Smart decimation method for fusion research data

Abstract

New fusion research experiments will generate massive experimental data. In this context, fusion research appears in the scope of the big data where both search and access functions require new approaches and optimizations. One common data access functionality is decimation. It allows to retrieve a limited number of points of the total available. One classical mechanism of decimation is downsampling by an integer factor called step. It works by selecting a value every ‘step’ number of values. The main characteristic of classical decimation is that the selected values are uniformly distributed along the total. However, in case of time evolution experimental signals, the relevancy of data is not uniform. There are some intervals where the provided information is more complex and richer, and usually more interesting from the user point of view. This contribution presents a new data decimation technology for unidimensional time evolution signals where the limited number of accessed points are distributed following the criterion of data interest level. The new method implements, on one hand, a heuristic function which is able to determine the level of interest of an interval based on its data characteristics, and on the other hand, a selection algorithm where points are distributed based on weighted intervals.