29th Symposium on Fusion Technology (SOFT 2016)

P4.076 Soft real-time analysis of ITER magnetics streaming data using SPECTRE

Sep 8, 2016, 2:20 PM

1h 40m

Foyer 2A (2nd floor), 3A (3rd floor) (Prague Congress Centre)

Board: 76

Poster D. Diagnostics, Data Acquisition and Remote Participation P4 Poster session

Speaker

Jonathan Hollocombe (Theory and Modelling)

Description

The SAGE ² 2 European Horizon 2020 project (grant agreement 671500), led by Seagate with 10 partners, is investigating the needs of future exascale storage systems for data intensive applications. CCFE is one of the partners and SPECTRE (SPECtral Research Engine) is one of the tools being developed to take advantage of the improved data I/O and throughput capability of the SAGE platform. SPECTRE is designed to emulate and process the levels of data acquisition required to ingest magnetics data once ITER is running. ITER will have about 200 magnetics sources each generating data at a rate ~40 MB/s giving an input of ~8 GB/s for the whole pulse. A typical pulse will last hundreds of seconds and create over 1 TB of magnetics data. Typically magnetics data are stored and processed off-line between pulses to generate diagnostic data vital for the preparation of the next experiment. SPECTRE, using streaming technologies such as Apache Spark or Apache Storm in concert with the SAGE platform, will greatly reduce the time needed to process this diagnostic data. In addition to the significant inter-pulse analysis speed up using SAGE & ‘Big Data’ technologies, it is possible to provide soft real-time (i.e. as close to real-time as feasible) diagnostics from live streaming data – an essential aid for experiment ROs in running and guiding the pulse. Synthetic data generated at levels matching those of ITER have been used to stress test SPECTRE and demonstrate the potential benefits of the SAGE platform and related technologies. Additionally, real data generated using prototype ITER data acquisition hardware being developed by F4E, CCFE and others ³3, will be analysed. 2 http://www.sagestorage.eu/ 3 A. J. N. Batista et al, “F4E prototype of a chopper digital integrator for the ITER magnetics”, this Conference.