Speaker
Sherry Mayo
(CSIRO)
Description
It is now possible to collect very rapid micro-CT scans at IMBL with data collection times down to 16s or less per scan. This makes it possible to collect three-dimensional data on physical systems that are changing over time.
While this provides an exciting scientific opportunity it also comes with significant challenges in terms of data storage, reconstruction and analysis. Dealing effectively with such 'Big data' challenges is increasingly important in order to ensure that we derive the most benefit from the opportunity that rapid data collection offers.
The authors recently acquired a large number of time-resolved datasets following the development of bread dough during proving and baking. The aim of the experiment was to determine differences in the rheological and machanical properties of dough made using different flours and salt additives.
This experiment resulted in over 460 datasets which required reconstruction and analysis and provided a good test-case for addressing the big-data challenge. This challenge has been addressed using tools available (and new tools developed) on the MASSIVE supercomputer cluster.
We outline the challenges and how they were addressed as well as providing some reflection on what might be needed in the future to meet our growing data needs.
| Keywords or phrases (comma separated) | Big data, IMBL, micro-CT, time-resolved, bread |
|---|
Primary author
Sherry Mayo
(CSIRO)
