20-21 November 2014
National Centre for Synchrotron Science
Australia/Melbourne timezone
Save the date: User Meeting 2015 - 26-27 November

Wrestling with Big Data at IMBL

20 Nov 2014, 17:30
1h 30m
NCSS Exhibition Area ()

NCSS Exhibition Area

Australian Synchrotron 800 Blackburn Road Clayton VIC 3168
Board: 608


Sherry Mayo (CSIRO)


It is now possible to collect very rapid micro-CT scans at IMBL with data collection times down to 16s or less per scan. This makes it possible to collect three-dimensional data on physical systems that are changing over time. While this provides an exciting scientific opportunity it also comes with significant challenges in terms of data storage, reconstruction and analysis. Dealing effectively with such 'Big data' challenges is increasingly important in order to ensure that we derive the most benefit from the opportunity that rapid data collection offers. The authors recently acquired a large number of time-resolved datasets following the development of bread dough during proving and baking. The aim of the experiment was to determine differences in the rheological and machanical properties of dough made using different flours and salt additives. This experiment resulted in over 460 datasets which required reconstruction and analysis and provided a good test-case for addressing the big-data challenge. This challenge has been addressed using tools available (and new tools developed) on the MASSIVE supercomputer cluster. We outline the challenges and how they were addressed as well as providing some reflection on what might be needed in the future to meet our growing data needs.
Keywords or phrases (comma separated) Big data, IMBL, micro-CT, time-resolved, bread

Primary author


Presentation Materials

There are no materials yet.
Your browser is out of date!

Update your browser to view this website correctly. Update my browser now