Computerized imaging experiments were first performed in the late 1980’s as scanning probe microscopies such as STM and AFM were developed. Later these techniques were applied to existing forms of scanning microscopy, including electron microscopy (STEM, SEM). Digital data acquisition enabled new microscopy modalities including a broad range of optical spectro-microscopies (confocal Raman, Photoluminescence, FLIM…) and synchrotron-based x-ray microscopies.
In the “classical” paradigm for scientific imaging, software tools were first developed by research groups with a focus on immediate results. These programs were often “user hostile” and subject to rapid obsolesce as the original authors moved on. When techniques became popular, commercial vendors developed imaging hardware, followed by the creation of usually-proprietary software to operate the instrument in predefined modes. While vendor-developed software makes powerful techniques accessible to a wide user base, the same software often prevents researchers from fully exploiting commercial equipment for advanced applications, where instruments from multiple vendors are combined with locally-built equipment to perform integrated experiments.
We want to do “Smart imaging”, performing experiments in the microscope, not just recording “images”, with adaptive control of data acquisition to maximize the “data-to-damage” ratio with nanoscale samples, combined with on-line physical analysis that feeds back to guide the experiment and the acquisition of microscopy data. Reaching this goal will require development of open-source software architectures that control the hardware – the experimental environment, the microscope with its detectors and spectroscopic tools – while integrating data visualization and theoretical analysis of real-time data streams.
In the Molecular Foundry, we are developing “ScopeFoundry”, a framework based on scientific python to control the hardware and visualization aspects of smart scanned imaging experiments. The issues are very similar for many types of scanned imaging, including analytical STEM, STM, AFM, STXM, NSOM and confocal optical spectroscopy.