Sequential writing of files

[gwitz]

Hi,

I'm trying to set up a microscope to save my multidimensional files (several fluorescence channels, z-stack, time) in the OME-Tiff format. Controlling the microscope with Matlab, I would like to be able to save file in OME-Tiff format directly from there. I successfully adapted available matlab code, but I'm stuck with the following problem: how can I write the OME-Tiff files piece by piece ? My goal is to scan a grid over time, and I would like to have one OME-Tiff for each xy stage position containing all colors, time points and z-stacks. To avoid running into memory problems, I intend to write each xy OME-Tiff file piece by piece each time I scan that specific xy position. I'm a beginner with the OME format, and so I guess there's a simple solution to that problem.

Thanks for your help,

Guillaume

[rleigh]

It's probably worth having a look at these references if you haven't already:

http://www.openmicroscopy.org/site/support/ome-model/ome-tiff/
http://www.openmicroscopy.org/site/support/ome-model/ome-tiff/specification.html
https://www.openmicroscopy.org/site/support/bio-formats4/developers/export.html#converting-to-multiple-files
https://github.com/openmicroscopy/bioformats/blob/develop/components/bio-formats/utils/MultiFileExportExample.java

The "Distributing an image sequence across multiple OME-TIFF files" and "Storing partial metadata blocks" sections at the bottom of the specification pages, and the use of changeOutputFiles to write planes to different files (last two links) are the most relevant bits.

However, because the OME-XML metadata is written in a single shot, it's not easy to incrementally update it during acquisition as you acquire each plane. You could save the planes during acquisition either as a single stack, a set of stack per xy position, or one file per plane, whatever is easiest, and write out the metadata at the end. Someone else might be better able to confirm this, but it looks like you might be able to use the OMETiffWriter to add the metadata to the existing images if you call setId, don't use saveBytes, and then just call changeOutputFile() for each tiff file, and then close(). You'll have had to set up the TiffData blocks appropriately for the entire dataset first. This does mean you would have to keep the data model in memory as you acquire; but the actual image data can be written to the TIFFs and discarded as you acquire, and then you just add the metadata when finished. You could use loci.formats.tiff.TiffSaver during acquisition to write out raw TIFF planes without the metadata.

Regards,
Roger Leigh

[gwitz]

Thanks for the suggestions. Writing simple tiff files and and adding the ome-xml at the end sounds like a good solution. In the meantime I found a very similar solution to my problem. Using bfsave, I write the ome-tiff file with only the first set of data (i.e. first time point of an xy point), but fill the metadata with what will be the final characteristics (all time points). Then using the Tiff Class in Matlab, I simply append the new data each time I scan that xy point. At the end, the data correspond to the metadata, and the file is perfectly readable as OME format.

An even better solution, would be of course to rewrite every time the correct metadata, in case the measurement gets interrupted. That will of course displace all the information in the tiff which is problematic. I'll look into the source code to find a function able to do that.

[rleigh]

The approach you've taken sounds very reasonable. Please do let us know if you have any other questions or run into any trouble. Bio-Formats and OME-TIFF were not originally geared towards image acquisition, being more focussed upon reading and conversion of file formats, so if there are particular features missing which you need or would make things easier or more efficient, or things which don't quite meet your requirements, please do let us know and we might be able to better support them in future releases.


Regards,
Roger