I am trying to find out more information about the support for Big TIFF export within OMERO Insight. I found this ticket which seems to indicate that work is in progress but is not mature:
http://trac.openmicroscopy.org/ome/ticket/4799#
We use time-lapse microscope images that regularly surpass 4GB in size. At current we can upload the images to OMERO but cannot export them as OME TIFF. We have to use the archive option to be able to extract the data in its original form. If I use 'Export as OME-TIFF' the activities manager takes a long time to process the task. It then says the image is exported but the file size on disc is 0 bytes.
If I log on to the server I can see the file being generated in a tmp directory under:
$OMERO_TMPDIR/omero/tmp/omero_omero/
I can monitor this file size using the watch command. However when the file hits 4Gb the export fails and the file is reset to size 0.
The same thing happens when I use the Gateway API to create an Exporter and use the generateTiff() method:
- Code: Select all
image_id = 3 # Use a file over 4GB
e = conn.createExporter();
e.addImage(image_id);
try:
length = e.generateTiff()
out = open('/tmp/test.ome.tif', 'wb')
read = 0
while True:
buf = e.read(read, 1000000);
out.write(buf)
if len(buf) < 1000000:
break
read += len(buf);
out.close()
finally:
e.close();
I've tried this a few times and I get various errors:
- The file reaches several GB until I get an ::Ice::CommunicatorDestroyedException thrown. The file is set to 0 bytes.
- After some processing time I get an ::omero::ApiUsageException thrown with message: Only one image supported for TIFF, not 0. However the file still continues to be generated on the server side until it reaches 4GB. It does not get reset to 0 bytes. This file is truncated since I cannot read the entire file into ImageJ using BioFormats (I get errors and blank planes).
What do you suggest for storage of our large files? Archiving them roughly doubles the amount of disk space used for each file (which is a costly solution). In addition any scripts that process the images and create new equivalent sized images cannot be exported since they have no corresponding original file; they only have the pixel data. Thus they would have to be split into smaller images, exported and then joined up outside of OMERO.
Thanks,
Alex
PS. Apologies for cross-posting. I tried this in the Developers forum a week ago and had no luck so I thought I'd try this forum instead.
