Open Microscopy Environment

OME Bioformats support for the Olympus raw file format (.oir) is incomplete. When a .oir file reaches a size larger than 1Gb, Olympus automatically begins splitting the file into sub-files, e.g. 'datafile.oir' will contain time points 0 through N, then 'datafile_00001' will contain time points N through M, etc... These files need to be loaded/treated as one large file, but OME Bioformats does not do this properly. (Note: the sub-files cannot simply be loaded as separate .oir files. See my previous post).

I would greatly appreciate Bioformats correcting this issue. If I upload a complete example of a large split .oir file, could someone work on this?

Note, the Olympus Viewer ImageJ plugin handles these split files correctly, but it is not scriptable or usable outside of ImageJ.

Hi Aaron,

OME Bioformats support for the Olympus raw file format (.oir) is incomplete. When a .oir file reaches a size larger than 1Gb, Olympus automatically begins splitting the file into sub-files, e.g. 'datafile.oir' will contain time points 0 through N, then 'datafile_00001' will contain time points N through M, etc... These files need to be loaded/treated as one large file, but OME Bioformats does not do this properly. (Note: the sub-files cannot simply be loaded as separate .oir files. See my previous post).

Linking back to viewtopic.php?f=13&t=8360 for the previous post mentioned above. From the OME standpoint, a short summary is that the Olympus OIR reader has no support for OIR datasets composed of multiple files such as the ones discussed above and in the previous thread. This is primarily due to the fact we had no such representative samples when the initial OIRReader was developed for Bio-Formats 5.5.0.

I would greatly appreciate Bioformats correcting this issue. If I upload a complete example of a large split .oir file, could someone work on this?

From the OME side, we can certainly record this format support addition in our public backlog. Given our current agenda and deadlines, it is unlikely the open-source team will have the time to prioritize this issue in the upcoming months.
It is possible that someone in the growing Bio-Formats contributing community can and wants to get involved. For this to happen, sharing representative datasets publicly is probably the best starting point if possible (see http://downloads.openmicroscopy.org/images/Olympus-OIR/ for instance).

Note, the Olympus Viewer ImageJ plugin handles these split files correctly, but it is not scriptable or usable outside of ImageJ.

Knowing that the Olympus Viewer ImageJ plugin works on this file is certainly great to hear since this is the reference external software we used to validate OIR filesets during the initial reader development.

Best,
Sebastien

Sebastien, Thank you for the reply.

Obviously, it would be great if the OME team could prioritize this fix after the current deadlines pass. Olympus is shipping more and more systems that use this format and datasets larger than a 1Gb are no longer uncommon.

To this end, and in case the public is interested in working on this, I have hosted an example of a multipart .oir file on AWS. Here are the links to the 4 parts:
https://s3.amazonaws.com/edu.stanford.c ... tipart.oir
https://s3.amazonaws.com/edu.stanford.c ... part_00001
https://s3.amazonaws.com/edu.stanford.c ... part_00002
https://s3.amazonaws.com/edu.stanford.c ... part_00003

Hi all,

I've also come up against this problem, using the new FV3000 which has very limited support for other file formats, and am struggling to deal with some quite large (60GB) datasets. I'd be happy to provide one (or a smaller test dataset) if it's useful, but it looks like Aaron already has.

Anyway, just wanted to add my vote for extended OIR support and am happy to test out any modifications to make it work.

Martin

Hi Martin (and Aaron),

Thanks for notifying us of the importance of this issue. We have starting the investigation of the modifications required in the reader to support large OIR datasets- see this card for more details.

Having extra datasets to confirm the validity of this preliminary work would definitely be useful. If you can generate a fileset below 2GB, feel free to upload it at http://qa.openmicroscopy.org.uk/qa/upload/. Otherwise we can send you private instructions for uploading your data to a FTP server.

Best,
Sebastien

Hi Sebastien,

Thanks for your help on this, it is very much appreciated. Most of the imaging I do is much larger than 2GB and I am happy to send a representative dataset that is ~30GB via FTP.

I am currently attempting to upload this one, which is under the 2GB limit:
20171020_JupGFPHisRFP_L3_30xSI_galvo_p5z_p1xy_better_than_nyquist.oir
20171020_JupGFPHisRFP_L3_30xSI_galvo_p5z_p1xy_better_than_nyquist_00001

However, as it is not supported correctly by OMERO, I'm unsure as to how successful this will be. My own uploads to OMERO result in a truncated dataset, which is not the intention.

Regards,
Martin

Hi Martin,

Thank you again for offering to send a few datasets. FTP instructions for large files have been sent to you via private message.

-Melissa