Open Microscopy Environment

I have a very large bigTiff file (~162GB uncompressed) created by exporting from Zeiss' Zen software. I had hoped to import this image into ImageJ (Fiji). However, when I try to import/open the image in Fiji (using the Bio-Formats plugin), I get the error message (see below): "Array size too large". I think that the Bio-Formats (Release 5.1.9, build date 14 April 2016, Revision c3065feb775a7ac0cc2cf6e35979331cca2418b) is installed and working since I can use the plugin to import cropped regions of the image, provided that the region is 24,000x18,000 pixels or smaller. Any thoughts as to whether my inability to import the bigTiff file is:

1) a limitation imposed by Bio-Formats? I think that the bigTiff specification theoretically can handle unrealistically large files, but there probably will be more realistic limits as to size of the file that the software can handle. Does Bio-Formats have its own cut-off size for the bigTiff files it can handle?

2) a limitation with my implementation of Fiji? The computer has 32GB RAM, of which Fiji can use up to ~23.7GB (24,473MB), so Fiji/Bio-Formats probably would need to use the hard-drive as virtual RAM to import the image. Is this the problem, or can Fiji/Bio-Format spool to the hard-drive?

3) some other problem with my installation of Fiji or the Bio-Source plugin? I just loaded Fiji onto a new computer and I haven't played with bigTiff files before, so I could have done something wrong/something could have gotten corrupted during installation.

As an aside, if I try to import a cropped region with dimensions of 32,000x24,000 pixels or 40,000x30,000 pixels, Fiji displays an image but it is corrupted. If I try to import a cropped region from the bigTiff file of 80,000x60,000 pixels, I get the same error message as if I try to import or open the entire bigTiff file.

The error message that we get when we try to open the bigTiff file is:

(Fiji Is Just) ImageJ 2.0.0-rc-47/1.50i; Java 1.8.0_66 [64-bit]; Windows 7 6.1; 66MB of 24473MB (<1%)
java.lang.IllegalArgumentException: Array size too large: 331197 x 174340 x 1
at loci.common.DataTools.safeMultiply32(DataTools.java:850)
at loci.common.DataTools.allocate(DataTools.java:824)
at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:155)
at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:335)
at loci.formats.DimensionSwapper.openBytes(DimensionSwapper.java:233)
at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:335)
at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:335)
at loci.plugins.util.ImageProcessorReader.openProcessors(ImageProcessorReader.java:186)
at loci.plugins.util.ImageProcessorReader.openProcessors(ImageProcessorReader.java:83)
at loci.plugins.util.ImageProcessorSource.getObject(ImageProcessorSource.java:72)
at loci.formats.cache.Cache.recache(Cache.java:204)
at loci.formats.cache.Cache.recache(Cache.java:224)
at loci.formats.cache.Cache.<init>(Cache.java:88)
at loci.plugins.util.BFVirtualStack.<init>(BFVirtualStack.java:126)
at loci.plugins.in.ImagePlusReader.createVirtualStack(ImagePlusReader.java:331)
at loci.plugins.in.ImagePlusReader.readImage(ImagePlusReader.java:276)
at loci.plugins.in.ImagePlusReader.readImages(ImagePlusReader.java:238)
at loci.plugins.in.ImagePlusReader.readImages(ImagePlusReader.java:216)
at loci.plugins.in.ImagePlusReader.openImagePlus(ImagePlusReader.java:111)
at loci.plugins.in.Importer.readPixels(Importer.java:149)
at loci.plugins.in.Importer.run(Importer.java:86)
at loci.plugins.LociImporter.run(LociImporter.java:78)
at ij.IJ.runUserPlugIn(IJ.java:216)
at ij.IJ.runPlugIn(IJ.java:180)
at ij.Executer.runCommand(Executer.java:137)
at ij.Executer.run(Executer.java:66)
at java.lang.Thread.run(Thread.java:745)

Thanks for your help & ideas!

- - Jim

Hi,

Thank you for submitting your issue.

To start with, the error log is expected. The size of the whole image exceeds the maximum array size, allowed in Java.

So the only reliable way to opening these files is to use the “crop on import” option, to open tiles such that,
"The total number of bytes in a tile is less than the maximum array size (2GB)”.

All your later options (>24000*18000) are exceeding this limit and thus the corresponding stack trace/error.

Unfortunately the limitation comes from the Java side, and there is not much that we can do on this front.
Hope that helps.

Best,
Balaji

Unfortunately the limitation comes from the Java side, and there is not much that we can do on this front.

On the ImageJ side, we plan to implement a tiled image browser... eventually. In principle it will be doable with the new ImageJ2/ImgLib2 data model—it is just a matter of development effort.

Many thanks!

- - Jim