bfconvert "File is too large" Error

[mdifranc]

Hi all,

I'm using v5.1.2 and trying to use bfconvert on the command line to convert a single series within a .vsi image pyramid into .ome.tiff. Here's the command that I am running:

./bfconvert -series 7 -compression LZW INPUT_IMAGE.vsi OUTPUT_IMAGE.ome.tiff

The program runs for a long time, and the OUTPUT_IMAGE.ome.tiff file grows to about 5.7 GB. Then this errors happens:

Exception in thread "main" loci.formats.FormatException: File is too large; call setBigTiff(true)
at loci.formats.out.TiffWriter.prepareToWriteImage(TiffWriter.java:343)
at loci.formats.out.TiffWriter.saveBytes(TiffWriter.java:221)
at loci.formats.out.OMETiffWriter.saveBytes(OMETiffWriter.java:190)
at loci.formats.tools.ImageConverter.convertTilePlane(ImageConverter.java:731)
at loci.formats.tools.ImageConverter.convertPlane(ImageConverter.java:617)
at loci.formats.tools.ImageConverter.testConvert(ImageConverter.java:549)
at loci.formats.tools.ImageConverter.main(ImageConverter.java:874)

I was under the impression that BigTIFFs could grow to 18,000 petabytes, give or take. Can anyone tell me what's happening here?

Matt

[sbesson]

Hi Matt,


by default the bfconvert does not enable the BigTiff option. To enable it, there are 2 possibilities:
1. either use the `-bigtiff` flag.

Code: Select all

bfconvert -bigtiff -series 7 -compression LZW INPUT_IMAGE.vsi OUTPUT_IMAGE.ome.tiff


2. or use a BigTiff extension e.g. `ome.btf` (see also https://github.com/openmicroscopy/bioformats/pull/1790)

Code: Select all

bfconvert -series 7 -compression LZW INPUT_IMAGE.vsi OUTPUT_IMAGE.ome.btf


Thanks for reporting this, we will try to amend the command-line tools documentation to mention these options.

Best,
Sebastien

[mdifranc]

Aha! Thanks, that makes sense. I thought this was default behavior, given the Exception message.

At the very least, it would be nice to have the program spit out an error up front, rather than running into it at the end.

Cheers,
Matt

[mlinkert]

Hi Matt,

Thanks for the feedback - the current behavior is not ideal, but it is intentional.

If a BigTIFF extension or the '-bigtiff' option are used, then Bio-Formats will always write a BigTIFF file. If it's guaranteed that more than 4GB of data will be written, then a BigTIFF will be written even if one wasn't explicitly requested. However, that will only happen if the images are written uncompressed, as we can't reliably determine the size of the output file in advance when compression is used.

We don't write BigTIFF by default because many software packages still do not support it - an error message on conversion seemed safer than quietly writing something that might not be usable outside of Bio-Formats.

Regards,
-Melissa