Page 1 of 1

bfconvert "File is too large" Error

PostPosted: Wed Jun 10, 2015 1:41 pm
by mdifranc
Hi all,

I'm using v5.1.2 and trying to use bfconvert on the command line to convert a single series within a .vsi image pyramid into .ome.tiff. Here's the command that I am running:

./bfconvert -series 7 -compression LZW INPUT_IMAGE.vsi OUTPUT_IMAGE.ome.tiff

The program runs for a long time, and the OUTPUT_IMAGE.ome.tiff file grows to about 5.7 GB. Then this errors happens:

Exception in thread "main" loci.formats.FormatException: File is too large; call setBigTiff(true)
at loci.formats.out.TiffWriter.prepareToWriteImage(TiffWriter.java:343)
at loci.formats.out.TiffWriter.saveBytes(TiffWriter.java:221)
at loci.formats.out.OMETiffWriter.saveBytes(OMETiffWriter.java:190)
at loci.formats.tools.ImageConverter.convertTilePlane(ImageConverter.java:731)
at loci.formats.tools.ImageConverter.convertPlane(ImageConverter.java:617)
at loci.formats.tools.ImageConverter.testConvert(ImageConverter.java:549)
at loci.formats.tools.ImageConverter.main(ImageConverter.java:874)

I was under the impression that BigTIFFs could grow to 18,000 petabytes, give or take. Can anyone tell me what's happening here?

Matt

Re: bfconvert "File is too large" Error

PostPosted: Wed Jun 10, 2015 2:46 pm
by sbesson
Hi Matt,


by default the bfconvert does not enable the BigTiff option. To enable it, there are 2 possibilities:
1. either use the `-bigtiff` flag.

Code: Select all
bfconvert -bigtiff -series 7 -compression LZW INPUT_IMAGE.vsi OUTPUT_IMAGE.ome.tiff


2. or use a BigTiff extension e.g. `ome.btf` (see also https://github.com/openmicroscopy/bioformats/pull/1790)

Code: Select all
bfconvert -series 7 -compression LZW INPUT_IMAGE.vsi OUTPUT_IMAGE.ome.btf


Thanks for reporting this, we will try to amend the command-line tools documentation to mention these options.

Best,
Sebastien

Re: bfconvert "File is too large" Error

PostPosted: Wed Jun 10, 2015 3:18 pm
by mdifranc
Aha! Thanks, that makes sense. I thought this was default behavior, given the Exception message.

At the very least, it would be nice to have the program spit out an error up front, rather than running into it at the end.

Cheers,
Matt

Re: bfconvert "File is too large" Error

PostPosted: Thu Jun 11, 2015 4:09 pm
by mlinkert
Hi Matt,

Thanks for the feedback - the current behavior is not ideal, but it is intentional.

If a BigTIFF extension or the '-bigtiff' option are used, then Bio-Formats will always write a BigTIFF file. If it's guaranteed that more than 4GB of data will be written, then a BigTIFF will be written even if one wasn't explicitly requested. However, that will only happen if the images are written uncompressed, as we can't reliably determine the size of the output file in advance when compression is used.

We don't write BigTIFF by default because many software packages still do not support it - an error message on conversion seemed safer than quietly writing something that might not be usable outside of Bio-Formats.

Regards,
-Melissa