Tile naming when importing tiled CZI image

[dsudar]

Hi all,

When I import a Zeiss CZI image that consists of a tiled acquisition, the importer creates separate images for each of the tiles with the same name and a sequence number added. Unfortunately the sequence numbering format results in a order of the list that is alphabetically not in the correct order. Thus when I arrange the thumbnails in the original tile pattern, the resulting view is not correct. See:

Capture.GIF (135.12 KiB) Viewed 3880 times

Could a small change be made in BF or the Importer where the sequence number format naming scheme is something like "#00000" so that they will always appear in correct order when sorted alphabetically?

Of course, even better would be if the importer would re-constitute the tiles into a single large image similar to what happens with some of the slide scanner formats such as Aperio SVS, Leica SCN, etc. This is quite relevant for CZI as well since Zeiss's Axioscan.Z1 slide scanner produces these type of .czi files.

I have uploaded a small example CZI file to illustrate the issue under QA entry: 10335.

Thanks,
- Damir

[mlinkert]

Hi Damir,

As always, thank you for the feedback.

Could a small change be made in BF or the Importer where the sequence number format naming scheme is something like "#00000" so that they will always appear in correct order when sorted alphabetically?

A fix for this is under review (for inclusion in 5.1.0):

https://github.com/openmicroscopy/bioformats/pull/1412

That should guarantee that .czi image names sort correctly alphabetically and numerically.

Of course, even better would be if the importer would re-constitute the tiles into a single large image similar to what happens with some of the slide scanner formats such as Aperio SVS, Leica SCN, etc. This is quite relevant for CZI as well since Zeiss's Axioscan.Z1 slide scanner produces these type of .czi files.

We unfortunately do not have plans to do that for .czi data. Aperio SVS, Leica SCN, and other formats specific to slide scanning have well-defined ways of putting the tiles back together - there is usually no overlap between tiles, and no registration is necessary. Most of the multi-tile .czi data that we've seen requires more advanced registration to have the tiles put back together correctly, so the tiles are imported separately; if we were to attempt registration in Bio-Formats, there would be no way to get the original tiles back if what Bio-Formats does is wrong.

Regards,
-Melissa

[dsudar]

Hi Melissa,
And, as always, thanks for the quick response.

With the sequence number fix, it'll be possible to display the thumbnails in the appropriate arrangement to illustrate how the tiling was done so that will be very useful. Thanks!

Yes, I understand the issue with .czi tiles. One interesting thing: the Zeiss ZEN 2012 software has the possibility to export the tiled composite as a .ome.tiff file and that file can be opened nicely with Fiji/BioFormats. However, that same file has trouble importing into OMERO and I'm still figuring out what the error is precisely. One excerpt from PixelData-0.log says:

2014-11-07 14:20:24,075 INFO [ ome.io.nio.FilePathResolver] (2-thread-4) Metadata only file, resulting path: /data/OMERO/ManagedRepository/dsudar_2/2014-11/07/14-20-07.683/K19(A).K17.K8_A1.MDAMB468_40x-1T4x5_s1.ome.tiff
2014-11-07 14:20:24,227 INFO [ ome.io.nio.PixelsService] (2-thread-4) Creating BfPixelBuffer: /data/OMERO/ManagedRepository/dsudar_2/2014-11/07/14-20-07.683/K19(A).K17.K8_A1.MDAMB468_40x-1T4x5_s1.ome.tiff Series: 0
2014-11-07 14:20:24,227 INFO [ ome.io.nio.PixelsService] (2-thread-4) Destination pyramid tile size: java.awt.Dimension[width=256,height=256]
2014-11-07 14:20:24,227 INFO [ ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:75881 1/1596 (0%).
2014-11-07 14:20:30,414 INFO [ ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:75881 160/1596 (9%).
2014-11-07 14:20:36,563 INFO [ ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:75881 319/1596 (19%).
2014-11-07 14:20:42,395 INFO [ ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:75881 478/1596 (29%).
2014-11-07 14:20:48,726 INFO [ ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:75881 637/1596 (39%).
2014-11-07 14:20:54,693 INFO [ ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:75881 796/1596 (49%).
2014-11-07 14:21:01,626 INFO [ ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:75881 955/1596 (59%).
2014-11-07 14:21:08,058 INFO [ ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:75881 1114/1596 (69%).
2014-11-07 14:21:13,660 INFO [ ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:75881 1273/1596 (79%).
2014-11-07 14:21:20,076 INFO [ ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:75881 1432/1596 (89%).
2014-11-07 14:21:25,835 INFO [ ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:75881 1591/1596 (99%).
2014-11-07 14:21:25,983 INFO [ ome.io.nio.PixelsService] (2-thread-4) SUCCESS -- Pyramid created for pixels id:75881
2014-11-07 14:21:25,984 INFO [ loci.formats.in.MinimalTiffReader] (2-thread-4) Reading IFDs
2014-11-07 14:21:25,988 INFO [ loci.formats.in.MinimalTiffReader] (2-thread-4) Populating metadata
2014-11-07 14:21:25,990 WARN [ loci.formats.in.JPEG2000MetadataParser] (2-thread-4) Unknown JPEG 2000 box 0x2900 at 6798
2014-11-07 14:21:25,990 INFO [ loci.formats.in.JPEG2000MetadataParser] (2-thread-4) File is a raw codestream not a JP2.
2014-11-07 14:21:25,992 WARN [ loci.formats.in.JPEG2000MetadataParser] (2-thread-4) Unknown JPEG 2000 box 0x2900 at 22456960
2014-11-07 14:21:25,992 INFO [ loci.formats.in.JPEG2000MetadataParser] (2-thread-4) File is a raw codestream not a JP2.
2014-11-07 14:21:25,994 WARN [ loci.formats.in.JPEG2000MetadataParser] (2-thread-4) Unknown JPEG 2000 box 0x2900 at 44929305
2014-11-07 14:21:25,994 INFO [ loci.formats.in.JPEG2000MetadataParser] (2-thread-4) File is a raw codestream not a JP2.
2014-11-07 14:21:25,995 WARN [ loci.formats.in.JPEG2000MetadataParser] (2-thread-4) Unknown JPEG 2000 box 0x2900 at 67783592
2014-11-07 14:21:25,995 INFO [ loci.formats.in.JPEG2000MetadataParser] (2-thread-4) File is a raw codestream not a JP2.
2014-11-07 14:21:25,997 INFO [ loci.formats.in.TiffReader] (2-thread-4) Checking comment style
2014-11-07 14:21:25,997 INFO [ loci.formats.in.BaseTiffReader] (2-thread-4) Populating OME metadata
2014-11-07 14:21:27,507 ERROR [ ome.services.pixeldata.PixelDataHandler] (2-thread-4) Failed to handle pixels 75881
java.lang.RuntimeException: An uncaught runtime exception has occurred jj2000.j2k.codestream.CorruptedCodestreamException: First marker after SOC must be SIZ 6a70
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.initializeRead(J2KReadState.java:735) ~[jai_imageio.jar:na]
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.<init>(J2KReadState.java:241) ~[jai_imageio.jar:na]
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KImageReader.readRaster(J2KImageReader.java:551) ~[jai_imageio.jar:na]
at loci.formats.services.JAIIIOServiceImpl.readRaster(JAIIIOServiceImpl.java:174) ~[formats-bsd.jar:na]
at loci.formats.codec.JPEG2000Codec.decompress(JPEG2000Codec.java:281) ~[formats-bsd.jar:na]
at loci.formats.tiff.TiffCompression.decompress(TiffCompression.java:282) ~[formats-bsd.jar:na]
at loci.formats.tiff.TiffParser.getTile(TiffParser.java:705) ~[formats-bsd.jar:na]
at loci.formats.tiff.TiffParser.getSamples(TiffParser.java:923) ~[formats-bsd.jar:na]
at loci.formats.tiff.TiffParser.getSamples(TiffParser.java:745) ~[formats-bsd.jar:na]
at loci.formats.in.MinimalTiffReader.openBytes(MinimalTiffReader.java:288) ~[formats-bsd.jar:na]
at loci.formats.FormatReader.openBytes(FormatReader.java:883) ~[formats-api.jar:na]
at loci.formats.FormatReader.openBytes(FormatReader.java:855) ~[formats-api.jar:na]
at ome.io.bioformats.OmeroPixelsPyramidWriter.recompressSeries(OmeroPixelsPyramidWriter.java:115) ~[romio.jar:na]
at ome.io.bioformats.OmeroPixelsPyramidWriter.postProcess(OmeroPixelsPyramidWriter.java:88) ~[romio.jar:na]
at ome.io.bioformats.OmeroPixelsPyramidWriter.close(OmeroPixelsPyramidWriter.java:52) ~[romio.jar:na]
at ome.io.bioformats.BfPyramidPixelBuffer.closeWriter(BfPyramidPixelBuffer.java:439) ~[romio.jar:na]
at ome.io.bioformats.BfPyramidPixelBuffer.close(BfPyramidPixelBuffer.java:674) ~[romio.jar:na]
at ome.io.nio.PixelsService.makePyramid(PixelsService.java:321) ~[romio.jar:na]
at ome.services.pixeldata.PixelDataHandler.process(PixelDataHandler.java:137) [server.jar:na]
at ome.services.pixeldata.PixelDataHandler.doWork(PixelDataHandler.java:88) [server.jar:na]
at sun.reflect.GeneratedMethodAccessor100.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_11]
at java.lang.reflect.Method.invoke(Method.java:601) ~[na:1.7.0_11]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307) [org.springframework.aop.jar:3.0.1.RELEASE-A]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) [org.springframework.aop.jar:3.0.1.RELEASE-A]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) [org.springframework.aop.jar:3.0.1.RELEASE-A]
at ome.services.util.Executor$Impl$Interceptor.invoke(Executor.java:576) [server.jar:na]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [org.springframework.aop.jar:3.0.1.RELEASE-A]
at ome.security.basic.NullEventHandler.invoke(NullEventHandler.java:39) [server.jar:na]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [org.springframework.aop.jar:3.0.1.RELEASE-A]
at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:111) [org.springframework.orm.jar:3.0.1.RELEASE-A]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [org.springframework.aop.jar:3.0.1.RELEASE-A]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108) [org.springframework.transaction.jar:3.0.1.RELEASE-A]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [org.springframework.aop.jar:3.0.1.RELEASE-A]
at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:241) [server.jar:na]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [org.springframework.aop.jar:3.0.1.RELEASE-A]
at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:116) [server.jar:na]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [org.springframework.aop.jar:3.0.1.RELEASE-A]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) [org.springframework.aop.jar:3.0.1.RELEASE-A]
at $Proxy68.doWork(Unknown Source) [na:na]
at ome.services.util.Executor$Impl.execute(Executor.java:457) [server.jar:na]
at ome.services.util.Executor$Impl.execute(Executor.java:401) [server.jar:na]
at ome.services.pixeldata.PixelDataThread.go(PixelDataThread.java:220) [server.jar:na]
at ome.services.pixeldata.PixelDataThread.access$000(PixelDataThread.java:49) [server.jar:na]
at ome.services.pixeldata.PixelDataThread$1.call(PixelDataThread.java:195) [server.jar:na]
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [na:1.7.0_11]
at java.util.concurrent.FutureTask.run(FutureTask.java:166) [na:1.7.0_11]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_11]
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [na:1.7.0_11]
at java.util.concurrent.FutureTask.run(FutureTask.java:166) [na:1.7.0_11]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [na:1.7.0_11]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [na:1.7.0_11]
at java.lang.Thread.run(Thread.java:722) [na:1.7.0_11]
Caused by: java.lang.RuntimeException: jj2000.j2k.codestream.CorruptedCodestreamException: First marker after SOC must be SIZ 6a70
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.initializeRead(J2KReadState.java:460) ~[jai_imageio.jar:na]
... 52 common frames omitted
Caused by: jj2000.j2k.codestream.CorruptedCodestreamException: First marker after SOC must be SIZ 6a70
at jj2000.j2k.codestream.reader.HeaderDecoder.extractMainMarkSeg(HeaderDecoder.java:1895) ~[jai_imageio.jar:na]
at jj2000.j2k.codestream.reader.HeaderDecoder.<init>(HeaderDecoder.java:2395) ~[jai_imageio.jar:na]
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.initializeRead(J2KReadState.java:456) ~[jai_imageio.jar:na]
... 52 common frames omitted
2014-11-07 14:21:27,508 ERROR [ ome.services.util.ServiceHandler] (2-thread-4) Method interface ome.services.util.Executor$Work.doWork invocation took 63492
2014-11-07 14:21:27,514 INFO [ ome.io.nio.FilePathResolver] (2-thread-1) Metadata only file, resulting path: /data/OMERO/ManagedRepository/dsudar_2/2014-11/07/14-20-07.683/K19(A).K17.K8_A1.MDAMB468_40x-1T4x5_s1.ome.tiff
2014-11-07 14:21:27,515 ERROR [ ome.services.util.ServiceHandler] (2-thread-1) Method interface ome.services.util.Executor$Work.doWork invocation took 63499

I also uploaded this file to the QA system under the same record 10335.

Thanks,
- Damir

[mlinkert]

Hi Damir,

Unfortunately, it looks like the file upload was not successful. If the .ome.tiff is smaller than 2GB, could you please upload it as a new QA issue? If it's larger than 2GB, we can send alternate upload instructions privately.

Regards,
-Melissa

[dsudar]

Hi Melissa,

I uploaded the file again under QA issue: 10341
It's about 190MB

Since Fiji/Bioformats can read this file fine, it doesn't appear to be a Bioformats issue but rather something with the OMERO importer.

Thanks for looking into this.
- Damir

[bramalingam]

Hi,

Thank you for submitting your file. We were able to reproduce the bug.

I have filed a ticket on this regard,
https://trac.openmicroscopy.org.uk/ome/ticket/12631

You have been added to the CC list, and will get notified when there is an update.

Best,
Balaji