PlaneID support for Columbus imports

[ok88]

Hi,

We're trying to import a dataset from Columbus that has 1 384 well plate, 6 fields, and 9 planes per well into Omero. It seems like only the first plane (z-stack) is imported and the rest are ignored.
I looked through the source code for the ColumbusReader.java here (https://github.com/openmicroscopy/biofo ... eader.java) and it seems like the "PlaneID" field in our ImageIndex.ColumbusIDX.xml isn't included.

Can Omero/bioformats import the planes correctly from this format?
We're using Omero 5.4.6 (bioformats 5.8.2). Attached is the .XML file.

Thanks!

[dgault]

You are correct that the PlaneID is not currently being parsed and that could certainly be added to the reader, however I would still have expected the reader to have parsed the remaining metadata for the plane and included it as part of the fileset. If you have a small fileset (<2GB) or subset which reproduces the problem would you be able to upload it to https://www.openmicroscopy.org/qa2/qa/upload/

David Gault

[ok88]

Hi David,
Thanks for the quick reply. I uploaded a zip file with images from one field (3 channels at 9 different planes) and the corresponding XML for the plate). Let me know what you think the best way to resolve this is.
Thanks again,
Oren

[dgault]

Thanks Oren, we received the uploaded samples successfully. I will continue to test and debug them and provide an update to this thread tomorrow for you.

[dgault]

Just as an update to the testing, I have been able to reproduce the issue with the submitted fileset and debug the problem further. The image metadata is being parsed correctly with the exception of the PlaneID. This means that allow the reader has identified all of the image files correctly the series remains incorrect.

I have tested a potential fix which would treat PlaneID similar to FieldID and allow for the correct seriesCount to be used. This will require some further work and testing. I have also opened a Trello card to track this issue on the Bio-Formats inbox: https://trello.com/c/1tAKDsuU/261-plane ... r-columbus

[ok88]

Hi David, Thanks for finding/correcting this bug.
Can you provide a patch for the latest stable release with this correction?
Thanks again!
Oren

[dgault]

Hi Oren, this fix will not make it in for the next 5.9.1 patch release but I have opened the PR for testing so we can begin to look at scheduling it for a future release: https://github.com/openmicroscopy/bioformats/pull/3193

[etienne.dumoulin]

Hi David,

I work with Oren. We would like to load this plate for a demo due in 10 days. Is there anyway I can patch the current fix to an omero installation, load the plate and un-patch the changes?

Do you have some pointers for doing those kind of things?

Thanks,

Etienne

[dgault]

Hi Etienne,

The only Bio-Formats component impacted by that particular change is the formats-gpl.jar file.
You can find the latest successful build of this jar file at https://web-proxy.openmicroscopy.org/we ... pl/target/

If you wished to patch an OMERO server to include this you can simply replace the formats-gpl jar in the folder: /lib/server

As a word of caution, this is not something which we would normally recommend and the unreleased jars should be considered experimental. The build will also contains changes to other readers which have not yet been tested and released.

[etienne.dumoulin]

Hi David,

Unfortunately the Jar is not 100% in isolation I get:

Code: Select all

$ ./OMERO.server/bin/omero import <myfolder>
Created session adb1270f-934d-47fc-9925-4f30bfcb3355 (Kshitij@localhost:4064). Idle timeout: 10 min. Current group: system
2018-09-11 17:00:42,317 147        [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.4.6-ice36-b87
2018-09-11 17:00:42,325 155        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.8.2 revision: 6bc10be63e934ed12e76c3ffd83ecd2bca6312a1 date: 18 April 2018
2018-09-11 17:00:42,360 190        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2018-09-11 17:00:42,600 430        [      main] ERROR  formats.importer.cli.CommandLineImporter - Error during import process.
java.lang.NoSuchFieldError: mergeSubIFDs
        at loci.formats.in.NikonReader.<init>(NikonReader.java:119) ~[formats-gpl.jar:5.8.2]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_181]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_181]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_181]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_181]
        at java.lang.Class.newInstance(Class.java:442) ~[na:1.8.0_181]
        at loci.formats.ImageReader.<init>(ImageReader.java:129) ~[formats-api.jar:5.8.2]
        at loci.formats.in.FilePatternReader.<init>(FilePatternReader.java:77) ~[formats-bsd.jar:5.8.2]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_181]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_181]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_181]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_181]
        at java.lang.Class.newInstance(Class.java:442) ~[na:1.8.0_181]
        at loci.formats.ImageReader.<init>(ImageReader.java:129) ~[formats-api.jar:5.8.2]
        at loci.formats.ImageReader.<init>(ImageReader.java:116) ~[formats-api.jar:5.8.2]
        at ome.formats.importer.OMEROWrapper.createReader(OMEROWrapper.java:148) ~[blitz.jar:na]
        at ome.formats.importer.OMEROWrapper.<init>(OMEROWrapper.java:93) ~[blitz.jar:na]
        at ome.formats.importer.OMEROWrapper.<init>(OMEROWrapper.java:89) ~[blitz.jar:na]
        at ome.formats.importer.cli.CommandLineImporter.<init>(CommandLineImporter.java:141) ~[blitz.jar:na]
        at ome.formats.importer.cli.CommandLineImporter.main(CommandLineImporter.java:964) ~[blitz.jar:na]


Any suggestion would be welcome. On my side, I will try to figure out if I can use a Columbus instance to import the data to re-export it into a friendlier format. But I am not too sure if that is possible.