Unable to read Olympus apl database

[dgault]

Hi Klaus,

My apologies for the very late response as I missed your last question. The fixes I have submitted have now been accepted and are being set to be included in the next Bio-Formats release 5.1.9 which is currently scheduled for release next week.

With Thanks,
David Gault

[klaus]

Hi David,

I have just tried the new BioFormats (5.1.9) but unfortunately I have still some problems with the databases that I sent as examples. Here is a brief sum up of my tests so far:

Database "example2":
The pictures load OK but the metadata is somehow mixed up. When I display the metadata I find that the number of records in the metadata (33) does not match the number of pictures (11). The info in the metadata fields is partly correct but some info is missing or shows incorrect values. At least one record name is missing completely (for picture 9, should be "PGP").

Database "ExampleDatabase":
This database does not load at all. Here is the error message:

Code: Select all

(Fiji Is Just) ImageJ 2.0.0-rc-47/1.50i; Java 1.8.0_66 [32-bit]; Windows 7 6.1; 39MB of 1187MB (3%)

java.lang.ArrayIndexOutOfBoundsException
   at java.lang.System.arraycopy(Native Method)
   at loci.formats.tiff.TiffParser.getSamples(TiffParser.java:1021)
   at loci.formats.tiff.TiffParser.getSamples(TiffParser.java:783)
   at loci.formats.in.APLReader.openBytes(APLReader.java:151)
   at loci.formats.ChannelFiller.openBytes(ChannelFiller.java:156)
   at loci.formats.ChannelFiller.openBytes(ChannelFiller.java:148)
   at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:196)
   at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:157)
   at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:335)
   at loci.formats.DimensionSwapper.openBytes(DimensionSwapper.java:233)
   at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:335)
   at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:335)
   at loci.plugins.util.ImageProcessorReader.openProcessors(ImageProcessorReader.java:186)
   at loci.plugins.in.ImagePlusReader.readProcessors(ImagePlusReader.java:416)
   at loci.plugins.in.ImagePlusReader.readPlanes(ImagePlusReader.java:380)
   at loci.plugins.in.ImagePlusReader.readImage(ImagePlusReader.java:277)
   at loci.plugins.in.ImagePlusReader.readImages(ImagePlusReader.java:238)
   at loci.plugins.in.ImagePlusReader.readImages(ImagePlusReader.java:216)
   at loci.plugins.in.ImagePlusReader.openImagePlus(ImagePlusReader.java:111)
   at loci.plugins.in.Importer.readPixels(Importer.java:149)
   at loci.plugins.in.Importer.run(Importer.java:86)
   at loci.plugins.LociImporter.run(LociImporter.java:78)
   at ij.IJ.runUserPlugIn(IJ.java:216)
   at ij.IJ.runPlugIn(IJ.java:180)
   at ij.Executer.runCommand(Executer.java:137)
   at ij.Executer.run(Executer.java:66)
   at java.lang.Thread.run(Thread.java:745)


Database "LifeScienceDemo":
The importer loads only one picture although the database contains a lot more.

Database "plain":
The importer throws an error message: "Sorry, there was a problem during import" and the log window shows:

Code: Select all

loci.formats.FormatException: Could not find a directory with TIFF files.
   at loci.formats.in.APLReader.initFile(APLReader.java:359)
   at loci.formats.FormatReader.setId(FormatReader.java:1426)
   at loci.plugins.in.ImportProcess.initializeFile(ImportProcess.java:505)
   at loci.plugins.in.ImportProcess.execute(ImportProcess.java:143)
   at loci.plugins.in.Importer.showDialogs(Importer.java:140)
   at loci.plugins.in.Importer.run(Importer.java:76)
   at loci.plugins.LociImporter.run(LociImporter.java:78)
   at ij.IJ.runUserPlugIn(IJ.java:216)
   at ij.IJ.runPlugIn(IJ.java:180)
   at ij.Executer.runCommand(Executer.java:137)
   at ij.Executer.run(Executer.java:66)
   at java.lang.Thread.run(Thread.java:745)


If necessary, I could try to produce report sheets for the pictures in one of the databases. This could help to identify the correct metadata. Just tell me were I should send it.

Thanks a lot for all the work so far!

Klaus

[dgault]

Hi Klaus,

Thanks for following up with further feedback on this. It does indeed look like some of the metadata is not being parsed or saved correctly for both the LifScienceDemo and example2 datasets. This will require further investigation on our side to look into the parsing of the metadata fields in more depth.

The ExampleDatabase fileset we still have some known issues to fix in order to get it loading correctly which is the open ticket https://trac.openmicroscopy.org/ome/ticket/13174#ticket. I will further update this ticket with the metadata issues listed above also. Hopefully we can make further improvements to the APL Reader to resolve these issues.

The reported error on "plain" databse however I am not able to reproduce here. Are the associated tiffs in the directory B25UN899_DocumentFiles?

With thanks,
David

[klaus]

Hi David,

the issue with the "plain" database was my own stupid fault. Apparently during zip, copy, paste, unzip... I managed to create an additional subfolder between the folder with the apl-file and the folder with the tiff-files. Without this everything is OK.

Just one additional question: Will the importer also work when the database contains a few thousand pictures?

Greetings,
Klaus

[sbesson]

Hi Klaus,

thanks for taking the time to re-test the plain database example and letting us know of the outcome.

On the multiple files front, we have several example of file formats spanning over multiple hundreds or thousands of files especially in the high-content screening domain. So opening large APL filesets should also work. If you encounter opening or performance issues when dealing with filesets of such scale, let us know via the forum or the mailing list.

Best,
Sebastien