In place import hanging on an ome.tiff

[thusharaw]

I'm seeing the in place import hang on a ome.tiff file. It is stuck here :

"main" #1 prio=5 os_prio=0 tid=0x00007fe6e800b800 nid=0x993 runnable [0x00007fe6f0f30000]
java.lang.Thread.State: RUNNABLE
at loci.formats.in.TiffReader.parseCommentGeneric(TiffReader.java:433)
at loci.formats.in.TiffReader.initStandardMetadata(TiffReader.java:194)
at loci.formats.in.BaseTiffReader.initMetadata(BaseTiffReader.java:97)
at loci.formats.in.BaseTiffReader.initFile(BaseTiffReader.java:584)
at loci.formats.FormatReader.setId(FormatReader.java:1397)
at loci.formats.in.OMETiffReader.initFile(OMETiffReader.java:840)
at loci.formats.FormatReader.setId(FormatReader.java:1397)
at loci.formats.ImageReader.setId(ImageReader.java:839)
at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:650)
at loci.formats.ChannelFiller.setId(ChannelFiller.java:223)
at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:650)
at loci.formats.ChannelSeparator.setId(ChannelSeparator.java:291)
at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:650)
at loci.formats.Memoizer.setId(Memoizer.java:649)
at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:650)
at ome.formats.importer.ImportCandidates.singleFile(ImportCandidates.java:427)
at ome.formats.importer.ImportCandidates.handleFile(ImportCandidates.java:576)
at ome.formats.importer.ImportCandidates.execute(ImportCandidates.java:384)
at ome.formats.importer.ImportCandidates.<init>(ImportCandidates.java:222)
at ome.formats.importer.ImportCandidates.<init>(ImportCandidates.java:174)
at ome.formats.importer.cli.CommandLineImporter.<init>(CommandLineImporter.java:147)
at ome.formats.importer.cli.CommandLineImporter.main(CommandLineImporter.java:955)

The importer is taking more than a single core (120 - 140%% shown on top) and it seems it is trying to increase a buffer used for the "generic comment". Without seeing the code, this seems eerily familiar as the growing buffer problem where unless for each new data, the buffer is doubled (or increased more than strictly necessary for the new data), it creates a slowdown.

I'm using the latest OMERO 5.3.3.

I increased the memory used for Blitz to be 50% so in my system, it is running with 12G of RAM. But that probably doesn't matter as the process that is churning is not the server but the importer.

I used JAVA_OPTS="-Xmx12G" before stating the in place importer. But it was still getting stuck at the same place.

thx,
thushara

[thusharaw]

Digging a bit further, perhaps this code can be made faster by doubling the buffer used to hold the comment for each comment line:

Code: Select all

private void parseCommentGeneric(String comment) {
    if (comment == null) return;
    String[] lines = comment.split("\n");
    if (lines.length > 1) {
      comment = "";
      for (String line : lines) {
        int eq = line.indexOf('=');
        if (eq != -1) {
          String key = line.substring(0, eq).trim();
          String value = line.substring(eq + 1).trim();
          addGlobalMeta(key, value);
        }
        else if (!line.startsWith("[")) {
          comment += line + "\n";
        }
      }
      addGlobalMeta("Comment", comment);
      description = comment;
    }
}

[sbesson]

Hi thusara,

thanks your report including the detailed jstack output. As mentioned below, the issue seems to lie primarily at the Bio-Formats level with some inefficiency while parsing large amount of metadata.

I am cross-linking this forum thread to the currently open Pull Request which proposes an implementation to improve the handling of metadata string.
We will definitely test the performance of OME-TIFF metadata processing with this addition both at the Bio-Formats and the OMERO level and report the outcome on the Pull Request.

Best,
Sebastien