CellomicsReader modification

[kuba97531]

Hi,

I am not sure if this is the right place to ask about it.

I am using CellomicsReader to read images. I have found that a huge part of time is spend in checking if a file is hidden.

in line 140 of
loci/formats/in/CellomicsReader.java

Code: Select all

(...)
String[] list = parent.list(true);


Changing the boolean literal to false (not checking if files are hidden) improves performance of the reader greatly.

Is this possible to get something like this in the new bioformats release?

Code: Select all

private static boolean noHiddenFiles = true;
public static void setNoHiddenFiles(boolean v) { noHiddenFiles = v; }

(...)
String[] list = parent.list(noHiddenFiles);


Or can I submit a patch myself somewhere?

[rleigh]

Hi,

It's certainly possible to submit patches, either by opening a pull request on github (https://github.com/openmicroscopy/bioformats), or by sending a patch to ome-devel@lists.openmicroscopy.org.uk .

It looks like in the Location class, the main cause of the slowness is here:

Code: Select all

        if (!noHiddenFiles || !(name.startsWith(".") ||
          new Location(file.getAbsolutePath(), name).isHidden()))
        {
          files.add(name);
        }


The overhead is in creating a new Location instance and calling isHidden. Given that we check for dotfiles before this as an optimisation (it's also done by isHidden), the latter check only needs performing on Windows platforms, so could be omitted entirely on Unix and MacOS to improve the performance for all users of this method.

In CellomicsReader, we do this check in the inner loop:

Code: Select all

        if (plateName.equals(getPlateName(f)) &&
          (checkSuffix(f, "c01") || checkSuffix(f, "dib")))
        {
          pixelFiles.add(new Location(parent, f).getAbsolutePath());
        }


Since we are filtering the filenames for a .c01 or .deb suffix, we could move the hidden check to the inside of the body here, if it's still needed. For example:

Code: Select all

      String[] list = parent.list();
      for (String f : list) {
        if (plateName.equals(getPlateName(f)) &&
          (checkSuffix(f, "c01") || checkSuffix(f, "dib")))
        {
          Location loc = new Location(parent, f);
          if (!loc.isHidden())
            pixelFiles.add(loc.getAbsolutePath());
        }
      }


Since we have to create a Location class here /anyway/, the extra overhead is minimal. Another alternative would be to use Location.listFiles() to get an array of Location objects, which would also prevent creating two instances per file.

Regards,
Roger