processing time issues

[cagatay]

I realized that I posted this on the OMERO discussion board instead of bioformats. So moving the question here...

I have a czi file that I am trying to process in batches. Crop all the regions using bftools and then process them using some image processing library. I am using the latest bftools as far as I can tell.

[cagatay@p53 bftools]$ ./bfconvert -version
Version: 5.5.3
Build date: 5 July 2017
VCS revision: 7bed1b04d071667d7b8a2e8c5d42f05fbfc07ecc
[cagatay@p53 bftools]$
------------------------

When I try this on Imagej with bioformats package, after loading the image, I click the crop on load and voila, in a second I get the region in front of me. When I try this on the bfconvert it takes more than a minute per crop operation. I am wondering why the processing times are so different. Given that I'll be running this for the whole image, it will takes days if not weeks to crop all the regions in the image. Any ideas?

[cagatay@p53 bftools]$ export BF_MAX_MEM=50g
[cagatay@p53 bftools]$ ./bfconvert -series 1 -crop 30000,30000,1024,1024 ../x40-01.czi output-512x512-crop.tiff
../x40-01.czi
ZeissCZIReader initializing ../x40-01.czi
[Zeiss CZI] -> output-512x512-crop.tiff [Tagged Image File Format]
Converted 1/1 planes (100%)
[done]
82.68s elapsed (140.0+200.0ms per plane, 82337ms overhead)
[cagatay@p53 bftools]$

[dgault]

Hi,

I have carried out some profiling of the two scenarios you described and there is indeed a significant difference. From looking at the results it appears that for bfconvert the majority of the time (95%) is actually spent generating and writing out the XML metadata to file, and most of that time is specifically writing the data. When opening in FIJI the plugin is not writing metadata to file until you export it, so that is the main difference between these scenarios.

In saying that it does seem like a very long time to be writing XML data to file. When I load the metadata back in I can again see a very long read time to parse the XML back. I have opened a Trello card on the Bio-Formats inbox for follow investigation into these read and write times: https://trello.com/c/LXSLMzIa/165-long- ... read-times

David Gault

[cagatay]

Thank you David,

let me know if I can be of any help. My only other option at this point is to write some fiji plugin to do this in batch mode and I'm trying to avoid that.

Cheers,
Cagatay

[cagatay]

I ended up writing the Fiji plugin if anyone needs it. It's been >10 years since I wrote in Java and this is a quick and dirty solution, so don't judge

Code: Select all

import ij.*;
import ij.process.*;
import ij.gui.*;
import java.awt.*;
import ij.plugin.*;
import ij.plugin.frame.*;
import loci.plugins.BF;
import loci.plugins.in.*;
import loci.common.*;

public class My_Plugin implements PlugIn {

   public void run(String arg) {

     String id = "/Users/cbilgin/x40-01.czi";
     try{
      ImporterOptions options = new ImporterOptions();
      options.setId(id);
      options.setAutoscale(true);
      options.setCrop(true);
      options.setCropRegion(0, new Region(60000, 60000, 512, 512));
      options.setColorMode(ImporterOptions.COLOR_MODE_COMPOSITE);
      //...etc.
      ImagePlus[] imps = BF.openImagePlus(options);
      imps[0].show();
     }
     catch(Exception ioex){
      System.out.println(ioex);
      // do nothing!
     }

   }

}

[bramalingam]

Hi,

Thank you for submitting your code snippet. That looks good to read images.

We have a couple of example code snippets on our bio-formats: ImageJ Overview page,
https://www-legacy.openmicroscopy.org/s ... rs/imagej/

and for your purposes you could take a look at,
https://github.com/openmicroscopy/biofo ... Image.java

This also has an example method for applying lookup tables, similarly you could add a method for cropping images.

For the second part of your workflow (writing the cropped images), please take a look at the following page,
https://www-legacy.openmicroscopy.org/s ... brary.html

This contains example java code snippets for reading and writing images, and specifically :
https://github.com/openmicroscopy/biofo ... ample.java

would give you an idea of how to export cropped regions in an image.

Hope that helps.

Best,
Balaji