feature request: bfconvert to generate pyramid in OME-TIFF

[dsudar]

Hi team,

With the new OMERO Pyramid and WSI handling in 5.7, I would like to request new features in OME-TIFF and bfconvert: have bfconvert create the pyramid in the OME-TIFF output files. In one of our workflows we create our own large images and would like to optimize the OMERO import and other downstream use by having the pyramid already in the OME-TIFF file. For now that would be easiest accomplished if bfconvert can do that.

Thanks,
- Damir

[sbesson]

Hi Damir,

With the new OMERO Pyramid and WSI handling in 5.7, I would like to request new features in OME-TIFF and bfconvert: have bfconvert create the pyramid in the OME-TIFF output files

The main technological requirement for this feature request would be to add support for pyramidal resolutions in the OME-TIFF file format specification and possibly in the OME Data Model. See this issue for an initial discussion on possible ways to extend this specification.

With a new backwards-compatible release of the file format specification and potentially the OME Model API component, the OME-TIFF readers and writers would also require to be updated both in Bio-Formats and OME Files C++ to handle pyramidal OME-TIFFs.

This feature has been on the wishlist for some time but we have not prioritized it yet. As mentioned above, a number of critical decisions on the model and the OME-TIFF specification would first need to be agreed upon to move forward.

In one of our workflows we create our own large images and would like to optimize the OMERO import and other downstream use by having the pyramid already in the OME-TIFF file.

As you are generating your own large images, storing the pyramidal levels precomputed locally within the image file format makes complete sense and would certainly reduce the burden on the OMERO server.

In addition to the discussion above, there might be other possibilities worth mentioning:

• the first one would be to store the pixel data using the OMERO TIFF-based pyramid format,
• the second option would be to store your pyramids using another open file format read by Bio-Formats with existing pyramid support. At the time of this post, the Imaris File Format would probably be the primary candidate.

For now that would be easiest accomplished if bfconvert can do that.

One workflow question: os your input data another file format that is also read via Bio-Formats or are you primarily interested in using the writing API to generate multi-resolution images?

Best,
Sebastien

[dsudar]

Hi Sebastien,

Thanks for addressing my request and yes, I see that there is still a long road ahead with having such functionality fully within the OME-TIFF infrastructure. In my opinion it would be a very worthwhile road however considering how many microscopy imaging modalities are moving very rapidly to very large format images including WSI, lightsheet, various blockface EM methods (FIB/ and SBF/ SEM), etc. And of course, updating the premier truly open microscopy image format to having that functionality is very important. I'm sure I'm preaching to the choir here

As you are generating your own large images, storing the pyramidal levels precomputed locally within the image file format makes complete sense and would certainly reduce the burden on the OMERO server.

Indeed, one benefit of having the pyramids in the original files will lower the burden on the server during import but (and I may be wrong here) I also noticed that files with built-in pyramids appear to render much faster than files for which the pyramid was created upon import and stored in the repo. Does that make sense?

In addition to the discussion above, there might be other possibilities worth mentioning:

the first one would be to store the pixel data using the OMERO TIFF-based pyramid format,
the second option would be to store your pyramids using another open file format read by Bio-Formats with existing pyramid support. At the time of this post, the Imaris File Format would probably be the primary candidate.

So option 1 will have to wait, correct? But indeed I looked at what other formats DO support pyramids and I hadn't realized that Bitplane has made .ims an open file format. That's definitely an option until OME-TIFF can do it. I see that the format specification is on: http://open.bitplane.com/Default.aspx?tabid=268 and reader code (in Java) is presumably available in the Bio-Formats library. Are you aware of any writer code (ideally C/C++) freely available?

One workflow question: is your input data another file format that is also read via Bio-Formats or are you primarily interested in using the writing API to generate multi-resolution images?

I have 2 different workflows related to this:
- files are created by a FEI software called MAPS which generates very large plain TIFFs and I've been struggling with those (see e.g.: http://www.openmicroscopy.org/community ... f=4&t=8380) so I was thinking to pass them through bfconvert and make them into something more manageable.
- for our highly multiplex IF work, we take a series of multi-channel WSI images and create a highly multi-channel WSI image set (30-100 channels) in our own software and I'm looking how to save that out (ideally into an open format) so it can be imported into OMERO and downstream into other visualization/analysis software (incl. Glencoe's PathViewer). So for this an API, ideally OME-Files, would be needed.

Cheers,
- Damir

[sbesson]

Hi Damir,

Indeed, one benefit of having the pyramids in the original files will lower the burden on the server during import but (and I may be wrong here) I also noticed that files with built-in pyramids appear to render much faster than files for which the pyramid was created upon import and stored in the repo. Does that make sense?

From the rendering engine perspective there should be no fundamental difference in the way internal OMERO pyramids or native pyramids are handled. Performance discrepancies could be be reflect the way the pixel data is stored and accessed on disk. In this case, reading performance will probably strongly depend on the specifics of the file format and the implementation. Do you have any performance metrics using image of similar dimensionalities (number of resolutions, resolution sizes) and pixel types?

So option 1 will have to wait, correct?

Actually, the internal OMERO pyramid format is completely independent of OME-TIFF. In terms of specification, the next minor release of the OME Model and Formats technical documentation will include a minimal description of this format. In terms of implementation, the Java writer is currently located in the OMERO codebase.

But indeed I looked at what other formats DO support pyramids and I hadn't realized that Bitplane has made .ims an open file format. That's definitely an option until OME-TIFF can do it. I see that the format specification is on: http://open.bitplane.com/Default.aspx?tabid=268 and reader code (in Java) is presumably available in the Bio-Formats library. Are you aware of any writer code (ideally C/C++) freely available?

Yes, Bio-Formats and most specifically the ImarisHDFReader supports the reading of the Imaris file format. With regard to writer code, I am not aware of any reference library. Note also that the Bitplane specification page mentions using the hdf5 library directly.

Best,
Sebastien

[dsudar]

Hi Sebastien,

From the rendering engine perspective there should be no fundamental difference in the way internal OMERO pyramids or native pyramids are handled. Performance discrepancies could be be reflect the way the pixel data is stored and accessed on disk. In this case, reading performance will probably strongly depend on the specifics of the file format and the implementation. Do you have any performance metrics using image of similar dimensionalities (number of resolutions, resolution sizes) and pixel types?

Upon a closer look, I think I was misled by the fact that some large image formats (specifically .svs and .czi (w/ jepg-xr)) perform very well during rendering but that's mostly because the pixel data is compressed while for similarly sized other files the pixel data is uncompressed or minimally compressed and thus much slower to retrieve and render.

For pre-generating and storing the pyramid into the files, I am also looking at TIFF itself. Unfortunately there does not appear to be a formal and standardized way to store the pyramid but there are a number of ad-hoc approaches:
1) how it's done in .svs which is simply a tiff (or bigtiff) with the lower resolutions in subsequent series entries,
2) how the IIPImage server appears to like it (see: http://iipimage.sourceforge.net/documentation/images/) where the lower resolutions are stuck into the time axis (horrible idea ...), and
3) whatever ImageMagick and VIPS do which again appears to be somewhat different.
There was a discussion about this on the libtiff forum (http://maptools-org.996276.n3.nabble.co ... 13896.html) which Roger participated in. Since BioFormats already handles Aperio's SVS approach, that might be the easiest but what are you thinking to do for OME.TIFF?

I'll keep the Imaris format on my radar but for usability, convenience of sharing, and true openness, my preference is to move as closely as possible to whatever OME-TIFF is going to be.

Cheers,
- Damir

[rleigh]

Hi Damir,

One question I have regarding pyramids is whether you have particular requirements for how the pyramids are created. The OMERO pyramids are using JPEG2000 to create the pyramids. With Bio-Formats bfconvert/OME Files, we would need to do the low-pass filtering and decimation ourselves for saving in TIFF. While there are plenty of different algorithms to achieve this, with various different tradeoffs, are there any which are preferred/standard for quantitative biological imaging, analysis and rendering?

For example, we could use lanczos2 (or 3?) to filter out high frequencies and then bicubic decimation for the size reduction. Would that be acceptable, or are there better choices?

We could certainly make such choices configurable to allow the user to select the algorithms to use if there are reasons for using different algorithms for different purposes.


Regards,
Roger

[dsudar]

Hi Roger,

Sorry for my delayed response. It's partly because I was trying to educate myself a little more and I now understand that this whole issue is indeed a bit more of a quagmire than I thought.

So in short I think you're right that the question which encoding/compression algorithm to use depends on the purpose so unfortunately that probably means that the writing/reading routines will ultimately need to support multiple encodings. So that can rapidly become a big deal.

However, maybe this cat can be skinned piecemeal as follows:
1) initially just define clearly how OME-TIFF requires pyramids to be laid out in the file similarly to how OME-TIFF defines how Z planes, channels, time series, etc. are done. This to avoid the many different methods employed by Aperio, Ventana, Philips, etc. And provide a defined way how to store an overview image and a slide label image into the file.
2) again initially just support the already defined compression schemes that OME-TIFF already supports such as JPEG, LZW, etc. And have those apply on per-tile or per-IFD basis as they currently do. I believe the Aperio approach is pretty much this way.
3) then in a next step, also support more advanced encodings such as JPEG2000 which do the pyramid and compression encoding all in one and that gives the community a bit of time to contemplate tricky questions such as what you are bringing up.

I know that even the above is not as simple as I make it sound and I'm trying to gather input from multiple sources to tighten up and be more specific about such an approach. And with this message I'm asking what the OME community and especially the OME team thinks about that.

Cheers,
- Damir