different results with ome.tiff and ome.xml

[dwight]

I'm trying to convert an Olympus OIB image to OME and I'm getting two different results depending on whether I use OME-XML or OME-TIFF as output. The microscope file has two channels and in the end results I'd expect to see 2 Image elements, both with 2 channel and 2 BinData (2 channels 394x2500 pixels and 2 reference pictures 512x512 pixels).
Output from showinf tells me that one Image series has disappeared:

Code: Select all

Reading core metadata
Filename = 36032aeb31.ome
Series count = 1
Series #0 :
    Image count = 2
    RGB = false (1)
    Interleaved = false
    Indexed = false (false color)
    Width = 394
    Height = 2500
    SizeZ = 1
    SizeT = 1
    SizeC = 2
    Thumbnail size = 20 x 128
    Endianness = intel (little)
    Dimension order = XYCZT (certain)
    Pixel type = uint16
    Valid bits per pixel = 16
    Metadata complete = false
    Thumbnail series = false
    -----
    Plane #0 <=> Z 0, C 0, T 0
    Plane #1 <=> Z 0, C 1, T 0


Reading pixel data (0-1)
    Read 2/2 planes (100%)
[done]
0.579s elapsed (289.5ms per plane)


With OME-XML I get one image tag with two channels and four BinData elements. Since there is only one Pixel element then the pixel size and count information for the reference image disappears. Below are the relevant parts of the XML:

Code: Select all

<Instrument ID="Instrument:0">
<Image ID="Image:0" Name="Series 1">
<AcquisitionDate>2012-07-26T10:30:29</AcquisitionDate>
<InstrumentRef ID="Instrument:0"/>
<Pixels DimensionOrder="XYCZT" ID="Pixels:0" PhysicalSizeX="0.207" PhysicalSizeY="0.207" PhysicalSizeZ="1.0" SizeC="2" SizeT="1" SizeX="394" SizeY="2500" SizeZ="1" TimeIncrement="1.0" Type="uint16">
<Channel EmissionWavelength="515" ExcitationWavelength="488" ID="Channel:0:0" IlluminationType="Epifluorescence" Name="CH1" SamplesPerPixel="1">

<Channel ExcitationWavelength="488" ID="Channel:0:1" IlluminationType="Epifluorescence" Name="CH2" SamplesPerPixel="1">

<BinData BigEndian="false" Length="0" xmlns="http://www.openmicroscopy.org/Schemas/BinaryFile/2012-06"/>
<BinData BigEndian="false" xmlns="http://www.openmicroscopy.org/Schemas/BinaryFile/2012-06"/>
<BinData BigEndian="false" xmlns="http://www.openmicroscopy.org/Schemas/BinaryFile/2012-06"/>
<BinData BigEndian="false" xmlns="http://www.openmicroscopy.org/Schemas/BinaryFile/2012-06"/>
<BinData BigEndian="false" xmlns="http://www.openmicroscopy.org/Schemas/BinaryFile/2012-06"/>


For an OME-TIFF image, the contents are correct

Code: Select all

Series count = 2
Series #0 :
    Image count = 2
    RGB = false (1)
    Interleaved = false
    Indexed = false (false color)
    Width = 394
    Height = 2500
    SizeZ = 1
    SizeT = 1
    SizeC = 2
    Thumbnail size = 20 x 128
    Endianness = intel (little)
    Dimension order = XYCZT (certain)
    Pixel type = uint16
    Valid bits per pixel = 16
    Metadata complete = true
    Thumbnail series = false
    -----
    Plane #0 <=> Z 0, C 0, T 0
    Plane #1 <=> Z 0, C 1, T 0

Series #1 :
    Image count = 2
    RGB = false (1)
    Interleaved = false
    Indexed = false (false color)
    Width = 512
    Height = 512
    SizeZ = 1
    SizeT = 1
    SizeC = 2
    Thumbnail size = 128 x 128
    Endianness = intel (little)
    Dimension order = XYCZT (certain)
    Pixel type = uint16
    Valid bits per pixel = 16
    Metadata complete = true
    Thumbnail series = false
    -----
    Plane #0 <=> Z 0, C 0, T 0
    Plane #1 <=> Z 0, C 1, T 0


Reading series #0 pixel data (0-1)
    Read 2/2 planes (100%)
[done]
0.232s elapsed (116.0ms per plane)


I'd use the OME-TIFF format if I could, but my program is in Python and I can read XML without having to install extra packages for reading tiff files. I'd appreciate any help on getting the correct data but using the OME-XML file.

[wmoore]

I'm not sure if I understand what you want exactly. Are you simply trying to use Bioformats to get the OME-XML for an Olympus file?

I have no experience of converting to OME-TIFF using BioFormats, but I have used this command to extract OME-XML for proprietary image files:

I guess the path depends on your class-path etc, but this is what I used:

Code: Select all

$ java -Xmx512m loci.formats.tools.ImageInfo -nopix -omexml-only -no-sas -xmlspaces 4 /path/to/file.oib


This prints out to console, but you could use > to direct it to an xml file (and call the whole command from Python too I guess).

Code: Select all

$ java -Xmx512m loci.formats.tools.ImageInfo -nopix -omexml-only -no-sas -xmlspaces 4 /path/to/file.oib > output.ome.xml


[dwight]

Thanks for the reply. Let me clarify myself a bit. My end-goal is to open images from microscopes (Olympus, Zeiss, etc) in Python. I've been using OME tools to convert images into OME-XML and then importing the image (metadata and pixel data (BinData elements) both) into Python by parsing the OME-XML file. Up till now this has worked without problem. But I've recently started using multiple channel images and it seems that the OME-XML that is produced by ImageConverter does not contain all the necessary data about my image. More specifically it piles all the BinData into a single Image element, which results in me not being able to 1) get the pixel size and image dimension information for the series and 2) forces me to guess which of the BinData elements belong to which series.

On the other hand, if I convert the microscope images into OME-TIFF format then the XML is correct. Unfortunately I don't have a way of reading TIFF images in a platform independent way and without installing extra Python packages. That's why it would be great if the XML contained in OME-XML would correspond to what is in OME-TIFF (as far as metadata is concerned).

The command you provided gives out the correct XML with both Image elements present. Unfortunately it still leaves the problem of obtaining the pixel data.

Is there a reason why there is a difference between the amount of Image elements in the XML in OME-XML and OME-TIFF generated from the same microscope file?

[mlinkert]

Thank you for clarifying the problem - as it turns out, this is only an issue with files that contain multiple Images and multiple channels.

We now have a fix pending review for inclusion in the next release:

https://github.com/openmicroscopy/bioformats/pull/246

[dwight]

Thanks for the quick fix! I've rebuilt loci_tools.jar and now the OME-XML files contain both Pixel elements. Unfortunately a new problem has surface. The BinData elements are not within the Pixel elements. As far as I understand the schema, the BinData elements should not be children of the Image element. The generated XML also fails validation.

Code: Select all

   <Image ID="Image:0" Name="Series 1">
      <AcquisitionDate>
         2012-07-26T10:30:29
      </AcquisitionDate>
      <InstrumentRef ID="Instrument:0">
      </InstrumentRef>
      <ObjectiveSettings ID="Objective:0:0">
      </ObjectiveSettings>
      <Pixels DimensionOrder="XYCZT" ID="Pixels:0" PhysicalSizeX="0.207" PhysicalSizeY="0.207" PhysicalSizeZ="1.0" SizeC="2" SizeT="1" SizeX="394" SizeY="2500" SizeZ="1" TimeIncrement="1.0" Type="uint16">
         <Channel EmissionWavelength="515" ExcitationWavelength="488" ID="Channel:0:0" IlluminationType="Epifluorescence" Name="CH1" SamplesPerPixel="1">
            <LightSourceSettings ID="LightSource:0:0" Wavelength="488">
            </LightSourceSettings>
            <DetectorSettings ID="Detector:0:0">
            </DetectorSettings>
            <LightPath>
               <DichroicRef ID="Dichroic:0:1">
               </DichroicRef>
               <EmissionFilterRef ID="Filter:0:0">
               </EmissionFilterRef>
            </LightPath>
         </Channel>
         <Channel ExcitationWavelength="488" ID="Channel:0:1" IlluminationType="Epifluorescence" Name="CH2" SamplesPerPixel="1">
            <LightSourceSettings ID="LightSource:0:1" Wavelength="488">
            </LightSourceSettings>
            <DetectorSettings ID="Detector:0:1">
            </DetectorSettings>
            <LightPath>
               <DichroicRef ID="Dichroic:0:3">
               </DichroicRef>
               <EmissionFilterRef ID="Filter:0:1">
               </EmissionFilterRef>
            </LightPath>
         </Channel>
         <BinData xmlns="http://www.openmicroscopy.org/Schemas/BinaryFile/2012-06" BigEndian="false" Length="0">
         </BinData>
      </Pixels>
      <BinData xmlns="http://www.openmicroscopy.org/Schemas/BinaryFile/2012-06" Length="1970000" BigEndian="false" Compression="zlib">
...


This affects single channel images as well

Code: Select all

   <Image ID="Image:0" Name="Series 1">
      <AcquisitionDate>
         2012-08-20T10:02:02
      </AcquisitionDate>
      <InstrumentRef ID="Instrument:0">
      </InstrumentRef>
      <ObjectiveSettings ID="Objective:0:0">
      </ObjectiveSettings>
      <Pixels DimensionOrder="XYZCT" ID="Pixels:0" PhysicalSizeX="0.295" PhysicalSizeY="0.295" PhysicalSizeZ="1.0" SizeC="1" SizeT="1" SizeX="372" SizeY="10000" SizeZ="1" TimeIncrement="1.0" Type="uint16">
         <Channel EmissionWavelength="527" ExcitationWavelength="488" ID="Channel:0:0" IlluminationType="Epifluorescence" Name="CH1" SamplesPerPixel="1">
            <LightSourceSettings ID="LightSource:0:0" Wavelength="488">
            </LightSourceSettings>
            <DetectorSettings ID="Detector:0:0">
            </DetectorSettings>
            <LightPath>
               <DichroicRef ID="Dichroic:0:1">
               </DichroicRef>
               <EmissionFilterRef ID="Filter:0:0">
               </EmissionFilterRef>
            </LightPath>
         </Channel>
         <BinData xmlns="http://www.openmicroscopy.org/Schemas/BinaryFile/2012-06" BigEndian="false" Length="0">
         </BinData>
      </Pixels>
      <BinData xmlns="http://www.openmicroscopy.org/Schemas/BinaryFile/2012-06" Length="7440000" BigEndian="false" Compression="zlib">
...


Also, there are some weird zero-length BinData elements in Pixel. But those were present already before the new update and so far I've just ignored them.

I have implemented a way of reading OME-TIFF files as an alternative, but at the moment data access from TIFF is roughly 4 times slower then from XML, so I'd be very happy if the OME-XML would work. Please let me know I there is anything I can help with so as to resolve this issue.

[mlinkert]

Sorry about that - I should have tested the fix more carefully.

There is now one more set of fixes under review here:

https://github.com/openmicroscopy/bioformats/pull/265

[dwight]

Everything seems to work now! Thanks a lot!

Edit: One side-effect I've now noticed now is that the time taken for conversion has more than doubled. For the same image the conversion times are:
OIB -> OME-TIFF: 7 seconds
OIB -> OME-XML (old code): 8 seconds
OIB -> OME-XML (new code): 22 seconds

[sbesson]

Hi,

thank you for reporting the conversion time increase. We noticed the same side-effect during the review of https://github.com/openmicroscopy/bioformats/pull/265. We are working on a way to keep the same level of performance and will keep you posted about this.

Sebastien