OME Structure

[visitech]

I'm current working on our software to write OME-TIFFs and wanted to know where the best place to store the following:

- Scan rate (Fps) of the confocal
- Wavelength (nm) of the laser
- Channel power (%) of the laser
- Dichroic & Barrier filter

Also I have ran the OME validator with some test images and it came out with no errors. However when I try to load it into JImage, it returns errors when attempting to read the meta-data.

Edit: Is there any other way of checking the validity of our OME structure or is it just through the OME.qa website?

Thanks,
Matt.

[ajpatterson]

Hello Matt,

> I'm current working on our software to write OME-TIFFs and wanted to know where the best place to store the following:

Always great to hear from people writing to our file format.

> - Scan rate (Fps) of the confocal

We do not have a location to directly store a scan rate.
We have a TimeIncrement on Pixels:
http://www.openmicroscopy.org/Schemas/D ... tml#Pixels

We also have a value DeltaT to store the time at acquisition of each Plane
http://www.openmicroscopy.org/Schemas/D ... html#Plane

> - Wavelength (nm) of the laser

http://www.openmicroscopy.org/Schemas/D ... html#Laser

> - Channel power (%) of the laser

The constant power of the laser in milliwatts[mW] can be store in LightSource
http://www.openmicroscopy.org/Schemas/D ... ightSource

If you want to vary the percentage with channel then you need to use a LightSourceSettings object in each channel. This has Attenuation %:
http://www.openmicroscopy.org/Schemas/D ... ceSettings

> - Dichroic & Barrier filter

The filter system is described in the docs here:
http://www.openmicroscopy.org/site/supp ... erset.html
Your barrier filter would map to the models Emission filter.
You probably want to define a LightPath and populate it:
http://www.openmicroscopy.org/Schemas/D ... #LightPath

> Also I have ran the OME validator with some test images and it came out with no errors. However when I try to load it into JImage, it returns errors when attempting to read the meta-data.

The validator can only check the structure of the file and internal references. It does not validate the values stored in the metadata other than checking they are the correct type.

> Edit: Is there any other way of checking the validity of our OME structure or is it just through the OME.qa website?

All the QA website does is submit files. I have been validation them manually. As there was no email address attached to the files I had no way to reply. This may we a bug with the QA system, we are checking. I have added comments to the different files sets I think you have submitted, links to them are below:
http://qa.openmicroscopy.org.uk/qa/feed ... b6c18254bb
http://qa.openmicroscopy.org.uk/qa/feed ... f7ea33f1e3
http://qa.openmicroscopy.org.uk/qa/feed ... 46977062ce

I was using the xmlvalid command, part of the bioformat command line tools:
http://www.openmicroscopy.org/site/supp ... t=xmlvalid
It can also process OME-TIFF files, e.g. xmlvalid myfile.ome.tiff

Another useful command is showinf, this performs a test read of a file using bio-formats.

Oh, if you are going to be writing Florsence Lifetime data check out:
http://www.openmicroscopy.org/site/supp ... orage.html
and our first solution using these structures:
http://www.openmicroscopy.org/site/prod ... er/flimfit

Hope this helps,

Andrew

[visitech]

Hi,

Sorry for the late reply since I have been tied up with other things. Your post was very helpful and cleared up a lot of the queries we had but we are still encountering some issues.

I have made numerous replies on the QA feedback but it seems like it does not send.

Here is the link to the QA: https://www.openmicroscopy.org/qa2/qa/f ... 26912ddac1

So at the moment I am looking into the Z-stack files that aren't importing.
Currently we define in the Pixels tag the number of Z-Stacks (SizeZ) being equal to time frames (SizeT).

e.g. 10 stacks - SizeZ = 10 & SizeT = 10.

I assume that this maybe the problem since it is then expecting 20 entries instead of the 10 that we write?
The reason we did this is that as we move in Z, time also passes so we assumed that there would be 10 time frames with 10 Z-stack positions.
So to fix this would we simply make the SizeZ attribute always = 1?

We also have another issue where the files we generate do not load into ImageJ when loading through the Bio-formats plugin. Here is the error we are getting when we try to import and attempt to display the OME-XML meta data.

Code: Select all

java.lang.RuntimeException: org.xml.sax.SAXParseException: The markup in the document following the root element must be well-formed.
   at ome.xml.model.XMLAnnotation.asXMLElement(XMLAnnotation.java:251)
   at ome.xml.model.XMLAnnotation.asXMLElement(XMLAnnotation.java:221)
   at ome.xml.model.StructuredAnnotations.asXMLElement(StructuredAnnotations.java:612)
   at ome.xml.model.StructuredAnnotations.asXMLElement(StructuredAnnotations.java:593)
   at ome.xml.model.OME.asXMLElement(OME.java:781)
   at ome.xml.model.OME.asXMLElement(OME.java:673)
   at loci.formats.ome.AbstractOMEXMLMetadata.dumpXML(AbstractOMEXMLMetadata.java:117)
   at loci.formats.ome.OMEXMLMetadataImpl.dumpXML(OMEXMLMetadataImpl.java:99)
   at loci.formats.services.OMEXMLServiceImpl.getOMEXML(OMEXMLServiceImpl.java:396)
   at loci.plugins.in.DisplayHandler.displayOMEXML(DisplayHandler.java:108)
   at loci.plugins.in.Importer.run(Importer.java:85)
   at loci.plugins.LociImporter.run(LociImporter.java:81)
   at ij.IJ.runUserPlugIn(IJ.java:196)
   at ij.IJ.runPlugIn(IJ.java:160)
   at ij.Executer.runCommand(Executer.java:131)
   at ij.Executer.run(Executer.java:64)
   at java.lang.Thread.run(Thread.java:619)
Caused by: org.xml.sax.SAXParseException: The markup in the document following the root element must be well-formed.
   at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:249)
   at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
   at ome.xml.model.XMLAnnotation.asXMLElement(XMLAnnotation.java:247)


The file we used to test the load to ImageJ was multiTest1_1.ome.tif.

Thanks,

Matt.

[wmoore]

Hi, apologies for the delay - Andrew is away just now.

If you have a stack of planes that are conceptually a Z-stack, then you should set SizeZ = 10 and SizeT = 1. Although it's true that every plane will have a different timestamp, you should only use SizeT > 1 for movies.

You can record the time that a Plane is acquired using the DeltaT attribute of Plane:
http://www.openmicroscopy.org/Schemas/D ... ane_DeltaT

I'm going to have to leave your second question to Andrew or someone else, but we'll reply ASAP,

Will.

[wmoore]

For what it's worth, I tried looking at the SAX parsing error but didn't get very far. I got the same error as you on opening that ome.xml file in ImageJ with BF. But when I use tiffcomment and xmlvalid from http://www.openmicroscopy.org/site/supp ... linetools/ to extract ome.xml and then validate it, I get no errors (using the same loci_tools.jar as in ImageJ.

Code: Select all

$ xmlvalid ../multiTest1_1.ome.xml
Parsing schema path
http://www.openmicroscopy.org/Schemas/OME/2013-06/ome.xsd
Validating ../multiTest1_1.ome.xml
No validation errors found.


Playing a bit more with bf tools, bfconvert failed to read the file (same error as in ImageJ):

Code: Select all

$ bfconvert ../7574/multiTest1_1.ome.tif ../7574/output_multiTest1_1.ome.tiff
../7574/multiTest1_1.ome.tif
Reading IFDs
Populating metadata
[OME-TIFF] -> ../7574/output_multiTest1_1.ome.tiff [OME-TIFF]
Reading IFDs
Populating metadata
Reading IFDs
Populating metadata
Reading IFDs
Populating metadata
   Converted 33/40 planes (82%)
   Converted 40/40 planes (100%)
[Fatal Error] :1:79: The markup in the document following the root element must be well-formed.
Exception in thread "main" java.lang.RuntimeException: org.xml.sax.SAXParseException: The markup in the document following the root element must be well-formed.
   at ome.xml.model.XMLAnnotation.asXMLElement(XMLAnnotation.java:251)
   at ome.xml.model.XMLAnnotation.asXMLElement(XMLAnnotation.java:221)
   at ome.xml.model.StructuredAnnotations.asXMLElement(StructuredAnnotations.java:612)
   at ome.xml.model.StructuredAnnotations.asXMLElement(StructuredAnnotations.java:593)
   at ome.xml.model.OME.asXMLElement(OME.java:781)
   at ome.xml.model.OME.asXMLElement(OME.java:673)
   at loci.formats.ome.AbstractOMEXMLMetadata.dumpXML(AbstractOMEXMLMetadata.java:109)
   at loci.formats.ome.OMEXMLMetadataImpl.dumpXML(OMEXMLMetadataImpl.java:99)
   at loci.formats.services.OMEXMLServiceImpl.getOMEXML(OMEXMLServiceImpl.java:393)
   at loci.formats.out.OMETiffWriter.setupServiceAndMetadata(OMETiffWriter.java:243)
   at loci.formats.out.OMETiffWriter.close(OMETiffWriter.java:104)
   at loci.formats.ImageWriter.close(ImageWriter.java:456)
   at loci.formats.tools.ImageConverter.testConvert(ImageConverter.java:481)
   at loci.formats.tools.ImageConverter.main(ImageConverter.java:632)
Caused by: org.xml.sax.SAXParseException: The markup in the document following the root element must be well-formed.
   at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:249)
   at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
   at ome.xml.model.XMLAnnotation.asXMLElement(XMLAnnotation.java:247)
   ... 13 more


Trying $ showinf gets a bit further, then hangs.
This, together with the SAX error suggests the problem is in the XML annotation. Does it parse OK if you remove that?

Code: Select all

$ showinf ../7574/multiTest1_1.ome.tif
Checking file format [OME-TIFF]
Initializing reader
Reading IFDs
Populating metadata
Initialization took 0.355s

Reading core metadata

..
Reading pixel data (0-39)
Reading IFDs
Populating metadata
Reading IFDs
Populating metadata
Reading IFDs
Populating metadata
   Read 40/40 planes (100%)
[done]
0.323s elapsed (8.075ms per plane)

Launching image viewer

Reading global metadata

Reading metadata


[mlinkert]

The problem is that the XMLAnnotation value does not contain a valid root element, which is required for the annotation to be properly parsed and validated. Changing the XMLAnnotation so that it looks something like:

Code: Select all

<SA:XMLAnnotation ID="..." Namespace="...">
            <SA:Value>
                <Root>
                <!-- contents of current value here -->
                </Root>
            </SA:Value>
</SA:XMLAnnotation>


should solve the problem. "Root" could be replaced with anything else you want.

[ajpatterson]

Hello Matt,

It looks like you have found a point where the specification and Bio-Formats implementation differ.
Sorry about that!

According the the specification it is valid to have an unlimited number of child nodes inside the Value element.

The Bio-Formats code however expects there to be a single child element, though this of course can itself contain multiple child elements.

I will look into this and see how we want to resolve this long term.

In the mean time Melissa's approach is the correct one.

Thanks,

Andrew