Why not having own schema besides OME one?

[Steffen_SI]

Hallo,
as I know (please correct me if I am wrong) it is not possible to have an own schema next to the ome one? Because I thought of simply using an own schema for all our tags which are not specified yet in ome and when they will get part of ome in future, I could simply move the tag from our schema to the ome's one.

I thought of something like
<OME xmlns="http://www.openmicroscopy.org/Schemas/OME/2009-09" xmlns:OUR_LAB="http://www.xyz.OUR_LAB" ...>

Regards,
Steffen

[cxallan]

You just need to have a different top level element. There is nothing preventing you from having a tag layout such as:

Code: Select all

<OurLab>
    <OME>
        ...
    </OME>
    <OurLabTagOne>
        ...
    </OurLabTagOne>
    <OurLabTagTwo>
        ...
    </OurLabTagTwo>
    ...
</OurLab>


This is of course not something we encourage and is not a valid OME document. It also cannot be read by OME-XML compliant software. Which tags do you find currently missing from the OME schema specifically?

[Steffen_SI]

Hey,

of cource I dont want to have a file which is no more supported by any ome software. But there will always be tags which are not considered, no matter how complex the schema is.

In our case e.g.

- DPSS as an laser type
- some more tags to describe the experiments like "date"
- additional instruments, als "tube lens", "excitation lens"...

just to name a few.

I got an reply, telling me that it is working, I hope that it is OK if I post it here
---------the reply------------
Hi Steffan,

Absolutely, you have the right idea. If you have created your own schema, and are hosting it on your web server, you can indeed reference it in that fashion. Then you just use the "OUR_LAB" namespace with elements from your schema.

Of course, anything in your XML blocks not from the OME schema will likely be ignored by Bio-Formats and other OME-compatible software.

Let us know on the OME users mailing list (http://lists.openmicroscopy.org.uk/mail ... ome-users/) if you have any problems with it.
-------------------------------------
------------my answer-------------
What I described in my post (below) is not working. Let me give you an example, probably you will find my mistake.

In the ome schema of 2009 (see http://www.openmicroscopy.org/Schemas/OME/2009-09) you will find a tag called "MiddleName" in "Experimenter", which is not there in the schema of 2008.

If you say

Code: Select all

<OME xmlns="http://www.openmicroscopy.org/Schemas/OME/2008-09"  xmlns:ome2009="http://www.openmicroscopy.org/Schemas/OME/2009-09" ....>
...
<Experimenter ID="Experimenter:sillig">
    <FirstName>Stefan</FirstName>
    <ome2009:MiddleName>Bernhard</ome2009:MiddleName>
    <LastName>Schnell</LastName>
    <Email>xyz@mpimf.de</Email>
    <Institution>MaxPlanckInstituteMF</Institution>
    <GroupRef ID="Group:Lightmicroscopy1"/>
</Experimenter>
...


it will not be validated.
-------------------------
Why is it not possible to embed tag like in the example above? It would be absolutely flexible, fit any needs and ome software could read the ome-conform tags.

Regards Steffen

[cxallan]

Without an xsd:any declaration at every level what you're saying would not be possible. This is not how we structure our schema for many reasons. Fundamentally It makes parsing and using instance documents incredibly cumbersome due to the sheer number of possibilities and the need to hard code tag types to know what is expected.

The extensibility mechanism we have at present is called structured annotations. These are primitive types that can be associated to various levels of the schema and include an XML annotation with an xsd:any like block. You can read about how structured annotations are used in OMERO here:

http://trac.openmicroscopy.org.uk/omero ... nnotations

and you can have a look at the schema here:

http://www.openmicroscopy.org/Schemas/SA/2009-09/SA.xsd

This allows us to control the areas of the schema that are dynamic, facilitate structured tags within an instance document and ensure that certain blocks remain true to the specification.

[Steffen_SI]

But it is only possible to use the structered annotations at some limited places. Take the example of the "DPSS" as an additional laser type for example. Structered annotations will not help much here.

You say if it would be possible to edit tags freely (such as

Code: Select all

<ourlab:TubeLens>...</ourlab:TubeLens>

) the documents will become too cumbersome. But the labs will know what the document is saying (as it exactly refers to theirs needs) and the ome-software goes trough the documents very fast, just reading the well known tags. And if more and more tags are implemented in future ome schemas, one can simply move the "ourlab"-tags to ome-tags.

I think that would be the best solution. What do you mean?

Steffen

[cxallan]

As all OME objects can be referenced by ID, structured annotations can point to any object in the entire instance document. So one can in fact use structured annotations anywhere. Their only restriction is that their location within the tag hierarchy is limited.

The strategy you propose where anyone can write any tags anywhere is not in line with the goal of OME schema development, established controlled vocabulary design or data modeling. We have chosen to limit the extension points for a number of simple reasons that include tag clutter, consistent naming, consistent use of tags vs. attributes and ease of adoption.

Using your tube lens as an example:

Code: Select all

<Objective ID="Objective:1">
...
</Objective>
...
<StructuredAnnotations>
   <XMLAnnotation>
      <ourlab:TubeLens>
          <ObjectiveRef ID="Objective:1"/>
          ...
      </ourlab:TubeLens>
    </XmlAnnotation>
</StructuredAnnotations>
...


The above allows you to extend your modeling however you wish, making whatever additional statements about the objective that you wish, use whatever additional schemas, etc. and keeps the structure that many OME implementers value.

I agree that inline tag extension has utility and many may prefer it, unfortunately it does not fit with our style or our design choices. It's absence from valid OME XML instance documents also does not limit their extensibility.

[nic]

hi,

Working on the same problem like Steffen, extending the ome schema with individual data, I have some questions about using StructredAnnotations:

1. Why are there only a few types that can be annotated? Wouldn't it make sense to add the AnnotationRef element to every existing type?
That would give everyone the opportunity to extend the ome schema, without interrupting the validator.
Is it possible to access the annotated data at the right position using AnnotionRefs in OMERO.insight?

2. If I use (like in your example) the XMLAnnotation for individual data referencing on an OME Objekt, how would this affect the view of metadata in OMERO.insight? Would it be possible to access the data at the right position?

3. Last Question: What ist the Namespace attribute of the StructuredAnnotations elements for? I can't find it in the SA schema, but the validator is asking for it.

Thanks,

nic

[jmoore]

Hi nic,

1. Why are there only a few types that can be annotated? Wouldn't it make sense to add the AnnotationRef element to every existing type?
That would give everyone the opportunity to extend the ome schema, without interrupting the validator.
Is it possible to access the annotated data at the right position using AnnotionRefs in OMERO.insight

This is something we've also discussed many times internally. In the end there's a a trade-off between flexibility and simplicity. What element would you like to annotate which 1) isn't annotatable or 2) isn't attached to an annotatable element? We'd definitely be interested in making sure all of these are supported. However, the complexity of parsing a file which can have embedded data at any point is prohibitive. There's a similar issue when tying to find a meaningful way to display all the possible extensions in the UI.

Instead, the current method would encourage schema-extenders to use larger blocks of their own XML which reference one or multiple items from the OME-XML section via IDs and REFs. This block can then get attached at the most appropriate level (e.g. Image, Project, Plane, Channel)

2. If I use (like in your example) the XMLAnnotation for individual data referencing on an OME Objekt, how would this affect the view of metadata in OMERO.insight? Would it be possible to access the data at the right position?

What would you expect to see? It's certainly possible to make the raw XML available, but would you like to see it parsed in some way? I'm not sure if it would work for you, but XSL comes to mind.

3. Last Question: What ist the Namespace attribute of the StructuredAnnotations elements for? I can't find it in the SA schema, but the validator is asking for it.

Which schema version are you looking at/using? The namespace is intended to describe the source/intent of a structured annotation as defined by the annotator. In the case of an XML annotation, this is redundant because the namespace of the XML can be used for a similar purpose; for the other annotation types, however, this becomes necessary to avoid parsing the annotation itself.

Cheers,
~J.

[nic]

hi!

Thank you for your quick and detailed answer.

I am using the 2009/09 SA schema and just found the Namespace attribute... For some reason I didn't find it the last time

[...] In the case of an XML annotation, this is redundant because the namespace of the XML can be used for a similar purpose; for the other annotation types, however, this becomes necessary to avoid parsing the annotation itself.

Does that mean, the annotations are not parsed at all?
Because my main concern is not only to add the annotations ome-conform, but to view the annotation of an element at the right position of the tree structure of the meta data.
Unfortunately I haven't been able to work with OMERO.insight yet (dont have server access yet), but I am sure it is possible to show the ome metadata in a tree view or something, right?

With the AnnotationRef element you should be able to show the annotation of an element at the right position:
e.g. (using Steffen's example of a DPSS laser) :

-ome
--Instrument
---Microscope
---Lightsource
----Laser ( [...], Type="Other")
-----Annotation( -> "Laser: DPSS")
.
.
.


The annotation element is just the reference to any existing annotation. This would be a great way to extend the lasertype at the exact position.

Using XMLAnnotations, with a reference on the LaserID wouldn't show the individual data at the right position in the tree view. So I would expect it as an own element at the end, just showing the xml code.
You are right, to display the data in a proper way, it would be necessary to use xsl.

nic

[cxallan]

The support for parsing and associating annotations from XML documents at import isn't very well supported in OMERO Beta 4.1.x. It is one of the things we will be improving in 4.2.x. How we will potentially visualize annotations is still a bit up in the air but a tree like structure, as you've already outlined, is something we have discussed. More details should be available as we get closer to working on annotations for 4.2.0.