real-world coordinates for correlative microscopy

[dsudar]

Hi all,

With the great step forward in 5.1 of having real units on most of the information dimensions, I thought it would be opportune to bring up the topic of real-world coordinates and coordinate conversions again. In a brief talk at the 2014 Paris meeting (see: http://downloads.openmicroscopy.org/pre ... elatve.pdf) I touched on this topic in the context of correlative microscopy where we are trying to relate optical microscopy with electron microscopy images. Vendors of correlative systems have mostly chosen to go the way of sample holders that provide a "fixed" reference so samples can be moved between OM and EM and data collected with these systems can then (using their own proprietary software) be registered. With this message I hope to start a discussion how such functionality might become part of OMERO and provide a generalized approach to multi-modal image handling, dealing with "models" of detected objects in images separated from the pixels/voxels, etc.

Just to kick of the discussion, here follows a summary of a discussion I had with a few OME team members at the Paris meeting (after a glass of wine so ...... ):
-----------
1) On how to think about a "real-world coordinate system" we all
thought that for each image a 4x4 matrix describing a 3d affine
transform would be sufficient to capture that image's relation to
the "real-world" and thus allow it to be mapped onto real world
coordinates. The real-world coordinate system could either come from
some "outside" registration markers (e.g. part of the sample holder
or substrate) encoded in the image metadata, or could be imposed by
the user (or automatic image processing routine) by identifying
specific registration markers in an image, or the default would be
that the "first" image in a set would be considered the origin and
would get the identity matrix. A whole lot yet to be thought through
but all sounded right to me to enable multi-modal, multi-resolution,
correlative, etc. imaging. (oh, all hinges on having real units!!).
2) Fairly closely linked to #1 we discussed how a generalized new
(3D) ROI system could encode shapes (e.g. from segmenting the image
into objects) in the form of meshes or something. And by making this
ROI "layer" live inside the real-world coordinate system, it would
be de-coupled from the image and could thus be associated with other
images (e.g. from different imaging modes of the same sample) as
overlays and to define ROIs in that other image, but could also be
rendered and worked with independently of the pixels/voxels. E.g. in
a graphics rendering environment or such.
-------------------

Thanks for any thoughts on this.
Cheers,
- Damir

[jrswedlow]

Hi Damir-

Thanks for this. A very useful and insightful (re)start of this conversation.

As with all things in OME, we take these things in steps-- Units was a first, necessary prerequisite.

We need real world input and expertise on how to take this work forward. The info you've included above is a good start. Certainly the first thing to do is to establish the common elements that occur across many different systems and modalities-- these are candidates for addition to our data modeling and ultimately expression in software.

Will you be returning to the Paris meeting this year? Would be good to continue the conversation there, and start developing a draft spec we can target for community distribution and comment later in 2015.

Thanks again.

Cheers,

Jason

[dsudar]

Hi OME team,

I'm resurrecting this old topic mostly because it provides the "why" of my question.

I was reading up on your work on ROI folders, transforms, etc. on the Blog and it's all very exciting to think how to use that when it becomes available. One thing I couldn't deduce from the description of the plans was whether the new ROIs will have unit support and such so they can be associated with images of different scales/resolutions/etc. and properly scale themselves.

Cheers,
- Damir

[mtbc]

Dear Damir,

While I suppose that ROIs have some implicit units in being related to an image that may have physical pixel size set, I am afraid that OMERO 5.3 goes no further than the new 2016-06 OME-XML schema in not adding any new units to ROIs. The issue does remain very much on our radar though so thank you for mentioning this: I have noted your interest in Trello which will help us to prioritize future work.

Cheers,
Mark