High Content Screening in omero

[heg]

I have two questions about High Content Screening data in omero:

1.-
I usually store one file with all the acquired data from one position. This contains also the metadata of the position (such as what was in that position). However, in most examples I have seen the full content of the plate/plates is stored in each file. Is this an omero requirement or a suggestion? Can it be avoided?

2.-
I currently have a Matlab program to visualize HCS data. It plots (in different ways) data precalculated from the images and stored in a mysql database. Data and presentation layers are separated, so it should be straightforward to migrate to omero. However, I am not sure how to store the precalculated values in omero. Should I keep them in my own separate table (outside omero) or is there any suggested place for storing these (such as in metadata or features)?

Thanks,

H

[jmoore]

1.-
I usually store one file with all the acquired data from one position. This contains also the metadata of the position (such as what was in that position). However, in most examples I have seen the full content of the plate/plates is stored in each file. Is this an omero requirement or a suggestion? Can it be avoided?

It is not a requirement. Assuming you are using multiple OME-TIFF (cF. this thread) then as long as the OME metadata in the headers of the per-position files is consistent, you should be able to import them. There currently isn't an example of this, though one would obviously be immensely valuable.

2.-
I currently have a Matlab program to visualize HCS data. It plots (in different ways) data precalculated from the images and stored in a mysql database. Data and presentation layers are separated, so it should be straightforward to migrate to omero. However, I am not sure how to store the precalculated values in omero. Should I keep them in my own separate table (outside omero) or is there any suggested place for storing these (such as in metadata or features)?

I've just committed initial support for a new service (omero.grid.TablesPrx) which should provide just this. At the moment, we are testing using http://pytables.org as the storage mechanism. This may change, but the API will hopefully remain the same.

This is really very, very new, but if you are interested in helping us test, keep an eye on: OmeroTables on the developer wiki, or drop me a line.

[heg]

Going back to point 1, saving all the information from the plate is not a requirement from the format but seems to be enforced by the API. Take a look at the ome.xml that I generated for the second well (index
number 2). As you can see, in addition to the info about Well:1 there is also a part for Well:0.

I have removed all the usual <xml and <OME xmlns to avoid cluttering.

Code: Select all

<Image ID="Image:0">
<Pixels BigEndian="true" DimensionOrder="XYZCT" ID="Pixels:0" PixelType="uint16" SizeC="1" SizeT="1" SizeX="2" SizeY="2" SizeZ="1">
<Bin:BinData Length="12">QzCuHo+g23M=</Bin:BinData></Pixels>
<LogicalChannel ID="LogicalChannel:0" SamplesPerPixel="1"></LogicalChannel></Image>
<SPW:Plate ID="Plate:0" Name="Plate Name">
<SPW:Well ID="Well:0"></SPW:Well>
<SPW:Well Column="3" ID="Well:1" Row="2">
<SPW:ReagentRef ID="Reagent Name"></SPW:ReagentRef>
<SPW:WellSample ID="WellSample:0" Index="0"></SPW:WellSample></SPW:Well></SPW:Plate></OME>

Maybe I have misunderstood the what is the index. I thought is the physical/logical (of my experiment) but it seems to be the logical index withing ONE file. The same happens with objectives/filters/plates ... basically everything.

The question is: if I want to upload data coming from a screening, one file per well. The only way to avoid putting all the information about all plates in each file is that, the plate will be allways Plate:0 and the well will be always Well:0. Will this be problematic for the analysis/display interface for screening that you are planning?

Regards,

Hernan

[crueden]

Are you looking for a way to reference Wells (or other metadata fields) consistently across multiple OME-TIFF files? If so, try using LSIDs instead of internal IDs for your ID strings: http://www.ome-xml.org/wiki/LSID

An LSID is a globally unique identifier, and should preserve linkages across multiple data sources such as OME-TIFF constituent files.

[heg]

Thanks for the suggestion. I have tried using LSID for the objectives with unexpected results. I created an ome file with the following metadata (I have stripped all the usual headers):

Code: Select all

<Image ID="Image:0">
<Pixels BigEndian="true" DimensionOrder="XYZCT" ID="Pixels:0" PixelType="uint16" SizeC="1" SizeT="1" SizeX="2" SizeY="2" SizeZ="1">
<Bin:BinData Length="12">xyhc2cxMqdc=</Bin:BinData></Pixels>
<LogicalChannel ID="LogicalChannel:0" SamplesPerPixel="1"></LogicalChannel>
<ObjectiveRef ID="urn:lsid:omero.mpi-dortmund.mpg.de:Objective:TestObj20"></ObjectiveRef></Image>
<Instrument ID="Instrument:0">
<Objective ID="urn:lsid:omero.mpi-dortmund.mpg.de:Objective:TestObj20" Manufacturer="Olympus" Model="UPLANSAPO">
<NominalMagnification>20</NominalMagnification></Objective></Instrument>

The picture got uploaded properly and the Objective Metadata is visible in OMERO.insight. Then, I wanted to upload another image taken with the same objective. I thought it was not necessary to repeat again the same metadata: the info is already stored and the LSID is unique for that objective. Therefore, I created the following OME

Code: Select all

<Image ID="Image:0">
<Pixels BigEndian="true" DimensionOrder="XYZCT" ID="Pixels:0" PixelType="uint16" SizeC="1" SizeT="1" SizeX="2" SizeY="2" SizeZ="1">
<Bin:BinData Length="12">fU2MBaxRf04=</Bin:BinData></Pixels>
<LogicalChannel ID="LogicalChannel:0" SamplesPerPixel="1"></LogicalChannel>
<ObjectiveRef ID="urn:lsid:omero.mpi-dortmund.mpg.de:Objective:TestObj20"></ObjectiveRef></Image>

But I got the following error in OMERO.importer

java.lang.RuntimeException: omero.ApiUsageException
serverStackTrace = "ome.conditions.ApiUsageException: Missing reference handler for urn:lsid:omero.mpi-dortmund.mpg.de:Objective:TestObj20(null) --> omero.model.ObjectiveSettings:0(ome.model.acquisition.ObjectiveSettings:Hash_20019631) reference.
at ome.formats.OMEROMetadataStore.updateReferences(OMEROMetadataStore.java:333)

Why this error? Is this a misunderstanding from my side about how LSID should work or a bug in the software?

Hernan

[cxallan]

For performance and scalability reasons OMERO does not check the LSID of every object. Also, because LSID resolution is such a hairy issue that is implementation dependent we suggest that you include the entire metadata structure and not rely on LSIDs exclusively.

We're currently examining LSID resolution in more detail but this is how it currently works.

[heg]

I think that having the LSID resolution will be very useful if it is complemented by a user interface to edit this data. This will remove a lot of overhead from the microscope workflow and from the data files.

Thanks for the reply,

Hernan