File format of the images stored in OMERO.data/Pixels

[bpavie]

Hi ,

I successfully installed the last version of OMERO server and could connect to it both using the web interface and the java clients (on ubuntu Ubuntu 13.04)
I have just a quick question: which format is used to stored the uploaded images? If I understand well, images are uploaded to the server and stored in OMERO.data/Pixels, but using which format ?

I could see that thumbnails are in JPEG format:
~/apps/OMERO/OMERO.data/Thumbnails$ file -i 855
855: image/jpeg; charset=binary
But in Pixels, it looks to be some binary format
~/apps/OMERO/OMERO.data/Pixels$ file -i 855
855: application/octet-stream; charset=binary

How I could open them and retrieve the image?
I tried to use loci to open them (using fiji) without success.

Thanks,
Benjamin

[ajpatterson]

Hello Benjamin,

The files stored in this folder are the raw pixel values in a format for efficient access by the OMERO server. They are designed only to be read by the server.

If you explain what your are trying to achieve we might be able to give you some pointers.

Ta,

Andrew

[bpavie]

Hi Andrew,

I was just wondering what was the format and if you had specification of it somewhere, and a way to access it. The idea was to avoid duplicating data and so not use the archive option. We currently have several terabytes (around 10) so if we could avoid to duplicate it, that will save us some storage...

I also have 2 unrelated questions:

My dataset are actually all plates and have been exported like this:

A1/image1_channel_1.tif
A1/image1_channel_2.tif
A1/image2_channel_1.tif
A1/image2_channel_2.tif
A2/image1_channel_1.tif
A2/image1_channel_2.tif
etc...

Once I imported those images to the database, is there an simple way to group the channel for each channel or should I do that before to upload the images? I tried to use the Combine_Images.py but it looks it is generating only one image. Is there an other easy way to do it or should I write my own script (based onCombine_Images.py ).
Also I would like to organize my dataset by plate. I saw the script Dataset_To_Plate.py and that generate some screen grouping . However, this script seems to handle 1 image/plate and I have multiple... Is there another way to indicate which images go to which well or should I write a script/code to do it (based on Dataset_To_Plate )?

Thanks
Benjamin

[wmoore]

Hi,

I was just wondering what was the format and if you had specification of it somewhere, and a way to access it. The idea was to avoid duplicating data and so not use the archive option. We currently have several terabytes (around 10) so if we could avoid to duplicate it, that will save us some storage...

The format of the Pixels file is really pure pixel data (no header etc) since all the metadata, dimensions etc is stored in the database pixels table.
However, your question is quite timely in that we are getting close to the first release candidate of what we've been calling OMERO.FS 'file system'. In OMERO.FS, imported images are simply stored in their original format (and name) and these original files are used to access Pixel data. Therefore, no Pixel objects are created and there is no data duplication. See http://trac.openmicroscopy.org.uk/ome/milestone/OMERO-5

I'm afraid there's no "simple" way to get "Combine_Images.py" to generate multiple images. If you are acquiring your data using a common format (E.g. commercial system) then we should be able to extend Bio-Formats to recognise this and import the data 'correctly' into plates in OMERO. However, if you have you own in-house acquisition and format, then you could either combine the channels before import using ImageJ / Bio-Formats etc on the command line or adapt the Combine_Images script to do what you want.

The Dataset_To_Plate script should certainly handle multiple images per plate. Did you see the notes and demo at http://www.openmicroscopy.org/site/supp ... ripts.html ?

E.g. if you have these images (in the following order sorted by name)

A1.tif
A2.tif
A3.tif
B1.tif
B2.tif
B3.tif

and you want columns A & B and rows 1,2,3 then you'd simply choose that you want the 'First Axis' to be row and the 'First Axis Count':3. Then choose to name columns by 'letter' and rows by 'number'.

[bpavie]

Thanks for your answer, i'm looking forward to see this OMERO.FS!!
Just to be sure, is the The Dataset_To_Plate script don't assume that you have continuous wells notation so if I have let say images in wells A1 A3 A5 B2 B4 only, then the script won't work... Am I right?
Anyway, based on this script, I should be able to write a swing interface to let the user pick which well contain the images...

Thanks again for your answers,

Benjamin

[wmoore]

You are correct that Dataset_To_Plate assumes every Well has an Image.

If you know that your images are always going to be named with their Well position 'A5' etc, then you probably don't want to make the user pick a Well for every Image. Probably just want to update the Dataset_To_Plate script to use image names. But if you need a more manual approach then an interactive GUI is the only way.

Let us know if you need any more pointers...

Will.

[cxallan]

Hi Benjamin,

I guess one final thing to mention about the files in the binary repository, Andrew has already touched on it, is that the format of them is both undocumented and non-public. When we say both of these things it is absolutely not our intention to be secretive. We take this position so that we can at any time, change the format, layout or location without adversely affecting our user base. You really should treat the files in the Pixels directory of the binary repository as opaque just like you would the contents of your PostgreSQL data directory.

-Chris