Grouping of multiple files; creation of data slices?

[gyellen]

I have been exporting a new format to OME-TIFF, and have some questions about using the OME data model (eventually with OMERO, though we do not yet have an installation).

Suppose I write a set of OME-TIFF files: each file contains multiple image sets from one short period of time. The full set of files constitute an experiment (not in the OMERO sense), but there is no direct metafile data linking the files - they are independent of each other.

1) In loading these to OMERO, can they be consolidated as a single 'dataset'? Does that allow processed files (ROI files, image summaries) created by scripts to be stored and linked with them?

2) Can a new metadata file, linked to the original images, create a 'slice' (possibly not the correct OMERO term) that consists of (for instance) a single image from each file, constructing a new T dimension? Or would it be necessary to duplicate the original images?

Thanks - gary

[ajpatterson]

Hello Gary,

If you are importing the files using the command line importer via a script then you can collect your "experiment" together into a dataset.

For command line import options see:
http://www.openmicroscopy.org/site/supp ... mport.html

I think that having more that one file using the same pixel data is likely to cause problem and confuse the import process. The data will end up duplicated in the OMERO server, but that is true of any file that is imported twice.

A collection of ome-tiff files with two different ome-xml companion files would not validate correctly. Each ome-tiff file in this arrangement needs to contain the unique identifier of it's metadata companion. A single file could not point to two companions.

It could make more sense to write a script that would run inside the OMERO server to create the slice image from the full image. This would possibly also result in some data duplication.

Hope this helps.

Andrew

[gyellen]

Thanks, Andrew.

I'm intrigued by the statement in the schema that "Images can belong to more than one Dataset, and a Dataset may contain more than one Image. Images contain one or more DatasetRef elements to specify what datasets they belong to."

This seems like exactly the functionality that I'm after. I would like to specify when saving the single acquisition OME-TIFF that its data (or, more specifically, its Images) belong to a specific Dataset (by name? by UUID?). If I understand this correctly, the metadata do not have to include specifications for all other members of the dataset.

[BTW, in general I have a very hard time reading through the schema or any of the online docs and figuring out how to use the Bio-Formats/LOCI Java calls to create a particular notation - it usually involves trial and error. Is there a good place to look for example code?]

Furthermore, the description makes it sound like I could have all three image series ("Image"s) be part of one Dataset (embracing all the data for one of my "experiment"s), but then have another Dataset that embraces all of the "Image:0"s, another for all of the "Image:1"s, etc. Is this correct?

I have now done this provisionally in an OME-TIFF file by using

Code: Select all

   %% create the Datasets
    metadata.setDatasetID('Dataset:All',0);
    metadata.setDatasetName('BigExperimentToday',0);
    metadata.setDatasetID('Dataset:dfi',1);
    metadata.setDatasetName('BigExperimentToday_DFI',1);
    metadata.setDatasetID('Dataset:xyt',2);
    metadata.setDatasetName('BigExperimentToday_XYT',2);
    metadata.setDatasetID('Dataset:dfh',3);
    metadata.setDatasetName('BigExperimentToday_DFH',3);


and then associating individual image series with two datasets, the overall one and the specific one. For instance, for the second series ("Image:1"):

Code: Select all

            metadata.setDatasetImageRef(['Image:' sSeries],0,series);
            metadata.setDatasetImageRef(['Image:' sSeries],2,series);


These metadata are written without errors and imported in a sensible-looking way into Fiji.

When imported into OMERO, will this result in grouping of images from multiple files into the various Datasets, so that 'BigExperimentToday' includes all of the images from multiple files, and 'BigExperimentToday_XYT' includes all of the Image:1 series?

As a secondary issue, will the Datasets be linked to the Experiment and Instrument metadata in each component image, or do I need to somehow set ExperimentRef's and InstrumentRef's for each Image?

Thanks,
Gary

[This same idea may also apply to Dan's query in "Ome-metadata distributed across multiple tiff-files"]

[jmoore]

Hi Gary,

This seems like exactly the functionality that I'm after. I would like to specify when saving the single acquisition OME-TIFF that its data (or, more specifically, its Images) belong to a specific Dataset (by name? by UUID?). If I understand this correctly, the metadata do not have to include specifications for all other members of the dataset.

You're right that a DatasetImageRef would link an Image into a Dataset in the model without having the full Dataset present. On import, however, we've been conservative about allow Datasets in for exactly the same reason, and so as of OMERO 5.0, the datasets will be stripped on import.

Furthermore, the description makes it sound like I could have all three image series ("Image"s) be part of one Dataset (embracing all the data for one of my "experiment"s), but then have another Dataset that embraces all of the "Image:0"s, another for all of the "Image:1"s, etc. Is this correct?

Currently, the relationship between Datasets and Images is many-to-many, so you can certainly do just this.

When imported into OMERO, will this result in grouping of images from multiple files into the various Datasets, so that 'BigExperimentToday' includes all of the images from multiple files, and 'BigExperimentToday_XYT' includes all of the Image:1 series?

As above, currently no, it won't.

As a secondary issue, will the Datasets be linked to the Experiment and Instrument metadata in each component image, or do I need to somehow set ExperimentRef's and InstrumentRef's for each Image?

An Images containment in a Dataset won't have any effect on links to Experimenters or Instruments for the individual Images. You'll definitely need to set the Refs.

Cheers,
~Josh