Upload/Download OME-Tiff

[sgaric]

Hi all,

I am sorry if I am reposting the question but search engine wouldn't let me search for any keyword that would help me.

Currently I am writing some Java code that I want to use to download OME-tiff images from the OMERO server using omero client Java APIs. I looked through some examples and I am just not sure what is the way (or the best way to try to do this). This is my example scenario. I've uploaded a number of images into OMERO server and put them all in one Project/Dataset. I did not do any modification to them in OMERO so they are as they were when I uploaded them. Now what I want to do is connect to the server, get the dataset using its OMERO ID and then iterate through the images contained in that dataset and save each of those images on my local file system. What I've done so far is I've created my session object

Code: Select all

this.conn = new omero.client(this.hostname, this.port);
this.session = conn.createSession(username, password);


Then I used the example code provided in schoola documentation to retrieve a dataset using its ID and the get images contained within:

Code: Select all

IContainerPrx proxy = this.session.getContainerService();
ParametersI param = new ParametersI();
param.leaves();
List<omero.model.IObject> results = proxy.loadContainerHierarchy(Dataset.class.getName(), Arrays.asList(datasetid), param);
if (results.size() == 0) return null;
DatasetData dataset = new DatasetData((Dataset) results.get(0));
Set<ImageData> images = dataset.getImages();
Iterator<ImageData> j = images.iterator();


And then for the last bit I just iterated over those images trying to download them. I used two different approaches:

Code: Select all

ImageData image;
while (j.hasNext()) {
        image = j.next();
        File newfile = new File(tempdir.getAbsolutePath() + File.separator + image.getName());
        this.conn.download(image.getId(), newfile);
} 


This works and it does download "things" but this doesn't always give me the actual tiff file. It sometimes just gives me a metadata block, some script or some XML file.

Other approach I used was with the RawFileStorePrx object:

Code: Select all

ImageData image;
while (j.hasNext()) {
        image = j.next();
        RawFileStorePrx filestore = this.session.createRawFileStore();
        filestore.setFileId(image.getId());

        byte[] imagebytes = new byte[(int) filestore.size()];
        FileOutputStream fos = new FileOutputStream(tempdir.getAbsolutePath() + File.separator + image.getName());
        BufferedOutputStream dest = new BufferedOutputStream(fos, BUFFER);
        imagebytes = filestore.read(0, (int) filestore.size());
        dest.write(imagebytes.length);
}


But as it was the case with other example this just doesn't give me the actual content of the image and I just get odd metadata/script/XML file instead.

Once I have this working I will be attempting to use my code to upload ome-tiff files to OMERO server so if anyone can think of some pointers there it would be great. All I need is sample code or link to some part of OMERO documentation that would help me out with this and I will be set.

Any help with pointing me in the right direction would be appreciated,
Regards,
Slavisa

[sgaric]

Forgot to say, the images were uploaded using OMEROinsight client,

Regards,
Slavisa

[cxallan]

You need to retrieve the Image IDs and export each Image individually to its own OME-TIFF file. The RawFileStore is only for files that have been archived to OMERO.

I've whipped up a full isolated example which should do what you want here:

https://gist.github.com/1261989

[sgaric]

You need to retrieve the Image IDs and export each Image individually to its own OME-TIFF file. The RawFileStore is only for files that have been archived to OMERO.

I've whipped up a full isolated example which should do what you want here:

https://gist.github.com/1261989

Thank you so much for this Chris. This does exactly what I needed to do. There are couple of follow up questions. How would I go about importing a file into the OMERO server?

To give you a better picture of what I am working on here I will just quickly explain it. We work with computational workflows where there are number of possible sources that will provide us with OME-Tiff images. There could be direct link to a microscope or we will just load the images from local files system. What we want to do is automatically store these into OMERO and at the same time appropriately assign these images to the correct datasets.

The second part of our work is actually doing some processing on those images and as is that part works well but we just wanted to add the option to get the images directly from OMERO, which is where ExporterPrx example above comes in.

Thanks again for the helps so far and in advance for any advice you may have with the questions in this post,
Regards,
Slavisa

[cxallan]

How would I go about importing a file into the OMERO server?

There are a few ways to do that. The obvious one via the OMERO.insight, via the command line with bin/omero import, or customized in your own Java code by using ImportLibrary and its dependencies.

To give you a better picture of what I am working on here I will just quickly explain it. We work with computational workflows where there are number of possible sources that will provide us with OME-Tiff images. There could be direct link to a microscope or we will just load the images from local files system. What we want to do is automatically store these into OMERO and at the same time appropriately assign these images to the correct datasets.

Makes complete sense.

The second part of our work is actually doing some processing on those images and as is that part works well but we just wanted to add the option to get the images directly from OMERO, which is where ExporterPrx example above comes in.

[sgaric]

Hello again,

I've made some progress with creating a Java client code that is using ImportLibrary to import OME-tiff files into OMERO server. I borrowed a bit of code from CommandLineImporter.java, basically creating a custom ImportConfig object and then using that to import a group of images into OMERO server. The problem I have with this is the actual naming convention of imported images. The default behaviour is to use the full path of the image being imported as the name of the image in OMERO.

Digging through the actual code in CLI and GUI import codes I've seen that there are ImportConfig options that you should be able to use to accomplish this (see below) but for some reason this is not yielding desired results for me (this is inside a function called SetupImageImportConfig():

Code: Select all

this.importconfig = new ImportConfig();
...      
this.importconfig.setCustomImageNaming(true);
this.importconfig.setNumOfDirectories(0);
...


I would like to have this work so I can use a single importCandidates call on an array of images rather than modifying config for every image I am importing and then calling importCandidates on each image. What I've done so far I've created two functions, one that will update the "this.importconfig" for every image before importing that image; and second that imports all images using the same "this.importconfig". Code is below:
Example 1: - Update config and importCandidate per image

Code: Select all

public void DatasetImageImportSingle (long datasetid, LinkedList<File> images) throws Exception {
   this.SetupImageImportConfig(datasetid);
   this.importconfig.loadAll();
      
      
   String[] filenames = new String[1];
   for (File image : images) {
      filenames[0] = image.getAbsolutePath();
      this.importconfig.imageName.set(image.getName());
         
      OMEROWrapper reader = new OMEROWrapper(this.importconfig);
      OMEROErrorHandler handler = new OMEROErrorHandler(this.importconfig);
      reader.setMetadataOptions(new DefaultMetadataOptions(MetadataLevel.ALL));
      ImportCandidates candidates = new ImportCandidates(reader, filenames, handler);
         
      OMEROMetadataStoreClient store = this.importconfig.createStore();
      ImportLibrary library = new ImportLibrary(store, reader);
      library.importCandidates(this.importconfig, candidates);
   }
}


This is the one I would love to use but as I said my desired result doesn't happen even when I set what I think are the right options in ImportConfig object.
Example 2: - Import group of images with one importCandidates call

Code: Select all

public void DatasetImageImportGroup (long datasetid, LinkedList<File> images) throws Exception {
   this.SetupImageImportConfig(datasetid);
   this.importconfig.loadAll();
      
   String[] filenames = new String[images.size()];
   int index=0;
   for (File image : images) {
      filenames[index++] = image.getAbsolutePath();
   }
   OMEROWrapper reader = new OMEROWrapper(this.importconfig);
   OMEROErrorHandler handler = new OMEROErrorHandler(this.importconfig);
   
   reader.setMetadataOptions(new DefaultMetadataOptions(MetadataLevel.ALL));
   ImportCandidates candidates = new ImportCandidates(reader, filenames, handler);
      
   OMEROMetadataStoreClient store = this.importconfig.createStore();
   ImportLibrary library = new ImportLibrary(store, reader);
   library.importCandidates(this.importconfig, candidates);
}


If anyone has any ideas how I can get it so that import names the imported images the way I would like it,I'd like to hear it,
Thanks in advance,
Slavisa

[cxallan]

ImportCandidates is definitely not going to do what you want. What you really want to do is prepare a list of ImportContainers and then use the ImportLibrary.importImage() method in a tight loop (which is pretty much what ImportCandidates does behind the scenes anyway).

[JakeNewman]

This thread has been very useful to me, but I'm still a little stuck.

After much effort, I've written JAVA/Matlab code to download an OME-TIFF, Extract the XML, Add the XML to a new image and I'm now at the point where I want to upload the new OME-TIFF.

Sgaric, did you successfully get your code to upload to your OMERO server in the end?

Code: Select all

public void DatasetImageImportSingle (long datasetid, LinkedList<File> images) throws Exception {
   this.SetupImageImportConfig(datasetid);
   this.importconfig.loadAll();
     
     
   String[] filenames = new String[1];
   for (File image : images) {
      filenames[0] = image.getAbsolutePath();
      this.importconfig.imageName.set(image.getName());
         
      OMEROWrapper reader = new OMEROWrapper(this.importconfig);
      OMEROErrorHandler handler = new OMEROErrorHandler(this.importconfig);
      reader.setMetadataOptions(new DefaultMetadataOptions(MetadataLevel.ALL));
      ImportCandidates candidates = new ImportCandidates(reader, filenames, handler);
         
      OMEROMetadataStoreClient store = this.importconfig.createStore();
      ImportLibrary library = new ImportLibrary(store, reader);
      library.importCandidates(this.importconfig, candidates);
   }
}

Any help would be greatly appreciated, thanks.

Jake

[JakeNewman]

Continued: I've found some code on my computer which claims to be able to upload OME-TIFFs to the omero server. I'm dubious, but (once again) I'm failing at the first hurdle. The code makes an OMEROMetadataStoreClient object, which apparently is in ome.formats. Which .jar file contains that library? I can't find it anywhere.

Thanks

[cxallan]

The ome.formats.OMEROMetadataStoreClient class is included as part of omero_client.jar.