Import images using the Gateway

[rguichard]

Dear Omero team,
A quick question, I am developing a in-house image importer for Omero in Python. I have a simple question, can I use the Gateway to import images without the CLI ?
Best regards
Roland Guichard

[jmoore]

Hi Roland,

I am developing a in-house image importer for Omero in Python. I have a simple question, can I use the Gateway to import images without the CLI ?

The BlitzGateway doesn't yet provide a Python-only import mechanism, though it is likely something we'll be interested in as well in the coming months.

In the meantime, you might take a look at the test_reimport.py file. The methods startUpload and uploadFileset reproduce what the Java importer does in pure Python.

The major drawback is not having access to Bio-Formats in order to determine the list of files that need uploading. If you can manage that internally, however, then there's no reason not to work directly from Python.

Let us know how it goes!
~Josh

[wmoore]

Josh,
In that example, createSynthetic() seems to first import a 16 x 16 multi-image file and duplicate it. But do startUpload() and uploadFileset() need to use a newImg created in this way, or can you create a newImage (with unknown dimensions) in a simpler way?

[jmoore]

Hi Roland,

Apologies. You can definitely create a new image this way, but the test_reimport.py class is trying to do too many things for this example. The method I was looking for had actually been refactored out into the `omero.testlib` package:

https://github.com/openmicroscopy/openmicroscopy/blob/develop/components/tools/OmeroPy/src/omero/testlib/__init__.py#L1340

Code: Select all

    def full_import(self, client):
        """
        Re-usable method for a basic import
        """
        mrepo = self.get_managed_repo(client)
        folder = self.create_test_dir()
        fileset = self.create_fileset(folder)
        settings = self.create_settings()

        proc = mrepo.importFileset(fileset, settings)
        try:
            return self.assert_import(client, proc, folder)
        finally:
            proc.close()


That should hopefully be a bit more straight-forward.

All the best,
~Josh

[wmoore]

Thanks Josh,

I created a stand-alone script to import, taking and tweaking code from the testlib.

https://gist.github.com/will-moore/f540 ... 8b38a077ce

This imports all the images in the target folder as a single 'Fileset' in OMERO.
A Fileset is what BioFormats recognises as a single import, where the files are part of the same collection.
E.g. A deltavision file + log file, or Leica fileset consisting of multiple tiffs + image.lei file.

If there are additional unrelated files that are not part of the first Fileset that BioFormats recognises, they will be uploaded by the code above and stored on the server, but ignored by the import (won't show up as separate images).

I don't know if there's a way to group a collection of files into separate filesets, but probably not since BioFormats functionality is really server-side in this case.
So you'd have to know somehow, which files to include in each fileset when you do the upload & import.

Hope that helps,

Will.

[wmoore]

Josh,

I guess we should consider making this code available in the BlitzGateway or similar place, instead of only in testlib, where it can't really be used for real import.

Will.

[jmoore]

Follow https://trello.com/c/3zH1PSo7/311-omeropy-importlibrary if interested.
~J.