Batch export with Python

[jstitlow]

Is there a way to call the CLI export command from Python?

Or more to the point, after retrieving all of the images from a dataset with the code below, how do I save them in their original format (or OME.tiff)?

Many thanks!
j

Code: Select all

import omero
from omero.gateway import BlitzGateway
import os
import getpass

USER = os.environ['USER']
PASS = getpass.getpass("Enter Password:")
HOST = os.environ['HOST']

conn = BlitzGateway(USER, PASS, host=HOST, port=4064)
conn.connect()
conn.getSession().setTimeToIdle(rlong(60*60*1000))

DATASET = 16822

dataset = conn.getObject('Dataset', DATASET)

# make a dict of name: image
images = {}
for image in dataset.listChildren():
        filename = image.getName()
        print filename


[wmoore]

Hi Josh

This should do what you want.
NB: it assumes that there are not duplicate file names (will get overwritten)

Code: Select all

import omero
from omero.gateway import BlitzGateway

conn = BlitzGateway("username", "password", host="localhost", port=4064)
conn.connect()
conn.SERVICE_OPTS.setOmeroGroup(-1)

dataset_id = 11509

for image in conn.getObject("Dataset", dataset_id).listChildren():

  for orig_file in image.getImportedImageFiles():

    file_name = orig_file.getName()
    print "Downloading...", file_name

    with open(file_name, "wb") as f:
      for chunk in orig_file.getFileInChunks(buf=2621440):
        f.write(chunk)

conn.close()


Hope that works for you,

Will.

[wmoore]

If you want to be able to run this from the command line, passing in username, password, dataset_id, you can follow the pattern at e.g. https://github.com/ome/training-scripts ... te_ROIs.py

[wmoore]

If you want to download images linked to Tags, via command-line, save this code as "download_original_files_by_tag.py"

Code: Select all

import argparse
from omero.gateway import BlitzGateway

def run(username, password, tag_id, host, port):
    conn = BlitzGateway(username, password, host=host, port=port)
    try:
        conn.connect()
        for link in conn.getAnnotationLinks("Image", ann_ids=[tag_id]):
            image = link.getParent()
            print 'Image', image.getName()
            for orig_file in image.getImportedImageFiles():
                file_name = orig_file.getName()
                print " Downloading...", file_name

                with open(file_name, "wb") as f:
                  for chunk in orig_file.getFileInChunks(buf=2621440):
                    f.write(chunk)
           
    except Exception as exc:
            print "Error while deleting annotations: %s" % str(exc)
    finally:
        conn.close()

def main(args):
    parser = argparse.ArgumentParser()
    parser.add_argument('username')
    parser.add_argument('password')
    parser.add_argument('tag_id')
    parser.add_argument('--server', help="OMERO server hostname")
    parser.add_argument('--port', default=4064, help="OMERO server port")
    args = parser.parse_args(args)
    run(args.username, args.password, args.tag_id, args.server, args.port)

if __name__ == '__main__':
    import sys
    main(sys.argv[1:])


Then on the command-line you can do:

Code: Select all

$ python download_original_files_by_tag.py username password 44409 --server server.address


[jstitlow]

That's perfect, thank you Will!

[jstitlow]

Hi Will,

The export code gives an error (below) when exporting images that were created from a fileset, e.g., .mvd2.

Is there a way around this?

Code: Select all

Traceback (most recent call last):
  File "/usr/people/bioc1301/src/OMERO_scripts/OMERO_export_dataset_CLI.py", line 20, in <module>
    for image in dataset.listChildren():
AttributeError: 'NoneType' object has no attribute 'listChildren'


[jstitlow]

Removing .list.children(), i.e,

Code: Select all

#for image in dataset.listChildren():
for image in dataset:


shows that the dataset is not iterable:

Code: Select all

Traceback (most recent call last):
  File "/usr/people/bioc1301/src/OMERO_scripts/OMERO_export_dataset_CLI.py", line 21, in <module>
    for image in dataset:
TypeError: 'NoneType' object is not iterable


[jmoore]

Code: Select all

AttributeError: 'NoneType' object has no attribute 'listChildren'


Here NoneType means that your original call, "dataset = conn.getObject('Dataset', DATASET)", didn't return anything. We need to back up to that point for now. Can you double check the dataset ID? Does the user your logging in as have permission to view it?

~Josh.

[jstitlow]

Permissions might be the problem as I am now experiencing other password issues, let me get back to you on this...

[jmoore]

You might need to check the server log for a resource exhaustion of some form.
~Josh.