Page 1 of 2

Batch export with Python

PostPosted: Thu Jun 14, 2018 10:43 am
by jstitlow
Is there a way to call the CLI export command from Python?

Or more to the point, after retrieving all of the images from a dataset with the code below, how do I save them in their original format (or OME.tiff)?

Many thanks!
j

Code: Select all
import omero
from omero.gateway import BlitzGateway
import os
import getpass

USER = os.environ['USER']
PASS = getpass.getpass("Enter Password:")
HOST = os.environ['HOST']

conn = BlitzGateway(USER, PASS, host=HOST, port=4064)
conn.connect()
conn.getSession().setTimeToIdle(rlong(60*60*1000))

DATASET = 16822

dataset = conn.getObject('Dataset', DATASET)

# make a dict of name: image
images = {}
for image in dataset.listChildren():
        filename = image.getName()
        print filename

Re: Batch export with Python

PostPosted: Fri Jun 15, 2018 3:44 pm
by wmoore
Hi Josh

This should do what you want.
NB: it assumes that there are not duplicate file names (will get overwritten)

Code: Select all
import omero
from omero.gateway import BlitzGateway

conn = BlitzGateway("username", "password", host="localhost", port=4064)
conn.connect()
conn.SERVICE_OPTS.setOmeroGroup(-1)

dataset_id = 11509

for image in conn.getObject("Dataset", dataset_id).listChildren():

  for orig_file in image.getImportedImageFiles():

    file_name = orig_file.getName()
    print "Downloading...", file_name

    with open(file_name, "wb") as f:
      for chunk in orig_file.getFileInChunks(buf=2621440):
        f.write(chunk)

conn.close()


Hope that works for you,

Will.

Re: Batch export with Python

PostPosted: Fri Jun 15, 2018 3:46 pm
by wmoore
If you want to be able to run this from the command line, passing in username, password, dataset_id, you can follow the pattern at e.g. https://github.com/ome/training-scripts ... te_ROIs.py

Re: Batch export with Python

PostPosted: Fri Jun 15, 2018 10:02 pm
by wmoore
If you want to download images linked to Tags, via command-line, save this code as "download_original_files_by_tag.py"

Code: Select all
import argparse
from omero.gateway import BlitzGateway

def run(username, password, tag_id, host, port):
    conn = BlitzGateway(username, password, host=host, port=port)
    try:
        conn.connect()
        for link in conn.getAnnotationLinks("Image", ann_ids=[tag_id]):
            image = link.getParent()
            print 'Image', image.getName()
            for orig_file in image.getImportedImageFiles():
                file_name = orig_file.getName()
                print " Downloading...", file_name

                with open(file_name, "wb") as f:
                  for chunk in orig_file.getFileInChunks(buf=2621440):
                    f.write(chunk)
           
    except Exception as exc:
            print "Error while deleting annotations: %s" % str(exc)
    finally:
        conn.close()

def main(args):
    parser = argparse.ArgumentParser()
    parser.add_argument('username')
    parser.add_argument('password')
    parser.add_argument('tag_id')
    parser.add_argument('--server', help="OMERO server hostname")
    parser.add_argument('--port', default=4064, help="OMERO server port")
    args = parser.parse_args(args)
    run(args.username, args.password, args.tag_id, args.server, args.port)

if __name__ == '__main__':
    import sys
    main(sys.argv[1:])


Then on the command-line you can do:
Code: Select all
$ python download_original_files_by_tag.py username password 44409 --server server.address

Re: Batch export with Python

PostPosted: Mon Jun 18, 2018 9:25 am
by jstitlow
That's perfect, thank you Will!

Re: Batch export with Python

PostPosted: Tue Jun 19, 2018 7:36 pm
by jstitlow
Hi Will,

The export code gives an error (below) when exporting images that were created from a fileset, e.g., .mvd2.

Is there a way around this?

Code: Select all
Traceback (most recent call last):
  File "/usr/people/bioc1301/src/OMERO_scripts/OMERO_export_dataset_CLI.py", line 20, in <module>
    for image in dataset.listChildren():
AttributeError: 'NoneType' object has no attribute 'listChildren'

Re: Batch export with Python

PostPosted: Tue Jun 19, 2018 8:24 pm
by jstitlow
Removing .list.children(), i.e,

Code: Select all
#for image in dataset.listChildren():
for image in dataset:


shows that the dataset is not iterable:

Code: Select all
Traceback (most recent call last):
  File "/usr/people/bioc1301/src/OMERO_scripts/OMERO_export_dataset_CLI.py", line 21, in <module>
    for image in dataset:
TypeError: 'NoneType' object is not iterable

Re: Batch export with Python

PostPosted: Tue Jun 19, 2018 8:46 pm
by jmoore
Code: Select all
AttributeError: 'NoneType' object has no attribute 'listChildren'


Here NoneType means that your original call, "dataset = conn.getObject('Dataset', DATASET)", didn't return anything. We need to back up to that point for now. Can you double check the dataset ID? Does the user your logging in as have permission to view it?

~Josh.

Re: Batch export with Python

PostPosted: Tue Jun 19, 2018 9:33 pm
by jstitlow
Permissions might be the problem as I am now experiencing other password issues, let me get back to you on this...

Re: Batch export with Python

PostPosted: Tue Jun 19, 2018 9:43 pm
by jmoore
You might need to check the server log for a resource exhaustion of some form.
~Josh.