Need CLI script to download all images with a given tagID

[jstitlow]

What is the best approach?
Thanks!
j

[jmoore]

Hi j,

can you tell us a little more about what you'd like the final result to look like? The two best options from the CLI will be bash or pure python. A rough break down in bash:

My available test data:

Code: Select all

[jamoore@ome-c6100-1 dist]$ bin/omero -q hql "select a.id, count(i.id) from ImageAnnotationLink l join l.child a join l.parent i where a.class = ome.model.annotations.TagAnnotation group by a.id"
# | Col1  | Col2
---+-------+------
0 | 12786 | 1
1 | 12901 | 3
2 | 20259 | 2
3 | 12907 | 1
4 | 20260 | 1
5 | 12908 | 1
6 | 20253 | 1
7 | 20257 | 3
8 | 20258 | 3
(9 rows)


Inspecting the images:

Code: Select all

[jamoore@ome-c6100-1 dist]$ bin/omero -q hql "select i.id, i.name, p.sizeX, p.sizeY from ImageAnnotationLink l join l.child a join l.parent i join i.pixels p where a.id = 12901"
# | Col1  | Col2                      | Col3  | Col4
---+-------+---------------------------+-------+-------
0 | 21905 | UMD001_ORO.svs [Series 3] | 1280  | 501
1 | 21904 | UMD001_ORO.svs [Series 2] | 416   | 482
2 | 21903 | UMD001_ORO.svs [Series 1] | 17317 | 11041
(3 rows)



Getting the images:

Code: Select all

[jamoore@ome-c6100-1 dist]$ for x in $(bin/omero -q hql "select i.id, i.name, p.sizeX, p.sizeY from ImageAnnotationLink l join l.child a join l.parent i join i.pixels p where a.id = 12901" --style=plain | head -n 2 | cut -f2 -d,); do bin/omero -q export Image:$x -f- >/dev/null ; done


This however downloads as OME-TIFF which will have some limitations. If you are looking to get the original files themselves, you'll need to use `bin/omero fs ls Fileset:$FILESET_ID`.

Cheers,
~Josh

[jstitlow]

Thanks Josh!

That works.

Using the script that exports the original file, how do you name the output file with the original filename?

[jmoore]

Thanks Josh!

That works.

Glad to hear it.

Using the script that exports the original file, how do you name the output file with the original filename?

Sorry, you lost me. Which script are you referring to?
~Josh

[jstitlow]

This script exports the images and gives them the name of the image ID:

Code: Select all

for x in $(/opt/OMERO.server-5.4.5-ice35-b83/bin/omero -q hql "select i.id, i.name, p.sizeX, p.sizeY from ImageAnnotationLink l join l.child a join l.parent i join i.pixels p where a.id = 2113196" --style=plain | head -n 2 | cut -f2 -d,); do /opt/OMERO.server-5.4.5-ice35-b83/bin/omero -q export Image:$x -f- >$x.ome.tiff ; done


How to export the images and give them the name of the image filename instead of the image ID?

Also the script above exports as ome.tiff and you suggested an alternative script to export the image in its original file format. I get an error when running that script, though probably I am implementing it wrong...?

Code: Select all

for x in $(/opt/OMERO.server-5.4.5-ice35-b83/bin/omero -q hql "select i.id, i.name, p.sizeX, p.sizeY from ImageAnnotationLink l join l.child a join l.parent i join i.pixels p where a.id = 2113196" --style=plain | head -n 2 | cut -f2 -d,); do /opt/OMERO.server-5.4.5-ice35-b83/bin/omero fs ls Fileset:$FILESET_ID>$x ; done
usage: /opt/OMERO.server fs ls [-h] fileset
/opt/OMERO.server fs ls: error: argument fileset: invalid proxy value: 'Fileset:'


[jmoore]

The BASH is obviously getting gnarly. Does the Python code from https://www.openmicroscopy.org/community/viewtopic.php?t=8521&p=19519#p19519 get you what you want then?

~Josh

[jstitlow]

Yes.

Do you have a simple Python script to import data into OMERO? Would be helpful for naive users like me to have import/export Python scripts where we can just plugin a path/datasetID and upload/download.

The export script you reference above is perfect, would be handy to have something similar for import.

We have a nice BASH script for batch importing all files in a given directory:

Code: Select all

for i in $1; do
        /opt/OMERO.server-5.4.5-ice35-b83/bin/omero import ${i} -T Dataset:+name:$2\
        --group 'davisgroup'
done


.. which can be called with command line arguments for 1=local directory where the data are located; and 2=OMERO dataset name.

So how to convert that to Python?
Thanks!
j

[jmoore]

Do you have a simple Python script to import data into OMERO? Would be helpful for naive users like me to have import/export Python scripts where we can just plugin a path/datasetID and upload/download.

We've and especially Will has been looking into supporting a native Python importer, but nothing at the moment is released. The most flexible option at the moment is to script the CLI:

Code: Select all

from omero.cli import cli_login
with cli_login() as cli:
    cli.invoke("import --help")


where in a loop or similar you could call each of the `bin/omero import` statements that you are currently calling from bash.

The export script you reference above is perfect, would be handy to have something similar for import.

We have a nice BASH script for batch importing all files in a given directory:

Code: Select all

for i in $1; do
        /opt/OMERO.server-5.4.5-ice35-b83/bin/omero import ${i} -T Dataset:+name:$2\
        --group 'davisgroup'
done


.. which can be called with command line arguments for 1=local directory where the data are located; and 2=OMERO dataset name.

Did you see any of the presentations on the --bulk option to bin/omero import during #ome2018? e.g. http://downloads.openmicroscopy.org/presentations/2018/Users-Meeting/Workshops/Metadata-Import/import.pdf

It's a somewhat different, more declarative supported alternative. There, you'd have input YML & CSV files with the sets of parameters you want in your loop, and is what's in use for the IDR (e.g. idr0042)

~Josh

[jstitlow]

Thanks Josh,

I like the omero.cli approach.

Re: --bulk option for bin/omero import, will have a look at this for our screen datasets.

Currently trying to write basic batch input.py/output.py scripts for everyday use in our lab and for sharing data with collaborators.

Appreciate the help,
j