DropBox questions

[helmerj]

Hi!

In my attempt to automate the import of large numbers of metaXpress tif images I have some more question in regard to the import itself.

requirements:

- Import images for different users using an "import" account
- import to either existing or non existing projects/datasets
- large number of files (100k to 500k, 2MB each)

• is it possible or at least planned for importer-cli to allow an Omero user to import images for other users? E.g. have an admin "import" account being able to import for other regular users without knowing the users password but assigning the images to said user. Right now specifying a dataset id belonging to a different user throws a security violation error.
• is is possible to have images in the DropBox being assigned to either existing projects/datasets or even better to create projects/dataset that do not exist before the import? Right now images do show up under Images in Insight but project and dataset definitions in the OME header block are ignored
• What are the amount of images the DropBox mechanism can handle? On the website it says: "normal acquisition rates". well that is bit unspecific now isn't it!? I am afraid that 100k to 500k is a bit much if there already is a warning...
• How reliable is the DropBox mechanism in the first place

If anyone could share their experience with the import of larger amounts of data into Omero that would be rather helpful.

Cheers Juergen

[cblackburn]

Hi Juergen,

Is it possible or at least planned for importer-cli to allow an Omero user to import images for other users? E.g. have an admin "import" account being able to import for other regular users without knowing the users password but assigning the images to said user. Right now specifying a dataset id belonging to a different user throws a security violation error.

This isn't possible at the moment but it's a potentially useful feature. I have created a ticket:

https://trac.openmicroscopy.org.uk/omero/ticket/4033

is is possible to have images in the DropBox being assigned to either existing projects/datasets or even better to create projects/dataset that do not exist before the import? Right now images do show up under Images in Insight but project and dataset definitions in the OME header block are ignored

The only option at present is to set the cli-importer flags in the fs config to import to an already existing dataset but this would need setting for each separate dateset and then restarting DropBox each time. Again, I've created a ticket for a more general solution:

http://trac.openmicroscopy.org.uk/omero/ticket/4032

How reliable is the DropBox mechanism in the first place

The reliability depends on the underlying file monitoring systems. This varies between OSes. If a new file is detected and it is importable then it should be imported. At normal acquisition rates all files are seen. The monitoring systems can be tripped up by lots of very small files being written very quickly but copying across image files shouldn't present a problem.

The whole FS system (including DropBox) is ongoing work which will become a priority in the next few months. We have an overarching requirement for its improvement and development.

http://trac.openmicroscopy.org.uk/omero/ticket/2128

We would welcome any contributions to this requirement.

Colin