Public data - alternative method

[dsudar]

Hi all,

Following up on a discussion I had with Ola at the OME meeting, this is to initiate a discussion about a alternative way to make data in OMERO publicly accessible.

My question to Ola was: is there a way to make data that is in a Read-Annotate group publicly accessible but with all the read-only protections of a properly set up public repository access as Ola explained in her presentation: http://downloads.openmicroscopy.org/pre ... ository/#/
But I was asking whether I could make the data publicly accessible while leaving it in the R-A group where our internal data generation folks are still analyzing, annotating, QA'ing, etc. the data. This question has come up before in the forum: http://www.openmicroscopy.org/community ... 9987#p9987 As discussed in that thread, I want to avoid duplicating the data or having to split the data over multiple groups and making life difficult for internal "full access" users.

After some discussion Ola came up with an interesting concept: use the Share mechanism (http://help.openmicroscopy.org/sharing-data.html#share) to share the data that needs to be public with the "public user" that has been set up in a R-O public group per the instructions. While this will not preserve the organization of data as it exists in the R-A group where it resides, it will become accessible publicly through the WebGateway and similar mechanisms. For my needs that is sufficient so I was all gung-ho to try it out.

However, when trying to start setting up a test site I ran into a few issues and have some questions:
- For images in regular projects/datasets the approach appears to work fine but not so for images in SPW. When you select those, they cannot be "shared" probably because the Share button mechanism only allows to share images and not Wells which is what gets selected in a SPW. Maybe a small change to consider in a next version?
- While creating shares works by selecting images (with the restriction above), it would be more convenient if one could select entire datasets, projects, s/p/w's to publish them. Similar to #1, maybe a next OMERO version could allow that type of selection and have an iterator select all the images in the parent structure?
- Lacking the #1 and #2 functionalities at this time and in general, wanting to do this programmatically, I looked for a CLI or (Python) API way to use the Share functionality. I found that there is an API (http://downloads.openmicroscopy.org/ome ... Share.html) to do this but I couldn't find some examples to show how to use that functionality and it looks a bit daunting.

I hope this approach might also be useful to others and that any tools or new functions would be generally welcome.

Thanks to Ola for coming up with this promising approach.
Cheers,
- Damir

[atarkowska]

Hi Damir,

Thank you for bringing up our important discussion into the forum.

However, when trying to start setting up a test site I ran into a few issues and have some questions:
- For images in regular projects/datasets the approach appears to work fine but not so for images in SPW. When you select those, they cannot be "shared" probably because the Share button mechanism only allows to share images and not Wells which is what gets selected in a SPW. Maybe a small change to consider in a next version?
- While creating shares works by selecting images (with the restriction above), it would be more convenient if one could select entire datasets, projects, s/p/w's to publish them. > Similar to #1, maybe a next OMERO version could allow that type of selection and have an iterator select all the images in the parent structure?

We are in the process of investigating potential solutions, such as sharing entire group without need to manually choose individual objects. Would such a solution work for you?

- Lacking the #1 and #2 functionalities at this time and in general, wanting to do this programmatically, I looked for a CLI or (Python) API way to use the Share functionality. I found that there is an API (http://downloads.openmicroscopy.org/ome ... Share.html) to do this but I couldn't find some examples to show how to use that functionality and it looks a bit daunting.

IShare service was designed as a proof of concept and didn't get many feedback. Functionality is limited to images only, but I am sure we can work together on bringing more functionality to the service. Probably the best place to look for examples are integration tests https://github.com/openmicroscopy/openm ... _ishare.py

I hope that will help.

I will keep you posting.

Ola

[wmoore]

Hi,

Maybe I'm missing something, but if you were to simply disable all the public urls that allow annotation (in the same way we do to disallow E.g. running scripts for public users in a read-only group) then would that allow you to use a read-annotate group for public data?

Will.

[atarkowska]

Will,

It is clearly stated that existing solution is designed for read-only group https://www.openmicroscopy.org/site/sup ... ublic-user. Following your suggestion may lead to a high-severity vulnerability. I wouldn't recommend anyone to try that out without extensive security testing!

Ola

[dsudar]

Hi Ola,

DS: While creating shares works by selecting images (with the restriction above), it would be more convenient if one could select entire datasets, projects, s/p/w's to publish them. > Similar to #1, maybe a next OMERO version could allow that type of selection and have an iterator select all the images in the parent structure?

We are in the process of investigating potential solutions, such as sharing entire group without need to manually choose individual objects. Would such a solution work for you?

That would be a reasonable solution especially if that is relatively easy to implement. Long term it would be desirable to have more granular control over which data to publish/expose and which data to keep private but still keep it all in one group.

IShare service was designed as a proof of concept and didn't get many feedback. Functionality is limited to images only, but I am sure we can work together on bringing more functionality to the service. Probably the best place to look for examples are integration tests https://github.com/openmicroscopy/openm ... _ishare.py

Great. I'll look at the code you indicate. I will probably come back with questions and appreciate your help.

Thanks,
- Damir

[atarkowska]

Hi Damir,

With the recent security release we had to make shares image centric. You should be still good to test and share images, original rendering settings, etc. but no annotations, ROIs, and containers these images are stored in.

At the moment, if you wish to play yourself feel free to look at the PR https://github.com/openmicroscopy/openm ... 4771/files that may help you to customize the graph and allow shareing more objects. In addition to hibernate filters second check determining access to the object is done by BlobShareStore. But please keep in mind that this is temporary solution and may change in the future as we are targeting a more flexible solution that will suit your needs for the 5.3.x line. At the moment sharing an entire group is not an option.

Let us know how is that going and if you have any more questions.

Ola