Page 1 of 1

Enforcing storage limit per user/group

PostPosted: Mon Jan 29, 2018 6:10 pm
by ratamero
Hi there!

we're in the process of setting up an OMERO install at Warwick Medical School - we expect this to use large amounts of storage in the next few years (petabyte-scale), so ideally we'd want to enforce maximum storage limits per group/PI. I did some searching on my own, but couldn't find any way of doing it. Is it a feature that exists? If not, is it a feature that is planned in the near future? :)

Thanks in advance!

Re: Enforcing storage limit per user/group

PostPosted: Mon Jan 29, 2018 7:23 pm
by jmoore
OMERO does not currently provide a quota feature. The closest that is provided by default is the ability to configure the layout of /OMERO/ManagedRepository so that you can both define quotas at the filesystem level as well as expand storage as your server grows.

See http://downloads.openmicroscopy.org/presentations/2014/fs-workshop-paris/#/14 for graphical examples of various configurations.

Regards,
~Josh

Re: Enforcing storage limit per user/group

PostPosted: Tue Jan 30, 2018 11:56 am
by ratamero
Cool, thanks! That will hopefully be enough to enforce quotas. I've edited omero.properties to use:

omero.fs.repo.path=%groupId%/%user%_%userId%//%year%-%month%/%day%/%time%

I've restarted the server, but it still seems that new data is being saved in the old repo path format - is tere anything I'm missing here?

Re: Enforcing storage limit per user/group

PostPosted: Tue Jan 30, 2018 12:28 pm
by mtbc
I can't see what you'd be missing. In a local test build I tried changing mine to:
Code: Select all
$ bin/omero config get omero.fs.repo.path
%groupId%/%user%_%userId%//%year%-%month%/%day%/%time%

Code: Select all
$ find ManagedRepository/ -name '*.dcm'
ManagedRepository/user-2_3/2018-01/30/12-19-06.099/ankle.dcm
ManagedRepository/3/user-2_3/2018-01/30/12-20-08.576/brain.dcm

I imported "ankle.dcm" before the change, then restarted the server with the new configuration and imported "brain.dcm".

My initial advice would be to,
  • use "config get" to double-check the configuration
  • restart the server
  • double-check that you're finding a genuinely new import in "ManagedRepository/".
If there's still a baffling lack of picking up the new configuration, please go ahead and send us the output of "bin/omero admin diagnostics" and paste a transcript of the checks you did via shell then we'll try to help you to get to the bottom of what's going on.

Cheers,
Mark

Re: Enforcing storage limit per user/group

PostPosted: Tue Jan 30, 2018 1:41 pm
by ratamero
For some reason omero.properties seem to be completely ignored - bin/omero config get omero.fs.repo.path returned empty. By doing bin/omero config set omero.fs.repo.path with the correct string, it seems to work now. Thanks again!

Re: Enforcing storage limit per user/group

PostPosted: Tue Jan 30, 2018 2:04 pm
by jmoore
ratamero wrote:For some reason omero.properties seem to be completely ignored


Ah, that would explain it. See the top of the file for an explanation.

Let us know if you have any other issues.
~Josh.

Re: Enforcing storage limit per user/group

PostPosted: Tue Jan 30, 2018 2:14 pm
by ratamero
Serves me right for not reading things :lol: