Multiple Omero Data Directories

[dwj]

Hi OMERO developers.

In our OMERO environment we use a network share as ‘omero.data.dir’. That works very smoothly.
Our network administration offers network shares with a maximum size of 16TB. Since we need more than 16TB, we could get a second network share with the same size of 16TB.

Is it possible to configure OMERO to use two or more network shares as ‘omero.data.dir’ so that we can use the complete 32TB?

I also think about using a Software-RAID to abstract multiple network shares to one big share. Does anybody have experiences with a software RAID? We use CentOS 7 as operation system.

Thanks,
Daniel

[kennethgillen]

Hi OMERO developers.

Hi Daniel,

Is it possible to configure OMERO to use two or more network shares as ‘omero.data.dir’ so that we can use the complete 32TB?

Yes, The OMERO managed repository, where newly imported data ends up, could be organised in such a way that some data is on one of your existing storage paths (in this case your network share) and newly imported data goes to another.

Our docs show the process under https://docs.openmicroscopy.org/latest/ ... repository - a good diagram showing the repository as new data is added is at http://downloads.openmicroscopy.org/pre ... paris/#/16

The binary repository layout itself (including managed repository) is discussed in our docs here: https://docs.openmicroscopy.org/latest/ ... itory.html

I also think about using a Software-RAID to abstract multiple network shares to one big share. Does anybody have experiences with a software RAID? We use CentOS 7 as operation system.

Software RAID isn't designed to work with remote filesystems like NFS mounts. You would be better off with a different filesystem type which does support sizes >16TB. There are a few options out there, which can be discussed with the team who are providing the storage infrastructure you're currently using.

Hope that's useful.

Kenny

[dwj]

Hi Kenny,

Thanks for your hints.
I changed the ‘omero.managed.dir’ settings and all new imaged files are stored in that new repository. So that works fine.

Is it possible that new images for one group (or user) are stored in one repository and the images of a second group (or user) are stored in a second repository?

Thanks,
Daniel

[jmoore]

Hi Daniel,

Is it possible that new images for one group (or user) are stored in one repository and the images of a second group (or user) are stored in a second repository?

There's only one repository, but you can change the template prefix to additionally take group information into account.

Note though that this only holds for new imports and if a user changes groups the data is not automatically migrated.

Cheers,
~Josh

[dwj]

OK,
thank you Josh!