Page 1 of 1

postgres volume recommended size

PostPosted: Thu Jan 04, 2018 3:35 am
by moisbo
Hi,

(First thanks for the responses!)

I would like to create a volume for postgres in my OMERO.db vm. I would like to know your thoughts on the recommended volume size for postgres database. I see there are some recommendations on the docs, but not specifically for the postgres data_directory size.

Cheers,
Moises

Re: postgres volume recommended size

PostPosted: Thu Jan 04, 2018 9:35 am
by manics
Hi Moises

Could you give us some details about your OMERO server, such as number of users, number of images/datasets, types of images, whether you plan to store additional metadata, etc?

Re: postgres volume recommended size

PostPosted: Fri Mar 02, 2018 12:43 am
by moisbo
Hi Manics,

Appreciate your reply. We are gathering that information and will get back to you. For the moment we are implementing in a filesystem where we are able to grow the storage on the postgres database with an initial 10GB and it is being monitored.

Many Thanks,
Moises

Re: postgres volume recommended size

PostPosted: Mon Mar 12, 2018 1:15 am
by moisbo
Hi Manics,

Could you give us some details about your OMERO server, such as number of users

Around 150 Users
number of images/datasets

1.8 million images, around 10 TB per year
types of images

raw images from microscope
whether you plan to store additional metadata, etc?

YES

Many Thanks,
Moises

Re: postgres volume recommended size

PostPosted: Mon Mar 12, 2018 3:53 pm
by manics
Hi Moises

Here are the stats for our internal production OMERO:
- 30 TB data (375,000 images)
- 70 GB postgres
- 450 users
- 270,000 annotations

For comparison the IDR (https://idr.openmicroscopy.org/):
- 50TB data (3,000,000 images)
- 150 GB postgres
- 3,200,000 annotations

So I think around 40GB for postgres would be a good value for you to start with, especially if it's easy for you to expand it later. If you prefer to minimise storage changes then 100GB would be safer.

Re: postgres volume recommended size

PostPosted: Thu Mar 15, 2018 2:56 am
by moisbo
Hi Manics,

Thanks a lot for your response. As a side question (I don't know if I need to do a new post)

We are currently hosting on a VM with 16GB RAM with . Would you care to share your configuration?
This is ours:
Code: Select all
omero.jvmcfg.heap_size=12g
omero.jvmcfg.percent.blitz=60
omero.jvmcfg.perm_gen=512m
omero.jvmcfg.strategy=percent


We would like to get more, however, we need to build a strong case for the IT department to provide for more.

Cheers,
Moises

Re: postgres volume recommended size

PostPosted: Thu Mar 15, 2018 10:15 am
by manics
Our internal OMERO is a VMWare VM with 32 GB RAM, and the following settings:
Code: Select all
omero.jvmcfg.percent.blitz=50
omero.jvmcfg.percent.indexer=20
omero.jvmcfg.percent.pixeldata=20
omero.jvmcfg.system_memory=17000


Most of the setup is done with Ansible, if you're interested have a look here: https://github.com/openmicroscopy/prod- ... o.yml#L270

Simon

Re: postgres volume recommended size

PostPosted: Thu Mar 15, 2018 10:39 pm
by moisbo
Thanks for that. We've deployed with Ansible as well so that is helpful

Moises