300 concurrent users

[tla]

Hello,
Our client is looking for a system to support 300(!) concurrent users in a teaching environment.
Do you think OMERO can scale to this? If so, how much hardware will they need?
I am sorry to be rather vague but we don't know how many images etc they will be using.
Regards

[atarkowska]

Hi,

We have about 50-100 users using virtual microscope at once in a class. Performance seems to be fairly good (even we do not use any caching). I think, vanilla OMERO shouldn't have any issues with 300 concurrent users at all. It may requires OMERO performance optimization https://www.openmicroscopy.org/site/sup ... mance.html as well as turning on webserver caching, etc.

The OMERO.server is deployed on a Dell PowerEdge C6220 computer with 2 x Hex Core Xeon CPUs @ 2.50GHz (Intel® HT Technology), 96GB of RAM, running CentOS 6, PostgreSQL 9.4, Nginx, with locally attached spinning disk.

Here is a benchmark run on our server for 50 concurrent users making 200 requests per user (roughly equal to opening 10 tiled images at once).

Code: Select all

Concurrency Level:      50
Time taken for tests:   592.923 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      273560000 bytes
HTML transferred:       271460000 bytes
Requests per second:    16.87 [#/sec] (mean)
Time per request:       2964.616 [ms] (mean)
Time per request:       59.292 [ms] (mean, across all concurrent requests)
Transfer rate:          450.56 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       20   43  55.0     37     840
Processing:   347 2915 595.0   2684    5233
Waiting:      344 2914 594.7   2683    5232
Total:        839 2958 588.3   2728    5272

What kind of data your client would like to host? Are they compressed pyramidal-tiled structures?

Ola

[tla]

Hi,
Thanks for great reply
Our client has told us in response:
We'll mostly be using Hamamatsu NDP images, there'll be around a dozen per class so caching should work pretty well I guess. Maybe we should get more RAM for the caching or would that be on the disks?
Maybe an SSD-cached RAID card like an Adpaptec 71650Q would help?
TIA
tla

[atarkowska]

You may want to also check how nginx cache could be used in viewtopic.php?f=5&t=7802#p15741

Ola

[kennethgillen]

Hi tla,

We'll mostly be using Hamamatsu NDP images, there'll be around a dozen per class so caching should work pretty well I guess. Maybe we should get more RAM for the caching or would that be on the disks?

Bio-Formats' supported formats list has a pixel's rating of "Fair" for NDPI images. It'd be worth testing a selection of the images with either your own test implementation or our OMERO demo server.

Maybe we should get more RAM for the caching or would that be on the disks?
Maybe an SSD-cached RAID card like an Adpaptec 71650Q would help?

There are certainly many ways to go about this. As Ola has mentioned, the recent thread discussing the use of Nginx's FastCGI cache is likely to be of use. This appears to be cache to RAM, not disk. I've not heard of anyone using SSD RAID cache cards specifically, but we have certainly heard good feedback from having the OMERO database on SSD.

I'd be inclined to try RAM and caching, with local SSD, and then you've the option to add something like PCIe SSD down the line, provided your server form-factor supports that.

Good luck with the deployment, and let us know how it goes. I'm sure the community would hugely benefit from hearing how you get on. Once you're happy with your configuration, feel free to contribute to our Deployment Questionnaire which would help us feed back to the rest of the community, once you're happy with your configuration.

All the best,

Kenny

[atarkowska]

Hi tla,

We'll mostly be using Hamamatsu NDP images, there'll be around a dozen per class so caching should work pretty well I guess...

Bio-Formats' supported formats list has a pixel's rating of "Fair" for NDPI images. It'd be worth testing a selection of the images with either your own test implementation or our OMERO demo server.

In 5.1 release we made significant improvement in NDPI reader. If you still see some incorrect metadata please send us a sample file with a short description of what is wrong using https://www.openmicroscopy.org/qa2/qa/upload/

Ola

[tla]

Thanks again for helpful replies. Will let you know once we get the go-ahead for this.