OMERO virtual appliance extremely slow

[remco]

Hi,

In order to evaluate some software that would connect to a OMERO server, I installed the most recent OMERO virtual appliance from http://downloads.openmicroscopy.org/ome ... nce/5.1.3/
I run WIndows (8.1) on a very high end PC (64 GB RAM, SSD's and 12 core processing).
All went smooth, except that port 8080 on localhost is already occupied, so I used another port there (8888). I then also increased the amount of memory for OMERO and the amount of cores, as per the Troubleshooting guide.
But still, importing a single 200 MB (mrc) file took over 2.5 hours, whereas I would expect this to take only a second or two max! The first 60MB go fast, and then everything completely stalls.
If anyone could give some advise/hints/tips or even better: a solution, that would be greatly appreciated!

kind regards
Remco

[manics]

Hi Remco

Could you tell us what your current VM settings are (number of CPUs, memory, etc)? Is there anything in the OMERO.server logs (/home/omero/OMERO.server/var/log) or the system logs (/var/log) in the VM?

Can you also give us some details of the file e.g. type of image data, etc? If you want you can upload it to https://www.openmicroscopy.org/qa2/qa/upload/ and we can check whether it imports normally on our test server.

Thanks, Simon

[kennethgillen]

Hi Remco,

The OMERO configuration of your virtual appliance would also be useful (bin/omero config get).

If anyone could give some advise/hints/tips or even better: a solution, that would be greatly appreciated!

Another option: if your network supports it, you are welcome to test the application against our Demo server based at the University of Dundee. http://www.openmicroscopy.org/site/supp ... erver.html

All the best,

Kenny

[remco]

Dear Simon and Kenny,

thank you very much for your quick replies!
I have uploaded the dataset to the link Simon provided (KLH_EFTEM.mrc)
In the end, it imports fine, but takes a really long time (30+ minutes now on the latest test) from OMERO.insight to get into the VA OMERO.server on the same physical hardware (using LAN settings, and Intel PRO/1000 MT Server (NAT) setting for the network card on the VM.
Furthermore I use 4096MB mem and 4 processors on the VM.

I looked into the directories for the log files, but to be honest, as a newbie, I do not know what to look for. If you want to I could try to send you the logs, or you could perhaps be a bit more specific on what logs I should look into and what to (not) expect in them?

the config get only gave four entries : the data dir to /home/omero/OMERO.data and three lines for name, pass and user, all of them with value ''omero'' (the default I guess).

Kenny, thanks for the offer for using the Demo server. Although I would prefer to have a VA locally for speed and being able to work offline (on my laptop) later, I have indeed submitted a request for an account, as it will for sure be informative to see how a properly setup external server behaves!

thanks again for any help!

kind regards
Remco

[remco]

Update : thanks to Kenny, I now have an account at the Dundee demo server.

I uploaded the same file that takes 30 min locally in the VA, in about 30 seconds (likely limited by bandwidth) to that server. So it seems the file is OK and there is really an issue with the VA.
Might it be that I needed to use another port (8888) instead of 8080?

kind regards and thanks again!
Remco

[manics]

Hi Remco

The web port shouldn't make a difference as long as 4063 and 4064 are unchanged.

Since you've got loads of memory on your host could you try increasing it to 8GB, restart the VM, and import again? If that doesn't speed things up could you zip up the whole logs directory (/home/omero/OMERO.server/var/log) and either include it as a forum post attachment, or upload it to https://www.openmicroscopy.org/qa2/qa/upload/ ?

Thanks, Simon

[remco]

Hi,

increasing to 8Gb did not help (I did this on a somewhat smaller PC with 12GB mem, but that should still be OK). The log file tar file was slightly too large to be attached to this message, so I have uploaded it per your request. I hope you're able to find something in it, and I hope it will also be useful to you.

kind regards and thanks again for all the help!

Remco

[kennethgillen]

Hi Remco,

I hope you're able to find something in it, and I hope it will also be useful to you.

We'll certainly be investigating, Remco. To let you know, we can replicate the behaviour with your dataset and our release version Virtual Appliance, so thank you for submitting that dataset.

kind regards and thanks again for all the help!

Thank _you_ for being in touch and providing us with the sample dataset, feedback and your time in this dialogue. It's most useful.

All the best,

Kenny

[jmoore]

I've opened a trello card to track our work on tracking down this issue: https://trello.com/c/SFjFGmvd/99-slow-va-import

We'll let you know if there are any further questions/discoveries!

All the best,
~Josh.

[remco]

Thank you all for your support! Let me know if I can be of any assistance!

kind regards
Remco