Download as... restrictions

[PaulVanSchayck]

Hello,

I've got an interesting use case. From the webclient, the "Download as [format]" option will zip all files together when multiple images are selected. This will give a delay in the UI while the file is generated. One of our users was unaware of this, and repeatedly restarted the request (70 PNG images, started over 10 to 15 times in the space of a minute, re-rendered to PNG).

Obviously, issues arise from this. Eventually the 8GB of HEAP memory of the JVM wil run out.

Code: Select all

2015-03-19 13:47:57,279 ERROR [        ome.services.util.ServiceHandler] (Server-395) java.lang.Error:  Wrapped Exception: (java.lang.OutOfMemoryError):
Java heap space
java.lang.OutOfMemoryError: Java heap space
        at omeis.providers.re.RGBIntBuffer.<init>(RGBIntBuffer.java:69) ~[common.jar:na]

I've a few questions about this:

- All the http requests that were started will eventually time out, or throw a 505 error. What's worrying however, is that the HEAP memory is not released even several hours later. New completely unrelated tasks will also run OOM. This seems like a bug to me. Do you want the Blitz log of this?

- I've seen several PR on Github related to file Download restrictions. Will any of these help with this in 5.1?

- Is it intended that a PNG image is re-rendered to PNG when "Download as PNG" is selected?

Thanks,

Paul

[wmoore]

Hi Paul,

Can you tell me what version of OMERO you are using?
Do you see a dialog box appear when you choose to download PNG for multiple images (see screenshot)?
I forget which OMERO release we added this exactly, but this should only allow you a single click to start the zip creation and download, so unless you launch this dialog multiple times then it shouldn't be possible to start multiple downloads.

When the download is complete (http request returns) then all open services (rendering engines etc) will be closed. However, I see that we're not closing the rendering engine for each image as we're preparing the zip, so this should certainly be fixed - ticket created: http://trac.openmicroscopy.org/ome/ticket/12794

The "Download restrictions" PRs are more about permissions, E.g. for public repositories rather than performance. However, in future this may include a "Max Size" configuration to restrict the number or size of files that users are allowed to download.

It does seem crazy to render a PNG as PNG, but if you simply want to download the original PNG then you should use the "Download" option, not the "Export PNG" option.
The "Export PNG" will take into account the current rendering settings etc, and is the same behaviour for all images in OMERO. It will be tricky for the code to guess that you really want the Original PNGs.

Hope that helps - let us know how the repeated download was started.
Cheers,

Will.

[PaulVanSchayck]

Hi Will,

This is OMERO 5.0.3. Yes, the dialog box appears. The user in question was not a 100% sure how he created the multiple requests. I've managed to recreate it by both opening 4 windows, and by refreshing once I had pressed "OK" and then starting it again.

Good to know that it was indeed an issue with the non released memory. Thanks for the PR.

Yeah, I see the use of the re-rendering as PNG. However, for the novice user this causes a bit of confusion. I've seen this confusion now multiple times. I'm not sure how to exactly solve this. I guess "Download Original File" is too long as a label instead of "Download..."

Thanks,

Paul

[wmoore]

Hi Paul,

Unfortunately there's no way I can think of preventing multiple downloads from multiple windows (or page refreshes). Perhaps we could make the dialog a little more obvious.

Similarly with the "Download..." vv "Download Original File". I'm not sure which of these would be clearest to most users. Thanks for your feedback - we'll discuss etc....

Cheers,

Will.