Uploading large tiff genereated wrong image on OMERO 5 RC1

[bpavie]

I am currently testing OMERO 5 RC1 and I was testing it with some large tiff files, and realized that there is a bug on how they are managed (looks to be that the image is stored in a duplicate pyramidal format that is wrong). The result is that I can download fine the original image but I can't visualize it properly:

viewer.png (69.16 KiB) Viewed 9370 times

The tiff file has been saved from a dm3 file using Fiji.
The upload of the dm3 file and the rendering using the Omero viewer is working fine.

However (and this is probably an other issue), I am not able to open both of them using the Omero plugin in Fiji. Smaller image from other dataset are however open fine using the plugin. I didn't notice any error in the Blitz-0.log file during the opening, it is just hanging for ever from the user point of view until you have to cancel it...

Let me know if you want a sample of the data.
DM3 file is an 8000x8000 16 bits file 1 channel for a size of 122 MB, same thing for the TIFF.


Benjamin

[jmoore]

Hi Benjamin,

A sample file uploaded to http://qa.openmicroscopy.org.uk/qa/upload/ would be ideal. If possible, could you also include the PixelData-0.log file, zipped, from var/log on the server?

Cheers,
~Josh

[bpavie]

Hi Josh,

I uploaded the 2 files:
http://qa.openmicroscopy.org.uk/qa/feed ... 7c6ba7d6c2
http://qa.openmicroscopy.org.uk/qa/feed ... be5e323cc8

Best,
Benjamin

[jmoore]

There's an OutOfMemory exception in the log file, which is fairly likely causing the corrupted pyramid file:

Code: Select all

2014-01-29 16:12:17,174 INFO  [                ome.io.nio.PixelsService] (3-thread-2) Creating BfPixelBuffer: /opt/OMERO/OMERO.data/ManagedRepository/bpavie_2/2014-01/29/16-12-03.239/Prefix_3VBSED_slice_0000.tif Series: 0
2014-01-29 16:12:17,178 INFO  [                ome.io.nio.PixelsService] (3-thread-2) Destination pyramid tile size: java.awt.Dimension[width=256,height=256]
2014-01-29 16:12:17,179 INFO  [                ome.io.nio.PixelsService] (3-thread-2) Pyramid creation for Pixels:4701 1/1024 (0%).
2014-01-29 16:12:18,564 INFO  [       loci.formats.in.MinimalTiffReader] (3-thread-2) Reading IFDs
2014-01-29 16:12:18,569 INFO  [       loci.formats.in.MinimalTiffReader] (3-thread-2) Populating metadata
2014-01-29 16:12:18,579 INFO  [              loci.formats.in.TiffReader] (3-thread-2) Checking comment style
2014-01-29 16:12:18,580 INFO  [          loci.formats.in.BaseTiffReader] (3-thread-2) Populating OME metadata
2014-01-29 16:12:19,106 ERROR [        ome.services.util.ServiceHandler] (3-thread-2) java.lang.Error:  Wrapped Exception: (java.lang.OutOfMemoryError):
Java heap space
java.lang.OutOfMemoryError: Java heap space
        at loci.formats.tiff.TiffParser.getSamples(TiffParser.java:897) ~[scifio.jar:na]
        at loci.formats.tiff.TiffParser.getSamples(TiffParser.java:745) ~[scifio.jar:na]
        at loci.formats.in.MinimalTiffReader.openBytes(MinimalTiffReader.java:292) ~[scifio.jar:na]
        at loci.formats.DelegateReader.openBytes(DelegateReader.java:219) ~[scifio.jar:na]
        at loci.formats.ImageReader.openBytes(ImageReader.java:435) ~[scifio.jar:na]
        at loci.formats.ChannelFiller.openBytes(ChannelFiller.java:197) ~[scifio.jar:na]
        at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:226) ~[scifio.jar:na]
        at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:332) ~[scifio.jar:na]
        at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:332) ~[scifio.jar:na]
        at loci.formats.MinMaxCalculator.openBytes(MinMaxCalculator.java:273) ~[scifio.jar:na]
        at ome.io.bioformats.BfPixelsWrapper.getTile(BfPixelsWrapper.java:345) ~[romio.jar:na]


The PixelData service is still at the default 256 MB:

Code: Select all

2014-01-29 16:02:55,932 INFO  [      ome.services.util.JvmSettingsCheck] (      main) Max Memory (MB):   =    227
2014-01-29 16:02:55,936 INFO  [      ome.services.util.JvmSettingsCheck] (      main) OS Memory (MB):    =   7983


Could you spare up to 1GB for it? See http://www.openmicroscopy.org/site/support/omero4/sysadmins/unix/server-installation.html#jvm-memory-settings for instructions.

Cheers,
~Josh

[bpavie]

Thanks, that solve the problem.
I did the following modification in the OMERO.server/etc/grid/templates.xml

Code: Select all

<server-template id="PixelDataTemplate">
      <parameter name="index"/>
      <parameter name="dir"/>
      <parameter name="config" default="default"/>
      <server id="PixelData-${index}" exe="${JAVA}" activation="always" pwd="${OMERO_HOME}">
        <target name="heap-dump">
            <option>-XX:+HeapDumpOnOutOfMemoryError</option>
        </target>
        <target name="heap-dump-tmp">
            <option>-XX:+HeapDumpOnOutOfMemoryError</option>
            <option>-XX:HeapDumpPath=/tmp</option>
        </target>
        <!-- <option>-Xmx256M</option> -->
        <option>-Xmx1024M</option>

It would be nice to describe the following more in details in the documentation. I didn't fully realized that they was multiple service (Blitz, PixelData, Indexer and Repository), each one with their own memory setting, so I modified only the Blitz one... Is that also recommended to increase the memory for the indexer and the repository ?

Thanks,
Benjamin

[Sethur]

Hi Benjamin & Josh,

I agree on this one, this has to be included in the documentation in more detail. Showing a reg-ex search and replace command is not enough for sys admins to even realize that there is more than one memory setting. Also, there has to be a description of what makes sense in small, medium or large user environments, etc. and what impact the settings have on the functioning of the OMERO server.

Personally, I did not use the rex-ex replace in the beginning and only now realized that I missed the memory settings for all the other services besides Blitz.

Cheers,

Tristan

[sbesson]

Hi Tristan and Benjamin,

thanks for all the valuable feedback on this thread. We are agree our sysadmin documentation still lacks many details like the description of the server templates. An initial documentation Pull Request is currently under examination to improve its Advanced configuration section.

Regarding other server instances (Repository, Indexer), according to your logs, there is no reason to increase their default memory settings.

Best regards,
Sebastien