Many Pyramid Files with 0 kB

[albot]

Hi,

We recently found that in the pixel dirs many files are present which are all 0kb files.

How could we trigger that those files are recalculated? We thought that the pyramid files are generated based on other pixel from OMERO based on the source image?

Thanks a lot for your feedback.

Best regards
Alex

[jmoore]

Hi Alex,

Sorry, several questions before I feel safe in answering:

• Are there matching non-pyramid files in the same directory? i.e. if you are seeing empty /OMERO/Pixels/100_pyramid files is there also an /OMERO/Pixels/100 file?
• What imported file types are causing the errors? All of them?
• What size are the failing files?
• What does "bin/omero admin diagnostics" show?
• Are there any errors in var/log/PixelData-0.log? in Blitz-0.log?

Cheers,
~Josh

[rdccmemg]

Dear Josh,

No matching non-pyramid files in the same directory.
The logs.zip attachment contains the diagnostics output and a snip out of the pixeldata-0.log.
The files are ordinary jpg files from a consumer camera. Resultion 4608x2592.
Some show up fine others don't and have the 0kb pyramid file.

Thanks,
Raf

[jmoore]

Thanks, Alex. The following certainly looks like a smoking gun:

Code: Select all

013-08-22 20:38:49,342 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Creating BfPixelBuffer: /home/ict/apps/OMERO/OMERO.data/Files/Dir-034/34397 Series: 0
2013-08-22 20:38:49,342 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Destination pyramid tile size: java.awt.Dimension[width=256,height=256]
2013-08-22 20:38:49,342 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:24392 1/594 (0%).
2013-08-22 20:38:49,722 ERROR [                ome.io.nio.PixelsService] (2-thread-1) FAIL -- Error during tile population


Is /home/ict/apps/OMERO/OMERO.data/Pixels/Dir-024/24392_pyramid one of the empty files? If so, could you send us /home/ict/apps/OMERO/OMERO.data/Files/Dir-034/34397 to test?

Also, have you changed the memory settings on PixelData at all?

Cheers,
~Josh

[rdccmemg]

Hi Josh,

you can download the file here http://perswww.kuleuven.be/~u0039776/

I did not change memory setting on Pixeldata in templates.xml. it's still on 256MB, is that the prob?

I did at one point increase the memory on BlitzTemplate from512M to 2048M because we were seeing java OOM errors.
thx for having a look.

Best,
Raf

[rdccmemg]

So I increased memory to 4096M and maxpermsize to 512M.

Then I renamed a 0byte pyramid fila and the according thumbnail (also 0kb).
Restarted OMERO and tried to view them again via webinterface.
Thumbnail and pyramid are recreated and seem fine now.

Thus if I delete all 0kb pyramids and thumbnails, it should recreate correctly? Can you confirm this?

thanks!
Raf

[jmoore]

It should certainly be safe to delete any 0KB files under either /Pixels or /Thumbnails (with the caveat that with amazingly bad timing you could delete a file that has just been created). One possibility would be to use `find -mtime +1 ...` to chose files that haven't been modified in the past day.

If there's anything else we can help with in the clean up process, just let us know.
~Josh.

P.S https://trac.openmicroscopy.org.uk/ome/ticket/11395 opened to provide a helper for this provide.

[rdccmemg]

because images are of resolution 4608x2592.

should I change
omero.pixeldata.max_plane_width=3192
omero.pixeldata.max_plane_height=3192

to at least 4608?
in pixeldata log there were some reports of
"Expected positive value for PhysicalSizeX; got Infinity"
I'm suspecting this has to to with the max values defined in pixeldata?

Best,
Raf

[jmoore]

because images are of resolution 4608x2592.

should I change
omero.pixeldata.max_plane_width=3192
omero.pixeldata.max_plane_height=3192

to at least 4608?

You can but it will put a higher memory burden on your server. Pyramid generation is an attempt to pre-process larger images into a format that's easier/quicker to work with. Especially jpegs are difficult since the entire image must be read into memory.

in pixeldata log there were some reports of
"Expected positive value for PhysicalSizeX; got Infinity"
I'm suspecting this has to to with the max values defined in pixeldata?

This is a metadata warning and should be unrelated to generation itself.

Cheers,
~Josh

[rdccmemg]

Bulk removing the 0kb files does not work. It works for the 1st and all other fail with the same 0kb result.

In blitz log I found this which seems odd:

Code: Select all

2013-08-23 12:26:02,825 INFO  [                ome.io.nio.PixelsService] (.Server-15) Using Pyramid BfPixelBuffer: /home/ict/apps/OMERO/OMERO.data/Pixels/Dir-024/24371_pyramid
2013-08-23 12:26:02,835 INFO  [       loci.formats.in.MinimalTiffReader] (l.Server-3) Populating metadata
2013-08-23 12:26:02,836 ERROR [                ome.io.nio.PixelsService] (.Server-15) Error instantiating pixel buffer: /home/ict/apps/OMERO/OMERO.data/Pixels/Dir-024/24371_pyramid
java.lang.RuntimeException: java.io.IOException: Permission denied
        at ome.io.bioformats.BfPyramidPixelBuffer.isLockedByOthers(BfPyramidPixelBuffer.java:377)
        at ome.io.bioformats.BfPyramidPixelBuffer.initializeReader(BfPyramidPixelBuffer.java:185)
        at ome.io.bioformats.BfPyramidPixelBuffer.<init>(BfPyramidPixelBuffer.java:171)
        at ome.io.bioformats.BfPyramidPixelBuffer.<init>(BfPyramidPixelBuffer.java:141)
        at ome.io.nio.PixelsService.createPyramidPixelBuffer(PixelsService.java:636)
        at ome.io.nio.PixelsService.getPixelBuffer(PixelsService.java:410)
        at ome.logic.RenderingSettingsImpl.resetDefaults(RenderingSettingsImpl.java:500)
        at ome.logic.RenderingSettingsImpl.resetDefaults(RenderingSettingsImpl.java:1324)
        at ome.services.ThumbnailBean._resetDefaults(ThumbnailBean.java:1266)


No idea where this permission denied comes from...
What I changed as well was
"omero.pixeldata.threads=4
omero.pixeldata.max_plane_height=5000
omero.pixeldata.max_plane_width=5000"

Currently I only increased memory for the BlitzTemplate, should I also increase memory on other server-templates (PixelDataTemplate for instance?)?

appreciate your help!
Best,

Raf