incorrect importing of Zeiss SmartSEM created TIFF

[dmankus]

Hello,
I've found a bug in the bio-formats plugins for ImageJ. When importing a Zeiss TIFF acquired with SmartSEM software, it uses the wrong metadata to set pixel size. Zeiss saves two metadata fields, one is "Pixel Size" and the other is "Image Pixel Size." The two will be the same if the image acquired is 1024 pixels in width. When other acquisition settings are chosen, "Pixel Size" remains the same, but "Image Pixel Size" updates to the true value.

I've uploaded 3 images to demonstrate the effect here: https://keybase.pub/dmankus/

Has this previously been reported? Is there a fix or a workaround?
I'm using up-to-date as of writing FIJI, running Bio-Formats Plugins for ImageJ Release 5.90-SNAPSHOT Build Date: 27 June 2018

[sbesson]

Hi,

many thanks for reporting this issue and providing some sample data allowing to reproduce it. I can confirm that the physical size populated by Bio-Formats 5.8.2 is the same across the three different images although it should be different.

I have captured this issue as well as some initial technical details from my investigation in https://trello.com/c/V4SzWlb7/254-leore ... ze-parsing. Briefly, the LEOReader which is used for reading these types of formats makes minimal and partly incorrect use of the TIFF tag containing all the metadata.

Without any change, the only workaround would be to write some code client-side to re-parse the tag and extract the correct Image Pixel Size value. The next step would be to update the reader parsing logic to read and populate the correct metadata. In terms of commitment, our current priority is to finalize our next minor release of Bio-Formats 5.9.0 and we will have no immediate capacity. If anyone has the time to help with this fix, they should definitely free to take ownership and we will be happy to give guidance and feedback.

Best,
Sebastien

[dmankus]

Sebastien,
Thanks for looking into and confirming the issue for me, I'd be happy to contribute in any way I can to fixing the issue, though I'm wholly unfamiliar with the Bio-Formats codebase. Some advice on getting started would be welcome.
Looking at the LEOReader code, it appears that it's attempting to populate the EHT, Filament, Working Distance, Physical Pixel Size, and Acquisition Date fields. In theory this seems like a reasonable subset of tags to begin with, at the least.
In practice, the imported image only has Working Distance and Physical Pixel Size populated. And Working distance is listed with units of µm, but the value is in meters.
As far as I understand what LEOReader is doing, I'm not sure why it's completely missing the other tags, as it looks like it ought to read those values fine.
-David

[dmankus]

Not really sure how to compile and test it locally, and this is a pretty hacked together attempt, but if we replace LEOReader Lines 105-109 with the following, the previously mentioned values should populate. I'm not sure if setting all physical sizes to micrometers is correct, but it's straightforward to change this to other units, if appropriate.

Code: Select all

    for (int row=36; row<lines.length; row++) {
     
      if (lines[row].equals("AP_IMAGE_PIXEL_SIZE")) {
// pixel size is stored in nm, converted now to micrometers
       double xSize = Double.parseDouble(lines[row+ 1].split("\\s+=\\s+")[1].replace(" nm","e+03"));
      }
      else if (lines[line].equals("AP_WD")) {
//working distance stored in mm, converting now to micrometers
       double workingDistance = Double.parseDouble(lines[row+ 1].split("\\s+=\\s+")[1].replace(" mm","e-03"));
      }
      else if (lines[line].equals("AP_ACTUALCURRENT")) {
        double filament = Double.parseDouble(lines[row+ 1].split("\\s+=\\s+")[1].replace(" A",""));
      }
      else if (lines[line].equals("AP_ACTUALKV")) {
        double eht = Double.parseDouble(lines[row+ 1].split("\\s+=\\s+")[1].replace(" kV","e+03").replace(" V",""));
      }
    }


note: 94-101 should probably be removed, or someone familiar with the expected time/date format can parse the metadata lines:

Code: Select all

AP_TIME
Time :15:47:48
AP_DATE
Date :26 Jun 2018


[dmankus]

The date issue wasn't as confusing as I thought, since DateTools is later called to fix formatting.

I have forked it on GitHub with the suggested edits in place: https://github.com/openmicroscopy/biofo ... 1?expand=1

I didn't test it, though, so I'm refraining from submitting a pull request. Perhaps someone else can check my work for me. Otherwise I'll have to go through installing all the tools when I have a chance in a few days.

[sbesson]

Hi David,

thanks for driving this. Having looked briefly at the changes, your patch feels like a good start.

You can find more instructions about building Bio-Formats at https://docs.openmicroscopy.org/bio-for ... index.html if you want to test your changes before submitted.

To get some feedback from the OME team, opening a Pull Request against the Bio-Formats repository is certainly your best option. Initially, we can certainly provide a code review before including your contribution in the daily builds testing Bio-Formats changes against our curated datasets.

Best,
Sebastien

[dmankus]

Did what I could, and while my testing shows the code seems to parse the text, I'm missing something and the result isn't understood down the line. I put in a pull request in the hopes someone can point out my errors.

[dgault]

Thank you for taking the time to open the PR, adding a link for anyone following this thread: https://github.com/openmicroscopy/bioformats/pull/3178

[sbesson]

Hi David,

thank you very much for the contribution to the Bio-Formats code base. We are currently reviewing the changes and looking into integrating them in an upcoming Bio-Formats release.

The sample files you uploaded to us will also be added to our curated QA repository for the testing of future Bio-Formats releases. To facilitate community contributions and the maintenance of this library, would it be possible to share those files publicly under a CC-BY license and make them available at https://downloads.openmicroscopy.org/images/? If so, we can provide a first location where you can review the content and licensing. If this is not possible, we will add these sample files to our curated repository but keep them private.

Best,
Sebastien

[dmankus]

Absolutely, I acquired the images specifically to troubleshoot this issue, so I'd be happy to have them used as part of the repository. Let me know what I'll need to do to release them officially, and if you need them re-uploaded in some capacity.