apparent reorganization of cropped .nd2 files
Posted: Wed Feb 22, 2017 7:45 pmI have found twice that when I crop a .nd2 file within Nikon NIS elements software, the resulting cropped .nd2 file is not interpreted correctly by the bioformats `showinf -omexml-only` command. The file in question was initially ~1 TB but was cropped to ~200GB. There were originally 19 series, ~145 time points, and 31 z-slices per time point, and the cropped file contains the same 19 series and 31 z-slices per time point, but now only 27 time points.
Within the .ome.xml metadata produced by `showinf` command, the indicated dimensionality is the original one (XYCZT), but the images are in a different order. Normally, the metadata has the following basic organization:
Thus, there should be 19 <Image></Image> blocks with 27*31=837 <Plane> lines within them, with the <Plane> lines organized as Timepoint -> z-slice (i.e., all z-slices for time point 1, followed by all z-slices for time point 2, etc.). This is what happens with uncropped files. However, after cropping, the organization, per the `showinf` command, is bewildering: the 19 series become 27 series (presumably the highest-level organization is now that of the timepoint), and within each series there are 31*19=589 <Plane> lines. However, the <Plane> lines do not appear in order of series/z but rather in order of increasing time, with X/Y/Z stage positions of adjacent lines being completely uncorrelated. The order is not Series(Z), i.e., consecutive z-slices for each series because the time-step is rather constant and much larger than that of the time between z-slices (exposure length + stage Z movement). Overall, the 27*31*19=15,903 images are arranged in groups of 27 x (31*19), but it's hard to say discern order beyond the fact that <Planes> occur in sequential time order within each group.
One other thing to note is that the size of the metadata file (.ome.xml) in relation to the .nd2 file is larger than normal. Typically, I found that .ome.xml files are about 100ppm of the corresponding .nd2 file (about 1 MB of .ome.xml metadata per 10 GB of .nd2 data). The .ome.xml files for cropped .nd2 files have sizes closer to what one would expect for the original, uncropped file, although most of the excess is explained by the <StructuredAnnotations> section rather than the <OME> section. Thus, I suspect that when NIS Elements writes the cropped file, it may leave behind metadata for data that was removed by cropping. Perhaps, these metadata remnants confuse the `showinf` command.
As I rely on the metadata generated by `showinf` to process my data sets, I find that I am unable to process any data set that has been cropped within NIS elements. Can anything be done to obtain the correct metadata using `showinf`?
Within the .ome.xml metadata produced by `showinf` command, the indicated dimensionality is the original one (XYCZT), but the images are in a different order. Normally, the metadata has the following basic organization:
- Code: Select all
<Image (series00)>
<Pixels>
<Plane timepoint 1, z 1>
<Plane timepoint 1, z 2>
...
<Plane timepoint 1, z 31>
<Plane timepoint 2, z 1>
...
</Pixels>
</Image>
<Image (series01)>
...
...
</Image>
Thus, there should be 19 <Image></Image> blocks with 27*31=837 <Plane> lines within them, with the <Plane> lines organized as Timepoint -> z-slice (i.e., all z-slices for time point 1, followed by all z-slices for time point 2, etc.). This is what happens with uncropped files. However, after cropping, the organization, per the `showinf` command, is bewildering: the 19 series become 27 series (presumably the highest-level organization is now that of the timepoint), and within each series there are 31*19=589 <Plane> lines. However, the <Plane> lines do not appear in order of series/z but rather in order of increasing time, with X/Y/Z stage positions of adjacent lines being completely uncorrelated. The order is not Series(Z), i.e., consecutive z-slices for each series because the time-step is rather constant and much larger than that of the time between z-slices (exposure length + stage Z movement). Overall, the 27*31*19=15,903 images are arranged in groups of 27 x (31*19), but it's hard to say discern order beyond the fact that <Planes> occur in sequential time order within each group.
One other thing to note is that the size of the metadata file (.ome.xml) in relation to the .nd2 file is larger than normal. Typically, I found that .ome.xml files are about 100ppm of the corresponding .nd2 file (about 1 MB of .ome.xml metadata per 10 GB of .nd2 data). The .ome.xml files for cropped .nd2 files have sizes closer to what one would expect for the original, uncropped file, although most of the excess is explained by the <StructuredAnnotations> section rather than the <OME> section. Thus, I suspect that when NIS Elements writes the cropped file, it may leave behind metadata for data that was removed by cropping. Perhaps, these metadata remnants confuse the `showinf` command.
As I rely on the metadata generated by `showinf` to process my data sets, I find that I am unable to process any data set that has been cropped within NIS elements. Can anything be done to obtain the correct metadata using `showinf`?