ZVI Timestamp

[chrub]

Hi,
I am having some trouble regarding timestamps in ZVI files:
i used the TimeStamp example:
https://github.com/openmicroscopy/biofo ... tamps.java

The problem is that the timestamps appear to be cut off. The results look like this:
Z 0, C 0, T 0 = 0.0 s
Z 0, C 0, T 1 = 0.0 s
Z 0, C 0, T 2 = 0.0 s
Z 0, C 0, T 3 = 0.0 s
Z 0, C 0, T 4 = 0.0 s
Z 0, C 0, T 5 = 0.0 s
Z 0, C 0, T 6 = 1.0 s
Z 0, C 0, T 7 = 1.0 s
Z 0, C 0, T 8 = 1.0 s
Z 0, C 0, T 9 = 1.0 s
etc.

When using proprietary Zeiss Software a higher time resolution is revealed (millisec. range)
Is there a way to extract the exact time from the metadata in zvi?
Thanks in advance!

chrub

[rleigh]

Hi,

There's a few possibilities here:

Until the last week, we didn't parse millisecond timestamps into the OME data model. It's possible that it's being reduced to second precision. However, this part of the output is using the deltaT from the start of acquisition which is double-precision floating point, so I'm fairly sure that the millisecond timestamp support would not affect this unless it's being converted to/from a Timestamp object internally.

Looking at the Zeiss reader, I think the problem is probably in the use of second precision in parseTimestamp():
https://github.com/openmicroscopy/bioformats/blob/develop/components/bio-formats/src/loci/formats/in/BaseZeissReader.java#L1012

If you could possibly upload a sample image using http://qa.openmicroscopy.org.uk/qa/upload/ then I can take a closer look at exactly what is being stored, and make sure that we are reading it correctly. Note that it would be helpful to have the exact date/time/timezone for the start of acquisition in addition to the file itself.

Note that a second problem with ZVI is the use of a very odd time base in their internal timestamps. http://trac.openmicroscopy.org.uk/ome/ticket/9660 It's not seconds, nor is it ms, ns, ps or some other whole decimal fraction of a second. And just to make things more annoying, it also varies over time(!) I've not yet determined accurately all the variants due to it being such high resolution I'm running into lots of floating point rounding issues. Unfortunately, I'm also limited to what I can observe based upon what I can see in AxioVision, which may also be imposing its own rounding error on its output (which is second precision only). I need to repeat it with a multi-precision maths library, and then it might make more sense. This basically means that there is currently a drift in ZVI timestamps due to the current timebase being slightly out, but which we will hopefully resolve soon as well. This shouldn't have any significant effect upon the precision of the relative inter-plane timings within your file, but it may mean that the accuracy of the acquisition dates may off by a bit.


Regards,
Roger

[chrub]

Hi,

I finally figured out how to read the time constants accurately.
I am referring to http://trac.openmicroscopy.org.uk/ome/ticket/9660
The data is not stored as a 64 bit long but rather is a 64 bit double.
The resulting number represents a TimeStamp since 1900, measured in DAYS.

This format can also be found in Excel documents so Apache POI gives you direct conversion functions:

Code: Select all

      
double dat = stream.readDouble();
Date a = org.apache.poi.ss.usermodel.DateUtil.getJavaDate(dat);
SimpleDateFormat sdf = new SimpleDateFormat();
sdf.applyPattern( "dd. MMMM yyyy hh:mm:ss:SS z" );
return sdf.format(a);


Do you see a possibility to get this included (with millisecond resolution) in the OME/loci ?
If you have any questions I would be happy to answer.

Also (for verification) I have written a very small demo for opening a .zvi and extracting the metainformation (especially the TimeStamps) myself using ApachePOI. If you are interested let me know.

Best regards,
chrub

[rleigh]

Hi,

Many thanks for looking into it. I had worked out that it was a double of the days since 1900 but didn't realise that it was what Excel used or that Apache POI provided a helper function to do the conversion, which is very handy--my conversion constants are currently off by 5 hours which was the main blocker for adding it. We've recently replaced (as a separate work) all Date/Calendar usage with joda DateTime to give us ms precision ISO8601 date support, so I'll probably write a conversion separately to do it in terms of DateTime rather than Date/Calendar. The POI helper is only a few lines of code, so looks quite straightforward to reimplement.

I've done the rest of the conversion to millisecond precision already, just needed a few tweaks to the date handling, so should be done fairly soon.

It would be very helpful to have a sample test file for the ms precision so that we can make sure that there are no regressions in the ms precision support in future releases. If you could possibly upload a sample file with http://qa.openmicroscopy.org.uk/qa/upload/ that would be very much appreciated.


Regards,
Roger Leigh

[arnon]

Dear Mr Leich,

I recently encountered the same problem that Chrub had. What is the current status if this issue?
Currently, I'm reading my zvi files using Matlab and the bfopen.m procedure. Then I read the metadata hash table stored at ans{1,2}. The time-stamps I get are in the same format as Chrub mentioned and I'm also interested in the millisec. field. Can I solve the problem somehow using Matlab only ?

Best regards,
Arnon

[rleigh]

Hi,

Regarding the timestamp precision, we will support millisecond precision in the 5.0.0 beta2 release, which will be released soon, and all subsequent 5.0.x releases. However, this is not supported in the 4.4.9 and earlier 4.4.x releases, and will support will not be added in any future 4.4 releases I'm afraid (it's too invasive to add to the stable release). The millisecond precision support is for timestamps in general, and will be used for all file formats, including ZVI.

Regarding the ZVI precision, there is also a bug in the timestamp calculations which I am working on a fix for. This is mostly done, but not yet available in any release. It will hopefully also be ready for the 5.0.0 beta2 release (or if not, very soon after). Once this is done, you will have correct timestamps with ms precision for all ZVI files.

Regards,
Roger