Partial import of data from InCell microscope

[wgeorgescu]

Hi,
I have been trying to import data from an InCell 6000 experiment into our OMERO 5.0.2 server using the command line. However, after the import completes with zero errors it has only imported 5 of 96 wells. Specifically it imports only wells A01-A05 even though this is a plate where every well has been imaged (16 fields per well). The command I used was ./omero import -s localhost -u user1 '/data/Experiments/140610/InCell6000/140610-4_iC2/20x Z1 16fields Gr Rd DAPI Cy5_140610-3_fix_2.xdce'. Do you have any suggestions as to how I might get all the data imported? You can download the xcde file from this link: https://app.box.com/s/mijymdgn4aratw9d1api
Thanks.

[rleigh]

Thanks for making the file available. It looks like the InCell reader isn't behaving correctly with this file. The reader is correctly picking up the wells, but it's not filling them with WellSamples (fields), so they are as a result appearing empty. There's nothing added after A5.

I have opened a ticket for this and CCd you on it. http://trac.openmicroscopy.org.uk/ome/ticket/12416


Kind regards,
Roger Leigh

[wgeorgescu]

Sounds good, thanks!

[wgeorgescu]

After investigating some more, it seems that the main problem is that the importer fails to recognize the dimension of the data set. For this set, we only acquired one Z slice per image but it was imported into OMERO as images with 21 Z-stacks. From field 1 to 9 it imports these images into wells A01-A05 and from field 10 to 16 it imports them into wells A01-A04. My guess now is that it imports all but three of the images but it gets the dimensionality wrong ( (9*5+7*4)*21=1533 images instead of 96*16*1=1536 images).

[mlinkert]

Thank you for the additional information - as you have likely seen, that's been added to the ticket. We'll update the ticket as soon as a fix is available.

[dsudar]

Hi Melissa,

I'm working with Walter on this and we found a couple very specific inconsistencies and outright errors in the GE INCell XML file that cause the problem. If it would be possible for the InCellReader code to take these 2 issues into account, then a lot more data from the InCell 6000 could be properly imported.

1) Even when the dataset is 2D (i.e. no Z slices), the XML file will still have a line in it saying and note that it actually says "enabled="true"" erroneously:

<ZDimensionParameters enabled="true" linked="true" number_of_slices="21" starting_position="0.0" step="1.0"/>

However ONLY if the "imaging_mode= "x-D" in the Wavelength section says "3-D", then the ZDimension... line should be considered.
Here's an example where it's only 2-D data and thus ZDimension... is not valid:

<Wavelength aperture_rows="50" fusion_wave="false" imaging_mode="2-D" index="0" label="" laser_power="100" open_aperture="true" show="false" z_section="0.0" z_slice="1" z_step="0.0">
<ExcitationFilter name="Blue" unit="nm" wavelength="488"/>
<EmissionFilter name="FITC" unit="nm" wavelength="525"/>
<Exposure time="500.0" unit="ms"/>
<HWAFOffset unit="µm" value="0.0"/>
<FocusOffset unit="µm" value="3.0"/>
<FusionData/>
</Wavelength>
<Wavelength aperture_rows="50" fusion_wave="false" imaging_mode="2-D" index="1" label="" laser_power="100" open_aperture="true" show="false" z_section="1.0" z_slice="0" z_step="0.0">
<ExcitationFilter name="Green" unit="nm" wavelength="561"/>
<EmissionFilter name="dsRed" unit="nm" wavelength="605"/>
<Exposure time="500.0" unit="ms"/>
<HWAFOffset unit="µm" value="0.0"/>
<FocusOffset unit="µm" value="3.0"/>
<FusionData/>
</Wavelength>
</Wavelengths>

2) Similarly, the section describing "TimeSchedule" is frequently in the file even if it is NOT a time series. Currently the BioFormats importer will consider it a time series if this section (example below) is in the file even if the "enabled" parameter says "false". So this bug could simply be squashed by following the "enabled" parameter.

<TimeSchedule displayHms="true" enabled="false" incubate_between_time_points="false" mode="look walk look" time_unit="sec">
<Waves>
<Wave emission_filter="dsRed" excitation_filter="Green" id="2" optical_z_sectioning="0"/>
<Wave emission_filter="FITC" excitation_filter="Blue" id="1" optical_z_sectioning="0"/>
</Waves>
<Times>
<TimePoint index="0" ms="0" selected="true">
<Events>
<AcqWave id="1"/>
<AcqWave id="2"/>
</Events>
</TimePoint>
<TimePoint index="1" ms="1200000" selected="true">
<Events>
<AcqWave id="1"/>
<AcqWave id="2"/>
</Events>
</TimePoint>
<TimePoint index="2" ms="2400000" selected="true">
<Events>
<AcqWave id="1"/>
<AcqWave id="2"/>
</Events>
</TimePoint>
.................etc ............

I manually edited the XML files and executed the import and the data was imported correctly using the current InCellReader. So at least for these 2 errors/inconsistencies in the InCell files as they come off the InCell6000, the fixes I suggested would resolve the issues I encountered.

Thanks,
- Damir

[mlinkert]

Hi Damir,

Thank you for investigating further - that's very helpful. I don't think we've seen any other data from InCell 6000 systems before, so perhaps this is changed behavior from the 2000/3000 systems.

In any case, I think we have more than enough information to fix this fairly quickly, so I've scheduled the ticket to be fixed for the upcoming 5.0.3 release.

Regards,
-Melissa

[dsudar]

Hi Melissa,

Thanks for taking this on for 5.0.3. While the full datasets are very large and not easily transferred to you, I do have a few example .xdce files that I can upload to the QA system (or other more convenient location). I have the following:
1) 96-wells, 2-channel, 16 fields/well, 2D images, single timepoint
2) 96-wells, 4-channel, 16 fields/well, 2D images, single timepoint
3) 96-wells, 2-channel, 12 fields/well, 3D images with 3 slices, single timepoint
4) 96-wells, 1-channel, 12 fields/well, 2D images, 20 timepoints
and finally a weird one that I haven't tried to import yet (and probably unlikely to be successful):
5) 96-wells, 4-channel, 16 fields/well, 2D images for 3 channels and a Z-stack of 5 slices for one of the channels

Let me know if you'd like them or some of them.
Cheers,
- Damir

[mlinkert]

Hi Damir,

We now have the fix under review here: https://github.com/openmicroscopy/bioformats/pull/1212

1) 96-wells, 2-channel, 16 fields/well, 2D images, single timepoint
2) 96-wells, 4-channel, 16 fields/well, 2D images, single timepoint
3) 96-wells, 2-channel, 12 fields/well, 3D images with 3 slices, single timepoint
4) 96-wells, 1-channel, 12 fields/well, 2D images, 20 timepoints
and finally a weird one that I haven't tried to import yet (and probably unlikely to be successful):
5) 96-wells, 4-channel, 16 fields/well, 2D images for 3 channels and a Z-stack of 5 slices for one of the channels

I'd expect (1) - (4) to just work with the above changes; (5) concerns me a bit though in that there is generally the assumption that all channels will have the same number of Z sections and timepoints. I suspect that with the above fix, you would get 4C x 5Z for every field, with 4 of the Z sections being all black for 3 of the channels, which is about the best we can do. If you import it and discover that it's doing something else, though, then we can certainly take a look.

-Melissa