Open Microscopy Environment

This issue applies to:
bfconvert
Version: 5.3.3
Build date: 1 February 2017
VCS revision: 5.3.3

When using bfconvert to convert .ND2 to .tiff (also .ome.tiff) there is a reproducible error when using the flags -series, -channel, -timepoint, and -z. The error is always "[Fatal Error] :1:72: Character reference "&#55663" is an invalid XML character." more details below. This error does not occur with bfconvert version 5.1.10. The ND2 files are created by a Nikon DS-Qi2 camera. The files are converted correctly (not strictly true, but that is a separate issue) when the flags are omitted.

This error is reproducible with any .nd2 produced on this camera (I have not tried other cameras), but I have made a blank .nd2 including multiple series, channels, timepoints, and z for testing purposes which I am happy to provide to anyone (it is 357 MB when compressed in .7z format or 1.54 GB in .nd2 format).

example of issue:
C:\Users\tbates3\Desktop\Temp\bftools-new> bfconvert C:\Users\tbates3\Desktop\Testing\supernd2.nd2 C:\Users\tbates3\Desktop\TiffOutput\test3.tiff
-This works and produces a multipage tiff file

C:\Users\tbates3\Desktop\Temp\bftools-new> bfconvert -debug -series 0 C:\Users\tbates3\Desktop\Testing\supernd2.nd2 C:\Users\tbates3\Desktop\TiffOutput\test3.tiff
No update needed
C:\Users\tbates3\Desktop\Testing\supernd2.nd2
Could not find loci.formats.in.SlideBook6Reader
java.lang.ClassNotFoundException: loci.formats.in.SlideBook6Reader
at java.net.URLClassLoader.findClass(Unknown Source) ~[na:1.8.0_121]
at java.lang.ClassLoader.loadClass(Unknown Source) ~[na:1.8.0_121]
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) ~[na:1.8.0
_121]
at java.lang.ClassLoader.loadClass(Unknown Source) ~[na:1.8.0_121]
at java.lang.Class.forName0(Native Method) ~[na:1.8.0_121]
at java.lang.Class.forName(Unknown Source) ~[na:1.8.0_121]
at loci.formats.ClassList.parseLine(ClassList.java:196) [bioformats_pack
age.jar:5.3.2]
at loci.formats.ClassList.parseFile(ClassList.java:258) [bioformats_pack
age.jar:5.3.2]
at loci.formats.ClassList.<init>(ClassList.java:138) [bioformats_package
.jar:5.3.2]
at loci.formats.ClassList.<init>(ClassList.java:122) [bioformats_package
.jar:5.3.2]
at loci.formats.ImageReader.getDefaultReaderClasses(ImageReader.java:79)
[bioformats_package.jar:5.3.2]
at loci.formats.ImageReader.<init>(ImageReader.java:116) [bioformats_pac
kage.jar:5.3.2]
at loci.formats.tools.ImageConverter.testConvert(ImageConverter.java:341
) [bioformats_package.jar:5.3.2]
at loci.formats.tools.ImageConverter.main(ImageConverter.java:880) [biof
ormats_package.jar:5.3.2]
processor is INTEL_64 os.arch is amd64
architecture is WINDOWS_64 os.name is windows 7
architecture is WINDOWS_64 os.name is windows 7
platform specific path is META-INF/lib/windows_64/
mappedLib is turbojpeg.dll
URL is jar:file:/C:/Users/tbates3/Desktop/Temp/bftools-new/bioformats_package.ja
r!/META-INF/lib/windows_64/turbojpeg.dll
URL path is file:/C:/Users/tbates3/Desktop/Temp/bftools-new/bioformats_package.j
ar!/META-INF/lib/windows_64/turbojpeg.dll
Extracting 'jar:file:/C:/Users/tbates3/Desktop/Temp/bftools-new/bioformats_packa
ge.jar!/META-INF/lib/windows_64/turbojpeg.dll' to 'C:\Users\tbates3\AppData\Loca
l\Temp\turbojpeg6688584493340154469.dll'
Loaded properties from: services.properties
Added interface interface loci.formats.services.POIService and implementation cl
ass loci.formats.services.POIServiceImpl
Added interface interface loci.formats.services.MDBService and implementation cl
ass loci.formats.services.MDBServiceImpl
Added interface interface loci.formats.services.JPEGTurboService and implementat
ion class loci.formats.services.JPEGTurboServiceImpl
Added interface interface loci.formats.services.JAIIIOService and implementation
class loci.formats.services.JAIIIOServiceImpl
Added interface interface loci.formats.services.WlzService and implementation cl
ass loci.formats.services.WlzServiceImpl
Added interface interface loci.formats.services.JHDFService and implementation c
lass loci.formats.services.JHDFServiceImpl
Added interface interface loci.formats.services.NetCDFService and implementation
class loci.formats.services.NetCDFServiceImpl
Added interface interface loci.formats.services.EXIFService and implementation c
lass loci.formats.services.EXIFServiceImpl
Added interface interface loci.formats.services.MetakitService and implementatio
n class loci.formats.services.MetakitServiceImpl
Added interface interface loci.formats.services.LuraWaveService and implementati
on class loci.formats.services.LuraWaveServiceImpl
Added interface interface loci.formats.services.OMEXMLService and implementation
class loci.formats.services.OMEXMLServiceImpl
Added interface interface loci.formats.services.JPEGXRService and implementation
class loci.formats.services.JPEGXRServiceImpl
Using mapped byte buffer? false
ND2Reader initializing C:\Users\tbates3\Desktop\Testing\supernd2.nd2
NativeND2Reader initializing C:\Users\tbates3\Desktop\Testing\supernd2.nd2
loci.formats.in.NativeND2Reader.initFile(C:\Users\tbates3\Desktop\Testing\supern
d2.nd2)
Searching for blocks
Parsing block 'ND2 FILE SIG' 0%
Parsing block 'ImageMetadat' 0%
Parsing block 'ImageCalibra' 0%
Parsing block 'ImageMetadat' 0%
Parsing block 'ImageCalibra' 0%
Parsing block 'ImageMetadat' 0%
Parsing block 'ImageAttribu' 0%
Parsing block 'ImageDataSeq' 0%
Parsing block 'ImageDataSeq' 1%
Parsing block 'ImageDataSeq' 2%
Parsing block 'ImageDataSeq' 3%
Parsing block 'ImageDataSeq' 5%
Parsing block 'ImageDataSeq' 6%
Parsing block 'ImageDataSeq' 7%
Parsing block 'ImageDataSeq' 8%
Parsing block 'ImageDataSeq' 10%
Parsing block 'ImageDataSeq' 11%
Parsing block 'ImageDataSeq' 12%
Parsing block 'ImageDataSeq' 13%
Parsing block 'ImageDataSeq' 15%
Parsing block 'ImageDataSeq' 16%
Parsing block 'ImageDataSeq' 17%
Parsing block 'ImageDataSeq' 18%
Parsing block 'ImageDataSeq' 20%
Parsing block 'ImageDataSeq' 21%
Parsing block 'ImageDataSeq' 22%
Parsing block 'ImageDataSeq' 23%
Parsing block 'ImageDataSeq' 25%
Parsing block 'ImageDataSeq' 26%
Parsing block 'ImageDataSeq' 27%
Parsing block 'ImageDataSeq' 28%
Parsing block 'ImageDataSeq' 30%
Parsing block 'ImageDataSeq' 31%
Parsing block 'ImageDataSeq' 32%
Parsing block 'ImageDataSeq' 33%
Parsing block 'ImageDataSeq' 35%
Parsing block 'ImageDataSeq' 36%
Parsing block 'ImageDataSeq' 37%
Parsing block 'ImageDataSeq' 38%
Parsing block 'ImageDataSeq' 40%
Parsing block 'ImageDataSeq' 41%
Parsing block 'ImageDataSeq' 42%
Parsing block 'ImageDataSeq' 43%
Parsing block 'ImageDataSeq' 45%
Parsing block 'ImageDataSeq' 46%
Parsing block 'ImageDataSeq' 47%
Parsing block 'ImageDataSeq' 48%
Parsing block 'ImageDataSeq' 49%
Parsing block 'ImageDataSeq' 51%
Parsing block 'ImageDataSeq' 52%
Parsing block 'ImageDataSeq' 53%
Parsing block 'ImageDataSeq' 54%
Parsing block 'ImageDataSeq' 56%
Parsing block 'ImageDataSeq' 57%
Parsing block 'ImageDataSeq' 58%
Parsing block 'ImageDataSeq' 59%
Parsing block 'ImageDataSeq' 61%
Parsing block 'ImageDataSeq' 62%
Parsing block 'ImageDataSeq' 63%
Parsing block 'ImageDataSeq' 64%
Parsing block 'ImageDataSeq' 66%
Parsing block 'ImageDataSeq' 67%
Parsing block 'ImageDataSeq' 68%
Parsing block 'ImageDataSeq' 69%
Parsing block 'ImageDataSeq' 71%
Parsing block 'ImageDataSeq' 72%
Parsing block 'ImageDataSeq' 73%
Parsing block 'ImageDataSeq' 74%
Parsing block 'ImageDataSeq' 76%
Parsing block 'ImageDataSeq' 77%
Parsing block 'ImageDataSeq' 78%
Parsing block 'ImageDataSeq' 79%
Parsing block 'ImageDataSeq' 81%
Parsing block 'ImageDataSeq' 82%
Parsing block 'ImageDataSeq' 83%
Parsing block 'ImageDataSeq' 84%
Parsing block 'ImageDataSeq' 86%
Parsing block 'ImageDataSeq' 87%
Parsing block 'ImageDataSeq' 88%
Parsing block 'ImageDataSeq' 89%
Parsing block 'ImageDataSeq' 91%
Parsing block 'ImageDataSeq' 92%
Parsing block 'ImageDataSeq' 93%
Parsing block 'ImageDataSeq' 94%
Parsing block 'ImageDataSeq' 96%
Parsing block 'ImageDataSeq' 97%
Parsing block 'ImageDataSeq' 98%
Parsing block 'ImageMetadat' 99%
Parsing block 'CustomData|R' 99%
Parsing block 'ImageEventsL' 99%
Parsing block 'ImageTextInf' 99%
Parsing block 'CustomDataVa' 99%
Parsing block 'CustomDataVa' 99%
Parsing block 'CustomDataVa' 99%
Parsing block 'CustomDataVa' 99%
Parsing block 'CustomDataVa' 99%
Parsing block 'CustomDataVa' 99%
Parsing block 'CustomData|C' 99%
Parsing block 'CustomData|P' 99%
Parsing block 'CustomData|P' 99%
Parsing block 'CustomData|X' 99%
Parsing block 'CustomData|Y' 99%
Parsing block 'CustomData|Z' 99%
Parsing block 'CustomData|T' 99%
Parsing block 'CustomData|A' 99%
Parsing block 'CustomData|A' 99%
Parsing block 'CustomData|A' 99%
Parsing block 'CustomDataVa' 99%
Parsing block 'ImageAttribu' 99%
Parsing block 'ND2 FILEMAP ' 99%
Could not parse XML
java.io.IOException: null
at loci.common.xml.XMLTools.parseXML(XMLTools.java:452) ~[bioformats_pac
kage.jar:5.3.2]
at loci.common.xml.XMLTools.parseXML(XMLTools.java:430) ~[bioformats_pac
kage.jar:5.3.2]
at loci.common.xml.XMLTools.parseXML(XMLTools.java:409) ~[bioformats_pac
kage.jar:5.3.2]
at loci.formats.in.NativeND2Reader.parseText(NativeND2Reader.java:2146)
[bioformats_package.jar:5.3.2]
at loci.formats.in.NativeND2Reader.initFile(NativeND2Reader.java:754) [b
ioformats_package.jar:5.3.2]
at loci.formats.FormatReader.setId(FormatReader.java:1399) [bioformats_p
ackage.jar:5.3.2]
at loci.formats.DelegateReader.setId(DelegateReader.java:291) [bioformat
s_package.jar:5.3.2]
at loci.formats.ImageReader.setId(ImageReader.java:839) [bioformats_pack
age.jar:5.3.2]
at loci.formats.tools.ImageConverter.testConvert(ImageConverter.java:380
) [bioformats_package.jar:5.3.2]
at loci.formats.tools.ImageConverter.main(ImageConverter.java:880) [biof
ormats_package.jar:5.3.2]
Caused by: org.xml.sax.SAXParseException: Content is not allowed in prolog.
at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Un
known Source) ~[bioformats_package.jar:5.3.2]
at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
~[bioformats_package.jar:5.3.2]
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) ~
[bioformats_package.jar:5.3.2]
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) ~
[bioformats_package.jar:5.3.2]
at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source) ~[
bioformats_package.jar:5.3.2]
at org.apache.xerces.impl.XMLDocumentScannerImpl$PrologDispatcher.dispat
ch(Unknown Source) ~[bioformats_package.jar:5.3.2]
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Un
known Source) ~[bioformats_package.jar:5.3.2]
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) ~[
bioformats_package.jar:5.3.2]
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) ~[
bioformats_package.jar:5.3.2]
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) ~[bioformat
s_package.jar:5.3.2]
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) ~[b
ioformats_package.jar:5.3.2]
at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Sour
ce) ~[bioformats_package.jar:5.3.2]
at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source) ~[bioforma
ts_package.jar:5.3.2]
at javax.xml.parsers.SAXParser.parse(Unknown Source) ~[na:1.8.0_121]
at loci.common.xml.XMLTools.parseXML(XMLTools.java:444) ~[bioformats_pac
kage.jar:5.3.2]
... 9 common frames omitted
[Fatal Error] :1:72: Character reference "&#55663" is an invalid XML character.
Exception in thread "main" java.lang.RuntimeException: org.xml.sax.SAXParseExcep
tion; lineNumber: 1; columnNumber: 72; Character reference "&#55663" is an inval
id XML character.
at ome.xml.model.XMLAnnotation.asXMLElement(XMLAnnotation.java:265)
at ome.xml.model.XMLAnnotation.asXMLElement(XMLAnnotation.java:234)
at ome.xml.model.StructuredAnnotations.asXMLElement(StructuredAnnotation
s.java:679)
at ome.xml.model.StructuredAnnotations.asXMLElement(StructuredAnnotation
s.java:660)
at ome.xml.model.OME.asXMLElement(OME.java:896)
at ome.xml.model.OME.asXMLElement(OME.java:773)
at ome.xml.meta.AbstractOMEXMLMetadata.dumpXML(AbstractOMEXMLMetadata.ja
va:110)
at ome.xml.meta.OMEXMLMetadataImpl.dumpXML(OMEXMLMetadataImpl.java:105)
at loci.formats.services.OMEXMLServiceImpl.getOMEXML(OMEXMLServiceImpl.j
ava:465)
at loci.formats.tools.ImageConverter.testConvert(ImageConverter.java:414
)
at loci.formats.tools.ImageConverter.main(ImageConverter.java:880)
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 72; Chara
cter reference "&#55663" is an invalid XML character.
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at ome.xml.model.XMLAnnotation.asXMLElement(XMLAnnotation.java:261)
... 10 more

Thank you,
Tim

Hi,

Thank you for reporting your issue.

We had recently fixed issues with our ImageConverter in this PR,
https://github.com/openmicroscopy/bioformats/pull/2737
The fix was included in the 5.3.3 release of Bio-Formats.

Testing with the files we have locally, we were unable to reproduce the issue that you have reported.

It would be great if you could submit one of your nd2 files to our QA server,
https://www.openmicroscopy.org/qa2/qa/upload/

This would help us troubleshoot the issue in better detail.

Best,
Balaji

I have uploaded a file called supernd2.zip which contains an nd2 with all of the available options in our software, NIS Elements AR (several of each timepoints, series, channels, z-series), but all images are of the inside of the camera shutter. This file gives the same results as my files containing actual data.

I also verified that my java is up to date with Version 8 build 121. My OS is Windows 7 Enterprise.

Cheers,
Tim

Hi,

Thank you for submitting your files.
We were able to reproduce the issue now with Bio-Foramts 5.3.3.

We will be filing a ticket on this front, please let us know if you would like to get added to the cc list on the ticket.

Best,
Balaji

Hello,

Just to let you know, as of 5.4.0 the error is now occurring every time regardless of the use of flags. 5.1.10 still works correctly though in all cases tested by me, so I will just keep using that for the time being.

Cheers,
Tim

Hi Tim,

Thanks for providing an update. I have added those details to the ticket (https://trac.openmicroscopy.org/ome/ticket/13322) and will increase the priority for further investigation.

David Gault