Page 1 of 1

Olympus CellR files not importing

PostPosted: Mon Dec 17, 2018 2:40 pm
by ratamero
Hi all,

we've been having multiple issues with Olympus CellR files (.apl, .mtb, .obsep, etc). Specifically, collecting files to import a fileset seems to be problematic. Many filesets are imported partially (i.e. not all files are collected correctly) and, in a few cases, the import process freezes completely when collecting files. Our newest example of this issue has been stuck at that stage of the import for 3 days already for a ~18GB fileset, so I don't think it's simply a performance issue.

Of course, since our offender is a large fileset with 18GB of data and a specific folder structure, I'm not sure what would be the best way to send you an example. Let me know how to proceed!

best regards

Erick

Re: Olympus CellR files not importing

PostPosted: Tue Dec 18, 2018 9:58 am
by jmoore
I've sent FTP instructions separately.
~Josh

Re: Olympus CellR files not importing

PostPosted: Thu Dec 20, 2018 12:53 am
by mlinkert
Thank you for uploading datasets. I can confirm that the larger dataset seems to enter an infinite loop during initialization, and that the smaller dataset omits the *.obsep and Storage.ini files from the imported fileset. It looks like all of the TIFF files in the smaller dataset are collected correctly (so the pixel data at least should be correct), but if you are seeing something different let us know.

The infinite loop issue can be tracked here:

https://trello.com/c/VWs0uUp8/302-infin ... le-parsing

and the missing files issue can be tracked here:

https://trello.com/c/1OYcvpa7/303-compa ... r-datasets

Re: Olympus CellR files not importing

PostPosted: Thu Dec 20, 2018 1:31 pm
by ratamero
Yeah, currently the pixel data seems to be correctly imported from the TIFFs, but the file structure and naming inside OMERO were not particularly useful (i.e. all fileset files are imported under the same name, which makes finding specific data in OMERO impossible). The group generating these filesets would love to use OMERO, but this has stopped them from doing so in the past, so a fix would be much appreciated! Thanks a lot for taking the time to look into it!