DropBox fails to import files with UTF8 symbols such as µ

[Sethur]

Hi,

apparently, DropBox with OMERO 5.0.5 has some issues importing files containing UTF-8 non ASCII chars such as µ. I only found this because one of our users was complaining that certain files did not appear in his "Orphaned Images" folder.

I did not yet try wether those files also cause issues during a manual import. Will this be fixed in the next release?

Cheers,

Tristan

[manics]

Hi Tristan

DropBox should work with unicode characters. Would you mind trying a manual import?

Cheers, Simon

[cblackburn]

Hi Tristan,

as Simon says there should not be a problem here. If after further investigation you do find some problem then either the logs from a CLI import or, if there is a problem specific to DropBox, the Monitor.log and DropBox.log files that cover a failed import would be helpful.

Cheers,

Colin

[Sethur]

Hi Colin & Simon,

sorry I had little time to dive into this in the last few days. I did another test DropBox upload of a file containing a UTF-8 "µ" though and it still did not work. Files without any UTF8 special chars work without issues. The relevant sections in the log files are:

MonitorServer.log:

Code: Select all

2015-02-16 23:42:52,953 INFO  [             fsserver.fsPyinotifyMonitor] (Thread-4  ) Modified file event of type IN_MODIFY at: /srv/omero_data/DropBox/tnowak/random-long-test-name-100µl-with-special-chars.lif
2015-02-16 23:42:56,505 INFO  [             fsserver.fsPyinotifyMonitor] (Thread-4  ) Modified file event of type IN_CLOSE_WRITE at: /srv/omero_data/DropBox/tnowak/random-long-test-name-100µl-with-special-chars.lif
2015-02-16 23:42:56,506 INFO  [             fsserver.fsPyinotifyMonitor] (Thread-4  ) Event notification : [('/srv/omero_data/DropBox/tnowak/random-long-test-name-100\xc2\xb5l-with-special-chars.lif', Create)]
2015-02-16 23:42:56,546 INFO  [        fsclient.fsNotificationScheduler] (Thread-5  ) Notification queue size = 1
2015-02-16 23:42:56,547 INFO  [        fsclient.fsNotificationScheduler] (Thread-5  ) Notification queue 1 items removed.
2015-02-16 23:42:56,547 INFO  [        fsclient.fsNotificationScheduler] (Thread-5  ) Notification queue size = 0
2015-02-16 23:42:56,547 INFO  [                fsserver.fsMonitorServer] (Thread-5  ) Event notification on monitor id= 17111dea-b1fb-11e4-bad8-001e6752dcf8


DropBox.log

Code: Select all

2015-02-16 23:42:56,549 INFO  [                            omero.remote] (Dummy-157 )  Meth: MonitorClientI.fsEventHappened
2015-02-16 23:42:56,549 INFO  [         fsclient.fsDropBoxMonitorClient] (Dummy-157 ) EVENT_RECORD::Cookie::1424126576.55::Batch::1
2015-02-16 23:42:56,549 INFO  [         fsclient.fsDropBoxMonitorClient] (Dummy-157 ) EVENT_RECORD::Cookie::1424126576.55::Create::/srv/omero_data/DropBox/tnowak/random-long-test-name-100µl-with-special-chars.lif
2015-02-16 23:42:56,566 INFO  [                            omero.remote] (Dummy-157 )  Rslt: None
2015-02-16 23:43:13,542 INFO  [         fsclient.fsDropBoxMonitorClient] (Thread-3  ) Processing 1 events (1 ids). 0 remaining
2015-02-16 23:43:13,543 INFO  [         fsclient.fsDropBoxMonitorClient] (Thread-3  ) Getting filesets on : set(['/srv/omero_data/DropBox/tnowak/random-long-test-name-100\xc2\xb5l-with-special-chars.lif'])
2015-02-16 23:43:14,673 INFO  [         fsclient.fsDropBoxMonitorClient] (Thread-3  ) EVENT_RECORD::Cookie::1424126594.67::Filesets::{}


The problem seems to be that the upload is not commenced after "Getting filesets...".

Can you reproduce this on your side?

Cheers,

Tristan

EDIT: I waited for 10 more minutes after the last message in DropBox.log. There is no further output.

[manics]

Hi Tristan

Did you also try a manual import for comparison? On the server run

Code: Select all

OMERO.server/bin/omero import random-long-test-name-100µl-with-special-chars.lif

and if it fails paste the output.

Could you also paste the output of locale when run as the omero system user? For example, as root:

Code: Select all

su - omero -c locale

Thanks, Simon

[Sethur]

Hi Simon,

as always, thanks for the quick support. It turns out that after updating from 5.0.6 (we used this, 5.0.5 was a typo) to 5.0.8, the DropBox upload seems to work fine when using specials chars.

Best,

Tristan