We're Hiring!

Open Microscopy Environment

The OME community

Skip to content

In-place import via DropBox using --transfer=ln fails

General user discussion about using the OMERO platform to its fullest. Please ask new questions at https://forum.image.sc/tags/omero
Please note:
Historical discussions about OMERO. Please look for and ask new questions at https://forum.image.sc/tags/omero

There are workflow guides for various OMERO functions on our help site - http://help.openmicroscopy.org

You should find answers to any basic questions about using the clients there.
Post a reply

In-place import via DropBox using --transfer=ln fails

Postby Sethur » Tue Jun 17, 2014 3:13 pm

Hi,

while using our DropBox with the new in-place import feature:

Code: Select all
omero.fs.importArgs=--transfer=ln


i.e. transfer via hard links, DropBox imports are failing with the error message below. Unfortunately, I was unable to deduce from this where the problem lies. Both the DropBox and the data storage are on the same physical partition, so there should be no issue there. We are using OMERO 5.0.2

Ideas to solve this would be greatly appreciated.

Cheers,

Tristan

Code: Select all
2014-06-17 16:24:04,812 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) ***** start of output from importer-cli to stderr *****
2014-06-17 16:24:04,812 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:01,492 307        [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.0.2-ice34-b26
2014-06-17 16:24:04,812 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:01,512 327        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.0.2-ice34-b26 revision: 0c4215a date: 27 May 2014
2014-06-17 16:24:04,812 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:01,520 335        [      main] INFO   formats.importer.cli.CommandLineImporter - Setting transfer to ln
2014-06-17 16:24:04,812 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:01,525 340        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2014-06-17 16:24:04,813 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:01,874 689        [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2014-06-17 16:24:04,813 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:02,223 1038       [      main] WARN   odel.enums.handlers.ImmersionEnumHandler - Unknown Immersion value 'IMM' will be stored as "Other"
2014-06-17 16:24:04,813 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:02,228 1043       [      main] WARN   odel.enums.handlers.ImmersionEnumHandler - Unknown Immersion value 'IMM' will be stored as "Other"
2014-06-17 16:24:04,813 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:02,230 1045       [      main] WARN   odel.enums.handlers.ImmersionEnumHandler - Unknown Immersion value 'IMM' will be stored as "Other"
2014-06-17 16:24:04,813 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:02,232 1047       [      main] WARN   odel.enums.handlers.ImmersionEnumHandler - Unknown Immersion value 'IMM' will be stored as "Other"
2014-06-17 16:24:04,813 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:02,234 1049       [      main] WARN   odel.enums.handlers.ImmersionEnumHandler - Unknown Immersion value 'null' will be stored as "Other"
2014-06-17 16:24:04,814 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:02,234 1049       [      main] WARN   del.enums.handlers.CorrectionEnumHandler - Unknown Correction value 'null' will be stored as "Other"
2014-06-17 16:24:04,814 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:02,250 1065       [      main] INFO      ome.formats.importer.ImportCandidates - 1 file(s) parsed into 1 group(s) with 1 call(s) to setId in 372ms. (376ms total) [0 unknowns]
2014-06-17 16:24:04,814 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:02,772 1587       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Attempting initial SSL connection to localhost:4064
2014-06-17 16:24:04,814 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:03,446 2261       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Insecure connection requested, falling back
2014-06-17 16:24:04,814 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:03,882 2697       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Server: 5.0.2
2014-06-17 16:24:04,814 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:03,882 2697       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Client: 5.0.2-ice34-b26
2014-06-17 16:24:04,814 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:03,882 2697       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Java Version: 1.7.0_55
2014-06-17 16:24:04,815 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:03,882 2697       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Name: Linux
2014-06-17 16:24:04,815 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:03,882 2697       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Arch: amd64
2014-06-17 16:24:04,815 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:03,882 2697       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Version: 3.11.0-23-generic
2014-06-17 16:24:04,815 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:04,058 2873       [      main] INFO   ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_PREPARATION
2014-06-17 16:24:04,815 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:04,553 3368       [      main] INFO   ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_START
2014-06-17 16:24:04,815 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:04,574 3389       [      main] INFO   .importer.transfers.HardlinkFileTransfer - Transferring /srv/omero_data/DropBox/tnowak/Worm_4x1-Tiled_Pinhole_100um_Tristan.lif...
2014-06-17 16:24:04,815 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:04,684 3499       [      main] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /srv/omero_data/DropBox/tnowak/Worm_4x1-Tiled_Pinhole_100um_Tristan.lif
2014-06-17 16:24:04,816 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:04,698 3513       [      main] ERROR  .importer.transfers.HardlinkFileTransfer - transfer process returned 1
2014-06-17 16:24:04,816 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:04,739 3554       [      main] ERROR  .importer.transfers.HardlinkFileTransfer - error in closing raw file store
2014-06-17 16:24:04,816 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) omero.ResourceError: null
2014-06-17 16:24:04,816 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.7.0_55]
2014-06-17 16:24:04,816 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) ~[na:1.7.0_55]
2014-06-17 16:24:04,816 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.7.0_55]
2014-06-17 16:24:04,816 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) ~[na:1.7.0_55]
2014-06-17 16:24:04,817 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at java.lang.Class.newInstance(Class.java:374) ~[na:1.7.0_55]
2014-06-17 16:24:04,817 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at IceInternal.BasicStream$DynamicUserExceptionFactory.createAndThrow(BasicStream.java:2142) ~[ice.jar:na]
2014-06-17 16:24:04,817 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at IceInternal.BasicStream.throwException(BasicStream.java:1564) ~[ice.jar:na]
2014-06-17 16:24:04,817 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at IceInternal.Outgoing.throwUserException(Outgoing.java:443) ~[ice.jar:na]
2014-06-17 16:24:04,817 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at omero.api._RawFileStoreDelM.close(_RawFileStoreDelM.java:462) ~[blitz.jar:na]
2014-06-17 16:24:04,817 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at omero.api.RawFileStorePrxHelper.close(RawFileStorePrxHelper.java:1580) ~[blitz.jar:na]
2014-06-17 16:24:04,818 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at omero.api.RawFileStorePrxHelper.close(RawFileStorePrxHelper.java:1543) ~[blitz.jar:na]
2014-06-17 16:24:04,818 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.transfers.AbstractFileTransfer.cleanupUpload(AbstractFileTransfer.java:125) ~[blitz.jar:na]
2014-06-17 16:24:04,818 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.transfers.AbstractExecFileTransfer.transfer(AbstractExecFileTransfer.java:63) [blitz.jar:na]
2014-06-17 16:24:04,818 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.ImportLibrary.uploadFile(ImportLibrary.java:414) [blitz.jar:na]
2014-06-17 16:24:04,818 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:477) [blitz.jar:na]
2014-06-17 16:24:04,818 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.ImportLibrary.importCandidates(ImportLibrary.java:271) [blitz.jar:na]
2014-06-17 16:24:04,818 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.cli.CommandLineImporter.start(CommandLineImporter.java:216) [blitz.jar:na]
2014-06-17 16:24:04,818 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.cli.CommandLineImporter.main(CommandLineImporter.java:721) [blitz.jar:na]
2014-06-17 16:24:04,819 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:04,740 3555       [      main] ERROR     ome.formats.importer.cli.ErrorHandler - FILE_EXCEPTION: /srv/omero_data/DropBox/tnowak/Worm_4x1-Tiled_Pinhole_100um_Tristan.lif
2014-06-17 16:24:04,819 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) java.lang.RuntimeException: transfer process returned 1
2014-06-17 16:24:04,819 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.transfers.AbstractExecFileTransfer.exec(AbstractExecFileTransfer.java:137) ~[blitz.jar:na]
2014-06-17 16:24:04,819 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.transfers.AbstractExecFileTransfer.transfer(AbstractExecFileTransfer.java:57) ~[blitz.jar:na]
2014-06-17 16:24:04,819 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.ImportLibrary.uploadFile(ImportLibrary.java:414) [blitz.jar:na]
2014-06-17 16:24:04,819 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:477) [blitz.jar:na]
2014-06-17 16:24:04,819 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.ImportLibrary.importCandidates(ImportLibrary.java:271) [blitz.jar:na]
2014-06-17 16:24:04,820 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.cli.CommandLineImporter.start(CommandLineImporter.java:216) [blitz.jar:na]
2014-06-17 16:24:04,820 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.cli.CommandLineImporter.main(CommandLineImporter.java:721) [blitz.jar:na]
2014-06-17 16:24:04,820 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:04,742 3557       [      main] ERROR        ome.formats.importer.ImportLibrary - Error on import
2014-06-17 16:24:04,820 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) java.lang.RuntimeException: transfer process returned 1
2014-06-17 16:24:04,820 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.transfers.AbstractExecFileTransfer.exec(AbstractExecFileTransfer.java:137) ~[blitz.jar:na]
2014-06-17 16:24:04,820 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.transfers.AbstractExecFileTransfer.transfer(AbstractExecFileTransfer.java:57) ~[blitz.jar:na]
2014-06-17 16:24:04,820 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.ImportLibrary.uploadFile(ImportLibrary.java:414) ~[blitz.jar:na]
2014-06-17 16:24:04,821 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:477) ~[blitz.jar:na]
2014-06-17 16:24:04,821 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.ImportLibrary.importCandidates(ImportLibrary.java:271) ~[blitz.jar:na]
2014-06-17 16:24:04,821 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.cli.CommandLineImporter.start(CommandLineImporter.java:216) [blitz.jar:na]
2014-06-17 16:24:04,821 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) at ome.formats.importer.cli.CommandLineImporter.main(CommandLineImporter.java:721) [blitz.jar:na]
2014-06-17 16:24:04,821 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) 2014-06-17 16:24:04,742 3557       [      main] INFO         ome.formats.importer.ImportLibrary - Exiting on error
2014-06-17 16:24:04,821 ERROR [         fsclient.fsDropBoxMonitorClient] (Thread-6  ) ***** end of output from importer-cli *****
2014-06-17 16:24:04,832 INFO  [                    omero.util.Resources] (Thread-8  ) Halted
Sethur
 
Posts: 112
Joined: Thu Jan 16, 2014 11:34 pm
Top

Re: In-place import via DropBox using --transfer=ln fails

Postby jmoore » Tue Jun 17, 2014 5:39 pm

Hi Tristan,

my apologies. A fix for the "not enough info on error" probably was included in https://github.com/openmicroscopy/openmicroscopy/pull/2242 but it didn't get included in 5.0.2.

I re-opened that patch as https://github.com/openmicroscopy/openmicroscopy/pull/2648 and can provide you a build or you can compile it yourself. It should only be needed client-side (i.e. a new copy of lib/client/blitz.jar) and should let us know more about what's going wrong.

Cheers,
~Josh.
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany
Top

Re: In-place import via DropBox using --transfer=ln fails

Postby Sethur » Fri Jun 20, 2014 9:20 am

Hi Josh,

a build would be very helpful since I did not setup my own compiler environment yet for OME.

Cheers,

Tristan
Sethur
 
Posts: 112
Joined: Thu Jan 16, 2014 11:34 pm
Top

Re: In-place import via DropBox using --transfer=ln fails

Postby sbesson » Fri Jun 20, 2014 2:46 pm

Hi Tristan,

the latest 5.0 build should now include Josh's PR mentioned previously in this thread. Select the Ice version you desire, then under Last Successful Artifacts, expand View, src and target and click on the artifact you need.
Please note these are continuous integration artifacts, have not been validated through the release QA process and may contain other devlopment bugs.

Let us know if this solves your issue,
Sebastien
User avatar
sbesson
Team Member
 
Posts: 421
Joined: Tue Feb 28, 2012 7:20 pm
Top

Re: In-place import via DropBox using --transfer=ln fails

Postby Sethur » Tue Jun 24, 2014 11:18 am

Hi Sebastien,

is this version safe to deploy? We currently only have a single production environment and no second server available for testing purposes.

Cheers,

Tristan
Sethur
 
Posts: 112
Joined: Thu Jan 16, 2014 11:34 pm
Top

Re: In-place import via DropBox using --transfer=ln fails

Postby sbesson » Wed Jun 25, 2014 1:38 pm

Hi Tristan,

the build linked previously is the result of our continuous integration and should not be considered as production quality. I would not recommend upgrading your production server with non-release builds. If you need to upgrade your server, I would wait for the 5.0.3 release which is due by the end of July.

Alternatively, to get more information about your error, you could try to download the OMERO.server from the link I pasted previously and use it as a client (i.e. use the CLI) to re-import the faulty file without upgrading your production server. The log of the client should now include Josh's PR and provide you with more information about the nature of the error if this is reproducible.

Best,
Sebastien
User avatar
sbesson
Team Member
 
Posts: 421
Joined: Tue Feb 28, 2012 7:20 pm
Top

Re: In-place import via DropBox using --transfer=ln fails

Postby Sethur » Mon Jul 21, 2014 8:16 am

Hi again,

I just wanted to point out that I have resolved the problem. It turned out to be insufficient file permissions after all.

Thanks for your support,

Tristan
Sethur
 
Posts: 112
Joined: Thu Jan 16, 2014 11:34 pm
Top
Post a reply

Return to User Discussion

Who is online

Users browsing this forum: Google [Bot] and 0 guests

Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group