Upload problem: Directory exists but is not registered.

[ehrenfeu]

Dear all,

we're having problems with uploading data on one of our user accounts. After hitting the "Import" button the upload process stops almost immediately and the attached error message is presented to the user (upon request). At first, an existing project and dataset were chosen as the target, but we re-tried with newly created ones, didn't work either.

Importing the same data with a different user account works flawlessly, so it's not a problem of the file(s) being uploaded.

I guess this is related to our recent upgrade of OMERO from 4.4.9 to 5.0.2, but I'm not 100% sure there.

Any thoughs?
~Niko

Code: Select all

omero.ResourceError
    serverStackTrace = "java.lang.RuntimeException: Directory exists but is not registered: CheckedPath(sautlo01_13/2014-06)
                        at ome.services.blitz.repo.PublicRepositoryI.assertFindDir(PublicRepositoryI.java:832)
                        at ome.services.blitz.repo.PublicRepositoryI.makeCheckedDirs(PublicRepositoryI.java:749)
                        at ome.services.blitz.repo.ManagedRepositoryI.makeCheckedDirs(ManagedRepositoryI.java:751)
                        at ome.services.blitz.repo.PublicRepositoryI.makeDir(PublicRepositoryI.java:715)
                        at ome.services.blitz.repo.RepositoryDaoImpl$13.doWork(RepositoryDaoImpl.java:701)
                        at sun.reflect.GeneratedMethodAccessor261.invoke(Unknown Source)
                        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                        at java.lang.reflect.Method.invoke(Method.java:601)
                        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
                        at ome.services.util.Executor$Impl$Interceptor.invoke(Executor.java:576)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        at ome.security.basic.EventHandler.invoke(EventHandler.java:154)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:111)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:241)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:116)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
                        at $Proxy69.doWork(Unknown Source)
                        at ome.services.util.Executor$Impl.execute(Executor.java:457)
                        at ome.services.blitz.repo.RepositoryDaoImpl.makeDirs(RepositoryDaoImpl.java:688)
                        at ome.services.blitz.repo.PublicRepositoryI.makeDir(PublicRepositoryI.java:690)
                        at ome.services.blitz.repo.ManagedRepositoryI.createTemplateDir(ManagedRepositoryI.java:562)
                        at ome.services.blitz.repo.ManagedRepositoryI.internalImport(ManagedRepositoryI.java:251)
                        at ome.services.blitz.repo.ManagedRepositoryI.importFileset(ManagedRepositoryI.java:218)
                        at omero.grid._ManagedRepositoryTie.importFileset(_ManagedRepositoryTie.java:73)
                        at omero.grid._ManagedRepositoryDisp.___importFileset(_ManagedRepositoryDisp.java:364)
                        at omero.grid._ManagedRepositoryDisp.__dispatch(_ManagedRepositoryDisp.java:516)
                        at IceInternal.Incoming.invoke(Incoming.java:159)
                        at Ice.ConnectionI.invokeAll(ConnectionI.java:2357)
                        at Ice.ConnectionI.dispatch(ConnectionI.java:1208)
                        at Ice.ConnectionI.message(ConnectionI.java:1163)
                        at IceInternal.ThreadPool.run(ThreadPool.java:302)
                        at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
                        at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:643)
                        at java.lang.Thread.run(Thread.java:722)
                        "
    serverExceptionClass = "java.lang.RuntimeException"
    message = "Directory exists but is not registered: CheckedPath(sautlo01_13/2014-06)"
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at java.lang.Class.newInstance(Class.java:374)
at IceInternal.BasicStream$DynamicUserExceptionFactory.createAndThrow(BasicStream.java:2142)
at IceInternal.BasicStream.throwException(BasicStream.java:1564)
at IceInternal.Outgoing.throwUserException(Outgoing.java:443)
at omero.grid._ManagedRepositoryDelM.importFileset(_ManagedRepositoryDelM.java:51)
at omero.grid.ManagedRepositoryPrxHelper.importFileset(ManagedRepositoryPrxHelper.java:81)
at omero.grid.ManagedRepositoryPrxHelper.importFileset(ManagedRepositoryPrxHelper.java:43)
at ome.formats.importer.ImportLibrary.createImport(ImportLibrary.java:351)
at org.openmicroscopy.shoola.env.data.OMEROGateway.importImageFile(OMEROGateway.java:6403)
at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.importCandidates(OmeroImageServiceImpl.java:232)
at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.importFile(OmeroImageServiceImpl.java:1348)
at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.importFile(ImagesImporter.java:77)
at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.access$000(ImagesImporter.java:53)
at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter$1.doCall(ImagesImporter.java:102)
at org.openmicroscopy.shoola.env.data.views.BatchCall.doStep(BatchCall.java:144)
at org.openmicroscopy.shoola.util.concur.tasks.CompositeTask.doStep(CompositeTask.java:226)
at org.openmicroscopy.shoola.env.data.views.CompositeBatchCall.doStep(CompositeBatchCall.java:126)
at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.exec(ExecCommand.java:165)
at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.run(ExecCommand.java:276)
at org.openmicroscopy.shoola.util.concur.tasks.AsyncProcessor$Runner.run(AsyncProcessor.java:91)
at java.lang.Thread.run(Thread.java:745)

at org.openmicroscopy.shoola.env.data.OMEROGateway.importImageFile(OMEROGateway.java:6448)
at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.importCandidates(OmeroImageServiceImpl.java:232)
at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.importFile(OmeroImageServiceImpl.java:1348)
at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.importFile(ImagesImporter.java:77)
at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.access$000(ImagesImporter.java:53)
at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter$1.doCall(ImagesImporter.java:102)
at org.openmicroscopy.shoola.env.data.views.BatchCall.doStep(BatchCall.java:144)
at org.openmicroscopy.shoola.util.concur.tasks.CompositeTask.doStep(CompositeTask.java:226)
at org.openmicroscopy.shoola.env.data.views.CompositeBatchCall.doStep(CompositeBatchCall.java:126)
at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.exec(ExecCommand.java:165)
at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.run(ExecCommand.java:276)
at org.openmicroscopy.shoola.util.concur.tasks.AsyncProcessor$Runner.run(AsyncProcessor.java:91)
at java.lang.Thread.run(Thread.java:745)
Caused by: omero.ResourceError
    serverStackTrace = "java.lang.RuntimeException: Directory exists but is not registered: CheckedPath(sautlo01_13/2014-06)
                        at ome.services.blitz.repo.PublicRepositoryI.assertFindDir(PublicRepositoryI.java:832)
                        at ome.services.blitz.repo.PublicRepositoryI.makeCheckedDirs(PublicRepositoryI.java:749)
                        at ome.services.blitz.repo.ManagedRepositoryI.makeCheckedDirs(ManagedRepositoryI.java:751)
                        at ome.services.blitz.repo.PublicRepositoryI.makeDir(PublicRepositoryI.java:715)
                        at ome.services.blitz.repo.RepositoryDaoImpl$13.doWork(RepositoryDaoImpl.java:701)
                        at sun.reflect.GeneratedMethodAccessor261.invoke(Unknown Source)
                        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                        at java.lang.reflect.Method.invoke(Method.java:601)
                        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
                        at ome.services.util.Executor$Impl$Interceptor.invoke(Executor.java:576)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        at ome.security.basic.EventHandler.invoke(EventHandler.java:154)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:111)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:241)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:116)
                        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
                        at $Proxy69.doWork(Unknown Source)
                        at ome.services.util.Executor$Impl.execute(Executor.java:457)
                        at ome.services.blitz.repo.RepositoryDaoImpl.makeDirs(RepositoryDaoImpl.java:688)
                        at ome.services.blitz.repo.PublicRepositoryI.makeDir(PublicRepositoryI.java:690)
                        at ome.services.blitz.repo.ManagedRepositoryI.createTemplateDir(ManagedRepositoryI.java:562)
                        at ome.services.blitz.repo.ManagedRepositoryI.internalImport(ManagedRepositoryI.java:251)
                        at ome.services.blitz.repo.ManagedRepositoryI.importFileset(ManagedRepositoryI.java:218)
                        at omero.grid._ManagedRepositoryTie.importFileset(_ManagedRepositoryTie.java:73)
                        at omero.grid._ManagedRepositoryDisp.___importFileset(_ManagedRepositoryDisp.java:364)
                        at omero.grid._ManagedRepositoryDisp.__dispatch(_ManagedRepositoryDisp.java:516)
                        at IceInternal.Incoming.invoke(Incoming.java:159)
                        at Ice.ConnectionI.invokeAll(ConnectionI.java:2357)
                        at Ice.ConnectionI.dispatch(ConnectionI.java:1208)
                        at Ice.ConnectionI.message(ConnectionI.java:1163)
                        at IceInternal.ThreadPool.run(ThreadPool.java:302)
                        at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
                        at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:643)
                        at java.lang.Thread.run(Thread.java:722)
                        "
    serverExceptionClass = "java.lang.RuntimeException"
    message = "Directory exists but is not registered: CheckedPath(sautlo01_13/2014-06)"
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at java.lang.Class.newInstance(Class.java:374)
at IceInternal.BasicStream$DynamicUserExceptionFactory.createAndThrow(BasicStream.java:2142)
at IceInternal.BasicStream.throwException(BasicStream.java:1564)
at IceInternal.Outgoing.throwUserException(Outgoing.java:443)
at omero.grid._ManagedRepositoryDelM.importFileset(_ManagedRepositoryDelM.java:51)
at omero.grid.ManagedRepositoryPrxHelper.importFileset(ManagedRepositoryPrxHelper.java:81)
at omero.grid.ManagedRepositoryPrxHelper.importFileset(ManagedRepositoryPrxHelper.java:43)
at ome.formats.importer.ImportLibrary.createImport(ImportLibrary.java:351)
at org.openmicroscopy.shoola.env.data.OMEROGateway.importImageFile(OMEROGateway.java:6403)
... 12 more


[jmoore]

Hi Niko,

we've seen one other case of this. Can you confirm that there is a directory under /OMERO/ManagedRepository of the form: "sautlo01_13/2014-06"? Does it have anything in it?

Cheers,
~Josh

[ehrenfeu]

Hi Josh,

it's there, but completely empty:

Code: Select all

$ find /export/omero/OMERO/ManagedRepository/sautlo01_13/
/export/omero/OMERO/ManagedRepository/sautlo01_13/
/export/omero/OMERO/ManagedRepository/sautlo01_13/2014-06
/export/omero/OMERO/ManagedRepository/sautlo01_13/2014-06/04

Cheers,
~Niko

[jmoore]

Hi Niko,

if the directory is empty, you should be able to just use `rmdir` to get rid of it and then subsequent imports should succeed. If you have any idea of what might have happened to leave the directory dangling, we'd be very interested. (In another case, for example, the server ran out of memory while attempting an import).

All the best,
~Josh

[ehrenfeu]

Hi Josh,

perfect, this solved the problem! As we had a misconfiguration after the upgrade (our own fault), the server was in fact running out of memory and this particular upload process of this user did crash back then. So I'd assume this was for the same reason than in that other case.

Thanks for your help & enjoy the weekend!
~Niko