OMERO Bulk import

[alexr]

Dear omero users,

I am currently trying to setup an automated inplace import from a network share into omero server.
As a first step I try to implement the inplace import using the "bulk.yml" method.

The test scenario is like this:

omero-server:
mounted file-server in: /media/CCHL-User
Omero-ManagedRepository in /OMERO/ManagedRepository
Omero-binaries in /home/omerouser/OMERO.server/bin/omero

the following users are installed on the omero-server:
omero = admin
omerouser = account that runs omero server
inplace = inplace import user, with read/write access to ManagedRepository

Then I generated a bulk.yml file to import my test images. It looks like this:

Code: Select all

---
continue: "true"
transfer: "ln_s"
exclude: "clientpath"
checksum_algorithm: "File-Size-64"
logprefix: "logs/"
output: "yaml"
path: "toImport.tsv"
columns:
    - target
    - path


Together with that I made the file "toImport.tvs" that looks like this:

Code: Select all

Dataset:name:DermalStem   /media/CCHL-User/Alex/03-Microscopy/2018/2018-06-08-DermalStemCells_dissoiationTest/digested+filtered01.zvi
Dataset:name:DermalStem   /media/CCHL-User/Alex/03-Microscopy/2018/2018-06-08-DermalStemCells_dissoiationTest/digested+filtered02.zvi
Dataset:name:DermalStem   /media/CCHL-User/Alex/03-Microscopy/2018/2018-06-08-DermalStemCells_dissoiationTest/digested+filtered03.zvi
Dataset:name:DermalStem   /media/CCHL-User/Alex/03-Microscopy/2018/2018-06-08-DermalStemCells_dissoiationTest/digested02.zvi
Dataset:name:DermalStem   /media/CCHL-User/Alex/03-Microscopy/2018/2018-06-08-DermalStemCells_dissoiationTest/digested03.zvi
Dataset:name:DermalStem   /media/CCHL-User/Alex/03-Microscopy/2018/2018-06-08-DermalStemCells_dissoiationTest/digested01.zvi


I then tried to perform the import with the following command, logged in as omerouser:
/home/omerouser/OMERO.server/bin/omero import --bulk /OMERO/ManagedRepository/bulk.yml
the toImport.tsv is in the same location as the bulk.yml

I also tried the following command:
/home/omerouser/OMERO.server/bin/omero import --bulk /OMERO/ManagedRepository/bulk.yml
the toImport.tsv is in the same location as the bulk.yml -u 'Alex R'
Again no error message, but no upload either.
Here is the console output:

Code: Select all

Server: [localhost:4064]
Username: [inplace]Alex R
Password:
Created session c0dc9d22-436e-4841-96ce-9afaa0d341a2 (Alex R@localhost:4064). Idle timeout: 10 min. Current group: system
2018-06-12 14:38:20,038 833        [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.4.5-ice36-b83
2018-06-12 14:38:20,095 890        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.7.3 revision: 34c7ca439a617601ce1ac5dcbdcdcfc6f0bfdaf1 date: 11 January 2018
2018-06-12 14:38:20,385 1180       [      main] INFO   formats.importer.cli.CommandLineImporter - Adding exclusion: clientpath
2018-06-12 14:38:20,434 1229       [      main] INFO   formats.importer.cli.CommandLineImporter - Setting checksum algorithm to File-Size-64
2018-06-12 14:38:20,443 1238       [      main] INFO   formats.importer.cli.CommandLineImporter - Setting transfer to ln_s
2018-06-12 14:38:20,453 1248       [      main] INFO   formats.importer.cli.CommandLineImporter - Setting output format: yaml
2018-06-12 14:38:20,467 1262       [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2018-06-12 14:38:21,408 2203       [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2018-06-12 14:38:22,130 2925       [      main] WARN   del.enums.handlers.CorrectionEnumHandler - Unknown Correction value 'Achroplan' will be stored as "Other"
2018-06-12 14:38:22,491 3286       [      main] INFO      ome.formats.importer.ImportCandidates - 1 file(s) parsed into 1 group(s) with 1 call(s) to setId in 1065ms. (1083ms total) [0 unknowns]
2018-06-12 14:38:23,519 4314       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Attempting initial SSL connection to localhost:4064
2018-06-12 14:38:24,711 5506       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Insecure connection requested, falling back
2018-06-12 14:38:25,547 6342       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Server: 5.4.5
2018-06-12 14:38:25,548 6343       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Client: 5.4.5-ice36-b83
2018-06-12 14:38:25,548 6343       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Java Version: 1.8.0_171
2018-06-12 14:38:25,548 6343       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Name: Linux
2018-06-12 14:38:25,548 6343       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Arch: amd64
2018-06-12 14:38:25,549 6344       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Version: 4.13.0-43-generic
2018-06-12 14:38:26,810 7605       [      main] INFO          ome.formats.importer.ImportConfig - Using import target: Dataset:name:DermalStem
2018-06-12 14:38:27,229 8024       [      main] INFO   .importer.exclusions.ClientPathExclusion - ClientPath match for filename: media/CCHL-User/Alex/03-Microscopy/2018/2018-06-08-DermalStemCells_dissoiationTest/digested+filtered01.zvi

==> Summary
0 files uploaded, 0 filesets created, 0 images imported, 0 errors in 0:00:01.366
#only for first file, the rest is identical, then:
==> Summary
0 files uploaded, 0 filesets created, 0 images imported, 0 errors in 0:00:03.176
!! 06/12/18 14:39:13.406 error: 29 communicators not destroyed during global destruction.omerouser@comero:/media/CCHL-User$


Only at the end I get the error message: " error: 29 communicators not destroyed during global destruction"

Is there the option to specify the user in the yml file or in the tsv file?

Any help would be great,
thanks
Alex

[jmoore]

Hi Alex,

The issue is:

Code: Select all

exclude: "clientpath"


which leads to:

Code: Select all

2018-06-12 14:38:27,229 8024       [      main] INFO   .importer.exclusions.ClientPathExclusion - ClientPath match for filename: media/CCHL-User/Alex/03-Microscopy/2018/2018-06-08-DermalStemCells_dissoiationTest/digested+filtered01.zvi


i.e. the importer is saying, "You've already imported this and you've asked me to exclude imports that match based on client path, so I'm happily doing nothing".

Only at the end I get the error message: " error: 29 communicators not destroyed during global destruction"

Please ignore this, we hope a future Ice upgrade will make them go away.

Is there the option to specify the user in the yml file or in the tsv file?

Interesting idea. I have not tested this. (Unless you mean can the exclude be per-user? If so, not currently. It's not overly difficult to add but not something that can be done without a release.)

Cheers,
~Josh

[alexr]

Hi Josh,
thanks for the quick help. Can I change the "exclude" option to something else to allow multiple imports? Or do I simply delete the line?
Thanks
Alex

[jmoore]

Just delete the line if you don't want any exclusion logic.
~Josh

[alexr]

Hi Josh,
I tried that and when I restarted the import I got a file lock error:

Code: Select all

bin/omero import --bulk /OMERO/ManagedRepository/bulk.yml -u inplace
Server: [localhost:4064]
Reconnected to session f18e9dd5-2b74-45d1-b035-33397a166e88 (inplace@localhost:4064). Idle timeout: 10 min. Current group: system


2018-06-13 14:14:11,939 727        [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.4.5-ice36-b83
2018-06-13 14:14:11,974 762        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.7.3 revision: 34c7ca439a617601ce1ac5dcbdcdcfc6f0bfdaf1 date: 11 January 2018
2018-06-13 14:14:12,201 989        [      main] INFO   formats.importer.cli.CommandLineImporter - Setting transfer to ln_s
2018-06-13 14:14:12,213 1001       [      main] INFO   formats.importer.cli.CommandLineImporter - Setting checksum algorithm to File-Size-64
2018-06-13 14:14:12,218 1006       [      main] INFO   formats.importer.cli.CommandLineImporter - Setting output format: yaml
2018-06-13 14:14:12,223 1011       [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2018-06-13 14:14:13,113 1901       [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2018-06-13 14:14:13,839 2627       [      main] WARN   del.enums.handlers.CorrectionEnumHandler - Unknown Correction value 'Achroplan' will be stored as "Other"
2018-06-13 14:14:14,189 2977       [      main] INFO      ome.formats.importer.ImportCandidates - 1 file(s) parsed into 1 group(s) with 1 call(s) to setId in 1060ms. (1075ms total) [0 unknowns]
2018-06-13 14:14:14,819 3607       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Attempting initial SSL connection to localhost:4064
2018-06-13 14:14:15,890 4678       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Insecure connection requested, falling back
2018-06-13 14:14:16,785 5573       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Server: 5.4.5
2018-06-13 14:14:16,785 5573       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Client: 5.4.5-ice36-b83
2018-06-13 14:14:16,785 5573       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Java Version: 1.8.0_171
2018-06-13 14:14:16,785 5573       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Name: Linux
2018-06-13 14:14:16,785 5573       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Arch: amd64
2018-06-13 14:14:16,785 5573       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Version: 4.13.0-43-generic
2018-06-13 14:14:18,063 6851       [      main] INFO          ome.formats.importer.ImportConfig - Using import target: Dataset:name:DermalStem
2018-06-13 14:14:18,204 6992       [      main] ERROR        ome.formats.importer.ImportLibrary - Error on import
java.lang.RuntimeException: Cannot exclusively use the managed repository.

Likely no ManagedRepositoryPrx is being returned from the server.
This could point to a recent server crash. Ask your server administrator
to check for stale .lock files under the OMERO data directory. This
is particularly likely on a server using NFS.

   at ome.formats.importer.ImportLibrary.checkManagedRepo(ImportLibrary.java:750) ~[blitz.jar:na]
   at ome.formats.importer.ImportLibrary.createImport(ImportLibrary.java:338) ~[blitz.jar:na]
   at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:506) ~[blitz.jar:na]
   at ome.formats.importer.ImportLibrary.importCandidates(ImportLibrary.java:289) ~[blitz.jar:na]
   at ome.formats.importer.cli.CommandLineImporter.start(CommandLineImporter.java:289) [blitz.jar:na]
   at ome.formats.importer.cli.CommandLineImporter.main(CommandLineImporter.java:967) [blitz.jar:na]
2018-06-13 14:14:18,228 7016       [      main] INFO         ome.formats.importer.ImportLibrary - Continuing after error

==> Summary
0 files uploaded, 0 filesets created, 0 images imported, 0 errors in 0:00:01.171



Then I checked with fslocks for locked files:

Code: Select all

compiz           1733  POSIX   0B READ  0 1073741826 1073742335 /
compiz           1733  POSIX   0B READ  0        128        128 /
smbd            22925  POSIX   0B WRITE 0          0          0 /run
smbd            22925  POSIX   0B READ  0          4          4 /run
icegridnode      5027  POSIX   4B WRITE 0          0          0 /home/omerouser/OMERO.server-5.4.5-ice36-b8
icebox           5079  POSIX   4B WRITE 0          0          0 /home/omerouser/OMERO.server-5.4.5-ice36-b8
python           5083  FLOCK   0B WRITE 0          0          0 /home/omerouser/omero/tmp/omero_omerouser/5
python           5257  FLOCK   0B WRITE 0          0          0 /home/omerouser/omero/tmp/omero_omerouser/5
java             5060  POSIX  31B WRITE 0          0          0 /OMERO/.omero/repository/9f6b12af-2ef3-4071
java             5060  POSIX  31B WRITE 0          0          0 /home/omerouser/OMERO.server-5.4.5-ice36-b8
(unknown)        5355  FLOCK   0B WRITE 0          0          0 /
gunicorn         5391  FLOCK   0B WRITE 0          0          0 /home/omerouser/omero/tmp/omero_omerouser/5
gunicorn         5389  FLOCK   0B WRITE 0          0          0 /home/omerouser/omero/tmp/omero_omerouser/5
gunicorn         5390  FLOCK   0B WRITE 0          0          0 /home/omerouser/omero/tmp/omero_omerouser/5
gunicorn         5395  FLOCK   0B WRITE 0          0          0 /home/omerouser/omero/tmp/omero_omerouser/5
gunicorn         5397  FLOCK   0B WRITE 0          0          0 /home/omerouser/omero/tmp/omero_omerouser/5


I manually removed the one from
/OMERO/.omero/repository/9f6b12af-2ef3-4071../
But this did not help.

Does anybody have a suggestion?
Thanks Alex

[jmoore]

What does:

Code: Select all

bin/omero fs repos


show? (Also, you might send us your Blitz-0.log as well as master.out and master.err since it'll probably be the next thing we ask for)

~Josh.

[alexr]

Hi Josh,

here is the result of /bin/omero fs repos

Code: Select all

# | Id | UUID                                 | Type   | Path                                                     
---+----+--------------------------------------+--------+----------------------------------------------------------
0 | 57 | 1f613370-de42-44c0-8965-ed94cbb0881e | Public | /OMERO                                                   
1 | 58 | ScriptRepo                           | Script | /home/omerouser/OMERO.server-5.4.5-ice36-b83/lib/scripts
2 | 59 | 842746fd-4734-46bf-b55c-952bcc59e0ec | Public | /OMERO/ManagedRepository                                 
(3 rows)
!! 06/15/18 08:53:53.737 error: communicator not destroyed during global destruction.


Here is the tail of the Blitz-0.log

Code: Select all

   at ome.security.basic.EventHandler.invoke(EventHandler.java:153) ~[server.jar:na]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:111) ~[spring-orm.jar:3.0.1.RELEASE]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108) ~[spring-tx.jar:3.0.1.RELEASE]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249) ~[server.jar:na]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121) ~[server.jar:na]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) ~[spring-aop.jar:3.0.1.RELEASE]
   at com.sun.proxy.$Proxy72.doWork(Unknown Source) ~[na:na]
   at ome.services.util.Executor$Impl.execute(Executor.java:443) ~[server.jar:na]
   at ome.services.util.Executor$Impl.execute(Executor.java:387) ~[server.jar:na]
   at ome.services.blitz.repo.AbstractRepositoryI.takeover(AbstractRepositoryI.java:210) ~[blitz.jar:na]
   at sun.reflect.GeneratedMethodAccessor933.invoke(Unknown Source) ~[na:na]
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
   at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
   at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:273) [spring-core.jar:3.0.1.RELEASE]
   at org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean$MethodInvokingJob.executeInternal(MethodInvokingJobDetailFactoryBean.java:264) [spring-context-support.jar:3.0.1.RELEASE]
   at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86) [spring-context-support.jar:3.0.1.RELEASE]
   at org.quartz.core.JobRunShell.run(JobRunShell.java:203) [quartz.jar:1.5.2]
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_171]
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_171]
   at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]
2018-06-15 08:56:41,305 INFO  [ome.services.sessions.state.SessionCache] (2-thread-2) Synchronizing session cache. Count = 6
2018-06-15 08:56:41,306 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Executor.doWork -- ome.services.sessions.SessionManagerImpl.reload[8ab87180-41e6-4ed3-a25c-aefd3be5fcab]
2018-06-15 08:56:41,306 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Args:   [null, InternalSF@244922133]
2018-06-15 08:56:41,307 INFO  [         ome.security.basic.EventHandler] (2-thread-2)  Auth:   user=0,group=0,event=null(Sessions),sess=ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a
2018-06-15 08:56:41,316 INFO  [                 org.perf4j.TimingLogger] (2-thread-2) start[1529045801306] time[10] tag[omero.call.success.ome.services.sessions.SessionManagerImpl$6.doWork]
2018-06-15 08:56:41,316 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Rslt:   (ome.model.meta.Experimenter:Id_0, ome.model.meta.ExperimenterGroup:Id_0, (ome.model.enums.AdminPrivilege:ReadSession:Hash_1890811569, ome.model.enums.AdminPrivilege:DeleteScriptRepo:Hash_66779577, ome.model.enums.AdminPrivilege:ModifyUser:Hash_-2112703882, ... 12 more), ... 5 more)
2018-06-15 08:56:41,316 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Executor.doWork -- ome.services.sessions.SessionManagerImpl.reload[f32a5489-603d-4fe7-b2f6-df31e9cb8a6a]
2018-06-15 08:56:41,316 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Args:   [null, InternalSF@244922133]
2018-06-15 08:56:41,317 INFO  [         ome.security.basic.EventHandler] (2-thread-2)  Auth:   user=0,group=0,event=null(Sessions),sess=ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a
2018-06-15 08:56:41,324 INFO  [                 org.perf4j.TimingLogger] (2-thread-2) start[1529045801316] time[7] tag[omero.call.success.ome.services.sessions.SessionManagerImpl$6.doWork]
2018-06-15 08:56:41,324 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Rslt:   (ome.model.meta.Experimenter:Id_102, ome.model.meta.ExperimenterGroup:Id_0, (ome.model.enums.AdminPrivilege:ReadSession:Hash_1890811569, ome.model.enums.AdminPrivilege:DeleteScriptRepo:Hash_66779577, ome.model.enums.AdminPrivilege:ModifyUser:Hash_-2112703882, ... 12 more), ... 5 more)
2018-06-15 08:56:41,324 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Executor.doWork -- ome.services.sessions.SessionManagerImpl.reload[d471c14f-3ef5-4fbb-a410-60a6b222114d]
2018-06-15 08:56:41,324 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Args:   [null, InternalSF@244922133]
2018-06-15 08:56:41,325 INFO  [         ome.security.basic.EventHandler] (2-thread-2)  Auth:   user=0,group=0,event=null(Sessions),sess=ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a
2018-06-15 08:56:41,332 INFO  [                 org.perf4j.TimingLogger] (2-thread-2) start[1529045801324] time[7] tag[omero.call.success.ome.services.sessions.SessionManagerImpl$6.doWork]
2018-06-15 08:56:41,332 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Rslt:   (ome.model.meta.Experimenter:Id_0, ome.model.meta.ExperimenterGroup:Id_0, (ome.model.enums.AdminPrivilege:ReadSession:Hash_1890811569, ome.model.enums.AdminPrivilege:DeleteScriptRepo:Hash_66779577, ome.model.enums.AdminPrivilege:ModifyUser:Hash_-2112703882, ... 12 more), ... 5 more)
2018-06-15 08:56:41,332 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Executor.doWork -- ome.services.sessions.SessionManagerImpl.reload[689b2f27-2ec1-468d-b8ed-34d459b3996f]
2018-06-15 08:56:41,332 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Args:   [null, InternalSF@244922133]
2018-06-15 08:56:41,333 INFO  [         ome.security.basic.EventHandler] (2-thread-2)  Auth:   user=0,group=0,event=null(Sessions),sess=ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a
2018-06-15 08:56:41,339 INFO  [                 org.perf4j.TimingLogger] (2-thread-2) start[1529045801332] time[6] tag[omero.call.success.ome.services.sessions.SessionManagerImpl$6.doWork]
2018-06-15 08:56:41,339 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Rslt:   (ome.model.meta.Experimenter:Id_3, ome.model.meta.ExperimenterGroup:Id_0, (ome.model.enums.AdminPrivilege:ReadSession:Hash_1890811569, ome.model.enums.AdminPrivilege:DeleteScriptRepo:Hash_66779577, ome.model.enums.AdminPrivilege:ModifyUser:Hash_-2112703882, ... 12 more), ... 5 more)
2018-06-15 08:56:41,339 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Executor.doWork -- ome.services.sessions.SessionManagerImpl.reload[8ce69697-75a4-4c7c-b517-9e6a95e232ba]
2018-06-15 08:56:41,339 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Args:   [null, InternalSF@244922133]
2018-06-15 08:56:41,340 INFO  [         ome.security.basic.EventHandler] (2-thread-2)  Auth:   user=0,group=0,event=null(Sessions),sess=ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a
2018-06-15 08:56:41,346 INFO  [                 org.perf4j.TimingLogger] (2-thread-2) start[1529045801339] time[6] tag[omero.call.success.ome.services.sessions.SessionManagerImpl$6.doWork]
2018-06-15 08:56:41,346 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Rslt:   (ome.model.meta.Experimenter:Id_2, ome.model.meta.ExperimenterGroup:Id_0, (ome.model.enums.AdminPrivilege:ReadSession:Hash_1890811569, ome.model.enums.AdminPrivilege:DeleteScriptRepo:Hash_66779577, ome.model.enums.AdminPrivilege:ModifyUser:Hash_-2112703882, ... 12 more), ... 5 more)
2018-06-15 08:56:41,346 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Executor.doWork -- ome.services.sessions.SessionManagerImpl.reload[ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a]
2018-06-15 08:56:41,346 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Args:   [null, InternalSF@244922133]
2018-06-15 08:56:41,347 INFO  [         ome.security.basic.EventHandler] (2-thread-2)  Auth:   user=0,group=0,event=null(Sessions),sess=ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a
2018-06-15 08:56:41,351 INFO  [                 org.perf4j.TimingLogger] (2-thread-2) start[1529045801346] time[5] tag[omero.call.success.ome.services.sessions.SessionManagerImpl$6.doWork]
2018-06-15 08:56:41,351 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Rslt:   (ome.model.meta.Experimenter:Id_0, ome.model.meta.ExperimenterGroup:Id_0, (ome.model.enums.AdminPrivilege:ReadSession:Hash_1890811569, ome.model.enums.AdminPrivilege:DeleteScriptRepo:Hash_66779577, ome.model.enums.AdminPrivilege:ModifyUser:Hash_-2112703882, ... 12 more), ... 5 more)
2018-06-15 08:56:41,351 INFO  [                 org.perf4j.TimingLogger] (2-thread-2) start[1529045801306] time[45] tag[omero.sessions.synchronization]
2018-06-15 08:56:41,351 INFO  [ome.services.sessions.state.SessionCache] (2-thread-2) Synchronization took 45 ms.
2018-06-15 08:56:59,668 INFO  [ ome.services.blitz.fire.SessionManagerI] (.Server-20) Found session locally: 8ce69697-75a4-4c7c-b517-9e6a95e232ba
2018-06-15 08:56:59,668 INFO  [ ome.services.blitz.fire.SessionManagerI] (.Server-20) Rejoining session ServiceFactoryI(session-544867a3-29b5-4148-a1bd-6b1cdbc6a062/8ce69697-75a4-4c7c-b517-9e6a95e232ba) (agent=OMERO.web)
2018-06-15 08:56:59,675 INFO  [o.services.sessions.SessionContext$Count] (.Server-14) -Reference count: 8ce69697-75a4-4c7c-b517-9e6a95e232ba=0
2018-06-15 08:56:59,675 INFO  [                      omero.cmd.SessionI] (.Server-14) cleanupSelf(ServiceFactoryI(session-544867a3-29b5-4148-a1bd-6b1cdbc6a062/8ce69697-75a4-4c7c-b517-9e6a95e232ba)).
2018-06-15 08:57:00,028 INFO  [ ome.services.blitz.fire.SessionManagerI] (2-thread-5) Performing requestHeartbeats
2018-06-15 08:57:06,557 INFO  [ ome.services.blitz.fire.SessionManagerI] (.Server-14) Found session locally: 689b2f27-2ec1-468d-b8ed-34d459b3996f
2018-06-15 08:57:06,558 INFO  [ ome.services.blitz.fire.SessionManagerI] (.Server-14) Rejoining session ServiceFactoryI(session-a9e2a75f-a797-48d2-b362-47db859bb569/689b2f27-2ec1-468d-b8ed-34d459b3996f) (agent=OMERO.web)
2018-06-15 08:57:06,564 INFO  [o.services.sessions.SessionContext$Count] (.Server-19) -Reference count: 689b2f27-2ec1-468d-b8ed-34d459b3996f=0
2018-06-15 08:57:06,564 INFO  [                      omero.cmd.SessionI] (.Server-19) cleanupSelf(ServiceFactoryI(session-a9e2a75f-a797-48d2-b362-47db859bb569/689b2f27-2ec1-468d-b8ed-34d459b3996f)).
2018-06-15 08:57:08,831 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Executor.doWork -- ome.services.blitz.repo.LegacyRepositoryI.takeover[]
2018-06-15 08:57:08,832 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Args:   [null, InternalSF@244922133]
2018-06-15 08:57:08,844 INFO  [         ome.security.basic.EventHandler] (2-thread-2)  Auth:   user=0,group=0,event=38028(Internal),sess=ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a
2018-06-15 08:57:08,844 WARN  [       ome.services.blitz.repo.FileMaker] (2-thread-2) Failed to delete lock file: /OMERO/ManagedRepository/.omero/repository/9f6b12af-2ef3-4071-9b64-6ab409058ee3/.lock
2018-06-15 08:57:08,847 INFO  [                 org.perf4j.TimingLogger] (2-thread-2) start[1529045828832] time[15] tag[omero.call.success.ome.services.blitz.repo.AbstractRepositoryI$GetOrCreateRepo.doWork]
2018-06-15 08:57:08,847 INFO  [        ome.services.util.ServiceHandler] (2-thread-2)  Rslt:   java.io.FileNotFoundException: /OMERO/ManagedRepository/.omero/repository/9f6b12af-2ef3-4071-9b64-6a
2018-06-15 08:57:08,848 ERROR [      o.s.blitz.repo.AbstractRepositoryI] (2-thread-2) Failed during repository takeover
java.io.FileNotFoundException: /OMERO/ManagedRepository/.omero/repository/9f6b12af-2ef3-4071-9b64-6ab409058ee3/repo_uuid (Permission denied)
   at java.io.RandomAccessFile.open0(Native Method) ~[na:1.8.0_171]
   at java.io.RandomAccessFile.open(RandomAccessFile.java:316) ~[na:1.8.0_171]
   at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243) ~[na:1.8.0_171]
   at ome.services.blitz.repo.FileMaker.init(FileMaker.java:82) ~[blitz.jar:na]
   at ome.services.blitz.repo.AbstractRepositoryI$GetOrCreateRepo.doWork(AbstractRepositoryI.java:335) ~[blitz.jar:na]
   at sun.reflect.GeneratedMethodAccessor315.invoke(Unknown Source) ~[na:na]
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
   at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
   at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307) ~[spring-aop.jar:3.0.1.RELEASE]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) ~[spring-aop.jar:3.0.1.RELEASE]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) ~[spring-aop.jar:3.0.1.RELEASE]
   at ome.services.util.Executor$Impl$Interceptor.invoke(Executor.java:562) ~[server.jar:na]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at ome.security.basic.EventHandler.invoke(EventHandler.java:153) ~[server.jar:na]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:111) ~[spring-orm.jar:3.0.1.RELEASE]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108) ~[spring-tx.jar:3.0.1.RELEASE]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249) ~[server.jar:na]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121) ~[server.jar:na]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) ~[spring-aop.jar:3.0.1.RELEASE]
   at com.sun.proxy.$Proxy72.doWork(Unknown Source) ~[na:na]
   at ome.services.util.Executor$Impl.execute(Executor.java:443) ~[server.jar:na]
   at ome.services.util.Executor$Impl.execute(Executor.java:387) ~[server.jar:na]
   at ome.services.blitz.repo.AbstractRepositoryI.takeover(AbstractRepositoryI.java:210) ~[blitz.jar:na]
   at sun.reflect.GeneratedMethodAccessor933.invoke(Unknown Source) ~[na:na]
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
   at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
   at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:273) [spring-core.jar:3.0.1.RELEASE]
   at org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean$MethodInvokingJob.executeInternal(MethodInvokingJobDetailFactoryBean.java:264) [spring-context-support.jar:3.0.1.RELEASE]
   at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86) [spring-context-support.jar:3.0.1.RELEASE]
   at org.quartz.core.JobRunShell.run(JobRunShell.java:203) [quartz.jar:1.5.2]
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_171]
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_171]
   at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]
2018-06-15 08:57:59,665 INFO  [ ome.services.blitz.fire.SessionManagerI] (.Server-19) Found session locally: 8ce69697-75a4-4c7c-b517-9e6a95e232ba
2018-06-15 08:57:59,665 INFO  [ ome.services.blitz.fire.SessionManagerI] (.Server-19) Rejoining session ServiceFactoryI(session-bf7c2b23-0b45-4f11-a502-8466693e4d5d/8ce69697-75a4-4c7c-b517-9e6a95e232ba) (agent=OMERO.web)
2018-06-15 08:57:59,669 INFO  [o.services.sessions.SessionContext$Count] (.Server-20) -Reference count: 8ce69697-75a4-4c7c-b517-9e6a95e232ba=0
2018-06-15 08:57:59,670 INFO  [                      omero.cmd.SessionI] (.Server-20) cleanupSelf(ServiceFactoryI(session-bf7c2b23-0b45-4f11-a502-8466693e4d5d/8ce69697-75a4-4c7c-b517-9e6a95e232ba)).
2018-06-15 08:58:06,560 INFO  [ ome.services.blitz.fire.SessionManagerI] (.Server-20) Found session locally: 689b2f27-2ec1-468d-b8ed-34d459b3996f
2018-06-15 08:58:06,561 INFO  [ ome.services.blitz.fire.SessionManagerI] (.Server-20) Rejoining session ServiceFactoryI(session-87d40a45-ff56-4188-8c94-d94deb2a2fd1/689b2f27-2ec1-468d-b8ed-34d459b3996f) (agent=OMERO.web)
2018-06-15 08:58:06,566 INFO  [o.services.sessions.SessionContext$Count] (.Server-14) -Reference count: 689b2f27-2ec1-468d-b8ed-34d459b3996f=0
2018-06-15 08:58:06,566 INFO  [                      omero.cmd.SessionI] (.Server-14) cleanupSelf(ServiceFactoryI(session-87d40a45-ff56-4188-8c94-d94deb2a2fd1/689b2f27-2ec1-468d-b8ed-34d459b3996f)).
2018-06-15 08:58:08,823 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Executor.doWork -- ome.services.blitz.repo.LegacyRepositoryI.takeover[]
2018-06-15 08:58:08,823 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Args:   [null, InternalSF@244922133]
2018-06-15 08:58:08,833 INFO  [         ome.security.basic.EventHandler] (2-thread-1)  Auth:   user=0,group=0,event=38029(Internal),sess=ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a
2018-06-15 08:58:08,833 WARN  [       ome.services.blitz.repo.FileMaker] (2-thread-1) Failed to delete lock file: /OMERO/ManagedRepository/.omero/repository/9f6b12af-2ef3-4071-9b64-6ab409058ee3/.lock
2018-06-15 08:58:08,836 INFO  [                 org.perf4j.TimingLogger] (2-thread-1) start[1529045888823] time[13] tag[omero.call.success.ome.services.blitz.repo.AbstractRepositoryI$GetOrCreateRepo.doWork]
2018-06-15 08:58:08,836 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Rslt:   java.io.FileNotFoundException: /OMERO/ManagedRepository/.omero/repository/9f6b12af-2ef3-4071-9b64-6a
2018-06-15 08:58:08,837 ERROR [      o.s.blitz.repo.AbstractRepositoryI] (2-thread-1) Failed during repository takeover
java.io.FileNotFoundException: /OMERO/ManagedRepository/.omero/repository/9f6b12af-2ef3-4071-9b64-6ab409058ee3/repo_uuid (Permission denied)
   at java.io.RandomAccessFile.open0(Native Method) ~[na:1.8.0_171]
   at java.io.RandomAccessFile.open(RandomAccessFile.java:316) ~[na:1.8.0_171]
   at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243) ~[na:1.8.0_171]
   at ome.services.blitz.repo.FileMaker.init(FileMaker.java:82) ~[blitz.jar:na]
   at ome.services.blitz.repo.AbstractRepositoryI$GetOrCreateRepo.doWork(AbstractRepositoryI.java:335) ~[blitz.jar:na]
   at sun.reflect.GeneratedMethodAccessor315.invoke(Unknown Source) ~[na:na]
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
   at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
   at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307) ~[spring-aop.jar:3.0.1.RELEASE]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) ~[spring-aop.jar:3.0.1.RELEASE]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) ~[spring-aop.jar:3.0.1.RELEASE]
   at ome.services.util.Executor$Impl$Interceptor.invoke(Executor.java:562) ~[server.jar:na]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at ome.security.basic.EventHandler.invoke(EventHandler.java:153) ~[server.jar:na]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:111) ~[spring-orm.jar:3.0.1.RELEASE]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108) ~[spring-tx.jar:3.0.1.RELEASE]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249) ~[server.jar:na]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121) ~[server.jar:na]
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[spring-aop.jar:3.0.1.RELEASE]
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) ~[spring-aop.jar:3.0.1.RELEASE]
   at com.sun.proxy.$Proxy72.doWork(Unknown Source) ~[na:na]
   at ome.services.util.Executor$Impl.execute(Executor.java:443) ~[server.jar:na]
   at ome.services.util.Executor$Impl.execute(Executor.java:387) ~[server.jar:na]
   at ome.services.blitz.repo.AbstractRepositoryI.takeover(AbstractRepositoryI.java:210) ~[blitz.jar:na]
   at sun.reflect.GeneratedMethodAccessor933.invoke(Unknown Source) ~[na:na]
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
   at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
   at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:273) [spring-core.jar:3.0.1.RELEASE]
   at org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean$MethodInvokingJob.executeInternal(MethodInvokingJobDetailFactoryBean.java:264) [spring-context-support.jar:3.0.1.RELEASE]
   at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86) [spring-context-support.jar:3.0.1.RELEASE]
   at org.quartz.core.JobRunShell.run(JobRunShell.java:203) [quartz.jar:1.5.2]
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_171]
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_171]
   at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]
2018-06-15 08:58:41,298 INFO  [ome.services.sessions.state.SessionCache] (2-thread-1) Synchronizing session cache. Count = 6
2018-06-15 08:58:41,298 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Executor.doWork -- ome.services.sessions.SessionManagerImpl.reload[8ab87180-41e6-4ed3-a25c-aefd3be5fcab]
2018-06-15 08:58:41,298 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Args:   [null, InternalSF@244922133]
2018-06-15 08:58:41,299 INFO  [         ome.security.basic.EventHandler] (2-thread-1)  Auth:   user=0,group=0,event=null(Sessions),sess=ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a
2018-06-15 08:58:41,306 INFO  [                 org.perf4j.TimingLogger] (2-thread-1) start[1529045921298] time[8] tag[omero.call.success.ome.services.sessions.SessionManagerImpl$6.doWork]
2018-06-15 08:58:41,306 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Rslt:   (ome.model.meta.Experimenter:Id_0, ome.model.meta.ExperimenterGroup:Id_0, (ome.model.enums.AdminPrivilege:ReadSession:Hash_1890811569, ome.model.enums.AdminPrivilege:DeleteScriptRepo:Hash_66779577, ome.model.enums.AdminPrivilege:ModifyUser:Hash_-2112703882, ... 12 more), ... 5 more)
2018-06-15 08:58:41,307 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Executor.doWork -- ome.services.sessions.SessionManagerImpl.reload[f32a5489-603d-4fe7-b2f6-df31e9cb8a6a]
2018-06-15 08:58:41,307 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Args:   [null, InternalSF@244922133]
2018-06-15 08:58:41,307 INFO  [         ome.security.basic.EventHandler] (2-thread-1)  Auth:   user=0,group=0,event=null(Sessions),sess=ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a
2018-06-15 08:58:41,314 INFO  [                 org.perf4j.TimingLogger] (2-thread-1) start[1529045921307] time[6] tag[omero.call.success.ome.services.sessions.SessionManagerImpl$6.doWork]
2018-06-15 08:58:41,314 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Rslt:   (ome.model.meta.Experimenter:Id_102, ome.model.meta.ExperimenterGroup:Id_0, (ome.model.enums.AdminPrivilege:ReadSession:Hash_1890811569, ome.model.enums.AdminPrivilege:DeleteScriptRepo:Hash_66779577, ome.model.enums.AdminPrivilege:ModifyUser:Hash_-2112703882, ... 12 more), ... 5 more)
2018-06-15 08:58:41,314 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Executor.doWork -- ome.services.sessions.SessionManagerImpl.reload[d471c14f-3ef5-4fbb-a410-60a6b222114d]
2018-06-15 08:58:41,314 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Args:   [null, InternalSF@244922133]
2018-06-15 08:58:41,314 INFO  [         ome.security.basic.EventHandler] (2-thread-1)  Auth:   user=0,group=0,event=null(Sessions),sess=ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a
2018-06-15 08:58:41,321 INFO  [                 org.perf4j.TimingLogger] (2-thread-1) start[1529045921314] time[6] tag[omero.call.success.ome.services.sessions.SessionManagerImpl$6.doWork]
2018-06-15 08:58:41,321 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Rslt:   (ome.model.meta.Experimenter:Id_0, ome.model.meta.ExperimenterGroup:Id_0, (ome.model.enums.AdminPrivilege:ReadSession:Hash_1890811569, ome.model.enums.AdminPrivilege:DeleteScriptRepo:Hash_66779577, ome.model.enums.AdminPrivilege:ModifyUser:Hash_-2112703882, ... 12 more), ... 5 more)
2018-06-15 08:58:41,321 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Executor.doWork -- ome.services.sessions.SessionManagerImpl.reload[689b2f27-2ec1-468d-b8ed-34d459b3996f]
2018-06-15 08:58:41,321 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Args:   [null, InternalSF@244922133]
2018-06-15 08:58:41,322 INFO  [         ome.security.basic.EventHandler] (2-thread-1)  Auth:   user=0,group=0,event=null(Sessions),sess=ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a
2018-06-15 08:58:41,328 INFO  [                 org.perf4j.TimingLogger] (2-thread-1) start[1529045921321] time[7] tag[omero.call.success.ome.services.sessions.SessionManagerImpl$6.doWork]
2018-06-15 08:58:41,328 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Rslt:   (ome.model.meta.Experimenter:Id_3, ome.model.meta.ExperimenterGroup:Id_0, (ome.model.enums.AdminPrivilege:ReadSession:Hash_1890811569, ome.model.enums.AdminPrivilege:DeleteScriptRepo:Hash_66779577, ome.model.enums.AdminPrivilege:ModifyUser:Hash_-2112703882, ... 12 more), ... 5 more)
2018-06-15 08:58:41,328 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Executor.doWork -- ome.services.sessions.SessionManagerImpl.reload[8ce69697-75a4-4c7c-b517-9e6a95e232ba]
2018-06-15 08:58:41,328 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Args:   [null, InternalSF@244922133]
2018-06-15 08:58:41,329 INFO  [         ome.security.basic.EventHandler] (2-thread-1)  Auth:   user=0,group=0,event=null(Sessions),sess=ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a
2018-06-15 08:58:41,335 INFO  [                 org.perf4j.TimingLogger] (2-thread-1) start[1529045921328] time[6] tag[omero.call.success.ome.services.sessions.SessionManagerImpl$6.doWork]
2018-06-15 08:58:41,335 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Rslt:   (ome.model.meta.Experimenter:Id_2, ome.model.meta.ExperimenterGroup:Id_0, (ome.model.enums.AdminPrivilege:ReadSession:Hash_1890811569, ome.model.enums.AdminPrivilege:DeleteScriptRepo:Hash_66779577, ome.model.enums.AdminPrivilege:ModifyUser:Hash_-2112703882, ... 12 more), ... 5 more)
2018-06-15 08:58:41,335 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Executor.doWork -- ome.services.sessions.SessionManagerImpl.reload[ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a]
2018-06-15 08:58:41,335 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Args:   [null, InternalSF@244922133]
2018-06-15 08:58:41,336 INFO  [         ome.security.basic.EventHandler] (2-thread-1)  Auth:   user=0,group=0,event=null(Sessions),sess=ed86cdc0-0ac1-47d0-a06d-df89e4b66c0a
2018-06-15 08:58:41,341 INFO  [                 org.perf4j.TimingLogger] (2-thread-1) start[1529045921335] time[5] tag[omero.call.success.ome.services.sessions.SessionManagerImpl$6.doWork]
2018-06-15 08:58:41,341 INFO  [        ome.services.util.ServiceHandler] (2-thread-1)  Rslt:   (ome.model.meta.Experimenter:Id_0, ome.model.meta.ExperimenterGroup:Id_0, (ome.model.enums.AdminPrivilege:ReadSession:Hash_1890811569, ome.model.enums.AdminPrivilege:DeleteScriptRepo:Hash_66779577, ome.model.enums.AdminPrivilege:ModifyUser:Hash_-2112703882, ... 12 more), ... 5 more)
2018-06-15 08:58:41,341 INFO  [                 org.perf4j.TimingLogger] (2-thread-1) start[1529045921298] time[43] tag[omero.sessions.synchronization]
2018-06-15 08:58:41,341 INFO  [ome.services.sessions.state.SessionCache] (2-thread-1) Synchronization took 43 ms.


The master.out is empty.

The master.err looks like this (tail):

Code: Select all

2018-05-04 10:12:45,034 2655       [      main] INFO      ome.formats.importer.ImportCandidates - 10 file(s) parsed into 2 group(s) with 2 call(s) to setId in 1132ms. (1159ms total) [0 unknowns]
2018-05-04 10:13:02,328 679        [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.4.5-ice36-b83
2018-05-04 10:13:02,349 700        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.7.3 revision: 34c7ca439a617601ce1ac5dcbdcdcfc6f0bfdaf1 date: 11 January 2018
2018-05-04 10:13:02,447 798        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2018-05-04 10:13:03,148 1499       [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2018-05-04 10:13:04,592 2943       [      main] INFO      ome.formats.importer.ImportCandidates - 10 file(s) parsed into 3 group(s) with 3 call(s) to setId in 1417ms. (1443ms total) [0 unknowns]
2018-05-04 10:13:12,224 657        [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.4.5-ice36-b83
2018-05-04 10:13:12,246 679        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.7.3 revision: 34c7ca439a617601ce1ac5dcbdcdcfc6f0bfdaf1 date: 11 January 2018
2018-05-04 10:13:12,348 781        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2018-05-04 10:13:13,038 1471       [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2018-05-04 10:13:14,239 2672       [      main] INFO      ome.formats.importer.ImportCandidates - 10 file(s) parsed into 2 group(s) with 2 call(s) to setId in 1178ms. (1197ms total) [0 unknowns]
2018-05-04 10:14:46,232 659        [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.4.5-ice36-b83
2018-05-04 10:14:46,252 679        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.7.3 revision: 34c7ca439a617601ce1ac5dcbdcdcfc6f0bfdaf1 date: 11 January 2018
2018-05-04 10:14:46,359 786        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2018-05-04 10:14:47,047 1474       [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2018-05-04 10:14:47,780 2207       [      main] INFO      ome.formats.importer.ImportCandidates - 6 file(s) parsed into 1 group(s) with 1 call(s) to setId in 714ms. (733ms total) [0 unknowns]
2018-05-04 10:37:03,057 806        [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.4.5-ice36-b83
2018-05-04 10:37:03,080 829        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.7.3 revision: 34c7ca439a617601ce1ac5dcbdcdcfc6f0bfdaf1 date: 11 January 2018
2018-05-04 10:37:03,228 977        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2018-05-04 10:37:04,062 1811       [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2018-05-04 10:37:06,165 3914       [      main] INFO      ome.formats.importer.ImportCandidates - 10 file(s) parsed into 5 group(s) with 5 call(s) to setId in 2065ms. (2101ms total) [0 unknowns]
2018-05-04 10:37:09,790 745        [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.4.5-ice36-b83
2018-05-04 10:37:09,812 767        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.7.3 revision: 34c7ca439a617601ce1ac5dcbdcdcfc6f0bfdaf1 date: 11 January 2018
2018-05-04 10:37:09,953 908        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2018-05-04 10:37:10,779 1734       [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2018-05-04 10:37:12,527 3482       [      main] INFO      ome.formats.importer.ImportCandidates - 10 file(s) parsed into 4 group(s) with 4 call(s) to setId in 1715ms. (1747ms total) [0 unknowns]
2018-05-04 10:37:16,003 645        [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.4.5-ice36-b83
2018-05-04 10:37:16,033 675        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.7.3 revision: 34c7ca439a617601ce1ac5dcbdcdcfc6f0bfdaf1 date: 11 January 2018
2018-05-04 10:37:16,136 778        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2018-05-04 10:37:16,935 1577       [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2018-05-04 10:37:18,728 3370       [      main] INFO      ome.formats.importer.ImportCandidates - 10 file(s) parsed into 4 group(s) with 4 call(s) to setId in 1768ms. (1792ms total) [0 unknowns]
2018-05-04 10:37:22,815 707        [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.4.5-ice36-b83
2018-05-04 10:37:22,840 732        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.7.3 revision: 34c7ca439a617601ce1ac5dcbdcdcfc6f0bfdaf1 date: 11 January 2018
2018-05-04 10:37:22,951 843        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2018-05-04 10:37:23,677 1569       [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2018-05-04 10:37:26,433 4325       [      main] INFO      ome.formats.importer.ImportCandidates - 10 file(s) parsed into 9 group(s) with 9 call(s) to setId in 2720ms. (2755ms total) [0 unknowns]
2018-05-04 10:37:29,997 722        [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.4.5-ice36-b83
2018-05-04 10:37:30,031 756        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.7.3 revision: 34c7ca439a617601ce1ac5dcbdcdcfc6f0bfdaf1 date: 11 January 2018
2018-05-04 10:37:30,137 862        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2018-05-04 10:37:31,055 1780       [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2018-05-04 10:37:32,911 3636       [      main] INFO      ome.formats.importer.ImportCandidates - 10 file(s) parsed into 3 group(s) with 3 call(s) to setId in 1815ms. (1855ms total) [0 unknowns]
2018-05-04 10:37:37,532 1348       [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.4.5-ice36-b83
2018-05-04 10:37:37,553 1369       [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.7.3 revision: 34c7ca439a617601ce1ac5dcbdcdcfc6f0bfdaf1 date: 11 January 2018
2018-05-04 10:37:37,657 1473       [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2018-05-04 10:37:38,411 2227       [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2018-05-04 10:37:40,007 3823       [      main] INFO      ome.formats.importer.ImportCandidates - 10 file(s) parsed into 3 group(s) with 3 call(s) to setId in 1568ms. (1595ms total) [0 unknowns]
2018-05-04 10:37:57,466 671        [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.4.5-ice36-b83
2018-05-04 10:37:57,488 693        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.7.3 revision: 34c7ca439a617601ce1ac5dcbdcdcfc6f0bfdaf1 date: 11 January 2018
2018-05-04 10:37:57,607 812        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2018-05-04 10:37:58,401 1606       [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2018-05-04 10:37:58,404 1609       [      main] INFO      ome.formats.importer.ImportCandidates - 1 file(s) parsed into 0 group(s) with 0 call(s) to setId in 0ms. (3ms total) [0 unknowns]
2018-05-07 13:22:33,681 726        [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.4.5-ice36-b83
2018-05-07 13:22:33,709 754        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.7.3 revision: 34c7ca439a617601ce1ac5dcbdcdcfc6f0bfdaf1 date: 11 January 2018
2018-05-07 13:22:33,821 866        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2018-05-07 13:22:34,609 1654       [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2018-05-07 13:22:35,595 2640       [      main] INFO      ome.formats.importer.ImportCandidates - 5 file(s) parsed into 2 group(s) with 2 call(s) to setId in 955ms. (986ms total) [0 unknowns]
2018-05-07 13:26:37,267 739        [      main] INFO          ome.formats.importer.ImportConfig - OMERO Version: 5.4.5-ice36-b83
2018-05-07 13:26:37,291 763        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 5.7.3 revision: 34c7ca439a617601ce1ac5dcbdcdcfc6f0bfdaf1 date: 11 January 2018
2018-05-07 13:26:37,407 879        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2018-05-07 13:26:38,208 1680       [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2018-05-07 13:26:39,122 2594       [      main] INFO      ome.formats.importer.ImportCandidates - 4 file(s) parsed into 2 group(s) with 2 call(s) to setId in 891ms. (914ms total) [0 unknowns]
-! 05/08/18 15:19:41.804 OMERO.Glacier2: warning: unable to contact permissions verifier `BlitzVerifier@BlitzAdapters'
   Reference.cpp:1637: Ice::NoEndpointException:
   no suitable endpoint available for proxy `BlitzVerifier -t -e 1.1 @ BlitzAdapters'
-! 05/08/18 15:19:41.838 OMERO.Glacier2: warning: unable to contact session manager `BlitzManager@BlitzAdapters'
   Reference.cpp:1637: Ice::NoEndpointException:
   no suitable endpoint available for proxy `BlitzManager -t -e 1.1 @ BlitzAdapters'
!! 05/08/18 15:19:57.491 error: communicator not destroyed during global destruction.-! 5/8/18 15:22:29:990 warning: Blitz-0-Ice.ThreadPool.Server-9: dispatch exception:
   identity: session-0ad4e193-8b34-4b1d-b6ea-f606a9dd9dd2/3ad7243c-be60-4e8d-a440-c155a4decc51
   facet:
   operation: destroy
   remote host: 130.83.98.66 remote port: 56742
   Ice.ObjectAdapterDeactivatedException
       name = "BlitzAdapter"
      at Ice.ObjectAdapterI.checkForDeactivation(ObjectAdapterI.java:1170)
      at Ice.ObjectAdapterI.findFacet(ObjectAdapterI.java:498)
      at Ice.ObjectAdapterI.find(ObjectAdapterI.java:491)
      at omero.cmd.SessionI.unregisterServant(SessionI.java:633)
      at omero.cmd.SessionI.cleanServants(SessionI.java:449)
      at omero.cmd.SessionI.cleanServants(SessionI.java:379)
      at omero.cmd.SessionI.doDestroy(SessionI.java:369)
      at omero.cmd.SessionI.destroy(SessionI.java:297)
      at omero.api._ServiceFactoryTie.destroy(_ServiceFactoryTie.java:65)
      at Glacier2._SessionDisp.___destroy(_SessionDisp.java:94)
      at omero.api._ServiceFactoryDisp.__dispatch(_ServiceFactoryDisp.java:1263)
      at IceInternal.Incoming.invoke(Incoming.java:221)
      at Ice.ConnectionI.invokeAll(ConnectionI.java:2536)
      at Ice.ConnectionI.dispatch(ConnectionI.java:1145)
      at Ice.ConnectionI.message(ConnectionI.java:1056)
      at IceInternal.ThreadPool.run(ThreadPool.java:395)
      at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
      at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832)
      at java.lang.Thread.run(Thread.java:748)
   
-! 5/8/18 15:22:29:912 warning: Blitz-0-Ice.ThreadPool.Server-0: dispatch exception:
   identity: session-ea157632-ae45-4852-aab9-b05748744caa/585d6eff-6c30-4ff8-9c3f-28ed0238c2ba
   facet:
   operation: destroy
   remote host: 130.83.98.66 remote port: 56744
   Ice.ObjectAdapterDeactivatedException
       name = "BlitzAdapter"
      at Ice.ObjectAdapterI.checkForDeactivation(ObjectAdapterI.java:1170)
      at Ice.ObjectAdapterI.findFacet(ObjectAdapterI.java:498)
      at Ice.ObjectAdapterI.find(ObjectAdapterI.java:491)
      at omero.cmd.SessionI.unregisterServant(SessionI.java:633)
      at omero.cmd.SessionI.cleanServants(SessionI.java:449)
      at omero.cmd.SessionI.cleanServants(SessionI.java:379)
      at omero.cmd.SessionI.doDestroy(SessionI.java:369)
      at omero.cmd.SessionI.destroy(SessionI.java:297)
      at omero.api._ServiceFactoryTie.destroy(_ServiceFactoryTie.java:65)
      at Glacier2._SessionDisp.___destroy(_SessionDisp.java:94)
      at omero.api._ServiceFactoryDisp.__dispatch(_ServiceFactoryDisp.java:1263)
      at IceInternal.Incoming.invoke(Incoming.java:221)
      at Ice.ConnectionI.invokeAll(ConnectionI.java:2536)
      at Ice.ConnectionI.dispatch(ConnectionI.java:1145)
      at Ice.ConnectionI.message(ConnectionI.java:1056)
      at IceInternal.ThreadPool.run(ThreadPool.java:395)
      at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
      at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832)
      at java.lang.Thread.run(Thread.java:748)
   
!! 05/08/18 15:22:30.603 error: communicator not destroyed during global destruction.-! 05/08/18 15:25:56.004 OMERO.Glacier2: warning: unable to contact permissions verifier `BlitzVerifier@BlitzAdapters'
   Reference.cpp:1637: Ice::NoEndpointException:
   no suitable endpoint available for proxy `BlitzVerifier -t -e 1.1 @ BlitzAdapters'
-! 05/08/18 15:25:56.005 OMERO.Glacier2: warning: unable to contact session manager `BlitzManager@BlitzAdapters'
   Reference.cpp:1637: Ice::NoEndpointException:
   no suitable endpoint available for proxy `BlitzManager -t -e 1.1 @ BlitzAdapters'
!! 05/08/18 15:26:11.400 error: communicator not destroyed during global destruction.!! 05/08/18 17:53:51.908 error: communicator not destroyed during global destruction.-! 05/08/18 17:55:27.589 OMERO.Glacier2: warning: unable to contact permissions verifier `BlitzVerifier@BlitzAdapters'
   Reference.cpp:1637: Ice::NoEndpointException:
   no suitable endpoint available for proxy `BlitzVerifier -t -e 1.1 @ BlitzAdapters'
-! 05/08/18 17:55:27.590 OMERO.Glacier2: warning: unable to contact session manager `BlitzManager@BlitzAdapters'
   Reference.cpp:1637: Ice::NoEndpointException:
   no suitable endpoint available for proxy `BlitzManager -t -e 1.1 @ BlitzAdapters'
!! 05/08/18 17:55:43.760 error: communicator not destroyed during global destruction.!! 05/08/18 17:56:14.126 error: communicator not destroyed during global destruction.-! 05/25/18 11:52:30.452 icegridnode: warning: exception occurred while deactivating `OMERO.Glacier2' using process proxy:
   Network.cpp:2357: Ice::ConnectionRefusedException:
   connection refused: Connection refused
-! 05/25/18 11:52:30.515 icegridnode: warning: exception occurred while deactivating `Processor-0' using process proxy:
   Network.cpp:2357: Ice::ConnectionRefusedException:
   connection refused: Connection refused
-! 05/25/18 11:52:30.515 icegridnode: warning: exception occurred while deactivating `Tables-0' using process proxy:
   Network.cpp:2357: Ice::ConnectionRefusedException:
   connection refused: Connection refused
-! 05/25/18 11:52:30.515 icegridnode: warning: exception occurred while deactivating `OMERO.IceStorm' using process proxy:
   Network.cpp:2357: Ice::ConnectionRefusedException:
   connection refused: Connection refused
-! 05/25/18 11:57:57.632 OMERO.Glacier2: warning: unable to contact permissions verifier `BlitzVerifier@BlitzAdapters'
   Reference.cpp:1637: Ice::NoEndpointException:
   no suitable endpoint available for proxy `BlitzVerifier -t -e 1.1 @ BlitzAdapters'
-! 05/25/18 11:57:57.667 OMERO.Glacier2: warning: unable to contact session manager `BlitzManager@BlitzAdapters'
   Reference.cpp:1637: Ice::NoEndpointException:
   no suitable endpoint available for proxy `BlitzManager -t -e 1.1 @ BlitzAdapters'
!! 05/25/18 11:58:13.174 error: communicator not destroyed during global destruction.-! 5/31/18 13:40:39:364 warning: Blitz-0-Ice.ThreadPool.Server-65: dispatch exception:
   identity: session-1d7e24a4-d386-4c00-ab53-3b6b307a386e/16ab0dfd-51bb-45db-bc5c-ee9d86285f2d
   facet:
   operation: destroy
   remote host: 130.83.98.66 remote port: 39720
   Ice.ObjectAdapterDeactivatedException
       name = "BlitzAdapter"
      at Ice.ObjectAdapterI.checkForDeactivation(ObjectAdapterI.java:1170)
      at Ice.ObjectAdapterI.findFacet(ObjectAdapterI.java:498)
      at Ice.ObjectAdapterI.find(ObjectAdapterI.java:491)
      at omero.cmd.SessionI.unregisterServant(SessionI.java:633)
      at omero.cmd.SessionI.cleanServants(SessionI.java:449)
      at omero.cmd.SessionI.cleanServants(SessionI.java:379)
      at omero.cmd.SessionI.doDestroy(SessionI.java:369)
      at omero.cmd.SessionI.destroy(SessionI.java:297)
      at omero.api._ServiceFactoryTie.destroy(_ServiceFactoryTie.java:65)
      at Glacier2._SessionDisp.___destroy(_SessionDisp.java:94)
      at omero.api._ServiceFactoryDisp.__dispatch(_ServiceFactoryDisp.java:1263)
      at IceInternal.Incoming.invoke(Incoming.java:221)
      at Ice.ConnectionI.invokeAll(ConnectionI.java:2536)
      at Ice.ConnectionI.dispatch(ConnectionI.java:1145)
      at Ice.ConnectionI.message(ConnectionI.java:1056)
      at IceInternal.ThreadPool.run(ThreadPool.java:395)
      at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
      at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832)
      at java.lang.Thread.run(Thread.java:748)
   
-! 5/31/18 13:40:39:377 warning: Blitz-0-Ice.ThreadPool.Server-67: dispatch exception:
   identity: session-0dc373e7-1d97-43e8-8c47-d26def7472e4/80ac50ba-1cbf-4fbe-afc8-676ec8325145
   facet:
   operation: destroy
   remote host: 130.83.98.66 remote port: 39718
   Ice.ObjectAdapterDeactivatedException
       name = "BlitzAdapter"
      at Ice.ObjectAdapterI.checkForDeactivation(ObjectAdapterI.java:1170)
      at Ice.ObjectAdapterI.findFacet(ObjectAdapterI.java:498)
      at Ice.ObjectAdapterI.find(ObjectAdapterI.java:491)
      at omero.cmd.SessionI.unregisterServant(SessionI.java:633)
      at omero.cmd.SessionI.cleanServants(SessionI.java:449)
      at omero.cmd.SessionI.cleanServants(SessionI.java:379)
      at omero.cmd.SessionI.doDestroy(SessionI.java:369)
      at omero.cmd.SessionI.destroy(SessionI.java:297)
      at omero.api._ServiceFactoryTie.destroy(_ServiceFactoryTie.java:65)
      at Glacier2._SessionDisp.___destroy(_SessionDisp.java:94)
      at omero.api._ServiceFactoryDisp.__dispatch(_ServiceFactoryDisp.java:1263)
      at IceInternal.Incoming.invoke(Incoming.java:221)
      at Ice.ConnectionI.invokeAll(ConnectionI.java:2536)
      at Ice.ConnectionI.dispatch(ConnectionI.java:1145)
      at Ice.ConnectionI.message(ConnectionI.java:1056)
      at IceInternal.ThreadPool.run(ThreadPool.java:395)
      at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
      at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832)
      at java.lang.Thread.run(Thread.java:748)
   
!! 05/31/18 13:40:39.903 error: communicator not destroyed during global destruction.-! 05/31/18 13:41:58.994 OMERO.Glacier2: warning: unable to contact permissions verifier `BlitzVerifier@BlitzAdapters'
   Reference.cpp:1637: Ice::NoEndpointException:
   no suitable endpoint available for proxy `BlitzVerifier -t -e 1.1 @ BlitzAdapters'
-! 05/31/18 13:41:58.995 OMERO.Glacier2: warning: unable to contact session manager `BlitzManager@BlitzAdapters'
   Reference.cpp:1637: Ice::NoEndpointException:
   no suitable endpoint available for proxy `BlitzManager -t -e 1.1 @ BlitzAdapters'
!! 05/31/18 13:42:14.585 error: communicator not destroyed during global destruction.-! 6/13/18 13:01:20:907 warning: Blitz-0-Ice.ThreadPool.Server-182: dispatch exception:
   identity: session-fbf08c62-1b01-419e-8f92-c39324d80526/68a7066a-3718-46a4-90a7-729df9a9f42f
   facet:
   operation: destroy
   remote host: 130.83.98.66 remote port: 57554
   Ice.ObjectAdapterDeactivatedException
       name = "BlitzAdapter"
      at Ice.ObjectAdapterI.checkForDeactivation(ObjectAdapterI.java:1170)
      at Ice.ObjectAdapterI.findFacet(ObjectAdapterI.java:498)
      at Ice.ObjectAdapterI.find(ObjectAdapterI.java:491)
      at omero.cmd.SessionI.unregisterServant(SessionI.java:633)
      at omero.cmd.SessionI.cleanServants(SessionI.java:449)
      at omero.cmd.SessionI.cleanServants(SessionI.java:379)
      at omero.cmd.SessionI.cleanupSelf(SessionI.java:345)
      at omero.cmd.SessionI.destroy(SessionI.java:309)
      at omero.api._ServiceFactoryTie.destroy(_ServiceFactoryTie.java:65)
      at Glacier2._SessionDisp.___destroy(_SessionDisp.java:94)
      at omero.api._ServiceFactoryDisp.__dispatch(_ServiceFactoryDisp.java:1263)
      at IceInternal.Incoming.invoke(Incoming.java:221)
      at Ice.ConnectionI.invokeAll(ConnectionI.java:2536)
      at Ice.ConnectionI.dispatch(ConnectionI.java:1145)
      at Ice.ConnectionI.message(ConnectionI.java:1056)
      at IceInternal.ThreadPool.run(ThreadPool.java:395)
      at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
      at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832)
      at java.lang.Thread.run(Thread.java:748)
   
-! 6/13/18 13:01:21:014 warning: Blitz-0-Ice.ThreadPool.Server-189: dispatch exception:
   identity: session-5111dcce-b0c8-4b28-b174-dc2e44d656c3/68a7066a-3718-46a4-90a7-729df9a9f42f
   facet:
   operation: destroy
   remote host: 130.83.98.66 remote port: 57554
   Ice.ObjectAdapterDeactivatedException
       name = "BlitzAdapter"
      at Ice.ObjectAdapterI.checkForDeactivation(ObjectAdapterI.java:1170)
      at Ice.ObjectAdapterI.findFacet(ObjectAdapterI.java:498)
      at Ice.ObjectAdapterI.find(ObjectAdapterI.java:491)
      at omero.cmd.SessionI.unregisterServant(SessionI.java:633)
      at omero.cmd.SessionI.cleanServants(SessionI.java:449)
      at omero.cmd.SessionI.cleanServants(SessionI.java:379)
      at omero.cmd.SessionI.doDestroy(SessionI.java:369)
      at omero.cmd.SessionI.destroy(SessionI.java:297)
      at omero.api._ServiceFactoryTie.destroy(_ServiceFactoryTie.java:65)
      at Glacier2._SessionDisp.___destroy(_SessionDisp.java:94)
      at omero.api._ServiceFactoryDisp.__dispatch(_ServiceFactoryDisp.java:1263)
      at IceInternal.Incoming.invoke(Incoming.java:221)
      at Ice.ConnectionI.invokeAll(ConnectionI.java:2536)
      at Ice.ConnectionI.dispatch(ConnectionI.java:1145)
      at Ice.ConnectionI.message(ConnectionI.java:1056)
      at IceInternal.ThreadPool.run(ThreadPool.java:395)
      at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
      at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832)
      at java.lang.Thread.run(Thread.java:748)
   
!! 06/13/18 13:01:21.707 error: communicator not destroyed during global destruction.-! 06/13/18 13:02:39.667 OMERO.Glacier2: warning: unable to contact permissions verifier `BlitzVerifier@BlitzAdapters'
   Reference.cpp:1637: Ice::NoEndpointException:
   no suitable endpoint available for proxy `BlitzVerifier -t -e 1.1 @ BlitzAdapters'
-! 06/13/18 13:02:39.668 OMERO.Glacier2: warning: unable to contact session manager `BlitzManager@BlitzAdapters'
   Reference.cpp:1637: Ice::NoEndpointException:
   no suitable endpoint available for proxy `BlitzManager -t -e 1.1 @ BlitzAdapters'
!! 06/13/18 13:02:55.155 error: communicator not destroyed during global destruction.


Thanks again for your help.

Alex

[jmoore]

Odd,

Code: Select all

java.io.FileNotFoundException: /OMERO/ManagedRepository/.omero/repository/9f6b12af-2ef3-4071-9b64-6ab409058ee3/repo_uuid (Permission denied)


Do you have any ideas how the permissions on this file could have been changed? Move of directory? selinux activated?

~Josh

[alexr]

I did not change any permissions on the files nor implemented any selinux.

Owner of the file is my inplace user (the one with r/w permissions for the ManagedRepository) and the file access code is 775.

It also appears that repo_uuid has no content.

Best wishes Alex

[jmoore]

Hi Alex,

I would expect all files under .omero to belong to the OMERO server user and be writeable by it. I assume the group settings don't allow OMERO to modify the file? If so, that would explain the error, but now how things came to be this way.

~J.