Moving Images between groups via sql

[jmoore]

Assuming you have a session key "ce70411e-ffa7-11e0-bbb3-58b035fbf9a7" then you can login via:

Code: Select all


HOST=localhost
SESS=ce70411e-ffa7-11e0-bbb3-58b035fbf9a7

bin/omero -s $HOST -k $SESS export -f Image1.ome.tif Image:1
bin/omero -s $HOST -k $SESS sessions group "MyOtherGroup"
bin/omero -s $HOST -k $SESS import Image1.ome.tif


Cheers,
~Josh

[Manz]

It worked! Excellent!
Thanks!

[Manz]

I'm finding the sessions are not being set, and it keeps repasting the images in the same group:-

Code: Select all

OMEROserver/OMERO.server-Beta-4.3.0/bin/omero -s localhost -k $SESS sessions group "M_public" Joined session 3f94ce4e-fd81-4eb4-9384-ad0409308a5c (M@localhost:4064). Idle timeout: 10.0 min. Current group: M_private
Traceback (most recent call last):
  File "home/OMEROserver/OMERO.server-Beta-4.3.0/bin/omero", line 123, in <module>
    rv = omero.cli.argv()
  File "/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omero/cli.py", line 1172, in argv
    cli.invoke(args[1:])
  File "home/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omero/cli.py", line 722, in invoke
    stop = self.onecmd(line, previous_args)
  File "home/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omero/cli.py", line 791, in onecmd
    self.execute(line, previous_args)
  File "home/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omero/cli.py", line 871, in execute
    args.func(args)
  File "home/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omero/plugins/sessions.py", line 359, in group
    sf.setSecurityContext(omero.model.ExperimenterGroupI(group_id, False))
  File "home/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omero_API_ice.py", line 198, in setSecurityContext
    return _M_omero.api.ServiceFactory._op_setSecurityContext.invoke(self, ((obj, ), _ctx))
omero.SecurityViolation: exception ::omero::SecurityViolation
{
    serverStackTrace = ome.conditions.SecurityViolation: SecurityContext change cancelled:
Client 731ceceb-d19f-47ee-ba1c-43d9ef9920d7 has active stateful services:

7f:0:1:1:-26a14d0a:13343e62852:-7ffbomero.api.ThumbnailStore
7f:0:1:1:-26a14d0a:13343e62852:-7ffaomero.api.RenderingEngine
Client 7034b8d5-34b1-4fee-b29c-ce85dab04cb8 has active stateful services:

7f:0:1:1:-26a14d0a:13343e62852:-7ffbomero.api.ThumbnailStore
7f:0:1:1:-26a14d0a:13343e62852:-7ffaomero.api.RenderingEngine
Client 8d3ea5fb-28e3-4872-915a-fb874dce75f6 has active stateful services:

7f:0:1:1:-26a14d0a:13343e62852:-7ffbomero.api.ThumbnailStore
7f:0:1:1:-26a14d0a:13343e62852:-7ffaomero.api.RenderingEngine
Client 3751e7a6-2fe3-4881-af22-536cef62acb8 has active stateful services:

7f:0:1:1:-26a14d0a:13343e62852:-7ffbomero.api.ThumbnailStore
7f:0:1:1:-26a14d0a:13343e62852:-7ffaomero.api.RenderingEngine

   at ome.services.sessions.events.ChangeSecurityContextEvent.throwIfCancelled(ChangeSecurityContextEvent.java:61)
   at ome.services.sessions.SessionManagerImpl.setSecurityContext(SessionManagerImpl.java:1114)
   at ome.services.blitz.impl.ServiceFactoryI.setSecurityContext(ServiceFactoryI.java:318)
   at omero.api._ServiceFactoryDisp.___setSecurityContext(_ServiceFactoryDisp.java:393)
   at omero.api._ServiceFactoryDisp.__dispatch(_ServiceFactoryDisp.java:1379)
   at IceInternal.Incoming.invoke(Incoming.java:159)
   at Ice.ConnectionI.invokeAll(ConnectionI.java:2037)
   at Ice.ConnectionI.message(ConnectionI.java:972)
   at IceInternal.ThreadPool.run(ThreadPool.java:577)
   at IceInternal.ThreadPool.access$100(ThreadPool.java:12)
   at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:971)

    serverExceptionClass = ome.conditions.SecurityViolation
    message = SecurityContext change cancelled:
Client 731ceceb-d19f-47ee-ba1c-43d9ef9920d7 has active stateful services:

7f:0:1:1:-26a14d0a:13343e62852:-7ffbomero.api.ThumbnailStore
7f:0:1:1:-26a14d0a:13343e62852:-7ffaomero.api.RenderingEngine
Client 7034b8d5-34b1-4fee-b29c-ce85dab04cb8 has active stateful services:

7f:0:1:1:-26a14d0a:13343e62852:-7ffbomero.api.ThumbnailStore
7f:0:1:1:-26a14d0a:13343e62852:-7ffaomero.api.RenderingEngine
Client 8d3ea5fb-28e3-4872-915a-fb874dce75f6 has active stateful services:

7f:0:1:1:-26a14d0a:13343e62852:-7ffbomero.api.ThumbnailStore
7f:0:1:1:-26a14d0a:13343e62852:-7ffaomero.api.RenderingEngine
Client 3751e7a6-2fe3-4881-af22-536cef62acb8 has active stateful services:

7f:0:1:1:-26a14d0a:13343e62852:-7ffbomero.api.ThumbnailStore
7f:0:1:1:-26a14d0a:13343e62852:-7ffaomero.api.RenderingEngine

}



[jmoore]

Hi Amanda,

are you using the session for anything else? I tested with an existing exception and saw the error you saw. But then I tried with a fresh session and got:

Code: Select all

~/git/dist $ bin/omero -C -s root@localhost login
Password:
Created session e896c698-483d-40d7-a93b-778f1dbb0420 (root@localhost:4064). Idle timeout: 10.0 min. Current group: system

~/git/dist $ bin/omero -s root@localhost export Image:251 -f /tmp/xkcd.ome.tif
Using session e896c698-483d-40d7-a93b-778f1dbb0420 (root@localhost:4064). Idle timeout: 10.0 min. Current group: system

~/git/dist $ bin/omero sessions group default
Using session e896c698-483d-40d7-a93b-778f1dbb0420 (root@localhost:4064). Idle timeout: 10.0 min. Current group: system
Group 'system' (id=0) switched to 'default' (id=203)

~/git/dist $ bin/omero import /tmp/xkcd.ome.tif
Using session e896c698-483d-40d7-a93b-778f1dbb0420 (root@localhost:4064). Idle timeout: 10.0 min. Current group: default
2011-10-27 10:03:56,895 0          [      main] INFO          ome.formats.importer.ImportConfig  - OMERO Version: 4.3.3
2011-10-27 10:03:56,900 5          [      main] INFO          ome.formats.importer.ImportConfig  - Bioformats version: 4.3.3-DEV revision: c21c787 date: 17 October 2011
2011-10-27 10:03:56,908 13         [      main] INFO   formats.importer.cli.CommandLineImporter  - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2011-10-27 10:03:57,203 308        [      main] INFO      ome.formats.importer.ImportCandidates  - Depth: 4 Metadata Level: MINIMUM
2011-10-27 10:03:58,248 1353       [      main] INFO      ome.formats.importer.ImportCandidates  - 1 file(s) parsed into 1 group(s) with 1 call(s) to setId in 1042ms. (1045ms total) [0 unknowns]
2011-10-27 10:04:01,276 4381       [      main] INFO       ome.formats.OMEROMetadataStoreClient  - Attempting initial SSL connection to localhost:4064
2011-10-27 10:04:01,805 4910       [      main] INFO       ome.formats.OMEROMetadataStoreClient  - Insecure connection requested, falling back
2011-10-27 10:04:02,007 5112       [      main] INFO       ome.formats.OMEROMetadataStoreClient  - Server: 4.3.3
2011-10-27 10:04:02,007 5112       [      main] INFO       ome.formats.OMEROMetadataStoreClient  - Client: Beta4.3.3
2011-10-27 10:04:02,007 5112       [      main] INFO       ome.formats.OMEROMetadataStoreClient  - Java Version: 1.6.0_26
2011-10-27 10:04:02,007 5112       [      main] INFO       ome.formats.OMEROMetadataStoreClient  - OS Name: Mac OS X
2011-10-27 10:04:02,007 5112       [      main] INFO       ome.formats.OMEROMetadataStoreClient  - OS Arch: x86_64
2011-10-27 10:04:02,007 5112       [      main] INFO       ome.formats.OMEROMetadataStoreClient  - OS Version: 10.6.8
2011-10-27 10:04:02,287 5392       [      main] INFO         ome.formats.importer.ImportLibrary  - File format: OME-TIFF
2011-10-27 10:04:02,287 5392       [      main] INFO         ome.formats.importer.ImportLibrary  - Base reader: loci.formats.in.OMETiffReader
2011-10-27 10:04:02,287 5392       [      main] INFO         ome.formats.importer.ImportLibrary  - Metadata only import? false
2011-10-27 10:04:02,287 5392       [      main] INFO         ome.formats.importer.ImportLibrary  - Archiving enabled? false
2011-10-27 10:04:02,287 5392       [      main] INFO         ome.formats.importer.ImportLibrary  - Container metadata only import? false
2011-10-27 10:04:02,288 5393       [      main] INFO         ome.formats.importer.ImportLibrary  - Reader is not of HCS domain, use metafile: true
2011-10-27 10:04:02,826 5931       [      main] INFO   ormats.importer.cli.LoggingImportMonitor  - IMPORT_DONE
Imported pixels:
253
2011-10-27 10:04:02,826 5931       [      main] INFO      ome.formats.importer.cli.ErrorHandler  - Number of errors: 0
~/git/dist $



Could you try a fresh session per import/export cycle?

Cheers,
~Josh.

[Manz]

I can get this to work if I get the session from the sessions table and manually command line it as below. Export works as well, and the image appears in the new group. Sometimes it takes a few goes to get the correct session and i've found the ones that usually work have useragent=Python service. It runs as well in a function called imageImExviaOmero (more on that below)

Code: Select all


OMERO.server-Beta-4.3.0/bin/omero -s localhost -k 960f3742-093d-4302-8b16-063d8f743014 sessions group "Manz_public"
Joined session 960f3742-093d-4302-8b16-063d8f743014 (root@localhost:4064). Idle timeout: 10.0 min. Current group: Manz_public
Group 'Manz_public' (id=3) is already active

OMEROserver/OMERO.server-Beta-4.3.0/bin/omero -s localhost -k 0cb10f5c-d52e-461c-968d-1981617bb9fc import /OMERO/DropBox/Manz/TMNT.giftransfer.ome.tif -d 152




But then when I put it in OMERO to run, I get issues. I'll paste my code and the issues below. I think that I am not getting the correct session, either the session is already being used or claims that it only shows read-only permission.

selectfromdb

Code: Select all

import re
import dbconnect,string
import subprocess
import shlex


def imageImExviaOmero(imgid,PubGroupName,PublicDsId):
   #copies image n reimports image into ds that's public
   #need to be logged into omero private group
   sQuery="select owner_id, name from image where id="
   ownerid=str(dbconnect.selectfromdb(sQuery,imgid)[0][0])
   extendedfilename=((str(dbconnect.selectfromdb(sQuery,imgid)[0][1]))).replace(" ","_")
   #get session info
   sQuery1=string.join(["select uuid FROM session WHERE useragent='Python service' ORDER BY started desc limit 1"],"")
#   sQuery1=string.join(["select uuid FROM session WHERE owner=",ownerid," AND useragent='Python service' ORDER BY closed desc limit 1"],"")
   session=str(dbconnect.selectfromdb(sQuery1,"")[0][0])
   omerohomedir="/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0"
   exportstr=string.join([omerohomedir,"/bin/omero -s localhost -k ",session," export --file ",extendedfilename,"transfer.ome.tif Image:",imgid],"")
   #need to import using
   changesessstr=string.join([omerohomedir,'/bin/omero -s localhost -k ',session,' sessions group "',PubGroupName,'"'],"")
   importstr=string.join([omerohomedir,"/bin/omero -s localhost -k ",session," import ",extendedfilename,"transfer.ome.tif -d ",PublicDsId],"")
   a=subprocess.call([exportstr],shell=True)
   a=subprocess.call([changesessstr],shell=True)
   a=subprocess.call([importstr],shell=True)
   print (exportstr)
   print (changesessstr)
   print (importstr)
   print subprocess.call(["whoami"],shell=True)
   return (string.join([extendedfilename,"transfer.ome.tif"],""))



Code: Select all

def mvImgtoDsPublicUsingexim(imgId):
   #Mvs image to its corresponding public dataset
   import string
   import ImageImExviaOmero
   #get img ds n group
   #get ds n group for public ds
   sQuery="select c.child, a.id, a.permissions,a.group_id, b.name, a.name from dataset a JOIN experimentergroup b ON a.group_id=b.id JOIN datasetimagelink c ON c.parent=a.id WHERE c.child="
   imgId=re.sub("[^0-9]","",imgId)
   origImgInfo=selectfromdb(sQuery,imgId)
   publicGroup=re.sub("_private","_public",origImgInfo[0][4])
   origDsName=origImgInfo[0][5]
   sQuery1=string.join(["select  a.id, a.permissions,a.group_id,b.name, a.name from dataset a JOIN experimentergroup b ON a.group_id=b.id LEFT JOIN datasetimagelink c ON c.parent=a.id  WHERE b.name='",publicGroup,"' AND a.name='",origDsName,"'"],"")
   publicData=uniq(selectfromdb(sQuery1,""))
   publicDsId=str(publicData[0][0])
   publicPermiss="-39"
   publicGroupId=str(publicData[0][2])
   publicDsId=str(duplicatedsQuery(imgId)[0][0])
   #export the image
   #move image to new dataset- done by imageImExviaOmero, just need to get the publicDsIdvis
   newfilename=ImageImExviaOmero.imageImExviaOmero(imgId,publicGroup,publicDsId)
   print newfilename
        ....
   return string.join(["The image ",imgId," has been moved to the group ", publicGroupId, " with permissions " ,publicPermiss],"")


I'm not sure whether to get a fresh session I need to do something different with conn=kwargs['conn']

Code: Select all

@isUserConnected    # wrapper handles login (or redirects to webclient login). Connection passed in **kwargs
def publishimage(request, **kwargs):
   imgid=request.POST['imageIds']
   conn = kwargs['conn']
   from django.template import Context, Template
   import dbconnect,string
   #things to do to prepare for image publication
   #create dataset in corresponding public group
   dsresults=dbconnect.duplicatedsQuery(imgid)
   #will return a public dataset if exists that corresponds with a private on of id dsid
   #find out what ds it sits in

   dbconnect.mvImgtoDsPublicUsingexim(imgid)
   return HttpResponse("Your image has been published and moved to Public.")



If I leave Python service in as useragent

select uuid FROM session WHERE useragent='Python service' ORDER BY started desc limit 1"

Code: Select all

select a.id, a.name from dataset a JOIN experimentergroup b ON a.group_id=b.id WHERE a.group_id=3 AND a.name='Dataset2Private'
already exists
Publishing collection
select a.id, a.name from dataset a JOIN experimentergroup b ON a.group_id=b.id WHERE a.group_id=3 AND a.name='Dataset2Private'
already exists
usage: /home/manzomero/OMEROserver/OMERO.server export [-h] -f FILE
                                                       [-t {TIFF,XML}]
                                                       [--iterate]
                                                       obj
/home/manzomero/OMEROserver/OMERO.server export: error: argument -f/--file: invalid NewFileType('wb') value: '/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omeroweb/webtest/transfer2.301.ome.tiftransfer.ome.tif'
Bad session key
Bad session key
/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/bin/omero -s localhost -k b19ff66b-4acb-489e-a4a0-579386eb13a0 export --file /home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omeroweb/webtest/transfer2.301.ome.tiftransfer.ome.tif Image:303
/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/bin/omero -s localhost -k b19ff66b-4acb-489e-a4a0-579386eb13a0 sessions group "Manz_public"
/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/bin/omero -s localhost -k b19ff66b-4acb-489e-a4a0-579386eb13a0 import /home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omeroweb/webtest/transfer2.301.ome.tiftransfer.ome.tif -d 152
manzomero
0
/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omeroweb/webtest/transfer2.301.ome.tiftransfer.ome.tif



Otherwise if I leave it as:-
"select uuid FROM session WHERE owner=",ownerid," ORDER BY started desc limit 1"

I know export isn't working as there's already a file with the same name in that location,

Code: Select all

[28/Oct/2011 06:01:14] "GET /webclient/render_thumbnail/639/ HTTP/1.1" 200 3259
select a.id, a.name from dataset a JOIN experimentergroup b ON a.group_id=b.id WHERE a.group_id=3 AND a.name='Dataset2Private'
already exists
Publishing collection
select a.id, a.name from dataset a JOIN experimentergroup b ON a.group_id=b.id WHERE a.group_id=3 AND a.name='Dataset2Private'
already exists
usage: /home/manzomero/OMEROserver/OMERO.server export [-h] -f FILE
                                                       [-t {TIFF,XML}]
                                                       [--iterate]
                                                       obj
/home/manzomero/OMEROserver/OMERO.server export: error: argument -f/--file: invalid NewFileType('wb') value: '/OMERO/DropBox/Manz/Tmnt01.giftransfer.ome.tif'
Joined session 3b2c2401-8bb5-4329-bfd2-2dac6d996601 (Manz@localhost:4064). Idle timeout: 10.0 min. Current group: Manz_private
Traceback (most recent call last):
  File "/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/bin/omero", line 123, in <module>
    rv = omero.cli.argv()
  File "/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omero/cli.py", line 1172, in argv
    cli.invoke(args[1:])
  File "/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omero/cli.py", line 722, in invoke
    stop = self.onecmd(line, previous_args)
  File "/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omero/cli.py", line 791, in onecmd
    self.execute(line, previous_args)
  File "/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omero/cli.py", line 871, in execute
    args.func(args)
  File "/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omero/plugins/sessions.py", line 359, in group
    sf.setSecurityContext(omero.model.ExperimenterGroupI(group_id, False))
  File "/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/lib/python/omero_API_ice.py", line 198, in setSecurityContext
    return _M_omero.api.ServiceFactory._op_setSecurityContext.invoke(self, ((obj, ), _ctx))
omero.SecurityViolation: exception ::omero::SecurityViolation
{
    serverStackTrace = ome.conditions.SecurityViolation: SecurityContext change cancelled:
Client fbfef3e2-b404-404a-b44b-93ec2d77e0bd has active stateful services:

7f:0:1:1:-627648d0:13348d65098:-7ff4omero.api.ThumbnailStore
7f:0:1:1:-627648d0:13348d65098:-7ff5omero.api.RenderingEngine
Client 144010a9-3f16-4a2e-b5e5-edda26ed92b4 has active stateful services:

7f:0:1:1:-627648d0:13348d65098:-7ff4omero.api.ThumbnailStore
7f:0:1:1:-627648d0:13348d65098:-7ff5omero.api.RenderingEngine

   at ome.services.sessions.events.ChangeSecurityContextEvent.throwIfCancelled(ChangeSecurityContextEvent.java:61)
   at ome.services.sessions.SessionManagerImpl.setSecurityContext(SessionManagerImpl.java:1114)
   at ome.services.blitz.impl.ServiceFactoryI.setSecurityContext(ServiceFactoryI.java:318)
   at omero.api._ServiceFactoryDisp.___setSecurityContext(_ServiceFactoryDisp.java:393)
   at omero.api._ServiceFactoryDisp.__dispatch(_ServiceFactoryDisp.java:1379)
   at IceInternal.Incoming.invoke(Incoming.java:159)
   at Ice.ConnectionI.invokeAll(ConnectionI.java:2037)
   at Ice.ConnectionI.message(ConnectionI.java:972)
   at IceInternal.ThreadPool.run(ThreadPool.java:577)
   at IceInternal.ThreadPool.access$100(ThreadPool.java:12)
   at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:971)

    serverExceptionClass = ome.conditions.SecurityViolation
    message = SecurityContext change cancelled:
Client fbfef3e2-b404-404a-b44b-93ec2d77e0bd has active stateful services:

7f:0:1:1:-627648d0:13348d65098:-7ff4omero.api.ThumbnailStore
7f:0:1:1:-627648d0:13348d65098:-7ff5omero.api.RenderingEngine
Client 144010a9-3f16-4a2e-b5e5-edda26ed92b4 has active stateful services:

7f:0:1:1:-627648d0:13348d65098:-7ff4omero.api.ThumbnailStore
7f:0:1:1:-627648d0:13348d65098:-7ff5omero.api.RenderingEngine

}
Joined session 3b2c2401-8bb5-4329-bfd2-2dac6d996601 (Manz@localhost:4064). Idle timeout: 10.0 min. Current group: Manz_private
2011-10-28 06:01:23,171 5          [      main] INFO          ome.formats.importer.ImportConfig  - OMERO Version: 4.3.0-77345008
2011-10-28 06:01:23,178 12         [      main] INFO          ome.formats.importer.ImportConfig  - Bioformats version: 4.3.0 revision: b0a8381 date: 11 June 2011
2011-10-28 06:01:23,280 114        [      main] INFO   formats.importer.cli.CommandLineImporter  - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2011-10-28 06:01:24,615 1449       [      main] INFO      ome.formats.importer.ImportCandidates  - Depth: 4 Metadata Level: MINIMUM
2011-10-28 06:01:30,900 7734       [      main] INFO      ome.formats.importer.ImportCandidates  - 1 file(s) parsed into 1 group(s) with 1 call(s) to setId in 6261ms. (6285ms total) [0 unknowns]
2011-10-28 06:01:31,576 8410       [      main] INFO       ome.formats.OMEROMetadataStoreClient  - Attempting initial SSL connection to localhost:4064
2011-10-28 06:01:35,023 11857      [      main] INFO       ome.formats.OMEROMetadataStoreClient  - Insecure connection requested, falling back
2011-10-28 06:01:41,105 17939      [      main] WARN                    ome.system.UpgradeCheck  - UPGRADE AVAILABLE:Please upgrade to Beta-4.3.3 See http://trac.openmicroscopy.org.uk/omero for the latest version
2011-10-28 06:01:41,231 18065      [      main] INFO       ome.formats.OMEROMetadataStoreClient  - Server: 4.3.0
2011-10-28 06:01:41,231 18065      [      main] INFO       ome.formats.OMEROMetadataStoreClient  - Client: Dev Build
2011-10-28 06:01:41,231 18065      [      main] INFO       ome.formats.OMEROMetadataStoreClient  - Java Version: 1.6.0_22
2011-10-28 06:01:41,231 18065      [      main] INFO       ome.formats.OMEROMetadataStoreClient  - OS Name: Linux
2011-10-28 06:01:41,231 18065      [      main] INFO       ome.formats.OMEROMetadataStoreClient  - OS Arch: amd64
2011-10-28 06:01:41,231 18065      [      main] INFO       ome.formats.OMEROMetadataStoreClient  - OS Version: 2.6.38-8-generic
2011-10-28 06:01:41,345 18179      [      main] ERROR  formats.importer.cli.CommandLineImporter  - Error during import process.
java.lang.RuntimeException: omero.SecurityViolation
    serverStackTrace = "ome.conditions.SecurityViolation: Cannot read ome.model.containers.Dataset:Id_152
                           at ome.security.basic.BasicACLVoter.throwLoadViolation(BasicACLVoter.java:112)
                           at ome.security.CompositeACLVoter.throwLoadViolation(CompositeACLVoter.java:86)
                           at ome.security.ACLEventListener.onPostLoad(ACLEventListener.java:104)
                           at org.hibernate.engine.TwoPhaseLoad.initializeEntity(TwoPhaseLoad.java:250)
                           at org.hibernate.loader.Loader.initializeEntitiesAndCollections(Loader.java:898)
                           at org.hibernate.loader.Loader.doQuery(Loader.java:773)
                           at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:270)
                           at org.hibernate.loader.Loader.loadEntity(Loader.java:1953)
                           at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:86)
                           at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:76)
                           at org.hibernate.persister.entity.AbstractEntityPersister.load(AbstractEntityPersister.java:3270)
                           at org.hibernate.event.def.DefaultLoadEventListener.loadFromDatasource(DefaultLoadEventListener.java:496)
                           at org.hibernate.event.def.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:477)
                           at org.hibernate.event.def.DefaultLoadEventListener.load(DefaultLoadEventListener.java:227)
                           at org.hibernate.event.def.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:147)
                           at org.hibernate.impl.SessionImpl.fireLoad(SessionImpl.java:1082)
                           at org.hibernate.impl.SessionImpl.immediateLoad(SessionImpl.java:1020)
                           at org.hibernate.proxy.AbstractLazyInitializer.initialize(AbstractLazyInitializer.java:176)
                           at org.hibernate.Hibernate.initialize(Hibernate.java:428)
                           at ome.logic.QueryImpl$1.doInHibernate(QueryImpl.java:174)
                           at org.springframework.orm.hibernate3.HibernateTemplate.doExecute(HibernateTemplate.java:406)
                           at org.springframework.orm.hibernate3.HibernateTemplate.execute(HibernateTemplate.java:339)
                           at ome.logic.QueryImpl.get(QueryImpl.java:157)
                           at sun.reflect.GeneratedMethodAccessor290.invoke(Unknown Source)
                           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                           at java.lang.reflect.Method.invoke(Method.java:616)
                           at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
                           at ome.security.basic.EventHandler.invoke(EventHandler.java:150)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                           at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:111)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                           at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                           at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:231)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                           at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:116)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                           at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
                           at $Proxy75.get(Unknown Source)
                           at sun.reflect.GeneratedMethodAccessor290.invoke(Unknown Source)
                           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                           at java.lang.reflect.Method.invoke(Method.java:616)
                           at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
                           at ome.security.basic.BasicSecurityWiring.invoke(BasicSecurityWiring.java:83)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                           at ome.services.blitz.fire.AopContextInitializer.invoke(AopContextInitializer.java:43)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                           at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
                           at $Proxy75.get(Unknown Source)
                           at sun.reflect.GeneratedMethodAccessor684.invoke(Unknown Source)
                           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                           at java.lang.reflect.Method.invoke(Method.java:616)
                           at ome.services.blitz.util.IceMethodInvoker.invoke(IceMethodInvoker.java:179)
                           at ome.services.throttling.Callback.run(Callback.java:56)
                           at ome.services.throttling.InThreadThrottlingStrategy.callInvokerOnRawArgs(InThreadThrottlingStrategy.java:56)
                           at ome.services.blitz.impl.AbstractAmdServant.callInvokerOnRawArgs(AbstractAmdServant.java:136)
                           at ome.services.blitz.impl.QueryI.get_async(QueryI.java:110)
                           at omero.api._IQueryTie.get_async(_IQueryTie.java:127)
                           at omero.api._IQueryDisp.___get(_IQueryDisp.java:170)
                           at omero.api._IQueryDisp.__dispatch(_IQueryDisp.java:516)
                           at IceInternal.Incoming.invoke(Incoming.java:159)
                           at Ice.ConnectionI.invokeAll(ConnectionI.java:2037)
                           at Ice.ConnectionI.message(ConnectionI.java:972)
                           at IceInternal.ThreadPool.run(ThreadPool.java:577)
                           at IceInternal.ThreadPool.access$100(ThreadPool.java:12)
                           at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:971)
                        "
    serverExceptionClass = "ome.conditions.SecurityViolation"
    message = "Cannot read ome.model.containers.Dataset:Id_152"
   at ome.formats.OMEROMetadataStoreClient.getTarget(OMEROMetadataStoreClient.java:2208)
   at ome.formats.importer.ImportLibrary.importCandidates(ImportLibrary.java:216)
   at ome.formats.importer.cli.CommandLineImporter.start(CommandLineImporter.java:141)
   at ome.formats.importer.cli.CommandLineImporter.main(CommandLineImporter.java:386)
Caused by: omero.SecurityViolation
    serverStackTrace = "ome.conditions.SecurityViolation: Cannot read ome.model.containers.Dataset:Id_152
                           at ome.security.basic.BasicACLVoter.throwLoadViolation(BasicACLVoter.java:112)
                           at ome.security.CompositeACLVoter.throwLoadViolation(CompositeACLVoter.java:86)
                           at ome.security.ACLEventListener.onPostLoad(ACLEventListener.java:104)
                           at org.hibernate.engine.TwoPhaseLoad.initializeEntity(TwoPhaseLoad.java:250)
                           at org.hibernate.loader.Loader.initializeEntitiesAndCollections(Loader.java:898)
                           at org.hibernate.loader.Loader.doQuery(Loader.java:773)
                           at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:270)
                           at org.hibernate.loader.Loader.loadEntity(Loader.java:1953)
                           at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:86)
                           at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:76)
                           at org.hibernate.persister.entity.AbstractEntityPersister.load(AbstractEntityPersister.java:3270)
                           at org.hibernate.event.def.DefaultLoadEventListener.loadFromDatasource(DefaultLoadEventListener.java:496)
                           at org.hibernate.event.def.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:477)
                           at org.hibernate.event.def.DefaultLoadEventListener.load(DefaultLoadEventListener.java:227)
                           at org.hibernate.event.def.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:147)
                           at org.hibernate.impl.SessionImpl.fireLoad(SessionImpl.java:1082)
                           at org.hibernate.impl.SessionImpl.immediateLoad(SessionImpl.java:1020)
                           at org.hibernate.proxy.AbstractLazyInitializer.initialize(AbstractLazyInitializer.java:176)
                           at org.hibernate.Hibernate.initialize(Hibernate.java:428)
                           at ome.logic.QueryImpl$1.doInHibernate(QueryImpl.java:174)
                           at org.springframework.orm.hibernate3.HibernateTemplate.doExecute(HibernateTemplate.java:406)
                           at org.springframework.orm.hibernate3.HibernateTemplate.execute(HibernateTemplate.java:339)
                           at ome.logic.QueryImpl.get(QueryImpl.java:157)
                           at sun.reflect.GeneratedMethodAccessor290.invoke(Unknown Source)
                           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                           at java.lang.reflect.Method.invoke(Method.java:616)
                           at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
                           at ome.security.basic.EventHandler.invoke(EventHandler.java:150)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                           at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:111)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                           at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                           at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:231)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                           at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:116)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                           at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
                           at $Proxy75.get(Unknown Source)
                           at sun.reflect.GeneratedMethodAccessor290.invoke(Unknown Source)
                           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                           at java.lang.reflect.Method.invoke(Method.java:616)
                           at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
                           at ome.security.basic.BasicSecurityWiring.invoke(BasicSecurityWiring.java:83)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                           at ome.services.blitz.fire.AopContextInitializer.invoke(AopContextInitializer.java:43)
                           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
                           at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
                           at $Proxy75.get(Unknown Source)
                           at sun.reflect.GeneratedMethodAccessor684.invoke(Unknown Source)
                           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                           at java.lang.reflect.Method.invoke(Method.java:616)
                           at ome.services.blitz.util.IceMethodInvoker.invoke(IceMethodInvoker.java:179)
                           at ome.services.throttling.Callback.run(Callback.java:56)
                           at ome.services.throttling.InThreadThrottlingStrategy.callInvokerOnRawArgs(InThreadThrottlingStrategy.java:56)
                           at ome.services.blitz.impl.AbstractAmdServant.callInvokerOnRawArgs(AbstractAmdServant.java:136)
                           at ome.services.blitz.impl.QueryI.get_async(QueryI.java:110)
                           at omero.api._IQueryTie.get_async(_IQueryTie.java:127)
                           at omero.api._IQueryDisp.___get(_IQueryDisp.java:170)
                           at omero.api._IQueryDisp.__dispatch(_IQueryDisp.java:516)
                           at IceInternal.Incoming.invoke(Incoming.java:159)
                           at Ice.ConnectionI.invokeAll(ConnectionI.java:2037)
                           at Ice.ConnectionI.message(ConnectionI.java:972)
                           at IceInternal.ThreadPool.run(ThreadPool.java:577)
                           at IceInternal.ThreadPool.access$100(ThreadPool.java:12)
                           at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:971)
                        "
    serverExceptionClass = "ome.conditions.SecurityViolation"
    message = "Cannot read ome.model.containers.Dataset:Id_152"
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
   at java.lang.Class.newInstance0(Class.java:372)
   at java.lang.Class.newInstance(Class.java:325)
   at IceInternal.BasicStream$DynamicUserExceptionFactory.createAndThrow(BasicStream.java:2243)
   at IceInternal.BasicStream.throwException(BasicStream.java:1632)
   at IceInternal.Outgoing.throwUserException(Outgoing.java:442)
   at omero.api._IQueryDelM.get(_IQueryDelM.java:547)
   at omero.api.IQueryPrxHelper.get(IQueryPrxHelper.java:660)
   at omero.api.IQueryPrxHelper.get(IQueryPrxHelper.java:632)
   at ome.formats.OMEROMetadataStoreClient.getTarget(OMEROMetadataStoreClient.java:2204)
   ... 3 more
/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/bin/omero -s localhost -k 3b2c2401-8bb5-4329-bfd2-2dac6d996601 export --file /OMERO/DropBox/Manz/Tmnt01.giftransfer.ome.tif Image:686
/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/bin/omero -s localhost -k 3b2c2401-8bb5-4329-bfd2-2dac6d996601 sessions group "Manz_public"
/home/manzomero/OMEROserver/OMERO.server-Beta-4.3.0/bin/omero -s localhost -k 3b2c2401-8bb5-4329-bfd2-2dac6d996601 import /OMERO/DropBox/Manz/Tmnt01.giftransfer.ome.tif -d 152



[jmoore]

I can get this to work if I get the session from the sessions table and manually command line it as below. Export works as well, and the image appears in the new group. Sometimes it takes a few goes to get the correct session and i've found the ones that usually work have useragent=Python service. It runs as well in a function called imageImExviaOmero (more on that below)

Oh. Ok. I didn't realize that is what you were doing. That's definitely not likely to work reliably since those sessions belong to other clients/services and so could be doing ANYTHING, including changing their groups. I.e. you'd have a race condition.

In general, accessing the OMERO DB directly is not supported, since we make no guarantees on what could be going on in there. Probably your best bet at the moment is to hard-code your password into a script like those on the mainteance page. Ticket 6957 intends to add passwordless login for admins, but that's not in place yet.

You might also want to take a look at this email thread for a similar problem.

Cheers,
~Josh.

[Manz]

I've worked out that I could include the following lines just above my function and get a new session... i think anyway. It works 90% of the time, sometimes I have to relogin

Code: Select all

   conn = getBlitzConnection(request, useragent="OMERO.web")
   active_group = request.REQUEST.get('active_group')


By the way, what's a 'race condition'?

[jmoore]

A race condition is a situation in which the outcome is dependent on the order in which actions take place. For example, while using someone else's session, if the session owner did nothing while you were trying to export and import, then things may have worked fine. But if the session owner also tried to call "group set", for example, then the two of you would be racing to be the most recent call since that would determine the group which the image was imported to.

If you can figure out under what conditions you're having to re-login (the 10%) please let us know.

Cheers,
~Josh.

[Manz]

Hey Guys!

We are looking at the next phase of our project, and I am hoping to incorporate the chgrp component into the work we have done so far. You mentioned the following link and that you were hoping to get this into Omero server in the future. Does it have a rough ETA yet?
Otherwise.... I had a look at the link below and (not being familiar with git) are unsure how to incorporate this with our current server install. Is it just a case of identifying different files and copying them across? I noticed a few of the folders there aren't in our current install. (Sorry, it is a bit of a noob question)

Our server version is 4.3.1-6d4b8c61-b2602

https://github.com/joshmoore/openmicros ... components

[jmoore]

We're just starting to evaluate that branch ourselves, but none of the clients will support the new chgrp method for some time (1-2 months). There is a command-line version which you could test if you wanted, but you shouldn't do so on your production data.

To do that: make a copy of your binary data and of your database following the instructions under https://www.openmicroscopy.org/site/support/omero4/server/backup-and-restore. Then checkout that git branch and build from source:

Code: Select all

./build.py


The dist/ directory contains a full build just like you would download for 4.3.1. In there, you can configure your server:

Code: Select all

bin/omero config ...


Or copy over your configuration file:

Code: Select all

cp OLD_DIRECTORY/etc/grid/config.xml dist/etc/grid/config.xml


Note: if you plan on keeping your original server running, you will need to change the configuration to point at the separate directory, and change the ports. The "easiest" thing to do would be to evaluate this code on a test box rather than your production server.

Once all of that is set, start it up with bin/omero admin start

Cheers,
~Josh