Open Microscopy Environment

Hi,

I currently attach file to dataset using matlab code (similar to the one given on omero website). This code worked for years now. I recently updated to omero 5.2.5 and face a permission issue. I create the annotation from a tif file with the code below

Code: Select all

   iUpdate = session.getUpdateService(); % service used to write object
    fa= writeFileAnnotation(session, nameOfFile);


nameOfFile = '{matlab}_mask_2'
(this is a tif file)
However, when I try to link the annotation to the dataset (owned by Z, member of group C), I already did that on the very same dataset from the same user (let's call it J, within the same group C and owner of group C), and connected with group C as security context, I now get the error below. The command is

Code: Select all

link = linkAnnotation(session,fa,'dataset',str2num(candidateDatasetID));

By the way, I usually finish by

Code: Select all

iUpdate.saveObject(link);

but it is no longer mentioned on the website https://www.openmicroscopy.org/site/support/omero5.2/developers/Matlab.html. Does that mean that it is dispensable now ?

The created annotation looks to have different permission from psql even if I have no clue how to interpret the figure I got in the permission field (-8 now, -40 before). This is likely a change in the default behaviour, isn't it ?
I used command

Code: Select all

psql -h localhost -U omero omero -c "select * from annotation where id=237239"

indeed, annotation create in the exemple reads
discriminator | id | description | permissions | ns | version | timevalue | textvalue | boolvalue | termvalue | longvalue | doublevalue | creation_id | external_id | group_id | owner_id | update_id | file | name
---------------------+--------+-------------+-------------+----+---------+-----------+-----------+-----------+-----------+-----------+-------------+-------------+-------------+----------+----------+-----------+---------+------
/type/OriginalFile/ | 237239 | | -8 | | | | | | | | | 154466595 | | 3 | 2 | 154466595 | 1162142 |


An old annotation created by the same code reads
discriminator | id | description | permissions | ns | version | timevalue | textvalue | boolvalue | termvalue | longvalue | doublevalue | creation_id | external_id | group_id | owner_id | update_id | file | name
---------------------+--------+-------------+-------------+----+---------+-----------+-----------+-----------+-----------+-----------+-------------+-------------+-------------+----------+----------+-----------+---------+------
/type/OriginalFile/ | 235851 | | -40 | | | | | | | | | 66313652 | | 3 | 2 | 66313652 | 1011728 |

Could you help me to solve the weird permission issue ?
Thank you so much

Best

Jacques

------------------------- error obtained when linking the annotation ---------------------
Error using linkAnnotation (line 58)
Java exception occurred:
omero.SecurityViolation
serverStackTrace = "ome.conditions.SecurityViolation: Cannot read ome.model.annotations.FileAnnotation:Id_237239
at ome.security.basic.BasicACLVoter.throwLoadViolation(BasicACLVoter.java:167)
at ome.security.CompositeACLVoter.throwLoadViolation(CompositeACLVoter.java:91)
at ome.security.ACLEventListener.onPostLoad(ACLEventListener.java:102)
at org.hibernate.engine.TwoPhaseLoad.initializeEntity(TwoPhaseLoad.java:250)
at org.hibernate.loader.Loader.initializeEntitiesAndCollections(Loader.java:898)
at org.hibernate.loader.Loader.doQuery(Loader.java:773)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:270)
at org.hibernate.loader.Loader.loadEntity(Loader.java:1953)
at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:86)
at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:76)
at org.hibernate.persister.entity.AbstractEntityPersister.load(AbstractEntityPersister.java:3270)
at org.hibernate.event.def.DefaultLoadEventListener.loadFromDatasource(DefaultLoadEventListener.java:496)
at org.hibernate.event.def.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:477)
at org.hibernate.event.def.DefaultLoadEventListener.load(DefaultLoadEventListener.java:227)
at org.hibernate.event.def.DefaultLoadEventListener.proxyOrLoad(DefaultLoadEventListener.java:285)
at org.hibernate.event.def.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:152)
at org.hibernate.impl.SessionImpl.fireLoad(SessionImpl.java:1082)
at org.hibernate.impl.SessionImpl.get(SessionImpl.java:999)
at org.hibernate.event.def.DefaultMergeEventListener.entityIsDetached(DefaultMergeEventListener.java:453)
at ome.security.basic.MergeEventListener.entityIsDetached(MergeEventListener.java:184)
at org.hibernate.event.def.DefaultMergeEventListener.onMerge(DefaultMergeEventListener.java:255)
at ome.security.basic.MergeEventListener.onMerge(MergeEventListener.java:90)
at org.hibernate.impl.SessionImpl.fireMerge(SessionImpl.java:871)
at org.hibernate.impl.SessionImpl.merge(SessionImpl.java:853)
at org.hibernate.engine.CascadingAction$6.cascade(CascadingAction.java:279)
at org.hibernate.engine.Cascade.cascadeToOne(Cascade.java:392)
at org.hibernate.engine.Cascade.cascadeAssociation(Cascade.java:335)
at org.hibernate.engine.Cascade.cascadeProperty(Cascade.java:204)
at org.hibernate.engine.Cascade.cascade(Cascade.java:161)
at org.hibernate.event.def.AbstractSaveEventListener.cascadeBeforeSave(AbstractSaveEventListener.java:451)
at org.hibernate.event.def.DefaultMergeEventListener.mergeTransientEntity(DefaultMergeEventListener.java:336)
at org.hibernate.event.def.DefaultMergeEventListener.entityIsTransient(DefaultMergeEventListener.java:303)
at org.springframework.orm.hibernate3.support.IdTransferringMergeEventListener.entityIsTransient(IdTransferringMergeEventListener.java:59)
at ome.security.basic.MergeEventListener.entityIsTransient(MergeEventListener.java:157)
at org.hibernate.event.def.DefaultMergeEventListener.onMerge(DefaultMergeEventListener.java:258)
at ome.security.basic.MergeEventListener.onMerge(MergeEventListener.java:90)
at org.hibernate.event.def.DefaultMergeEventListener.onMerge(DefaultMergeEventListener.java:84)
at ome.security.basic.MergeEventListener.onMerge(MergeEventListener.java:76)
at org.hibernate.impl.SessionImpl.fireMerge(SessionImpl.java:861)
at org.hibernate.impl.SessionImpl.merge(SessionImpl.java:845)
at org.hibernate.impl.SessionImpl.merge(SessionImpl.java:849)
at ome.logic.UpdateImpl.internalMerge(UpdateImpl.java:274)
at ome.logic.UpdateImpl$2.run(UpdateImpl.java:121)
at ome.logic.UpdateImpl$2.run(UpdateImpl.java:118)
at ome.logic.UpdateImpl.doAction(UpdateImpl.java:310)
at ome.logic.UpdateImpl.doAction(UpdateImpl.java:302)
at ome.logic.UpdateImpl.saveAndReturnObject(UpdateImpl.java:118)
at sun.reflect.GeneratedMethodAccessor514.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at ome.security.basic.EventHandler.invoke(EventHandler.java:154)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:111)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy91.saveAndReturnObject(Unknown Source)
at sun.reflect.GeneratedMethodAccessor514.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at ome.security.basic.BasicSecurityWiring.invoke(BasicSecurityWiring.java:93)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at ome.services.blitz.fire.AopContextInitializer.invoke(AopContextInitializer.java:43)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy91.saveAndReturnObject(Unknown Source)
at sun.reflect.GeneratedMethodAccessor537.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at ome.services.blitz.util.IceMethodInvoker.invoke(IceMethodInvoker.java:172)
at ome.services.throttling.Callback.run(Callback.java:56)
at ome.services.throttling.InThreadThrottlingStrategy.callInvokerOnRawArgs(InThreadThrottlingStrategy.java:56)
at ome.services.blitz.impl.AbstractAmdServant.callInvokerOnRawArgs(AbstractAmdServant.java:140)
at ome.services.blitz.impl.UpdateI.saveAndReturnObject_async(UpdateI.java:62)
at sun.reflect.GeneratedMethodAccessor536.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at omero.cmd.CallContext.invoke(CallContext.java:78)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy92.saveAndReturnObject_async(Unknown Source)
at omero.api._IUpdateTie.saveAndReturnObject_async(_IUpdateTie.java:92)
at omero.api._IUpdateDisp.___saveAndReturnObject(_IUpdateDisp.java:180)
at omero.api._IUpdateDisp.__dispatch(_IUpdateDisp.java:351)
at IceInternal.Incoming.invoke(Incoming.java:222)
at Ice.ConnectionI.invokeAll(ConnectionI.java:2482)
at Ice.ConnectionI.dispatch(ConnectionI.java:1258)
at Ice.ConnectionI.message(ConnectionI.java:1213)
at IceInternal.ThreadPool.run(ThreadPool.java:321)
at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:693)
at java.lang.Thread.run(Thread.java:745)
"
serverExceptionClass = "ome.conditions.SecurityViolation"
message = "Cannot read ome.model.annotations.FileAnnotation:Id_237239"
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at java.lang.Class.newInstance(Unknown Source)
at IceInternal.BasicStream.createUserException(BasicStream.java:2615)
at IceInternal.BasicStream.access$300(BasicStream.java:12)
at IceInternal.BasicStream$EncapsDecoder10.throwException(BasicStream.java:3099)
at IceInternal.BasicStream.throwException(BasicStream.java:2077)
at IceInternal.Outgoing.throwUserException(Outgoing.java:538)
at omero.api._IUpdateDelM.saveAndReturnObject(_IUpdateDelM.java:260)
at omero.api.IUpdatePrxHelper.saveAndReturnObject(IUpdatePrxHelper.java:802)
at omero.api.IUpdatePrxHelper.saveAndReturnObject(IUpdatePrxHelper.java:780)

jacques2020 wrote:Hi,

Hi Jacques,

I currently attach file to dataset using matlab code (similar to the one given on omero website). This code worked for years now. I recently updated to omero 5.2.5 and face a permission issue.

Is it possible that the permissions on the group changed?

Could you help me to solve the weird permission issue ?
Thank you so much

Could you show us the relevant lines from `bin/omero group list`?

Cheers,
~Josh.

Hi Josh,

thank you for your answer.
I have not changed the group permission actively. It appear in the output of the command as I set it long ago:
3 | CeDRE | rwrw-- | False | 3 | 17

Any other idea ?
Thanks

Best

Jacques

What version of OMERO were you using before upgrading to 5.2.5?
Are you sure that the Annotation and the Dataset are in the same group?

See "Working in a different group" on that docs page:

Code: Select all

eventContext = session.getAdminService().getEventContext();
groupId = eventContext.groupId;


Will.

What version of OMERO were you using before upgrading to 5.2.5?

5.2.2

Are you sure that the Annotation and the Dataset are in the same group?

Group is 3 for the file annotation

Code: Select all

fa.getDetails().getGroup().getId().getValue()
ans =

     3

seesion is also opened on this group

Code: Select all

eventContext = session.getAdminService().getEventContext();
groupId = eventContext.groupId

groupId =

     3

But the dataset is in the generic user group although it appear in the CeDRE panel (group 3) in insight client. The owner of the dataset created it long ago and was member of CeDRE (group 3). The project containing the dataset is also with group=1. So I am confused. user (group 1) is a system group that cannot be displayed in insight. Indeed, Z who owns the dataset is not member of group user right now (it was created 5 years ago). Is that a part of the issue?

Code: Select all

ds=getDatasets(session,str2num(candidateDatasetID))

ds =

omero.model.DatasetI@16e22064

K>> ds.getDetails().getGroup().getId().getValue()

ans =

     1

Does that mean that all model with group=1 should be re-attributed to an other group. This may be a long work ! Is there an other way ?

Many thanks

Best

Jacques

Hi Jacques,

this is sufficiently complicated that I think I need a test to show me what's going on. My first attempt is:

Code: Select all

    def testUserGroupLinkage(self):
        admin = self.sf.getAdminService()
        update = self.sf.getUpdateService()
        ugid = admin.getSecurityRoles().userGroupId
        ogid = admin.getEventContext().groupId

        # (1) We start off by assuming ogid == 3 as in your example
        # -------------------------------------------------------------------

        # (2) Then create a file annotation in the user's current group
        # -------------------------------------------------------------------
        fa = FileAnnotationI()
        fa = update.saveAndReturnObject(fa)
        assert fa.details.group.id.val == ogid

        # (3) Then we create a dataset in the "user" group (id=1)
        # -------------------------------------------------------------------
        ds = DatasetI()
        ds.setName(rstring("testUserGroupLinkage"))
        ds = update.saveAndReturnObject(ds)
        assert ds.details.group.id.val == ogid
        self.root.sf.setSecurityContext(ExperimenterGroupI(ogid, False))
        self.root.sf.getAdminService().moveToCommonSpace([ds])

        # (4) Now we link the two together.
        # -------------------------------------------------------------------
        dal = DatasetAnnotationLinkI()
        dal.parent = ds.proxy()
        dal.child = fa.proxy()
        dal = update.saveAndReturnObject(dal)


Does that sound correct?
~Josh

Hi Josh,

reading the code, I would point out that the dataset belong to an user different from the one used to connect and owning the annotation. To clarify
dataset : user Z , group 1 (user), pre-existing
annotation : created by the code with user J, group 3 (CeDRE)
connection as user J, group 3 as security context.

I tried to use the code to reproduce the issue but I don't know what is "sf.root". I must say that I am not very fluent in omero.python. Below is what I did so far but it of course stopped when reaching sf.root.

Thank you for your help

Best

Jacques

-----------------

Code: Select all

from omero.gateway import *
from omero.model import *

class MyClass:
    def __init__(self):
      conn = MyBlitzGateway('J', '******', host='host', port=4064)
      connected = conn.connect()
      self.sf=conn
      self.root.sf=conn

    def testUserGroupLinkage(self):
            admin = self.sf.getAdminService()
            update = self.sf.getUpdateService()
            ugid = admin.getSecurityRoles().userGroupId
            ogid = admin.getEventContext().groupId

            # (1) We start off by assuming ogid == 3 as in your example
            # -------------------------------------------------------------------

            # (2) Then create a file annotation in the user's current group
            # -------------------------------------------------------------------
            fa = FileAnnotationI()
            fa = update.saveAndReturnObject(fa)
            assert fa.details.group.id.val == ogid

            # (3) Then we create a dataset in the "user" group (id=1)
            # -------------------------------------------------------------------
            ds = DatasetI()
            ds.setName(rstring("testUserGroupLinkage"))
            ds = update.saveAndReturnObject(ds)
            assert ds.details.group.id.val == ogid
            self.root.sf.setSecurityContext(ExperimenterGroupI(ogid, False))
            self.root.sf.getAdminService().moveToCommonSpace([ds])

            # (4) Now we link the two together.
            # -------------------------------------------------------------------
            dal = DatasetAnnotationLinkI()
            dal.parent = ds.proxy()
            dal.child = fa.proxy()
            dal = update.saveAndReturnObject(dal)

c = MyClass()
c.testUserGroupLinkage()



Thanks for the explanation. Here's my updated (but unfortunately still passing) code:

Code: Select all

    def testUserGroupLinkage(self):
        # See https://www.openmicroscopy.org/community/
        #     viewtopic.php?t=8106&p=17424#p17424

        # The default logged in user we'll consider user_j
        admin_j = self.sf.getAdminService()
        update_j = self.sf.getUpdateService()

        group_1 = admin_j.getSecurityRoles().userGroupId
        group_3 = admin_j.getEventContext().groupId

        user_z = self.new_client(group=group_3, owner=True)
        update_z = user_z.sf.getUpdateService()

        # (A) Create a FileAnnotation as user J in group 3
        # ------------------------------------------------
        fa_j3 = FileAnnotationI()
        fa_j3 = update_j.saveAndReturnObject(fa_j3)
        assert fa_j3.details.group.id.val == group_3

        # (B) Create a Dataset as user Z in group 1 (user)
        # ------------------------------------------------
        ds_z1 = DatasetI()
        ds_z1.setName(rstring("testUserGroupLinkage"))
        ds_z1 = update_z.saveAndReturnObject(ds_z1)
        assert ds_z1.details.group.id.val == group_3
        self.root.sf.setSecurityContext(ExperimenterGroupI(group_3, False))
        self.root.sf.getAdminService().moveToCommonSpace([ds_z1])

        # (C) & Link the two together as user J in group 3
        # ------------------------------------------------
        dal_j3 = DatasetAnnotationLinkI()
        dal_j3.parent = ds_z1.proxy()
        dal_j3.child = fa_j3.proxy()
        dal_j3 = update_j.saveAndReturnObject(dal_j3)



Looking at https://github.com/openmicroscopy/openmicroscopy/blob/v5.2.5/components/tools/OmeroM/src/annotations/linkAnnotation.m#L52, the issue, I believe, is that the MATLAB code assumes that the parent group is assumed to be correct whereas in your example it would be the child.

If I change the last line of my example to

Code: Select all

        dal_j3 = update_j.saveAndReturnObject(dal_j3,
                                              {"omero.group": str(group_1)})


then I get:

Code: Select all

E           serverExceptionClass = ome.conditions.SecurityViolation
E           message = Cannot read ome.model.annotations.FileAnnotation:Id_606
E       }


That being said, the linkAnnotations.m function hasn't changed for some time, so there may be something additional going on that I've missed.

Cheers,
~Josh

Hi Josh,

this is the group issue you pointed out.
If I modify the call to link annotation to \

Code: Select all

link = linkAnnotation_perso(session,fa,'dataset',str2num(candidateDatasetID),session.getAdminService().getEventContext().groupId);

with linked annotation perso mimmicking linkAnnotaiton except definition

Code: Select all

function link = linkAnnotation_perso(session, annotation, parentType, parentId,group)

and l52 replaced by

Code: Select all

if nargin<=4
    group = parent.getDetails().getGroup().getId().getValue();
end

then it works correctly.

For the python, I can have a try if you wish but I don't know what is sf.root in your code. (see my attempted class in the previous post). The bug might be matlab specific since attaching the annotation to the dataset within insight worked fine (annotation created in matlab, breakpoint on the line with linkAnnotation, retrieval in insight by ID, and attaching it from uploaded). So far, I can live with the work around but might also test the python if it helps you to fix the bug for future release.

Thank you very much for your help.

Best

Jacques

If I modify the call to link annotation to \

Code: Select all

link = linkAnnotation_perso(session,fa,'dataset',str2num(candidateDatasetID),session.getAdminService().getEventContext().groupId);

with linked annotation perso mimmicking linkAnnotaiton except definition

Code: Select all

function link = linkAnnotation_perso(session, annotation, parentType, parentId,group)

and l52 replaced by

Code: Select all

if nargin<=4
    group = parent.getDetails().getGroup().getId().getValue();
end

then it works correctly.

Excellent!

For the python, I can have a try if you wish but I don't know what is sf.root in your code. (see my attempted class in the previous post).

Sorry for not explaining. `self.root` is a part of the OMERO.py integration test framework. `self.client` is an automatically created user, specially for the current test and self.root is an admin that can be used for various tweaks. In your case, you'd want to create a new BlitzGateway as as admin user for that part.

The bug might be matlab specific since attaching the annotation to the dataset within insight worked fine (annotation created in matlab, breakpoint on the line with linkAnnotation, retrieval in insight by ID, and attaching it from uploaded). So far, I can live with the work around but might also test the python if it helps you to fix the bug for future release.

I think your fix to linkAnnotations.m is a correct one and it'd be worth integrating. I.e. this particular issue is matlab specific. The BlitzGateway in Python may have a very similar problem. If you have time to test it, it'd be very much appreciated.

Thank you very much for your help.

Gladly.
~Josh.