statistics/serverside computations on Pixels

[achessel]

Hi all,

A bit of a simple question but just in case: are simple image statistics (ie mean, max, min, median, mode...) stored somewhere serverside? Or the histogram? Or a way to compute them without having to download the whole thing? And/or same question for a z-projection? I could not find anything in the docs but it would make my life so much easier if it were the case that I thought I would ask

Basically I would need those numbers, but for several Tb worth of plates, so just downloading them would take forever...

Many thanks

[manics]

If you imported the images with the default options they'll have the min and max calculated automatically for display purposes:

Code: Select all

bin/omero hql 'select c,i.id,s.globalMin,s.globalMax from Channel c join fetch c.statsInfo s join fetch c.pixels p join fetch p.image i where i.id=101'
Using session 2d903c3e-54e1-488e-aaab-4a7bd300e028 (spli@localhost:4064). Idle timeout: 10 min. Current group: data_repo
# | Col1         | Col2 | Col3 | Col4 
---+--------------+------+------+-------
0 | ChannelI:101 | 101  | 0.0  | 255.0
1 | ChannelI:102 | 101  | 0.0  | 255.0
2 | ChannelI:103 | 101  | 0.0  | 255.0
(3 rows)


I don't think there's an easy way to get other stats, or per-plane stats, someone else might know better though.

Simon

[achessel]

Yes, I think that's what I was dimly remembering and looking for, thanks for pointing it out. It may not be enough for my purpose though, I would need at least the mean...

Thanks

[wmoore]

Hi,

I'm afraid if you want average values you're going to have to read all the pixel data for each plane.
Probably the most efficient way of doing this is to use a server-side python script (to avoid moving data very far). See the examples at:

http://www.openmicroscopy.org/site/supp ... ata-access

If you use getPlanes() then this only opens the raw file store once per image (instead of per plane) and provides a generator of planes, so you never have all the data in hand at once.
E.g.

Code: Select all

zctList = []
for z in range(sizeZ):
    for c in range(sizeC):
        for t in range(sizeT):
            zctList.append((z, c, t))

for p in pixels.getPlanes(zctList):
    print p.mean()


You can use an existing script, or an example like
https://github.com/openmicroscopy/openm ... m_Image.py
as a basis for getting started.

Hope that helps.

Will.

[achessel]

Thanks, that's the conclusion I was reaching as well.

Is there not a getStack in OMERO.python? I am more use to the Matlab bindings... A generator of planes does work though (and is more pythonic maybe? )

[manics]

I don't think there is. You could easily write a wrapper in Python to concatenate the planes from the generator, and if we were to implement it in our API that's probably what we'd end up doing anyway, since Ice has a maximum object size (around 64MB by default in OMERO).

Edit: Just taken a look and there's something in the low level RawPixelsStore- so you could reimplement OMERO.matlab's getStack.m in Python

[achessel]

A bit of a tangent, but while I am here: the hql query to get min and max above works well on the CLI but fails in matlab (I am still on 5.0 if that helps):

Code: Select all

q=['select c,i.id,s.globalMin,s.globalMax from Channel c join fetch c.statsInfo s join fetch c.pixels p join fetch p.image i where i.id= ' num2str(imId) ]

q =

select c,i.id,s.globalMin,s.globalMax from Channel c join fetch c.statsInfo s join fetch c.pixels p join fetch p.image i where i.id= 129816

>> mMs = query.findAllByQuery(q,[]);
Java exception occurred:
Ice.MarshalException
    reason = (null)
   at IceInternal.BasicStream.createUserException(BasicStream.java:2620)
   at IceInternal.BasicStream.access$300(BasicStream.java:12)
   at IceInternal.BasicStream$EncapsDecoder10.throwException(BasicStream.java:3099)
   at IceInternal.BasicStream.throwException(BasicStream.java:2077)
   at IceInternal.Outgoing.throwUserException(Outgoing.java:538)
   at omero.api._IQueryDelM.findAllByQuery(_IQueryDelM.java:276)
   at omero.api.IQueryPrxHelper.findAllByQuery(IQueryPrxHelper.java:769)
   at omero.api.IQueryPrxHelper.findAllByQuery(IQueryPrxHelper.java:741)
Caused by: Ice.MarshalException
    reason = "expected type id but received `'"
   at IceInternal.BasicStream.typeToClass(BasicStream.java:2734)
   at IceInternal.BasicStream.findClass(BasicStream.java:2646)
   at IceInternal.BasicStream.createUserException(BasicStream.java:2612)
   ... 7 more

Another tangent while I am still here: how to get the index of the channel (ie the coordinate in the c axis) in hql in the query above? Or alternatively where is stored the laser wavelength, which I can see is set in the acquisition panel in insight? I checked lightSource.wavelength but it seemed unset...

Many thanks

[sbesson]

Hi Anatole,

answering your first question, the reason is that the hql CLI plugin is using queryService.projection. If you want yo use findAllbyQuery in your MATLAB client, you could do the following

Code: Select all

>> c=q.findAllByQuery(['select c from Channel c join fetch c.statsInfo as s left join fetch c.pixels as p left outer join fetch p.image as i where i.id = ' num2str(iId)], [])

c =

[omero.model.ChannelI@26149e4, omero.model.ChannelI@413a18f4, omero.model.ChannelI@4d3c5ca0]


and then read the information from the returned channel objects. Otherwise, you should be able to use projection as as follows:

Code: Select all

>> c=q.projection(['select c,i.id,s.globalMin,s.globalMax from Channel c join fetch c.statsInfo as s left join fetch c.pixels as p left outer join fetch p.image as i where i.id = ' num2str(iId)], [])

c =

[[omero.rtypes$RObjectI@6a65a8e6, omero.rtypes$RLongI@5a43, omero.rtypes$RDoubleI@40831000, omero.rtypes$RDoubleI@40ac3a00], [omero.rtypes$RObjectI@3833ca9c, omero.rtypes$RLongI@5a43, omero.rtypes$RDoubleI@40550000, omero.rtypes$RDoubleI@40aede00], [omero.rtypes$RObjectI@503ae497, omero.rtypes$RLongI@5a43, omero.rtypes$RDoubleI@404d8000, omero.rtypes$RDoubleI@40adac00]]


Best,
Sebastien