Pass image object to compiled Matlab program

[Hang]

Hi all,

I'm trying to access an image on Omero, and then pass it to an compiled Matlab program for some processing, then pass the image back to Omero to store as result image.

So, is there an Object/API available for me to pass an image btw Omero and Matlab?

Thanks,

Hang

[jmoore]

Hi Hang,

So, is there an Object/API available for me to pass an image btw Omero and Matlab?

The examples under Reading data show an `Image` object that can be received from the server. This is a metadata object and you can certainly pass it back and forth, including making changes to the metadata in order to update the server's entry.

However, this does not contain the binary data. For accessing the pixels, you will need to work with planes or, preferably, tiles of image data.

Cheers,
~Josh

[Hang]

Hi,

A quick rundown on what I want to accomplish: The user can start a python script from Omero client to process an image that's on the remote Omero server with the algorithm I wrote in Matlab.

So the idea is to call the Matlab executable within a Python scripting service using "subprocess.Popen (Because unlike OMERO.matlab, I want this Matlab executable to be called from the server side). I'm thinking about two possible approaches:

A. With Python language bindings, I know I can do:

Code: Select all

parameter_map = client.getInputs(unwrap=True)

# create a wrapper so we can use the Blitz Gateway.
conn = BlitzGateway(client_obj=client)

And then getting image plane:

Code: Select all

image = conn.getObject("Image", imageId)
size_z = image.getSizeZ()
size_c = image.getSizeC()
size_t = image.getSizeT()
z, t, c = 0, 0, 0                     # first plane of the image
pixels = image.getPrimaryPixels()
plane = pixels.getPlane(z, c, t)      # get a numpy array.

Can I then pass/convert this numpy array to Matlab?

Or,

B. Are you suggesting me to directly use the scripts (like loadOmero.m , connectOmero.m, getImages.m ) from OMERO.matlab? Meaning establish the connection to the server in Matlab script rather than Python script. And then do the rest of the work all in Matlab script? And eventually convert the whole thing to a big Matlab executable?

I'm not sure whether Plan A or B would be more feasible?

Thanks,

Hang

[jmoore]

Hi,

Morning, Hang

I'm thinking about two possible approaches:

A. With Python language bindings, I know I can do:

Code: Select all

parameter_map = client.getInputs(unwrap=True)

# create a wrapper so we can use the Blitz Gateway.
conn = BlitzGateway(client_obj=client)

And then getting image plane:

Code: Select all

image = conn.getObject("Image", imageId)
size_z = image.getSizeZ()
size_c = image.getSizeC()
size_t = image.getSizeT()
z, t, c = 0, 0, 0                     # first plane of the image
pixels = image.getPrimaryPixels()
plane = pixels.getPlane(z, c, t)      # get a numpy array.

Can I then pass/convert this numpy array to Matlab?

This is certainly what something like mlab is trying to enable. But our experience with mlabwrap is quite old.

Or,

B. Are you suggesting me to directly use the scripts (like loadOmero.m , connectOmero.m, getImages.m ) from OMERO.matlab? Meaning establish the connection to the server in Matlab script rather than Python script. And then do the rest of the work all in Matlab script? And eventually convert the whole thing to a big Matlab executable?

This sounds most like what I had in mind during my previous answer. At the end of the day, OMERO.processor is only doing the following steps when it kicks off a Python script:

• create a temporary directory
• write a config file
• setup an environment
• launch the subprocess

You can definitely make use of the config file in the subdirectory to run a native MATLAB script which talks to the server by using the Java language binding.


As a quick aside, in case the intermediate Python layer becomes a problem, I'd add in a plan C -- the infrastructure is also in place to have OMERO.processor directly call MATLAB:

• https://github.com/openmicroscopy/openmicroscopy/blob/develop/etc/omero.properties#L230
• https://github.com/openmicroscopy/openmicroscopy/blob/develop/components/server/resources/ome/services/service-scripts.xml#L35
• https://github.com/openmicroscopy/openmicroscopy/blob/develop/components/tools/OmeroPy/src/omero/processor.py#L701

However, since we don't ship MATLAB scripts, we don't regularly test this and the syntax is much more verbose, but my assumption is that a script like the one below would work. The only thing that I needed to configure was making the OMERO.matlab toolbox needs to be made available as if with `addpath`.

I'm not sure whether Plan A or B would be more feasible?

Comparing all three possibilities, unless you are familiar & comfortable with Python and libraries like mwrap, I'd try B or perhaps C if you're willing to try out something new.

Cheers,
~Josh


Example script:

Code: Select all

function script(void)
    [client, session] = loadOmero();
    session.detachOnDestroy();
    try
        parse = client.getProperty('omero.scripts.parse');
        if length(parse) > 0
            params = javaObject('omero.grid.JobParams');
            params.name = 'my-first-matlab-script' ;
            params.description = 'some info';
            params.stdoutFormat = 'text/plain';
            params.stderrFormat = 'text/plain';

            input1 = javaObject('omero.grid.Param');
            input1.optional = false;
            input1.description = 'A required string';
            input1.prototype = javaMethod('rstring', 'omero.rtypes', '');

            params.inputs = javaObject('java.util.HashMap');
            params.inputs.put('input1', input1);

            params = javaMethod('rinternal', 'omero.rtypes', params);
            client.setOutput('omero.scripts.parse', params);
        else
            input1 = client.getInput('input1').getValue();
            disp('Found input:');
            disp(input1);
            disp('Do something here');
        end
    catch e
        client.closeSession();
        rethrow e
    end
    client.closeSession();
end


[Hang]

hi,

Code: Select all

function plane = testOmero(id)

omero_client_1 = loadOmero('ceb-cad2');
session = omero_client_1.createSession('omero','1234');
omero_client_1.enableKeepAlive(60);
images = getImages(session, 1);
image = images(1);
plane = getPlane(session, image, 0,0,0);

end

I compiled this simple code to an executable and tried to run it, but I'm getting error:

Code: Select all

No class omero.client can be located on the Java class path

Error in connectOmero (line 66)

Error in loadOmero (line 101)

I tried to put all the jar files from the libs folder(commons-collections.jar, commons-lang.jar, guava.jar,omero_client.jar... ...) as the required files before compile it into the exe, and I also tried to put the jars under the same directory as the exe while trying to run it. But these two methods don't work..

Anyway, is omero.client a class in one of the jar files?

Thanks,

Hang

[jmoore]

hi,

Morning,

I tried to put all the jar files from the libs folder(commons-collections.jar, commons-lang.jar, guava.jar,omero_client.jar... ...) as the required files before compile it into the exe, and I also tried to put the jars under the same directory as the exe while trying to run it. But these two methods don't work..

I have to admit I have basically no experience with compiling MATLAB applications. However, looking around I see for example https://de.mathworks.com/matlabcentral/answers/244613-editing-classpath-for-compiled-program which suggests either adding the files with deploytool or modifying a classpath file. https://de.mathworks.com/matlabcentral/answers/97072-how-should-i-include-a-jar-file-in-my-compiled-application-with-matlab-compiler-4-1-r14sp1 additionally talks about the `-a` flag to deploytool as well as the JAVAADDPATH environment variable.

My hope would be that the logic in getOmeroJars.m and loadOmero.m would work even when compiled, but apparently something else is needed. Could you give us the output of getOmeroJars in your test application?

Anyway, is omero.client a class in one of the jar files?

omero_client.jar definitely has an omero.client class.

~Josh

[Hang]

Hi,

I tried compiling it again today and it worked. The Matlab exe is able find the class path for all the jar files. Sorry I must have made some mistakes yesterday.

So, to sum it up:

Add the lib folder under OMERO.matlab (the one that has all the jar files) into the application package works. So Josh is right:
"The logic in getOmeroJars.m and loadOmero.m would work even when compiled"

Thanks,

Hang

[jmoore]

Hi Hang,

glad to hear it. Does that mean that the .m script under lib/scripts is working for you? If so, could you post your final working version? We'll look into adding it to the documentation for others who follow suit.

~Josh.

[Hang]

Are you referring to the "lib/scripts" directory in Omero.server that stores all the python scripts?

If so, yes I got it to work. But there is no .m script. Again I'm using python to call a Matlab exe.

Here is my python script:

Code: Select all

import omero
import omero.scripts as scripts
from omero.gateway import BlitzGateway
from omero.rtypes import rstring, rlong, robject, unwrap
import omero.util.script_utils as script_utils
from omero.util.tiles import TileLoopIteration, RPSTileLoop
from omero.model import PixelsI

import os
import sys
from omero.rtypes import wrap
from PIL import Image
from io import StringIO
import shlex, subprocess
# from subprocess import call

if 'LD_LIBRARY_PATH' not in os.environ:
    os.environ['LD_LIBRARY_PATH'] = '/usr/local/MATLAB/MATLAB_Runtime/v94/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/sys/opengl/lib/glnxa64'

def run_script():
    """
    The main entry point of the script, as called by the client via the
    scripting service, passing the required parameters.
    """
    data_types = [rstring('Dataset'), rstring('Image')]

    client = scripts.client(
        'test.py',
        """Count total number of cells and classify infected/normal cells.
""",

        scripts.String(
            "Data_Type", optional=False, grouping="1",
            description="Choose Images via their 'Dataset' or directly by "
            " 'Image' IDs.", values=data_types, default="Image"),

        scripts.List(
            "IDs", optional=False, grouping="2",
            description="List of Dataset IDs or Image IDs to "
            " process.").ofType(rlong(0)),

        scripts.String(
            "Container_Name", grouping="3",
            description="Option: put Images in new Dataset with this name"
            " OR use this name for new Image stacks, if 'Make_Image_Stack')",
            default="From_Malria_screener"),

        scripts.Bool(
            "Make_Image_Stack", grouping="4", default=False,
            description="If true, make a single Image (stack) from all the"
            " ROIs of each parent Image"),

        version="0.1.0",
        authors=["Hang Yu", "Malaria Screener Team"],
        institutions=["National Institutes of Health"],
        contact="hang.yu@nih.gov",
    )

    try:
        parameter_map = client.getInputs(unwrap=True)

        # create a wrapper so we can use the Blitz Gateway.
        conn = BlitzGateway(client_obj=client)

   # print 'Success:', os.environ['LD_LIBRARY_PATH']
   
   # call Matlab executale
   #subprocess.Popen(['/home/omero/magicsquare', '5'])
   subprocess.call(['/home/omero/testOmero', '1'])

   message = "Session is over."

        client.setOutput("Message", rstring(message))
       
    finally:
        client.closeSession()


if __name__ == "__main__":
    run_script()

As you can see in the script, testOmero is the Matlab exe being called.

And the script for testOmero:

Code: Select all

function plane = testOmero(id)

% Connect to Omero server and create a session
omero_client_1 = loadOmero('ceb-cad2');
session = omero_client_1.createSession('omero','1234');
omero_client_1.enableKeepAlive(60);

images = getImages(session, 2);
image = images(1);
pixels = image.getPrimaryPixels;
sizeX = pixels.getSizeX.getValue();
sizeY = pixels.getSizeY.getValue();
sizeZ = pixels.getSizeZ.getValue();
sizeT = pixels.getSizeT.getValue();
sizeC = pixels.getSizeC.getValue();
plane1 = getPlane(session, image, 0,0,0);
plane2 = getPlane(session, image, 0,1,0);
plane3 = getPlane(session, image, 0,2,0);

image_new = cat(3, plane1, plane2, plane3);

end

Cheers,
Hang

[jmoore]

Are you referring to the "lib/scripts" directory in Omero.server that stores all the python scripts?

If so, yes I got it to work. But there is no .m script. Again I'm using python to call a Matlab exe.

Understood and thanks for the complete example!

Cheers,
~Josh.