Run Compiled Matlab program from Client

[Hang]

Hi all,

Is it possible to run a compiled matlab program from the Omero.insight?

I saw this tool Omero.mtool, but it's a standalone Matlab application that requires to be installed separately from the client.

I wonder if I can:

1. Call compiled matlab program via python scripts?

2. OR using some other way to embed a compiled matlab program to the client.

FYI: I know that MATLAB functionality can be mixed into Python scripts. But currently we are having issue installing Matlab on the server for our own reason. And that's why I'm seeking this route with matlab executables.

[sbesson]

Hi

Is it possible to run a compiled matlab program from the Omero.insight?

I saw this tool Omero.mtool, but it's a standalone Matlab application that requires to be installed separately from the client.

If you are looking at called a MATLAB application client-side, it should be possible. The application will still need to be installed separately. From the OMERO.insight client, you should be able to launch it like any other third-party applications using the Open With... options when right clicking on an object.

1. Call compiled matlab program via python scripts?

If you are looking into invoking compiled MATLAB application server-side (where Python scripts are executed), it should also be possible provided both the program and the MCR are available to the server. If you are writing an OMERO.script, I would expect you should be able to use `subprocess.Popen` using the full path to the compiled program executable and the expected arguments.

FYI: I know that MATLAB functionality can be mixed into Python scripts. But currently we are having issue installing Matlab on the server for our own reason. And that's why I'm seeking this route with matlab executables.

This approach makes complete sense. In the case of OMERO.mtools mentioned above, the MATLAB source code lives on GitHub. In addition, for each release the tools are compiled into a MATLAB executable for deployment and distribution purposes (see downloads page).
We have not had feedback from anyone who would have used this binary MATLAB application via scripts yet but it would be very interesting to hear from your experience.

Best,
Sebastien

[Hang]

Hi there,

I'm trying to use Omero's python scripting service to call a compiled Matlab executable. Omero.server is installed on the Linux server, so is MCR.

We tried to place the matlab executable on the Linux server under:

1. /home/yuh5/Desktop/magicsquare/for_testing/magicsquare

2. /home/omero/magicsquare

But both getting error when launch the python script:

Code: Select all

omero script launch 354
Using session f3d059e2-446e-453e-974f-4a7428f8e80f (yuh5@localhost:4064). Idle timeout: 10 min. Current group: system
Enter value for "IDs": 1
Job 424 ready
Waiting....
Callback received: FINISHED

   *** start stdout (id=522)***
   * Color of each channel:
   *
   *** end stdout ***


   *** start stderr (id=523)***
   * /home/omero/magicsquare: error while loading shared libraries: libmwlaunchermain.so: cannot open shared object file: No such file or directory
   *
   *** end stderr ***


   *** out parameters ***
   * Message=Session is over.
   ***  done ***

FYI:
1. If we directly execute the compiled Matlab executable (with ./magicsquare), it will run successfully.

2. We found that when LD_LIBRARY_PATH was set to point to MCR's paths, Omero server would fail to start until LD_LIBRARY_PATH was cleared. This seems to suggest Omero is not compatible with MCR's dynamic objects. If you could, please point us to a success case that demonstrated a working example of Omero calling MATLAB compiled executable with MCR.

[Hang]

Before you mentioned:

"If you are looking into invoking compiled MATLAB application server-side (where Python scripts are executed), it should also be possible provided both the program and the MCR are available to the server."

What should be done to make sure that both MCR and the executable are available to the Omero server?

Hang

[sbesson]

Hi Hang,

it seems we have initiated two separate threads (see also viewtopic.php?f=6&t=8487) which are strongly related but have different symptoms and errors.

Trying to summarize the current state of the problem:

- your `magicsquare` script is a compiled MATLAB executable wrapped in OMERO.script using `subprocess.Popen`
- you are encountering errors independently of the location of the executable i.e. `/home/yuh5/Desktop/magicsquare/for_testing/` or `/home/omero`
- is the error you are receiving `cannot open shared object file` as shown here, `Permission denied` as in viewtopic.php?f=6&t=8487#p19257 or do you have both error messages depending on the conditions and if yes which?

For this particular error, I also suspect something might have to be set in terms of environment variables. Could you share your OMERO environment as shown by

Code: Select all

/bin/omero admin diagnostics

?

Best,
Sebastien


Best,
Sebastien

[Hang]

admin diagnostics

================================================================================
OMERO Diagnostics (admin) 5.4.4-ice36-b82
================================================================================

Commands: java -version 1.8.0 (/usr/bin/java)
Commands: python -V 2.7.12 (/usr/bin/python)
Commands: icegridnode --version error:'NoneType' object has no attribute 'group'
Commands: icegridadmin --version 3.6.4 (/usr/bin/icegridadmin)
Commands: psql --version 9.6.8 (/usr/bin/psql)

Server: icegridnode running
Server: Blitz-0 icegridadmin: /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/x86_64-linux-gnu/libIceUtil.so.36)
Server: DropBox icegridadmin: /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/x86_64-linux-gnu/libIceUtil.so.36)
Server: FileServer icegridadmin: /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/x86_64-linux-gnu/libIceUtil.so.36)
Server: Indexer-0 icegridadmin: /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/x86_64-linux-gnu/libIceUtil.so.36)
Server: MonitorServer icegridadmin: /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/x86_64-linux-gnu/libIceUtil.so.36)
Server: OMERO.Glacier2 icegridadmin: /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/x86_64-linux-gnu/libIceUtil.so.36)
Server: OMERO.IceStorm icegridadmin: /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/x86_64-linux-gnu/libIceUtil.so.36)
Server: PixelData-0 icegridadmin: /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/x86_64-linux-gnu/libIceUtil.so.36)
Server: Processor-0 icegridadmin: /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/x86_64-linux-gnu/libIceUtil.so.36)
Server: Tables-0 icegridadmin: /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/x86_64-linux-gnu/libIceUtil.so.36)
Server: TestDropBox icegridadmin: /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/x86_64-linux-gnu/libIceUtil.so.36)

Log dir: /home/omero/OMERO.server/var/log exists
Log files: Blitz-0.log 67.0 MB errors=32 warnings=120
Log files: DropBox.log 25.0 KB errors=18 warnings=13
Log files: FileServer.log 4.0 KB
Log files: Indexer-0.log 858.0 KB errors=36 warnings=84
Log files: MonitorServer.log 9.0 KB
Log files: PixelData-0.log 605.0 KB errors=36 warnings=83
Log files: Processor-0.log 1.0 MB errors=53 warnings=94
Log files: Tables-0.log 139.0 KB errors=36 warnings=70
Log files: TestDropBox.log n/a
Log files: master.err 20.0 KB errors=11 warnings=34
Log files: master.out 0.0 KB
Log files: Total size 70.54 MB


Environment:OMERO_HOME=/home/omero/OMERO.server
Environment:OMERO_NODE=(unset)
Environment:OMERO_MASTER=(unset)
Environment:OMERO_USERDIR=(unset)
Environment:OMERO_TMPDIR=(unset)
Environment:PATH=/home/omero/bin:/home/omero/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/snap/bin:/home/omero/OMERO.server/bin
Environment:PYTHONPATH=/home/omero/OMERO.server/lib/python:/usr/local/MATLAB/MATLAB_Runtime/v94/mcr/toolbox/matlab/external/interfaces/python
Environment:ICE_HOME=(unset)
Environment:LD_LIBRARY_PATH=/usr/local/MATLAB/MATLAB_Runtime/v94/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/sys/opengl/lib/glnxa64
Environment:DYLD_LIBRARY_PATH=(unset)

icegridadmin: /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/x86_64-linux-gnu/libIceUtil.so.36)
OMERO data dir:'/OMERO' Exists? True Is writable? True
OMERO temp dir:'/home/omero/omero/tmp' Exists? True Is writable? True (Size: 0)

JVM settings: Blitz-${index} -Xmx5051m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: Indexer-${index} -Xmx3367m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: PixelData-${index} -Xmx5051m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: Repository-${index} -Xmx3367m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions

[Hang]

Hi Sebastien,

1. I'm not sure what 'wrapped in' mean, but yes I'm using 'subprocess.Popen' to call that Matlab executable 'magicsquare'.

2. Yes, I was encountering errors independently of the location of the executable. I had the `Permission denied` error, but after I placed the executable under `/home/omero`, now I'm getting and only getting error:

Code: Select all

* /home/omero/magicsquare: error while loading shared libraries: libmwlaunchermain.so: cannot open shared object file: No such file or directory     


Hang

[Hang]

Hi Sebastien,

In terms of what I want to do: Call a Matlab executable using Omero's python scripting service.

If it's not too much to ask, would you please write me an instruction on how to get this done, so I can compare and see if I'm missing anything or doing something wrong? Or point me to an example?

Thanks,

Hang

[rleigh]

The presence of shared libraries from the matlab runtime on the OMERO runtime paths is a potential source of problems. The versions of libraries in the matlab runtime are often quite outdated, and this could cause problems. If you can run matlab without having to add its library paths to the system library paths, I think this would be for the best.

[Hang]

Hi rleigh,

Thanks for your suggestions, however:

1. We need to use matlab runtime(MCR) instead of matlab because then there is no license issue. Calling a compiled or not compiled matlab program from Omero without MCR installed will have a license issue as far as I know.

2. We tried to add MCR's library path both to the system and Omero account. When adding the path to system library path, it caused Omero server failed to start. When adding MCR's library path to Omero, it still can not find MCR libraries.

So, we are stuck here at this point...Any other suggestions?

Or, do you think I should proceed with Omero regarding of what I'm trying to accomplish here?

Thanks,

Hang