Open Microscopy Environment

I have stepped through http://www.openmicroscopy.org/site/support/omero5.2/sysadmins/unix/server-installation.html a few times and have reached a waypoint where I believe I have all the components installed, but I am certain that I am missing a lot of configuration bits.

I am running FreeBSD 10.2 and am attempting to install Omero Server 5.2.5-ice36-b28. My 'admin diagnostics' output is below. I have no idea whether I am very close, or quite a ways off at this point. I am not expecting any web services to function, as I have not yet gone through any of http://www.openmicroscopy.org/site/support/omero5.2/sysadmins/unix/install-web.html.

Is it reasonable to expect 'omero admin start' to work even though the Omero.web components are not yet ready? I'm just looking for some metric of success on the work done so far, so that I can troubleshoot what I've got, and then build from there.

The most obvious point is that icegridnode is not running. I don't see any obvious script installed that would start the icegridnode process. At least, running 'icegridamin -s &' didn't do it. Clues appreciated.

Thank you for any suggestions, including pointers to documentation that I should have read, or requests to please post this or that.

Jim

Code: Select all

omero : 14:55:14 /home/omero/omero> OMERO.server/bin/omero admin diagnostics

================================================================================
OMERO Diagnostics 5.2.5-ice36-b28
================================================================================
       
Commands:   java -version                  1.8.0     (/usr/local/bin/java)
Commands:   python -V                      2.7.12    (/usr/local/bin/python)
Commands:   icegridnode --version          3.6.2     (/usr/local/bin/icegridnode)
Commands:   icegridadmin --version         3.6.2     (/usr/local/bin/icegridadmin)
Commands:   psql --version                 9.4.9     (/usr/local/bin/psql)

Server:     icegridnode                    not started

Log dir:    /usr/home/omero/omero/OMERO.server/var/log exists
Log files:  Blitz-0.log                    102.0 KB      errors=52   warnings=0   
Log files:  DropBox.log                    1.0 KB        errors=6    warnings=0   
Log files:  FileServer.log                 0.0 KB       
Log files:  Indexer-0.log                  103.0 KB      errors=70   warnings=0   
Log files:  MonitorServer.log              0.0 KB       
Log files:  OMEROweb.log                   n/a
Log files:  PixelData-0.log                103.0 KB      errors=70   warnings=0   
Log files:  Processor-0.log                30.0 KB       errors=16   warnings=8   
Log files:  Tables-0.log                   n/a
Log files:  TestDropBox.log                n/a
Log files:  master.err                     3.0 KB        errors=2    warnings=6   
Log files:  master.out                     0.0 KB       
Log files:  Total size                     0.35 MB


Environment:OMERO_HOME=(unset)             
Environment:OMERO_NODE=(unset)             
Environment:OMERO_MASTER=(unset)           
Environment:OMERO_USERDIR=(unset)         
Environment:OMERO_TMPDIR=(unset)           
Environment:PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin:/home/omero/bin
Environment:PYTHONPATH=(unset)             
Environment:ICE_HOME=(unset)               
Environment:LD_LIBRARY_PATH=(unset)       
Environment:DYLD_LIBRARY_PATH=(unset)     

Cannot list deployed applications.
OMERO data dir:'/OMERO'                       Exists? True      Is writable? True
OMERO temp dir:'/home/omero/omero/tmp'        Exists? True      Is writable? True   (Size: 0)

JVM settings: Blitz-${index}                -Xmx3860m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: Indexer-${index}              -Xmx2573m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: PixelData-${index}            -Xmx3860m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: Repository-${index}           -Xmx2573m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions

OMERO.web status... [ERROR] You are deploying OMERO.web using Apache and mod_wsgi. OMERO.web does not provide any management for the daemon process which communicates with Apache child processes using UNIX sockets to handle a request. Please check Apache directly.
Django version: 1.8.13


Code: Select all

omero : 15:19:52 /home/omero/omero> time OMERO.server/bin/omero admin start
No descriptor given. Using etc/grid/default.xml
Waiting on startup. Use CTRL-C to exit
.............................
Failed to startup some components after 300 seconds
Calling "stop" on remaining components
Waiting on shutdown. Use CTRL-C to exit
......
real    6m7.791s
user    0m11.724s
sys     0m4.238s


Code: Select all

omero : 15:29:29 /home/omero/omero> sockstat
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS     
pgsql    postgres   67182 3  dgram  -> /var/run/log
pgsql    postgres   67182 8  udp4   10.160.92.49:38228   10.160.92.49:38228
pgsql    postgres   67181 3  dgram  -> /var/run/log
pgsql    postgres   67181 8  udp4   10.160.92.49:38228   10.160.92.49:38228
pgsql    postgres   67180 3  dgram  -> /var/run/log
pgsql    postgres   67180 8  udp4   10.160.92.49:38228   10.160.92.49:38228
pgsql    postgres   67179 3  dgram  -> /var/run/log
pgsql    postgres   67179 8  udp4   10.160.92.49:38228   10.160.92.49:38228
pgsql    postgres   67178 3  dgram  -> /var/run/log
pgsql    postgres   67178 8  udp4   10.160.92.49:38228   10.160.92.49:38228
pgsql    postgres   67176 3  dgram  -> /var/run/log
pgsql    postgres   67176 4  tcp4   10.160.92.49:5432    *:*
pgsql    postgres   67176 5  stream /tmp/.s.PGSQL.5432
pgsql    postgres   67176 8  udp4   10.160.92.49:38228   10.160.92.49:38228
root     cron       38308 4  dgram  -> /var/run/logpriv
smmsp    sendmail   38304 3  dgram  -> /var/run/log
root     sendmail   38301 3  dgram  -> /var/run/logpriv
root     sendmail   38301 4  tcp4   10.160.92.49:25      *:*
root     syslogd    38245 4  dgram  /var/run/log
root     syslogd    38245 5  dgram  /var/run/logpriv
root     syslogd    38245 6  udp4   10.160.92.49:514     *:*

jimlong wrote:Is it reasonable to expect 'omero admin start' to work even though the Omero.web components are not yet ready?

I'd think so.

The most obvious point is that icegridnode is not running. I don't see any obvious script installed that would start the icegridnode process. At least, running 'icegridamin -s &' didn't do it. Clues appreciated.

The biggest clues at this stage should be in var/log/Blitz-0.log.

~Josh

Thank you for your reply. There was a pg_hba.conf error which I've fixed. In manual PostgreSQL testing I had not noticed this, because psql defaults to a socket connection to the server, whereas it appears that Omero uses an IP-based connection. FreeBSD jails do not have a 127.0.0.1 loopback address. Only the master host does. So allowing 127.0.0.1/32 in pg_hba.conf was not useful. My typical workaround for this is to edit /etc/hosts in the jail and define 'localhost' as the physical IP of the jail. However, it seems that PostgreSQL can't parse a pg_hba.conf entry of the form:

Code: Select all

host    all             all             localhost/32        trust

PostgreSQL was logging:

Code: Select all

Aug 22 16:29:02 omero postgres[54486]: [3-1] FATAL:  could not load pg_hba.conf

Even though all the perms and ownership were correct. Replacing 'localhost' with the physical IP enabled PG to accept both socket and IP connections. Now Omero.admin does start, and I have fewer errors:

Code: Select all

omero : 16:48:26 /home/omero/omero/OMERO.server> rm -rf var/log; mkdir var/log; bin/omero admin start;
No descriptor given. Using etc/grid/default.xml
Waiting on startup. Use CTRL-C to exit
omero : 16:49:47 /home/omero/omero/OMERO.server> bin/omero admin diagnostics

================================================================================
OMERO Diagnostics 5.2.5-ice36-b28
================================================================================
       
Commands:   java -version                  1.8.0     (/usr/local/bin/java)
Commands:   python -V                      2.7.12    (/usr/local/bin/python)
Commands:   icegridnode --version          3.6.2     (/usr/local/bin/icegridnode)
Commands:   icegridadmin --version         3.6.2     (/usr/local/bin/icegridadmin)
Commands:   psql --version                 9.4.9     (/usr/local/bin/psql)

Server:     icegridnode                    running
Server:     Blitz-0                        active (pid = 57744, enabled)
Server:     DropBox                        inactive (disabled)
Server:     FileServer                     active (pid = 57796, enabled)
Server:     Indexer-0                      active (pid = 57789, enabled)
Server:     MonitorServer                  active (pid = 57895, enabled)
Server:     OMERO.Glacier2                 active (pid = 57839, enabled)
Server:     OMERO.IceStorm                 active (pid = 57842, enabled)
Server:     PixelData-0                    active (pid = 57846, enabled)
Server:     Processor-0                    active (pid = 57848, enabled)
Server:     Tables-0                       inactive (disabled)
Server:     TestDropBox                    inactive (enabled)

Log dir:    /usr/home/omero/omero/OMERO.server-5.2.5-ice36-b28/var/log exists
Log files:  Blitz-0.log                    655.0 KB      errors=17   warnings=5   
Log files:  DropBox.log                    0.0 KB        errors=3    warnings=0   
Log files:  FileServer.log                 0.0 KB       
Log files:  Indexer-0.log                  1.0 KB       
Log files:  MonitorServer.log              0.0 KB       
Log files:  OMEROweb.log                   n/a
Log files:  PixelData-0.log                2.0 KB       
Log files:  Processor-0.log                60.0 KB       errors=17   warnings=4   
Log files:  Tables-0.log                   n/a
Log files:  TestDropBox.log                n/a
Log files:  master.err                     1.0 KB        errors=1    warnings=1   
Log files:  master.out                     0.0 KB       
Log files:  Total size                     0.72 MB


Environment:OMERO_HOME=(unset)             
Environment:OMERO_NODE=(unset)             
Environment:OMERO_MASTER=(unset)           
Environment:OMERO_USERDIR=(unset)         
Environment:OMERO_TMPDIR=(unset)           
Environment:PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin:/home/omero/bin
Environment:PYTHONPATH=(unset)             
Environment:ICE_HOME=(unset)               
Environment:LD_LIBRARY_PATH=(unset)       
Environment:DYLD_LIBRARY_PATH=(unset)     

OMERO SSL port:4064                           
OMERO TCP port:4063                           
OMERO data dir:'/OMERO'                       Exists? True      Is writable? True
OMERO temp dir:'/home/omero/omero/tmp'        Exists? True      Is writable? True   (Size: 0)

JVM settings: Blitz-${index}                -Xmx3860m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: Indexer-${index}              -Xmx2573m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: PixelData-${index}            -Xmx3860m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: Repository-${index}           -Xmx2573m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions

OMERO.web status... [ERROR] You are deploying OMERO.web using Apache and mod_wsgi. OMERO.web does not provide any management for the daemon process which communicates with Apache child processes using UNIX sockets to handle a request. Please check Apache directly.
Django version: 1.8.13


The 17 errors reported in Blitz-0.log are all of the form:

Code: Select all

omero : 16:58:13 /home/omero/omero/OMERO.server> grep -nw Error var/log/Blitz-0.log
313:2016-08-22 16:50:06,977 INFO  [        ome.services.util.ServiceHandler] (l.Server-0)  Args:        [Error]
543:2016-08-22 16:50:09,220 INFO  [        ome.services.util.ServiceHandler] (l.Server-7)  Args:        [Error]
773:2016-08-22 16:50:11,672 INFO  [        ome.services.util.ServiceHandler] (l.Server-3)  Args:        [Error]
992:2016-08-22 16:50:13,559 INFO  [        ome.services.util.ServiceHandler] (l.Server-4)  Args:        [Error]
1245:2016-08-22 16:50:15,452 INFO  [        ome.services.util.ServiceHandler] (l.Server-2)  Args:       [Error]
1498:2016-08-22 16:50:17,260 INFO  [        ome.services.util.ServiceHandler] (l.Server-7)  Args:       [Error]
1762:2016-08-22 16:50:19,148 INFO  [        ome.services.util.ServiceHandler] (l.Server-5)  Args:       [Error]
1992:2016-08-22 16:50:20,963 INFO  [        ome.services.util.ServiceHandler] (l.Server-4)  Args:       [Error]
2222:2016-08-22 16:50:22,877 INFO  [        ome.services.util.ServiceHandler] (l.Server-3)  Args:       [Error]
2452:2016-08-22 16:50:24,812 INFO  [        ome.services.util.ServiceHandler] (l.Server-8)  Args:       [Error]
2682:2016-08-22 16:50:26,795 INFO  [        ome.services.util.ServiceHandler] (l.Server-1)  Args:       [Error]
2912:2016-08-22 16:50:28,712 INFO  [        ome.services.util.ServiceHandler] (l.Server-2)  Args:       [Error]
3142:2016-08-22 16:50:30,625 INFO  [        ome.services.util.ServiceHandler] (l.Server-9)  Args:       [Error]
3372:2016-08-22 16:50:32,515 INFO  [        ome.services.util.ServiceHandler] (l.Server-0)  Args:       [Error]
3602:2016-08-22 16:50:34,284 INFO  [        ome.services.util.ServiceHandler] (l.Server-7)  Args:       [Error]
3832:2016-08-22 16:50:36,059 INFO  [        ome.services.util.ServiceHandler] (l.Server-6)  Args:       [Error]
4051:2016-08-22 16:50:37,896 INFO  [        ome.services.util.ServiceHandler] (l.Server-4)  Args:       [Error]

The log context around the first error (for example) is:

Code: Select all

omero : 16:58:20 /home/omero/omero/OMERO.server> grep -5w Error var/log/Blitz-0.log  |  head -12
2016-08-22 16:50:06,955 INFO  [    ome.security.basic.BasicEventContext] (l.Server-8)  cctx:    group=0
2016-08-22 16:50:06,956 INFO  [         ome.security.basic.EventHandler] (l.Server-8)  Auth:    user=0,group=0,event=null(User),sess=c32b0fb8-9e2c-4a8d-9ddb-9739fe1c6526
2016-08-22 16:50:06,973 INFO  [                 org.perf4j.TimingLogger] (l.Server-8) start[1471909806952] time[21] tag[omero.call.success.ome.services.JobBean.attach]
2016-08-22 16:50:06,974 INFO  [        ome.services.util.ServiceHandler] (l.Server-8)  Rslt:    ome.model.jobs.JobStatus:Id_5
2016-08-22 16:50:06,977 INFO  [        ome.services.util.ServiceHandler] (l.Server-0)  Meth:    interface ome.api.JobHandle.setStatus
2016-08-22 16:50:06,977 INFO  [        ome.services.util.ServiceHandler] (l.Server-0)  Args:    [Error]
2016-08-22 16:50:06,979 INFO  [    ome.security.basic.BasicEventContext] (l.Server-0)  cctx:    group=0
2016-08-22 16:50:06,995 INFO  [         ome.security.basic.EventHandler] (l.Server-0)  Auth:    user=0,group=0,event=412368(User),sess=c32b0fb8-9e2c-4a8d-9ddb-9739fe1c6526
2016-08-22 16:50:07,014 INFO  [       ome.security.basic.CurrentDetails] (l.Server-0) Adding log:UPDATE,class ome.model.jobs.ParseJob,1302
2016-08-22 16:50:07,034 INFO  [                 org.perf4j.TimingLogger] (l.Server-0) start[1471909806977] time[56] tag[omero.call.success.ome.services.JobBean.setStatus]
2016-08-22 16:50:07,035 INFO  [        ome.services.util.ServiceHandler] (l.Server-0)  Rslt:    Running
--

Luckily, those lines are all false-positives. The logging level is "INFO" and the string "[ERROR]" is being passed as an argument. In future versions, the changes in PR 4787 will remove them from the diagnostics counts.

Cheers,
~Josh.

My next hurdle is an error in master.err:

Code: Select all

omero : /home/omero/omero/OMERO.server> cat var/log/master.err
-! 08/23/16 13:30:40.910 icegridnode: warning: failed to join the multicast group for IceGrid discovery:
   endpoints = udp -h "239.255.0.1" -p 4061
   Network.cpp:2061: Ice::SocketException:
   socket exception: Can't assign requested address
Traceback (most recent call last):
  File "lib/python/runTables.py", line 15, in <module>
    import omero.tables
  File "/usr/home/omero/omero/OMERO.server-5.2.5-ice36-b28/lib/python/omero/tables.py", line 11, in <module>
    import numpy
  File "/usr/local/lib/python2.7/site-packages/numpy/__init__.py", line 180, in <module>
    from . import add_newdocs
  File "/usr/local/lib/python2.7/site-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/usr/local/lib/python2.7/site-packages/numpy/lib/__init__.py", line 8, in <module>
    from .type_check import *
  File "/usr/local/lib/python2.7/site-packages/numpy/lib/type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "/usr/local/lib/python2.7/site-packages/numpy/core/__init__.py", line 14, in <module>
    from . import multiarray
[b]ImportError: /lib/libgcc_s.so.1: version GCC_4.6.0 required by /usr/local/lib/gcc48/libgfortran.so.3 not found[/b]


There is some prior work on that error string available on the net, but I can't find anything that helps.
Actually, I don't yet understand where that message is coming from, because according to ldd, /usr/local/lib/gcc48/libgfortran.so.3 does not use /lib/libgcc_s.so.1. It uses /usr/local/lib/gcc48/libgcc_s.so.1:

Code: Select all

omero : /home/omero/omero/OMERO.server> ldd /usr/local/lib/gcc48/libgfortran.so.3
/usr/local/lib/gcc48/libgfortran.so.3:
        libquadmath.so.0 => /usr/local/lib/gcc48/libquadmath.so.0 (0x801717000)
        libm.so.5 => /lib/libm.so.5 (0x801952000)
        libgcc_s.so.1 => /usr/local/lib/gcc48/libgcc_s.so.1 (0x801b7b000)
        libc.so.7 => /lib/libc.so.7 (0x800821000)

readelf(1) says that /usr/local/lib/gcc48/libgcc_s.so.1 does indeed have a GCC_4.6.0 version tag.

So at present, I am struggling to figure out why /lib/libgcc_s.so.1 is getting loaded at all .... and by whom. Then it's hopefully just a matter of re-building that package to ensure that it links against the newer version of libgcc_s.so.

One related question I have is about Gcc and Clang. Are both required, or is either one acceptable?

As the docs correctly state, clang is the default toolchain on FreeBSD. My machine has 3.4.1, which is supported for my target Omero version 5.2.

Code: Select all

omero : /home/omero/omero/OMERO.server> clang --version
FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
Target: x86_64-unknown-freebsd10.2
Thread model: posix

However, my installation log shows that when I installed the Python packages from the FreeBSD repo:

Code: Select all

# pkg install graphics/py-pillow math/py-matplotlib math/py-numpy devel/py-tables science/py-scipy

That pulled in a lot of dependencies, including:

Code: Select all

   gcc: 4.8.5_2

I'm not sure why that is yet, but it definitely shows up in the script(8) output from my installation process. Also noteworthy is:

Code: Select all

Message from gcc-4.8.5_2:
To ensure binaries built with this toolchain find appropriate versions
of the necessary run-time libraries, you may want to link using

  -Wl,-rpath=/usr/local/lib/gcc48

For ports leveraging USE_GCC, USES=compiler, or USES=fortran this happens
transparently.

I'm going to take a hard look at the installation log to see what software got installed after that point that I had to build via FreeBSD's ports system (which, for non-BSD readers, compiles from source) vs. the pkg system (which installs pre-built binaries).

Hi Jim,

I'm certainly out of my depth when it comes to FreeBSD, but https://lists.freebsd.org/pipermail/freebsd-python/2015-June/008420.html might suggest adding `import numpy` to the top of lib/python/runTables.py.

Cheers,
~Josh

I have found that I can get around the problem with:

Code: Select all

$ export LD_LIBRARY_PATH=/usr/local/lib/gcc48

before doing the usual:

Code: Select all

$ bin/omero admin start
No descriptor given. Using etc/grid/default.xml
Waiting on startup. Use CTRL-C to exit

This reduces var/log/master.err to:

Code: Select all

-! 08/23/16 16:20:53.799 icegridnode: warning: failed to join the multicast group for IceGrid discovery:
   endpoints = udp -h "239.255.0.1" -p 4061
   Network.cpp:2061: Ice::SocketException:
   socket exception: Can't assign requested address

So now I'm back to Novice Mode: is this a warning that I should be concerned about?

It feels like I'm getting close to a clean startup/shutdown of the Omero admin subsystem.

Code: Select all

omero : /home/omero/omero/OMERO.server> rm -rf var/log; mkdir var/log; bin/omero admin start
No descriptor given. Using etc/grid/default.xml
Waiting on startup. Use CTRL-C to exit
omero : /home/omero/omero/OMERO.server> bin/omero admin diagnostics       

================================================================================
OMERO Diagnostics 5.2.5-ice36-b28
================================================================================
       
Commands:   java -version                  1.8.0     (/usr/local/bin/java)
Commands:   python -V                      2.7.12    (/usr/local/bin/python)
Commands:   icegridnode --version          3.6.2     (/usr/local/bin/icegridnode)
Commands:   icegridadmin --version         3.6.2     (/usr/local/bin/icegridadmin)
Commands:   psql --version                 9.4.9     (/usr/local/bin/psql)

Server:     icegridnode                    running
Server:     Blitz-0                        active (pid = 11185, enabled)
Server:     DropBox                        inactive (disabled)
Server:     FileServer                     active (pid = 11230, enabled)
Server:     Indexer-0                      active (pid = 11235, enabled)
Server:     MonitorServer                  active (pid = 11239, enabled)
Server:     OMERO.Glacier2                 active (pid = 11286, enabled)
Server:     OMERO.IceStorm                 active (pid = 11289, enabled)
Server:     PixelData-0                    activating (enabled)
Server:     Processor-0                    inactive (disabled)
Server:     Tables-0                       inactive (disabled)
Server:     TestDropBox                    inactive (enabled)

Log dir:    /usr/home/omero/omero/OMERO.server-5.2.5-ice36-b28/var/log exists
Log files:  Blitz-0.log                    19.0 KB       errors=0    warnings=1   
Log files:  DropBox.log                    0.0 KB        errors=3    warnings=0   
Log files:  FileServer.log                 0.0 KB       
Log files:  Indexer-0.log                  1.0 KB       
Log files:  MonitorServer.log              0.0 KB       
Log files:  OMEROweb.log                   n/a
Log files:  PixelData-0.log                1.0 KB       
Log files:  Processor-0.log                n/a
Log files:  Tables-0.log                   n/a
Log files:  TestDropBox.log                n/a
Log files:  master.err                     0.0 KB        errors=0    warnings=1   
Log files:  master.out                     0.0 KB       
Log files:  Total size                     0.02 MB


Environment:OMERO_HOME=(unset)             
Environment:OMERO_NODE=(unset)             
Environment:OMERO_MASTER=(unset)           
Environment:OMERO_USERDIR=(unset)         
Environment:OMERO_TMPDIR=(unset)           
Environment:PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin:/home/omero/bin
Environment:PYTHONPATH=(unset)             
Environment:ICE_HOME=(unset)               
Environment:LD_LIBRARY_PATH=/usr/local/lib/gcc48
Environment:DYLD_LIBRARY_PATH=(unset)     

OMERO SSL port:4064                           
OMERO TCP port:4063                           
OMERO data dir:'/OMERO'                       Exists? True      Is writable? True
OMERO temp dir:'/home/omero/omero/tmp'        Exists? True      Is writable? True   (Size: 0)

JVM settings: Blitz-${index}                -Xmx3860m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: Indexer-${index}              -Xmx2573m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: PixelData-${index}            -Xmx3860m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: Repository-${index}           -Xmx2573m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions

OMERO.web status... [ERROR] You are deploying OMERO.web using Apache and mod_wsgi. OMERO.web does not provide any management for the daemon process which communicates with Apache child processes using UNIX sockets to handle a request. Please check Apache directly.
Django version: 1.8.13


Hi Jim,

jimlong wrote:I have found that I can get around the problem with:

Code: Select all

$ export LD_LIBRARY_PATH=/usr/local/lib/gcc48

before doing the usual:

Code: Select all

$ bin/omero admin start
No descriptor given. Using etc/grid/default.xml
Waiting on startup. Use CTRL-C to exit

Good to hear you found a solution.

This reduces var/log/master.err to:

Code: Select all

-! 08/23/16 16:20:53.799 icegridnode: warning: failed to join the multicast group for IceGrid discovery:
   endpoints = udp -h "239.255.0.1" -p 4061
   Network.cpp:2061: Ice::SocketException:
   socket exception: Can't assign requested address

So now I'm back to Novice Mode: is this a warning that I should be concerned about?

There will be some services (processor, tables) which don't function properly unless there is access to the registry (port 4061). Have you checked if anything else might be using this port? Firewall/jail settings? There are some debugging suggestions in https://forums.zeroc.com/discussion/6638/problem-with-python-server-of-printer-example that you could try out.

Cheers,
~Josh

Thanks for all your pointers, Josh. Just two quick notes before I spend more time with the ZeroC forum link you provided.

I do have a unicast socket listening on port 4061 -- is that sufficient?

Code: Select all

omero /home/omero/omero/OMERO.server> sockstat -4l | grep ':406'
omero    glacier2ro 11286 13 tcp4   10.16.92.49:4064    *:*
omero    glacier2ro 11286 14 tcp4   10.16.92.49:4063    *:*
omero    icegridnod 11183 33 tcp4   10.16.92.49:4061    *:*

I have no firewalling in place on the demo server I am provisioning. However, FreeBSD jails by design cannot modify their networking environment, such as by downing an interface or adding or removing IPs. It is possible to configure a jail with its own virtual network stack, which then does give the jail full control, but my institution has not had any need to do that in the past. Perhaps now we do! If need be I could move my test installation onto a low-end piece of hardware and give it full access to the machine, instead of trying to provision Omero inside a jailed ZFS filesystem. If possible, my preferred approach would be to confine Omero's choice of multicast IP, and provision the jail with a predefined IP to use. A jailed application will not be able to unilaterally assign itself a network address, especially since Omero runs without superuser privileges and I don't see any setuid binaries:

Code: Select all

omero /home/omero/omero/OMERO.server> find . -perm +4000 | wc -l
       0

How is Omero choosing the particular multicast IP 239.255.0.1 -- is that configured somewhere? I am inexperienced with multicast, but if Omero can be constrained to a particular static multicast IP, I can probably provision the jail configuration to have that IP available for use.

Also, perhaps a note for the documentation team:

http://www.openmicroscopy.org/site/support/omero5.2/search.html?q=multicast

yields no hits and

https://www.google.com/#q=multicast+site:openmicroscopy.org&filter=0

very few. I cannot find much on the Omero site about which subsystems of Omero use multicast networking, or any multicast-related configuration details, with the slight exception of:

http://downloads.openmicroscopy.org/omero/5.2.4/api/ome/services/messages/GlobalMulticaster.html

Thanks again!

Jim

jimlong wrote:I do have a unicast socket listening on port 4061 -- is that sufficient?

Code: Select all

omero /home/omero/omero/OMERO.server> sockstat -4l | grep ':406'
omero    glacier2ro 11286 13 tcp4   10.16.92.49:4064    *:*
omero    glacier2ro 11286 14 tcp4   10.16.92.49:4063    *:*
omero    icegridnod 11183 33 tcp4   10.16.92.49:4061    *:*

Hmmm.... those are tcp4 sockets. I'd assume you're having trouble getting a udp one.

I have no firewalling in place on the demo server I am provisioning. However, FreeBSD jails by design cannot modify their networking environment, such as by downing an interface or adding or removing IPs. It is possible to configure a jail with its own virtual network stack, which then does give the jail full control, but my institution has not had any need to do that in the past. Perhaps now we do!

Other than opening ports 80, 443, 4063, and 4064, there's usually nothing needed in the way of networking for an OMERO installation.

If need be I could move my test installation onto a low-end piece of hardware and give it full access to the machine, instead of trying to provision Omero inside a jailed ZFS filesystem. If possible, my preferred approach would be to confine Omero's choice of multicast IP, and provision the jail with a predefined IP to use. A jailed application will not be able to unilaterally assign itself a network address, especially since Omero runs without superuser privileges and I don't see any setuid binaries:

Code: Select all

omero /home/omero/omero/OMERO.server> find . -perm +4000 | wc -l
       0

Certainly no setuid binaries or anything of that nature. OMERO is intended to be run as a single, non-admin user.

How is Omero choosing the particular multicast IP 239.255.0.1 -- is that configured somewhere? I am inexperienced with multicast, but if Omero can be constrained to a particular static multicast IP, I can probably provision the jail configuration to have that IP available for use.

See http://www.iana.org/assignments/multicast-addresses/multicast-addresses.xhtml -- 239.* counts as "organization scope":

Code: Select all

239.0.0.0-239.255.255.255       Organization-Local Scope   [David_Meyer][RFC2365]   1997-01-01


so I assume ZeroC picked this value at random. You can see the value in their docs in various places, for example IceGridAdmin.

Also, perhaps a note for the documentation team:

http://www.openmicroscopy.org/site/support/omero5.2/search.html?q=multicast

yields no hits and

https://www.google.com/#q=multicast+site:openmicroscopy.org&filter=0

very few. I cannot find much on the Omero site about which subsystems of Omero use multicast networking, or any multicast-related configuration details,

Thanks for the heads up. Typically this does tend to "just one work", certainly on a single host machine. It may take some back and forth with you to figure out just what we need to document here.

All the best,
~Josh