Running multiple instances on the same host in parallel

[bernhard]

Hello all!

Is it possible to run multiple instances of the Omero4 server application in parallel. The different instances have different databases and data repositories. The purpose would be a development server with several instances for different development parties, like developers and testers.

Thanks for info! Bernhard

[jmoore]

Sure, if you are using the default IceGrid configuration, then the only requirements are to change the Registry port, the Glacier2 port, the connection information for your database, and the omero.data.dir.

The Registry port is specified in the *.cfg files:

Code: Select all

$ grep -r 4061 etc/
etc/internal.cfg:Ice.Default.Locator=IceGrid/Locator:tcp -h 127.0.0.1 -p 4061
etc/master.cfg:IceGrid.Registry.Client.Endpoints=tcp -h 127.0.0.1 -p 4061


The Glacier2 port is specified in etc/grid/default.xml (or windefault.xml or Windows):

Code: Select all

      <server-instance template="Glacier2Template"
        client-endpoints="tcp -p 4063"
        session-timeout="300"
        server-endpoints="tcp -h 127.0.0.1"/>


The configuration is a bit more complicated if you are using the System properties set via "bin/omero config". Each server ("bin/omero admin start") loads the default properties if "OMERO_CONFIG" is not set. If you set "OMERO_CONFIG" before running "admin start", each instance will pick up separate values.

Code: Select all

cd production
export OMERO_CONFIG=production
bin/omero config set omero.data.dir /OMERO/
bin/omero config set omero.db.name production
bin/omero admin start

cd ../testing
export OMERO_CONFIG=testing
bin/omero config set omero.data.dir /OMERO_TESTING
bin/omero config set omero.db.name testing
bin/omero admin start


The alternative to maintaining two sets of config and switching by environment variable is to specify the values directly in each of the default.xml files, which only has the down-side of needing to manage the permissions on that file to prevent password-snopping.



Updated: added requirement to change the registry port.

[bernhard]

Hi Josh!

Thanks for the quick response!

The configuration is a bit more complicated if you are using the System properties set via "bin/omero config". Each server ("bin/omero admin start") loads the default properties if "OMERO_CONFIG" is not set. If you set "OMERO_CONFIG" before running "admin start", each instance will pick up separate values.

Code: Select all

cd production
export OMERO_CONFIG=production
bin/omero config set omero.data.dir /OMERO/
bin/omero config set omero.db.name production
bin/omero admin start

cd ../testing
export OMERO_CONFIG=testing
bin/omero config set omero.data.dir /OMERO_TESTING
bin/omero config set omero.db.name testing
bin/omero admin start


What's the meaning of the env vars? Are they pointing somewhere? Where are the new configs written to if I specify the env var?

The alternative to maintaining two sets of config and switching by environment variable is to specify the values directly in each of the default.xml files, which only has the down-side of needing to manage the permissions on that file to prevent password-snopping.

That's fine, I guess it's always an admin doing the configuration for all the instances.

Thanks! Bernhard

[jmoore]

What's the meaning of the env vars? Are they pointing somewhere? Where are the new configs written to if I specify the env var?

You can see what configs you have available with "bin/config all". Each of those will be either "default" or the value of OMERO_CONFIG before running a command or the value of the "bin/omero config def <sth>" command. I.e. the env var is just a specifier for a set of properties that a server instance should use. (They are stored in your Java preferences. See ticket:800 for more information.)

[bernhard]

Hi Josh!

I tried the approach using different configuration descriptors in etc/grid/default.xml and test.xml, the relavant part that is different looks like this

default.xml

Code: Select all

<properties id="Profile">
      <!--
      Any properties beginning with "omero" set here will
      be passed to the servers as if via "-Domero.example=value"
      were set on the command line. These values may be overwritten
      by `bin/omero config` values.
      -->
      <property name="omero.db.name" value="omero4"/>
      <property name="omero.db.user" value="omero"/>
      <property name="omero.db.pass" value="some_password"/>
      <property name="omero.data.dir" value="/OMERO/OMERO4"/>
    </properties>

<node name="master">
      <server-instance template="Glacier2Template"
        client-endpoints="tcp -p 4063"
        session-timeout="300"
        server-endpoints="tcp -h 127.0.0.1"/>
      <server-instance template="BlitzTemplate" index="0" config="default"/>
      <server-instance template="IndexerTemplate" index="0"/>
      <server-instance template="DropBoxTemplate"/>
      <server-instance template="FSTemplate"/>
      <server-instance template="ProcessorTemplate" index="0"/>
      <server-instance template="ShellTemplate" id="Web" act="on-demand"/>
      <server-instance template="StormTemplate"/>
</node>


test.xml

Code: Select all

<properties id="Profile">
      <!--
      Any properties beginning with "omero" set here will
      be passed to the servers as if via "-Domero.example=value"
      were set on the command line. These values may be overwritten
      by `bin/omero config` values.
      -->
      <property name="omero.db.name" value="omero4_test"/>
      <property name="omero.db.user" value="omero_test"/>
      <property name="omero.db.pass" value="some_password"/>
      <property name="omero.data.dir" value="/OMERO/OMERO4_test/"/>
    </properties>

<node name="master">
      <server-instance template="Glacier2Template"
        client-endpoints="tcp -p 5063"
        session-timeout="300"
        server-endpoints="tcp -h 127.0.0.1"/>
      <server-instance template="BlitzTemplate" index="0" config="default"/>
      <server-instance template="IndexerTemplate" index="0"/>
      <server-instance template="DropBoxTemplate"/>
      <server-instance template="FSTemplate"/>
      <server-instance template="ProcessorTemplate" index="0"/>
      <server-instance template="ShellTemplate" id="Web" act="on-demand"/>
      <server-instance template="StormTemplate"/>
    </node>



Starting each sever individually does not work:

Code: Select all

$ omero admin start etc/grid/default.xml
$ omero admin start etc/grid/test.xml
icegridnode: failure occurred in daemon


Here's the corresponding error in the master.err log file:
06/29/09 16:00:57.017 icegridnode: error: an IceGrid registry is already running and listening on the clientendpoints `tcp -h 127.0.0.1 -p 4061'

Having one server started and deploying the other descriptor replaces the previous servant.

Code: Select all

$omero admin start etc/grid/default.xml
# connect to port 4063 works
$omero admin deploy etc/grid/test.xml
# connect to port 5063 works, but 4063 is dead


Thanks for help! Bernhard

It's not an urgent request. I just wanted to let you know that it seems to be a little bit more involved to get two servers running in parallel.

[jmoore]

Sorry about that Bernhard, I forgot to include the requirement to change the Registry port (4061) in one of the two instances. I've updated my entry above. ~J