OMERO data dir config

[jwarren]

Hi

I have two questions (using v5.1.2):

1) I've configured and omero instance using the config set command line to point at a directory - but every time I try to access a larger image (not thumbnail) omero complains with the Error instantiating pixel buffer and says it's looking in a directory that isn't specified in the config anymore. Is there anywhere else I need to change the location it's pointing at for OMERO data? Restarted admin and web many times but makes no difference - also tried relative directory specified - but always points to the old directory that it doesn't have access to.

2) We are setting up dev-> pre-release-> production omeros in vms. I'd like to be able to use exactly the same omero directory but configured differently for each so they point at a different databases and a local copy of the OMERO data directory etc. Is there a way to do this?

Many thanks

Jonathan.

[kennethgillen]

Hi Jonathan,

Re 1, to help others reply to this thread, could you please paste the output of the "config get", "admin diagnostics" and "fs repos" commands:

Code: Select all

OMERO-CURRENT/bin/omero config get --hide-password

Code: Select all

OMERO-CURRENT/bin/omero admin diagnostics

Code: Select all

OMERO-CURRENT/bin/omero fs repos

All the best,

Kenny

[jmoore]

Hi

Hi Jonathan,

while waiting on logs, here's a an idea for:

2) We are setting up dev-> pre-release-> production omeros in vms. I'd like to be able to use exactly the same omero directory but configured differently for each so they point at a different databases and a local copy of the OMERO data directory etc. Is there a way to do this?

One way (of likely several) to do this might be to:

• symlink the var/ directory to something outside of the OMERO.server directory
• point the omero.db.* properties and omero.data.dir to some locally resolvable name
• chmod the rest of OMERO.server read-only
• port-forward (or similar) a postgresql port connection to the locally resolvable name
• mount a unique data directory to the locally resolvable name

The only remaining gotcha might be the lib/script directory depending on if and how you are using scripts.

Cheers,
~Josh

[jwarren]

config get:

Ice.IPv6=0
omero.data.dir=../../omero_data/OMERO
omero.db.host=pgsql-hxvm-018.ebi.ac.uk
omero.db.name=mipadev
omero.db.pass=********
omero.db.user=omero
omero.jvmcfg.max_system_memory=12000
omero.jvmcfg.percent.blitz=50
omero.jvmcfg.percent.indexer=20
omero.jvmcfg.percent.pixeldata=20
omero.jvmcfg.system_memory=14000
omero.web.application_host=http://wwwdev.ebi.ac.uk/mi/media/omero:80/
omero.web.application_server=fastcgi-tcp
omero.web.debug=True
omero.web.public.enabled=True
omero.web.public.password=********
omero.web.public.url_filter=webgateway|webclient/annotation
omero.web.public.user=public_user
omero.web.static_url=/mi/media/static/

bin/omero admin diagnostics

================================================================================
OMERO Diagnostics 5.1.2-ice35-b45
================================================================================

WARNING:omero.util.UpgradeCheck:UPGRADE AVAILABLE:Please upgrade to 5.1.3 See http://trac.openmicroscopy.org.uk/omero for the latest version

Commands: java -version 1.6.0 (/usr/bin/java)
Commands: python -V 2.6.6 (/usr/bin/python)
Commands: icegridnode --version 3.5.1 (/net/isilonP/public/rw/homes/mi_adm/omero_stage_ves-hx-ca/Ice-3.5.1/bin/icegridnode)
Commands: icegridadmin --version 3.5.1 (/net/isilonP/public/rw/homes/mi_adm/omero_stage_ves-hx-ca/Ice-3.5.1/bin/icegridadmin)
Commands: psql --version 8.4.20 (/usr/bin/psql)

Server: icegridnode running
Server: Blitz-0 active (pid = 13497, enabled)
Server: DropBox active (pid = 13509, enabled)
Server: FileServer active (pid = 13519, enabled)
Server: Indexer-0 active (pid = 13527, enabled)
Server: MonitorServer active (pid = 13518, enabled)
Server: OMERO.Glacier2 active (pid = 13520, enabled)
Server: OMERO.IceStorm active (pid = 13536, enabled)
Server: PixelData-0 active (pid = 13522, enabled)
Server: Processor-0 active (pid = 13524, enabled)
Server: Tables-0 inactive (disabled)
Server: TestDropBox inactive (enabled)

Log dir: /net/isilonP/public/rw/homes/mi_adm/omero_stage_ves-hx-ca/OMERO.server-5.1.2-ice35-b45/var/log exists

Log files: .nfs000000018b44850c0001b0bf 5.0 MB errors=42 warnings=30
Log files: Blitz-0.log 217.0 MB errors=168 warnings=773
Log files: Blitz-0.log.1 524.0 MB errors=46 warnings=3232
Log files: DropBox.log 32.0 KB errors=8 warnings=12
Log files: FileServer.log 6.0 KB errors=3 warnings=0
Log files: Indexer-0.log 10.0 MB errors=2638 warnings=528
Log files: MonitorServer.log 17.0 KB errors=2 warnings=0
Log files: OMEROweb.log 3.0 MB errors=52 warnings=0
Log files: OMEROweb.log.1 5.0 MB errors=293 warnings=2
Log files: OMEROweb.log.2 882.0 KB errors=16 warnings=9
Log files: OMEROweb.log.3 903.0 KB errors=18 warnings=9
Log files: OMEROweb.log.4 930.0 KB errors=17 warnings=8
Log files: OMEROweb.log.5 1.0 MB errors=14 warnings=8
Log files: OMEROweb_request.log 101.0 KB errors=0 warnings=597
Log files: PixelData-0.log 5.0 MB errors=1353 warnings=270
Log files: Processor-0.log 83.0 KB errors=34 warnings=26
Log files: Tables-0.log n/a
Log files: TestDropBox.log n/a
Log files: master.err 19.0 KB errors=14 warnings=10
Log files: master.out 0.0 KB
Log files: Total size 775.21 MB

Parsing Blitz-0.log:[line:987906] Did you create your omero.data.dir? E.g. /OMERO

Environment:OMERO_HOME=(unset)
Environment:OMERO_NODE=(unset)
Environment:OMERO_MASTER=(unset)
Environment:OMERO_USERDIR=(unset)
Environment:OMERO_TMPDIR=(unset)
Environment:PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/net/isilonP/public/rw/homes/mi_adm/bin:/net/isilonP/public/rw/homes/mi_adm/omero_stage_ves-hx-ca/Ice-3.5.1/bin
Environment:PYTHONPATH=:/net/isilonP/public/rw/homes/mi_adm/omero_stage_ves-hx-ca/Ice-3.5.1/python:/net/isilonP/public/rw/homes/mi_adm/omero_stage_ves-hx-ca/OMERO.server-5.1.2-ice35-b45/lib/python
Environment:ICE_HOME=(unset)
Environment:LD_LIBRARY_PATH=:/net/isilonP/public/rw/homes/mi_adm/omero_stage_ves-hx-ca/lib:/net/isilonP/public/rw/homes/mi_adm/omero_stage_ves-hx-ca/Ice-3.5.1/lib64/
Environment:DYLD_LIBRARY_PATH=(unset)

OMERO SSL port:4064
OMERO TCP port:4063
OMERO data dir:'../../omero_data/OMERO' Exists? True Is writable? True
OMERO temp dir:'/net/isilonP/public/rw/homes/mi_adm/omero/tmp' Exists? True Is writable? True (Size: 0)

JVM settings: Blitz -Xmx6000m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions # Settings({'max_system_memory': '12000', 'system_memory': '14000', 'percent': '50'})
JVM settings: Indexer -Xmx2400m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions # Settings({'max_system_memory': '12000', 'system_memory': '14000', 'percent': '20'})
JVM settings: Pixeldata -Xmx2400m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions # Settings({'max_system_memory': '12000', 'system_memory': '14000', 'percent': '20'})
JVM settings: Repository -Xmx1200m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions # Settings({'max_system_memory': '12000', 'system_memory': '14000'})

OMERO.web status... [RUNNING] (PID 15071)

bin/omero fs repos
Previously logged in to ves-ebi-cf:4064 as root

[kennethgillen]

Hi Jonathan,

config get:
bin/omero fs repos
Previously logged in to ves-ebi-cf:4064 as root

After logging into OMERO, there was no more output from that command?

Many thanks,

Kenny

[jwarren]

Hi Kenny

Yep- that was it! Strikes me as odd considering it's talking about a different machine and omero instance than the one we are looking at which would be pointing at the nfs directory that it complains about... but If I stop the omero in the current directory the server stops...which leads me to think the web pages are pointing at the correct omero instance.

I should also mention that I used an in-place import so that may affect the moving of the data dir?

I'm just looking at the fs command now - as it's the first time I've encountered it.

[jwarren]

Im now thinking the nfs directory from where the images were imported via the in-place importer is not accessible and so we get the error message: Error instantiating pixel buffer ..ManagedRepository/root/some-date/image.tiff

??

I guess as it was an in-place import the original nfs directory needs to be accessible?

[jwarren]

I get exactly the same response from the fs command on all my machines that as far as I know are running ok. They all talk about the same machine access even though they are running on a different machine.

[jwarren]

So my next question is as I can't access the same nfs directory from all our instances due to some being hosted in london and some here in cambridge - am I better off using the normal importer rather than the in place importer. After upload now the pyramid creation happens in the background for all images uploaded? So if I wait a while then I can set the data directory to be read only and replicate?

[kennethgillen]

Hi Jonathan

Im now thinking the nfs directory from where the images were imported via the in-place importer is not accessible and so we get the error message: Error instantiating pixel buffer ..ManagedRepository/root/some-date/image.tiff

??

I guess as it was an in-place import the original nfs directory needs to be accessible?

You're absolutely right on this point. The in-place import is for the use case where the data is permanently kept outside of OMERO (or shared with other users) - but it _absolutely must_ be kept available to OMERO. If that won't be the case, another type of import transfer option should be used. [1]

These limitations are detailed on our OMERO documentation page on in-place import: see https://www.openmicroscopy.org/site/sup ... mport.html

[1] https://www.openmicroscopy.org/site/sup ... er-options

All the best,

Kenny