Log level / verbosity

[ehrenfeu]

Hi,

we're experiencing a very fast growth of OMERO's logfiles, even though the system is not (yet) under heavy usage. For example, the current Blitz-0.log is approx. 10 days old and already 200 MB large.

I did a quick check which categories are the chattiest ones:

Code: Select all

   1025        ome.logic.JavaImageScalingService
   1132     ome.security.basic.BasicEventContext
   3045                ome.services.ThumbnailCtx
   4583  ome.services.blitz.fire.SessionManagerI
   5040                 ome.io.nio.PixelsService
   5843           omeis.providers.re.HSBStrategy
   7557                       omero.cmd.SessionI
  12546 ome.services.sessions.state.SessionCache
  40945        ome.security.basic.CurrentDetails
  47331              omeis.providers.re.Renderer
  96201              ome.io.nio.RomioPixelBuffer
178147          ome.security.basic.EventHandler
291255                  org.perf4j.TimingLogger
725537         ome.services.util.ServiceHandler


Checking the forum and the wiki docs, I didn't find anything about how to decrease the verbosity on a production-like system. So my next step would be to add new entries (or adjust existing ones) for at least the most verbose categories from above to the logging configuration in etc/log4j.xml - or do you have any other suggestions?

Cheers
~Niko

[cxallan]

That's exactly how you modify the verbosity. The file is read periodically during runtime so changing logging verbosity either up or down does not require a server restart.

[ehrenfeu]

Yep, I've figured that out for the periodic reading from the comments in the file

Is there a common set of rules that has proven reasonable on a production machine? Otherwise I'll just filter out the stuff producing the most messages and re-enable them as needed...

[cxallan]

The reason why those are the defaults is that the OMERO team considers those reasonable for most users. It helps us extensively with debugging a lot of problems without users having to fiddle around with logging levels. We regularly use even more verbose logging configurations in "production" as the benefits of ad hoc debugging, as far as we're concerned, vastly outweigh the storage costs. So, if what's there is not working for you I'd just filter out what you feel you need to.

[jmoore]

I'd just add that if you have an issue and you send us a log file without ome.security.basic.EventHandler it will be the first thing I get you to change.

org.perf4j, on the other hand, is far less used (though interesting sometimes to look back at historically)

Cheers,
~Josh

[ehrenfeu]

The reason why those are the defaults is that the OMERO team considers those reasonable for most users. It helps us extensively with debugging a lot of problems without users having to fiddle around with logging levels. We regularly use even more verbose logging configurations in "production" as the benefits of ad hoc debugging, as far as we're concerned, vastly outweigh the storage costs. So, if what's there is not working for you I'd just filter out what you feel you need to.

Well, I'm fine either way. It's not a real storage costs issue (some hundred megabytes of logfiles probably disappear in the overall accounting for everyone running a real image database anyway), I was just curious if this is a bug in our configuration and if the logfiles are a bit more handy in the common scenario...

[ehrenfeu]

I'd just add that if you have an issue and you send us a log file without ome.security.basic.EventHandler it will be the first thing I get you to change.

org.perf4j, on the other hand, is far less used (though interesting sometimes to look back at historically)

Cheers,
~Josh

Well, I guess we'll stick to the default settings then

As long as I don't have to open them in some stupid editor, I don't care too much about their size

Thanks Chris & Josh for your help!
~Niko

[cxallan]

I'd just add that if you have an issue and you send us a log file without ome.security.basic.EventHandler it will be the first thing I get you to change.

org.perf4j, on the other hand, is far less used (though interesting sometimes to look back at historically)

Cheers,
~Josh

Well, I guess we'll stick to the default settings then

As long as I don't have to open them in some stupid editor, I don't care too much about their size

Thanks Chris & Josh for your help!
~Niko

No problem at all.

[kennethgillen]

For anyone coming across this thread from 2014 onwards, editing OMERO-DIST/etc/logback.xml can alter the server's log level.

Best,

Kenny