Issues with the built-in web server

[ehrenfeu]

Hi all,

we're currently experiencing problems with the built-in web server. We use it to "deploy" the webstart client, so people will always be provided with the correct version of insight.

The web server process gets started (manually, using "OMERO.server/bin/omero web start") on the omero machine and is running in a screen session. At some point yesterday apparently it stopped reacting to requests, although the process itself didn't show any strange messages when I reattached the screen session this morning. I stopped the process using Ctrl-C, which seemed to work. Strangely during the shutdown a bunch of errors was printed to the console (I don't think this was an issue of screen holding back the output as I pressed enter several times in this console before I tried to shutdown the web service).

After the process had terminated I tried to restart it again, but it was complaining the port was already in use. So I had to manually kill (-15) the python process serving "manage.py runserver 0.0.0.0:4080 --noreload". With the next attempt, the service would come up again and provide the web interface on 4080.

So we thought it should be fine for the moment, but as soon as the user that noticed the problem yesterday tried to connect to the web server again, we got the same messages again that were already printed during the previous (Ctrl-C) shutdown:

Code: Select all

Traceback (most recent call last):
  File "/home/omeronas/OMERO.server/lib/python/django/core/servers/basehttp.py", line 284, in run
    self.finish_response()
  File "/home/omeronas/OMERO.server/lib/python/django/core/servers/basehttp.py", line 324, in finish_response
    self.write(data)
  File "/home/omeronas/OMERO.server/lib/python/django/core/servers/basehttp.py", line 403, in write
    self.send_headers()
  File "/home/omeronas/OMERO.server/lib/python/django/core/servers/basehttp.py", line 467, in send_headers
    self.send_preamble()
  File "/home/omeronas/OMERO.server/lib/python/django/core/servers/basehttp.py", line 382, in send_preamble
    self._write('HTTP/%s %s\r\n' % (self.http_version,self.status))
  File "/home/omeronas/OMERO.server/lib/python/django/core/servers/basehttp.py", line 511, in _write
    self.stdout.write(data)
  File "/usr/lib64/python2.6/socket.py", line 324, in write
    self.flush()
  File "/usr/lib64/python2.6/socket.py", line 303, in flush
    self._sock.sendall(buffer(data, write_offset, buffer_size))
error: [Errno 32] Broken pipe
----------------------------------------
Exception happened during processing of request from ('131.152.18.53', 54262)
Traceback (most recent call last):
  File "/usr/lib64/python2.6/SocketServer.py", line 283, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib64/python2.6/SocketServer.py", line 309, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib64/python2.6/SocketServer.py", line 322, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/home/omeronas/OMERO.server/lib/python/django/core/servers/basehttp.py", line 570, in __init__
    BaseHTTPRequestHandler.__init__(self, *args, **kwargs)
  File "/usr/lib64/python2.6/SocketServer.py", line 618, in __init__
    self.finish()
  File "/usr/lib64/python2.6/SocketServer.py", line 661, in finish
    self.wfile.flush()
  File "/usr/lib64/python2.6/socket.py", line 303, in flush
    self._sock.sendall(buffer(data, write_offset, buffer_size))
error: [Errno 32] Broken pipe
----------------------------------------


How could we debug this any further? We definitely need to sort out this issue as this particular user will have a very strong impact on the success of our OMERO

Could it be related to the browser / OS on the user's machine? Sounds like a very strange guess to me, but currently I can't think of anything else that's different from our machines (where it's still working) to the user's machine. Can't even imagine network latencies as an issue here, as the user currently is connected to the local network.

Any help is greatly appreciated!

Thanks,
Niko

[cxallan]

I'd strongly suggest you not use the built-in web server in production. There are so many things that can affect the single threaded development server and it can only service one request at a time. Is there a particular reason you have decided to use it?

[ehrenfeu]

Thanks for the fast reply. According to our IT department, they decided to put the setup of the "production" web server on hold for

incompatibility of proposed "fastcgi" method with Apache 2.4.x (or Apache 2.2.x)

Guess then we have to get this sorted?

[cxallan]

Which Linux distribution are you deploying on? There is support for fastcgi with Apache 2.4.x and 2.2.x. We run this configuration regularly.

[ehrenfeu]

CentOS release 6.3 (Final)

[cxallan]

Have them add the RPMForge Yum repository (http://wiki.centos.org/AdditionalResour ... s/RPMForge) and install mod_fastcgi.

[ehrenfeu]

Apparently, mod_fcgid is already there with our apache installation. Do you have any experience with that or would you rather prefer mod_fastcgi?

[cxallan]

mod_fcgid will not work, it does not support sockets and must perform process management itself. We only support the use of mod_fastcgi.

[ehrenfeu]

Ah, thanks. That's very useful information!