lightweight bioformats aware image server sought

[malcook]

I think OMERO might provide a solution, or a piece of it, to a problem at hand. I am seeking implementation ideas.

I seek a server implementing a URL API allowing to retrieve subsets of the MANY THOUSANDS of small TIFF images embedded in an bio-formats compatible (Amnis FlowSight) document.

I expect my users _could_ load them into an OMERO server and address them that way, however I think I need something more lightweight and providing a more transparent process to my users.

What I _think_ I want is a server that ingests bioformat compatible files "on demand", and somehow caches the meta-data for the most recently files, allowing for fast access.

The server could be deployed to the users localhost (PCs) or even on a central linux server having a shared file system (obviating the need for the documents to be moved or transmitted).

My users in the laboratory currently export the TIFFs to the file system and can address them that way. It is quite performant, but I want to relieve my user of the need to perform this export. It clutters their drives, makes needless copies, and requires them to adhere to naming conventions when exporting.

I'm currently thinking something might be cobbled using the python bioformats wrappers and a lightweight web framework (such as flask).

Perhaps some subset of features of OMERO could be deployed to this effect, however I hope to avoid the overhead of committing to permanently saving the meta-data of images, and it would have to provide quick ingest and teardown. So far, I have only read a bit about OMERO but have not deployed it myself.

The "client" of the URL API will be an instance of TIBCO Spotfire running on the local desktop. This should not matter to the implementation, but I mention it so you can visualize my problem a little better. Spotfire is used to analyze and plot matrices of feature values extracted from the images embedded in the AMNIS Flowsight document. On occasion, the user may select some datapoints and select to see the underlying image. The numeric image ID is a key which is embedded in a URL template to retrieve the image. Currently, I'm retrieving TIFFs exported to the file system. I want now instead to extract them "on demand" using the proposed server.

To boot, I envision submitting a document to such a server to possibly be integrated into the MS Windows desktop as a right-click action. Or something as simple and transparent.

Does such a thing already exist?

Is this a trivial application of OMERO?

Is is bound to be allot more complicated than a quick python/bioformats/flask caching server?

Am I leaving out essential details?

Thanks for your consideration and assistance.

[jmoore]

What I _think_ I want is a server that ingests bioformat compatible files "on demand", and somehow caches the meta-data for the most recently files, allowing for fast access.

Your first step should likely be looking at the "Memoizer" class then. See https://www.openmicroscopy.org/site/support/bio-formats5.1/developers/matlab-dev.html#improving-reading-performance

...

Perhaps some subset of features of OMERO could be deployed to this effect, however I hope to avoid the overhead of committing to permanently saving the meta-data of images, and it would have to provide quick ingest and teardown. So far, I have only read a bit about OMERO but have not deployed it myself.

A light-version of OMERO has certainly come up before, but it's not something we provide at the moment.

Does such a thing already exist?

Not from the OME team itself, but we're certainly interested in hearing what anyone else has come up with.

Is this a trivial application of OMERO?

In some sense, yes, since from what I can tell OMERO will do everything you're asking for here, but it brings the burden of running the DB etc.

Is is bound to be allot more complicated than a quick python/bioformats/flask caching server?

It always is. https://en.wikipedia.org/wiki/Fallacies ... _computing First concern that comes to mind is security (login policies, etc) There's certainly a balance here. If what you really want is simplicity, then the framework you spell out makes sense and is even something we'd like to support in the mainline. There's a tendency for these things to grow though.

All the best,
~Josh.

[malcook]

Josh - splendid reply - thanks - I will surely share the experience if we proceed with the effort - memoize looks great if I were writing in Java - any reason not instead to plan to work in python/sinatra (where I am more comfortable) - do you think it would be less much performant as a result of the addition bridge?

[jmoore]

I will surely share the experience if we proceed with the effort

Sounds good. Thanks.

- memoize looks great if I were writing in Java - any reason not instead to plan to work in python/sinatra (where I am more comfortable)

I was thinking more you'll need Memoizer regardless. In way of a quick crash course:

• Typically one doesn't know what type of file format is being read, so `ImageReader` is instantiated and `setId` called with a path to the file.
• To speed things up if the file format is known, `SpecificReader` is instantiated instead.
• In either case, if to speed things up substantially more, either instance can be wrapped with a `Memoizer`.

- do you think it would be less much performant as a result of the addition bridge?

OMERO.web currently already does Python -> remote invocation -> Java for all of its communication, so I don't think what you're proposing will be substantially faster or slower.

Cheers,
~Josh