Automatically generate key/value pairs

[kverbist]

Hello,

I also posted this question at the Bio-Formats section, but to no avail. Perhaps this question belongs here.

What I'm trying to accomplish is that once the file becomes imported, and read by the bio formats jar for analyzing meta data, I can write some code which also converts some meta data into either tags or key/value pairs such that the MRC files are queryable later on. I understand that it is not possible to seach on Meta data.

Can someone point me in the right direction? I have adapted some code in the bio formats Java files, and am also capable of programming in Python. So all I really need is some advice in which files it is best to write some code for automatic key/value pair generation.

Regards,
KVerbist

[sbesson]

Hi,

Following up of the discussion on viewtopic.php?f=13&t=7948

I understand that it is not possible to seach on Meta data.

Right, the OriginalMetadata read from Bio-Formats is not stored in the database and is not indexed by Lucene.

What I'm trying to accomplish is that once the file becomes imported, and read by the bio formats jar for analyzing meta data, I can write some code which also converts some meta data into either tags or key/value pairs such that the MRC files are queryable later on.

You should be able to load the original metadata of an imported image by submitting an OriginalMetadataRequestI command to the serer. For each key/value in the metadata table, you should be able to create a corresponding map annotation and link it to the image using the update service.

Can someone point me in the right direction? I have adapted some code in the bio formats Java files, and am also capable of programming in Python.

You should be able to make similar calls in Python or Java. See for instance this method of the Python BlitzGateway used to retrieve the original metadat given an image ID.

Let us know if you need more information or examples.
Best,
Sebastien

[kverbist]

Hello Sebastien,

Thanks for your feedback. I think I need to explain a little better what I want to accomplish.

Basically the idea is to keep using Omero Insight. Right now the extra custom meta data which I store through:

Code: Select all

in.seek(HEADER_SIZE);
addGlobalMeta("a_tilt", in.readFloat());
addGlobalMeta("b_tilt", in.readFloat());

addGlobalMeta("x_stage", in.readFloat());
addGlobalMeta("y_stage", in.readFloat());
addGlobalMeta("z_stage", in.readFloat());

addGlobalMeta("x_shift", in.readFloat());
addGlobalMeta("y_shift", in.readFloat());

addGlobalMeta("defocus", in.readFloat());
addGlobalMeta("exp_time", in.readFloat());
addGlobalMeta("mean_int", in.readFloat());
addGlobalMeta("tilt_axis", in.readFloat());
addGlobalMeta("pixel_size", in.readFloat());
addGlobalMeta("magnification", in.readFloat());
addGlobalMeta("ht", in.readFloat());
addGlobalMeta("binning", in.readFloat());
addGlobalMeta("appliedFocus", in.readFloat());


I can already see when I check Insight for the meta data.

However, what I really want now, is that when you perform a search in InSight, I can either search for the keyword "x_stage" or for its value, and then it will recover this record. I know I can manage this by manually converting all meta data into actual key/value pairs in Insight, or by creating tags or description.
So basically I want to avoid having to do all this manually, but at the same time as making meta data, I would like to make either tags or key/value pairs for this MRC record.

So in short, I can already retrieve the meta data, that is not the issue. I need to convert this meta data in searchable tags or key/value pairs such that the search option in Insight will actually find the MRC records based on my keywords which reflect one or more of the custom meta data I entered. And it would be great if I could insert these tags or key/value pairs at the same time I am adding the meta data, so in the Bio-Formats library.

Thank you and regards,
KVerbist

[sbesson]

Hi KVerbist,

thanks for the clarification. The classes I pointed you at in my previous post would only work in the context of a post-import script that would convert automatically (some of) the original metadata into map annotations on the server.

If you want to have these map annotations created at import time, you will need to create the relevant annotations and link them to the image in the custom reader. An advanced example of such annotation creation/linking is done in the readers defining a modulo annotation. A simpler example is achieved by the FakeReader which can be used to generate test images with linked annotations.

In the reader source code, you might want to look at this section which creates various types of annotations and links them the the image. For key/value pairs, you may want to investigate the MetadataStore.setMapAnnotation* methods.

Best,
Sebastien

[wmoore]

Hi,

One other option that might be a bit more work, but is probably the "right" way to do this is to add a custom "bridge" to the server that tells Lucene how to index the Original Metadata.

See https://www.openmicroscopy.org/site/sup ... idges.html

There's a lot on the forums about this from user 'bhcho', since they wrote a parser for OMERO.editor xml files.
This is several years old now but may still be useful.
E.g.
viewtopic.php?f=6&t=553
viewtopic.php?f=6&t=934&p=3408
and you can find more by searching forums!

[kverbist]

Hi guys,

Thanks for all the advice. This week I will try to make one of the suggested options work.

Regards,
KVerbist