Importing Metadata from a separate text file

[sduerrenberger]

I have images from a Hitachi S-4800 with the metadata in a separate text file. Is it possible to import the images together with the metadata from the files? At the moment I'm only abe to import the images and the metadata is lost.

The images are tif-files, the text file has the same name as the image with a .txt extension. The text files look like this:

Code: Select all

[SemImageFile]
InstructName=S-4800
SerialNumber=HI-9018-0003
DataNumber=ZMB-UniBS
SampleName=ambrosia
Format=tif
ImageName=ambrosia_q01.tif
Directory=Z:\Marcel\Ambrosia\
Date=09/22/2006
Time=16:36:49
Media=NE[ ]
DataSize=1280x960
PixelSize=55.12153
SignalName=SE(M)
AcceleratingVoltage=5000 Volt
DecelerationVoltage=0 Volt
Magnification=1800
WorkingDistance=8300 um
EmissionCurrent=18400 nA
LensMode=Normal
PhotoSize=1000
Vacuum=
MicronMarker=30000
SubMagnification=0
SubSignalName=
SpecimenBias=0 V
Condencer1=1000
ScanSpeed=Slow1
CalibrationScanSpeed=25
ColorMode=Grayscale
ColorPalette=
ScreenMode=Small Screen
Comment=
KeyWord1=pollen
KeyWord2=Biologz
Condition=Vacc=5kV   Mag=x1.80k   WD=8.3mm
DataDisplayCombine=1
StageType=2
StagePositionX=26900000
StagePositionY=20871000
StagePositionR=0
StagePositionZ=8000000
StagePositionT=0


[wmoore]

The simplest, but perhaps not the 'nicest' solution is to import the image as normal, then use the Insight client to attach the txt file to the image. Select the image -> In the right-hand panel choose: attachment [+] then upload the local file. This keeps the metadata with the image and you can read the text file in Insight using the 'view' option after attaching the file.

However, this is not nice for a large number of images.
If I were going to do this, I would write a python script that connected to the server, found the images (E.g. by dataset ID) and uploaded the local file to the matching named image.
A potential improvement is that you could upload the txt file as an "original_metadata.txt" attachment, which would mean that it would be displayed in Insight under the Acquisition tab "Original Metadata" for the image, without having to open a separate window to view the data.

I can help you with this, but you'd need to get set up with Python etc.
Have a read here and let me know if you're interested. http://trac.openmicroscopy.org.uk/ome/w ... tBeginners

Cheers,

Will.

[sduerrenberger]

Thank you for your detailed reply! We do have a large number of images and we need to make it as easy for the users as possible.

Regarding the python script, is there a possibility to run it within Importer or does the user have to run it manually after importing? That would still require the user to log into Insight, get the Dataset ID and then run the python script (entering the ID and local path) if I understand that correctly.

I already picture people forgetting to run the script and wondering where their metadata went.

[cxallan]

What Will has mentioned above with original_metadata.txt is quite tricky overall and something that we use internally for extra metadata. There are multiple places where you must know exactly what to do with annotation namespaces and the like, not for the faint of hear. It's also not something I would suggest using in place of the proper solution which is to build a Bio-Formats reader for your image data and metadata.

Writing a Bio-Formats reader is really what needs to happen as this satisfies both the technical and user requirements that you've mentioned. No user we've ever met likes to do extra steps on import.

So...

How consistent are these metadata files? Does the file come from the instrument itself? Is there one for each image?

[sduerrenberger]

Having a Bio-Formats reader available would be a great solution, but how hard would it be to implement? And to add it to the current Omero version?

How consistent are these metadata files? Does the file come from the instrument itself? Is there one for each image?

The files are very consistent. They are saved from the instrument itself which allows to save tiff, jpeg or bitmap, but saves the metadata in this separate file for all of the formats. The metadata file always has the same keys on the same lines, only the values change.

The metadata file is saved for every image and always has the same name as the image with a .txt extension.

The Tiff image itself doesn't seem to have any metadata at all.

[cxallan]

It would probably take us a few days to do, unless you want to try and embark on the process yourself. With data provided soon we may be able to get the file format included in Beta-4.3, it just depends on how busy things are on the file format front between now and release.

I assume you could get us some representative datasets so that we might be able to support the file format?

[sduerrenberger]

It would probably take me a long time to do it myself so if you can add it, that would be great! I'm not familiar with the inner workings of bio formats. It would be great to have it in the next release of course.

Please tell me how I can help best. I can provide all of the datasets needed from the microscope. I assume you need a dataset with each of the three image formats used?

[mlinkert]

Please tell me how I can help best. I can provide all of the datasets needed from the microscope. I assume you need a dataset with each of the three image formats used?

It would be best if you could send at least one (preferably two or three) datasets in each format, so that we can adequately test the reader(s). I have sent instructions privately on how to send data to us; if you have any trouble, please do let us know.

Regards,
-Melissa