Page 1 of 1

New API and custom Bridge for 4.3.x

PostPosted: Tue Nov 29, 2011 4:51 pm
by bhcho
Hi all,

We are moving ourselves to 4.3.x.
The first step for me is to have a corresponding Bridge that works on 4.3.x OMERO server.
I confirmed my bridge is called from Indexer log.
But it seems like there's major change in APIs.
here's my bridge java code (which was working from 4.2.1)
Code: Select all
/*
*   $Id$
*
*   Copyright 2008 Glencoe Software, Inc. All rights reserved.
*   Use is subject to license terms supplied in LICENSE.txt
*/

package edu.cmu.search.bridges;

import edu.cmu.search.bridges.ExperimentFileParser;
import edu.cmu.search.bridges.RunPython;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.Reader;
import java.util.HashMap;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Collections;
import java.util.Iterator;

import java.util.Properties;
import java.io.*;
import org.apache.commons.io.filefilter.*;


//import omero.api.IMetadataPrx;
//import omero.api.ServiceFactoryPrx;
//import omero.api.ServiceFactory;
//import omero.api.RawFileStorePrx;
//import omero.api.RawFileStore;
//import omero.model.IObject;

import ome.model.annotations.Annotation;
import ome.model.annotations.FileAnnotation;

import ome.model.containers.Dataset;
import ome.model.containers.DatasetImageLink;
import ome.model.containers.Project;
import ome.model.containers.ProjectDatasetLink;
import ome.model.core.Image;
import ome.model.core.OriginalFile;
import ome.services.fulltext.BridgeHelper;
import ome.services.fulltext.SimpleLuceneOptions;



import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.hibernate.search.bridge.FieldBridge;
import org.hibernate.search.bridge.LuceneOptions;

/**
* Example custom {@link FieldBridge} implementation which parses all
* {@link Image} names from a {@link Project} and inserts them into the index
* for that {@link Project}.
*
* @author Josh Moore, josh at glencoesoftware.com
* @since 3.0-Beta3
*/
public class ExperimentFileBridge extends BridgeHelper {
 
//     ExperimentFileParser exParser;
  /**
   * If the "value" argument is a {@link Project}, this
   * {@link FieldBridge bridge} gathers all images and adds them to the index
   * with a slightly reduced boost value. The field name of the image name is
   * "image_name" but the values are also added to the
   * {@link BridgeHelper#COMBINED} field via the
   * {@link #add(Document, String, String, org.apache.lucene.document.Field.Store, org.apache.lucene.document.Field.Index, Float)}
   * method.
   */
  @Override
  public void set(final String name, final Object value,
                  final Document document, final LuceneOptions _opts) {
   
    if (value instanceof Image) {
      logger().info("Scheduling all Experiment Files of " + value + " for re-indexing");
      ExperimentFileParser exParser = new ExperimentFileParser();
      final Image i = (Image) value;
      String user = i.getDetails().getOwner().getOmeName();
      String group = i.getDetails().getGroup().getName();
      Long imageId = i.getId();
     
      // this can be used later. compare the created time with the current time.
      // if the difference is small, then you trigger the feature calculation
      Long update = i.getDetails().getUpdateEvent().getTime().getTime();
      Long creation = i.getDetails().getCreationEvent().getTime().getTime();
      java.util.Date today = new java.util.Date();
      Long now = today.getTime();
     
     
      //RunPython rp0 = new RunPython();
      //String st = rp0.testCreateUpdate(creation, update, now);
     
      /******************* Block Start for triggering feature calculation     ******************/
     
      RunPython rp = new RunPython();
      logger().info("Before Triggering Feature Calculation for " + value);
      Boolean isFeatureFile = false;
      for (Annotation a : i.linkedAnnotationList()) {
        logger().info("Scheduling all Experiment Files of " + " for re-indexing");
        logger().info("Scheduling all Experiment Files of " + a + " for re-indexing");
        if (a instanceof FileAnnotation) {
          FileAnnotation ann2 = (FileAnnotation) a;
         
          String fName = ann2.getFile().getName();
          String subfName = fName.substring(fName.length()-3);
         
          if ((subfName != null)){
            if (subfName.equalsIgnoreCase(".h5")){
              isFeatureFile = true;
            }
          }
         
          if (!isFeatureFile){
            for (final DatasetImageLink dil : i.unmodifiableDatasetLinks()) {
              //logger().info("WIthin Triggering Feature Calculation for " + value);
              Long datasetID = dil.parent().getId();
              //logger().info("WIthin Triggering Feature Calculation for " +  Long.toString(datasetID));
              String status = rp.runFeatCalc(Long.toString(imageId), Long.toString(datasetID), user, group);
              //String status = rp.runFeatCalc(Long.toString(imageId), Long.toString(datasetID),Long.toString(creation), Long.toString(now));
              logger().info("WIthin Triggering Feature Calculation STATUS: " +  status);
            }
          }
        }
      }
     
      /******************* Block Ends  ******************/
     
      /******************* Block Start for Experiment File Indexing to Lucene Index     ******************/
      String tagNs = null;
      String tagName = null;
      String subtagName = null;   
      String XML = null;
      HashMap <String, String> map = null;
      // get the annotations for the image and look for a FileAnnotation
      for (Annotation a : i.linkedAnnotationList()) {
        if (a instanceof FileAnnotation) {
          FileAnnotation ann = (FileAnnotation) a;
          tagNs = ann.getNs();
          tagName = ann.getFile().getName();
          subtagName = tagName.substring(tagName.length()-7);
         
          if ((tagNs != null) && (subtagName != null)){
            if (tagNs.equalsIgnoreCase("openmicroscopy.org/omero/editor/experiment") && subtagName.equalsIgnoreCase("cpe.xml")){
              OriginalFile file = ann.getFile();
              ome.io.nio.OriginalFilesService fileservice = (ome.io.nio.OriginalFilesService)ome.system.OmeroContext.getManagedServerContext().getBean("/OMERO/Files");
              String path = fileservice.getFilesPath(file.getId());
              String format = file.getMimetype();
              map =  exParser.parse(new File(path));
             
              // parse the XML string
              Iterator<String> iterator = map.keySet().iterator();
              while (iterator.hasNext()) {
                String key = (String) iterator.next();
                add(document, key, map.get(key), _opts);
              }
            }
          }
        }
      }
      /******************* Block Ends  ******************/                 
    }
  }
}


there's no compile error.
from the Indexer log, I saw the following part has been successfully went through.
Code: Select all
logger().info("Before Triggering Feature Calculation for " + value);

But after that, I don't see any log, meaning there might be many run time errors from the line of..
Code: Select all
for (Annotation a : i.linkedAnnotationList()) {


Could anyone give me the link for API documents of 4.3.x so that I can get help in order to update my code?

Best,
BK

Re: New API and custom Bridge for 4.3.x

PostPosted: Tue Nov 29, 2011 5:20 pm
by wmoore

Re: New API and custom Bridge for 4.3.x

PostPosted: Tue Nov 29, 2011 6:55 pm
by bhcho
Hi Will,

Thanks and I'll take a look at those links.

the way I debug my bridge is..
1. modify my java code by inserting some loggers
2. generate the jar file and drop it into server
3. restart OMERO server
4. import an image via OMERO.importer
5. look at the Indexer log file in server.

Is there any easier/faster way to do this?

BK

Re: New API and custom Bridge for 4.3.x

PostPosted: Wed Nov 30, 2011 8:55 am
by jmoore
Hi BK,

it won't do exactly what you want, but you may look into modifying "components/server/src/ome/services/fulltext/Main.java" in order to index a single image. It's run by components/tools/OmeroPy/src/omero/plugins/admin.py:
Code: Select all
bin/omero admin reindex ...


With that command, for example, you can re-index all images in the foreground. But that can take some time. If you added in another subcommand:
Code: Select all
bin/omero admin reindex --image 1

then you wouldn't have to wait long at all, including no server restart etc.

Btw, if you do implement such a command, feel free to open a pull request and we'll integrate it into the rest of the code base.

cheers,
~Josh.