Apache SOLR and GeneOntology: Creating the JQUERY-UI client (with autocompletion)

In the previous post I showed how to use Apache SOLR to index the content of gene ontology. In this post I will show how the SOLR server can be used as a source of data for a "JQuery-UI autocomplete" HTML input.

The JQuery autocomplete Widget expects an JSON array of Objects with label and value properties :

[ { label: "Choice1", value: "value1" }, ... ]

We need to tell SOLR to return the GO terms formatted this way.

Fortunately SOLR provides a mechanism to captures the output of the normal XML response and applies a XSLT transformation to it.

The following XSLT stylesheet transforms the XML response to JSON, it is created in the conf/xslt directory (apache-solr-3.5.0/example/solr/conf/xslt/solr2json.xsl)


We can test the stylesheet by adding the tr parameter in the query URL:

$ curl "http://localhost:8983/solr/select?q=go_definition:cancer&wt=xslt&tr=solr2json.xsl"

[
    {
        "label": "Z-phenylacetaldoxime metabolic process",
        "value": "GO:0018983"
    },
    {
        "label": "epothilone metabolic process",
        "value": "GO:0050813"
    },
    {
        "label": "epothilone biosynthetic process",
        "value": "GO:0050814"
    },
    {
        "label": "aflatoxin biosynthetic process",
        "value": "GO:0045122"
    },
    {
        "label": "aflatoxin metabolic process",
        "value": "GO:0046222"
    },
    {
        "label": "aflatoxin catabolic process",
        "value": "GO:0046223"
    }
]

The jquery client

Let's create the HTML/JQuery Client, I have downloaded the component for jquery and jquery-ui in the apache-solr-3.5.0/example/work/Jetty_xxxx_solr.war__solr__xxxx/webapp/ (and that's a bad idea, everything will be deleted with the SOLR server will be stopped or re-deployed).

I've also created the following HTML page 'solr.html' in the same directory: Let's visit http://localhost:8983/solr/solr.html to test the page:


(image via openwetware)

 That worked ! :-)

That's it,

Pierre

Indexing the content of Gene Ontology with apache SOLR

Via Wikipedia:"Solr (http://lucene.apache.org/solr/) is an open source enterprise search platform from the Apache Lucene project. Its major features include powerful full-text search, hit highlighting, faceted search, dynamic clustering, database integration, and rich document (e.g., Word, PDF) handling. Providing distributed search and index replication, Solr is highly scalable." In the this post, I'll show how I've used SOLR to index the content of GeneOntology.

Download and install SOLR

Download from http://mirrors.ircam.fr/pub/apache/lucene/solr/3.5.0/apache-solr-3.5.0.tgz.

tar xvfz apache-solr-3.5.0.tgz
rm apache-solr-3.5.0.tgz

Configure schema.xml

We need to tell SOLR about the which fields of GO will be indexed, what are their type, how they should be tokenized and parsed. This information is defined in the schema.xml. The following components will be indexed: accession, name, synonym and definition. Edit apache-solr-3.5.0/example/solr/conf/schema.xml and add the following <fields>:

<field name="go_name" type="text_general" indexed="true" stored="true" multiValued="false"/>
<field name="go_synonym" type="text_general" indexed="true" stored="true" multiValued="true"/>
<field name="go_definition" type="text_general" indexed="true" stored="true" multiValued="false"/>

Start the SOLR server

In this example, the SOLR server is started using the simple Jetty server provided in the distribution:

$ cd apache-solr-3.5.0/example/example
$ java -jar start.jar

(...)

Indexing Gene Ontology

Go is downloaded as RDF/XML from http://archive.geneontology.org/latest-termdb/go_daily-termdb.rdf-xml.gz
 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE go:go PUBLIC "-//Gene Ontology//Custom XML/RDF Version 2.0//EN" "http://www.geneontology.org/dtd/go.dtd">

<go:go xmlns:go="http://www.geneontology.org/dtds/go.dtd#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
    <rdf:RDF>
        <go:term rdf:about="http://www.geneontology.org/go#GO:0000001">
            <go:accession>GO:0000001</go:accession>
            <go:name>mitochondrion inheritance</go:name>
            <go:synonym>mitochondrial inheritance</go:synonym>
            <go:definition>The distribution of mitochondria, including the mitochondrial genome, into daughter cells after mitosis or meiosis, mediated by interactions between mitochondria and the cytoskeleton.</go:definition>
            <go:is_a rdf:resource="http://www.geneontology.org/go#GO:0048308" />
            <go:is_a rdf:resource="http://www.geneontology.org/go#GO:0048311" />
        </go:term>
        <go:term rdf:about="http://www.geneontology.org/go#GO:0000002">
            <go:accession>GO:0000002</go:accession>
            <go:name>mitochondrial genome maintenance</go:name>
            <go:definition>The maintenance of the structure and integrity of the mitochondrial genome; includes replication and segregation of the mitochondrial chromosome.</go:definition>
            <go:is_a rdf:resource="http://www.geneontology.org/go#GO:0007005" />
            <go:dbxref rdf:parseType="Resource">
                <go:database_symbol>InterPro</go:database_symbol>
(...)

We now need to transform this XML file to another XML file that can be indexed by the SOLR server.  

"You can modify a Solr index by POSTing XML Documents containing instructions to add (or update) documents, delete documents, commit pending adds and deletes, and optimize your index."

The following XSLT stylesheet is used to transform the RDF/XML for GO:

$ xsltproc --novalid go2solr.xsl go_daily-termdb.rdf-xml.gz > add.xml
$ cat add.xml

Before indexing the current disk usage under apache-solr-3.5.0 is 136Mo. We can now use the java utiliy post.jar to index GeneOntology.

 $ cd  ~/package/apache-solr-3.5.0/example/exampledocs
 $ java -jar post.jar  add.xml

SimplePostTool: version 1.4
SimplePostTool: POSTing files to http://localhost:8983/solr/update..
SimplePostTool: POSTing file jeter.xml
SimplePostTool: COMMITting Solr index changes..

After indexing, the disk usage under apache-solr-3.5.0 is 153Mo.

Querying

Search for the GO terms having go:definition containing "cancer" a go:name containing "genome" but discard those having go:definition containing "metabolism".

 curl "http://localhost:8983/solr/select/?q=go_definition%3Acancer+go_name%3Agenome+-go_definition%3Ametabolism&version=2.2&start=0&rows=10&indent=on"

Same query, but return the result as a JSON structure:

 curl "http://localhost:8983/solr/select/?q=go_definition%3Acancer+go_name%3Agenome+-go_definition%3Ametabolism&version=2.2&start=0&rows=10&indent=on&wt=json"

That's it, Pierre