Showing posts with label xsd. Show all posts
Showing posts with label xsd. Show all posts

09 January 2013

A XML schema (xsd) for GeneOntology

The GeneOntology can be downloaded as a RDF/XML file from http://archive.geneontology.org/latest-termdb/go_daily-termdb.rdf-xml.gz.
Although it is a RDF file, the structure of the file remains the same. As a consequence, it is shipped with a DTD that describes the structure of the document ( http://www.geneontology.org/dtd/go.dtd ).
I've just written a XML schema (XSD) for this RDF file. This schema is available on github at:
https://github.com/lindenb/xsd-sandbox/tree/master/schemas/bio/go.

Validation with xmllint

The RDF file is successfully validated against my xsd schema:
$ curl "http://archive.geneontology.org/latest-termdb/go_daily-termdb.rdf-xml.gz" |\
 gunzip -c | grep -v "<!DOCTYPE " > go.xml

xmllint  --noout --schema go.xsd go.xml
go.xml validates
Note: I've ignored the elements defined in the DTD but absent in the RDF file.

Code Generation with XJC

XJC can be used to generate the java classes for this schema:
xjc go.xsd 
parsing a schema...
compiling a schema...
org/w3/_1999/_02/_22_rdf_syntax_ns_/ObjectFactory.java
org/w3/_1999/_02/_22_rdf_syntax_ns_/RDF.java
org/w3/_1999/_02/_22_rdf_syntax_ns_/package-info.java
org/geneontology/dtds/go/AbstractRelation.java
org/geneontology/dtds/go/Go.java
org/geneontology/dtds/go/IsA.java
org/geneontology/dtds/go/NegativelyRegulates.java
org/geneontology/dtds/go/ObjectFactory.java
org/geneontology/dtds/go/PartOf.java
org/geneontology/dtds/go/PositivelyRegulates.java
org/geneontology/dtds/go/Regulates.java
org/geneontology/dtds/go/package-info.java

Java Parsing

... and we can parse the terms of GO with java without writing a new parser and without any dependencies. For example, the following code parses the whole ontology and prints it to stdout as XML:
import java.io.InputStream;
import java.io.StringWriter;
import org.geneontology.dtds.go.*;
import org.w3._1999._02._22_rdf_syntax_ns_.*;
import javax.xml.namespace.QName;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBElement;
import javax.xml.bind.Unmarshaller;
import javax.xml.bind.Marshaller;
import javax.xml.transform.stream.StreamSource;

public class TestGo
    {
    public static void main(String[] args) throws Exception
        {
 JAXBContext jaxbCtxt=JAXBContext.newInstance("org.geneontology.dtds.go:org.w3._1999._02._22_rdf_syntax_ns_");
 Marshaller marshaller = jaxbCtxt.createMarshaller();
 Unmarshaller unmarshaller=jaxbCtxt.createUnmarshaller();
        marshaller.setProperty("jaxb.formatted.output",true);
        Object go=unmarshaller.unmarshal(new java.io.File("go.xml"));
        marshaller.marshal(go, System.out);
        }
    }
compile and run:
$javac TestGo.java \
  org/w3/_1999/_02/_22_rdf_syntax_ns_/ObjectFactory.java \
  org/geneontology/dtds/go/ObjectFactory.java

$ java TestGo  | head -n 100
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<go xmlns="http://www.geneontology.org/dtds/go.dtd#" xmlns:ns2="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
    <ns2:RDF>
        <term ns2:about="http://www.geneontology.org/go#GO:0000001">
            <accession>GO:0000001</accession>
            <name>mitochondrion inheritance</name>
            <synonym>mitochondrial inheritance</synonym>
            <definition>The distribution of mitochondria, including the mitochondrial genome, into daughter cells after mitosis or meiosis, mediated by interactions between mitochondria and the cytoskeleton.</definition>
            <is_a ns2:resource="http://www.geneontology.org/go#GO:0048308"/>
            <is_a ns2:resource="http://www.geneontology.org/go#GO:0048311"/>
        </term>
        <term ns2:about="http://www.geneontology.org/go#GO:0000002">
            <accession>GO:0000002</accession>
            <name>mitochondrial genome maintenance</name>
            <definition>The maintenance of the structure and integrity of the mitochondrial genome; includes replication and segregation of the mitochondrial chromosome.</definition>
            <is_a ns2:resource="http://www.geneontology.org/go#GO:0007005"/>
            <dbxref ns2:parseType="Resource">
                <database_symbol>InterPro</database_symbol>
                <reference>IPR009446</reference>

That's it,

Pierre

PS: many thanks to @bdoughan for his help on SO.

18 September 2012

Describing protein-protein interactions in XML: customizing the xsd-schema with JAXB, my notebook.

Say, you want to describe a network of protein-protein interaction using a XML format. Your XML schema will contain a set of

  • Articles/References
  • Proteins
  • Interactions
Here is a simple XSD schema for this model:
<?xml version="1.0" encoding="UTF-8"?>
<schema xmlns="http://www.w3.org/2001/XMLSchema" xmlns:tns="http://www.example.org/" targetNamespace="http://www.example.org/" elementFormDefault="qualified">
  <complexType name="article">
    <sequence>
      <element name="title" type="string"/>
      <element name="year" type="gYear"/>
    </sequence>
    <attribute name="pmid" type="ID" use="required"/>
  </complexType>
  <complexType name="protein">
    <sequence>
      <element name="acn" type="ID"/>
      <element name="description" type="string"/>
    </sequence>
  </complexType>
  <complexType name="interaction">
    <sequence>
      <element name="pmids" type="int" minOccurs="1" maxOccurs="unbounded"/>
      <element name="proteins" type="IDREF" minOccurs="1" maxOccurs="unbounded"/>
    </sequence>
  </complexType>
  <complexType name="interactome">
    <sequence>
      <element name="article" type="tns:article" minOccurs="0" maxOccurs="unbounded"/>
      <element name="protein" type="tns:protein" minOccurs="0" maxOccurs="unbounded"/>
      <element name="interaction" type="tns:interaction" minOccurs="0" maxOccurs="unbounded"/>
    </sequence>
  </complexType>
  <element name="interactome" type="tns:interactome"/>
</schema>
Here, the attibutes 'type="ID"' and 'type="IDREF"' are used to link the entities (One protein can be part of several interactions,....).
One can generate the java classes for those types using: ${JAVA_HOME}/bin/xjc:
$ xjc  interactome.xsd
parsing a schema...
compiling a schema...
org/example/Article.java
org/example/Interaction.java
org/example/Interactome.java
org/example/ObjectFactory.java
org/example/Protein.java
org/example/package-info.java
Problem: xjc doesn't know the exact nature of the links created between ID and IDREF. What kind of object should return the method 'getProteins' of the class 'Interaction' ? In consequence, xjc generates the following code:

$ more org/example/Interaction.java

    (...)
    protected List<JAXBElement<Object>> proteins;
    (...)
    public List<JAXBElement<Object>> getProteins()

We can tell xjc about those link by creating a binding file (JXB). In the following file, we tell XJC that the entities linked by 'proteins' should be some instances of 'Protein':
<?xml version="1.0" encoding="UTF-8"?>
<jxb:bindings xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc" xmlns:jxb="http://java.sun.com/xml/ns/jaxb" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" jxb:version="2.1">
  <jxb:bindings schemaLocation="interactome.xsd">
    <jxb:bindings node="/xs:schema/xs:complexType[@name=' interaction']/xs:sequence">
      <jxb:bindings node="xs:element[@name=' proteins']">
        <jxb:property>
          <jxb:baseType name="Protein"/>
        </jxb:property>
      </jxb:bindings>
    </jxb:bindings>
  </jxb:bindings>
</jxb:bindings>

Invoking XJC with the bindings:
$ xjc -b interactome.jxb  interactome.xsd
parsing a schema...
compiling a schema...
org/example/Article.java
org/example/Interaction.java
org/example/Interactome.java
org/example/ObjectFactory.java
org/example/Protein.java
org/example/package-info.java

The generated class 'Interaction.java' now contains the correct java type:


$ more org/example/Interaction.java

   (...)
    protected List<Protein> proteins;
    (...)
    public List<Protein> getProteins() {
     (....)

That's it,
Pierre

05 September 2012

Customizing the java classes for the NCBI generated by XJC

Reminder: XJC is the Java XML Binding Compiler. It automates the mapping between XML documents and Java objects:

XSD (aka XML-schema)
+
XJC
=
JAVA Classes

The code generated by XJC allows to :
  • Unmarshal XML content into a Java representation
  • Access and update the Java representation
  • Marshal the Java representation of the XML content into XML content

For example, the following XML-Schema (tinyseq.xsd) describes a TinySeq-XML document returned by the NCBI.

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:annotation>
    <xs:documentation> XML schema for NCBI tinyseq format</xs:documentation>
  </xs:annotation>
  <xs:complexType name="TSeqSet_t">
    <xs:annotation>
      <xs:documentation>Set of sequences</xs:documentation>
    </xs:annotation>
    <xs:sequence>
      <xs:element ref="TSeq" maxOccurs="unbounded"/>
    </xs:sequence>
  </xs:complexType>
  <xs:complexType name="TSeq_t">
    <xs:annotation>
      <xs:documentation>A Tiny Sequence</xs:documentation>
    </xs:annotation>
    <xs:sequence>
      <xs:element name="TSeq_seqtype">
        <xs:complexType>
          <xs:attribute name="value">
            <xs:simpleType>
              <xs:restriction base="xs:string">
                <xs:enumeration value="nucleotide"/>
                <xs:enumeration value="protein"/>
              </xs:restriction>
            </xs:simpleType>
          </xs:attribute>
        </xs:complexType>
      </xs:element>
      <xs:element name="TSeq_gi" type="xs:long"/>
      <xs:element name="TSeq_accver" type="xs:string"/>
      <xs:element name="TSeq_sid" type="xs:string"/>
      <xs:element name="TSeq_taxid" type="xs:long"/>
      <xs:element name="TSeq_orgname" type="xs:string"/>
      <xs:element name="TSeq_defline" type="xs:string"/>
      <xs:element name="TSeq_length" type="xs:nonNegativeInteger"/>
      <xs:element name="TSeq_sequence" type="xs:string"/>
    </xs:sequence>
  </xs:complexType>
  <xs:element name="TSeqSet" type="TSeqSet_t"/>
  <xs:element name="TSeq" type="TSeq_t"/>
</xs:schema>

This xml-schema can be compiled with XJC:

${JAVA_HOME}/bin/xjc -d . -p generated tinyseq.xsd
parsing a schema...
compiling a schema...
generated/ObjectFactory.java
generated/TSeqSetT.java
generated/TSeqT.java

$ more generated/TSeqT.java

package generated;
(...)
@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "TSeq_t", propOrder = {  "tSeqSeqtype",  "tSeqGi",  "tSeqAccver","tSeqSid",(...),"tSeqSequence"})
public class TSeqT {
    @XmlElement(name = "TSeq_seqtype", required = true)
    protected TSeqT.TSeqSeqtype tSeqSeqtype;
    @XmlElement(name = "TSeq_gi")
    protected long tSeqGi;
    @XmlElement(name = "TSeq_accver", required = true)
    protected String tSeqAccver;
    @XmlElement(name = "TSeq_sid", required = true)
    protected String tSeqSid;
    @XmlElement(name = "TSeq_taxid")
    protected long tSeqTaxid;
    @XmlElement(name = "TSeq_orgname", required = true)
    protected String tSeqOrgname;
    @XmlElement(name = "TSeq_defline", required = true)
    protected String tSeqDefline;
    @XmlElement(name = "TSeq_length", required = true)
    @XmlSchemaType(name = "nonNegativeInteger")
    protected BigInteger tSeqLength;
    @XmlElement(name = "TSeq_sequence", required = true)
    protected String tSeqSequence;
(...)
    }

But XJC doesn't know how to generate some classical java functions like 'hashCode', 'equals' or 'toString' or to add some custom methods to your classes.

Hopefully the standard distribution of XJC comes with a plugin named -Xinject-code whch injects some custom code in the classes generated by XJC.

XSD (aka XML-schema)
+
java xml binding (jxb)
+
XJC
=
Customized JAVA Classes

For example, if we want to add a toString method to the class TSeqT, we're going to write the following "java xml binding file" (jxb) which alters the initial xml schema:

<?xml version="1.0" encoding="UTF-8"?>
<jxb:bindings 
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc"
xmlns:jxb="http://java.sun.com/xml/ns/jaxb"
xmlns:ci="http://jaxb.dev.java.net/plugin/code-injector"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
jxb:extensionBindingPrefixes="ci "
jxb:version="2.1"

>

<jxb:bindings schemaLocation="tinyseq.xsd">
        <!-- here we use an XPATH expression to tell xjc about which part
 of the XML schema we want to change -->
 <jxb:bindings node="/xs:schema/xs:complexType[@name='TSeq_t']">
  <ci:code>

 /** toString : returns the gi and the defline  */
 public String toString()
  {
  return "gi:"+getTSeqGi()+"|"+getTSeqDefline();
  }

</ci:code>
 </jxb:bindings>
</jxb:bindings>

</jxb:bindings>

Below, I wrote a larger JXB file 'tinyseq.jxb' which injects the following methods:
  • 'equals' method for TSeq
  • 'hashCode' method for TSeq
  • 'toString' method for TSeq
  • 'printAsFasta' method for TSeq
  • 'getTSeqSetbyId' method for TSeqSet. A static function fetching a TinySeq sequence from the NCBI for a given 'gi'
  • a 'main' method for TSeqSet. It loops over a list of 'gi's, fetches the sequences (using NCBI-EFetch) and prints the sequences as FASTA
<?xml version="1.0" encoding="UTF-8"?>
<jxb:bindings 
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc"
xmlns:jxb="http://java.sun.com/xml/ns/jaxb"
xmlns:ci="http://jaxb.dev.java.net/plugin/code-injector"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
jxb:extensionBindingPrefixes="ci "
jxb:version="2.1"

>

<jxb:bindings schemaLocation="tinyseq.xsd">
 <jxb:bindings node="/xs:schema/xs:complexType[@name='TSeq_t']">
  <ci:code>
 /* print this sequence as fasta */
 public void printAsFasta(java.io.PrintStream out)
  {
  String s=getTSeqSequence();
  out.print(">gi:"+getTSeqGi()+"|"+ getTSeqAccver() +"|"+getTSeqDefline());
  for(int i=0;i &lt; s.length();++i)
   {
   if(i%60==0) out.println();
   out.print(s.charAt(i));
   }
  out.println();
  }

 /** equals: two  TSeq are equal if they have the same gi */
 @Override
 public boolean equals(Object o)
  {
  if(o==this) return true;
  if(o==null || o.getClass()!=this.getClass()) return false;
  return this.getTSeqGi()==TSeqT.class.cast(o).getTSeqGi();
  }

 /** hashCode : use gi */
 @Override
 public int hashCode()
  {
  return  (int)(this.getTSeqGi()^(this.getTSeqGi()>>>32));
  }

 /** toString : returns the gi and the defline  */
 public String toString()
  {
  return "gi:"+getTSeqGi()+"|"+getTSeqDefline();
  }

</ci:code>
 </jxb:bindings>




 <jxb:bindings node="/xs:schema/xs:complexType[@name='TSeqSet_t']">
  <ci:code>
 /** get TSeqSetT from a given gi */
 public static TSeqSetT getTSeqSetbyId(long gi)
  throws javax.xml.bind.JAXBException, javax.xml.bind.UnmarshalException , java.io.IOException
  {
  /** find the JAXB context in the defined path */
  javax.xml.bind.JAXBContext jc = javax.xml.bind.JAXBContext.newInstance(TSeqSetT.class,TSeqT.class);
  javax.xml.bind.Unmarshaller u = jc.createUnmarshaller();
  String uri="http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=nucleotide&amp;rettype=fasta&amp;retmode=xml&amp;id="+gi;
  /** read the sequence */
  return u.unmarshal(new javax.xml.transform.stream.StreamSource(uri),TSeqSetT.class).getValue();
  }

 /** main: takes a list of gi and prints the sequences as fasta */
 public static void main(String args[]) throws Exception
  {
  for(int optind=0;optind &lt; args.length;++optind)
   {
   TSeqSetT tss=TSeqSetT.getTSeqSetbyId(Long.parseLong(args[optind]));
   for(generated.TSeqT seq:tss.getTSeq()) seq.printAsFasta(System.out);
   }
  }

</ci:code>

</jxb:bindings>

</jxb:bindings>

</jxb:bindings>

Compile the schema, compile the java classes and execute

$ xjc -target 2.1 -verbose -Xinject-code -extension -d . -p generated -b tinyseq.jxb tinyseq.xsd
parsing a schema...
compiling a schema...
[INFO] generating code
unknown location

generated/ObjectFactory.java
generated/TSeqSetT.java
generated/TSeqT.java

$ javac generated/*.java
$ java generated.TSeqSetT 25 26 27
>gi:25|X53813.1|Blue Whale heavy satellite DNA
TAGTTATTCAACCTATCCCACTCTCTAGATACCCCTTAGCACGTAAAGGAATATTATTTG
GGGGTCCAGCCATGGAGAATAGTTTAGACACTAGGATGAGATAAGGAACACACCCATTCT
AAAGAAATCACATTAGGATTCTCTTTTTAAGCTGTTCCTTAAAACACTAGAGTCTTAGAA
ATCTATTGGAGGCAGAAGCAGTCAAGGGTAGCCTAGGGTTAGGGTTAGGCTTAGGGTTAG
GGTTAGGGTACGGCTTAGGGTACTGTTTCGGGGAGGGGTTCAGGTACGGCGTAGGGTATG
GGTTAGGGTTAGGGTTAGGGTTAGTGTTAGGGTTAGGGCTCGGTTTAGGGTACGGGTTAG
GATTAGGGTACGTGTTAGGGTTAGGGTAGGGCTTAGGGTTAGGGTACGTGTTAGGGTTAG
GG
>gi:26|X53814.1|Blue Whale heavy satellite DNA
TAGTTATTAAACCTATCCCACTCTCTAGATACACCTTAGCACGTAAAGGAATATTATTTG
GGGGTCCAGACATGGAGAAGAGTTTAGACACTAGGATAAGATAAGGAACACACCCATTCT
AAAGAAATCACATTAGGATTCTCTTTTTAAGCTGTTCCTTAAAACTCTAGTGCTTAGGAA
ATCTATTGGAGGCAGAAGCAGTCAAGGGTAGCCTAGGGTTAGGGTTAGGCTTATGGTTAG
GGCTAGGGTACGGCTTAGGGTACGGATTCGGGGAGGGGTTCGGGTACGGCGTAGGGTATG
GGTTAGGGTTAGCGTTAGTGTTAGGGTTAGGGCTCGGTTTAGGGTACGGGTTAGGATTAG
GGTACGTGTTAGGGTTAGGGTAGGGGTTAGGGTTAGGGTACGCGTTAGGGTTAGGG
>gi:27|Z18633.1|B.physalus gene for large subunit rRNA
AACCAGTATTAGAGCACTGCCTGCCCGGTGACTAATCGTTAAACGGCCGCGGTATCCTGA
CCGTGCAAAGGTAGCATAATCACTTGTTCTCTAATTAGGGACTTGTATGAATGGCCACAC
GAGGGTTTTACTGTCTCTTACTTTTAATCAGTGAAATTGACCTCTCCGTGAAGAGGCGGA
GATAACAAAATAAGACGAGAAGACCCTATGGAGCTTCAATTAATCAACCCAAAAACCATA
ACCTTAAACCACCAAGGGATAACAAAACCTTATATGGGCTGACAATTTCGGTTGGGGTGA
CCTCGGAGTACAAAAAACCCTCCGAGTGATTAAAACTTAGGCCCACTAGCCAAAGTACAA
TATCACTTATTGATCCAATCCTTTGATCAACGGAACAAGTTACCCTAGGGATAACAGCGC
AATCCTATTCTAGAGTCCATATCGACAATAGGGTTTACGACCTCGATGTTGGATCAGGAC
ATCCTAATGGTGCAGCTGCTATTAAGGGTTCGTTTGTT
That's it,


Pierre

07 December 2009

Playing with SOAP. Implementing a WebService for the LocusTree Server

Image via wikipediaIn a previous post I've described the LocusTree server and showed how the wsimport command can be used to generate the java code that will query a Web-Service. Today, I've implemented a few web services in the LocusTree server but I wrote the entire code generating the SOAP messages rather than using the Java API for Web Services (JAXWS-API) because 1) I wanted to learn about the SOAP internals 2) I wanted to return a big volume of data to the client by writing a stream of data rather than building a xml response and then echoing the xml tree (DOM).
Ok. In this example I'm going to describe a WebService returning a list of chromosomes for a given organism-id:

The WSDL file.

The signature of our function is something like: getChromosomesByOrganismId(int orgId). In the WSDL file (the file describing our web services), the operation is called getChromosomes . The input for this function will be a tns:getChromosomes and the object returned by this function is a tns:GetChromosomesResponse. The prefix 'tns' is a reference to a xml schema that is will to defined later.
<portType name="LocusTree">
<operation name="getChromosomes">
<input message="tns:getChromosomes"/>
<output message="tns:GetChromosomesResponse"/>
</operation>
</portType>

We now define those two messages (input and output parameters) for this web service. The input value (the organism-id) is defined in an external xml schema as an element named 'tns:getChromosomes'. The ouput value (a list of chromosomes) is defined in an external xml schema as an element named 'tns:Chromosomes'.
<message name="getChromosomes">
<part name="parameters" element="tns:getChromosomes"/>
</message>
<message name="GetChromosomesResponse">
<part name="parameters" element="tns:Chromosomes"/>
</message>

But where can we find this external schema ? It is referenced in the WSDL file under the <types> element. The following 'types' says that the schema describing our objects is available at 'http://localhost:8080//locustree/static/ws/schema.xsd'
<types>
<xsd:schema>
<xsd:import namespace="http://webservices.cephb.fr/locustree/" schemaLocation="http://localhost:8080//locustree/static/ws/schema.xsd"/>
</xsd:schema>
</types>
The Http protocol will be used to send and receive the SOAP messages, so we have to bind our method 'getChromosomesByOrganismsId' to this protocol.
<soap:binding transport="http://schemas.xmlsoap.org/soap/http"
style="document"/>
<operation name="getChromosomes">
<soap:operation/>
<input>
<soap:body use="literal"/>
</input>
<output>
<soap:body use="literal"/>
</output>
</operation>
</binding>
Finally, we tell the client where the server is located
<service name="LocusTreeService">
<port name="LocusTreeSePort" binding="tns:LocusTreePortBinding">
<soap:address location="http://localhost:8080//locustree/locustree/soap"/>
</port>
</service>

The XSD Schema

This XML schema describes the structures that will be used and returned by the server.We need to describe what is...

A Chromosome

A Chromosome is a structure holding an ID, a name, a length, an organism-id etc...
<xs:complexType name="Chromosome">
<xs:annotation>
<xs:documentation>A Chromosome</xs:documentation>
</xs:annotation>
<xs:sequence>
<xs:element name="id" type="xs:int" nillable="false" />
<xs:element name="organismId" type="xs:int" nillable="false" />
<xs:element name="name" type="xs:string" nillable="false"/>
<xs:element name="length" type="xs:int" nillable="false"/>
<xs:element name="metadata" type="xs:string"/>
</xs:sequence>
</xs:complexType>

A List of Chromosomes

.. is just a sequence of Chromosomes
<xs:complexType name="Chromosomes">
<xs:annotation>
<xs:documentation>Set of Chromosomes</xs:documentation>
</xs:annotation>
<xs:sequence>
<xs:element ref="tns:Chromosome" maxOccurs="unbounded"/>
</xs:sequence>
</xs:complexType>

GetChromosome

This is the structure that is used as a parameter for our web service. It just holds an organism-id.
<xs:complexType name="getChromosomes">
<xs:annotation>
<xs:documentation>return the chromosomes for a given organism </xs:documentation>
</xs:annotation>
<xs:sequence>
<xs:element name="organismId" type="xs:int" nillable="false">
<xs:annotation>
<xs:documentation>The Organism Id</xs:documentation>
</xs:annotation>
</xs:element>
</xs:sequence>
</xs:complexType>



All in one, here is the WSDL file:
<definitions
xmlns="http://schemas.xmlsoap.org/wsdl/"
xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
xmlns:tns="http://webservices.cephb.fr/locustree/"
targetNamespace="http://webservices.cephb.fr/locustree/"
name="LocusTreeWebServices">

<types>
<xsd:schema>
<xsd:import namespace="http://webservices.cephb.fr/locustree/" schemaLocation="http://localhost:8080//locustree/static/ws/schema.xsd"/>
</xsd:schema>
</types>
<message name="getChromosomes">
<part name="parameters" element="tns:getChromosomes"/>
</message>
<message name="GetChromosomesResponse">
<part name="parameters" element="tns:Chromosomes"/>
</message>
<portType name="LocusTree">
<operation name="getChromosomes">
<input message="tns:getChromosomes"/>
<output message="tns:GetChromosomesResponse"/>
</operation>
</portType>
<binding name="LocusTreePortBinding" type="tns:LocusTree">
<soap:binding transport="http://schemas.xmlsoap.org/soap/http" style="document"/>
<operation name="getChromosomes">
<soap:operation/>
<input>
<soap:body use="literal"/>
</input>
<output>
<soap:body use="literal"/>
</output>
</operation>
</binding>
<service name="LocusTreeService">
<port name="LocusTreeSePort" binding="tns:LocusTreePortBinding">
<soap:address location="http://localhost:8080//locustree/locustree/soap"/>
</port>
</service>
</definitions>

...and the XSD/Schema file:
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:tns="http://webservices.cephb.fr/locustree/"
targetNamespace="http://webservices.cephb.fr/locustree/"
elementFormDefault="qualified">


<xs:annotation>
<xs:documentation>XML schema for LocusTreeWebServices</xs:documentation>
</xs:annotation>

<xs:element name="Chromosomes" type="tns:Chromosomes"/>
<xs:element name="Chromosome" type="tns:Chromosome"/>
<xs:element name="getChromosomes" type="tns:getChromosomes"/>

<xs:complexType name="getChromosomes">
<xs:annotation>
<xs:documentation>return the chromosomes for a given organism </xs:documentation>
</xs:annotation>
<xs:sequence>
<xs:element name="organismId" type="xs:int" nillable="false">
<xs:annotation>
<xs:documentation>The Organism Id</xs:documentation>
</xs:annotation>
</xs:element>
</xs:sequence>
</xs:complexType>

<xs:complexType name="Chromosomes">
<xs:annotation>
<xs:documentation>Set of Organisms</xs:documentation>
</xs:annotation>
<xs:sequence>
<xs:element ref="tns:Chromosome" maxOccurs="unbounded"/>
</xs:sequence>
</xs:complexType>

<xs:complexType name="Chromosome">
<xs:annotation>
<xs:documentation>A Chromosome</xs:documentation>
</xs:annotation>
<xs:sequence>
<xs:element name="id" type="xs:int" nillable="false"/>
<xs:element name="organismId" type="xs:int" nillable="false"/>
<xs:element name="name" type="xs:string" nillable="false"/>
<xs:element name="length" type="xs:int" nillable="false"/>
<xs:element name="metadata" type="xs:string"/>
</xs:sequence>
</xs:complexType>


</xs:schema>

Generating the client

The stubs on the client side are generated using the ${JAVA_HOME}/bin/wsimport command:
> wsimport -keep http://localhost:8080/locustree/locustree/soap?wsdl
parsing WSDL...
generating code...
compiling code...
> find fr
fr/cephb/webservices/locustree/Chromosomes.java
fr/cephb/webservices/locustree/ObjectFactory.java
fr/cephb/webservices/locustree/LocusTreeService.java
fr/cephb/webservices/locustree/LocusTree.java
fr/cephb/webservices/locustree/GetChromosomes.java
fr/cephb/webservices/locustree/Chromosome.java
(...)
> more fr/cephb/webservices/locustree/LocusTree.java
package fr.cephb.webservices.locustree;
(...)
public interface LocusTree
{
(...)
public List<Chromosome> getChromosomes(int organismId);
}
Ok, the function was successfully generated. Let's test it with a tiny java program:

file Test.java
import fr.cephb.webservices.locustree.*;

public class Test
{
public static void main(String args[])
{
LocusTreeService service=new LocusTreeService();
LocusTree locustree=service.getLocusTreeSePort();
final int organismId=36;
for(Chromosome chrom:locustree.getChromosomes(organismId))
{
System.out.println(
chrom.getId()+"\t"+
chrom.getName()+"\t"+
chrom.getOrganismId()+"\t"+
chrom.getLength()
);
}

}
}

Compiling and running:
javac -cp . Test.java
java -cp . Test
1 chr1 36 247249719
2 chr2 36 242951149
3 chr3 36 199501827
4 chr4 36 191273063
5 chr5 36 180857866
6 chr6 36 170899992
7 chr7 36 158821424
8 chr8 36 146274826
9 chr9 36 140273252
10 chr10 36 135374737
11 chr11 36 134452384
12 chr12 36 132349534
13 chr13 36 114142980
14 chr14 36 106368585
15 chr15 36 100338915
(...)

SOAP internals


Here is the XML/SOAP query for getChromosomes sent to the sever via a POST query.
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
<S:Body>
<getChromosomes xmlns='http://webservices.cephb.fr/locustree/'>
<organismId>36</organismId>
</getChromosomes>
</S:Body>
</S:Envelope>
This can be checked using curl:
curl \
-X POST\
-H "Content-Type: text/xml" \
-d '<?xml version="1.0" ?>;<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"><S:Body><getChromosomes xmlns="http://webservices.cephb.fr/locustree/"><organismId>36</organismId></getChromosomes></S:Body></S:Envelope>' \
'http://localhost:8080/locustree/locustree/soap'
And here is the (my) response from the server:
<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" >
<Body>
<ceph:Chromosomes>
<ceph:Chromosome>
<ceph:id>1</ceph:id>
<ceph:organismId>36</ceph:organismId>
<ceph:name>chr1</ceph:name>
<ceph:length>247249719</ceph:length>
<ceph:metadata>{'type':'autosomal','size':247249719}</ceph:metadata>
</ceph:Chromosome>
<ceph:Chromosome>
<ceph:id>2</ceph:id>
<ceph:organismId>36</ceph:organismId>
<ceph:name>chr2</ceph:name>
<ceph:length>242951149</ceph:length>
<ceph:metadata>{'type':'autosomal','size':242951149}</ceph:metadata>
</ceph:Chromosome>
<ceph:Chromosome>
<ceph:id>3</ceph:id>
<ceph:organismId>36</ceph:organismId>
<ceph:name>chr3</ceph:name>
<ceph:length>199501827</ceph:length>
<ceph:metadata>{'type':'autosomal','size':199501827}</ceph:metadata>
</ceph:Chromosome>
<ceph:Chromosome>
<ceph:id>4</ceph:id>
<ceph:organismId>36</ceph:organismId>
<ceph:name>chr4</ceph:name>
<ceph:length>191273063</ceph:length>
<ceph:metadata>{'type':'autosomal','size':191273063}</ceph:metadata>
</ceph:Chromosome>
(...)
</ceph:Chromosomes>
</Body>
</Envelope>

On the server/servlet side I've decoded the SOAP query using javax.xml.soap.MessageFactory;. It looks like that
(...)
MimeHeaders headers=new MimeHeaders();
Enumeration<?> e=req.getHeaderNames();
(... copy the http headers to 'headers' ...);
SOAPMessage message=getMessageFactory().createMessage(headers,req.getInputStream());
SOAPBody body=message.getSOAPBody();
Iterator<?> iter=body.getChildElements();
while(iter.hasNext())
{
SOAPElement child =SOAPElement.class.cast(iter.next());
Name name= child.getElementName();
if(!name.getURI().equals(child.getNamespaceURI())) continue;
if(name.getLocalName().equals("getChromosomes"))
{
processGetChromosomes(w,message,child,req,res);
return;
}
}
And I'm streaming the response using the XML Streaming API (StaX).
(...)
w.writeStartElement(pfx, "Chromosomes", getTargetNamespace());
w.writeAttribute(XMLConstants.XMLNS_ATTRIBUTE,XMLConstants.XML_NS_URI,pfx,getTargetNamespace());
for(ChromInfo ci:model.getChromsomesByOrganismId(getTransaction(), organismId))
{
w.writeStartElement(pfx, "Chromosome", getTargetNamespace());
w.writeStartElement(pfx, "id", getTargetNamespace());
w.writeCharacters(String.valueOf(ci.getId()));
w.writeEndElement();
w.writeStartElement(pfx, "organismId", getTargetNamespace());
w.writeCharacters(String.valueOf(ci.getOrganismId()));
w.writeEndElement();
(...)
w.writeEndElement();
}
w.writeEndElement();(...)

And as a final note, I'll cite this tweet I received today from Paul Joseph Davis :-)


@yokofakun Everytime someone uses SOAP, an angel cries.
Mon Dec 07 16:50:41



That's it !
Pierre

03 September 2009

Generating a C Pull Parser for dbSNP with XSLT



I've used the XSD schema describing dbSNP to generate a C "Pull parser" reading the content of the dbSNP XML files. To transform the schema into a C code I wrote the following XSLT stylesheet:. This stylesheet was specifically developed for dbSNP so it might not handle a more complicated schema (for example a schema that would use <xsd:elementType> ). Basically the C code generated is a scaffold for a Pull Parser using the libxml2 library. For example, here is a simplified snippet of code handling the tag <Assembly/>
/** A collection of genome sequence records (curated gene regions (NG's),
contigs (NWNT's) and chromosomes (NC/AC's) produced by a genome
sequence project. Structure is populated from ContigInfo tables. */

static int processAssembly(StatePtr state)
{
int returnValue=EXIT_SUCCESS;
int success;
int nodeType;
const int isEmptyElement= xmlTextReaderIsEmptyElement(state -> reader);

/** Name of the group(s) or organization(s) that generated the assembly */
xmlChar* assemblySourceAttr=NULL;

//(...) declare other attributes

assemblySourceAttr= xmlTextReaderGetAttribute(
state->reader,
BAD_CAST "assemblySource"
);

//(...) other attributes

if(!isEmptyElement)
{
success = xmlTextReaderRead( state -> reader );
if(!success)
{
fprintf( state->error,"In Assembly I/O Error. xmlTextReaderRead returned \n");
returnValue = EXIT_FAILURE;
goto cleanup;
}
nodeType = xmlTextReaderNodeType( state -> reader );


/* process childNode <Component/> */

while(nodeType == XML_READER_TYPE_ELEMENT)
{
if(xmlStrcmp(
xmlTextReaderConstName(state -> reader),
BAD_CAST "Component"
)!=0)
{
break;
}

if(processComponent(state)!=EXIT_SUCCESS)
{
returnValue = EXIT_FAILURE;
goto cleanup;
}

/* read next event */
success= xmlTextReaderRead(state->reader);
if(!success)
{
returnValue = EXIT_FAILURE;
fprintf( state->error,"In Assembly/Component I/O Error.\n");
goto cleanup;
}
nodeType=xmlTextReaderNodeType(state->reader);
}

/* process childNode <SnpStat/> */
(...)

}//end of if(!isEmptyElement)

cleanup:

//free attributes
if(assemblySourceAttr!=NULL)
{
xmlFree(assemblySourceAttr);
}
//(...) other attributes
return returnValue;
}
Using this prototype I was able to quickly write a fast parser , for example echoing a JSON description of the SNPs of the Human Mitochondrial Genome (time: 0.11user 0.01system 0:00.19elapsed 66%CPU).
[
{
"rsId":8896,
"seq5":"GGTGTTGGTTCTCTTAATCTTTAACTTAAAAGGTTAATGCTAAGTTAGCTTTACAGTGGGCTCTAGAGGGGG
TAGAGGGGGTG",
"observed":"C/T",
"seq3":"TATAGGGTAAATACGGGCCCTATTTCAAAGATTTTTAGGGGAATTAATTCTAGGACGATGGGCATGAAACTGTGGTTTGCTCCACAGATTTCAGAGCATT"
}
,
{
"rsId":8936,
"seq5":"ACTACGGCGGACTAATCTTCAACTCCTACATACTTCCCCCATTATTCCTAGAACCAGGCGACCTGCGACTCCTTGACGTTGACAATCGAGTAGTACTCCCGATTGAAGCCCCCATTCGTATAATAATTACATCACAAGACGTCTTGCACTCATGAGCTGTCCCCACATTAGGCTTAAAAACAGATGCAATTCCCGGACGT",
"observed":"A/C/T",
"seq3":"TAAACCAAACCACTTTCACCGCTACACGACCGGGGGTATACTACGGTCAATGCTCTGAAATCTGTGGAGCAAACCACAGTTTCATGCCCATCGTCCTAGAATTAATTCCCCTAAAAATCTTTGAAATAGGGCCCGTATTTACCCTATAGCACCCCCTCTACCCCCTCTAGAGCCCACTGTAAAGCTAACTTAGCATTAAC"
}
(...)
{
"rsId":72619366,
"seq5":"TGCTTACAAGCAAGTACAGCAATCAACCTTCAACTATCACACATCAACTGCAACTCCAAAGCCACCCCTCACCCACTAGGATACCAACAAACCTACCCAC",
"observed":"C/T",
"seq3":"CTTAACAGTACATAGTACATAAAGCCATTTACCGTACATAGCACATTACAGTCAAATCCCTTCTCGTCCCCATGGATGACCCCCCTCAGATAGGGGTCCC"
}
]



That's it
Pierre