Showing posts with label sql. Show all posts
Showing posts with label sql. Show all posts

21 May 2016

Playing with the @ORCID_Org / @ncbi_pubmed graph. My notebook.

"ORCID provides a persistent digital identifier that distinguishes you from every other researcher and, through integration in key research workflows such as manuscript and grant submission, supports automated linkages between you and your professional activities ensuring that your work is recognized. "
I've recently discovered that pubmed now integrates ORCID identfiers.

And there are several minor problems, I found some articles where the ORCID id is malformed or where different people use the same ORCID-ID:







You can download the papers containing some orcid Identifiers using the entrez query http://www.ncbi.nlm.nih.gov/pubmed/?term=orcid[AUID].
I've used one of my tools pubmeddump to download the articles asXML and I wrote PubmedOrcidGraph to extract the author's orcid.
<?xml version="1.0" encoding="UTF-8"?>
<PubmedArticleSet>
  <!--Generated with PubmedOrcidGraph https://github.com/lindenb/jvarkit/wiki/PubmedOrcidGraph - Pierre Lindenbaum.-->
  <PubmedArticle pmid="27197243" doi="10.1101/gr.199760.115">
    <year>2016</year>
    <journal>Genome Res.</journal>
    <title>Improved definition of the mouse transcriptome via targeted RNA sequencing.</title>
    <Author orcid="0000-0002-4078-7413">
      <foreName>Giovanni</foreName>
      <lastName>Bussotti</lastName>
      <initials>G</initials>
      <affiliation>EMBL, European Bioinformatics Institute, Cambridge, CB10 1SD, United Kingdom;</affiliation>
    </Author>
    <Author orcid="0000-0002-4449-1863">
      <foreName>Tommaso</foreName>
      <lastName>Leonardi</lastName>
      <initials>T</initials>
      <affiliation>EMBL, European Bioinformatics Institute, Cambridge, CB10 1SD, United Kingdom;</affiliation>
    </Author>
    <Author orcid="0000-0002-6090-3100">
      <foreName>Anton J</foreName>
      <lastName>Enright</lastName>
      <initials>AJ</initials>
      <affiliation>EMBL, European Bioinformatics Institute, Cambridge, CB10 1SD, United Kingdom;</affiliation>
    </Author>
  </PubmedArticle>
  <PubmedArticle pmid="27197225" doi="10.1101/gr.204479.116">
    <year>2016</year>
    <journal>Genome Res.</journal>
(...)
Now, I want to insert those data into a sqlite3 database. I use the XSLT stylesheet below to convert the XML into some SQL statement.
<?xml version="1.0"?>
<xsl:stylesheet
 xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
 version="1.0"
    xmlns:xalan="http://xml.apache.org/xalan"
    xmlns:str="xalan://com.github.lindenb.xslt.strings.Strings"
    exclude-result-prefixes="xalan str"
 >
<xsl:output method="text"/>
<xsl:variable name="q">'</xsl:variable>

<xsl:template match="/">
create table author(orcid text unique,name text,affiliation text);
create table collab(orcid1 text,orcid2 text,unique(orcid1,orcid2));
begin transaction;
<xsl:apply-templates select="PubmedArticleSet/PubmedArticle"/>
commit;
</xsl:template>

<xsl:template match="PubmedArticle">
<xsl:for-each select="Author">
<xsl:variable name="o1" select="@orcid"/>insert or ignore into author(orcid,name,affiliation) values ('<xsl:value-of select="$o1"/>','<xsl:value-of select="translate(concat(lastName,' ',foreName),$q,' ')"/>','<xsl:value-of select="translate(affiliation,$q,' ')"/>');
<xsl:for-each select="following-sibling::Author">insert or ignore into collab(orcid1,orcid2) values(<xsl:variable name="o2" select="@orcid"/>
<xsl:choose>
 <xsl:when test="str:strcmp( $o1 , $o2) < 0">'<xsl:value-of select='$o1'/>','<xsl:value-of select='$o2'/>'</xsl:when>
 <xsl:otherwise>'<xsl:value-of select='$o2'/>','<xsl:value-of select='$o1'/>'</xsl:otherwise>
</xsl:choose>);
</xsl:for-each>
</xsl:for-each>
</xsl:template>
</xsl:stylesheet>

This stylesheet contains an extension 'strmcp' for the xslt processor xalan to compare two XML strings
This extension is just used to always be sure that the field "orcid1" in the table "collab" is always lower than "orcid2" to avoid duplicates pairs.
./src/xslt-sandbox/xalan/dist/xalan -XSL orcid2sqlite.xsl -IN orcid.xml

create table author(orcid text unique,name text,affiliation text);
create table collab(orcid1 text,orcid2 text,unique(orcid1,orcid2));
begin transaction;
insert or ignore into author(orcid,name,affiliation) values ('0000-0002-4078-7413','Bussotti Giovanni','EMBL, European Bioinformatics Institute, Cambridge, CB10 1SD, United Kingdom;');
insert or ignore into collab(orcid1,orcid2) values('0000-0002-4078-7413','0000-0002-4449-1863');
insert or ignore into collab(orcid1,orcid2) values('0000-0002-4078-7413','0000-0002-6090-3100');
insert or ignore into author(orcid,name,affiliation) values ('0000-0002-4449-1863','Leonardi Tommaso','EMBL, European Bioinformatics Institute, Cambridge, CB10 1SD, United Kingdom;');
insert or ignore into collab(orcid1,orcid2) values('0000-0002-4449-1863','0000-0002-6090-3100');
insert or ignore into author(orcid,name,affiliation) values ('0000-0002-6090-3100','Enright Anton J','EMBL, European Bioinformatics Institute, Cambridge, CB10 1SD, United Kingdom;');
(...)
and those sql statetements are loaded into sqlite3:
./src/xslt-sandbox/xalan/dist/xalan -XSL orcid2sqlite.xsl -IN orcid.xml |\
 sqlite3 orcid.sqlite

The next step is to produce a gexf+xml file to play with the orcid graph in gephi.
I use the following bash script to convert the sqlite3 database to gexf+xml.
DB=orcid.sqlite

cat << EOF
<?xml version="1.0" encoding="UTF-8"?>
<gexf xmlns="http://www.gexf.net/1.2draft" xmlns:viz="http://www.gexf.net/1.1draft/viz" version="1.2">
<meta>
<creator>Pierre Lindenbaum</creator>
<description>Orcid Graph</description>
</meta>
<graph defaultedgetype="undirected" mode="static">

<attributes class="node">
<attribute type="string" title="affiliation" id="0"/>
</attributes>
<nodes>
EOF

sqlite3 -separator ' ' -noheader  ${DB} 'select orcid,name,affiliation from author' |\
 sed  -e 's/&/&/g' -e "s/</\</g" -e "s/>/\>/g" -e "s/'/\'/g"  -e 's/"/\"/g' |\
 awk -F ' ' '{printf("<node id=\"%s\" label=\"%s\"><attvalue for=\"0\" value=\"%s\"/></node>\n",$1,$2,$3);}'

echo "</nodes><edges>"
sqlite3 -separator ' ' -noheader  ${DB} 'select orcid1,orcid2 from collab' |\
 awk -F ' ' '{printf("<edge source=\"%s\" target=\"%s\"/>\n",$1,$2);}'
echo "</edges></graph></gexf>"



The output is saved and then loaded into gephi.






That's it,

Pierre

01 May 2013

Inserting the result of a BLAST into a Database using XSLT.

Here is the XML output of a BLAST:

<?xml version="1.0"?>
<!DOCTYPE BlastOutput PUBLIC "-//NCBI//NCBI BlastOutput/EN" "NCBI_BlastOutput.dtd">
<BlastOutput>
  <BlastOutput_program>tblastn</BlastOutput_program>
  <BlastOutput_version>TBLASTN 2.2.27+</BlastOutput_version>
  <BlastOutput_reference>Stephen F. Altschul, Thomas L. Madden, Alejandro A. Sch&amp;auml;ffer, Jinghui Z
hang, Zheng Zhang, Webb Miller, and David J. Lipman (1997), &quot;Gapped BLAST and PSI-BLAST: a new generation 
of protein database search programs&quot;, Nucleic Acids Res. 25:3389-3402.</BlastOutput_reference>
  <BlastOutput_db>nr</BlastOutput_db>
  <BlastOutput_query-ID>52385</BlastOutput_query-ID>
  <BlastOutput_query-def>myseq</BlastOutput_query-def>
  <BlastOutput_query-len>30</BlastOutput_query-len>
  <BlastOutput_param>
    <Parameters>
      <Parameters_matrix>BLOSUM62</Parameters_matrix>
      <Parameters_expect>10</Parameters_expect>
      <Parameters_gap-open>11</Parameters_gap-open>
      <Parameters_gap-extend>1</Parameters_gap-extend>
      <Parameters_filter>L;</Parameters_filter>
    </Parameters>
  </BlastOutput_param>
<BlastOutput_iterations>
<Iteration>
  <Iteration_iter-num>1</Iteration_iter-num>
  <Iteration_query-ID>52385</Iteration_query-ID>
  <Iteration_query-def>myseq</Iteration_query-def>
  <Iteration_query-len>30</Iteration_query-len>
<Iteration_hits>
<Hit>
  <Hit_num>1</Hit_num>
  <Hit_id>gi|110624327|dbj|AK225891.1|</Hit_id>
  <Hit_def>Homo sapiens mRNA for zinc finger CCCH-type containing 7B variant, clone: FCC121C02</Hit_def>
  <Hit_accession>AK225891</Hit_accession>
  <Hit_len>1829</Hit_len>
  <Hit_hsps>
    <Hsp>
      <Hsp_num>1</Hsp_num>
      <Hsp_bit-score>62.3882</Hsp_bit-score>
      <Hsp_score>150</Hsp_score>
      <Hsp_evalue>4.01658e-10</Hsp_evalue>
      <Hsp_query-from>1</Hsp_query-from>
      <Hsp_query-to>30</Hsp_query-to>
      <Hsp_hit-from>250</Hsp_hit-from>
      <Hsp_hit-to>339</Hsp_hit-to>
      <Hsp_query-frame>0</Hsp_query-frame>
      <Hsp_hit-frame>1</Hsp_hit-frame>
      <Hsp_identity>30</Hsp_identity>
      <Hsp_positive>30</Hsp_positive>
      <Hsp_gaps>0</Hsp_gaps>
      <Hsp_align-len>30</Hsp_align-len>
      <Hsp_qseq>MERQKRKADIEKGLQFIQSTLPLKQEEYEA</Hsp_qseq>
      <Hsp_hseq>MERQKRKADIEKGLQFIQSTLPLKQEEYEA</Hsp_hseq>
      <Hsp_midline>MERQKRKADIEKGLQFIQSTLPLKQEEYEA</Hsp_midline>
    </Hsp>
  </Hit_hsps>
</Hit>
<Hit>
  <Hit_num>2</Hit_num>
  <Hit_id>gi|6176337|gb|AF188530.1|AF188530</Hit_id>
  <Hit_def>Homo sapiens ubiquitous tetratricopeptide containing protein RoXaN mRNA, partial cds</Hit_def&g
t;
  <Hit_accession>AF188530</Hit_accession>
  <Hit_len>2398</Hit_len>
  <Hit_hsps>
    <Hsp>
      <Hsp_num>1</Hsp_num>
      <Hsp_bit-score>62.3882</Hsp_bit-score>
      <Hsp_score>150</Hsp_score>
      <Hsp_evalue>4.12279e-10</Hsp_evalue>
      <Hsp_query-from>1</Hsp_query-from>
      <Hsp_query-to>30</Hsp_query-to>
      <Hsp_hit-from>105</Hsp_hit-from>
      <Hsp_hit-to>194</Hsp_hit-to>
      <Hsp_query-frame>0</Hsp_query-frame>
      <Hsp_hit-frame>3</Hsp_hit-frame>
      <Hsp_identity>30</Hsp_identity>
      <Hsp_positive>30</Hsp_positive>
      <Hsp_gaps>0</Hsp_gaps>
      <Hsp_align-len>30</Hsp_align-len>
      <Hsp_qseq>MERQKRKADIEKGLQFIQSTLPLKQEEYEA</Hsp_qseq>
      <Hsp_hseq>MERQKRKADIEKGLQFIQSTLPLKQEEYEA</Hsp_hseq>
      <Hsp_midline>MERQKRKADIEKGLQFIQSTLPLKQEEYEA</Hsp_midline>
    </Hsp>
  </Hit_hsps>
(...)
We want to insert that XML file into a database. I wrote the following XSLT stylesheet , it transforms the blast-xml into a set of SQL statements for sqlite3.
xsltproc --novalid blast2sqlite.xsl blast.xml 

create table BlastOutput(
(...)


BEGIN TRANSACTION;

insert into BlastOutput(
 program,
 version,
 reference,
 db,
 query_ID,
 query_def,
 query_len
 )
 values (
  'tblastn',
  'TBLASTN 2.2.27+',
  'Stephen F. Altschul, Thomas L. Madden, Alejandro A. Sch&auml;ffer, Jinghui Zhang,Zheng Zhang, Webb Miller, and David J. Lipman (1997), "Gapped BLAST and PSI-BLAST: a new generation of protein database search programs", Nucleic Acids Res. 25:3389-3402.',
  'nr',
  '52385',
  'myseq',
  30
  );


insert into Parameters(
 blastOutput_id,
 expect,
 matrix,
 sc_match,
 sc_mismatch,
 gap_open,
 gap_extend,
 filter
 )
select MAX(id),
 10,
 'BLOSUM62',
 NULL,
 NULL,
 11,
 1,
 'L;'
 from BlastOutput;
 


insert into Iteration(
 blastOutput_id,
 iter_num,
 query_id,
 query_def,
 query_len
 )
select MAX(id),
 1,
 '52385',
 'myseq',
 30
from BlastOutput;

insert into Hit(iteration_id,num,hit_id,def,accession,len)
select MAX(id),
 1,
 'gi|110624327|dbj|AK225891.1|',
 'Homo sapiens mRNA for zinc finger CCCH-type containing 7B variant, clone: FCC121C02',
 'AK225891',
 1829
from Iteration;
(...)
All in one you can redirect the output to sqlite3.
xsltproc --novalid blast2sqlite.xsl blast.xml |\
 sqlite3 input.db
and query the database:
$ sqlite3 -header -line input.sqlite \
 'select * from Hsp,Hit where Hsp.hit_id=Hit.id limit 2'

          id = 1
      hit_id = 1
         num = 1
   bit_score = 62.3882
       score = 150.0
      evalue = 4.01658e-10
  query_from = 1
    query_to = 30
    hit_from = 250
      hit_to = 339
 query_frame = 0
   hit_frame = 1
    identity = 30
    positive = 30
        gaps = 0
   align_len = 30
        qseq = MERQKRKADIEKGLQFIQSTLPLKQEEYEA
        hseq = MERQKRKADIEKGLQFIQSTLPLKQEEYEA
     midline = MERQKRKADIEKGLQFIQSTLPLKQEEYEA
          id = 1
iteration_id = 1
         num = 1
      hit_id = gi|110624327|dbj|AK225891.1|
         def = Homo sapiens mRNA for zinc finger CCCH-type containing 7B variant, clone: FCC121C02
   accession = AK225891
         len = 1829

          id = 2
      hit_id = 2
         num = 1
   bit_score = 62.3882
       score = 150.0
      evalue = 4.12279e-10
  query_from = 1
    query_to = 30
    hit_from = 105
      hit_to = 194
 query_frame = 0
   hit_frame = 3
    identity = 30
    positive = 30
        gaps = 0
   align_len = 30
        qseq = MERQKRKADIEKGLQFIQSTLPLKQEEYEA
        hseq = MERQKRKADIEKGLQFIQSTLPLKQEEYEA
     midline = MERQKRKADIEKGLQFIQSTLPLKQEEYEA
          id = 2
iteration_id = 1
         num = 2
      hit_id = gi|6176337|gb|AF188530.1|AF188530
         def = Homo sapiens ubiquitous tetratricopeptide containing protein RoXaN mRNA, partial cds
   accession = AF188530
         len = 2398
That's it,

Pierre

02 November 2012

Saving your tweets in a database using sqlite, rhino, scribe, javascript

In the current post, I 'll describe a simple method to save your tweets in a sqlite database using Mozilla Rhino.

Prerequisites

  • sqlite
  • Apache Rhino. I think it should be de-facto available when the java developer toolkit (JDK) is installed
  • Scribe, the simple OAuth library for Java . It also requires Apache codec

The config.js file

Open an account on https://dev.twitter.com/ and create an App to receive an API-key and an API-secret.
Create the following file 'config.js' filled with the correct parameters.

The javascript

The following javascript file opens a Oauth connection, retrieves the tweets and stores them into sqlite. I've commented the code, I hope it is clear enough.

Running the script using Rhino

scribe.libs=/path/to/scribe-1.3.2.jar:/path/to/commons-codec.jar
rhino.libs=/usr/share/java/js.jar:/usr/share/java/jline.jar
sqlite.libs=/path/to/sqlitejdbc-v056.jar
CLASSPATH=${rhino.libs}:${scribe.libs}:${sqlite.libs}

java -cp ${CLASSPATH} org.mozilla.javascript.tools.shell.Main -f twitter2sqlite.js
At the first time, the user is asked to authorize the application to use the twitter API

The script runs forever (Ctrl-C to break), listening to the new tweets.

As a test, I wrote the following tweet:


... and the tweet was later inserted in the database...

Sleep...

Inserted ({created_at:"Fri Nov 02 20:29:04 +0000 2012", id:264464160664981500, id_str:"264464160664981504", text:"wrote a tool to save my tweets: This is a test . ( #rhino, #jdbc, #sqlite, #scribe #javascript )", source:"web", truncated:false, in_reply_to_status_id:null, in_reply_to_status_id_str:null, in_reply_to_user_id:null, in_reply_to_user_id_str:null, in_reply_to_screen_name:null, geo:null, coordinates:null, place:null, contributors:null, retweet_count:0, entities:{hashtags:[{text:"rhino", indices:[51, 57]}, {text:"jdbc", indices:[59, 64]}, {text:"sqlite", indices:[66, 73]}, {text:"scribe", indices:[75, 82]}, {text:"javascript", indices:[83, 94]}], urls:[], user_mentions:[]}, favorited:false, retweeted:false})

Sleep...
Sleep...
Sleep...

Later, the tweets can be extracted using the sqlite command line:

$  sqlite3 tweets.sqlite 'select * from tweet'

264464160664981504|({created_at:"Fri Nov 02 20:29:04 +0000 2012", id:264464160664981500, id_str:"264464160664981504", text:"wrote a tool to save my tweets: This
264421310841638913|({created_at:"Fri Nov 02 17:38:47 +0000 2012", id:264421310841638900, id_str:"264421310841638913", text:"The tools for recalibration have cha
264264932097400832|({created_at:"Fri Nov 02 07:17:24 +0000 2012", id:264264932097400830, id_str:"264264932097400832", text:"@warandpeace you're welcome. Your sh
264158323287416832|({created_at:"Fri Nov 02 00:13:46 +0000 2012", id:264158323287416830, id_str:"264158323287416832", text:"Drawing of the day November 1, 2012.
264142732174438400|({created_at:"Thu Nov 01 23:11:49 +0000 2012", id:264142732174438400, id_str:"264142732174438400", text:"[delicious] PLOS Collections: How th
264064117558624256|({created_at:"Thu Nov 01 17:59:26 +0000 2012", id:264064117558624260, id_str:"264064117558624256", text:"I've added a stupid basic dependency
264025607724204034|({created_at:"Thu Nov 01 15:26:24 +0000 2012", id:264025607724204030, id_str:"264025607724204034", text:"in the desert lab, checking my on-go
264013563704795136|({created_at:"Thu Nov 01 14:38:33 +0000 2012", id:264013563704795140, id_str:"264013563704795136", text:"Drawing of the day November 1, 2012.
263996436679630848|({created_at:"Thu Nov 01 13:30:29 +0000 2012", id:263996436679630850, id_str:"263996436679630848", text:"RT @RealistComics: he's tall, dark a
263966759210590208|({created_at:"Thu Nov 01 11:32:34 +0000 2012", id:263966759210590200, id_str:"263966759210590208", text:"RT @guermonprez: #Aubry Un avion nor
263946369847398402|({created_at:"Thu Nov 01 10:11:33 +0000 2012", id:263946369847398400, id_str:"263946369847398402", text:"[delicious] OVal: object validation 
263946366919790593|({created_at:"Thu Nov 01 10:11:32 +0000 2012", id:263946366919790600, id_str:"263946366919790593", text:"[delicious] MyBatis #tweet: a first 
263941020729896960|({created_at:"Thu Nov 01 09:50:17 +0000 2012", id:263941020729896960, id_str:"263941020729896960", text:"RT @josh_wills: I have never been pr
263938670187388928|({created_at:"Thu Nov 01 09:40:57 +0000 2012", id:263938670187388930, id_str:"263938670187388928", text:"RT @softmodeling @peterneubauer: Usi
263936362716200960|({created_at:"Thu Nov 01 09:31:47 +0000 2012", id:263936362716200960, id_str:"263936362716200960", text:"declined to review an article about 
263934528186351616|({created_at:"Thu Nov 01 09:24:29 +0000 2012", id:263934528186351600, id_str:"263934528186351616", text:"@figshare Thanks, ( was http://t.co/
263815846139412480|({created_at:"Thu Nov 01 01:32:53 +0000 2012", id:263815846139412480, id_str:"263815846139412480", text:"Drawing of the day October 30, 2012.
263731855919026176|({created_at:"Wed Oct 31 19:59:09 +0000 2012", id:263731855919026180, id_str:"263731855919026176", text:"[delicious] An integrated map of gen
263726281647067136|({created_at:"Wed Oct 31 19:36:59 +0000 2012", id:263726281647067140, id_str:"263726281647067136", text:"RT @bryan_howie: 1000 Genomes paper 
263695076516052992|({created_at:"Wed Oct 31 17:33:00 +0000 2012", id:263695076516053000, id_str:"263695076516052992", text:"\"Forget your Past\" ( abandoned Bul

That's it
Pierre

08 May 2012

Downloading the XML data from the Exome Variant Server

From EVS: "The goal of the NHLBI GO Exome Sequencing Project (ESP) is to discover novel genes and mechanisms contributing to heart, lung and blood disorders by pioneering the application of next-generation sequencing of the protein coding regions of the human genome across diverse, richly-phenotyped populations and to share these datasets and findings with the scientific community to extend and enrich the diagnosis, management and treatment of heart, lung and blood disorders."

The NHLBI Exome Sequencing Project provides a download area but I wanted to build a local database for the richer XML data returned by their Web Services (previously described here on my blog ). The following java program sends some XML/SOAP requests to the EVS server for each chromosome using a genomic window of 150000 bp and parses the XML response.



it then dumps the results into a tab-delimited file:

Compilation:

javac DumpExomeVariantServerData.java

Execution:
java DumpExomeVariantServerData > input.evs.tsv

May 8, 2012 7:59:58 AM sandbox.DumpExomeVariantServerData fetchEvsData
INFO: 1:1-200011
May 8, 2012 8:00:02 AM sandbox.DumpExomeVariantServerData fetchEvsData
INFO: 1:200001-400011
May 8, 2012 8:00:03 AM sandbox.DumpExomeVariantServerData fetchEvsData
INFO: 1:400001-600011
May 8, 2012 8:00:04 AM sandbox.DumpExomeVariantServerData fetchEvsData
INFO: 1:600001-800011
May 8, 2012 8:00:05 AM sandbox.DumpExomeVariantServerData fetchEvsData
INFO: 1:800001-1000011
(...)

head input.evs.tsv
1 69116 <snpList><positionString>1:69116</positionString><chrPosition>69116</chr
1 69134 <snpList><positionString>1:69134</positionString><chrPosition>69134</chr
1 69270 <snpList><positionString>1:69270</positionString><chrPosition>69270</chr
1 69428 <snpList><positionString>1:69428</positionString><chrPosition>69428</chr
1 69453 <snpList><positionString>1:69453</positionString><chrPosition>69453</chr
1 69476 <snpList><positionString>1:69476</positionString><chrPosition>69476</chr
1 69496 <snpList><positionString>1:69496</positionString><chrPosition>69496</chr
1 69511 <snpList><positionString>1:69511</positionString><chrPosition>69511</chr
1 69552 <snpList><positionString>1:69552</positionString><chrPosition>69552</chr
1 69590 <snpList><positionString>1:69590</positionString><chrPosition>69590</chr

Inserting the EVS data in a sqlite database

We can now create a sqlite3 database to insert the data ...
$ sqlite3 evs.sqlite
sqlite> create table evsData(chrom TEXT NOT NULL,pos INT NOT NULL,xml TEXT NOT NULL);
sqlite> create index chrompos on evsData(chrom,pos);
sqlite> .separator "\t";
sqlite> .import "input.evs.tsv" evsData

... and query this database
$ sqlite3 evs.sqlite 'select xml from evsData where chrom="1" and pos=69552' |\
xmllint  --format -

<?xml version="1.0"?>
<snpList>
  <positionString>1:69552</positionString>
  <chrPosition>69552</chrPosition>
  <alleles>C/G</alleles>
  <uaAlleleCounts>C=4/G=4644</uaAlleleCounts>
  <aaAlleleCounts>C=0/G=2944</aaAlleleCounts>
  <totalAlleleCounts>C=4/G=7588</totalAlleleCounts>
  <uaMAF>0.0861</uaMAF>
  <aaMAF>0.0</aaMAF>
  <totalMAF>0.0527</totalMAF>
  <avgSampleReadDepth>143</avgSampleReadDepth>
  <geneList>OR4F5</geneList>
  <snpFunction>
    <chromosome>1</chromosome>
    <position>69552</position>
    <conservationScore>1.0</conservationScore>
    <conservationScoreGERP>-0.1</conservationScoreGERP>
    <snpFxnList>
      <mrnaAccession>NM_001005484.1</mrnaAccession>
      <fxnClassGVS>coding-synonymous</fxnClassGVS>
      <aminoAcids>none</aminoAcids>
      <proteinPos>154/306</proteinPos>
      <cdnaPos>462</cdnaPos>
      <pphPrediction>unknown</pphPrediction>
      <granthamScore>NA</granthamScore>
    </snpFxnList>
    <refAllele>G</refAllele>
    <ancestralAllele>C</ancestralAllele>
    <firstRsId>55874132</firstRsId>
    <secondRsId>0</secondRsId>
    <filters>SVM</filters>
    <clinicalLink>unknown</clinicalLink>
  </snpFunction>
  <conservationScore>1.0</conservationScore>
  <conservationScoreGERP>-0.1</conservationScoreGERP>
  <refAllele>G</refAllele>
  <altAlleles>C</altAlleles>
  <ancestralAllele>C</ancestralAllele>
  <chromosome>1</chromosome>
  <hasAtLeastOneAccession>true</hasAtLeastOneAccession>
  <rsIds>rs55874132</rsIds>
  <filters>SVM</filters>
  <clinicalLink>unknown</clinicalLink>
  <dbsnpVersion>dbSNP_129</dbsnpVersion>
  <uaGenotypeCounts>CC=0/CG=4/GG=2320</uaGenotypeCounts>
  <aaGenotypeCounts>CC=0/CG=0/GG=1472</aaGenotypeCounts>
  <totalGenotypeCounts>CC=0/CG=4/GG=3792</totalGenotypeCounts>
  <onExomeChip>false</onExomeChip>
  <gwasPubmedIds>unknown</gwasPubmedIds>
</snpList>

The Variation Toolkit

I also wrote a C++ program (that is part of my (always-beta) Variation Toolkit) to use this sqlite database to annotate some VCF-like files. See http://code.google.com/p/variationtoolkit/wiki/VcfEvs

Example 1

$ echo -e "#CHROM\tPOS\n1\t69511\n1\t69512\n1\t69552" |\
vcfevs -f evs.sqlite 

#CHROM  POS     evs.positionString      evs.chrPosition evs.alleles     evs.uaAlleleCounts      evs.aaAlleleCounts      evs.totalAlleleCounts   evs.uaMAF       evs.aaMAF       evs.totalMAF    evs.avgSampleReadDepth  evs.geneList    evs.conservationScore   evs.conservationScoreGERP       evs.refAllele   evs.altAllelesevs.ancestralAllele       evs.chromosome  evs.hasAtLeastOneAccession      evs.rsIds       evs.filters     evs.clinicalLink        evs.dbsnpVersion        evs.uaGenotypeCounts    evs.aaGenotypeCounts    evs.totalGenotypeCounts evs.onExomeChip evs.gwasPubmedIds
1       69511   1:69511 69511   G/A     G=4235/A=483    G=1707/A=1297   G=5942/A=1780   10.2374 43.1758 23.051  74      OR4F5   1.0     1.1     A       G       G1      true    rs75062661      PASS    unknown dbSNP_131       GG=1964/GA=307/AA=88    GG=703/GA=301/AA=498    GG=2667/GA=608/AA=586   false   unknown
1       69512   .       .       .       .       .       .       .       .       .       .       .       .       .       .       .       .       .       .       ..      .       .       .       .       .       .       .
1       69552   1:69552 69552   C/G     C=4/G=4644      C=0/G=2944      C=4/G=7588      0.0861  0.0     0.0527  143     OR4F5   1.0     -0.1    G       C       C1      true    rs55874132      SVM     unknown dbSNP_129       CC=0/CG=4/GG=2320       CC=0/CG=0/GG=1472       CC=0/CG=4/GG=3792       false   unknown

Example 2

$ echo -e "#CHROM\tPOS\n1\t69511\n1\t69512\n1\t69552" |\
vcfevs -f ~/WORK/20120506.evs.download/evs.sqlite -c uaMAF 
#CHROM  POS     evs.uaMAF
1       69511   10.2374
1       69512   .
1       69552   0.0861

Example 3

$ echo -e "#CHROM\tPOS\n1\t69511\n1\t69512\n1\t69552" |\
vcfevs -f evs.sqlite  -x -c _ | cut -c 1-200

#CHROM  POS     evs.xml
1       69511   <snpList><positionString>1:69511</positionString><chrPosition>69511</chrPosition><alleles>G/A</alleles><uaAlleleCounts>G=4235/A=483</uaAlleleCounts><aaAlleleCounts>G=1707/A=1297</aaAlleleCount
1       69512   .
1       69552   <snpList><positionString>1:69552</positionString><chrPosition>69552</chrPosition><alleles>C/G</alleles><uaAlleleCounts>C=4/G=4644</uaAlleleCounts><aaAlleleCounts>C=0/G=2944</aaAlleleCounts><to

That's it,

Pierre

26 July 2011

A mysql full-text parser searching for some SNPs

A mysql full-text parser server plugin can be used to replace or modify the built-in full-text parser.
This post a full-text parser plugin named bioparser that extracts the rs#### id from a text.

The source

The source of this plugin is available on github at:
https://github.com/lindenb/bioudf/blob/master/src/bioparser.c.

The work horse of the plugin is a simple function bioparser_parse scanning the SQL query or the TEXT. Each time a word starting with "rs" and followed by one or more number is found, it is added to the list of words to be find:
static int bioparser_parse(MYSQL_FTPARSER_PARAM *param)
{
char *curr=param->doc;
const char *begin=param->doc;
const char *end= begin + param->length;

param->flags = MYSQL_FTFLAGS_NEED_COPY;
while(curr+2<end)
{
if(tolower(*curr)=='r' &&
tolower(*(curr+1))=='s' &&
isdigit(*(curr+2)) &&
(curr==begin || IS_DELIM(*(curr-1) ) )
)
{
char* p=curr+2;
while(p!=end && isdigit(*p))
{
++p;
}
if(p==end || IS_DELIM(*p))
{
my_add_word(param,curr,p-curr);
}
curr=p;
}
else
{
curr++;
}
}

return 0;
}

Install the Plugin


mysql> INSTALL PLUGIN bioparser SONAME 'bioparser.so';
Query OK, 0 rows affected (0.00 sec)

mysql> show plugins;
+-----------------------+----------+--------------------+--------------+---------+
| Name | Status | Type | Library | License |
+-----------------------+----------+--------------------+--------------+---------+
(...)
| partition | ACTIVE | STORAGE ENGINE | NULL | GPL |
| bioparser | ACTIVE | FTPARSER | bioparser.so | GPL |
+-----------------------+----------+--------------------+--------------+---------+
21 rows in set (0.00 sec)

Invoke Plugin


create a table that will use the plugin:
mysql> create table pubmed(
abstract TEXT,
FULLTEXT (abstract) WITH PARSER bioparser
) ENGINE=MyISAM;

Insert some abstracts.
mysql> insert into pubmed(abstract) values("A predictive role in radiation pneumonitis (RP) development was observed for the LIG4 SNP rs1805388 (adjusted hazard ratio, 2.08; 95% confidence interval, 1.04-4.12; P = .037 for the CT/TT genotype vs the CC genotype). In addition, men with the TT genotype of the XRCC4 rs6869366 SNP and women with AG + AA genotypes of the XRCC5 rs3835 SNP also were at increased risk of developing severe RP.");
Query OK, 1 row affected (0.00 sec)

(...)

mysql> select abstract from pubmed;
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| abstract |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| A predictive role in radiation pneumonitis (RP) development was observed for the LIG4 SNP rs1805388 (adjusted hazard ratio, 2.08; 95% confidence interval, 1.04-4.12; P = .037 for the CT/TT genotype vs the CC genotype). In addition, men with the TT genotype of the XRCC4 rs6869366 SNP and women with AG + AA genotypes of the XRCC5 rs3835 SNP also were at increased risk of developing severe RP. |
| Nonhomologous end joining (NHEJ) is a pathway that repairs DNA double-strand breaks (DSBs) to maintain genomic stability in response to irradiation. The authors hypothesized that single nucleotide polymorphisms (SNPs) in NHEJ repair genes may affect clinical outcomes in patients with nonsmall cell lung cancer (NSCLC) who receive definitive radio(chemo)therapy. |
| The authors genotyped 5 potentially functional SNPs-x-ray repair complementing defective repair in Chinese hamster cells 4 (XRCC4) reference SNP (rs) number rs6869366 (-1394 guanine to thymine [-1394G?T] change) and rs28360071 (intron 3, deletion/insertion), XRCC5 rs3835 (guanine to adenine [G?A] change at nucleotide 2408), XRCC6 rs2267437 (-1310 cytosine to guanine [C?G) change], and DNA ligase IV (LIG4) rs1805388 (threonine-to-isoleucine change at codon 9 [T9I])-and estimated their associations with severe radiation pneumonitis (RP) (grade ?3) in 195 patients with NSCLC. |
| The current results indicated that NHEJ genetic polymorphisms, particularly LIG4 rs1805388, may modulate the risk of RP in patients with NSCLC who receive definitive radio(chemo)therapy. Large studies will be needed to confirm these findings. |
| The repair of DNA double-strand breaks (DSBs) is the major mechanism to maintain genomic stability in response to irradiation. We hypothesized that genetic polymorphisms in DSB repair genes may affect clinical outcomes among non-small cell lung cancer (NSCLC) patients treated with definitive radio(chemo)therapy. |
| We also found that RAD51 -135G>C and XRCC2 R188H SNPs were independent prognostic factors for overall survival (adjusted HR?=?1.70, 95% CI, 1.14-2.62, P?=?0.009 for CG/CC vs. GG; and adjusted HR?=?1.70; 95% CI, 1.02-2.85, P?=?0.043 for AG vs. GG, respectively) and that the SNP-survival association was most pronounced in the presence of RP. |
| A total of 291 patients (145 male/146 female, mean age (± S.D.) 52.2 (± 13.1) years) with PsA were examined clinically, by standard laboratory tests and their DNA was genotyped for the SNP rs2476601 (PTPN22 +1858 C/T). Allelic frequencies were determined and compared with 725 controls. |
| this is a test rs2476601, rs1805388, rs3835 and rs25 |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
8 rows in set (0.00 sec)

Now, test the plugin:

mysql> select
concat(left(abstract,40),"...") as ABSTRACT,
match(abstract) against("DNA double-strand) as SCORE
from pubmed group by 2 HAVING SCORE!=0;
Empty set (0.00 sec)

mysql> select
concat(left(abstract,40),"...") as ABSTRACT,
match(abstract) against("rs25") as SCORE
from pubmed group by 2 HAVING SCORE!=0;
+---------------------------------------------+--------------------+
| ABSTRACT | SCORE |
+---------------------------------------------+--------------------+
| this is a test rs2476601, rs1805388, rs3... | 1.8603347539901733 |
+---------------------------------------------+--------------------+
1 row in set (0.01 sec)

mysql> select
concat(left(abstract,40),"...") as ABSTRACT,
match(abstract) against("rs2476601 rs1805388 rs6869366") as SCORE
from pubmed group by 2 HAVING SCORE!=0 order by 2 desc;
+---------------------------------------------+--------------------+
| ABSTRACT | SCORE |
+---------------------------------------------+--------------------+
| A total of 291 patients (145 male/146 fe... | 1.086121916770935 |
| A predictive role in radiation pneumonit... | 1.0619741678237915 |
| this is a test rs2476601, rs1805388, rs3... | 1.0502985715866089 |
| The authors genotyped 5 potentially func... | 1.0388768911361694 |
+---------------------------------------------+--------------------+
4 rows in set (0.00 sec)

mysql> select
concat(left(abstract,40),"...") as ABSTRACT,
match(abstract) against("rs25,rs2476601,rs1805388,rs6869366") as SCORE
from pubmed group by 2 HAVING SCORE!=0 order by 2 desc;
+---------------------------------------------+--------------------+
| ABSTRACT | SCORE |
+---------------------------------------------+--------------------+
| this is a test rs2476601, rs1805388, rs3... | 2.9106333255767822 |
| A total of 291 patients (145 male/146 fe... | 1.086121916770935 |
| A predictive role in radiation pneumonit... | 1.0619741678237915 |
| The authors genotyped 5 potentially func... | 1.0388768911361694 |
+---------------------------------------------+--------------------+
4 rows in set (0.00 sec)

uninstall the plugin


mysql> UNINSTALL PLUGIN bioparser;
Query OK, 0 rows affected (0.00 sec)


That's it,

Pierre

11 May 2010

Playing with the C API for sqlite: inserting the hapmap data , my notebook.



In this post, I'll insert a Hapmap genotype file into a SQLite database using the C API for SQLIte. Of course, this is just a test and I would not use SQLite to store this kind of information. Moreover the speed of insertion of SQLite was deadly slow here.

The Genotype file

The first line contains the name of the individuals, the other rows of the file contain one SNP per line and the genotype for each individual:
rs# alleles chrom pos strand assembly# center protLSID assayLSID panelLSID QCcode NA06984 NA06985 NA06986 NA06989 NA06991 NA06993 NA06994 NA06995 NA06997 NA07000 NA07014 NA07019 NA07022 NA07029 NA07031 NA07034 NA07037 NA07045 NA07048 NA07051 NA07055 NA07056 NA07345 NA07346 NA07347 NA07348 NA07349 NA07357 NA07435 NA10830 NA10831 NA10835 NA10836 NA10837 NA10838 NA10839 NA10840 NA10843 NA10845 NA10846 NA10847 NA10850 NA10851 NA10852 NA10853 NA10854 NA10855 NA10856 NA10857 NA10859 NA10860 NA10861 NA10863 NA10864 NA10865 NA11829 NA11830 NA11831 NA11832 NA11839 NA11840 NA11843 NA11881 NA11882 NA11891 NA11892 NA11893 NA11894 NA11917 NA11918 NA11919 NA11920 NA11930 NA11931 NA11992 NA11993 NA11994 NA11995 NA12003 NA12004 NA12005 NA12006 NA12043 NA12044 NA12045 NA12056 NA12057 NA12144 NA12145 NA12146 NA12154 NA12155 NA12156 NA12234 NA12236 NA12239 NA12248 NA12249 NA12264 NA12272 NA12273 NA12275 NA12282 NA12283 NA12286 NA12287 NA12335 NA12336 NA12340 NA12341 NA12342 NA12343 NA12344 NA12347 NA12348 NA12375 NA12376 NA12383 NA12386 NA12399 NA12400 NA12413 NA12489 NA12546 NA12707 NA12708 NA12716 NA12717 NA12718 NA12739 NA12740 NA12748 NA12749 NA12750 NA12751 NA12752 NA12753 NA12760 NA12761 NA12762 NA12763 NA12766 NA12767 NA12775 NA12776 NA12777 NA12778 NA12801 NA12802 NA12812 NA12813 NA12814 NA12815 NA12817 NA12818 NA12827 NA12828 NA12829 NA12830 NA12832 NA12842 NA12843 NA12864 NA12865 NA12872 NA12873 NA12874 NA12875 NA12877 NA12878 NA12889 NA12890 NA12891 NA12892
rs6423165 A/G chrY 109805 + ncbi_b36 broad urn:LSID:affymetrix.hapmap.org:Protocol:GenomeWideSNP_6.0:3 urn:LSID:broad.hapmap.org:Assay:SNP_A-8572888:3 urn:lsid:dcc.hapmap.org:Panel:CEPH-60-trios:3 QC+ GG AG GG AG AG AG AA AA GG AA AG NN AG AA AG NN AA AG NN GG GG GG GG AA GG GG AG AG AA AG GG AG AG AG AA AG AG AG GG GG AA AA NN GG AG NN AA AG NN GG NN GG GG GG AA AG AA AA AG GG AG AG GG AG AA AG GG AG GG GG AG GG AG GG GG AG GG GG AA NN AG GG AG GG GG GG AG GG GG AA AA GG GG GG NN AG AG AG GG AG AG AA AA GG AA GG AA AG AA AG GG AG GG GG AG AG GG GG AG AG AG NN AG AG AG AG AG NN AA AA AG AG AG AG AG AA AA AG AG AA AG GG AG GG GG AG AG AG AG AG AG AA AG AG AG AG AA AA AG GG GG GG GG AG AG GG GG AG AA GG AG AG GG AG
rs6608381 C/T chrY 109922 + ncbi_b36 broad urn:LSID:affymetrix.hapmap.org:Protocol:GenomeWideSNP_6.0:3 urn:LSID:broad.hapmap.org:Assay:SNP_A-8528859:3 urn:lsid:dcc.hapmap.org:Panel:CEPH-60-trios:3 QC+ CC CT CC CT CT CT TT TT CC TT CT NN CT TT CT NN TT CC NN CC CC CC CC TT CC CC CT CT TT CT CC CT CT CT TT CT CT CT CC CC TT TT NN CC CT NN TT CT NN CC NN CC CC CC TT CT TT TT CT CC CT CT CC CT TT CT CC CT CC CC CT CC CT CC CC CT CC CC TT NN CT CC CT CC CC CC CT CC CC TT TT CC CC CC NN CT CT CT CC CT CT TT TT CC TT CC TT CT TT CT CC CT CC CC CT CT CC CC CT CT CT NN CT CT CT CT CT NN TT TT CT CT CT CT CT TT TT CT CT TT CT CC CT CC CC CT CT CC CT CT CC TT CT CT CT CT TT TT CT CC CC CC CC CT CT CC CC CT TT CC CT CT CC CT
rs6644972 A/G chrY 118624 + ncbi_b36 sanger urn:LSID:illumina.hapmap.org:Protocol:Human_1M_BeadChip:3 urn:LSID:sanger.hapmap.org:Assay:H1Mrs6644972:3 urn:lsid:dcc.hapmap.org:Panel:CEPH-60-trios:3 QC+ GG GG AG GG AG AG AG GG AG GG AG NN GG GG AG NN GG GG NN AG AG GG GG GG GG GG GG GG GG AG AA GG GG GG AG GG GG GG GG GG GG GG NN GG AG GG GG GG NN AG NN AG AG AA GG GG GG GG GG GG GG GG GG AG GG GG AA AG GG AG GG GG GG GG GG GG GG AG AG NN AG GG AG GG GG GG AG GG GG GG GG AA AA AG NN GG GG GG GG GG GG GG AG GG GG GG GG AG GG GG AG GG GG GG GG GG GG GG GG GG GG AG GG GG GG GG GG NN GG GG AG GG GG AG GG AG GG AG AA GG GG GG GG AG GG GG GG AG GG AG GG GG GG AG GG GG AG GG AG GG GG GG GG GG GG AG GG GG GG GG GG GG GG GG
rs6655397 A/G chrY 141935 + ncbi_b36 sanger urn:LSID:illumina.hapmap.org:Protocol:Human_1M_BeadChip:3 urn:LSID:sanger.hapmap.org:Assay:H1Mrs6655397:3 urn:lsid:dcc.hapmap.org:Panel:CEPH-60-trios:3 QC+ GG GG GG GG AG AG GG GG GG GG GG NN GG GG NN NN GG GG NN GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG NN AG GG GG GG GG NN AG NN GG GG GG NN AG GG GG GG AG GG GG AG GG GG GG GG AG GG AG GG GG GG GG AG AG GG GG GG NN AG GG GG GG GG GG GG GG GG GG GG GG GG GG NN GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG NN GG GG AG GG AG NN GG GG GG GG GG GG GG GG AG GG GG AG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG GG AG GG GG GG GG GG GG
rs6603172 C/T chrY 142439 + ncbi_b36 broad urn:LSID:affymetrix.hapmap.org:Protocol:GenomeWideSNP_6.0:3 urn:LSID:broad.hapmap.org:Assay:SNP_A-8527413:3 urn:lsid:dcc.hapmap.org:Panel:CEPH-60-trios:3 QC+ TT TT TT TT CT CT TT TT TT TT TT NN TT TT TT NN TT TT NN TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT NN CT TT TT TT TT NN CT NN TT TT TT TT CT TT TT TT CT TT TT CT TT TT TT TT CT TT CT TT TT TT TT CT CT TT TT TT NN CT TT TT TT TT TT TT TT TT TT TT TT TT TT NN TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT CT TT CT NN TT TT TT TT TT TT TT TT CT TT TT CT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT CT TT TT TT TT TT TT
rs6644970 A/G chrY 142664 + ncbi_b36 broad urn:LSID:affymetrix.hapmap.org:Protocol:GenomeWideSNP_6.0:3 urn:LSID:broad.hapmap.org:Assay:SNP_A-4207883:3 urn:lsid:dcc.hapmap.org:Panel:CEPH-60-trios:3 QC+ AG AA AA AG AG GG AG AG AA AA GG NN AG AA GG NN AG AG NN AG AG AG GG AG AG AG AA AG GG GG AA AG AG AG AG AG AA AA AG GG AG GG NN GG AG NN AG AG NN AG NN AA AG AA AA AG GG GG AG AG AG AG AG AG AA AG AA AG GG GG AG AG AA GG AG GG AA AG AG NN AG AG GG AG GG GG AG GG GG AA AG AA AG GG NN GG AA GG AG AG GG GG AA GG AA AG AG GG GG AG AG GG AG AG AG GG AG AG AG AG AG NN AG GG AG AG GG NN AA AG AA AA GG AG AG AG GG AG AG AG GG AG AA AG GG AA AA AG GG AA AG AG GG AA GG AA AA AG AG AG AG AG GG AG GG GG AG AG AG AA AA GG AG AG
rs11019 A/G chrY 159978 + ncbi_B36 affymetrix urn:LSID:affymetrix.hapmap.org:Protocol:GenomeWideSNP_6.0:2 urn:LSID:affymetrix.hapmap.org:Assay:SNP_A-4207841:2 urn:lsid:dcc.hapmap.org:Panel:CEPH-30-trios:1 QC+ NN AG NN NN AG GG AA NN NN AG NN AG GG AA NN AG NN NN GG NN GG AG GG NN NN GG NN GG NN GG AG GG NN NN GG AG NN NN NN AG AG NN NN NN NN AA AG GG GG AA AG GG GG NN NN GG GG AG GG AG AG NN AA AA NN NN NN NN NN NN NN NN NN NN GG AG GG AG AG GG AG AG GG GG NN GG GG AG AG GG GG AG AG AG AG AG GG GG GG NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN NN AG NN AA GG NN NN GG NN NN GG GG AG AA AG GG AA AA NN NN NN NN NN NN AG GG GG AG GG AG NN NN NN NN NN NN NN NN NN GG GG AG AG GG GG NN GG NN NN GG AG
rs1132353 C/T chrY 160116 + ncbi_b36 sanger urn:LSID:illumina.hapmap.org:Protocol:Human_1M_BeadChip:3 urn:LSID:sanger.hapmap.org:Assay:H1Mrs1132353:3 urn:lsid:dcc.hapmap.org:Panel:CEPH-60-trios:3 QC+ CC CT TT CT CT TT NN CT TT CT CC NN TT CC CT NN TT CT NN CT TT CT CT TT TT CC TT CT CC TT CT TT TT TT TT CT TT TT TT CT CT TT NN TT CT NN CT TT NN CC NN TT TT CC NN TT TT CT TT CT CT CC CC CC CT CT CC CT CC CT CT TT TT TT TT CT TT CT CT NN CT CT TT TT TT TT TT CC CT TT TT CT CT CT NN CT TT TT TT TT TT TT TT TT TT TT CT CT CC TT CT TT CT CT CT CT CT CC CC CC CC TT CC CT CT CT CC NN TT CT TT TT CT TT CT CT CC CT TT CC CC TT TT TT CT TT TT CT TT TT CT TT CT TT TT CT TT CT TT CT CT CT TT TT CT CT TT TT CT TT TT CC TT CT
rs35047434 C/T chrY 169118 + ncbi_b36 sanger urn:LSID:illumina.hapmap.org:Protocol:Human_1M_BeadChip:3 urn:LSID:sanger.hapmap.org:Assay:H1Mrs35047434:3 urn:lsid:dcc.hapmap.org:Panel:CEPH-60-trios:3 QC+ TT TT TT TT CT CT TT TT TT TT TT NN TT TT TT NN TT TT NN CT TT TT TT TT TT TT TT TT TT TT TT TT TT CT TT TT TT CT TT TT CT TT NN CT TT TT TT TT NN TT NN TT TT TT TT TT TT TT TT TT CT TT TT TT TT TT TT CT TT TT TT CT TT CT CT TT TT TT TT NN TT TT TT TT TT TT TT TT TT CT TT TT TT TT NN TT CT TT TT TT CT CT TT TT TT CT CT TT TT CT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT NN TT CT TT CT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT TT CT TT TT TT TT CT TT TT TT CT TT TT TT TT CT TT TT TT CC TT TT TT TT TT TT
(...)

Opening the SQLIte Database

if(sqlite3_open(env->fileout,&(env->connection))!=SQLITE_OK)
{
fprintf(stderr,"Cannot open sqlite file %s.\n",env->fileout);
exit(EXIT_FAILURE);
}

Closing the SQLIte Database

sqlite3_close(env->connection);

Creating a table

if(sqlite3_exec(env->connection,
"create table markers(id INTEGER PRIMARY KEY,name varchar(20) unique,chrom varchar(10),position integer)",
NULL,NULL,&error
)!=SQLITE_OK)
{
fprintf(stderr,"Cannot create table markers: %s\n",error);
exit(EXIT_FAILURE);
}

Filling a prepared statement and inserting some data

if(sqlite3_prepare(env->connection,
"insert into markers(name,chrom,position) values(?,?,?)",
-1,&pstmt_insert_marker,NULL
)!=SQLITE_OK)
{
fprintf(stderr,"Cannot compile insert into markers.\n");
exit(EXIT_FAILURE);
}
(...)

if(sqlite3_bind_text(
pstmt_insert_marker,1,
name,
strlen(name),
NULL)!=SQLITE_OK)
{
fprintf(stderr,"Cannot bind markers's name.\n");
exit(EXIT_FAILURE);
}
(
if(sqlite3_bind_text(
pstmt_insert_marker,2,
chrom,
strlen(chrom),
NULL)!=SQLITE_OK)
{
fprintf(stderr,"Cannot bind markers's chrom.\n");
exit(EXIT_FAILURE);
}
if( sqlite3_bind_int(
pstmt_insert_marker,3,
genomic_position
)!=SQLITE_OK)
{
fprintf(stderr,"Cannot bind markers's position.\n");
exit(EXIT_FAILURE);
}

if (sqlite3_step(pstmt_insert_marker) != SQLITE_DONE) {
fprintf(stderr,"Could not step insert marker.\n");
exit(EXIT_FAILURE);
}

marker_id = sqlite3_last_insert_rowid(env->connection);

sqlite3_reset(pstmt_insert_marker);

Creating a custom function for sqlite

Here I implemented a custom function named homozygous it returns true if a genotype is a string of two identical characters (but not 'N').
void is_homozygous(
sqlite3_context* context, /* context */
int argc, /* number of arguments */
sqlite3_value** argv /* arguments */
)
{
/* one text arg */
if(argc==1 &&
sqlite3_value_type(argv[0])==SQLITE_TEXT)
{
const char *alleles=(const char*)sqlite3_value_text(argv[0]);
if( alleles!=NULL &&
strlen(alleles)==2 &&
alleles[0]==alleles[1] &&
alleles[0]!='N')
{
/* set result to TRUE */
sqlite3_result_int(context,1);
return;
}
}
/* set result to FALSE */
sqlite3_result_int(context,0);
}

Telling sqlite about the new custom function

sqlite3_create_function(env->connection,
"homozygous",/* function name */
1,/* number of args */
SQLITE_UTF8,/* encoding */
NULL,
is_homozygous,/* the implementation callback */
NULL,
NULL
);

Implementing a callback for a simple 'SELECT' query

The following callback prints the results to stdout.
static int callback_select_homozygous(
void* notUsed,
int argc,/* number of args */
char** argv,/* arguments as string */
char** columns /* labels for the columns */
)
{
int i;
/* prints all the columns to stdout*/
for(i=0; i<argc; i++)
{
printf( "\"%s\": \"%s\"\n",
columns[i],
argv[i]!=NULL? argv[i] : "NULL"
);
}
printf("\n");
return 0;
}

Executing a SELECT query

Here, we select all the homoygous genotypes:
if(sqlite3_exec(
env->connection, /* An open database */
"select individuals.name,"
"markers.name,"
"markers.chrom,"
"markers.position,"
"genotypes.alleles "
"from "
"genotypes,"
"markers,"
"individuals "
"where "
"markers.id=genotypes.marker_id and "
"individuals.id=genotypes.individual_id and "
"homozygous(genotypes.alleles)=1"
, /* SQL to be evaluated */
callback_select_homozygous, /* Callback function */
NULL, /* 1st argument to callback */
&errormsg /* Error msg written here */
)!=SQLITE_OK)
{
fprintf(stderr,"Cannot select.\n");
exit(EXIT_FAILURE);
}

The full source code



Excuting the C program

[LOG]inserted marker ID.1
[LOG]inserted marker ID.2
[LOG]inserted marker ID.3
[LOG]inserted marker ID.4
[LOG]inserted marker ID.5
(... WAIT !!! )
"name": "NA12878"
"name": "rs6603251"
"chrom": "chrY"
"position": "240580"
"alleles": "CC"

"name": "NA12890"
"name": "rs6603251"
"chrom": "chrY"
"position": "240580"
"alleles": "CC"

"name": "NA12892"
"name": "rs6603251"
"chrom": "chrY"
"position": "240580"
"alleles": "CC"


That's it

Pierre

03 September 2009

Building a naive Interactome Database with Hibernate.

This is my notebook for building a naive database of protein-protein interactions with Hibernate (a java object/relational persistence and query service).

Files and Directories


./project
./project/lib
./project/src
./project/src/hibernate.cfg.xml
./project/src/org
./project/src/org/lindenb
./project/src/org/lindenb/hbn01
./project/src/org/lindenb/hbn01/Journal.java
./project/src/org/lindenb/hbn01/mapping.hbm.xml
./project/src/org/lindenb/hbn01/Article.java
./project/src/org/lindenb/hbn01/Main.java
./project/src/org/lindenb/hbn01/PMID.java
./project/src/org/lindenb/hbn01/Interactor.java
./project/src/org/lindenb/hbn01/Complex.java
./project/src/org/lindenb/hbn01/Protein.java
./project/src/org/lindenb/hbn01/PMIDType.java
./project/src/log4j.properties
./project/build
./project/bin
./derby.log
./build
./build/db
./Makefile

The components / Java Classes


Interactor


An abstract class defining a protein or a complex: Just a name and an ID.
package org.lindenb.hbn01;

public class Interactor
implements java.io.Serializable
{
private Long id;
private String name;
protected Interactor()
{
}

protected Interactor(String name)
{
setName(name);
}

private void setId(Long id)
{
this.id=id;
}

public Long getId()
{
return this.id;
}
public String getName()
{
return this.name;
}
public void setName(String name)
{
this.name=name;
}

@Override
public boolean equals(Object o)
{
if(o==this) return true;
if(o==null || o.getClass()!=getClass()) return false;
return getId().equals(Interactor.class.cast(o).getId());
}

@Override
public String toString()
{
return getClass().getName()+":"+getName()+"("+getId()+")";
}
}

Protein

Protein is a concrete subclass of Interactor. This could be an Unigene entry.
package org.lindenb.hbn01;

public class Protein
extends Interactor
{
public Protein()
{
}

public Protein(String name)
{
super(name);
}

@Override
public String toString()
{
return "Protein:"+getName();
}
}

Complex

Complex is a concrete subclass of Interactor. It is a Set of Interactors. It also contains a Set of Articles holding the references for those interactions.
package org.lindenb.hbn01;
import java.util.*;

public class Complex
extends Interactor
{
private Set<Interactor> partners= new HashSet<Interactor>();
private Set<Article> articles= new HashSet<Article>();
public Complex()
{
}

public Complex(String name)
{
super(name);
}

public Set<Interactor> getPartners()
{
return this.partners;
}
public void setPartners(Set<Interactor> partners)
{
this.partners = partners;
}
public Set<Article> getArticles()
{
return this.articles;
}

public void setArticles(Set<Article> articles)
{
this.articles = articles;
}
@Override
public String toString()
{
String s="Complex:"+getName()+". ID:"+getId()+" interacts with";
for(Interactor i: getPartners())
{
s+=" "+i.getName();
}
return s;
}
}

Article

an Article is a reference to a paper in Pubmed. I wanted to use the custom dataType in hibernate, so I used the class PMID rather than an Integer. Each Article is linked to a Journal.
package org.lindenb.hbn01;

public class Article
implements java.io.Serializable
{
private PMID pmid;
private String title;
private Integer year;
private String doi;
private Journal journal;

public Article()
{
}

public Article(PMID pmid,Journal journal,Integer year,String title)
{
setPmid(pmid);
setJournal(journal);
setYear(year);
setTitle(title);
}

public Journal getJournal()
{
return journal;
}

public String getDoi()
{
return this.doi;
}

public void setDoi(String doi)
{
this.doi=doi;
}

public void setJournal(Journal journal)
{
this.journal=journal;
}

private void setPmid(PMID pmid)
{
this.pmid=pmid;
}
public PMID getPmid()
{
return this.pmid;
}
public void setTitle(String title)
{
this.title=title;
}
public String getTitle()
{
return this.title;
}
public void setYear(Integer year)
{
this.year=year;
}
public Integer getYear()
{
return this.year;
}

public String toString()
{
return "("+getYear()+")\""+getTitle()+"\"."+getJournal().getTitle();
}
}

PMID

A custom type holding a Pubmed identifier
package org.lindenb.hbn01;
import org.hibernate.*;
import java.io.Serializable;

public class PMID
implements java.io.Serializable
{
private long pmid;

public PMID(String pmid)
{
this(new Long(pmid));
}

public PMID(long pmid)
{
this.pmid=pmid;
}

public long value()
{
return this.pmid;
}

public int hashCode()
{
return 31+(int)this.pmid;
}

public boolean equals(Object o)
{
if(o==this) return true;
if(o==null || !(o instanceof PMID)) return false;
return PMID.class.cast(o).pmid==this.pmid;
}

public String toString()
{
return String.valueOf(this.pmid);
}
}

Journal

A Journal is a NLM-Id and a title
package org.lindenb.hbn01;

public class Journal
implements java.io.Serializable
{
private long nlmId;
private String title;


public Journal()
{
}

public Journal(long nlmId,String title)
{
setNlmId(nlmId);
setTitle(title);
}

private void setNlmId(long nlmId)
{
this.nlmId=nlmId;
}
public Long getNlmId()
{
return this.nlmId;
}
public void setTitle(String title)
{
this.title=title;
}
public String getTitle()
{
return this.title;
}

public String toString()
{
return getTitle()+"["+getNlmId()+"]";
}
}

Using a Custom Type

PMIDType implements EnhancedUserType. Hibernate will use this class to manage the class PMID (how to read/write it from/to the database).
package org.lindenb.hbn01;
import org.hibernate.*;
import org.hibernate.usertype.EnhancedUserType;
import java.sql.*;
import java.io.Serializable;

public class PMIDType
implements EnhancedUserType
{


public int[] sqlTypes() {
return new int[]{Types.INTEGER};
}


public Object assemble(Serializable cached,
Object owner)
throws HibernateException
{
return cached;
}

public Serializable disassemble(Object value)
throws HibernateException
{
return Serializable.class.cast(value);
}

public boolean isMutable() { return false;}
public Object deepCopy(Object value)
{
return value;
}
public boolean equals(Object a, Object b)
{
return a==null?b==null:a.equals(b);
}

public int hashCode(Object x) throws HibernateException
{
return x==null?0:x.hashCode();
}

public Object nullSafeGet(ResultSet rs,
String[] names,
Object owner)
throws HibernateException,
SQLException
{
Object o = rs.getObject( names[0] );
if(rs.wasNull()) return null;
if(o instanceof Number)
{
return new PMID(Number.class.cast(o).longValue());
}
else if(o instanceof String)
{
return new PMID(String.class.cast(o));
}
throw new IllegalArgumentException("Bad class "+o.getClass());
}

public void nullSafeSet(PreparedStatement st,
Object value,
int index)
throws HibernateException, SQLException
{
if(value==null)
{
st.setNull( index, Types.INTEGER );
}
else
{
st.setLong(index,PMID.class.cast(value).value());
}
}


public Object replace(Object original,
Object target,
Object owner)
throws HibernateException
{
return original;
}

public Class<?> returnedClass()
{
return PMID.class;
}


public Object fromXMLString(String xmlValue)
{
return xmlValue==null? null: new PMID(new Long(xmlValue));
}
public String objectToSQLString(Object value)
{
return value==null? null: String.valueOf(PMID.class.cast(value).
value());
}
public String toXMLString(Object value)
{
return value==null? null: String.valueOf(PMID.class.cast(value).
value());
}
}

The mapping file

The file mapping.hbm.xml tells hibernate how the classes are linked to each others.
<!DOCTYPE hibernate-mapping PUBLIC
"-//Hibernate/Hibernate Mapping DTD 3.0//EN"
"http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">
<hibernate-mapping package="org.lindenb.hbn01" default-cascade="none" default-access="property" default-lazy="true" auto-import="true">

<class name="org.lindenb.hbn01.Article" table="Article" mutable="true" polymorphism="implicit" dynamic-update="false" dynamic-insert="false" select-before-update="false" optimistic-lock="version">
<meta attribute="class-description" inherit="true">A pubmed Article</meta>
<id name="pmid" column="pmid" type="org.lindenb.hbn01.PMIDType">
<meta attribute="field-description" inherit="true">Pubmed Identifier</meta>
<generator class="assigned"/>
</id>
<property name="title" not-null="true" unique="false" optimistic-lock="true" lazy="false" generated="never"/>
<property name="doi" unique="true" type="string" optimistic-lock="true" lazy="false" generated="never"/>
<property name="year" column="yearDate" type="integer" not-null="true" unique="false" optimistic-lock="true" lazy="false" generated="never"/>
<many-to-one name="journal" column="nlmId" not-null="true" unique="false" update="true" insert="true" optimistic-lock="true" not-found="exception" embed-xml="true"/>
</class>

<class name="Journal" mutable="true" polymorphism="implicit" dynamic-update="false" dynamic-insert="false" select-before-update="false" optimistic-lock="version">
<id name="nlmId" column="nlmId" type="long">
<generator class="assigned"/>
</id>
<property name="title" not-null="true" unique="false" optimistic-lock="true" lazy="false" generated="never"/>
</class>

<class name="Interactor" mutable="true" polymorphism="implicit" dynamic-update="false" dynamic-insert="false" select-before-update="false" optimistic-lock="version">
<id name="id" type="long">
<generator class="native"/>
</id>
<property name="name" not-null="true" unique="false" optimistic-lock="true" lazy="false" generated="never"/>

<joined-subclass name="Protein" dynamic-update="false" dynamic-insert="false" select-before-update="false">
<key column="interactorId" on-delete="noaction"/>
</joined-subclass>

<joined-subclass name="Complex" dynamic-update="false" dynamic-insert="false" select-before-update="false">
<key column="interactorId" on-delete="noaction"/>

<set name="partners" table="interactions" sort="unsorted" inverse="false" mutable="true" optimistic-lock="true" embed-xml="true">
<key column="complex_id" on-delete="noaction"/>
<many-to-many column="interactor_id" class="Interactor" embed-xml="true" not-found="exception" unique="false"/>
</set>

</joined-subclass>
</class>

</hibernate-mapping>

Configuring Hibernate


The file hibernate.cfg.xml describes the database we are using for persisting all the entities (driver, uri, login, password...). Here I've used JavaDB/Derby (Note: the 'unique' directive was ignored by Derby (?) ).
<!DOCTYPE hibernate-configuration PUBLIC
"-//Hibernate/Hibernate Configuration DTD 3.0//EN"
"http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">
<hibernate-configuration>

<session-factory>

<!-- Database connection settings -->
<property name="connection.driver_class">org.apache.derby.jdbc.EmbeddedDriver</property>
<property name="connection.url">jdbc:derby:build/db/derby/hibernate;create=true</property>
<property name="connection.username">sa</property>
<property name="connection.password"/>

<!-- JDBC connection pool (use the built-in) -->
<property name="connection.pool_size">1</property>

<!-- SQL dialect -->
<property name="dialect">org.hibernate.dialect.DerbyDialect</property>

<!-- Enable Hibernate's automatic session context management -->
<property name="current_session_context_class">thread</property>

<!-- Disable the second-level cache -->
<property name="cache.provider_class">org.hibernate.cache.NoCacheProvider</property>

<!-- Echo all executed SQL to stdout -->
<property name="show_sql">true</property>

<!-- Drop and re-create the database schema on startup -->
<property name="hbm2ddl.auto">create</property>

<mapping resource="org/lindenb/hbn01/mapping.hbm.xml"/>

</session-factory>

</hibernate-configuration>

Running


Building a Session factory

sessionFactory = new Configuration().configure().buildSessionFactory();

Creating an Interacome

Session session= getSessionFactory().getCurrentSession();
session.beginTransaction();
Journal journal = new Journal(1L,"PNAS");
session.save(journal);
Article article= new Article(new PMID(12234),journal,1988,"Article title 1");
article.setDoi("1");
session.save(article);
article= new Article(new PMID(456789),journal,1989,"Article title 2");
article.setDoi("2");
session.save(article);


Protein prot1= new Protein("prot1");
session.save(prot1);
Protein prot2= new Protein("prot2");
session.save(prot2);
Protein prot3= new Protein("prot3");
session.save(prot3);
Complex c1= new Complex("cplx1");
c1.getPartners().add(prot1);
c1.getPartners().add(prot2);
session.save(c1);
Complex c2= new Complex("cplx2");
c2.getPartners().add(prot1);
c2.getPartners().add(c1);
c2.getArticles().add(article);
session.save(c2);

session.getTransaction().commit();

Querying

Listing the Journals
Session session= getSessionFactory().getCurrentSession();
session.beginTransaction();
List list = session.createQuery("from Journal").list();

for(Object o:list)
{
System.out.println(o);
}
session.getTransaction().commit();

Listing the Articles
Session session= getSessionFactory().getCurrentSession();
session.beginTransaction();
List list = session.createQuery("from Article").list();

for(Object o:list)
{
System.out.println(o);
}
session.getTransaction().commit();
Listing the Interactors
Session session= getSessionFactory().getCurrentSession();
session.beginTransaction();
List list = session.createQuery("from Interactor").list();

for(Object o:list)
{
System.out.println("\n\n###\t"+o+"\n\n");
}
session.getTransaction().commit();

Full code

package org.lindenb.hbn01;

import org.hibernate.*;
import org.hibernate.cfg.*;
import java.util.*;

public class Main
{
private static final SessionFactory sessionFactory;

static
{
try
{
sessionFactory = new Configuration().configure().buildSessionFactory();
}
catch(Throwable err)
{
err.printStackTrace();
throw new ExceptionInInitializerError(err);
}
}

public static SessionFactory getSessionFactory()
{
return Main.sessionFactory;
}

private void listJournals()
{
Session session= getSessionFactory().getCurrentSession();
session.beginTransaction();
List list = session.createQuery("from Journal").list();

for(Object o:list)
{
System.out.println(o);
}
session.getTransaction().commit();
}

private void listArticles()
{
Session session= getSessionFactory().getCurrentSession();
session.beginTransaction();
List list = session.createQuery("from Article").list();

for(Object o:list)
{
System.out.println(o);
}
session.getTransaction().commit();
}

private void listInteractors()
{
Session session= getSessionFactory().getCurrentSession();
session.beginTransaction();
List list = session.createQuery("from Interactor").list();

for(Object o:list)
{
System.out.println("\n\n###\t"+o+"\n\n");
}
session.getTransaction().commit();
}

public void run()
{
Session session= getSessionFactory().getCurrentSession();
session.beginTransaction();
Journal journal = new Journal(1L,"PNAS");
session.save(journal);
Article article= new Article(new PMID(12234),journal,1988,"Article title 1");
article.setDoi("1");
session.save(article);
article= new Article(new PMID(456789),journal,1989,"Article title 2");
article.setDoi("2");
session.save(article);


Protein prot1= new Protein("prot1");
session.save(prot1);
Protein prot2= new Protein("prot2");
session.save(prot2);
Protein prot3= new Protein("prot3");
session.save(prot3);
Complex c1= new Complex("cplx1");
c1.getPartners().add(prot1);
c1.getPartners().add(prot2);
session.save(c1);
Complex c2= new Complex("cplx2");
c2.getPartners().add(prot1);
c2.getPartners().add(c1);
c2.getArticles().add(article);
session.save(c2);

session.getTransaction().commit();

listJournals();
listArticles();
listInteractors();
}

public static void main(String args[])
{
try
{
Main app= new Main();
app.run();
}
catch(Throwable err)
{
err.printStackTrace();
}
finally
{
if(Main.sessionFactory!=null) Main.sessionFactory.close();
}
System.out.println("Done.");
}
}

Compiling

LIB=${HIBERNATE_HOME}/lib
LIBS=${LIB}/antlr-2.7.6.jar:${LIB}/cglib-2.1.3.jar:${LIB}/asm.jar:${LIB}/asm-attrs.jar:${LIB}/commons-collections-2.1.1.jar:${LIB}/commons-logging-1.0.4.jar:${HIBERNATE_HOME}/hibernate3.jar:${LIB}/jta.jar:${LIB}/dom4j-1.6.1.jar:${LIB}/log4j-1.2.11.jar:${DERBY_HOME}/derby.jar
test:
cp -r project/src/* project/build
javac -cp ${LIBS} -d project/build -sourcepath project/build project/build/org/lindenb/hbn01/*.java
jar cvf project/bin/project.jar -C project/build .
java -cp ${LIBS}:project/bin/project.jar org.lindenb.hbn01.Main

Output

21:49:15,396 INFO Environment:514 - Hibernate 3.2.6
21:49:15,402 INFO Environment:547 - hibernate.properties not found
21:49:15,405 INFO Environment:681 - Bytecode provider name : cglib
21:49:15,409 INFO Environment:598 - using JDK 1.4 java.sql.Timestamp handling
21:49:15,457 INFO Configuration:1432 - configuring from resource: /hibernate.cfg.xml
21:49:15,458 INFO Configuration:1409 - Configuration resource: /hibernate.cfg.xml
21:49:15,546 INFO Configuration:559 - Reading mappings from resource : org/lindenb/hbn01/mapping.hbm.xml
21:49:15,682 INFO HbmBinder:300 - Mapping class: org.lindenb.hbn01.Article -> Article
21:49:15,751 INFO HbmBinder:300 - Mapping class: org.lindenb.hbn01.Journal -> Journal
21:49:15,752 INFO HbmBinder:300 - Mapping class: org.lindenb.hbn01.Interactor -> Interactor
21:49:15,778 INFO HbmBinder:873 - Mapping joined-subclass: org.lindenb.hbn01.Protein -> Protein
21:49:15,780 INFO HbmBinder:873 - Mapping joined-subclass: org.lindenb.hbn01.Complex -> Complex
21:49:15,781 INFO HbmBinder:1419 - Mapping collection: org.lindenb.hbn01.Complex.partners -> interactions
21:49:15,783 INFO Configuration:1547 - Configured SessionFactory: null
21:49:15,802 INFO DriverManagerConnectionProvider:41 - Using Hibernate built-in connection pool (not for production use!)
21:49:15,803 INFO DriverManagerConnectionProvider:42 - Hibernate connection pool size: 1
21:49:15,803 INFO DriverManagerConnectionProvider:45 - autocommit mode: false
21:49:16,036 INFO DriverManagerConnectionProvider:80 - using driver: org.apache.derby.jdbc.EmbeddedDriver at URL: jdbc:derby:build/db/derby/hibernate;create=true
21:49:16,036 INFO DriverManagerConnectionProvider:86 - connection properties: {user=sa, password=****}
21:49:18,144 INFO SettingsFactory:89 - RDBMS: Apache Derby, version: 10.2.2.1 - (538595)
21:49:18,145 INFO SettingsFactory:90 - JDBC driver: Apache Derby Embedded JDBC Driver, version: 10.2.2.1 - (538595)
21:49:18,158 INFO Dialect:152 - Using dialect: org.hibernate.dialect.DerbyDialect
21:49:18,165 INFO TransactionFactoryFactory:31 - Using default transaction strategy (direct JDBC transactions)
21:49:18,167 INFO TransactionManagerLookupFactory:33 - No TransactionManagerLookup configured (in JTA environment, use of read-write or transactional second-level cache is not recommended)
21:49:18,167 INFO SettingsFactory:143 - Automatic flush during beforeCompletion(): disabled
21:49:18,167 INFO SettingsFactory:147 - Automatic session close at end of transaction: disabled
21:49:18,168 INFO SettingsFactory:162 - Scrollable result sets: enabled
21:49:18,168 INFO SettingsFactory:170 - JDBC3 getGeneratedKeys(): disabled
21:49:18,169 INFO SettingsFactory:178 - Connection release mode: auto
21:49:18,169 INFO SettingsFactory:205 - Default batch fetch size: 1
21:49:18,170 INFO SettingsFactory:209 - Generate SQL with comments: disabled
21:49:18,170 INFO SettingsFactory:213 - Order SQL updates by primary key: disabled
21:49:18,170 INFO SettingsFactory:217 - Order SQL inserts for batching: disabled
21:49:18,170 INFO SettingsFactory:386 - Query translator: org.hibernate.hql.ast.ASTQueryTranslatorFactory
21:49:18,172 INFO ASTQueryTranslatorFactory:24 - Using ASTQueryTranslatorFactory
21:49:18,173 INFO SettingsFactory:225 - Query language substitutions: {}
21:49:18,173 INFO SettingsFactory:230 - JPA-QL strict compliance: disabled
21:49:18,173 INFO SettingsFactory:235 - Second-level cache: enabled
21:49:18,173 INFO SettingsFactory:239 - Query cache: disabled
21:49:18,174 INFO SettingsFactory:373 - Cache provider: org.hibernate.cache.NoCacheProvider
21:49:18,174 INFO SettingsFactory:254 - Optimize cache for minimal puts: disabled
21:49:18,174 INFO SettingsFactory:263 - Structured second-level cache entries: disabled
21:49:18,178 INFO SettingsFactory:283 - Echoing all SQL to stdout
21:49:18,178 INFO SettingsFactory:290 - Statistics: disabled
21:49:18,178 INFO SettingsFactory:294 - Deleted entity synthetic identifier rollback: disabled
21:49:18,178 INFO SettingsFactory:309 - Default entity-mode: pojo
21:49:18,179 INFO SettingsFactory:313 - Named query checking : enabled
21:49:18,206 INFO SessionFactoryImpl:161 - building session factory
21:49:18,472 INFO SessionFactoryObjectFactory:82 - Not binding factory to JNDI, no JNDI name configured
21:49:18,477 INFO SchemaExport:154 - Running hbm2ddl schema export
21:49:18,477 DEBUG SchemaExport:170 - import file not found: /import.sql
21:49:18,478 INFO SchemaExport:179 - exporting generated schema to database
21:49:18,482 DEBUG SchemaExport:303 - alter table Article drop constraint FK379164D684F84236
21:49:18,754 DEBUG SchemaExport:303 - alter table Complex drop constraint FK9BDFFC90D86C24B8
21:49:18,798 DEBUG SchemaExport:303 - alter table Protein drop constraint FK50CD6F63D86C24B8
21:49:18,834 DEBUG SchemaExport:303 - alter table interactions drop constraint FK4F6EF4A127BFBBC5
21:49:18,903 DEBUG SchemaExport:303 - alter table interactions drop constraint FK4F6EF4A1EBB9AEEF
21:49:19,029 DEBUG SchemaExport:303 - drop table Article
21:49:19,173 DEBUG SchemaExport:303 - drop table Complex
21:49:19,276 DEBUG SchemaExport:303 - drop table Interactor
21:49:19,410 DEBUG SchemaExport:303 - drop table Journal
21:49:19,537 DEBUG SchemaExport:303 - drop table Protein
21:49:19,654 DEBUG SchemaExport:303 - drop table interactions
21:49:19,814 DEBUG SchemaExport:303 - drop table hibernate_unique_key
21:49:19,896 DEBUG SchemaExport:303 - create table Article (pmid integer not null, title varchar(255) not null, doi varchar(255), yearDate integer not null, nlmId bigint not null, primary key (pmid))
21:49:20,063 DEBUG SchemaExport:303 - create table Complex (interactorId bigint not null, primary key (interactorId))
21:49:20,224 DEBUG SchemaExport:303 - create table Interactor (id bigint not null, name varchar(255) not null, primary key (id))
21:49:20,359 DEBUG SchemaExport:303 - create table Journal (nlmId bigint not null, title varchar(255) not null, primary key (nlmId))
21:49:20,580 DEBUG SchemaExport:303 - create table Protein (interactorId bigint not null, primary key (interactorId))
21:49:20,725 DEBUG SchemaExport:303 - create table interactions (complex_id bigint not null, interactor_id bigint not null, primary key (complex_id, interactor_id))
21:49:20,858 DEBUG SchemaExport:303 - alter table Article add constraint FK379164D684F84236 foreign key (nlmId) references Journal
21:49:20,997 DEBUG SchemaExport:303 - alter table Complex add constraint FK9BDFFC90D86C24B8 foreign key (interactorId) references Interactor
21:49:21,055 DEBUG SchemaExport:303 - alter table Protein add constraint FK50CD6F63D86C24B8 foreign key (interactorId) references Interactor
21:49:21,086 DEBUG SchemaExport:303 - alter table interactions add constraint FK4F6EF4A127BFBBC5 foreign key (interactor_id) references Interactor
21:49:21,202 DEBUG SchemaExport:303 - alter table interactions add constraint FK4F6EF4A1EBB9AEEF foreign key (complex_id) references Complex
21:49:21,331 DEBUG SchemaExport:303 - create table hibernate_unique_key ( next_hi integer )
21:49:21,376 DEBUG SchemaExport:303 - insert into hibernate_unique_key values ( 0 )
21:49:21,500 INFO SchemaExport:196 - schema export complete
21:49:21,501 WARN JDBCExceptionReporter:54 - SQL Warning: 10000, SQLState: 01J01
21:49:21,501 WARN JDBCExceptionReporter:55 - Database 'build/db/derby/hibernate' not created, connection made to existing database instead.
21:49:21,724 WARN JDBCExceptionReporter:54 - SQL Warning: 10000, SQLState: 01J01
21:49:21,724 WARN JDBCExceptionReporter:55 - Database 'build/db/derby/hibernate' not created, connection made to existing database instead.
Hibernate: insert into Journal (title, nlmId) values (?, ?)
Hibernate: insert into Article (title, doi, yearDate, nlmId, pmid) values (?, ?, ?, ?, ?)
Hibernate: insert into Article (title, doi, yearDate, nlmId, pmid) values (?, ?, ?, ?, ?)
Hibernate: insert into Interactor (name, id) values (?, ?)
Hibernate: insert into Protein (interactorId) values (?)
Hibernate: insert into Interactor (name, id) values (?, ?)
Hibernate: insert into Protein (interactorId) values (?)
Hibernate: insert into Interactor (name, id) values (?, ?)
Hibernate: insert into Protein (interactorId) values (?)
Hibernate: insert into Interactor (name, id) values (?, ?)
Hibernate: insert into Complex (interactorId) values (?)
Hibernate: insert into Interactor (name, id) values (?, ?)
Hibernate: insert into Complex (interactorId) values (?)
Hibernate: insert into interactions (complex_id, interactor_id) values (?, ?)
Hibernate: insert into interactions (complex_id, interactor_id) values (?, ?)
Hibernate: insert into interactions (complex_id, interactor_id) values (?, ?)
Hibernate: insert into interactions (complex_id, interactor_id) values (?, ?)
Hibernate: select journal0_.nlmId as nlmId1_, journal0_.title as title1_ from Journal journal0_
PNAS[1]
Hibernate: select article0_.pmid as pmid0_, article0_.title as title0_, article0_.doi as doi0_, article0_.yearDate as yearDate0_, article0_.nlmId as nlmId0_ from Article article0_
Hibernate: select journal0_.nlmId as nlmId1_0_, journal0_.title as title1_0_ from Journal journal0_ where journal0_.nlmId=?
(1988)"Article title 1".PNAS
(1989)"Article title 2".PNAS

Hibernate: select interactor0_.id as id2_, interactor0_.name as name2_, case when interactor0_1_.interactorId is not null then 1 when interactor0_2_.interactorId is not null then 2 when interactor0_.id is not null then 0 else -1 end as clazz_ from Interactor interactor0_ left outer join Protein interactor0_1_ on interactor0_.id=interactor0_1_.interactorId left outer join Complex interactor0_2_ on interactor0_.id=interactor0_2_.interactorId


### Protein:prot1




### Protein:prot2




### Protein:prot3


Hibernate: select partners0_.complex_id as complex1_1_, partners0_.interactor_id as interactor2_1_, interactor1_.id as id2_0_, interactor1_.name as name2_0_, case when interactor1_1_.interactorId is not null then 1 when interactor1_2_.interactorId is not null then 2 when interactor1_.id is not null then 0 else -1 end as clazz_0_ from interactions partners0_ left outer join Interactor interactor1_ on partners0_.interactor_id=interactor1_.id left outer join Protein interactor1_1_ on interactor1_.id=interactor1_1_.interactorId left outer join Complex interactor1_2_ on interactor1_.id=interactor1_2_.interactorId where partners0_.complex_id=?


### Complex:cplx1. ID:4 interacts with prot2 prot1


Hibernate: select partners0_.complex_id as complex1_1_, partners0_.interactor_id as interactor2_1_, interactor1_.id as id2_0_, interactor1_.name as name2_0_, case when interactor1_1_.interactorId is not null then 1 when interactor1_2_.interactorId is not null then 2 when interactor1_.id is not null then 0 else -1 end as clazz_0_ from interactions partners0_ left outer join Interactor interactor1_ on partners0_.interactor_id=interactor1_.id left outer join Protein interactor1_1_ on interactor1_.id=interactor1_1_.interactorId left outer join Complex interactor1_2_ on interactor1_.id=interactor1_2_.interactorId where partners0_.complex_id=?


### Complex:cplx2. ID:5 interacts with cplx1 prot1


21:49:22,076 INFO SessionFactoryImpl:769 - closing
21:49:22,076 INFO DriverManagerConnectionProvider:147 - cleaning up connection pool: jdbc:derby:build/db/derby/hibernate;create=true
Done


That's it!
Pierre