Object Oriented Programming with R: My notebook

In the following post, I describe how I've used the OOP features of R to create and use the following class hierarchy:

Your browser does not support the <CANVAS> element !

First the class Person is defined. It contains four fields : firstName, lastName, birthDate and birthPlace.

setClass("Person",
representation(
firstName="character",
lastName="character",
birthDate="Date",
birthPlace="character"
))

A kind of 'constructor' function can be called for Person to check that both firtsName and lastName are not empty:

setValidity("Person",
function(object)
{
length(object@firstName)>0 &&
length(object@lastName)>0
}
)

DeceasedPerson is a subClass of Person, it contains two more fields: deathPlace and deathDate:

setClass("DeceasedPerson",
representation(
deathDate="Date",
deathPlace="character"
),
contains="Person"
)

Scientist is another subClass of Person it contains one more field:'knownFor':

setClass("Scientist",
representation(
knownFor="character"
),
contains="Person"
)

Lastly, DeceasedScientist is a subClass of both Scientist and DeceasedPerson:

setClass("DeceasedScientist",
contains=c("Scientist","DeceasedPerson")
)

Let's define a 'generic' function 'age' returning the age of an individual from his 'birthdate':

age <- function(individual)
{
as.integer((Sys.Date()-individual@birthDate)/365)
}
setGeneric("age")

Polymorphism: for the DeceasedPerson another function will be used, it will calculate the age from both 'deathDate' and 'birthDate':

age.of.death <- function(individual)
{
as.integer((individual@deathDate-individual@birthDate)/365)
}
setMethod(age,signature=c("DeceasedPerson"),definition=age.of.death)

Ok, let's play with our class, we can first create a new instance of Scientist for Craig Venter:

craigVenter <-new(
"Scientist",
firstName="Craig",
lastName="Venter",
birthPlace="Salt Lake City",
birthDate=as.Date("1946-10-14", "%Y-%m-%d"),
knownFor=c("The Institute for Genomic Research","J. Craig Venter Institute")
)

... and Charles Darwin is a DeceasedScientist:

charlesDarwin <-new(
"DeceasedScientist",
firstName="Charles",
lastName="Darwin",
birthDate=as.Date("1809-02-12", "%Y-%m-%d"),
deathDate=as.Date("1882-04-19", "%Y-%m-%d"),
knownFor=c("Natural Selection","The Voyage of the Beagle")
)

Hey , we know where Charles was born!

charlesDarwin@birthPlace="Shrewsbury"

The following statement fails because the firstName is empty:

> try(new("Person",lastName="Darwin",birthDate=as.Date("1809-02-12", "%Y-%m-%d")),FALSE)
Error in validObject(.Object) : invalid class "Person" object: FALSE

Is Darwin a valid object?:

> validObject(charlesDarwin)
[1] TRUE

Print both individuals:

> charlesDarwin
An object of class “DeceasedScientist”
Slot "knownFor":
[1] "Natural Selection" "The Voyage of the Beagle"

Slot "firstName":
[1] "Charles"

Slot "lastName":
[1] "Darwin"

Slot "birthDate":
[1] "1809-02-12"

Slot "birthPlace":
[1] "Shrewsbury"

Slot "deathDate":
[1] "1882-04-19"

Slot "deathPlace":
character(0)

> craigVenter
An object of class “Scientist”
Slot "knownFor":
[1] "The Institute for Genomic Research" "J. Craig Venter Institute"

Slot "firstName":
[1] "Craig"

Slot "lastName":
[1] "Venter"

Slot "birthDate":
[1] "1946-10-14"

Slot "birthPlace":
[1] "Salt Lake City"

Let's use the 'is' operator:

> is(craigVenter,"Person")
[1] TRUE
> is(craigVenter,"DeceasedScientist")
[1] FALSE
> is(charlesDarwin,"DeceasedScientist")
[1] TRUE

Finally let's invoke the polymorhic function 'age' for both individuals:

> age(charlesDarwin)
[1] 73 #age.of.death was called
> age(craigVenter)
[1] 63 #generic "age'

Full source code

setClass("Person",
representation(
firstName="character",
lastName="character",
birthDate="Date",
birthPlace="character"
))

setValidity("Person",
function(object)
{
length(object@firstName)>0 &&
length(object@lastName)>0
}
)

setClass("DeceasedPerson",
representation(
deathDate="Date",
deathPlace="character"
),
contains="Person"
)

setClass("Scientist",
representation(
knownFor="character"
),
contains="Person"
)

age <- function(individual)
{
as.integer((Sys.Date()-individual@birthDate)/365)
}

setGeneric("age")

age.of.death <- function(individual)
{
as.integer((individual@deathDate-individual@birthDate)/365)
}


setClass("DeceasedScientist",
contains=c("Scientist","DeceasedPerson")
)

setMethod(age,signature=c("DeceasedPerson"),definition=age.of.death)

craigVenter <-new(
"Scientist",
firstName="Craig",
lastName="Venter",
birthPlace="Salt Lake City",
birthDate=as.Date("1946-10-14", "%Y-%m-%d"),
knownFor=c("The Institute for Genomic Research","J. Craig Venter Institute")
)

charlesDarwin <-new(
"DeceasedScientist",
firstName="Charles",
lastName="Darwin",
birthDate=as.Date("1809-02-12", "%Y-%m-%d"),
deathDate=as.Date("1882-04-19", "%Y-%m-%d"),
knownFor=c("Natural Selection","The Voyage of the Beagle")
)

try(new("Person",lastName="Darwin",birthDate=as.Date("1809-02-12", "%Y-%m-%d")),FALSE)

charlesDarwin@birthPlace="Shrewsbury"

validObject(charlesDarwin)

charlesDarwin
craigVenter


is(craigVenter,"Person")
is(craigVenter,"DeceasedScientist")
is(charlesDarwin,"DeceasedScientist")
age(charlesDarwin)
age(craigVenter)

That's it !
Pierre

Playing with Erlang (II)

transcripting a DNA to a RNA sequence with an anonymous function

lists:map(fun($T)->$U;(OTHER)->OTHER end,"AATAGCTGATCGACAATGTTAGCTAGGC").
>"AAUAGCUGAUCGACAAUGUUAGCUAGGC"

Testing if a symbol is an acid nucleic

> ACIDNUCLEICS="ATGCatgc".
"ATGCatgc"
> lists:member($A,ACIDNUCLEICS).
true
> lists:member($P,ACIDNUCLEICS).
false

Filtering a sequence with lists:filter

> IsAcidNucleic=fun(BASE)->lists:member(BASE,ACIDNUCLEICS) end.
#Fun<erl_eval.6.13229925>
> lists:filter( IsAcidNucleic, "1 ACGT ACGT ACGT ACGT ACGT ACGT 36").
"ACGTACGTACGTACGTACGTACGT"

Generating an array of numbers

> lists:seq(1,10).
[1,2,3,4,5,6,7,8,9,10]
> lists:seq(65,65+24).
"ABCDEFGHIJKLMNOPQRSTUVWXY"

The following notation says: add 32 to each item 'X' of the sequence 65 to 89.

> [X+32 || X<-lists:seq(65,65+24)].
"abcdefghijklmnopqrstuvwxy"

Create a list of structures {number,'X'} for each item 'X' of the sequence 1 to 5.

> [{number,X}|| X<-lists:seq(1,5)].
[{number,1},{number,2},{number,3},{number,4},{number,5}]

Create a list of structures {number,'X'} for each item 'X' of the sequence 1 to 100 where the modulus of X/15 is '0'.

> [{number,X}|| X<-lists:seq(1,100), X rem 15 =:= 0].
[{number,15},
{number,30},
{number,45},
{number,60},
{number,75},
{number,90}]

Create a list of all the pairs(x,y) for x and y between 1 and 4:

> [{pair,X,Y}|| X<-lists:seq(1,4),Y<-lists:seq(1,4)].
[{pair,1,1},
{pair,1,2},
{pair,1,3},
{pair,1,4},
{pair,2,1},
{pair,2,2},
{pair,2,3},
{pair,2,4},
{pair,3,1},
{pair,3,2},
{pair,3,3},
{pair,3,4},
{pair,4,1},
{pair,4,2},
{pair,4,3},
{pair,4,4}]

Create a list of all the pairs(x,y) for x and y between 1 and 4 having X==Y:

> [{pair,X,Y}|| X<-lists:seq(1,4),Y<-lists:seq(1,4),X=:=Y].
[{pair,1,1},{pair,2,2},{pair,3,3},{pair,4,4}]

A few is_XXXX functions

> is_integer("AAA").
false
> is_integer($A).
true
> is_list(1).
false
> is_list("AZAZAZ").
true
> is_atom(1).
false
> is_atom(hello).
true
> is_tuple(1).
false
> is_tuple({person,{name,"Pierre"}}).
true

A few functions:

% get the header of a list
> hd([100,99,98,97]).
100
% get the 3rd element of a list
> lists:nth(3,["EcoRI","HindIII","BamHI","PstI"]).
"BamHI"
>size({test,{name,"Hello"},{age,3}}).
3
> element(2,{test,{name,"Hello"},{age,3}}).
{name,"Hello"}
> abs(-99).
99

Translating a DNA to a protein.
The file bio.erl:

-module(bio).
-export([translate/1]).

%The standard genetic code table
geneticCode()->"FFLLSSSSYY**CC*WLLLLPPPPHHQQRRRRIIIMTTTTNNKKSSRRVVVVAAAADDEEGGGG".

%convert a base to an index in the table
base2index($T)-> 0;
base2index($C)-> 1;
base2index($A)-> 2;
base2index($G)-> 3.

%convert a codon to an index, lookup the table and return the amino acid
codon2AminoAcid(A1,A2,A3) -> lists:nth(
(base2index(A1)*16 + base2index(A2)*4 + base2index(A3))+1,
geneticCode())
.

%return an empty array if the argument is an empty sequence or a sequence with less than 3 bases
translate([])->[];
translate([_])->[];
translate([_,_])->[];
%translate the 3 first bases and append the remaining translation.
translate([A1,A2,A3|REMAIN])->[codon2AminoAcid(A1,A2,A3)|translate(REMAIN)].

Invoking bio:translate:

c(bio).

bio:translate("ATGGAGAGGCAGAAACGGAAGGCGGACATCGAGAAG").
"MERQKRKADIEK"

Records

A record file defines the structure of an object. Example: the following file snp.hrl defines the 'class' snp with its default values:

-record(snp,{
name="rs",
chrom="?",
chromStart=0,
chromEnd=0,
avHet= -1
}).

We can now use this class in an erlang shell:

%load the structure of a SNP
rr("snp.hrl").
[snp]
%create an empty instance of a snp
EMPTY_SNP= #snp{}.
#snp{name = "rs",chrom = "?",chromStart = 0,chromEnd = 0,
avHet = -1}
%create and fill an new instance of a snp
RS84=#snp{name="rs84",chrom="chr7",chromStart=25669317,chromEnd=25669318}.
#snp{name = "rs84",chrom = "chr7",chromStart = 25669317,
chromEnd = 25669318,avHet = -1}
%re-use the content of RS84 and fill the value of 'avHet'
RS84_2=RS84#snp{avHet=0.475045}.
#snp{name = "rs84",chrom = "chr7",chromStart = 25669317,
chromEnd = 25669318,avHet = 0.475045}
%extract the name and the avHet for RS84_2
#snp{name=NAME_RS84,avHet=AVHET_RS84}=RS84_2.
NAME_RS84.
"rs84"
AVHET_RS84.
0.475045

That's it
Pierre

Playing with Erlang (I)

I'm currently reading Joe Armstrong's "Programming Erlang". Here are a couple of notes about ERLANG.

Starting and stopping the Erlang shell

:~> erl
Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.2 (abort with ^G)
1> halt().
:~>

Simple Math

Input:

2*(3+4).
PI=3.14159.
R=2.
SURFACE=PI*R*R.
R=3.

Output:

1> 14
2> 3.14159
3> 2
4> 12.56636
##Variables in erlang are immutable R3 cannot be changed
5> ** exception error: no match of right hand side value 3

Two structure defining a SNP and a Gene are created with a Tuple:

RS94={snp,
{chrom,chr6},
{chromStart,98339675},
{chromEnd,98339676},
{name,"rs94"},
{strand,plus},
{func,unknown},
{avHet,0}
}.
RS47={snp,
{chrom,chr7},
{chromStart,11547645},
{chromEnd,11547646},
{name,"rs47"},
{strand,minus},
{func,missence},
{avHet,0.246}
}.
NP_056019={gene,
{chrom,chr7},
{txStart,11380695},
{txEnd,11838349},
{cdsStart,11381945},
{csdEnd,11838097},
{name,"NP_056019"},
{strand,minus},
{exonCount,27}
}.

Values are extracted using '_' as a placeholder for the unwanted variables.

{_,{_,_},{_,_},{_,_},{_,NAME_OF_RS94},{_,_},{_,_},{_,_}}=RS94.
NAME_OF_RS94.
"rs94"

Create a list of SNP:

LIST_OF_SNP1=[RS94,RS47].

Add P_056019 to this list and create a list of genomic objects:

LIST2=[NP_056019|LIST_OF_SNP1].

Extract the first and second element of LIST2, put the remaining list in LIST3:

[ITEM1,ITEM2|LIST3]=LIST2.
ITEM1.
{gene,{chrom,chr7},
{txStart,11380695},
{txEnd,11838349},
{cdsStart,11381945},
{csdEnd,11838097},
{name,"NP_056019"},
{strand,minus},
{exonCount,27}}
ITEM2.
{snp,{chrom,chr6},
{chromStart,98339675},
{chromEnd,98339676},
{name,"rs94"},
{strand,plus},
{func,unknown},
{avHet,0}}
LIST3.
[{snp,{chrom,chr7},
{chromStart,11547645},
{chromEnd,11547646},
{name,"rs47"},
{strand,minus},
{func,missence},
{avHet,0.246}}]

A String is just an Array of integer:

ALPHABET=[65,66,67,68,69,70].
"ABCDEF"

The dollar( $ ) notation is used to get the 'int' value of a 'char'.

HELLO=[$H,$e,$l,$l,$o].
"Hello"

Methods & Functions

A file named "bio.erl" is created. This file is a erlang module that contains a kind of polymorphic function distance returning the length of a given object (its number of arguments is '/1'). If this object is identified as a atom=snp the value "chromEnd-chromStart" is returned. Else, if atom=gene the value "txEnd-txStart" is returned.

-module(bio).
-export([distance/1]).
distance({snp,
{chrom,_},
{chromStart,START},
{chromEnd,END},
{name,_},
{strand,_},
{func,_},
{avHet,_}}
)-> END - START;
distance({gene,
{chrom,_},
{txStart,START},
{txEnd,END},
{cdsStart,_},
{csdEnd,_},
{name,_},
{strand,_},
{exonCount,_}
})-> END - START.

We now use this module:

c(bio).
RS94={snp,
{chrom,chr6},
{chromStart,98339675},
{chromEnd,98339676},
{name,"rs94"},
{strand,plus},
{func,unknown},
{avHet,0}
}.


RS47={snp,
{chrom,chr7},
{chromStart,11547645},
{chromEnd,11547646},
{name,"rs47"},
{strand,minus},
{func,missence},
{avHet,0.246}
}.

NP_056019={gene,
{chrom,chr7},
{txStart,11380695},
{txEnd,11838349},
{cdsStart,11381945},
{csdEnd,11838097},
{name,"NP_056019"},
{strand,minus},
{exonCount,27}
}.

bio:distance(RS94).
1
bio:distance(RS47).
1
bio:distance(NP_056019).
457654

We now want to calculate the GC percent of a DNA: the bio.erl file is modified as follow:

-module(bio).
-export([gcPercent/1]).
-export([distance/1]).
(...)

gc($A) -> 0.0;
gc($T) -> 0.0;
gc($C) -> 1.0;
gc($G) -> 1.0;
gc([])->0;
gc([BASE|REMAIN])->gc(BASE)+gc(REMAIN).

gcPercent(ADN)->100.0*(gc(ADN)/erlang:length(ADN)).

Here the method gc returns '1' or '0' if the argument is a base; returns 0 if the array is empty, or return the sum of the gc(first character of the string) plus the gc(remaining string). The method gcPercent divide the sum of gc by the length of the string and multiply it by 100.

c(bio).
bio:gcPercent("GCATG").
60.0

That's it.
Pierre

Hadoop, my notebook: HDFS

This post is about the Apache Hadoop, an open-source algorithm implementing the MapReduce algorithm. This first notebook focuses on HDFS, the Hadoop file system, and follows the great Yahoo! Hadoop Tutorial Home. Forget the clusters, I'm running this hadoop engine on my one and only laptop.

Downloading & Installing

~/tmp/HADOOP> wget "http://apache.multidist.com/hadoop/core/hadoop-0.19.1/hadoop-0.19.1.tar.gz
Saving to: `hadoop-0.19.1.tar.gz'

100%[======================================>] 55,745,146 487K/s in 1m 53s

2009-04-21 20:52:04 (480 KB/s) - `hadoop-0.19.1.tar.gz' saved [55745146/55745146]
~/tmp/HADOOP> tar xfz hadoop-0.19.1.tar.gz
~/tmp/HADOOP> rm hadoop-0.19.1.tar.gz
~/tmp/HADOOP> mkdir -p hdfs/data
~/tmp/HADOOP> mkdir -p hdfs/name
#hum... this step was not clear as I'm not a ssh guru. I had to give my root password to make the server starts
~/tmp/HADOOP> ssh-keygen -t rsa -P 'password' -f ~/.ssh/id_rsa
Generating public/private dsa key pair.
Your identification has been saved in /home/pierre/.ssh/id_rsa.
Your public key has been saved in /home/pierre/.ssh/id_rsa.pub.
The key fingerprint is:
17:c0:29:b4:56:d1:d3:dd:ae:d5:ba:3e:5b:33:b0:99 pierre@linux-zfgk
~/tmp/HADOOP> cat ~/.ssh/id_rsa.pub >> ~/.ssh/autorized_keys

Editing the Cluster configuration

Edit the file hadoop-0.19.1/conf/hadoop-site.xml.

<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
<description>This is the URI (protocol specifier, hostname, and port) that describes the NameNode (main Node) for the cluster.</description>
</property>
<property>
<name>dfs.data.dir</name>
<value>/home/pierre/tmp/HADOOP/hdfs/data</value>
<description>This is the path on the local file system in which the DataNode instance should store its data</description>
</property>
<property>
<name>dfs.name.dir</name>
<value>/home/pierre/tmp/HADOOP/hdfs/name</value>
<description>This is the path on the local file system of the NameNode instance where the NameNode metadata is stored.</description>
</property>
</configuration>

Formatting HDFS

HDFS the Hadoop Distributed File System "HDFS is a block-structured file system: individual files are broken into blocks of a fixed size. These blocks are stored across a cluster of one or more machines with data storage capacity. A file can be made of several blocks, and they are not necessarily stored on the same machine(...)If several machines must be involved in the serving of a file, then a file could be rendered unavailable by the loss of any one of those machines. HDFS combats this problem by replicating each block across a number of machines (3, by default)."

~/tmp/HADOOP> hadoop-0.19.1/bin/hadoop namenode -format
09/04/21 21:11:18 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = linux-zfgk.site/127.0.0.2
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 0.19.1
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.19 -r 745977; compiled by 'ndaley' on Fri Feb 20 00:16:34 UTC 2009
************************************************************/
Re-format filesystem in /home/pierre/tmp/HADOOP/hdfs/name ? (Y or N) Y
09/04/21 21:11:29 INFO namenode.FSNamesystem: fsOwner=pierre,users,dialout,video
09/04/21 21:11:29 INFO namenode.FSNamesystem: supergroup=supergroup
09/04/21 21:11:29 INFO namenode.FSNamesystem: isPermissionEnabled=true
09/04/21 21:11:29 INFO common.Storage: Image file of size 96 saved in 0 seconds.
09/04/21 21:11:29 INFO common.Storage: Storage directory /home/pierre/tmp/HADOOP/hdfs/name has been successfully formatted.
09/04/21 21:11:29 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at linux-zfgk.site/127.0.0.2
************************************************************/

Starting HDFS

~/tmp/HADOOP> hadoop-0.19.1/bin/start-dfs.sh
starting namenode, logging to /home/pierre/tmp/HADOOP/hadoop-0.19.1/bin/../logs/hadoop-pierre-namenode-linux-zfgk.out
Password:
localhost: starting datanode, logging to /home/pierre/tmp/HADOOP/hadoop-0.19.1/bin/../logs/hadoop-pierre-datanode-linux-zfgk.out
Password:
localhost: starting secondarynamenode, logging to /home/pierre/tmp/HADOOP/hadoop-0.19.1/bin/../logs/hadoop-pierre-secondarynamenode-linux-zfgk.out

Playing with HDFS

First Download a few SNP from UCSC/dbsnp into ~/local.xls.

~/tmp/HADOOP> mysql -N --user=genome --host=genome-mysql.cse.ucsc.edu -A -D hg18 -e 'select name,chrom,chromStart,avHet from snp129 where avHet!=0 and name like "rs12345%" ' > ~/local.xls

Creating directories

~/tmp/HADOOP> hadoop-0.19.1/bin/hadoop dfs -mkdir /user
~/tmp/HADOOP> hadoop-0.19.1/bin/hadoop dfs -mkdir /user/pierre

Copying a file "local.xls" from your local file system to HDFS

~/tmp/HADOOP> hadoop-0.19.1/bin/hadoop dfs -put ~/local.xls stored.xls

Recursive listing of HDFS

~/tmp/HADOOP> hadoop-0.19.1/bin/hadoop dfs -lsr /
drwxr-xr-x - pierre supergroup 0 2009-04-21 21:45 /user
drwxr-xr-x - pierre supergroup 0 2009-04-21 21:45 /user/pierre
-rw-r--r-- 3 pierre supergroup 308367 2009-04-21 21:45 /user/pierre/stored.xls

'cat' the first lines of the SNP file stored on HDFS:

~/tmp/HADOOP> hadoop-0.19.1/bin/hadoop dfs -cat /user/pierre/stored.xls | head
rs12345003 chr9 1765426 0.02375
rs12345004 chr9 2962430 0.055768
rs12345006 chr9 74304094 0.009615
rs12345007 chr9 73759324 0.112463
rs12345008 chr9 88421765 0.014184
rs12345013 chr9 78951530 0.104463
rs12345014 chr9 78542260 0.490608
rs12345015 chr9 10121973 0.201446
rs12345016 chr9 2698257 0.456279
rs12345027 chr9 8399632 0.04828

Removing a file. Note: "On startup, the NameNode enters a special state called Safemode." I could not delete a file before I used "dfsadmin -safemode leave".

~/tmp/HADOOP> hadoop-0.19.1/bin/hadoop dfsadmin -safemode leave
Safe mode is OFF
~/tmp/HADOOP> hadoop-0.19.1/bin/hadoop dfs -rm /user/pierre/stored.xls
Deleted hdfs://localhost:9000/user/pierre/stored.xls

Check there is NO file named stored.xls in the local file system !

~/tmp/HADOOP> find hdfs/
hdfs/
hdfs/data
hdfs/data/detach
hdfs/data/in_use.lock
hdfs/data/tmp
hdfs/data/current
hdfs/data/current/blk_3340572659657793789
hdfs/data/current/dncp_block_verification.log.curr
hdfs/data/current/blk_3340572659657793789_1002.meta
hdfs/data/current/VERSION
hdfs/data/storage
hdfs/name
hdfs/name/in_use.lock
hdfs/name/current
hdfs/name/current/edits
hdfs/name/current/VERSION
hdfs/name/current/fsimage
hdfs/name/current/fstime
hdfs/name/image
hdfs/name/image/fsimage

Stop HDFS

~/tmp/HADOOP> hadoop-0.19.1/bin/stop-dfs.sh
stopping namenode
Password:
localhost: stopping datanode
Password:
localhost: stopping secondarynamenode

Pierre

A Web Service for ONSolubility.

This post is about the ONSolubility project (For references search FriendFeed for Solubility). This post is about how I've used Egon's code to create a web service to query the data of solubility. Egon has already done a great job by using the google java spreasheet API to download Jean-Claude's Solubility data. On his side, Rajarshi Guha wrote an HTML page querying those data using the Google Query-API. Here I show how I have created a webservice searching for the measurements based on their solvent/solute/concentration.

Server Side

Classes

I've added some JAXB(Java Architecture for XML Binding) annotations to Egon's Measurement.java. Those annotations help the web-service compiler (wsgen) to understand how the data will be transmitted to the client.

@javax.xml.bind.annotation .XmlRootElement(name="Measurement")
public class Measurement
implements Serializable
{
(...)

Then we create the WebService ONService.java. This service is just a java class containing also a few annotations. First we flag the class as a webservice:

@javax.jws.WebService(
name="onsolubility",
serviceName="ons"
)
public class ONService
{

Then comes the function seach provided by this service. This function will download the data from google using Egon's API and will return a collection of Measurement based on their solute/solvent/concentration. Again the java annotations will help the compiler to implement the service

@WebMethod(action="urn:search",operationName="search")
public List search(
@WebParam(name="solute")String solute,
@WebParam(name="solvent")String solvent,
@WebParam(name="concMin")Double concMin,
@WebParam(name="concMax")Double concMax
) throws Exception
{....

. The web service is launched with only 3 lines of code (!).

ONService service=new ONService();
Endpoint endpoint = Endpoint.create(service);
endpoint.publish("http://localhost:8080/onsolubility");

Compilation

I've create a ant file invoking wsgen generating the stubs and installing the webservice. Here is the ouput

compile-webservice:
[javac] Compiling 1 source file to /home/pierre/tmp/onssolubility/ons.solubility.data/bin
[wsgen] command line: wsgen -classpath (...) -verbose ons.solubility.ws.ONService
[wsgen] Note: ap round: 1
[wsgen] [ProcessedMethods Class: ons.solubility.ws.ONService]
[wsgen] [should process method: search hasWebMethods: true ]
[wsgen] [endpointReferencesInterface: false]
[wsgen] [declaring class has WebSevice: true]
[wsgen] [returning: true]
[wsgen] [WrapperGen - method: search(java.lang.String,java.lang.String,java.lang.Double,java.lang.Double)]
[wsgen] [method.getDeclaringType(): ons.solubility.ws.ONService]
[wsgen] [requestWrapper: ons.solubility.ws.jaxws.Search]
[wsgen] [should process method: main hasWebMethods: true ]
[wsgen] [webMethod == null]
[wsgen] [ProcessedMethods Class: java.lang.Object]
[wsgen] ons/solubility/ws/jaxws/ExceptionBean.java
[wsgen] ons/solubility/ws/jaxws/Search.java
[wsgen] ons/solubility/ws/jaxws/SearchResponse.java
[wsgen] Note: ap round: 2

publish-webservice:
[java] Publishing Service on http://localhost:8080/onsolubility?WSDL

.
And... that's it. When I open my browser on http://localhost:8080/onsolubility?WSDL , I can now see the WSDL description/schema of this service.

Client Side

Writing a client using this api looks the same way I did for a previous post about the IntAct/EBI API where the wsimport command generated the stubs from the WSDL file. I then wrote a simple test ONServiceTest.java, invoking our service several times.

private void test(
String solute,
String solvent,
Double concMin,
Double concMax)
{
try
{
Ons service=new Ons();
Onsolubility port=service.getOnsolubilityPort();
List data=port.search(solute, solvent, concMin, concMax);
for(Measurement measure:data)
{
System.out.println(
" sample :\t"+measure.getSample()+"\n"+
" solute :\t"+measure.getSolute()+"\n"+
" solvent :\t"+measure.getSolvent()+"\n"+
" experiment:\t"+measure.getExperiment()+"\n"+
" reference :\t"+measure.getReference()+"\n"+
" conc :\t"+measure.getConcentration()+"\n"
);
}
} catch(Throwable err)

{
System.err.println("#error:"+err.getMessage());
}
}
private void test()
{
test(null,null,null,null);
test("4-nitrobenzaldehyde",null,null,null);
test("4-nitrobenzaldehyde",null,0.3,0.4);
}

Here is the output

ant test-webservice
Buildfile: build.xml
test-webservice
[wsimport] parsing WSDL...
[wsimport] generating code...
[javac] Compiling 1 source file to onssolubility/ons.solubility.data/bin
[java] ##Searching solute: null solvent: null conc: null-null
[java] sample : 9
[java] solute : D-Glucose
[java] solvent : THF
[java] experiment: 1
[java] reference : http://onschallenge.wikispaces.com/JennyHale-1
[java] conc : 0.00222
[java]
[java] sample : 6
[java] solute : D-Mannitol
[java] solvent : Methanol
[java] experiment: 1
[java] reference : http://onschallenge.wikispaces.com/JennyHale-1
[java] conc : 0.00548
[java]
(...)
[java]
[java] sample : 10
[java] solute : D-Mannitol
[java] solvent : THF
[java] experiment: 1
[java] reference : http://onschallenge.wikispaces.com/JennyHale-1
[java] conc : 0.01098
[java] ##Searching solute: 4-nitrobenzaldehyde solvent: null conc: 0.3-0.4
[java] sample : 2b
[java] solute : 4-nitrobenzaldehyde
[java] solvent : Methanol
[java] experiment: 212
[java] reference : http://usefulchem.wikispaces.com/exp212
[java] conc : 0.38

That's it and that's enough code for the week-end.

Pierre

Parsing JSON with javacc my Notebook.

Although I didn't wrote a new great programming language, I had a little experience with C lexers/parsers especialy with lex/yacc|flex/bison , a LALR parser. Now I'm programming in Java, I've been looking for the parsers available for this language: the most popular tools seems to be the top-down parsers javacc and antlr. In this post I show how I wrote a simple javacc parser reading a JSON entry (this is just an exercice, it is also easy to write this kind of parser using java.io.StreamTokenizer or java.util.Scanner).
First of all I found that the documentation was very limited and unlike Bison, I had the feeling that the javacc tutorial was a kind of "Look a the examples, isn't it cool ?"? I just writing my notebook here, so on my side I won't explain how I think I've understand how javacc works :-).
OK now, let's go back, to our JSON grammar and to the content of the javacc file.

The file is called JSONHandler.jj, it contains a java class JSONHandler with a main calling a method object after reading the input from stdin. This method will parse the json stream , transform it into a java object and echo it on stderr. The class is delimited by the keywords PARSER_BEGIN and PARSER_END

PARSER_BEGIN(JSONHandler)
public class JSONHandler {
public static void main(String args[])
{
try
{
JSONHandler parser = new JSONHandler(System.in);
Object o=parser.object();
System.err.println("JAVA OBJECT: "+o);
} catch(Exception err)
{
err.printStackTrace();
}
}
}
PARSER_END(JSONHandler)

Next we declare that the blank characters will be ignored:

SKIP :
{
" "
| "\t"
| "\n"
| "\r"
}

We then declare the lexical tokens (numbers, string, quoted strings) using a BNF grammar. As an example, here, a SIMPLE_QUOTE_LITERAL starts and ends with "\'" , it contains an unlimited number of ("escaped C special characters" or "normal characters").

TOKEN : /* LITERALS */
{
<#LETTER: ["_","a"-"z","A"-"Z"] >
| <#DIGIT: ["0"-"9"] >
| <#SIGN: ["-","+"]>
| <#EXPONENT: ("E"|"e") (<SIGN>)? (<DIGIT>)+
>
| <FLOATING_NUMBER: (<DIGIT>)* "." (<DIGIT>)* (<EXPO
NENT>)?
| (<DIGIT>)+ (<EXPONENT>) >
| <INT_NUMBER: (<DIGIT>)+ >
| <IDENTIFIER: <LETTER> (<LETTER>|<DIGIT>|<->)* >
| <#ESCAPE_CHAR: "\\" ["n","t","b","r","f","\\","'","\""]
>
| <SIMPLE_QUOTE_LITERAL:
"\'"
( (~["\'","\\","\n","\r"])
| <ESCAPE_CHAR>
)*
"\'"
>
|
<DOUBLE_QUOTE_LITERAL:
"\""
( (~["\"","\\","\n","\r"])
| <ESCAPE_CHAR>
)*
"\""
>
}

As we saw, the parser starts is job by invoking the object method. We expect here a JSON array or a JSON object or another identifier (null ||false||true||string). Those choices store their results in the variable o. This method returns a java.lang.Object.

public Object object():
{Object o;}
{
(
o=array()
| o= map()
| o= identifier()
)
{return o;}
}

JSON arrays will be returned as java.util.Vector<Object>. A JSON array is identified as starting by "[" and ending with "]", it contains zero or more JSON objects separated with a comma. Each time an element of this array is found, it is added in the vector. at the end, the vector is returned as the value of this array.

public Vector<Object> array():
{Vector<Object> vector= new Vector<Object>(); Object o;}
{
"[" ( o= object() {vector.addElement(o);} ("," o=object() {vector.addElement(o);} ) * )? "]"
{
return vector;
}
}

A JSON identifier can be a number, a string, a quoted string (need to be un-escaped), null, true or false. The content of this lexical token is obtained via the Token object which class is generated by javacc.

public Object identifier():
{Token t;}
{
(
t=<FLOATING_NUMBER>
{
return new Double(t.image);
}
| t=<INT_NUMBER>
{
return new Long(t.image);
}
| t=<IDENTIFIER>
{
if(t.image.equals("true"))
{
return Boolean.TRUE;
}
else if(t.image.equals("false"))
{
return Boolean.FALSE;
}
else if(t.image.equals("null"))
{
return null;
}
return t.image;
}
| t=<SIMPLE_QUOTE_LITERAL>
{
return unescape(t.image);
}
| t=<DOUBLE_QUOTE_LITERAL>
{
return unescape(t.image);
}
)
}

JSON Object will be returned as java.util.HashMap. A JSON Object starts with '{' and ends with '}'. In this case, we're passing this map as an argument each time the parser finds a pair of key/value.

public HashMap<String,Object> map():
{HashMap<String,Object> map= new HashMap<String,Object>(); }
{
"{" ( keyValue(map) ("," keyValue(map))*)? "}"
{
return map;
}
}

public void keyValue( HashMap<String,Object> map):
{Object k; Object v;}
{
(k=identifier() ":" v=object())
{
if(k==null) throw new ParseException("null cannot be used as key in object");
if(!(k instanceof String)) throw new ParseException(k.toString()+"("+k.getClass()+") cannot
be used as key in object");
map.put(k.toString(),v);
}
}

Compilation:

javacc JSONHandler.jj
Java Compiler Compiler Version 4.0 (Parser Generator)
(type "javacc" with no arguments for help)
Reading from file JSONHandler.jj . . .
Parser generated successfully.
javac JSONHandler.java

Testing:
Here is the content of test.json

{
organisms:[
{
id:10929,
name:"Bovine Rotavirus"
},
{
id:9606,
name:"Homo Sapiens"
}
],
proteins:[
{
label:"NSP3",
description:"Rotavirus Non Structural Protein 3",
organism-id: 10929,
acc: "ACB38353"
},
{
label:"EIF4G",
description:"eukaryotic translation initiation factor 4 gamma",
organism-id: 9606,
acc:"AAI40897"
}
],
interactions:[
{
label:"NSP3 interacts with EIF4G1",
pubmed-id:[77120248,38201627],
proteins:["ACB38353","AAI40897"]
}
]
}

This file is parsed, a *java* object is returned and printed on screen.

java JSONHandler < test.json
JAVA OBJECT: {organisms=[{id=10929, name=Bovine Rotavirus}, {id=9606, name=Homo Sapiens}],
proteins=[{description=Rotavirus Non Structural Protein 3, organism-id=10929, label=NSP3,
acc=ACB38353}, {description=eukaryotic translation initiation factor 4 gamma,
organism-id=9606, label=EIF4G, acc=AAI40897}], interactions=[{pubmed-id=[77120248, 38201627],
label=NSP3 interacts with EIF4G1, proteins=[ACB38353, AAI40897]}]}

Here is the complete javacc source code:

PARSER_BEGIN(JSONHandler)
import java.util.Vector;
import java.util.HashMap;

public class JSONHandler {



public static void main(String args[])
{
try
{
JSONHandler parser = new JSONHandler(System.in);
Object o=parser.object();
System.err.println("JAVA OBJECT: "+o);
} catch(Exception err)
{
err.printStackTrace();
}
}

/** unescape a C string */
private static String unescape(String s)
{
StringBuilder sb= new StringBuilder(s.length());
for(int i=1;i< s.length()-1;++i)
{
if(s.charAt(i)=='\\')
{
if(i+1< s.length()-1)
{
++i;
switch(s.charAt(i))
{
case '\n': sb.append('\n'); break;
case '\r': sb.append('\r'); break;
case '\\': sb.append('\\'); break;
case 'b': sb.append('\b'); break;
case 't': sb.append('\t'); break;
case 'f': sb.append('\f'); break;
case '\'': sb.append('\''); break;
case '\"': sb.append('\"'); break;
default: sb.append(s.charAt(i));
}
}
}
else
{
sb.append(s.charAt(i));
}
}
return sb.toString();
}

}

PARSER_END(JSONHandler)

SKIP :
{
" "
| "\t"
| "\n"
| "\r"
}


TOKEN : /* LITERALS */
{
<#LETTER: ["_","a"-"z","A"-"Z"] >
| <#DIGIT: ["0"-"9"] >
| <#SIGN: ["-","+"]>
| <#EXPONENT: ("E"|"e") (<SIGN>)? (<DIGIT>)+ >
| <FLOATING_NUMBER: (<DIGIT>)* "." (<DIGIT>)* (<EXPONENT>)?
| (<DIGIT>)+ (<EXPONENT>) >
| <INT_NUMBER: (<DIGIT>)+ >
| <IDENTIFIER: <LETTER> (<LETTER>|<DIGIT>|"-")* >
| <#ESCAPE_CHAR: "\\" ["n","t","b","r","f","\\","'","\""] >
| <SIMPLE_QUOTE_LITERAL:
"\'"
( (~["\'","\\","\n","\r"])
| <ESCAPE_CHAR>
)*
"\'"
>
|
<DOUBLE_QUOTE_LITERAL:
"\""
( (~["\"","\\","\n","\r"])
| <ESCAPE_CHAR>
)*
"\""
>
}



public Object object():
{Object o;}
{
(
o=array()
| o= map()
| o= identifier()
)
{return o;}
}

public Object identifier():
{Token t;}
{
(
t=<FLOATING_NUMBER>
{
return new Double(t.image);
}
| t=<INT_NUMBER>
{
return new Long(t.image);
}
| t=<IDENTIFIER>
{
if(t.image.equals("true"))
{
return Boolean.TRUE;
}
else if(t.image.equals("false"))
{
return Boolean.FALSE;
}
else if(t.image.equals("null"))
{
return null;
}
return t.image;
}
| t=<SIMPLE_QUOTE_LITERAL>
{
return unescape(t.image);
}
| t=<DOUBLE_QUOTE_LITERAL>
{
return unescape(t.image);
}
)
}

public Vector<Object> array():
{Vector<Object> vector= new Vector<Object>(); Object o;}
{
"[" ( o=object() {vector.addElement(o);} ("," o=object() {vector.addElement(o);} ) * )? "]"
{
return vector;
}
}

public HashMap<String,Object> map():
{HashMap<String,Object> map= new HashMap<String,Object>(); }
{
"{" ( keyValue(map) ("," keyValue(map))*)? "}"
{
return map;
}
}

public void keyValue( HashMap<String,Object> map):
{Object k; Object v;}
{
(k=identifier() ":" v=object())
{
if(k==null) throw new ParseException("null cannot be used as key in object");
if(!(k instanceof String)) throw new ParseException(k.toString()+"("+k.getClass()+") cannot be used as key in object");
map.put(k.toString(),v);
}
}

That's it. What's next ? jjtree is another component of the javacc package and seems to be a promising tool: it builds a tree structure from the grammar. The nodes of this tree can then be visited just like in a DOM/XML document and a language can be implemented, but here again, the documentation is succinct.

JAVA Native Interface (JNI): notebook

The JAVA Native Interface (JNI)allows Java code running in the Java virtual machine to call and (be called) some methods from libraries written in C/C++. I've learned this technology by trying to use the CURSES library (an API allowing the programmer to write text user interfaces in a terminal).

We starts by creating a java class: NCurses.java

1 /* A JNI test */
2 public class NCurses
3 {
4 public NCurses() {}
5 /** initializes the ncurses library */
6 public native static int install();
7 /** disposes the ncurses library */
8 public native static int uninstall();
9 /** set character on screen */
10 public native static void setCharAt(int x,int y,int ch);
11 /** update the screen */
12 public native static void refresh();
13 /** get the number of rows */
14 public native int getRowCount();
15 /** get the number of columns */
16 public native int getColumnCount();
17 /** inverse video */
18 public native int standout();
19 /** normal video */
20 public native int standend();
21
22 /** draw a string at the given position */
23 public void drawString(String s,int x, int y)
24 {
25 int maxx= getColumnCount();
26 for(int i=0;i< s.length() && i+x < maxx;++i)
27 {
28 if(i+3< s.length() && s.substring(i,i+3).equals("<b>"))
29 {
30 standout(); i+=2; continue;
31 }
32 else if(i+4< s.length() && s.substring(i,i+4).equals("</b>"))
33 {
34 standend(); i+=3; continue;
35 }
36 setCharAt(i+x,y,s.charAt(i));
37 }
38 }
39
40 public static void main(String[] args)
41 {
42 //load the C library
43 System.loadLibrary("NCurses");
44 //init curses
45 NCurses.install();
46
47 NCurses sample = new NCurses();
48
49 //draw a box
50 for(int i=1;i+1< sample.getRowCount();++i)
51 {
52 sample.setCharAt(1,i,'*');
53 sample.setCharAt(sample.getColumnCount()-2,i,'*');
54 }
55 for(int i=1;i+1< sample.getColumnCount();++i)
56 {
57 sample.setCharAt(i,1,'*');
58 sample.setCharAt(i,sample.getRowCount()-2,'*');
59 }
60 /* draw some strings */
61 int n=0;
62 for(int i=2;i+3< sample.getRowCount();i+=3)
63 {
64 sample.drawString("<b>Menu \t"+(++n)+"</b>: Hello Menu",5,i);
65 }
66 /* update the screen */
67 sample.refresh();
68 /* wait 10 sec */
69 try{ Thread.sleep(10000); } catch(Exception err) {}
70 //dispose ncurses
71 NCurses.uninstall();
72 }
73 }

This class contains some native methods that are not implemented in this class but they will be loading (line 43 by calling System.loadLibrary)

We can compile this class:

javac NCurses.java

We can run JAVAH on this class. javah will generate the C header file that are needed to implement our native methods.

javah NCurses

This generates the following file: NCurses.h

1 /* DO NOT EDIT THIS FILE - it is machine generated */
2 #include <jni.h>
3 /* Header for class NCurses */
4
5 #ifndef _Included_NCurses
6 #define _Included_NCurses
7 #ifdef __cplusplus
8 extern "C" {
9 #endif
10 /*
11 * Class: NCurses
12 * Method: install
13 * Signature: ()I
14 */
15 JNIEXPORT jint JNICALL Java_NCurses_install
16 (JNIEnv *, jclass);
17
18 /*
19 * Class: NCurses
20 * Method: uninstall
21 * Signature: ()I
22 */
23 JNIEXPORT jint JNICALL Java_NCurses_uninstall
24 (JNIEnv *, jclass);
25
26 /*
27 * Class: NCurses
28 * Method: setCharAt
29 * Signature: (III)V
30 */
31 JNIEXPORT void JNICALL Java_NCurses_setCharAt
32 (JNIEnv *, jclass, jint, jint, jint);
33
34 /*
35 * Class: NCurses
36 * Method: refresh
37 * Signature: ()V
38 */
39 JNIEXPORT void JNICALL Java_NCurses_refresh
40 (JNIEnv *, jclass);
41
42 /*
43 * Class: NCurses
44 * Method: getRowCount
45 * Signature: ()I
46 */
47 JNIEXPORT jint JNICALL Java_NCurses_getRowCount
48 (JNIEnv *, jobject);
49
50 /*
51 * Class: NCurses
52 * Method: getColumnCount
53 * Signature: ()I
54 */
55 JNIEXPORT jint JNICALL Java_NCurses_getColumnCount
56 (JNIEnv *, jobject);
57
58 /*
59 * Class: NCurses
60 * Method: standout
61 * Signature: ()I
62 */
63 JNIEXPORT jint JNICALL Java_NCurses_standout
64 (JNIEnv *, jobject);
65
66 /*
67 * Class: NCurses
68 * Method: standend
69 * Signature: ()I
70 */
71 JNIEXPORT jint JNICALL Java_NCurses_standend
72 (JNIEnv *, jobject);
73
74 #ifdef __cplusplus
75 }
76 #endif
77 #endif

I've implemented the C methods in the following file: NCurses.c

1 #include "NCurses.h"
2 #include <string.h>
3 #include <curses.h>
4 #include <signal.h>
5
6 JNIEXPORT jint JNICALL Java_NCurses_install(JNIEnv *env, jclass clazz)
7 {
8 //(void) signal(SIGINT, finish); /* arrange interrupts to terminate */
9
10 (void) initscr(); /* initialize the curses library */
11 keypad(stdscr, TRUE); /* enable keyboard mapping */
12 (void) nonl(); /* tell curses not to do NL->CR/NL on output */
13 (void) cbreak(); /* take input chars one at a time, no wait for \n */
14 (void) noecho(); /* don't echo input */
15 clear();
16 move(0,0);
17 refresh();
18 return 0;
19 }
20
21 JNIEXPORT jint JNICALL Java_NCurses_uninstall(JNIEnv *env, jclass clazz)
22 {
23 endwin();
24 return 0;
25 }
26
27 JNIEXPORT jint JNICALL Java_NCurses_getRowCount(JNIEnv *env, jobject obj)
28 {
29 int h, w;
30 getmaxyx(stdscr, h, w);
31 return h;
32 }
33
34
35 JNIEXPORT jint JNICALL Java_NCurses_getColumnCount(JNIEnv *env, jobject obj)
36 {
37 int h, w;
38 getmaxyx(stdscr, h, w);
39 return w;
40 }
41
42 /*
43 * Class: NCurses
44 * Method: setCharAt
45 * Signature: (III)V
46 */
47 JNIEXPORT void JNICALL Java_NCurses_setCharAt
48 (JNIEnv *env, jclass object, jint x, jint y, jint chr)
49 {
50 mvaddch(y,x,chr);
51 }
52
53 /*
54 * Class: NCurses
55 * Method: refresh
56 * Signature: ()V
57 */
58 JNIEXPORT void JNICALL Java_NCurses_refresh
59 (JNIEnv *env, jclass object)
60 {
61 refresh();
62 }
63
64
65 /*
66 * Class: NCurses
67 * Method: standout
68 * Signature: ()I
69 */
70 JNIEXPORT jint JNICALL Java_NCurses_standout
71 (JNIEnv *env, jobject object)
72 {
73 standout();
74 }
75
76 /*
77 * Class: NCurses
78 * Method: standend
79 * Signature: ()I
80 */
81 JNIEXPORT jint JNICALL Java_NCurses_standend
82 (JNIEnv * env, jobject object)
83 {
84 standend();
85 }

before compiling the C source, I set the variable LD_LIBRARY_PATH. For my machine it was:

export LD_LIBRARY_PATH=${PWD}:/usr/local/lib:/usr/lib

We can now compile NCurses.c and create the C library. You may need to add the path the java C headers which are contained in ${JAVA_HOME}/include/ and ${JAVA_HOME}/include/{your-machine}

gcc -fpic -c NCurses.c -o NCurses.o
gcc -shared -o libNCurses.so NCurses.o -lcurses

Let's run our java program:

java NCurses

The result looks like this:

 *********************************************
 *      Menu  1    : Hello Menu              *
 *                                           *
 *                                           *
 *      Menu  2    : Hello Menu              *
 *                                           *
 *                                           *
 *      Menu  3    : Hello Menu              *
 *                                           *
 *                                           *
 *      Menu  4    : Hello Menu              *
 *                                           *
 *                                           *
 *      Menu  5    : Hello Menu              *
 *                                           *
 *                                           *
 *      Menu  6    : Hello Menu              *
 *                                           *
 *                                           *
 *                                           *
 *********************************************

That's it.
Pierre

My fNotebook: Apache Tomcat / Bioinformatics

Hi all,
here is how I installed created and installed today a web application based on JSP (Java Server Page) and running on tomcat.

A prior knowledge on how deploying a web application with tomcat is required so this post is more a notebook than a tutorial.

First download tomcat 6.0, extract it:

wget -q "http://apache.cict.fr/tomcat/tomcat-6/v6.0.14/bin/apache-tomcat-6.0.14.tar.gz"
tar xfz apache-tomcat-6.0.14.tar.gz

Fetch the mysql java connector, extract it, and move in into the tomcat 'lib' folder

wget -q "ftp://ftp.inria.fr/pub/MySQL/Downloads/Connector-J/mysql-connector-java-5.1.5.tar.gz"
tar xfz mysql-connector-java-5.1.5.tar.gz
mv mysql-connector-java-5.1.5/mysql-connector-java-5.1.5-bin.jar apache-tomcat-6.0.14/lib/

I need the java standard template library JSTL library. I fetch and extract it.

wget "http://people.apache.org/builds/jakarta-taglibs/nightly/projects/standard/jakarta-taglibs-standard-20060823.tar.gz"
tar xfz jakarta-taglibs-standard-20060823.tar.gz

I create a database of snp.

mysql -u root -p -D test -e 'create table snp(chrom varchar(10) ,chromStart int not null,chromEnd int not null,name varchar(20) unique not null)'

I fill this database with a few snp from dbsnp@ucsc

mysql -N --user=genome --host=genome-mysql.cse.ucsc.edu -A -D hg18 -e 'select chrom,chromStart,chromEnd,name from snp126 where chrom="chrM" ' |\
gawk -F ' ' '{printf("insert into test.snp(chrom,chromStart,chromEnd,name) values (\"%s\",%s,%s,\"%s\");\n",$1,$2,$3,$4);}' |\
mysql -u login -p

I add a mysql connection pool in apache. In apache-tomcat-6.0.14/conf/context.xml , I add the following code just before the last tag </Context>

<Resource name="jdbc/MYSQL" auth="Container" type="javax.sql.DataSource"
maxActive="100" maxIdle="30" maxWait="10000"
username="login" password="yourpassword" driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/test?autoReconnect=true"/>

we also need to setup a few properties before running tomcat:

export JAVA_HOME /usr/your-path/java1.6
export CATALINA_HOME=${PWD}/apache-tomcat-6.0.14
export CATALINA_BASE=${PWD}/apache-tomcat-6.0.14

we can now run tomcat.

./apache-tomcat-6.0.14/bin/startup.sh
Using CATALINA_BASE: /home/pierre/tmp/TOMCAT/apache-tomcat-6.0.14
Using CATALINA_HOME: /home/pierre/tmp/TOMCAT/apache-tomcat-6.0.14
Using CATALINA_TMPDIR: /home/pierre/tmp/TOMCAT/apache-tomcat-6.0.14/temp
Using JRE_HOME: /usr/your-path/java1.6/jre

we then create a few new directories

mkdir -p ./src/jsp
mkdir -p ./src/org/lindenb/jsp

We create a first JSP Custom TAG in src/org/lindenb/jsp/Anchor2DbSNP.java. This custom JSP tag will be used to create a automatic anchor to dbSNP.

package org.lindenb.jsp;
import javax.servlet.jsp.*;
import javax.servlet.jsp.tagext.*;
import java.util.regex.*;

/**
* This is a simple printing a link to dbSNP.
*/
public class Anchor2DbSNP extends BodyTagSupport
{
static private final Pattern RS_PATTERN=Pattern.compile("rs[0-9]+");

public int doEndTag() throws JspException
{
try
{
BodyContent bodyContent= getBodyContent();
if(bodyContent==null) return EVAL_PAGE;
String input=bodyContent.getString().trim().toLowerCase();
if(RS_PATTERN.matcher(input).matches())
{
getPreviousOut().print(
"<a href='http://www.ncbi.nlm.nih.gov/SNP/snp_ref.cgi?rs="+
input.substring(2)+
"'>"+
input+
"</a>"
);
}
else
{
getPreviousOut().print(input);
}
} catch(java.io.IOException err)
{
throw new JspException(err);
}
return EVAL_PAGE;
}
}

Another two custom tags will be used to display a simple genomic map in SVG.

Here is src/org/lindenb/jsp/ChromosomeTag.java

package org.lindenb.jsp;
import javax.servlet.jsp.*;
import javax.servlet.jsp.tagext.*;
import java.util.regex.*;
import java.util.*;

public class ChromosomeTag extends BodyTagSupport
{
private static class Position
{
int position=0;
String name=null;
public Position(int position,String name)
{
this.position=position;
this.name=name;
}
}

private Vector<Position> items= null;
private int svgWidth=500;
private int itemHeight=20;


public int doStartTag() throws JspException
{
items= new Vector<Position>();
return EVAL_BODY_INCLUDE;
}

public void addPosition(int position,String name)
{
if(position<0 || name==null) return;
this.items.addElement(new Position(position,name));
}

public int doEndTag() throws JspException
{
//if(this.items.isEmpty()) return EVAL_PAGE;
int max=0;
int min=Integer.MAX_VALUE;
for(Position p:this.items)
{
max=Math.max(p.position,max);
min=Math.min(p.position,min);
}
try
{
JspWriter out= pageContext.getOut();
out.write("<svg xmlns:xlink='http://www.w3.org/1999/xlink' xmlns='http://www.w3.org/2000/svg' width='"+svgWidth+"' height='"+ (this.items.size()*itemHeight)+"' style='font-size:"+(itemHeight-10)+"pt;stroke-width:1;'>");
out.write("<rect x='0' y='0' width='"+svgWidth+"' height='"+ (this.items.size()*itemHeight)+"' style='fill:white; stroke:gray;'/>");
int y=0;
for(Position p:this.items)
{
int x= (int)(((p.position-min)/(float)(max-min))*(svgWidth-200))+100;
out.write("<line x1='"+x+"' y1='"+y+"' x2='"+x+"' y2='"+(y+itemHeight)+"' style='stroke:blue;'/>");

out.write("<line x1='0' y1='"+y+"' x2='"+svgWidth+"' y2='"+(y)+"' style='stroke:gray;'/>");

out.write("<text x='"+(x+4)+"' y='"+(y+5+itemHeight/2)+"' >"+p.name+"</text>");
y+=itemHeight;
}

out.write("</svg>");

}
catch(java.io.IOException err)
{
throw new JspException(err);
}
items=null;
return EVAL_PAGE;
}

public void release()
{
items=null;
}
}

and

src/org/lindenb/jsp/ChromItemTag.java

package org.lindenb.jsp;
import javax.servlet.jsp.*;
import javax.servlet.jsp.tagext.*;
import java.util.regex.*;


public class ChromItemTag extends BodyTagSupport
{
private int position=-1;
private String name="";

public void setPosition(int position) { this.position= position;}


public int doEndTag() throws JspException
{
BodyContent bodyContent= getBodyContent();
if(bodyContent!=null) this.name =bodyContent.getString().trim();
if(this.name==null || name.length()==0) this.name=String.valueOf(this.position);
Tag parent= findAncestorWithClass(this,ChromosomeTag.class);
if(parent==null) return EVAL_PAGE;
ChromosomeTag ct= ChromosomeTag.class.cast(parent);
ct.addPosition(this.position+20,this.name);
return EVAL_PAGE;
}

public void release()
{
name=null;
position=-1;
}
}

the file src/bio.tld is the file used to declare the three custom tags.

<?xml version="1.0" encoding="ISO-8859-1" ?>
<taglib xmlns="http://java.sun.com/xml/ns/j2ee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-jsptaglibrary_2_0.xsd"
version="2.0">

<description>Bioinfo JSP TAG library</description>
<display-name>Bioinfo</display-name>
<tlib-version>1.1</tlib-version>
<short-name>bio</short-name>
<uri>http://jsp.lindenb.org</uri>

<tag>
<name>rs</name>
<tag-class>org.lindenb.jsp.Anchor2DbSNP</tag-class>
<body-content>JSP</body-content>
<info>display a link to dbSNP</info>
</tag>

<tag>
<name>chrom</name>
<tag-class>org.lindenb.jsp.ChromosomeTag</tag-class>
<body-content>JSP</body-content>
<info>svg map</info>
</tag>

<tag>
<name>item</name>
<tag-class>org.lindenb.jsp.ChromItemTag</tag-class>
<body-content>JSP</body-content>
<info>svg item</info>
<attribute>
<name>position</name>
<required>true</required>
<rtexprvalue>true</rtexprvalue>
</attribute>
</tag>


</taglib>

the file cat src/jsp/page.jsp is our JSP. It displays a SVG map and a table of a few SNP. It uses the JSTL and our custom tags.

<jsp:root
xmlns:jsp="http://java.sun.com/JSP/Page"
xmlns:c="http://java.sun.com/jsp/jstl/core"
xmlns:sql="http://java.sun.com/jsp/jstl/sql"
xmlns:bio="http://jsp.lindenb.org"

version="2.0">
<jsp:directive.page contentType="text/xml; charset=iso-8859-1"/>
<jsp:output doctype-root-element="html"
doctype-public="-//W3C//DTD XHTML 1.0 Strict//EN"
doctype-system="http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"
omit-xml-declaration="true"
/>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>JSP Tutorial For Bioinformatics</title>
<!-- <meta http-equiv="Content-Type" content="application/xhtml+xml; charset=iso-8859-1" /> -->
</head>
<body>
<sql:query var="snps" dataSource="jdbc/MYSQL">select * from snp limit 10</sql:query>
<bio:chrom>
<c:forEach var="row" items="${snps.rows}">

<bio:item position="${row.chromStart}"><c:out value="${row.name}"/></bio:item>
</c:forEach>
</bio:chrom>

<sql:query var="snps" dataSource="jdbc/MYSQL">select * from snp limit 10</sql:query>
<table>
<tr><th>Position</th><th>Name</th></tr>
<c:forEach var="row" items="${snps.rows}">
<tr>
<td><c:out value="${row.chrom}"/>:<c:out value="${row.chromStart}"/>-<c:out value="${row.chromEnd}"/></td>
<td><bio:rs><c:out value="${row.name}"/></bio:rs></td>
</tr>
</c:forEach>
</table>
</body>
</html>
</jsp:root>

Tomcat needs src/web.xml as a descriptor to learn how to deploy this web application.

<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app
xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
version="2.5">
<display-name>Application Name</display-name>
<description>Application Description</description>


<taglib>
<taglib-uri>http://jsp.lindenb.org</taglib-uri>
<taglib-location>/WEB-INF/bio.tld</taglib-location>
</taglib>

<!-- see http://www.developer.com/java/ejb/article.php/1447551 -->
<taglib>
<taglib-uri>http://java.sun.com/jstl/fmt</taglib-uri>
<taglib-location>/WEB-INF/fmt.tld</taglib-location>
</taglib>

<taglib>
<taglib-uri>http://java.sun.com/jstl/fmt-rt</taglib-uri>
<taglib-location>/WEB-INF/fmt-rt.tld</taglib-location>
</taglib>

<taglib>
<taglib-uri>http://java.sun.com/jstl/core</taglib-uri>
<taglib-location>/WEB-INF/c.tld</taglib-location>
</taglib>

<taglib>
<taglib-uri>http://java.sun.com/jstl/core-rt</taglib-uri>
<taglib-location>/WEB-INF/c-rt.tld</taglib-location>
</taglib>

<taglib>
<taglib-uri>http://java.sun.com/jstl/sql</taglib-uri>
<taglib-location>/WEB-INF/sql.tld</taglib-location>
</taglib>

<taglib>
<taglib-uri>http://java.sun.com/jstl/sql-rt</taglib-uri>
<taglib-location>/WEB-INF/sql-rt.tld</taglib-location>
</taglib>

<taglib>
<taglib-uri>http://java.sun.com/jstl/x</taglib-uri>
<taglib-location>/WEB-INF/x.tld</taglib-location>
</taglib>

<taglib>
<taglib-uri>http://java.sun.com/jstl/x-rt</taglib-uri>
<taglib-location>/WEB-INF/x-rt.tld</taglib-location>
</taglib>


</web-app>

and we finally need src/build.xml to build all this stuff with ant.

<?xml version="1.0" encoding="ISO-8859-1"?>
<project name="Test" default="install" basedir=".">
<property name="tomcat.home" value="../apache-tomcat-6.0.14"/>
<property name="jstl.home" value="../jakarta-taglibs/standard"/>
<property name="webapps" value="${tomcat.home}/webapps"/>

<target name="compile">
<javac destdir="." srcdir="." debug="on">
<include name="org/lindenb/jsp/*.java"/>
<classpath>
<pathelement location="${jstl.home}/lib/jstl.jar"/>
<pathelement location="${jstl.home}/lib/standard.jar"/>
<pathelement location="${tomcat.home}/lib/servlet-api.jar"/>
<pathelement location="${tomcat.home}/lib/jsp-api.jar"/>
</classpath>
</javac>

<jar destfile="bio.jar"
basedir="."
includes="org/**"

/>

</target>

<target name="install" depends="compile">
<!-- yes, I know there is also war task... -->
<zip destfile="${webapps}/test.war">
<zipfileset dir="jsp" includes="*.jsp"/>
<zipfileset dir="." includes="web.xml" prefix="WEB-INF"/>
<zipfileset dir="${jstl.home}/tld" includes="*.tld" prefix="WEB-INF"/>
<zipfileset dir="." includes="*.tld" prefix="WEB-INF"/>
<zipfileset dir="${jstl.home}/lib" includes="*.jar" prefix="WEB-INF/lib"/>
<zipfileset dir="." includes="bio.jar" prefix="WEB-INF/lib"/>
</zip>
</target>

</project>

let's build this application. It creates a web archive (war) in the webapps folder of tomcat.

ant
Buildfile: build.xml

compile:
[jar] Building jar: /home/pierre/tmp/TOMCAT/src/bio.jar

install:
[zip] Building zip: /home/pierre/tmp/TOMCAT/apache-tomcat-6.0.14/webapps/test.war

BUILD SUCCESSFUL
Total time: 1 second

When you open "http://localhost:8080/test/page.jsp" you should get the following screen:

Pierre

“Getting Started In…”

In PLOS, today: This month, PLoS Computational Biology and the ISCB begin a series of short, practical articles for students and active researchers who want to learn more about new areas of computational biology and are unsure where or how to start. The aim of each article in the “Getting Started in…” series is to introduce the essentials: define the area and what it is about, highlight the debates and issues of relevance, and provide directions to the most relevant books, articles, or Web sites to find out more...

The first expert to inform, motivate, and inspire readers to consider a new direction is Dr. Xiaole Shirley Liu, who introduces tiling microarrays.

“Getting Started In…”: A Series Not to Miss: PLoS Computational Biology 3 (10), e224 (2007)
Getting Started in Tiling Microarray Analysis : PLoS Computational Biology 3 (10), e183 (2007)

Hello World ! My first FLEX2 Application.

Adobe Flex 2 software is a rich Internet application framework based on Adobe Flash that will enable you to productively create beautiful, scalable applications that can reach virtually anyone on any platform.

The SDK was downloaded and extracted from : http://download.macromedia.com/pub/flex/sdk/flex_sdk_2.zip.

File:first.mxml

<?xml version="1.0" encoding="utf-8"?>
<mx:Application
    xmlns:mx="http://www.adobe.com/2006/mxml"
    horizontalAlign="center" verticalAlign="center">
    <mx:Button id="myButton" label="Hello Bioinformatics World !" />
</mx:Application>

The file was compiled to flash using the mxmlc command (a java application):

>FLEX/SDK/bin/mxmlc --strict=true --file-specs first.mxml
Loading configuration file FLEX/SDK/frameworks/flex-config.xml
FLEX/test/first.swf (125402 bytes)
>  ls
first.mxml  first.swf

I then opened first.swf in firefox and...


Whaaaa ! It worked without any problem !!

I'M THE KING OF THE WORLD !!

This looks exciting ! I need a good tutorial ! I need a good tutorial ! I need ....


Pierre :-)

New blog: Beginning Python for Bioinformatics

Beginning Python for Bioinformatics is a new blog about... (guess what). The author's aim is to convert the book Beginning Perl for Bioinformatics into pyhton.

See also: Python course in Bioinformatics

Pierre