26 September 2006

MYSQL UDF, trees of data, hierarchy

In a previous post I described how to write a mysql user defined function (UDF) in C to create a translate-dna-to-protein function for mysql. Now, I've been playing with hierachies and I wrote a few UDFs to explore a tree of data; I guess this could have been done with a mysql stored procedure but I'm currently using an old server (and I need a deeper knowledge of mysql5 :-) ). The source code is available [here].
The tree itself is written as a sorted static const array of data so you cannot modify it via a mysql command but you'll have to recompile the code.

typedef struct Taxonomy
{
/** the ncbi-id */
taxon_type_t tax_id;
/** id of the parent */
taxon_type_t parent_id;
/** scientific name */
char name[TAXON_NAME_SIZE];
}Taxon,*const TaxonPtr;


It may also not be suitable for large trees. In my example, I've been using a subset of the NCBI taxonomy:

static const Taxon all_taxons[]={
{1, -1, "root"},
{2759, 131567, "Eukaryota"},
{6072, 33208, "Eumetazoa"},
{7711, 33511, "Chordata"},
{7742, 89593, "Vertebrata"},
{7776, 7742, "Gnathostomata"},
{8287, 117571, "Sarcopterygii"},
{9347, 32525, "Eutheria"},
{9443, 314146, "Primates"},
...)
};


but one could imagine something like this...

{1,-1,"root"},
{2,1,"world"},
{3,2,"europe"},
{4,3,"france"},
...


or...

{1,-1,"rdf:Property"},
{2,1,"foaf:knows"},
{3,2,"rel:friendOf"},
{4,2,"rel:childOf"},
...


the source taxonudf.c was successfuly compiled on my computer using the following command-line.

gcc -fPIC -shared -DDBUG_OFF -O3 -I/usr/include/mysql -lmysqlclient -o /usr/lib/taxonudf.o taxonudf.c


taxon_name returns the scientific name of an organism from a ncbi-tax-id.

mysql> create function taxon_name
returns string soname "taxonudf.o";
Query OK, 0 rows affected (0,00 sec)

mysql> select taxon_name(9606);
+------------------+
| taxon_name(9606) |
+------------------+
| Homo sapiens |
+------------------+
1 row in set (0,43 sec)


tax_id returns the ncbi-tax-id from from its name.

mysql> create function taxon_id
returns integer soname "taxonudf.o";
Query OK, 0 rows affected (0,00 sec)

mysql> select taxon_id("Homo sapiens");
+--------------------------+
| taxon_id("Homo sapiens") |
+--------------------------+
| 9606 |
+--------------------------+
1 row in set (0,00 sec)


taxon_childof returns wether a node in the hierarchy is a descendant of another node.

mysql> create function taxon_childof
returns integer soname "taxonudf.o";
Query OK, 0 rows affected (0,00 sec)

mysql> select taxon_childof(taxon_id("Homo"),taxon_id("Homo Sapiens"))
as "is Homo child of Homo.Sapiens ?";
+---------------------------------+
| is Homo child of Homo.Sapiens ? |
+---------------------------------+
| 0 |
+---------------------------------+
1 row in set (0,00 sec)


mysql> select taxon_childof(taxon_id("Homo Sapiens"),taxon_id("Homo"))
as "Is Homo.Sapiens descendant of Homo ?";
+--------------------------------------+
| Is Homo.Sapiens descendant of Homo ? |
+--------------------------------------+
| 1 |
+--------------------------------------+
1 row in set (0,00 sec)



taxon_com is an aggregate function which finds the common ancestral node in a set of node.

mysql> create aggregate function taxon_com
returns integer soname "taxonudf.o";
Query OK, 0 rows affected (0,00 sec)

mysql> create temporary table t1(cluster varchar(20),taxon int);
Query OK, 0 rows affected (0,07 sec)

mysql> insert into t1(cluster,taxon) values("A",251093),
("A",9781), ("A",37348),("B",9605),("B",9606),("B",63221),
("C",32523),("C",33154),("C",7776),("C",9443);
Query OK, 10 rows affected (0,03 sec)
Records: 10 Duplicates: 0 Warnings: 0

mysql> select cluster,taxon as "ncbi-id",taxon_name(taxon) as "Name",taxon_childof(taxon,taxon_id("Primates")) as "Is_Primate" from t1;
+---------+---------+-------------------------------+------------+
| cluster | ncbi-id | Name | Is_Primate |
+---------+---------+-------------------------------+------------+
| A | 251093 | Elephas antiquus | 0 |
| A | 9781 | Elephantidae gen. sp. | 0 |
| A | 37348 | Mammuthus | 0 |
| B | 9605 | Homo | 1 |
| B | 9606 | Homo sapiens | 1 |
| B | 63221 | Homo sapiens neanderthalensis | 1 |
| C | 32523 | Tetrapoda | 0 |
| C | 33154 | Fungi/Metazoa group | 0 |
| C | 7776 | Gnathostomata | 0 |
| C | 9443 | Primates | 1 |
+---------+---------+-------------------------------+------------+
10 rows in set (0,00 sec)

mysql> select cluster,taxon_com(taxon) as ncbi_id
from t1 group by cluster;
+---------+---------+
| cluster | ncbi_id |
+---------+---------+
| A | 9780 |
| B | 9605 |
| C | 33154 |
+---------+---------+
3 rows in set (0.00 sec)


That's all folks

03 September 2006

Scott McCloud at SciFoo 2006

I've just discovered on Flickr that Scott McCloud was present at SciFoo 2006.

scifoo


McLoud is the author of Understanding comics, one of the best book about comics I've ever read.

scifoo


In this book, McLoud introduced a map of visual iconography that took the shape of a triangle.

The lower left corner was visual resemblance (e.g., photography and realistic painting). The lower right included the products of what he called iconic abstraction (e.g., cartooning). And at the top were the denizens of the picture plane ("pure" abstraction) which ceased to make reference to any visual phenomena other than themselves. The move from realism to cartoons along the bottom edge was a move away from resemblance that still retained "meaning," so words, the next logical step in the progression, were included at far right, thereby enclosing anything in comics' visual vocabulary between the three points.

See also: http://www.scottmccloud.com/inventions/triangle/triangle.html


29 August 2006

My own little scifoo camp 2006.

Back from holidays at Montresor where, as the leader of Nature Network BrieComte Robert, I organized my own little private rainy scifoo camp.

My own private scifoo camp


The reason you didn't see me at SciFoo 2006

The reason you didn't see me at SciFoo 2006
The reason you didn't see me at SciFoo 2006


11 August 2006

The Life Sciences Semantic Web is Full of Creeps!

An article published in " Briefings in Bioinformatics Advance Access".

The Life Sciences Semantic Web is Full of Creeps!


Benjamin M. Good and Mark D. Wilkinson
Abstract:The Semantic Web for the Life Sciences (SWLS), when realized, will dramatically improve our ability to conduct bioinformatics analyses using the vast and growing stores of web-accessible resources. This ability will be achieved through the widespread acceptance and application of standards for naming, representing, describing and accessing biological information. The W3C-led Semantic Web initiative has established most, if not all, of the standards and technologies needed to achieve a unified, global SWLS. Unfortunately, the bioinformatics community has, thus far, appeared reluctant to fully adopt them. Rather, we are seeing what could be described as ‘semantic creep’--timid, piecemeal and ad hoc adoption of parts of standards by groups that should be stridently taking a leadership role for the community. We suggest that, at this point, the primary hindrances to the creation of the SWLS may be social rather than technological in nature, and that, like the original Web, the establishment of the SWLS will depend primarily on the will and participation of its consumers.

Mark Wilkinson is one of the creators of BioMoby. Bio Moby is a system for interoperability between biological data hosts and analytical services. Benjamin Good is a PhD student in the British Columbia Strategic Training Program in Bioinformatics. Both of them have a profile on connotea (users bgood, mwilkinson), group:Wilkinson Laboratory).

Although I'm convinced that the semantic web/RDF/XML model is the format of choice for any application (please ! use it for your output format !), I admit I never had the time and the technical knowledge about web services to really understand how BioMoby works , why I should use it and why I should use a LSID instead of an good old URI... :-)

hey, I'm going to ask them an offprint :-)

09 August 2006

A Bookmarklet for Offprint Requests

Hi, I'm pleased to share the javascript Bookmarklet I wrote today. A bookmarklet is a small JavaScript program that can be stored as a URL within a bookmark in most popular web browsers, or within hyperlinks on a web page. This bookmarklet invokes a new mail, from thunderbird, filled with a message requesting an offprint request fo an article. The first <a href="mailto:xxx@xxx.xxx"> tag found in the current page is used as the recipient of the mail and the subject is the title of the current page.


Here is the bookmarklet (you have to modify it by editing its properties in order to include your own message...):

Drag this Link: Offprint Request up to your Bookmarks Toolbar.


The bookmarklet was successfully tested on firefox/thunderbird with Bioinformatics: Building chromosome-wide LD maps Bioinformatics 2006 22(16):1933-1934 and NAR SYBR Green real-time telomeric repeat amplification protocol for the rapid quantification of telomerase activity Nucleic Acids Research, 2002, Vol. 31, No. 2 e3.

Example of mail generated from the previous paper from NAR:

From: me
To: xxxx@ucdavis.xxx
Subject: [offprint request] SYBR Green real-time telomeric repeat amplification protocol for the rapid quantification of telomerase activity -- Wege et al. 31 (2): e3 -- Nucleic Acids Research

Hi,
my name is Bruce Banner, I'm a nuclear physicist working on gamma radiations at Los-Alamos. Your recent paper titled

"SYBR Green real-time telomeric repeat amplification protocol for the rapid quantification of telomerase activity -- Wege et al. 31 (2): e3 -- Nucleic Acids Research"

caught my attention.
Would it be possible for you to forward me a PDF copy ? I thank you in advance.

Best Regards.

B. Banner

--
Bruce Banner PhD.
Gamma Radiation Laboratory
Los Alamos
http://www.marvel.com/universe/Hulk


07 August 2006

Drawing Networks, Interaction Networks, Social Networks

There is a cool presentation on google-tech-talks where Tamara Munzner introduce 15 algorithms describing how to draw a graph. That might be of interest for people working with "systems biology".



About those talks, the information is not displayed on google-video web site but it is now possible to get a RSS to keep track of the new talks at the following URL: http://video.google.com/videosearch?q=engEDU&so=1&output=rss (just add &output=rss at the end).

About networks, I've (re-)discovered a interesting source of data for protein-protein interactions. The database is called BioGrid(http://www.thebiogrid.org/index.php) and it was described in the database issue of NAR (BioGRID: a general repository for interaction datasets). I like this kind of resource because all interactions described in that database are manually curated from pubmed abstracts. This is much more rigourous than the large interaction maps screenings because, in the case of experiments using the yeast two hybrid system, for each distinct interaction, there should have some test to confirm the interaction. At the time I was using this assay, I used the method described by Bartel &. al (inverting the bait and the pray, co-immunoprecipitation, etc...) to confirm any interaction. But let's be fair, I have not read much bibliography about this subject since 1999...


The graphics from BioGrid can be exported in SVG...



....and the whole database can be downloaded in the PSI format.


PSI is a simple XML format describing

....interactors...
(...)
<proteinInteractor id="BIOGRID-109561">
<names>
<shortLabel>HGNC:5261</shortLabel>
</names>
<xref>
<primaryRef db="MIM" id="118190" secondary="" version=""/>
</xref>
<organism ncbiTaxId="9606">
<names>
<shortLabel>Human</shortLabel>
<fullName>Homo sapiens</fullName>
</names>
</organism>
</proteinInteractor>
(...)


... and interactions....

(...)
<interaction>
<names>
<shortLabel>1</shortLabel>
</names>
(...)
<participantList>
<proteinParticipant>
<proteinInteractorRef ref="BIOGRID-106718"/>
<role>bait</role>
</proteinParticipant>
<proteinParticipant>
<proteinInteractorRef ref="BIOGRID-106718"/>
<role>prey</role>
</proteinParticipant>
</participantList>
<interactionType>
<names>
<shortLabel>Invivo</shortLabel>
</names>
<xref>
<primaryRef db="" id="" secondary="" version=""/>
</xref>
</interactionType>
<xref>
<primaryRef db="pubmed" id="12081471" secondary="" version=""/>
</xref>
</interaction>
(...)


as far as I know,PSI is much more simplier than SBML or BIOPAX for describing simple protein-protein interactions (no biochemistry, no pKa, etc... however the format could be modernized by supporting a RDF/XML syntax). I also sent them a mail, asking how to sublit a new interaction but I still got no answer.


About social networks: David Wolber has introduced his social network in a google tech talk. There are a few good ideas about social networks: document and people are linked by a semantic property like in a FOAF document.

via google: Peoplicious is a collaborative research tool. Unlike systems such as del.icio.us, people are first-class data objects, along with documents. Users can create people, provide structured information about people (image, homepage, blog feed, delicious name, etc.), and create lists of people (people-tagging). Users can also bookmark documents and associate documents with people (personmarks). Any user can enter information about any person. Peoplicious is where del.icio.us meets RSS reader all within a modern day address book. There is a working prototype that can be accessed at: http://peoplicious.com/Technorati/lists. It is designed so that new sites can easily be created for particular domains (e.g., see http://peoplicious.com/USF/lists).

21 July 2006

NAR, Nature Aggademia & Bioinformatics web services

Yesterday, I've read the 150 abstracts of the special web server issue from NAR. About those 150 articles see my previous post about this. All thoses abstracts where tagged with connotea here.

As a Bioinformatician (yes I am), I like automated operations and most of all those
servers require a 'clickodrome'/web interface, that is to say,
a manual input: for example, I cannot use it to perform multiple
sequence analysis. NAR and other publishers should now require from
authors that the ouput of those tools should be written in a fully
parsable xhtml syntax, or better, in XML/RDF backed with a XSLT/xhtml stylesheet.
See also: BioMoby.

In the issue of NAR, there was a paper about Hubmed written by Alf Eaton. Alf now works at Nature and he just have released an experimental and beta social site called aggademia which aggregates the 50 most popular scientific blogs. At first sight, it works like postgenomic. I'm a little puzzled now with all those social web sites at Nature: connotea, Nature Network Boston, and (perhaps) now aggademia. On each of those site if had to register a new profile, I could create a group of user and send invitation ("Oh Nooo Pierre, not AGAIN one of your CENSORED social network"). Why not ("...you just have to......") a large integrated social scientific network ?


17 July 2006

NAR: The Web Server Issue 2006

Nucleic Acids Research has published its annual special issue devoted to web servers. The current issue contains... 150 papers (argh !).


10 July 2006

Mysql user defined function (UDF) for Bioinformatics.

MYSQL allows to create User Defined Functions (UDF). Written in 'C/C++', this kind of function can be used to embed bioinformatics into mysql. Here is an example of a function used to translate a DNA sequence into a protein directly in mysql.

#include <my_global.h>
#include <m_ctype.h>
#include <mysql.h>
#include <m_string.h>

/* a function translating 3 bases into an amino acid */
static char translation(char a,char b,char c);

/* The initialization function */
my_bool translate_init(UDF_INIT *initid, UDF_ARGS *args, char *message);
/* The deinitialization function */
void translate_deinit(UDF_INIT *initid);
/* The main function. This is where the function result is computed */
char *translate(UDF_INIT *initid, UDF_ARGS *args, char *result,
unsigned long *length, char *is_null, char *error);

/* The initialization function */
my_bool translate_init(
UDF_INIT *initid,
UDF_ARGS *args,
char *message
)
{
/* check the args */
if (!(args->arg_count == 1 && args->arg_type[0] == STRING_RESULT ))
{
strncpy(message,"Bad parameter expected a DNA",MYSQL_ERRMSG_SIZE);
return 1;
}
initid->maybe_null=1;
initid->ptr= (char*)malloc(0);

if(initid->ptr==NULL)
{
strncpy(message,"Out Of Memory",MYSQL_ERRMSG_SIZE);
return 1;
}
return 0;
}

/* The deinitialization function */
void translate_deinit(UDF_INIT *initid)
{
/* free the memory **/
if(initid->ptr!=NULL) free(initid->ptr);
}

/* The main function. This is where the function result is computed */
char *translate(UDF_INIT *initid, UDF_ARGS *args, char *result,
unsigned long *length, char *is_null, char *error)
{
long i;
long dnaLength= args->lengths[0];
const char *dna=args->args[0];
char *ptr=NULL;

if (dna==NULL) /* Null argument */
{
*is_null=1;
return NULL;
}
*length=dnaLength/3;
ptr= (char*)realloc(initid->ptr,sizeof(char)*(*length));
if(ptr==NULL)
{
*is_null=1;
*error=1;
strncpy(error,"Out Of Memory",MYSQL_ERRMSG_SIZE);
return NULL;
}
initid->ptr=ptr;
/* loop over the codons of the sequence */
int j=0;
for(i=0;i+2< dnaLength;i+=3)
{
initid->ptr[j++]=translation(dna[i],dna[i+1],dna[i+2]);
}

return initid->ptr;
}

/************************************
*
* translation
* a function translating 3 bases into an amino acid
*/
static
char translation(char base1,char base2,char base3)
{
(...)/* so obvious.... */
}


And here is the Makefile for my machine...

/usr/lib/translate.so:translate.c
gcc -fPIC -shared -I/usr/include/mysql -DDBUG_OFF -O3 -lmysqlclient -o $@ $<


... and the fragment from a session

mysql> CREATE FUNCTION translate RETURNS STRING SONAME 'translate.so';
Query OK, 0 rows affected (0,03 sec)

mysql> select translate("ATGGAGTCTACTCAGCAGATGGCTTCTTCTATTATTAATTCTTCATTTGAAGCT
AATTGATGGGTATTCAATATGACTACAATGAGGTATATACTAGAGTAAAGAGTAAATTTGATTTAGTTATGGATGATTC
GCAATTACTATTGATCAAGCTTTGAATGGAAAATTTAGTTCAGCGATTAGGAATAGAAATTGGATGACTGACTCTCGAA
TAAACTAAGAATTATGCTATCATCAAAAGGAATCGATCAGAAAATGAGAGTGCTTAATGCTTGTTTTAGTGTCAAGAGA
AATGTACTAGACTGATGAAAGACAAATTAGAACGTGGTGAAGTTGAAGTTGATGATTCCTTTGTTGAAGAGAAAATGGA
TATGAACAGTTAGAAAAGAGATTTGAGTCACTGAAACATCGGGTTAATGAGAAGTATAATCATTGGGTTCTTAAAGCTA
TCAAAATGTGATTT
") as NSP3;
+------------------------------------------------------------------------------ -------------------------------------------------------------------------------
| NSP3
+------------------------------------------------------------------------------
-------------------------------------------------------------------------------
| MESTQQMASSIINSSFEAAVVAATSTLELMGIQYDYNEVYTRVKSKFDLVMDDSGVKNNLIGKAITIDQALNGKFSS
SSKGIDQKMRVLNACFSVKRIPGKSSSIVKCTRLMKDKLERGEVEVDDSFVEEKMEVDTIDTKSRYEQLEKRFESLKHR

+------------------------------------------------------------------------------
-------------------------------------------------------------------------------
1 row in set (0,00 sec)