YOKOFAKUN: Visual Unix Pipeline

A few days ago I was presented a really impressive demo of Amadea. In a simplistic view, it looks like a "visual unix pipeline": just drag your 'grep','sort',etc... on your desktop, draw the links, choose your data sources (mysql, excel spreadsheets...) and run your analysis.

Via ISoft: The central window enables the user to draw and control the execution of the transformation process. Selecting the output of one of the operators automatically updates the data grid to reflect the transformations processed by this operator on the input table. The frames on the left of the screen give access to transformation operators. The parameters of each operator can be spelt out in the right of the screen.

I wondered if there is any other tool which acts like such a visual editor and which could help people from my lab to perform some simple operation on their data ?

On the other hand and without pretention, it might be easy to create a naive version of those unix filters in java just by extending java.io.InputStream. For example, for a Grep, I would write:

public class GrepInputStream
 extends InputStream
 {
 /** parent stream */
 private InputStream parent;

 /** regular expression */
 private Pattern pattern;

 /**  byte buffer */
 private byte byteBuffer[];

 /** position in buffer */
 private int curr;


 /** constructor */
 public GrepInputStream(
   InputStream parent,
   Pattern pattern)
  {
  this.parent= parent;
  this.pattern=pattern;
  this.byteBuffer=null;
  }

 @Override
 public int read() throws IOException
  {
  /** byte buffer already exists */
  if(byteBuffer!=null && curr< byteBuffer.length)
   {
   byte b= byteBuffer[curr++];
   if(curr== byteBuffer.length)
    {
    byteBuffer=null;
    curr=0;
    }
   return b;
   }

  while(true)
   {
   /** read next line from parent */
   int i;
   ByteArrayOutputStream byteOutputStream=new ByteArrayOutputStream();
   while((i=parent.read())!=-1)
    {
    byteOutputStream.write(i);
    if(i=='\n') break;//eol reached
    }
   this.byteBuffer= byteOutputStream.toByteArray();
   if(byteBuffer.length==0) return -1;

   /** creates the line. remove the CR if needed */
   String line= new String(
     this.byteBuffer,
     0,
     byteBuffer[byteBuffer.length-1]=='\n'?byteBuffer.length-1:byteBuffer.length
     );

   Matcher match= this.pattern.matcher(line);
   /* this line  matcth our pattern */
   if(match.find()) break;
   }
  this.curr=1;
  return this.byteBuffer[0];
  }

}

Pierre

YOKOFAKUN

04 December 2006

Visual Unix Pipeline

1 comment: