SOLR facet component customizations

Here is the use case:   Assume, you are developing e-commerce  search system, one fine day, someone from marketing department comes and says he wants to able to sort the brad facet differently ( neither increasing count order or lexicographical order  supported by the SOLR facets module. For example bring Samsung after Apple, for a tablet category brand facet) In this case, development team will end up customizing the sort at application layer for each facet field. I did this as  custom SOLR component. Following recipe will show.  I am not including all the component logic, but following logic which does parsing of facet component outputs & setting the order differently.

public void process(ResponseBuilder rb) throws IOException

      

  NamedList facet_counts = (NamedList)rb.rsp.getValues().get("facet_counts" );

   if (facet_counts == null) return;

   NamedList> facet_fields = (NamedList>) facet_counts.get( "facet_fields" );

           if (facet_fields == null) return;

           if( facet_fields != null ) {

           TreeMap sortedMap = new  TreeMap();

              int ctr=0;

            for( Map.Entry> facet : facet_fields ) {

                 String key =  facet.getKey() ;

                 if ( customFacetCache.containsKey(key )){

                    FacetObj facetDetails = customFacetCache.get(key);

                    if (facetDetails != null ){

                          String sortOrder = facetDetails.sortOrder;

                           sortedMap.clear();

                           for( Map.Entry entry : facet.getValue() ) {

                             sortedMap.put(entry.getKey(), entry.getValue().longValue() );               

                           }

                  if (sortOrder.equalsIgnoreCase("custom")){

                          ///now introduce your own custom order.

                  }

SOLR 4.2 features: Rest API to fetch SOLR schema, fields etc in JSON

Most f the SOLR business logic developers end up parsing solr schema with Luke request.
 & parsing the SOLR response structures etc in their logic.
 If you are a pure JSON geek, this aspect turn off.

 However starting with Solr 4.2 onwards, now SOLR supports
 REST API to request the the schema in json format.
Not only entire schema file, one can request, just few fields or field types,
dynamic fields, copy fields etc.
Wish they support wild cards. May be for  future.

For now this is solid beginning.

 Entier schema
 http://localhost:8983/solr/collection1/schema?wt=json 

Request price field:
 http://localhost:8983/solr/collection1/schema/fields/price?wt=json

 Request dynamic field ending with _I 
http://localhost:8983/solr/collection1/schema/dynamicfields/*_i?wt=json

 Request data field type.
 http://localhost:8983/solr/collection1/schema/fieldtypes/date?wt=json

Old  Style:

 LukeRequest request = new LukeRequest();
request.setShowSchema(true);
 request.setMethod(METHOD.GET);
LukeResponse response = request.process(getServer());
if(response != null && response.getFieldInfo() != null)
{ Map fieldInfoMap = response.getFieldInfo(); 
  Set fieldkeys = fieldInfoMap.keySet();
  for (String fieldKey : fieldkeys) { 
       ////do your stuff
  }
 } }

Sample R code to count number of the terms in end user queries & plot

dups<-function p="">  df<-read .csv="" csv="" input="" nbsp="" p="" query="" read="" strip.white="TRUE)" temr="">
  df[[1]] <- any="" cleanup="" df="" fixed="T)" gsub="" nbsp="" of="" p="" redirect="" term="">   ind <- df="" duplicate="" duplicated="" filter="" p="">
  new.df <- df="" ind="" p="">
  myh<-nchar 1="" gsub="" nbsp="" new.df="" p="">  #buckets
  one<- length="" myh="=1])</p">  two<- length="" myh="=2])</p">  three<- length="" myh="=3])</p">  four<- length="" myh="=4])</p">  five<- length="" myh="=5])</p">  six<- length="" myh="=6])</p">  seven<- length="" myh="=7])</p">  eight<- length="" myh="=8])</p">  cvec <- c="" eight="" five="" four="" nbsp="" one="" p="" seven="" six="" three="" two="">
  result.frame = as.data.frame(matrix(ncol=2, nrow=10))
  names(result.frame) = c("Number", "Total")
  # following is OK for now

  result.frame = rbind(result.frame, c(1, one))
  result.frame = rbind(result.frame, c(2, two))
  result.frame = rbind(result.frame, c(3, three))
  result.frame = rbind(result.frame, c(4, four))
  result.frame = rbind(result.frame, c(5, five))
  result.frame = rbind(result.frame, c(6, six))
  result.frame = rbind(result.frame, c(7, seven))
  result.frame = rbind(result.frame, c(8, eight))

  plot(result.frame$Number,result.frame$Total,pch=19,col="blue" , xlab="Number of terms in a query" ,ylab="Total")


  lines (result.frame$Number, result.frame$Total,lwd="4",col="red"  )
  lm1<-lm otal="" p="" result.frame="" umber="">  abline (lm1,lwd="4",col="green"  )


}

Moving from FAST to SOLR: Project Updates

Some background: Decision to move to SOLR was already made. Team hardly knows anything about FAST ESP. However business team know what they want in the new SOLR platform. (A clone of FAST based search system while addressing their pain points)

  Current status: Phase I of the project was completed & system is already in production (With all complexities, we wrapped the project in record time.)

  Positives: SOLR is fast in-terms of content indexing & searches. (Overall QPS is greater than FAST and current sizing issues are resolved.)

  Challenges: 1) During implementation we noticed strong customizations around business rules. This functionality is not available in SOLR/Lucene. I did some domain specific customizations. 2) We are replacing a search product with decent relevancy (because of all business rules) & we started late with relevancy. Relevancy echo system includes fine-tuning of similarity algorithms (tf/idf, B25 etc.) plus fine-tuning of synonyms/ spell-check modules. SOLR synonyms/spell check modules need more improvements/core bug fixes. Again I did more customizations to meet the needs. 3) Dynamic range facets & site taxonomy traversal/updates need future work. Basic stuff is working. However if the taxonomy changes often, doing incremental updates is a complex issue. For now, we have a workaround in place. Some extent business rules stuff was invented to work around some of these problems. Map reduce & Graph DB frameworks seems to solve issues around dynamic range facets/dynamic taxonomies. Exploring simple integration approaches between Hadoop/SOLR.

  Luck factor: Existing FAST based search was not completely leveraging FAST strong document processing capabilities(Linguistic normalization/ sentiment /Taxonomy etc). So we managed with little customizations around Lucene analyzers.

Lucene revolution conference 2012

After long time, I attended Lucene revolution conference. Overall this is good experience. Few sessions are too good & few missed the mark.

Personally I like the following sessions because of the contents/presenters energy/passion behind the search technology.

Automata Invasion
Challenges in Maintaining a High Performance Search Engine Written in Java
Updateable Fields in Lucene and other Codec Applications
Solr 4: The SolrCloud Architecture

Also “Stump the Chump” questions are interesting & learned quite a bit.
I won small prizes too. In general, Lucid imagination uploads the
conference vedios at the following location. Keep watching.
I also missed few good sessions in Big data area.

http://www.lucidimagination.com/devzone/videos-podcasts/conference-videos

Mapping hierarchical data in to SOLR

Experiments with Solr 4.0 (beta)

While modeling hierarchical data in XML easy, (for example org charts, Bill of materials structures) mapping to a persistent storage is very challenging. Relational SQL manages it however fetching/ updating the hierarchies is very difficult. Even books are written on mapping Tree/Graphs in to RDBMS world.

Consider simple hierarchy list looks like this:
Satya
Saketh
Dhanvi
Venkata
Dhasa

The most common and familiar known method is adjacency model, and it usually works as every node knows the adjacency node. (In SOLR world, ID contains in the unique value & parent field contains parent. Assume for the root node it is null or same.)

In SOLR each row is a document:

SOLRID Name Parent
01 Satya NULL
02 Saketh 01
03 Dhanvi 01
04 Venkata 01
05 Dhasa 04

Latest SOLR(4.0 beta?) Join functionality gives you the full hierarchy (or any piece of it) very quickly and easily.

Example queries:

1) Give me complete hierarchical list: q= {!join from=id to=parent}id:*
2) Give me immediate childs of Satya: q={!join from=id to=parent}id:Satya

An example SOLR query component to pull other objects

Consider Linked-in example. i.e. I have a set of connections. With single SOLRrequest, following SOLR component brings all first level connections information. In RDBMS world, typical example is employee table. i.e. bring me first level employees of manager. Assume single table contains both employee and manager information. (typical Adjacency list)

In terms of physical SOLR mapping, every document contains a connection field which contains the list(Adjacency list) of connections (root ids)

Configuration point of view, add the following to solrconfig.xml

public class ExampleComponent extends SearchComponent
{
  public static final String COMPONENT_NAME = "example";

  @Override
  public void prepare(ResponseBuilder rb) throws IOException
  {
  
  }
  @SuppressWarnings("unchecked")
  @Override
  public void process(ResponseBuilder rb) throws IOException
  {
     DocSlice slice = (DocSlice) rb.rsp.getValues().get("response");
       SolrIndexReader reader = rb.req.getSearcher().getReader();
       SolrDocumentList rl = new SolrDocumentList();
       int docId=0;//// at this point consider only one rootid.
       for (DocIterator it = slice.iterator(); it.hasNext(); ) {
         docId = it.nextDoc();
         Document doc = reader.document(docId);
         String id = (String)doc.get("id");
         String connections = (String)doc.get("contains");
         System.out.println("\n id:"+id+" contains-->"+connections);
         List<String> list = new ArrayList<String>();
         list.add(id);//add rootid too. If we have joins in solr4.0
   int pos = 0, end;
   while ((end = connections.indexOf(',', pos)) >= 0) {
    list.add(connections.substring(pos, end));
    pos = end + 1;
   }
   BooleanQuery bq = new BooleanQuery();
   Iterator<String> cIter = list.iterator();
   while (cIter.hasNext()) {
    String anExp = cIter.next();
    TermQuery tq = new TermQuery(new Term("id",anExp));
    bq.add(tq, BooleanClause.Occur.SHOULD);
   }
         SolrIndexSearcher searcher = rb.req.getSearcher();
         DocListAndSet results = new DocListAndSet();
         results.docList = searcher.getDocList(bq, null, null,0, 100,rb.getFieldFlags());
         System.out.println("\n results.docList-->"+results.docList.size() );
         rl.setNumFound(results.docList.size());
         rb.rsp.getValues().remove("response");
         rb.rsp.add("response", results.docList);
     }
  }

  @Override
  public String getDescription() {
    return "Information";
  }
  @Override
  public String getVersion() {
    return "Solr gur";
  }
  @Override
  public String getSourceId() {
    return "Satya Solr Example";
  }
  @Override
  public String getSource() {
    return "$URL:   }
  @Override
  public URL[] getDocs() {
    return null;
  }
}

Data format conversion in java

Basic need is we used to get xml date data type in different formats for data sources. However our federated search engine we are storing in a neutral format. (For simplicity sake.) After analyzing couple of formats, added this code in year 2005. Nowadays Team is using some apache package utils. But I don’t think they will something different. I am not seeing many options.

public static final String[] date_format_list = {
     "yyyy-MM-dd'T'HH:mm:ss'Z'",
     "yyyy-MM-dd'T'HH:mm:ss",
     "yyyy-MM-dd",
     "yyyy-MM-dd hh:mm:ss",
     "yyyy-MM-dd HH:mm:ss",
     "EEE MMM d hh:mm:ss z yyyy"
     /// add your own format here
   };

   public static Date parseDate(String d) throws ParseException {
     return parseInputWithFormats(d, date_format_list);
   }

   public static Date parseInputWithFormats(
           String dateValue,
           String[] formatList
   ) throws ParseException {
     if (dateValue == null || formatList == null || formatList.length == 0) {
       throw new IllegalArgumentException("dateValue is null");
     }
 
     if (dateValue.length() > 1
             && dateValue.startsWith("'")
             && dateValue.endsWith("'")
             ) {
       dateValue = dateValue.substring(1, dateValue.length() - 1);
     }

     SimpleDateFormat dateParser = null;
        for(int i=0;i < formatList.length;i++){
       String format = (String) formatList[i];
       if (dateParser == null) {
         dateParser = new SimpleDateFormat(format, Locale.US);
       } else {
         dateParser.applyPattern(format);
       }
       try {
         return dateParser.parse(dateValue);
       } catch (ParseException pe) {
        //pe.printStackTrace();
       }
     }
     throw new ParseException("Unable to parse the input date " + dateValue, 0);
   }

public static void main(String[] args) {
  String fromDt="";
  String nPattern = "yyyy-MM-dd'T'HH:mm:ss'Z'";
  SimpleDateFormat sdf = new SimpleDateFormat(nPattern);
  
  String currentValue="Fri Jul 22 04:22:14 CEST 2011";
  
   try{  
             fromDt = sdf.format(parseDate(currentValue.toString() )  );
      } catch (Exception e) {
       System.out.print("\n Case1: date format exception"+e.getMessage()+ " SOLR currentValue:"+currentValue);
       fromDt="";
      } 
      System.out.println("Case1. date as str---"+fromDt);
      
      currentValue="2011-07-21 21:22:14";
       try{
              fromDt = sdf.format(parseDate(currentValue.toString() )  );
       } catch (Exception e) {
        System.out.print("\n Case2: date format exception"+e.getMessage()+ " SOLR currentValue:"+currentValue);
        fromDt="";
       } 
      System.out.println("\n Cse2.  date as str---"+fromDt);
  }

Latest family picicture from Disney world.

This is from latest Disney world. (Photo from Disney pass port photo service. We took good number of phones with Disney theme characters. Photos are not brilliant by professional photographer standards. We must not blame them. They are shooting nonstop. However they are not bad too.)

Parsing complex Xsd file with Java code.

This one nearly 4+ year old code. Now Team is using apache xerces XSModel.
However this code is there for sanity check. Still this works fine.
(You need xsom.jar + relaxngDatatype.jar file. Google it. You will find the jars.)

public class XsdReader {
 public static void main (String args[]) 
 {
  XsdReader rr = new XsdReader();
  rr.parseSchema();
    }
 
 public void parseSchema()
 {  
  File file = new File("D:\\tmp\\books.xsd");
  try {
      XSOMParser parser = new XSOMParser();
      parser.parse(file);
      XSSchemaSet sset = parser.getResult();
      XSSchema mys = sset.getSchema(1);
      Iterator itr = sset.iterateSchema();
      while( itr.hasNext() ) {
        XSSchema s = (XSSchema)itr.next();
        System.out.println("Target namespace: "+s.getTargetNamespace());
        XSComplexType ct = mys.getComplexType("books");
        int ctr=0;
        if ( ct != null){
        Collection c = ct.getAttributeUses();
        Iterator i = c.iterator();while(i.hasNext()){
            XSAttributeDecl attributeDecl = i.next().getDecl(); 
           
            System.out.print("ctr="+ctr++ +"name:"+ attributeDecl.getName());
            System.out.print("    type: "+attributeDecl.getType());
            System.out.println("");
          }
        }
        Iterator jtr = s.iterateElementDecls();
        while( jtr.hasNext() ) {
          XSElementDecl e = (XSElementDecl)jtr.next();
          
          System.out.print( e.getName() );
          if( e.isAbstract() )
            System.out.print(" (abstract)");
          System.out.println();
        }
      }
    }
    catch (Exception exp) {
      exp.printStackTrace(System.out);
    }
  }
}

Lucene Standard Analyzer vs. Lingpipe EnglishStop Tokenizer Analyzer

This is old code from prototypes directory.
For some odd reason, I end up prototyping different analyzers for PLM space content vs 3ed party analyzers. (Basic need is which got better control on STOP words. At least based on my quick proto type, SOLR got easy constructs.)

Small sample code comparing both analyzers is included.
I did not see much difference for small input text.

public class AnalyzerTest {
 
  private static Analyzer analyzer;
   private static long perfTime = 0;

   public static void main(String[] args) {
     try {
 
         analyzer = new StandardAnalyzer(org.apache.lucene.util.Version.LUCENE_34);

         String str = "PLM technology refers to the group of software applications that create and manage the data that define a product and the process for building the product. Beyond just technology, PLM is a discipline that defines best practices for product definition, configuration management, change control, design release, and many other product and process-related procedures.";

         perfTime -= System.currentTimeMillis();
         displayTokensWithLuceneAnalyzer(analyzer, str);
         perfTime += System.currentTimeMillis();
         
         System.out.println("Lucene Analyzer: " + perfTime + " msecs.");
         
         perfTime -= System.currentTimeMillis();
         displayTokensWithLingpipeAnalyzer(str);
         perfTime += System.currentTimeMillis();
         
         System.out.println("Lingpipe Analyzer: " + perfTime + " msecs.");
         
   } catch (IOException ie) {
         System.out.println("IO Error " + ie.getMessage());
       }
       System.out.println("Time: " + perfTime + " msecs.");
       System.out.println("Ended");
        }

    private static void displayTokensWithLingpipeAnalyzer(String text)
       throws IOException {
     
     System.out.println("Inside LingpipeAnalyzer ");
     
         TokenizerFactory ieFactory  
            = IndoEuropeanTokenizerFactory.INSTANCE;
         
          TokenizerFactory factory
                = new EnglishStopTokenizerFactory(ieFactory);
               // = new IndoEuropeanTokenizerFactory();
        
     char[] cs =text.toCharArray();
     Tokenizer tokenizer = factory.tokenizer(cs, 0, cs.length);
     String[] tokens = tokenizer.tokenize();
     for (int i = 0; i < tokens.length; i++)
        System.out.println(tokens[i]);
     
     System.out.println("Total no. of Tokens: " +tokens.length );
     
    }
   private static void displayTokensWithLuceneAnalyzer(Analyzer analyzer, String text)
      throws IOException {
    System.out.println("Inside LuceneAnalyzer ");
        TokenStream tokenStream = analyzer.tokenStream("contents",new StringReader(text) );
    OffsetAttribute offsetAttribute = tokenStream.getAttribute(OffsetAttribute.class);
    CharTermAttribute charTermAttribute = tokenStream.getAttribute(CharTermAttribute.class);
       int length=0;
       
    while (tokenStream.incrementToken()) {
        int startOffset = offsetAttribute.startOffset();
        int endOffset = offsetAttribute.endOffset();
        String term = charTermAttribute.toString();
        System.out.println("term->"+term+ " start:"+startOffset+" end:"+endOffset);
        length++;
    }
       System.out.println("Total no. of Tokens: " + length);
  }
   
}

Java Set Operations

As you know, for a while I’ve been playing with Solr. Basically implementing federated search framework to aggregate different ERP, CRM, PLM space data sources.
This post is not about federated search but I keep using bunch of set operations to compare the search results, comparing the unique doc id etc.

public class SetOperations {
 public static  Set union(Set setA, Set setB) {
     Set tmp = new TreeSet(setA);
     tmp.addAll(setB);
     return tmp;
   }

   public static  Set intersection(Set setA, Set setB) {
     Set tmp = new TreeSet();
     for (T x : setA)
       if (setB.contains(x))
         tmp.add(x);
     return tmp;
   }

   public static  Set difference(Set setA, Set setB) {
     Set tmp = new TreeSet(setA);
     tmp.removeAll(setB);
     return tmp;
   }
   
   public static void main(String[] args) {
         
        SortedSet s1=  new TreeSet();
        s1.add("one");
        s1.add("two");

        SortedSet s2=  new TreeSet();
        s2.add("two");
        s2.add("three");
        s2.add("four");
         
        SortedSet result = (SortedSet) union(s1,s2);
        
        Iterator it = result.iterator();
        System.out.print("union result  -->");
         while (it.hasNext()) {
             String value = it.next();
             System.out.print(value+", ");
         }
         System.out.println("\n");
         
         result = (SortedSet) intersection(s1,s2);
         
      it = result.iterator();
      System.out.print("intersection result-->");
      while (it.hasNext()) {
              String value = it.next();
              System.out.print(value+ ", ");
      }
      System.out.println("\n");

         result = (SortedSet) difference(s1,s2);
         
      it = result.iterator();
      System.out.print("difference result-->");
      while (it.hasNext()) {
              String value = it.next();
              System.out.print(value+", ");
      }
      System.out.println("\n");
      
         /*
         SortedSet i1=  new TreeSet();
         i1.add(new Integer("1"));
         
         SortedSet i2=  new TreeSet();
         i2.add(new Integer("2"));
         
         SortedSet iresult = (SortedSet) union(i1,i2);
         
         Iterator iit = iresult.iterator();
         System.out.println("Integer union result");
          while (iit.hasNext()) {
           Integer value = iit.next();
              System.out.println(value+",");
          }
         */
   }
   
}

XSL recursion sample

In my use case, I am receiving part numbers as a big string and all are separated by comma(,), I need to split them in to array of elements.
Here is the sample code.

<xsl:template name="printChildObjects">
  <xsl:param name="inputString"/>
  <xsl:param name="delimiter"/>

  <xsl:choose>
    <xsl:when test="contains($inputString, $delimiter)">
    <xsl:variable name="aChild">
    
    <xsl:value-of select="substring-before($inputString,$delimiter)"/>
    </xsl:variable>
    <xsl:element name="field">
    <xsl:attribute name="name">
    <xsl:text> childObject</xsl:text>
     </xsl:attribute> 
     <xsl:value-of select="$aChild" /> 
     </xsl:element>
        <xsl:call-template name="printChildObjects">
           <xsl:with-param name="inputString" select="substring-after($inputString,$delimiter)"/>
           <xsl:with-param name="delimiter"
           
          select="$delimiter"/> 
            </xsl:call-template>
            </xsl:when> 
             <xsl:otherwise> 
             <xsl:choose> 
                <xsl:when test="$inputString != ''"> 
                <xsl:element name="field"> 
                <xsl:attribute name="name"> 
                <xsl:text> childObject</xsl:text> 
                     </xsl:attribute> 
                     <xsl:value-of select="$inputString" />    </xsl:element> 
             </xsl:when> 
             <xsl:otherwise>
         </xsl:otherwise> 
        </xsl:choose> 
      </xsl:otherwise> 
   </xsl:choose>
</xsl:template>

xsl:text must not contain child elements

Some info from old xsl cheetsheet.

Often there is a requirement to print a variable value in output.
Following xsl code will display compilation errors.
Simple workaround is pass it to a template

<xsl:text>

<xsl:value-of select="$objType"/> </xsl:text>

<!-- <xsl:call-template name="varValue"> <xsl:with-param name="value" select="$objType" /> </xsl:call-template> -->

<xsl:template name="varValue">

<xsl:param name="value" />

<xsl:choose> <xsl:when test="$value = ''"> <xsl:text>n/a</xsl:text> </xsl:when> <xsl:otherwise> <xsl:value-of select="$value" /> </xsl:otherwise> </xsl:choose> </xsl:template>

Typical industry search input requirements/ patterns

An object string property contains all possible values.
Now I want to filter the list with different kind of input patterns (aka single field search) In particular accepting wildcards (*,? & + etc escaping is fun in java.)
sample code:

String str="*+PATAC+*";
Pattern pat=Pattern.compile(".*\\+*\\+.*");

Matcher matcher=pat.matcher(str);
boolean flag=matcher.find(); // true;

Logger.println("1) matcher result->"+flag);
if ( flag == true)
Logger.println("pattern found"+str);

str = "adjkfh+PATAC+ajdskfhhk";

matcher=pat.matcher(str);
flag=matcher.find(); // true;

Logger.println("2) matcher result->"+flag);
if ( flag == true)
Logger.println("pattern found"+str);


str = "PATAC";

matcher=pat.matcher(str);
flag=matcher.find(); // true;

Logger.println("3) matcher result->"+flag);
if ( flag == true)
Logger.println("pattern found"+str);

str = "adjkfh+PATAC+";

matcher=pat.matcher(str);
flag=matcher.find(); // true;

Logger.println("4) matcher result->"+flag);
if ( flag == true)
Logger.println("pattern found"+str);

str = "+PATAC+testingsuffixchars";

matcher=pat.matcher(str);
flag=matcher.find(); // true;

Logger.println("5) matcher result->"+flag);
if ( flag == true)
Logger.println("pattern found"+str);

Sample code to create SOLR document from CSV file

Just one guy stopped by office & asked to index this legacy data. So quick & dirty solution is the following. (May be Perl etc are there, but this one gives me more flexibility.

public class CsvToSolrDoc
{
public String columnName(int i)
{
//workarounds workarounds
if ( i == 0) return "id";
if ( i == 1) return "what ever you want as field name";
return null;
}
public void csvToXML(String inputFile, String outputFile) throws java.io.FileNotFoundException, java.io.IOException
{
BufferedReader br = new BufferedReader(new FileReader(inputFile));
StreamTokenizer st = new StreamTokenizer(br);
String line = null;
FileWriter fw = new FileWriter(outputFile);
// Write the XML declaration and the root element
fw.write("\n");
fw.write("\n");
while ((line = br.readLine()) != null)
{
String[] values = line.split(",");
fw.write(" \n");
int i = 1;
for ( int j=0;j it is length; J++)
{
String colName = "field name=\""+columnName(j)+"\"";
fw.write("<" + colName + ">");
fw.write(values[j].trim());
fw.write( "\n");
}
fw.write(" \n");
}
// Now we're at the end of the file, so close the XML document,
// flush the buffer to disk, and close the newly-created file.
fw.write("\n");
fw.flush();
fw.close();
}

public static void main(String argv[]) throws java.io.IOException
{
CsvToSolrDoc cp = new CsvToSolrDoc();
cp.csvToXML("c:\\tmp\\m2.csv", "c:\\tmp\\m2.xml");
}

SOLR project stories. Lack of SOLR post filter support

For past few quarters, I am working on a project to implement security on object documents. My goals is Decorate every SOLR document with ACL field. This ACL field used to determine what users have access to this document. ACL syntax is something like +u (dave)-g(support) etc. My thoughts are process these ACL fields after search aka I want to subject the query component results via some kind of post filter. However SOLR is not offering any direct mechanism to specify this kind of post filter along with the search request. At Lucene level, there is an option to specify the filter however current AS IS implementation, it sucks.
It iterates entire document sets. For large documents this sucks. Also we need to SOLR distributed capabilities. Also during computing the ACL fields, I tried to encode users names etc with Base64, URLEncoder.encode etc. For small set of strings, this is working Ok. But for large sets, it is a pain. Ultimately affecting the search performance.
Another blocker.
Encode/decoder tes code.
startTime = System.currentTimeMillis();
String inputText = "Hello#$#%^#^&world";
for (int i =0;i<50000;i++)
{
String baseString = i+ " "+inputText;
encodedText = URLEncoder.encode(baseString,"UTF-8");
decodedText = URLDecoder.decode(encodedText, "UTF-8");
}
endTime = System.currentTimeMillis();

elapsedTime = endTime - startTime;

System.out.println( "\n URLEncoder/decoder Elapsed Time = " + elapsedTime + "ms");

>>>>
Elapsed Time = 2246ms

Few Inspiring Quotes from The Great Ones The Transformative Power of a Mentor

Adapt the pace of nature: her secret is patience – Emerson

Watch your thoughts; they become words
Watch you words; they become actions
Watch your actions; they become habits
Watch you habits; they become character
Watch your character; it becomes your destiny. – Unknown

I hope I shall always posse’s firmness and virtue enough to maintain what I consider the most enviable of all titles, the character of an Honest man – George Washington

The person who makes a success of living is the one who see his goal steadily and aims for it unsercingly. That is dedication – DeMille

What we say and what we do
Ultimately comes back to us
So let us own our responsibility
Place it our hands And carry it with dignity and strength -Anzaldua

Simple SOLR example code to index & search metadata

Roughly three years back, I did some a prototype to index/search unstructured content inside enterprise. At that time we are using Autonomy. So nobody cared. Basically all search engines does some kind of structured content searches & major focus on unstructured content i.e. PDF files to WS word document etc. So coming to the point, again I was asked to some prototyping. This time I am looking to solr framework because my C++ business tire makes requests to index/searches for content (mostly un structured content.) after downloading Solr, looking for some good examples. At the end, I started writing the following. I will port this kind of example with using CURL.

  protected URL solrUrl;
  public CommonsHttpSolrServer solr = null;

  public SolrServer startSolrServer() throws MalformedURLException,
  SolrServerException 
  {
     solr = new CommonsHttpSolrServer("http://localhost:8983/solr/"); 
     solr.setParser(new XMLResponseParser()); 

     return solr;
  }
   
   public TestQueryTool()
   {
          //will add more junk later
   }

 
   public static void main(String[] args)
   {
     
     TestQueryTool t = new TestQueryTool();   
     try
     {
            //1)start & work with existing SOLR instance
   t.startSolrServer();
 
   // 2)Now index content ( files later. metadata)
   t.indexBOAsDocument("uid1","bolt", "0001");
   t.indexBOAsDocument("uid2","nut", "0002");
   
   //3)now perform search
   t.performSearch("uid");

  } catch (MalformedURLException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  } catch (SolrServerException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  }
     

 }

  
 private void indexBOAsDocument(String uid, String id, String name)throws SolrServerException 
 {
  SolrInputDocument doc = new SolrInputDocument();
  
  doc.addField("id",  uid);
  doc.addField("part_name",  name);
  doc.addField("part_id", id);

  try {
   solr.add(doc);
   solr.commit();
  } catch (IOException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  }
 }

 
 public void performSearch(String qstring) throws SolrServerException 
 {
  SolrQuery query = new SolrQuery();
  //query.setQuery( "*:*" );
  
  query.setQuery("id:uid* OR part_name:nut");
  
        query.addSortField( "part_name", SolrQuery.ORDER.desc);
  QueryResponse response = solr.query(query);
  SolrDocumentList docs = response.getResults();
  info("docs size is ="+docs.size());
  Iterator iter =docs.iterator();

     while (iter.hasNext()) {
       SolrDocument resultDoc = iter.next();

       String part_name = (String) resultDoc.getFieldValue("part_name");
       String id = (String) resultDoc.getFieldValue("id"); //id is the uniqueKey field
        // Now you got id, part_name. End of story
   }
       
      }

Nice little book "The Great Ones" by Ridgely Goldsborough

Books come with two different stores. One side authors’ journey before, after a mentor & his own life changes and simple set of guide lines & the other side a simple fictional story. Through the tales, the author suggests importance of mentoring, benefits in ones life virtually every aspect of life. Many inspiring quotes, compact size & fictional story also effective one.

Following are simple code of conduct

1.Make a decision & commitment
2.Conceive and execute a plan
3.Take full responsibility
4.Embrace patience and temperance
5.Act with courage
6.Cultivate passion
7.Exercise discipline
8.Remain single minded
9.Demand integrity
10.Let go of past failure. ( mostly learn from them)
11.Pay the price

Interesting free book “Best Kept Secrets of Peer Code Review”

http://smartbear.com/codecollab-code-review-book.php

Order your FREE book with FREE shipping.
Also you can view sample chapters. Some of the point are interesting.
if you are doing code reviews or senior developer, take some time to read it.

I ordered one & after receiving the book, I will update this blog.

Dhanvi & Saketh still from 2009 India trip

Both Dhanvi & saketh likes the Mango a lot. However this is the first time, we took them to our own garden. Before going the garden, Dhanvi was very curious about the Mango tree & he says, will count all the mangos. At the end, after seeing the garden, he was tired in counting the number of trees itself & thrilled to see the groups of mangos handing at so low to the tree. ( see the back ground)

Long list of books waiting to be read - 1

Book title: What color is your parachute? 2010 by Richard N. Bolles

Nearly decade back, I read this book.
Now I picked the latest 2010 (Hard Times) edition.
During my Boston to Los Angeles transition, I looked at this book for some guidance.
Similar to old editions, book sticks to core ideas. However presentation & reference material was changed a lot. I am totally surprised to the information like 100 of job listing sites, starting home business etc. To some extent I am not following the latest trends. I have a job which I love & I do with tons of commitment. Because of this reason, last ten years, I never entertained any carrier change. So I am not aware of changing trends in Job markets. It does not matter if you are looking for a job or not; still it is worth reading.
Topic “Thing schools never taught us about job hunting” is always my favorite.

An interesting quote from the book
He or she who gets hired is not necessarily the one who can do the job the best; but, the one who knows the most about to get hired. – Richard Lathrop

Interesting little book “ReWealth”

Book: ReWealthby Storm Cunningham (Author)
Published by McGraw-Hill

Last weekend picked it from library. Contains very good information & what it takes to attain real sustainability.
I am not done but very interesting tales & full of quality quotes.
If I find more time, I will post more complete review here.


Give a man a fish, and you’ll feed him for a day.
Teach a man to fish, and he’ll buy a funny hat.
Talk to a hungry man about fish, and you’re a consultant.
—Scott Adams

The nation behaves well if it treats the natural resources as assets which it must turn over to the next generation INCREASED . . . in value. (emphasis added)
—Theodore Roosevelt

We really do not know how [the economy] works. . .
The old models just are not working.
—Alan Greenspan, former chairperson of the U.S.

If we become rich or poor as a nation, it’s because of water.
—Sunita Narain

Brief Kiva two years update.

I discovered Kiva in April 2007 via a PBS, tpt, Frontline program. Every month I am contributing little bit of my $ for good cause. To date, my Kiva loan portfolio contains over 225 loans, including 100 loans that have been repaid completely and then re-loaned & I reached my little goal of loaning 100 entrepreneurs always.

Birthday video

Still experimenting with my new Canon 5D & trial versions of photo story software’s. Following video is compressed from 120M in to 3.5 MB with audio added.

Eclipse Ganymede C/C++ CDT IDE platform & my first impressions

Steps

1.Make sure you have a stable JDK on your machine ( in my case, I am using JDK 1.5
2.Download MinGW from http://www.mingw.org/
& install (make sure enabling g++ installtion option)&
update your system PATH setting

After step 2, in your windows command prompt
if you type g++ -v it will display the following.
>>>
C:\a_eclipse_c_c++\eclipse-cpp-ganymede-win32\eclipse>g++ -v
Reading specs from C:/MinGW/bin/../lib/gcc/mingw32/3.4.5/specs etc. etc.
>>

3. Download eclipse & CDT environment from the following location.
http://www.eclipse.org/cdt/index.php
4. After installing eclipse, start eclipse.
5. Start with File->new->project->c++ give default name etc.
6. At this stage you are ready go with new a class or import some existing file.
7. If you are experienced user with Eclipse IDE(i.e. for java code) building & running the code is very easy. Else your have play with Run-> new configuration.
Unfortunately Eclipse-> Help Content sucks.
I finished from step(2) to step(7) in less than 30 minutes. I wrote few c & c++ programs & everything looking good. More later.

Integrate Full Text search functionality in to your applications with Lucene (searching)

Part 2: Now Full text searching.

public List search(){
List searchResult = new ArrayList();
IndexSearcher indexSearcher = null;

a) Get Indexer
b) Build default query parser
( since it was demo. Accept all wild cards characters in your inputs.

It is pure fun see how Lucene is responsing to wild queries.)

c) Now search the indexes.
( we already built them in the earlier step)

Hits hits = indexer.search(query);

My conclusions are
Lucene is a Pure Java product that provides:
* ranked searching ; best results returned first{ i need to test more here}
* Good numbers of query types:
phrase queries, wildcard queries, proximity queries, range queries etc
* fielded searching (e.g., title, path, contents)
* date-range searching
* sorting by any field
* multiple-index searching ( I am working on this one right now)
* allows simultaneous update and searching
I am looking forward to C/C++ implementation.

I love c++ stuff because it fits in our product stack very nicely.

I am wring custom code to parse our source code (C,C++ & CORBA)

Soon i will update this blog with key code blocks.

Integrate Full Text search functionality in to your applications with Lucene (indexing)

Part 1) Indexing your data.

Any Full Text search functionality involves indexing the data first. Lucene was no different in this approach. By indexing your data, it can perform high-performance full-text searching very fast. I did indexed 17,000 html files (my product documentation) in less than 5 minutes.

Creating Index writer & adding documents methods are key.
Rest f the methods for book keeping.
Following code indexes html, htm files in a folder. (It recursively iterates the nested folders & indexes each file)


////you data
public static final String dataDir = "D:\\webapps\\help";
//the directory that is used to store lucene index
private final String indexDir = "D:\\help_index";

public static String src1 = "";
public IndexWriter indexWriter;
public static int numF;
public static int numD;

public void openIndexWriter()throws IOException
{
Directory fsDirectory = FSDirectory.getDirectory(indexDir);
Analyzer analyzer = new StandardAnalyzer();
indexWriter = new IndexWriter(fsDirectory, true, analyzer);
indexWriter.setWriteLockTimeout(IndexWriter.WRITE_LOCK_TIMEOUT * 100 );
}

public void closeIndexWriter()throws IOException
{
indexWriter.optimize();
indexWriter.close();
}

public void indexFiles(String strPath) throws IOException
{
File src = new File(strPath);
if (src.isDirectory())
{
numD++;
String list[] = src.list();
try
{
for (int i = 0; i < list.length; i++)
{
src1 = src.getAbsolutePath() + File.separatorChar + list[i];
File file = new File(src1);
/*
* Try check like read/write access check etc.
*/
if ( file.isDirectory() )indexFiles(src1);
else
{
numF++;
if(src1.endsWith(".html") src1.endsWith(".htm")){
addDocument(src1, indexWriter);
}
}
}
}catch(java.lang.NullPointerException e){}
}
}
public boolean createIndex() throws IOException{
if(true == ifIndexExist()){
return true;
}
File dir = new File(dataDir);
if(!dir.exists()){
return false;
}
File[] htmls = dir.listFiles();
Directory fsDirectory = FSDirectory.getDirectory(indexDir);
Analyzer analyzer = new StandardAnalyzer();
IndexWriter indexWriter = new IndexWriter(fsDirectory, analyzer, true);
for(int i = 0; i < htmls.length; i++){
String htmlPath = htmls[i].getAbsolutePath();
if(htmlPath.endsWith(".html") htmlPath.endsWith(".htm")){
addDocument(htmlPath, indexWriter);
}
}
return true;
}
/**
* Add one document to the lucene index
*/
public void addDocument(String htmlPath, IndexWriter indexWriter){
//System.out.println("\n adding file to index "+htmlPath );
HTMLDocParser htmlParser = new HTMLDocParser(htmlPath);
String path = htmlParser.getPath();
String title = htmlParser.getTitle();
Reader content = htmlParser.getContent();
Document document = new Document();
document.add(new Field("path",path,Field.Store.YES,Field.Index.NO));
document.add(new Field("title",title,Field.Store.YES,Field.Index.TOKENIZED));
document.add(new Field("content",content));
try {
indexWriter.addDocument(document);
} catch (IOException e) {
e.printStackTrace();
}
}

im.openIndexWriter();
File src = new File(dataDir);
if(!src.exists()){
System.out.println("\n DATA DIR DOES NOT EXISTS" );
return;
}
long start = System.currentTimeMillis();
System.out.println("\n INDEXING STARTED" );
im.indexFiles(dataDir);
im.closeIndexWriter();
long end = System.currentTimeMillis();
long diff = (end-start)/1000;
System.out.println("\n Time consumed in Index the whole help=" +diff );
System.out.println("Number of files :\t"+numF);
System.out.println("Number of dirs :\t"+numD);
}

Populating MySql table from MS Excel { aka .csv} file

I am working on small project for a non profit organization.
{I felt Apache, PHP & MySQL combination fits their need.
I will explain about that application little later.}

I have already received some data in MS Excel file.
Strangely some trailing columns are missing in some records after saving Excel file in to csv file.

Following command fails saying column truncated MySQL errors.

mysql> load data infile 'C://bea//temple//dpexport.csv' into table donar
_info_4 fields terminated by ',' OPTIONALLY ENCLOSED BY '"' Lines terminated by
'\n';

After spend little more time with MySQL documentation,
I found out that IGNORE option does the magic &
I am able to load the csv files in my SQL table. Just add IGNORE next to infile.

Amazon Kindle is the next IPOD?

Few days back, I was little early to movie & I was waiting, sitting on a sofa, outside the theater.
Suddenly a young man sat next to me, seriously browsing web with his Amazon Kindle. (What made me curious was, he was reading one of my favorite web site, New York Times.) As I was paying attention to gadget , slowly he started talking all the good things about his new gadget & offered me to feel it. Quickly I checked the weight & look and feel of a blog & an e-book. Not so heavy & look and feel was very good & natural. I liked it a lot. I was so tempted after coming home, I checked Amazon website for Kindle.
(Little pricy, It was sold out & seems to be there is lot of demand for the gadget) Most of the reviews are positive. This incident remembers me another old incident with first experience with IPOD. Nearly4+ year's back, while coming back from my vacation, (India->London-> US), one person sitting next to me was explaining & talking & proud of owning a new IPOD. (If I remember correctly, it was the second month after IPOD launch.) Now I am seeing the same thing happening for Amazon's Kindle. I liked the way it was designed (automatic download content to the device)& convenience & thought process behind the Kindle. I was so surprised that Amazon came up with this kind of device. {After its initial launch as a major on line book store, this was the best thing from Amazon. My personal opinion. } Hoping that I will own Amazon Kindle very soon. It fits in to my taste.

An update & my progress in Kiva

Today I received an e-mail from kiva with happy Holidays message. Felt I like I need update my blog about my progress as a small kiva lender. I discovered Kiva in April 2007 via a PBS, tpt, Frontline program. I updated this same blog about this one. I was amazed by the Kiva founder’s sprit, website and the ease of loaning $25 to entrepreneurs around the world. I felt this is the best way to help “poorest of the poor” or "unlucky souls". I felt this is the right thing to do, So I was contributing to kiva whatever left in my savings account. (At the end, I am an immigrant here and working hard for my living.) I used to donate some good $ to few charities but for some reasons I felt this is the best way to make people responsible. { Donation is creating more dependency.} The fact is that I received no interest on my $ loan was immaterial. The important thing was being repaid in most cases. To date, my Kiva loan portfolio contains over 59 loans, including 5 loans that have been repaid completely and then re-loaned & I reached my little goal of loaning 50 entrepreneurs always. Now I am targetting 100 entrepreneurs in the next two years.
Following is my kiva lender page http://www.kiva.org/lender/dhasa

Java DefaultListModel performance issues

Recently I received a big customer escalation on search domain.
Basically end user was searching for enterprise information based on end user input criteria.
In our java rich client, we are showing a simple dialog to select the list of users in the enterprise. This action was consuming 10+ minutes.
Real culprit was UI works fine for simple 100 to 1000 users.
However customer is testing with 10K plus users.
After analyzing the all the code at server side & finally I looked in the client layer.
At client, server data is getting added to DefaultListModel with addElement() in a for loop.
Real culprit is addElement() method.
After seeing the implementation of the above method & its sequence of event calls &
Little bit of browsing the java forums, I found out that we should use the above class for large lists. Yes never use DefaultListModel directly. Still this problem exists in JDK 1.5 version. There are multiple solutions to this problem. Just Google it. You will find many.
I made a quick fix based on some suns forum advice. (Basically it is fast & I am seeing 90% improvement)

Steps:

1) Extend your DefaultListModel as shown below

class FastListModel extends DefaultListModel { private boolean listenersEnabled = true;
public boolean getListenersEnabled() { return listenersEnabled; }
public void setListenersEnabled(boolean enabled) { listenersEnabled = enabled; } public void fireIntervalAdded(Object source, int index0, int index1) { if (getListenersEnabled()) { super.fireIntervalAdded(source, index0, index1); } }
}

2) Add list listener to your list model

ListDataListener listener = new ListDataListener() { public void intervalAdded(ListDataEvent e) { list.ensureIndexIsVisible(e.getIndex1()); } public void intervalRemoved(ListDataEvent e) { } public void contentsChanged(ListDataEvent e) { } }; model.addListDataListener(listener);

3) Turn on & off listener explicitly

model.setListenersEnabled(false);

//add content for(int i = 0; i <>

// now enable the listers

model.setListenersEnabled(true);

New Java Runtime methods

If you building complex Java client & eclipse RCP based clients, always use
Java Runtime methods like maxMemory(), freeMemory() totalmemory() etc. to know your application or your module memory usage. It always helps.
Also use availableProcessors () elegantly if you application is spanning too many threads.

hooray XSLT now part of JDK 1.5

XSLT now part of JDK 1.5 itself.
(Thanks god, we don’t have to download xalan, xerces etc.
& setting big class paths to author style sheets.)

Minus points are Java implementation of xalan sucks badly.
It is working fine for simple transformation use cases &
but failing for the complex cases.

(I will write those test cases in the next post)

Following is the sample Test.java to transform the input xml in to HTML ( or another other output) using java Xalan libs.

import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.Source;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;
import org.w3c.dom.Document;
import java.io.FileInputStream;
import java.io.FileOutputStream;

public class Test
{
public static void main(String[] args) throws Exception
{

Source source = new StreamSource(new FileInputStream("C://AE_html.xsl"));

Transformer t = TransformerFactory.newInstance().newTransformer(source);

Document doc = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(new FileInputStream("c://users.xml"));

System.out.println("Transforming...");
t.transform(new DOMSource(doc),new StreamResult ( new FileOutputStream("C://users.html")) ); }
}


//System.out.println( "doc as text content"+ doc.getTextContent() );
//t.transform(new DOMSource(doc), new StreamResult(System.out));

MN State Fair pictures continued

We spend lot of time at pet animal section. Both Dhanvi & Saketh enjoyed a lot.
I observed big change in Poultry section. ( i.e. lots of new Rabbits. This change is good.)

MN State Fair pictures

Few pictures from flower stand. Compared to last year display, this year flower stand is less attractive. But a good one. I simply love this one. A must watch for me.

State Fair pictures

We went to State Fair over weekend. It became a ritual to visit State Fair every year.
Few picture from the fair. I will post more later.

More online references

Few more alternatives to wikipedia (www.wikipedia.org) I am finding more useful
Scholarpedia
www.scholarpedia.org

Conservapedia
www.conservapedia.org

Citizendium
www.citizendium.org

Became a member in kiva

As usual, yesterday, I was watching PBS documentaries.
But following program was really inspiring. (It is a small part in Frontline)

(A Little Goes a Long Way)
Watch it online first.
http://www.pbs.org/frontlineworld/stories/uganda601/

Immediately decided to be a member in kiva.
I believe in the concept (microfinance & direct lending & to the needy.)
I watched many programs on microfinance.
But I never know how to be part of it or Can an individual can join.
www.kava.org helped me to contribute.
At present I am thinking of contributing to the same for another one year.
Started with 6 & my goal is help 50 all the time.
Please read the FAQ etc in the www.kiva.org before jumping.

“It's a new, direct and sustainable way to fight global poverty, and the way I see it, I get a higher return on $25 helping someone build a future than the interest my checking account pays. “

Following is my lender page in kiva.
http://www.kiva.org/lender/dhasa

Saketh stills for passport applications. continued.

How to pull that camera?

See saketh stills for passport applications. continued.

How to stop these guys?

See saketh stills for passport applications. continued.

No more photos for the day. Got it.

See saketh stills for passport applications. continued.

How is my new still?

See saketh stills for passport applications. continued.

Why so many photos. I am sad but i dont give up.

See saketh stills for passport applications.

After struggling for 1/2 hour, finally his mom gave up for the day.

Saketh's first birthday photos

Saketh's first birthday invitation card

Created with using three jpg's.
Used photobuilder trial version software for this one.
Birthday photos will come soon.
I need to create web images.

Another Interesting Quote

Good programmers write good code. Great programmers write programs to generate it.
—Unknown

Interesting Quote

To change and to change for the better are two different things.

Funny but real

I believe everybody in the world should have guns. Citizens should have bazookas and rocket launchers too. I believe that all citizens should have their weapons of choice. However, I also believe that only I should have the ammunition. Because frankly, I wouldn't trust the rest of the goobers with anything more dangerous than [a] string.
—Scott Adams

Quotes of the Day

You cannot believe in God until you believe in yourself.
Swami Vivekananda
There is no security on this earth; there is only opportunity
Science without religion is lame, religion without science is blind.
Albert Einstein

Free & simple web picture software

it is free & very easy to use it & works with very small memory footprint.

New set of Dhanvi & Saketh photos

My Take on movie 'Heat"

Watched it on DVD over weekend.
Two great actors (Al Pacino & Robert Niro) acted quiet brilliantly.
It is a story about two people on the opposite of law.
Al Pacino as cop and Niro as the bad guy.
Movie starts with Niro & his team steeling some bonds from a firm in day light in LA. Next scene onwards Al Pacino investigation etc.
Director(Michael Mann) portrayed both characters quiet deeply.
The best part is confrontation between Al Pacino & Robert Niro.
I expected an alternate ending. (Let Niro go free or escape.)
A good & worth watching thriller movie.
This movie goes to my list of all time favorite thriller movies category.

My Take on movie “The insider (1999)”

Watched it on DVD over weekend. Quiet perfect drama.
Two great actors (Al Pacino & Russell Crowe) acted quiet brilliantly.
It is based on true story about a whistleblower who knows too much about inside of tobacco industry. Al Pacino acted as CBS 60 Minutes producer who will try to air whistleblower story on the TV at any cost. Whole movie is about these two people struggle to air the truth.
Again it is not an action movie but drama is so intense.
A good & worth watching movie.

Quotes of the Day

Justice Delayed is justice denied.

Men marry because they are tired; Women because they are curious. Finally both are disappointed.

Of there is a wrong way to do something, then someone will do it.

Lot of free photo prints

Click the following link to get nice package of free photo prints + shipping from shutterfly. https://www.shutterfly.com/secure/sign_up_childmag.jsp?http=www.shutterfly.com&re=/childmag/add_ppc_code.jsp

Use code "30UP-ROMI" to get additional 30 ( 4X6) prints.
Ordered nearly 100 photos for $10.

My take on “The Bourne Identity” movie.

This weekend both kids are sick , I stayed at home & sometimes working.
I watched The Bourne Identity and few kid’s Pooh movies. Just to give company to Dhanvi.

Coming to movie, this is a very neat and intelligent movie about a CIA agent. Everything perfect, no flaws really. Director showed Actor Matt Damon in a different dimension.
Movie starts with a body floating in sea water, with two bullets and lost memory.
As movie progresses, it is all about his past and identity and his life. Action sequences are quiet good and feel quiet natural and script is fast paced. No dull movement at all.
I regret not watch this movie so far. I heard it is good but never thought this good.
100% awesome movie for Spy/Action/Thriller fans.

My take on Dell Inspiron 600m Laptop

Couple of weeks back, I received my new Dell inspiron laptop 600m from Dell. Configuration is bit powerful. (I took it because I have big plans for video editing. Also it has 1 GB Ram with DVD R/W burner. CPU is 1.7 GHz mobile Pentium.)

I used it & like it lot. As usual I check CNET editor/user reviews before buying any gadgets. (I don’t know any other big user community.)

As far as laptop top is concerned, it is good. I love it a lot. It is light weight, thin laptop.
Also it is inexpensive compared to any other brands for the same configuration. (I paid $650 for this little machine.) Battery life is pretty good. (Nearly 3 hours.)
Lot of people complains about this. But I am very happy for batter life. I don’t know why anyone wants more than this. It came with NEC DVD burner. It is dam good. Working flawlessly. See my other post to see how I am burning DVDs.
The only negative point I see is, bottom left part of the laptop is pretty warm after couple of hours + usage. Seems to be hard drive is getting warm. For me it is OK. It is very rare for me to open laptop for couple of hours at home with kids playing around me.
Also I need to check inbuilt wireless and its performance. Sometimes it disconnects without any error messages. I need to check with dell fourms/support on this.
Other than couple of minor issues, if you want a nice, powerful laptop with in budget, I will definitely recommend dell 600m.

Dhanvi like number “3”

Last night when mama is giving some biscuits to Dhanvi, he is insisting I want 3.
In fact I observed this for quiet some time. He is always asking for quantity 3
I got little curiosity and asked why you like number “3”
Guess his response
.
.
.
.
.
.
.
.
.
.

because I am 3 years old.

Free Excel viewer

Want to open,view and print Excel files?
Try the above link. It is free.

It is the best option, if MS excel software is not installed on your machine.
Again it will not allow you to author excel documents.

Crazy Barnes & noble

Couple of week back, I ordered books from Barnes & noble (http://www.bn.com/) online store. Basically some clearance is going on right now & I ordered some stuff for my kids & few books for myself. (Story books, sticker kits, activity books etc.) Most of them are $1 & $2 books

Guess what, yesterday I received 11 different boxes from bn.com via ups. I was shocked seeing so many boxes. I don’t know why they did like that. Sometimes for multiple copies of the same item, they shipped the same item in different packages. It is truly stupid. If you to go US post office to mail those packages, usps will charge more then $5 per package. My order worth is $50 + No shipping. Assume bn gets deep discount(50%) from ups, still bn pays more than $25 to shipping only. Boxes, package material, laser printer based invoices and return slips consume the rest the of $25 not considering labor costs etc. I was so amazed the way bn online store business works. Instead of selling like this, they can donate to some schools or libraries for tax write off, it is much better & profit to bn core business. I feel this not the way to run a business. What do you think?

Again Barnes & Noble is my favorite store. Every week, I will visit at least once.
My comments are about their online order process system only.
If you are books lover, buy top 100 bargains from the following link.

http://www.barnesandnoble.com/saleannex/top100_cds2.asp?PID=4030&start=1&userid=yS36ilqCNH&cds2Pid=4030


Tail piece: I opened the all boxes one after the other to check the books. Dhanvi watched all the story, activity books once and put them one side and started playing with the boxes for whole two hours. (Assuming them as boats, brick etc.) One of my frind used to joke, if you take birthday gift to kids, kids play with the gift grip wrap more than the original toy. Same case with my kid too. Hoping that he will enjoy his new books, activity stuff etc.

Quotes of the Day

Idleness is sweet, and its consequences are cruel.
-John Quincy Adams (unverified)

It is better 100 guilty persons should escape than one innocent person should suffer.
-Chanakya Indian politician, strategist and writer

Since 1996 onwards I am using my e-mail.
I received so many e-mails with lot of their favorite quotes.
From here onwards, I will update my blog with all of my favorite quotes collection.
(It is huge and good collection (?).)

My way of DVD buring

- Rip with Shrink or Decrypter and remove any audio streams and/or subs
that I don't need
- Remove all the other crap I don't want with VobBlanker
- Fix the menus with MenuEdit if you want
- Burn with Nero
Most of the software is freeware & Nero software is available from www.download.com
(Trail version.)
I will add more links later.

Perfect & innovative movie "Memento" 2000

As I mentioned, I love movies. I always think, why not make a movie which role backs? (Again it is not flash back way of telling the story.)
Just try to tell the story for 10 mins. Then go back to another 10 minitues for what happened before and so on. This idea struck me after watching pulp fiction movie. But somebody else got the same idea & made a good thriller movie “Memento”. Plot is very simple. A revenge story. But as explained earlier, movie shows ending scene first and goes back wards. Perfect & innovative way of telling the story.

Dhanvi with santa

Dhanvi with santa.

This year santa gave lot of toys to Dhanvi.

Dhanvi in women's dress

After lot of mama's hard work, I am looking like Indian women?

Saketh as bala krishna

Saketh as Bala Krishna

dhanvi,saketh with mama

My wife always plays a lot with kids. Some times she roles, jumps like kids.
Sometimes all of them sit on the same chair. Following is an example. All are sitting in a laundry basket. Seems to be it is lot of fun. But I am always a silent watcher working with my Nikon 3700 camera.

Curious Saketh

Curious Saketh always lifts blinds to see outside. (Even in the dark.)
Only God know his findings.

Last few weeks films viewed:

(Out of 5 stars)
Batman begins (****)Best popcorn movie of this year.
Usual suspects (****) Good thriller.

I will write my take later. Too busy enjoying my vacation with kids.

Last weekend movies

Saw (2004)

It is quiet perfect thriller. (Sometimes disturbing too.)
Movie start with two men in old, abandoned, large bath room with their legs tied with a dead man in between. A note in their pocket says that you have kill the other person. It was a game played by a serial killer. Movie ends with who survived in that game.
Direction & script is very fast paced and hold you in your seat for whole two hours.
Very gripping screenplay. I watched many thrillers (silence of lambs, seven, bone collector etc) but this movie ending was awesome & stunning & twisting one.
Only negative point in this movie is, director did not explain why serial killer is doing all this. Seven, Bone collector etc shows their characters differently.
Movie ended in such a ways that there another part in is on the way.
It is very low budget movies & a big hit.
A worth watching for thriller fans.

I watched another awesome movie “Leon The professional” movie too.
I will add review later.

Chanakya (Indian politician, strategist and writer, 350 BC-275 BC) quotes

I always admire Chanakya's work.

{Right from 12 years old. Very important person in Indian history. I forgot almost all the history but not his name & his work.}


Few of his quotes


1) The biggest guru-mantra is: Never share your secrets with anybody. It will definitely destroy you.

2) There is some self-interest behind every friendship. There is no friendship without self-interests. This is a bitter truth.

3) A person should not be too honest. Straight trees are cut first and honest people are screwed first.

4) Before you start some work, always ask yourself three questions - Why am I doing it, What the results might be and Will I be successful. Only when you think deeply and find satisfactory answers to these questions, go ahead.

5) As soon as the fear approaches near, attack and destroy it.

6) The world's biggest power is the youth and beauty of a woman.

7) Once you start a working on something, don't be afraid of failure and don't abandon it. People who work sincerely are the happiest.

8) A man is great by deeds, not by birth.

9) Treat your kid like a darling for the first five years. For the next five/ten years, scold them. By the time they turn sixteen, treat them like a friend. Your grown up children are your best friends.

10) Books are as useful to a stupid person as a mirror is useful to a blind person.

11) Education is the best friend. An educated person is respected everywhere. Education beats the beauty and the youth.

12) Even if a snake is not poisonous, it should pretend to be venomous."

I will add more later.

My take on movie "Crash" (2004)

Tag line: You think you know who you are.

Movie start with cars crash on LA suburbs.
Immediately movie goes back to last 24 hours of
people who met this crash and lives.
Movie got very smart & intelligent script showcases not only characters,
but real representation of real people.

Shot in the background of LA
(Where you will see white, black, Asian, spanic and French etc)
live together & their personalities and racist feeling w.r.t others & on the system.

Again movie has everything.
Right & brilliant script, good casting & apt locations & good photography etc.

Very worth watching. Dont miss it.

My failed DVD recorder hacking story

During black Friday, I purchased Liteon DVD recorder.
It is working well. Easy to setup & records favorite TV shows.
But my main objective to buy this recorder is to make a copy of costly DVDs.
My little Dhanvi sometimes handles DVDs badly. Sometime he is very quiet nice.
For his age, he is doing great in handling DVDs. He select his own title from the collection & plays. We have to see what he is doing. He is quiet perfect on this.

Idea is to backup my own DVDs. So I searched net for any hacks.
I found following discussion forum for this.

http://club.cdfreaks.com/showthread.php?t=124298&page=3&pp=25

I tried it yesterday. But my hack attempt failed.
It is not working on my model Liteon LVW-1001HC1.
What a bad-luck. What a sad ending.

Wow. "Family Guy, Vol. 2: Season 3" Boxed DVD set for $10. Hurry

Ordered "Family Guy, Vol. 2: Season 3" boxed DVD set today from www.fye.com.
Check deal of the day section.
It is very good deal. ($30-$20 rebate.)
But for $10 you really can't go wrong.
Quiet neatly produced series. Sit, relax & enjoy.

Careful with blockbuster online rentals.

Couple of days back, on their website, I searched for movie “LEON”
and search resulted in many matches.
(Basically I saw picture of the DVD cover and read Leon: the professional etc.)
I selected one & added to my queue.
Yesterday I received movie from them & played DVD around 10PM.
Guess what happened. This is a bonus disk.
Contains some experts from stars, director, some other movie previews etc.
Not the movie, not even deleted scenes from the movie.
I am very disappointed.
Later checked blockbuster website.
It was mentioned that in title that it was Bonus disk only.
I am really surprised that people do rent “Just Bonus Disk?”
My wife scolded me. Don’t you read the complete title properly?

I said why somebody rents Bonus disk for movie price?
It is a supplementary DVD right?

My theory was, if you go to shop & buy the DVD movie you will get movie + bonus disks as a package. So if I am renting, I must receive the whole package.
But my theory was dead wrong with this.

This is first time I am renting DVD movies online, so be careful with the titles.
Blockbuster ships “Bonus” disk as movie for $.
Clever business or I am not knowledgeable to understand things properly?

So next time, if you rent movie, make sure you are ordering correct one.
Else you will end up with bonus DVD.
A crazy online DVD rental experience. What about you?

Hurray. Ordered "Die Hard" Boxed DVD set.

Ordered "Die Hard" boxed DVD set today during lunch time.
It is very good deal. ($30-$20 rebate.)
I am not fan of pure action movies or popcorn movies or Bruce.
But for $10 this is worth set to own.
Waiting and searching for good deal on Night Shyamalan DVD package deal.

Bad Weather & Dull weekend

Weather is too cold (upper 10’s F). This is my 5Th year in MN. But this is one of the worst December months. Single digits are common in Jan but not before new year eve. Want to go to shopping to Bestbuy but decided not to. Looking for good External hard drive & Laptop internal DVD recorder.
Did not do much except playing with kids.
Saketh started sitting on his own & he is trying to standup on his own.
Just playing with kids for the whole day & reading the pioneer press news paper.

Click following link too see MN Weather.
http://www.twincities.com/mld/twincities/13332539.htm
Minnesotans awoke Monday to the state's first hard cold snap of the winter, with below-zero temperatures in many parts of the state.............

Interesting Upcoming Movies

Went to Barnes and Nobel Friday night story time with family
Dhanvi & mama went to story time,
I am with saketh reading few magazines.

Seems to be the following forth-coming movie is very interesting.

King Kong from Lord of rings director, Naomi Watts (Ring fame Rachel) etc.
Production pictures are looking very good.
Click link http://www.kingkongmovie.com/ see to more details.

Just released

Syrian
Syrian script written by Traffic movie writer.
(Traffic is one of my favorite movie. )

Friday night movie Heavenly Creatures(1994)

I don’t know why this movie in my watch list. (May be Oscar nominated)
It is story about two close girl friends story.
The friendship so close is that, when their parents try to separate them,
they plans and kills one of their mother.
It is based on a true story happened in 1950's.
Again movie has great casting, locations, photography etc.

Don't rent it.
Safely you can borrow from your public library and watch it.
It is 100% for Drama lovers.

Recent photo with saketh

Dhanvi in Halloween Dress.

My younger son "Saketh" photo

Bedtime Story "Bad Lion" for Dhanvi

This is one of the bed time stories for my elder son Dhanvi.
This is my childhood text book story. Converted slightly for young boy’s purpose.
There is one big forest & one big bad lion. It is bad because it kills all animals. Since it is strong animal, no animal can fight with him. So all animals decide & go to him & say, lion don’t kill all of them. We will make turns & per day one animal will come to your home & u can eat. No more hunting for you. Bad lion accept it. Because it don’t have to run & fight with others. So per agreement, animal makes turns.

At first “A” like alligator, “B” like bear etc…. (I will try go with alphabet sequence. For most of the alphabets, he will say the animals.) When finally it is rabbits turn. Rabbitt is small animal but it is clever animal. So rabbit thinks, thinks and finally goes to bad lion home very late.

Bad lion is very hungry and angrily says why you are so late?

Then rabbit says “I saw another lion on the ways. It took lot of time to escape from that one”
Then lion says ‘Another lion in this forest? I am the king” Show me where it is?
Then rabbit takes him to one old well & says “it is in that well”
Bad lion climbs the well wall & sees inside.
I will ask Dhanvi what bad lion will see? He say a loud, its image.
Now story continues as
“Bad lion is strong but not clear” So it think that is another lion inside well.
So it says “I am the king of the forest. So surrender me else I will kill you”
But the same words comes back.
So angrily bad lion jumps in to well & what happens to bad lion?”
Dhanvi says “bad lion chachipoyiii." (Telugu word for "killed")
I will ask him why? He says “bad lion don’t know swimming” &

he continues " But I am learning. Now I am at ell stage."

(He remembers his swimming level.)
Normally I will stress him about “alphabets and animals” You have to be clever like rabbit. Think, think etc. Later swimming stuff. It is fun.
Slowly we move to another story.

My take on Ring Two (2005) movie

I liked Ring 1 a lot. But in part 1 raises more questions than answers.
In particular girl (samara story) the video tape origination and Why now suddenly she started killing? Part 1 did not answer those questions. I watched second part with some hope on some of these questions.

Now coming to second part review , Rachel & her boy moves from Seattle to Oregon & she get job in local new paper. Hoping that they will have better life. In the next scene, again young couple watching TV, video tape & death. (Very much similar to part 1.) Next scene onwards, director shows samara alive in the new place too. Later some of Rachel’s actions make her anger too. Now samara enters in to her son’s body. (These sequences of shots are good.) Now Rachel thinks that better to end samara before it is too late. She goes to original places and investigates Samara story. i.e. where she came from. (Story revolves around before samara adoption ) & movie ends with killingor stoping of samara.
A last major dialog was I am not your fucking mummy.

Part 2 address some of questions but again raises more questions, if samara is looking for Mummy and her love. I think the step mom have already provided that stuff. (In part1, they show stopmom tries a lot for a baby then finally, she adopts samara.) Again why suddenly Samara starts her evil acts? What prompted those? Part2 shows that a video tape spreads around the country and everyone makes a copy? If so, no investigation by any?
Samara original mom dialog “listen to baby”. She says, she tried to kill samara because she asked her to do. But they show samara as infant. Samara mom dairy content is confusing.
Like this many are unanswered.

Now coming to movie, there is not much plot and not much shock like part1. Nearly five & six shots are worth scaring. Actors did good job. In particular Rachel role is too good. Other casting, locations, photography etc are good & apt. Again do not except too much in part2.

At the end, I felt it is different way of story telling with lot of questions & answers left for viewer’s imagination.