2018 summer vacation UK/ROme

Location:British Museum

Interesting  idols from INDIA/south Asia  gallery

.

2018 Summer UK/Rome trip cntd 2

Location:Stonehenge   Day-3

Will add more later

2018 UK/Rome trip - part 1

Location: Windsor Castle , UK.
Day 1

I will add more content later.

Windsor Castle Guard change demo (only one guard)

VIN anonymization

Data anonymization is a type of information sanitation whose intent is privacy protection. It is the process of either encrypting or removing personally identifiable information from data sets, so that the people whom the data describe remain anonymous.
This is a sample to anonymize VINs i.e. systems sends vehicle data however it is difficult to trace-back to the original VIN except the source system,

Old code.. ( may be usefull. When i did this POC, i tested for 1 millions VINs.
SHA-256 hash function really worked well for this set. Never tested this for 20 million.. But based on N grams this model will work


import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.io.StreamTokenizer;
import java.io.Writer;
import java.util.*;

import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;


///goal is to read  millions of vins and aonymization the last 7 digits.
///this way VIN & its data can be share to anyone. however souring systems needs to trace back to 
// original VIN ... Steps... Use Java crypto API hash function w.r.t. simple java function
//So far no collisions on one million.. in case collisions, 
///thoughts are generating deterministic N grams based on hashed string..

public class VINAnonymization  {

   
   public static void main(String[] args) throws IOException {
   String stringToEncrypt ="FIXME";
       int ctr = 0;
    BufferedReader br = new BufferedReader(new FileReader("D:\\demo\\lvins.txt")); 
    
  try{
  
       MessageDigest messageDigest; 
       messageDigest = MessageDigest.getInstance("SHA-256");
       HashMap anomVinMap = new HashMap();
   StreamTokenizer st = new StreamTokenizer(br);
   String line = null;

   int loop =0;
while ((line = br.readLine()) != null){

   messageDigest.update(line.getBytes());
   byte[] mdbytes = messageDigest.digest();

      StringBuffer hexString = new StringBuffer();
    for (int i=0;i
        hexString.append(Integer.toHexString(0xFF & mdbytes[i]));
    }
    
    String anoVin = null;
  String nGramString = LRNGramString(hexString,7,0);
  if ( nGramString == null){
  System.out.println(loop+++";"+hexString.toString());
  continue;
  }
  anoVin = line.substring(0,9).concat(nGramString);
  if ( anomVinMap.containsKey(anoVin) ) {
  System.out.println("Collionson for a VIN-->"+anoVin+ "K:"+0);
  ctr++;
  ///Do this for two times... generate 3 ot 10 N grams in single call  & loop it
  //For now this is OK
  nGramString = LRNGramString(hexString,7,1);
  if ( nGramString == null) continue;
  anoVin = line.substring(0,9).concat(nGramString);
  if ( anomVinMap.containsKey(anoVin) ) {
  System.out.println("Collionson for a VIN-->"+anoVin+ "K:"+1);
  }else{
  anomVinMap.put(anoVin, line);
  }
  ctr++;
  nGramString = LRNGramString(hexString,7,2);
  if ( nGramString == null) continue;
  anoVin = line.substring(0,9).concat(nGramString);
  if ( anomVinMap.containsKey(anoVin) ) {
  System.out.println("Collionson for a VIN-->"+anoVin+ "K:"+2);
  }else{
  anomVinMap.put(anoVin, line);
  }
  }else{
  anomVinMap.put(anoVin, line);
  }
    messageDigest.reset();
  }

///Dump the map to CSV file or text file for comparisons
  System.out.println("Anonymization VIN map count"+anomVinMap.size());
  String eol = System.getProperty("line.separator");

  try (Writer writer = new FileWriter("D:\\demo\\out1.csv")) {
  for (Map.Entry entry : anomVinMap.entrySet()) {
    writer.append(entry.getKey())
          .append(',')
          .append(entry.getValue())
          .append(eol);
  }
  writer.flush();
  writer.close();
} catch (IOException ex) {
  ex.printStackTrace(System.err);
      }

  

} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally{
if (br != null ){
br.close();
}
}

         
   }
   
   //validation later
   static String LRNGramString (StringBuffer orgString, int legth, int position ){
   if ( orgString == null ) return null;
   return orgString.substring(position,legth).toUpperCase();
   
   }

}

Google finance "Create Portfolio from quotes"

For a while this feature, it is not working. Multiple time, i reported to the help section documentation  & as feedback. I guess no one is looking in to end user feedback.. If this is the case, why asking. Not sure................

https://finance.google.com/finance/portfolio/create?hash=XzZuRXIyV1NZYkgzSV9YY1UxdmFvS3NKWnJ3fDE1MTQwODgyNTY&ei=QCc_WrneEsywmAGV7Z_4DQ

400.
That’s an error.

The requested URL was not found on this server.
That’s all we know.

“The Disaster Artist “ Incredible homage to the best bad movie of Hollywood

The film is a story of friendship between Tommy and Greg from bay area film school.

Greg is baby faced young actors dreaming of big carrier in Hollywood and Tommy odd looking middle aged rich guy dreams of Hollywood. One fine day, they both movie to LA in pursuit of their Hollywood. After multiple unsuccessful attempt to get in to acting, finally they decide the make a movie “The Room”. Rest of the plot is around filming of the ‘The Room” movie. Tommy writes, produces and finally acts a lead actor in his story. Inspired based on 2003 Hollywood independent movie and subsequent book, this is a perfect homage. Loved every scene & one of the best comedy for this year. (at the end, during credits, they show both 2003 movie and 2017 movie scenes next to each other. Simply mind-blowing.)

emirates airlines USA to India air travel..(Inconsistent & poor Baggage allowance practices)

Last month, I went to India  ( from Austin,TX to Hyderabad)
My reservation confirmation e-mail says no fee for two check in bags and
no fee on CARRY-ON HAND BAGGAGE  & NO FEE CARRY ON PERSONAL ITEM.

From Austin to New York, travel is fine. From New York to Dubai is fine
However i am about to chicken Dubai to Hyderabad flight. 
At this point of time, group of people are weighing the carry-on & handbags and saying limit is exceeded & you have to pay.  My bad, i purchased few novels and books for reading/some food.. So i have to pay or drop 2 pounds of stuff.
I told, except books, i am carrying the same stuff from USA. but he is not listening. 
I clearly see racial profiling.  He simply allowed an american in the next line  without much screening.
Really felt bad for my situation. End-up throwing few items in to trash, ( finished novel, towel etc)

Now coming back from Hyderabad, again problem at Hyderabad. He simply saying, you have to take only one bag. No CARRY ON PERSONAL ITEM.. (I am carrying I7 based laptop ( little heavy) with charger, external mouse etc.) there is no space in other carry-on bag. basically this is small one even to put 15" laptop..
 This sucked so bad. end-up carrying laptop in the carry-on. ( leaving all laptop  bag etc.)

Emirates airlines needs to think/implement differently.

Seems to be "CARRY ON PERSONAL ITEM" is US domestic airline concept. If they are OK with this item in US outbound flights & they need to implement all way to destination by tagging or something else.
In my case, Dubai to Hyderabad. Emirate needs to act smartly for US outbound citizens.

In bound flight also, since the ticket is brought from US, they need to allow   "PERSONAL ITEM" from final destination.   My overall trip experience is spoiled with this kind of dump rules implementation.

2017 How not to do e-commerce (a case study)

This is based on my 2017 black Friday shopping experience.

Like many retails, frys.com, send promotion code to all the e-mail subscribers. So, I registered. Based on subscriber preference frys.com send daily e-mails. Up to this part, everything is OK.

For 2017 black Friday time also, I received e-mail with promo code. Both my kid’s laptop is not working so I decided to buy couple. Based on e-mails, I picked two laptops and I placed orders on 11/23/2017. After 10 or 15 minutes, I received an e-mail saying your order is ready for pickup. We are closed on thanks giving day & pick order on 11/24/2017.  I thought, wow. next day, I received the store around 10.15AM, went to straight checkout register line. After waiting for 90, minutes, I reached cashier and showed my order number. He took, my info went to back room and came back said, no one picked your items. Let me go and pick your items. After 40 minutes, he came back and said, one of your item is out of your stock. (standing with family, I was bit shocked.) finally paid bill and went to manager who is standing at service desk.  I show the order pick e-mail etc. He called someone & after 15 minutes, he said we are out of stock. Apologies. I told, if come next week, will you honor the prices. He said no. we don’t… So, dumb… while, I was standing, another customer came and he is saying, I came after driving 45 miles and his item is not there.  (like me so many disappointing parents.) after seeing this, I felt old circuit city stores experiance.  Absolute mess.

1)     If you are offering online sales, make sure your order processing system works.  (if ORDER is ready for pickup means they have mean it.)

2)     In my case, my chaser completed only two transactions per hour. Frys.com employees need to go to Walmart/target/kohls store. How quick they do the job.

3)     Don’t merge online order customer with in store promotion chases. There is is reason, we placed order online and we simply want to pick and go.  If you go to Kohls, even they have special parking slots for online customers and separate service line to process only ONLINE order. Please learn.

I will append more items later

2017 Back Friday experience with family

For past decade, I am not doing any shopping (At least visiting the physical store.) 

If required, I am purchasing via online.  However this year, I decided to give some experience to the kids. (Now they are in high/middle School) & they need laptops. SO I felt, this is the right time.  Basically they  are curious about one days record sales.  On thank giving day, I purchased couple of laptop from frys.com and for pickup, we went to store.  I will write one more post on frys.com ( how not to do e-commerce in today's world) however coming to main topic, on Fridays stores are opened at 6AM. We reached the store around 10.15 AM. Parking is mess. Based on my past experience, I told wife & kids to pick whatever you want come to line. I went directly checkout line. This is long. After sting 90 minutes in the line, I reached, cash register. ( in between, family wrapped, their pick & joined me.  Absolute shocking thing is my online order is not ready.  Cashier took, my order number and went on picking items for next 40 minutes. After spending 40 minutes, he came back and said one of my laptop is out of stock.   Finally we are out of the store around 12.15… My kids first feedback it, let’s avoid Black Friday shopping & in particular Frys. Let’s go to lunch.  

My Kiva updates after a decade

Today at Kiva we celebrate YOU!

10 years ago today, you joined Kiva to change lives around the world.

PDF to CSV file generator

few years back, when I  was working on Apache TIKA POC/evaluation/bench mark for document indexing with SOLR,

At the same time, i was exploring pdf document parsing with apache.pdfbox framework.

At that time, i wrote sample code to see the PDF box capabilities.

Exactly last year, my niece asked some help in parsing the PDF file. 

I fixed the java file for her need. Keeping it here for any later uses. 

import java.io.*;

import java.time.Duration;

import java.time.Instant;

import org.apache.pdfbox.cos.COSDocument;

import org.apache.pdfbox.pdfparser.PDFParser;

import org.apache.pdfbox.pdmodel.PDDocument;

import org.apache.pdfbox.util.PDFTextStripper;

public class PdfToCsvGenerator {

    public static void main(String[] args) {

        String string = null;

        BufferedWriter out = null;

        PDDocument pdDoc = null;

        COSDocument cosDoc = null;

        PDFTextStripper pdfStripper = null;

        try {

            System.out.println("Processing PDF file. please wait.");

            Instant start = Instant.now();

            PDFParser parser = new PDFParser(new FileInputStream("C:\\xxx\files\\final_result.pdf")); //TODO

            parser.parse();

            cosDoc = parser.getDocument();

            FileWriter fstream = new FileWriter("C:\\xxx\\files\\out.csv"); ///TODO this is output file

            out = new BufferedWriter(fstream);

            pdfStripper = new PDFTextStripper();

            pdDoc = new PDDocument(cosDoc);

            pdfStripper.setStartPage(1);

            pdfStripper.setEndPage(3); //TODO for now parse only first two pages. once everything is ready..extend it for all

            String parsedText = pdfStripper.getText(pdDoc);

            String lineSep = pdfStripper.getLineSeparator();

            String[] lines  = parsedText.split(lineSep);

            for ( String aLine : lines){

                if ( aLine!= null){

                    String trimLine = aLine.trim();

                    if (trimLine.matches("^[A-Z].*$")) {

                        System.out.println("Ignoring input line:"+trimLine);

                    }else{

                        String htno=null,subcode=null, subname=null, internal=null, ext=null, credit = null;

                        StringBuffer buf = new StringBuffer();

                        if (trimLine.contains(" ")){

                         String[] fields =    trimLine.split(" ");

                            boolean isSubject = true;

                            int i = 0;

                            for ( String s : fields) {

                                if (i == 0) {

                                    htno = s;

                                    i++;

                                    continue;

                                }

                                if (i == 1) {

                                    subcode = s;

                                    i++;

                                    continue;

                                }

                                if (isSubject == true) {

                                    //StringBuffer buf = new StringBuffer();

                                    if (s.matches("[A-Za-z-&/]+")) {

                                        buf.append(s+" ");

                                        i++;

                                        continue;

                                    } else {

                                        subname  = buf.toString();

                                        isSubject = false;

                                        internal = s;

                                        i++;

                                        continue;

                                    }

                                }

                                ext = s;

                                credit = fields[fields.length - 1];

                                break;

                            }

                            out.write(htno + "," + subcode + "," + subname + "," + internal + "," + ext + "," + credit + "\n");

                           // System.out.println("htno:" + htno + " subcode:" + subcode + " subject name:" + subname + " internal:" + internal + " ext:" + ext + " credit:" + credit);

                        }else{

                            System.out.println("Ignoring 2ed stage input line:"+trimLine);

                        }

                    }

                }

            }

            if (out != null) {

                out.flush();

                out.close();

            }

            System.out.println("processing is complete. Check output file.");

            Instant end = Instant.now();

            Duration timeElapsed = Duration.between(start, end);

            System.out.println("Time taken: "+ timeElapsed.getSeconds() +" seconds");

        } catch (FileNotFoundException e) {

            e.printStackTrace();

        } catch (IOException e) {

            e.printStackTrace();

        } finally {

        }

    }

}

Few dumbest SOLR/Cassandra implementations

1      I was in the Architecture review board and a project came for review.

It is kind of upgrade or tech refresh project.

Core ides is they moving simple Web application is moving old version of OS, WebLogic (yes. still companies use this king of heavy weight containers

IT BOM contains SOLR too. (Moving from Solr 3.2 to Solr 6.2)

Since I am not well versed with business domain, after meeting. I was asking

What is the motivation to go to Solr 6.x?

Answer was simple refresh & may be we will use new features. Solr cloud etc.

I asked how it is deployed. Answer was SOLR is running 3 different contains separately and they are load balanced.  I was asking it is master/slave etc. Answer is no.

We index all the content separately and monthly we update the content on each web logic instance. I said, this is incorrect thing. Are you going to fix with new Cloud architecture etc?

Answer is no. Still 3 isolated SOLR instances running separately in separate Elephant container and load balanced. I was shell shocked.  

Pure dumbest to the core.

I will expand this post later with more dump roll-outs.

Walmart (Free Pickup + Discount) rocks

Lately I end-up buying too many items from Walmart with Free Pickup + Discount option.
For most of the standard products, walmart.com prices are good and above pickup/discount is too good.
Amazon became more of prime member only.

Few example buys:
1) Product Title

Schlage FE595VCAM716ACC

Amazon is offering this one for prime members for ~$80 and Walmart
offered at same product 

with free pickup/discount offer.  

Similarly CATAN game.
( free pickup/discount is too good. in all my purchase
 it is matching or beating amazon prices.

at the same time, it is fast for me.

Nowadays, Amazon became to cleaver for non prime members. ( for free shipping, it says 5 to 8 business days)

and on 7 or 8 days, it is shipping. Kind of luring to buy prime.

At this point, still i am debating myself. $100 for prime &
$100 for sam'sclub membership. this list goes on.

Dhanvi News in the Round rock ISD school news

https://roundrockisd.org/rrisd-students-succeed-at-texas-k-12-chess-championship/

Saketh's story

This is at the beginning of his 6th grade.
( first he wrote the same in to 5 rough pages then moved in to word.)


Gochaun was hunting in the forest, he was chasing some Ulies. Ulies are buffalo like creatures with the

ability to camouflage into the background. Gochaun had chased them all morning, and was

going to make his camp when he saw something glint behind a tree. He decided to go check it

out. When he got to the tree he was pretty sure the thing was an egg, but it looked nothing like

any egg he had ever seen. The egg was half a foot in diameter and was a perfect sphere. It was

pure white with thin blue stripes running down. He decided he would probably be able to sell it,

and if that didn’t work he would cook it himself. He went back to his house, his house was on

the outskirts of a small town called Fjod. He was getting ready to go to the store when he heard

something crack. He checked his bag and saw a small crack as small as a thread. The egg

suddenly started shaking and a web of cracks appeared on the egg. The tiny head of a lizard, no

a dragon poked out of a hole in the egg, then the whole body came out of the egg, there were

tiny spikes on the dragon’s back and it had translucent wings. He touched the head of the

dragon and a searing pain ran up his arm. He also heard a voice in his mind “I am Fuun” he

replied telepathically “so it is Fuun?”, but Fuun was already what seemed to be asleep. In the

midst of his shock Gochaun did not notice Tajun approach. “You have one of the last dragons”

said Tajun, Gochaun turned to face Tajun, “what do you know about dragons” asked Gochaun

“very little or very less, dragons do extraordinary things” replied Tajun. Gochaun remembered

the pain in his arm and looked at his arm, he saw a strange tattoo of a limbless dragon running

down his whole arm. “That is the mark of a dracolite” explained Tajun then Tajun lifted his

sleeve and Gochaun saw a similar tattoo but had fire “why is your tattoo different than mine?”

asked Gochaun “my dragon  is fire and your dragon is ice” replied Tajun. During the

conversation neither Gochaun nor Tajun noticed Fuun was freezing Gochaun’s hair. Gochaun

turned to check on Fuun and saw what Fuun was doing, immediately Gochaun turned his bag

upside down and Fuun glided out and landed at Gochaun’s feet “he is hungry” said Tajun and he

handed Gochaun a chicken. Gochaun held a little bit out in his hand and Fuun gobbled it up

hungrily. “More” said Fuun telepathically, soon Gochaun had fed Fuun the whole chicken then

then Fuun was satisfied. Gochaun and Tajun talked some more and Gochaun agreed to have

Tajun as his mentor. Dokun the lord of all evil heard the whole conversation and planned to

kidnap Fuun and use him to take over the world. The next day Gochaun had his first lesson with

Tajun and Fuun had his first lesson with Libai (Tajun’s dragon). The next week Dokun striked,

while Fuun was in the woods exploring Dokun sent his demons from the nether realm to restrain

Fuun’s power and take him to the dragon’s cage the only place that can restrain a dragon. Fuun

knew it was no use to fight back when he had no power. Fuun mentally told Gochaun what

happened and out of his rage Gochaun teleported to Dokun, and started to fight. Gochaun and

Dokun fought for 7 days and 7 nights. When finally Dokun teleported Gochaun to the Madaar

desert and Gochaun realized that he needed to control his rage. Gochaun tried to teleport to

Fuun but he could not lock onto Fuun. He realized he needed to break into the castle by force

and he needed the help of the dwarves who lived on the other side of the desert so he set off.

LOL.. My first saleforce search results error

New generation error page 

A relocation journey from MN to TX

I don’t want to discuss why I made decision to move to TX from MN. (But money was a not factor.)

However I was told that there is no state tax in TX, you will save a lot after relocation etc. 

So in this post, i will starts with finance discussions. ( I have to start with something.)

In MN I used to own a small home (1750 Sq. foot & I used to pay ~2K property taxes.1% of home value)

After moving to TX, I brought a home 3300 Sq. foot & now I am paying 10.5K as taxes on 440K home. (Around 2.5% of home value)

In TX, I am not paying any state tax on my full time pay check. 

On similar salary, I used to pay 6K MN state tax.  

In summary, property tax in TX alone equates property tax in MN + state tax.

For one year, I rented an apartment before buying home here in Austin, TX

Again if are compare renting at MN vs, TX. 

In a good neighborhood, rent is minimum $300 to $400, more on similar size apartment compared to MN. 

On average for 12 months, this is equals MN state tax.

Bottom-line, don’t assume you are going to save a lot by moving to TX if you want to send kids to good schools. (In both approaches i.e. via rent or home.)

Coming to public services, for example, applying to kids’ passport etc., I used to spend 10 minutes at MN post offices. It was very quick. Here in Austin, TX, it is more than 3 hours process. 

Same case with Vehicle registration. For some reason, smaller teams serves larger public. 

Particular if both parents works, it is kind of ½ day forced vacation situation for this kind of simple situation. 

On side note, on average gas is 30 cents cheaper compared to MN. (It all depends on how many miles you drive your car. I don’t drive a lot for work...)

Another amazing about Austin TX is, school kids play lots of competitive games right from primary school. For example, my kids started playing chess and now they compete at many schools. We drive a lot to Dallas, Houston or San Antonio. Not only Chess, there is lots of competition for Tennis, Swimming etc. If you want to expose kids how competitive the worlds is this is very good place.

few Java Util methods used in batch programs

During processing of csv files or mainframe dumps and feeding to oracle/mysql via spring batch etc.


public static  convertStringSetToString(Set inputStrSet, char delimiter) {
    StringBuilder sb = new StringBuilder();
    if (inputStrSet != null) {
      Iterator iters = inputStrSet.iterator();

      while (iters.hasNext()) {
        String stringValue = iters.next();
        sb.append(stringValue);
        sb.append(delimiter);
      }
    }
    return sb.toString();
  }

  public  String convertStringListToString(List inputStrList, char delimiter) {
    StringBuilder sb = new StringBuilder();
    if (inputStrList != null) {
      Iterator iters = inputStrList.iterator();

      while (iters.hasNext()) {
        String stringValue = iters.next();
        sb.append(stringValue);
        sb.append(delimiter);
      }
    }
    return sb.toString();
  }


    public  String arrayToString(T[] array) {
      if (array != null) {
        StringBuilder builder = new StringBuilder();
        for (int i = 0; i < array.length; i++) {
          Object object = (Object) array[i];
          builder.append(String.valueOf(object));
          if (i < array.length - 1) {
            builder.append(", ");
          }
        }
        return builder.toString();
      } ///if null return ""?
  }

LONE STAR OPEN SCHOLASTIC 2015

Following are few pictures from weekend "LONE STAR OPEN SCHOLASTIC 2015" 

Both kids did good.We are kind of stressed out ( Austin to Houston & returned by mid night)
 but final out come is OK.

Casis Elementary Scholastic Chess Tournament

Location:

Casis Elementary

2710 Exposition Blvd

Austin, Texas 78703

It is very long day but Saketh did very good with all his games.

Not only individual first prize trophy, he contributed maximum points to his school trophy 

His rank was increased more than 300 point with this tournament. 

This is the scary part…

Dhanvi did not perform well. (However he has tough lineup)

Few pictures.