Final updates on my Kiva journey

 Following are final statistics. ( ~15 years of journey.)  

Nowadays, this year, I am unable to find anytime.

Following is public profile.

public profile

Nayagara pictures (First time i am able to capture rainbow just before ride)

 

Home Depot Absurd Pricing Example

See the enclosed.
 

A snapshot of EV vehicles location

Generated as part of EdX Python analysis exercise. (generated from of session logs)
 

Hive Timestamp column, default query results behavior, common misunderstanding and formatting to a target zone

    Similar to many RDBMS, Hive Timestamp datatype and it is stored as bigint and are stored as an offset from the UNIX epoch. Usually Timestamps are interpreted to be timezone-less. However end user runs a Hive query (via Hue or any tools), it returns the columns in the default server time zone. (in our case, EST ad team started saying your app is   not saving data in UTC, we need UTC format etc. etc.  After spending sometime with Hive language manual and with some of the past issues, I found an work around via Hive JIRA.

Idea of this query is vehicle sensor data is stored in data table.

Intent of the query is to know what is the value of a sensor on a given day.

select  ID, element_cd,

from_utc_timestamp(to_utc_timestamp(from_unixtime(unix_timestamp(timstm_column) ),'EST'), 'UTC'),

element_val

from  data_elements

where yr_nbr = 2019 and mth_nbr = 11 and day_nbr in (30)

and element_cd='POWER_LEVEL'

Lot’s of indirection. i.e. taking timestamp column & formatting it to unix timestamp and then first converting to default time zone and then to target time zone.  ( Above query returns timestamp value as UTC formatt.  My  HIVE servers are running in EST)

A diversified portfolio: 5% Inflation-Adjusted Income & moderate growth For Life

Usual concept is to hold 60% in stocks and 40% in bonds. While there are different methods/selections in to which stocks/bonds, I am using following allocations in 401K for each ongoing monthly contributions. Following are list of funds with contribution amounts.

     VEMAX( Vanguard Emerging Markets Stock Index Fund Admiral Sh )  5%

   VGENX ( Vanguard Energy Fund Investor Shares )   5%

   VGHCX ( Vanguard Health Care Fund Investor Shares ) 20%

   VGSLX ( Vanguard Real Estate Index Fund Admiral Shares) 10%

   VFIAX ( Vanguard 500 Index Fund Admiral Shares) 20%

   VWINX    Vanguard Wellesley Income Fund Investor Shares  (20%)

   VUSTX   Vanguard Long-Term Treasury Fund Investor Shares (20%)

No	Fund name	Annual dividends
1	VEMAX	4
2	VGENX	4
3	VGHCX	10
4	VGSLX	6
5	VFIAX	6
6	VUSTX	4

                                         Total 34/6 ~6%

Again I am not reinvesting all dividends automatically. All dividends sits in Vanguard money market account  with 1.X% monthly return. Again my intent is use this money in case of a deep corrections in the markets.  So far, last year above portfolio gave 5K dividends and moderate growth of 8%. Again last year is an exception with all time highs, but for past decade, my other portfolio is returning above average returns.  In my other portfolio, I am using 80% stocks and 20% bonds. ( other one is IRA and I am using little bit individual stocks too.  But same set of funds with different compositions.

The possible range of expected annual portfolio returns for the given period is 7.11% to 12.15%. 

I am not a financial analyst. I am not an expert in money, monetary policy, investment theories, theorems. My blog article should not be taken as advice, financial or otherwise, or as a statement of the right or proper way to do things. What I am is a regular IT guy, middle-aged, with a life full of past mistakes, most of which, I hope, I have learned enough from to continue forward in a positive direction. 

A long term IRA Portfolio Journal

I made plenty of mistakes my first year, primarily through trading and not investing. Again I joined stock markets race during Y2k melt-up situations. Everyone is talking about stocks during break time & this is first job in USA & took trading route. Just to make some quick bucks here.  But after 15K losses, With marriage, my philosophy has changed. All investing is limited to spouse IRA and my 401K only.  It essentially remains thus: build a core of 5 diversified funds with low expense ratio and 10 dividend paying equities, reinvest the dividends and allow them to grow. I don't have a lot of hard and fast rules for my holdings other than they must pay at least 5%. Also if a stock/fund give more than 25% of profits, sell 50% of position and lock the profits.

Currently, the retirement Portfolio consists of five funds and 20 odd stocks. Most of the stocks picks are 5 years old. I am holding some of these funds for 15+ years.

List of funds are:

Ø  VEMAX( Vanguard Emerging Markets Stock Index Fund Admiral Sh )

Ø   VGENX ( Vanguard Energy Fund Investor Shares )

Ø   VGPMX ( Vanguard Global Capital Cycles Fund )

Ø   VGHCX ( Vanguard Health Care Fund Investor Shares )

Ø   VGSLX ( Vanguard Real Estate Index Fund Admiral Shares)

Ø  VFIAX ( Vanguard 500 Index Fund Admiral Shares)

List of stocks includes

BEP

BIP

AMZA

REML

DX

CHI

GUT

ARCC

CII

EXG

APLE

BGR

RRC

PCI

AM

VEON

BGCP

AWP

CHMI

AMLP

GDX

I am not a financial analyst. I am not an expert in money, monetary policy, investment theories, theorems. My blog article should not be taken as advice, financial or otherwise, or as a statement of the right or proper way to do things. What I am is a regular guy, middle-aged, with a life full of past mistakes, most of which, I hope, I have learned enough from to continue forward in a positive direction. I am certainly not an expert in anything, but I am willing to learn, to ask right questions and take some risks.

At the end of December, the IRA is 20 years & I portfolio returned ~4.5% over last 20 years. But my point with this blog spot is long term view really works.  Regularly invest from paycheck and diversify and stay calm and ignore all the noise. Also always keep 5% of money in cash form. Sometime, if you find real value in anything, this helps.

2019 Summer Everglades summer trip and some advice

Will post entire trip details this week.

But for now, i will start with few pictures.

What's the biggest financial mistake you see many people make all the time?

Back when I was 28 years old, year 2000, I was making nearly ~100K minimum a year in IT field.

Every weekend, visiting A place for entertainment & spending, buying cool gadgets(At some point 3 TVs, Desktop, laptops, half dozen audio players etc.) On tops, based on  TV news, jumped in to Y2K stock market madness. But when stock market crashed, newly married, after a loss of job with 10K credit card debt, and on H1B Visa. Many sleepless night for short few weeks.

Luckily I found a job & saved me the entire scene.

Thinking backwards, I acted so dumb. Buying stocks based on news ( with margin). In my case, i used credit card money. plane DUMB.  So much of stuff for no reason.

fast forward 2019……..

Today, I am more comfortable.

 I learned how to save, invest, buying absolutely required stuff. More later

Funny amazon prime deals ( so far)

See the pictures below.
Why in the world, someone buys 3LB?
Jackup price and give 35% off.

My first Rome trip (between July 3ed to 6th) Day-1 & 2

Following is our complete trip summary

1) 3ed. We started from London Gatwick Airport and reached Rome around 10.30

I made earlier hotel reservation around  Vatican . So i at the airport,  we are walking towards bus station. However, on the way,  i nice shuttle operator stopped me and explains, i will offer $10 Euro ride to Vatican which cheaper than the train option. So i took Shuttle service for $40 ( family of 4)
We reached our apartment place. After quick refresh/baths and quick lunch ( yummy Pizza) we headed to Vatican  Museums
( i made reservation when i was USA, Following is the website. https://biglietteriamusei.vatican.va/musei/tickets/do_

Since, i have reservation, i simply walked directly to the entrance. 
( if you dont plan a head, then you will struck in long lines for 2+ hours. Lines under direct sun light.
Outside, i noticed lot's of people advertising, no lines. direct check etc for extra $)

Lot's of rush but we managed to enter the Museum by 2.30 PM. ( my actual timing was 3PM. But no one care. So many people, all they care is the entrance ticket)

Few advice to family folks. ( before entrance, eat a lot. + carry lot's of liquids. Water etc. 
During our visit time, temperature is around 33 degree C)

We stayed 3 hours in the  ( there is a short cut to reach  Vatican Basilica by skipping 40% of museum. But we choose to watch all rooms as much as possible.)

around 7PM we reached our hotel room. this 10 minutes walk to Vatican/Vatican Basilica)
After some refresh, we went to dinner.  Again pizza/ drinks for dinner.)

We wrapped our first day like this

2ed Day:
by 7.30 we finished our breakfast and walked to St. Peter's Basilica)
No many people even on Wednesday 7.30 AM.
We did spend quality time in the St. Peter's Basilica and outside. Nearly two hour gone just like that.

Later we walked to nearby but-stop and tourist center to purchase Roma passes. ( these centers are open from 9.30 AM.   Caution: they will not accept Cash So you have use Credit card only. For parent we bought Roma passes and for kids, i took 3 day bus passes only.  Kids under 18 are free for most of the attractions in our list. So we choose Roma passes for parents only.)
After that we picked a bus to Termi station and later one bus to catch Triva fountain.  (Initially finding bus numbers/stop is little tricky. But my older son cracked the code. Will write more on that topic later.)

2018 summer vacation UK/ROme

Location:British Museum

Interesting  idols from INDIA/south Asia  gallery

.

2018 Summer UK/Rome trip cntd 2

Location:Stonehenge   Day-3

Will add more later

2018 UK/Rome trip - part 1

Location: Windsor Castle , UK.
Day 1

I will add more content later.

Windsor Castle Guard change demo (only one guard)

VIN anonymization

Data anonymization is a type of information sanitation whose intent is privacy protection. It is the process of either encrypting or removing personally identifiable information from data sets, so that the people whom the data describe remain anonymous.
This is a sample to anonymize VINs i.e. systems sends vehicle data however it is difficult to trace-back to the original VIN except the source system,

Old code.. ( may be usefull. When i did this POC, i tested for 1 millions VINs.
SHA-256 hash function really worked well for this set. Never tested this for 20 million.. But based on N grams this model will work


import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.io.StreamTokenizer;
import java.io.Writer;
import java.util.*;

import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;


///goal is to read  millions of vins and aonymization the last 7 digits.
///this way VIN & its data can be share to anyone. however souring systems needs to trace back to 
// original VIN ... Steps... Use Java crypto API hash function w.r.t. simple java function
//So far no collisions on one million.. in case collisions, 
///thoughts are generating deterministic N grams based on hashed string..

public class VINAnonymization  {

   
   public static void main(String[] args) throws IOException {
   String stringToEncrypt ="FIXME";
       int ctr = 0;
    BufferedReader br = new BufferedReader(new FileReader("D:\\demo\\lvins.txt")); 
    
  try{
  
       MessageDigest messageDigest; 
       messageDigest = MessageDigest.getInstance("SHA-256");
       HashMap anomVinMap = new HashMap();
   StreamTokenizer st = new StreamTokenizer(br);
   String line = null;

   int loop =0;
while ((line = br.readLine()) != null){

   messageDigest.update(line.getBytes());
   byte[] mdbytes = messageDigest.digest();

      StringBuffer hexString = new StringBuffer();
    for (int i=0;i
        hexString.append(Integer.toHexString(0xFF & mdbytes[i]));
    }
    
    String anoVin = null;
  String nGramString = LRNGramString(hexString,7,0);
  if ( nGramString == null){
  System.out.println(loop+++";"+hexString.toString());
  continue;
  }
  anoVin = line.substring(0,9).concat(nGramString);
  if ( anomVinMap.containsKey(anoVin) ) {
  System.out.println("Collionson for a VIN-->"+anoVin+ "K:"+0);
  ctr++;
  ///Do this for two times... generate 3 ot 10 N grams in single call  & loop it
  //For now this is OK
  nGramString = LRNGramString(hexString,7,1);
  if ( nGramString == null) continue;
  anoVin = line.substring(0,9).concat(nGramString);
  if ( anomVinMap.containsKey(anoVin) ) {
  System.out.println("Collionson for a VIN-->"+anoVin+ "K:"+1);
  }else{
  anomVinMap.put(anoVin, line);
  }
  ctr++;
  nGramString = LRNGramString(hexString,7,2);
  if ( nGramString == null) continue;
  anoVin = line.substring(0,9).concat(nGramString);
  if ( anomVinMap.containsKey(anoVin) ) {
  System.out.println("Collionson for a VIN-->"+anoVin+ "K:"+2);
  }else{
  anomVinMap.put(anoVin, line);
  }
  }else{
  anomVinMap.put(anoVin, line);
  }
    messageDigest.reset();
  }

///Dump the map to CSV file or text file for comparisons
  System.out.println("Anonymization VIN map count"+anomVinMap.size());
  String eol = System.getProperty("line.separator");

  try (Writer writer = new FileWriter("D:\\demo\\out1.csv")) {
  for (Map.Entry entry : anomVinMap.entrySet()) {
    writer.append(entry.getKey())
          .append(',')
          .append(entry.getValue())
          .append(eol);
  }
  writer.flush();
  writer.close();
} catch (IOException ex) {
  ex.printStackTrace(System.err);
      }

  

} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally{
if (br != null ){
br.close();
}
}

         
   }
   
   //validation later
   static String LRNGramString (StringBuffer orgString, int legth, int position ){
   if ( orgString == null ) return null;
   return orgString.substring(position,legth).toUpperCase();
   
   }

}