Notice: 39% Increase in Monthly Insurance Premiums for Home and Auto Texas

Everything's Bigger in Texas, Including My Insurance Premiums hike:

From $380 to $530 aka 39% Price Increase! 

Waiting on other shocks.

Driving Safer with Smart Driver Scores: An Inside Look at Trip Scoring and Driver Behavior Analysis

 

In my earlier post, I mentioned that I work as a solution architect for a vehicle data platform. In this post, I will discuss how we use trip data information to score driver behavior, using what we call a "smart driver" score. Like a credit score, a driver score is an indicator of a driver's behavior, which can be used for things like usage-based insurance.

The term "smart driver" typically refers to an individual who can optimize their driving behavior to improve safety, fuel efficiency, and comfort. A smart driver might use advanced technologies, such as real-time traffic information or driver-assist features, to make informed decisions about how to operate their vehicle. They might also adjust their driving habits, such as reducing speed, avoiding sudden acceleration and braking, and following a consistent pace, to minimize fuel consumption and emissions.

Smart driver programs and initiatives are often designed to educate drivers about the benefits of safe and efficient driving, and to provide tools and resources to help drivers make informed decisions on the road. Some programs may also offer incentives, such as rewards or discounts, to encourage drivers to adopt safe and efficient driving practices.

In the context of connected and autonomous vehicles, the term "smart driver" may also refer to the vehicle's advanced systems and technologies, which are designed to optimize driving performance and provide a more comfortable and efficient driving experience.

The term "smart driver" is used by a variety of organizations and companies, including automotive manufacturers, government agencies, non-profit organizations, and technology companies.

Automotive manufacturers, such as Tesla and General Motors, may use the term to describe the advanced driver-assist technologies and safety features that they offer in their vehicles.

Government agencies, such as the National Highway Traffic Safety Administration (NHTSA) and the Environmental Protection Agency (EPA), may use the term to describe their initiatives to promote safe and efficient driving practices.

Non-profit organizations, such as the National Safety Council, may use the term to describe their educational programs that aim to promote safe and responsible driving practices.

Technology companies, such as telematics providers and software developers, may use the term to describe their products and services that aim to improve driving performance and provide a more efficient and connected driving experience.

Overall, the term "smart driver" is used to describe individuals and organizations that are focused on optimizing the driving experience, and promoting safe, efficient, and sustainable driving practices.

Core trip concepts, data collection, scoring methodology

When owner is driving vehicle from point A to point B, Ignation ON/OFF cycle,  smart driver system will rate your trip based on various factors such as the duration of the trip, the condition of the road, the traffic, and your driving experience. Here are some examples of factors you might consider when rating a trip:

Drive Time: How long did it take you to complete the trip? Was it faster or slower than expected?

Driving experience: Did you enjoy driving the vehicle? Was it easy to handle, or were there any challenges with maneuvering or controlling the vehicle?

Comfort: Was the trip comfortable? Did you experience any issues with the vehicle, such as discomfort from the seats, or noise from the engine or road?

Road condition: Was the road in good condition? Were there any obstacles, such as potholes or construction, that impacted your trip?

Traffic: Was there a lot of traffic on the road? Did you experience any delays or slowdowns due to congestion?

2023: The Year of Market Uncertainty?

The planet Saturn returns to the same position relative to Earth every 30 years, and it is believed that similar outcomes can be expected as seen in 1993 or 1963. These events may include an abundance of innovation, volatile markets, and instances of death and destruction. This is my first time tracing this phenomenon, and I am eager to see the outcome.

Currently, in USA, we are experiencing various financial conditions, such as high inflation and low employment rates. Need to final outcome.

In 1993, the global financial markets had a positive performance overall. The Dow Jones Industrial Average (DJIA) had a total return of 8.25% for the year, while the S&P 500 had a total return of 4.46%. The NASDAQ also had a positive performance, with a total return of 7.62%. The yield on the 10-year Treasury note also fell during this period, which generally indicates a strong bond market.

In terms of international markets, the Nikkei 225 in Japan had a total return of -0.31%, while the FTSE 100 in the United Kingdom had a total return of 3.57%. The DAX in Germany had a total return of 9.58%.

The U.S. economy continued to expand during this period, with the Gross Domestic Product (GDP) growing by 2.9% in 1993, and the unemployment rate falling to 6.9%. The inflation rate was also relatively low, at 3.0%.

It was worth mentioning that the interest rates were relatively high at that time, with the Federal funds rate at 3.25% by the end of 1993.

Overall, the financial markets in 1993 were characterized by steady growth, low inflation, and a strong bond market.

The Power of Data: Transforming Auto Insurance with Vehicle Management Systems

I am working in vehicle data management system design, development, and support.

While I can not specify fine details, (i.e. data used by specific auto insurance providers), data is being collected heavily and used by many.

1.     Risk assessment: By collecting and analyzing data about a vehicle's usage, insurance companies can better understand the risks associated with insuring that vehicle. For example, data about a vehicle's speed, acceleration, and location can be used to assess the likelihood of an accident occurring.

2.     Premium calculation: Based on the risk assessment, insurance companies can calculate the premium for insuring a vehicle, taking into account the specific risks associated with that vehicle.

3.     Fraud detection: By collecting data from onboard systems and other sources, insurance companies can detect instances of fraud, such as staged accidents or false claims.

4.     Claims management: In the event of a claim, insurance companies can use the data collected by the vehicle data management system to verify the details of the claim and assess the damage to the vehicle.

5.     Usage-based insurance: Insurance companies can offer usage-based insurance plans, which allow drivers to pay premiums based on the actual usage of their vehicles. By collecting and analyzing data about the usage of vehicles, insurance companies can determine the appropriate premium for each driver.

Here are some popular vehicle data signals that are commonly collected and analyzed by vehicle data management systems.

1.     Speed: The speed at which the vehicle is traveling can provide insights into the driving behavior of the driver and the likelihood of an accident occurring.

2.     Acceleration: The rate at which a vehicle accelerates can provide insight into the driving style of the driver and the likelihood of an accident occurring.

3.     Location: By tracking the location of a vehicle, insurance companies can determine the risk associated with insuring that vehicle, based on factors such as the road conditions, traffic density, and likelihood of accidents.

4.     Engine data: Engine data, such as RPM and fuel consumption, can provide insights into the health and performance of a vehicle, which can impact the likelihood of an accident and the cost of repairs.

5.     Brake data: By tracking the usage of a vehicle's brakes, insurance companies can assess the driving behavior of the driver and the likelihood of an accident occurring.

6.     Steering data: Steering data, such as the angle of the steering wheel and the force applied to the steering wheel, can provide insight into the driving style of the driver and the likelihood of an accident occurring.

7.     Fuel data: Fuel data, such as the amount of fuel consumed and the fuel efficiency of a vehicle, can provide insights into the driving behavior of the driver and the likelihood of an accident occurring.

2022 Stock Account Performance

 I used to be very conservative but sometime at the end of april, started expermenting with options & littlebit of crypto. end result is the following.   Need to see the rest of the years. good thing is option income via calls is good i.e. 2 to 3%.  For the rest of the year, i will continue the same. Still some capatial needs to be allocated. Hopefully put it work after Sept quarter/correction.

Sometimes slightly out of box thinking or outlier saves the entire portfolio

 I didn’t like the selection of funds that HAS provider offered me in HAS account.

So, I end up picking pimco commodity fund based on its past dividend returns.

See the result of my portfolio.  All my others account in RED, this is solid green. Amazing run plus great dividends. (May be at the end of Q3, will reduce the position size… I have a feeling, a great downturn ahead of us)

loving excalidraw

 noticed as part of a presentaion & I started using for my upcoming project. 

A case study on how to screw a great product & team

Business and systems architects got good vision on a critical enterprise data product. After some key decisions, everyone worked hard and initial few years later, a good product was deployed/operational.

Some wants to make/take it too great & at the same time, someone in the leadership thought, it is his baby and want to survive on it for rest of his life. so, he hired incompetent leaders to manage people.

This led to constantly delivered contributors are ignored but new commers are promoted. This news alone devastated the key guys. When they realized, they simply moved on within weeks.  Product is still evolving, and this happened. Now a good product became a mediocre product team and quality issues started (this is SAD & yet real story of a company which dreams BIG yet makes bad decisions.)

A  people leader needs to inspire, ( worst case stay neutral) else great turns in to shit quickly.

More later....

Apache Spark JSON pasing confusing errrors

input json:

{

      "shipping_address": {

        "street_address": "1600 Pen Avenue NW",

        "city": "Washington",

        "state": "DC",

         "type": "business",

         "additionalProperties": {

            "test": "one",

             "test1": "two"

          }

      }

}

spark code;

# File location and type

file_location = "/FileStore/tables/mock_example-1.json"

file_type = "json"

# CSV options

infer_schema = "false"

first_row_is_header = "false"

delimiter = ","

# The applied options are for CSV files. For other file types, these will be ignored.

df = spark.read.format(file_type) \

  .option("inferSchema", infer_schema) \

  .option("header", first_row_is_header) \

  .option("sep", delimiter) \

  .load(file_location)

display(df)

Since Spark 2.3, the queries from raw JSON/CSV files are disallowed when the referenced columns only include the internal corrupt record column

Solution: usuually spark expects one json message per line..

In general we use Notepad etc, to format JSON examples. ( just to validate the strucrue of the documents)... if U are saving formatted JSON, then spark will fail with the above error.

notepad JSON plugin offers compressJSON option too. so compress/save it. It works fine

Weekday ( Monday) pure EV averages distance driven and electric consumed usage

Based on 50K vehicles, average distance driven/kWh consumed.

In Box  plot form

Kwh consumption (data includes some outliers)

After outliers cleanup

Connected Car Q&A

 Everyday, I work on connected car data projects.

Lately few people repeatedly keep asking what it means. So I put brief Q&A. 

Something I am sharing. (small slice but hoping this is useful)

What is connected Car?

    A car with have access to the Internet and communicate with traditional automotive components, such as the engine and electronics, as well as the smart devices of a driver. All via telematics* system.

What type of car data are talking and how it is useful to a driver or vehicle owner or auto maker or 3ed party?

the most common use of car data is to improve the driving experience by collect the data about driver behavior events i.e., from ignition on to ignition off. This data improves following experiences for the driver.

·        Finding fuel location/battery charging station as needed

·        Local business searches and promotions

·        Journey route weather/traffic updates

·        Real-time data communications about any emergency situations (flat tire etc.), crash etc

·        Location sharing, fast theft response

·        Insurance discounts based on good driving behaviors/usage-based discounts

For vehicle manufactures

a)      Data helps to measure the performance/reliability of the vehicles. Data helps to pinpoint about unforeseen issue(s) with new & old vehicles. Data helps voluntary recall vehicles for specific issues.

b)     Data helps to catch fraudulent warranty clams/odo tampering issues

c)      Various service offering such as oil changes, end of brake pad changes etc.

d)     Offering customer services for example geo fence boundaries for family of drivers. With Teen drivers.

           For 3ed party companies:

     Vehicle location data helps in forecasting about live traffic conditions

    3ed parity insurance providers to offers discounts based on driving behaviors, usage.

      Automatic pothole information improves the road conditions

      Near real time weather data to forecast real time weather

 

Usually Telematic* systems are integrated with Satellite navigation systems and onboard computers and back office systems. Not only data collections, back office controls and refreshes the software inside cars too via over the air updates.

 

Final updates on my Kiva journey

 Following are final statistics. ( ~15 years of journey.)  

Nowadays, this year, I am unable to find anytime.

Following is public profile.

public profile

Nayagara pictures (First time i am able to capture rainbow just before ride)

 

Home Depot Absurd Pricing Example

See the enclosed.
 

A snapshot of EV vehicles location

Generated as part of EdX Python analysis exercise. (generated from of session logs)
 

Hive Timestamp column, default query results behavior, common misunderstanding and formatting to a target zone

    Similar to many RDBMS, Hive Timestamp datatype and it is stored as bigint and are stored as an offset from the UNIX epoch. Usually Timestamps are interpreted to be timezone-less. However end user runs a Hive query (via Hue or any tools), it returns the columns in the default server time zone. (in our case, EST ad team started saying your app is   not saving data in UTC, we need UTC format etc. etc.  After spending sometime with Hive language manual and with some of the past issues, I found an work around via Hive JIRA.

Idea of this query is vehicle sensor data is stored in data table.

Intent of the query is to know what is the value of a sensor on a given day.

select  ID, element_cd,

from_utc_timestamp(to_utc_timestamp(from_unixtime(unix_timestamp(timstm_column) ),'EST'), 'UTC'),

element_val

from  data_elements

where yr_nbr = 2019 and mth_nbr = 11 and day_nbr in (30)

and element_cd='POWER_LEVEL'

Lot’s of indirection. i.e. taking timestamp column & formatting it to unix timestamp and then first converting to default time zone and then to target time zone.  ( Above query returns timestamp value as UTC formatt.  My  HIVE servers are running in EST)

A diversified portfolio: 5% Inflation-Adjusted Income & moderate growth For Life

Usual concept is to hold 60% in stocks and 40% in bonds. While there are different methods/selections in to which stocks/bonds, I am using following allocations in 401K for each ongoing monthly contributions. Following are list of funds with contribution amounts.

     VEMAX( Vanguard Emerging Markets Stock Index Fund Admiral Sh )  5%

   VGENX ( Vanguard Energy Fund Investor Shares )   5%

   VGHCX ( Vanguard Health Care Fund Investor Shares ) 20%

   VGSLX ( Vanguard Real Estate Index Fund Admiral Shares) 10%

   VFIAX ( Vanguard 500 Index Fund Admiral Shares) 20%

   VWINX    Vanguard Wellesley Income Fund Investor Shares  (20%)

   VUSTX   Vanguard Long-Term Treasury Fund Investor Shares (20%)

No	Fund name	Annual dividends
1	VEMAX	4
2	VGENX	4
3	VGHCX	10
4	VGSLX	6
5	VFIAX	6
6	VUSTX	4

                                         Total 34/6 ~6%

Again I am not reinvesting all dividends automatically. All dividends sits in Vanguard money market account  with 1.X% monthly return. Again my intent is use this money in case of a deep corrections in the markets.  So far, last year above portfolio gave 5K dividends and moderate growth of 8%. Again last year is an exception with all time highs, but for past decade, my other portfolio is returning above average returns.  In my other portfolio, I am using 80% stocks and 20% bonds. ( other one is IRA and I am using little bit individual stocks too.  But same set of funds with different compositions.

The possible range of expected annual portfolio returns for the given period is 7.11% to 12.15%. 

I am not a financial analyst. I am not an expert in money, monetary policy, investment theories, theorems. My blog article should not be taken as advice, financial or otherwise, or as a statement of the right or proper way to do things. What I am is a regular IT guy, middle-aged, with a life full of past mistakes, most of which, I hope, I have learned enough from to continue forward in a positive direction. 

A long term IRA Portfolio Journal

I made plenty of mistakes my first year, primarily through trading and not investing. Again I joined stock markets race during Y2k melt-up situations. Everyone is talking about stocks during break time & this is first job in USA & took trading route. Just to make some quick bucks here.  But after 15K losses, With marriage, my philosophy has changed. All investing is limited to spouse IRA and my 401K only.  It essentially remains thus: build a core of 5 diversified funds with low expense ratio and 10 dividend paying equities, reinvest the dividends and allow them to grow. I don't have a lot of hard and fast rules for my holdings other than they must pay at least 5%. Also if a stock/fund give more than 25% of profits, sell 50% of position and lock the profits.

Currently, the retirement Portfolio consists of five funds and 20 odd stocks. Most of the stocks picks are 5 years old. I am holding some of these funds for 15+ years.

List of funds are:

Ø  VEMAX( Vanguard Emerging Markets Stock Index Fund Admiral Sh )

Ø   VGENX ( Vanguard Energy Fund Investor Shares )

Ø   VGPMX ( Vanguard Global Capital Cycles Fund )

Ø   VGHCX ( Vanguard Health Care Fund Investor Shares )

Ø   VGSLX ( Vanguard Real Estate Index Fund Admiral Shares)

Ø  VFIAX ( Vanguard 500 Index Fund Admiral Shares)

List of stocks includes

BEP

BIP

AMZA

REML

DX

CHI

GUT

ARCC

CII

EXG

APLE

BGR

RRC

PCI

AM

VEON

BGCP

AWP

CHMI

AMLP

GDX

I am not a financial analyst. I am not an expert in money, monetary policy, investment theories, theorems. My blog article should not be taken as advice, financial or otherwise, or as a statement of the right or proper way to do things. What I am is a regular guy, middle-aged, with a life full of past mistakes, most of which, I hope, I have learned enough from to continue forward in a positive direction. I am certainly not an expert in anything, but I am willing to learn, to ask right questions and take some risks.

At the end of December, the IRA is 20 years & I portfolio returned ~4.5% over last 20 years. But my point with this blog spot is long term view really works.  Regularly invest from paycheck and diversify and stay calm and ignore all the noise. Also always keep 5% of money in cash form. Sometime, if you find real value in anything, this helps.