Analytical Engine

In most organizations, customer satisfaction score is a nebulous measure and hard to map back to improvement opportunities (due to overall high scores across the board). Simplified customer satisfaction measures like net promoter scores are neither accurate nor actionable. Hence it was very refreshing to read John Aitchison’s post on how he is starting to think about customer satisfaction.

John proposes a new way to measure customer service and satisfaction similar to the new gymnastics scoring system, where you start with perfect score for execution and get deductions for mistakes.

His rationale is that people struggle with answering the positive abstractions of traditional customer satisfaction survey questions resulting in high scores and inconsistent scoring.

positives are ‘measured’, usually positive abstractions (e.g. the quality of the room) and these are added up in some manner. Working with such data is often rather unsatisfying, partly because the question measurement scale is often poor (most people give ratings at the high end), but partly I suspect that the data collection is focusing on the wrong problem.

People are not good with abstractions. And they are not good at telling you what is good about something (unless you are really skilled in the questioning), but can readily enough volunteer the specifics about what went wrong, how their expectations were disappointed.

Developing this core concept, the idea is to field a customer satisfaction survey where customers are asked to rate their a priori expectations from the service, identify the complaints or issues they had with the service, quality of the service and their overall satisfaction level of the service. It has been found that the gap between quality and a priori expectations is a strong indicator of overall satisfaction. This methodology is also actionable since the customers are only identifying issues they faced whose relative strength can be inferred using a regression analysis.

The trick is to find a way to ask and group the individual issues of the customers so that they retain their granularity and still can be used as variables in statistical modeling. One way that comes to mind, is creating an extensive and actionable issue root cause tree, similar to ones that call centers uses to code reasons for calling (at a much granular level) and then map all the individual issues to it. We have created such issues trees in the past and in our experience it is a useful tool to link operational improvement projects to customer issues. However we still need to explore how the issue tree can be used in context of a customer satisfaction survey.

This concept is worth further investigation, if you measure customer satisfaction or make decisions based on customer satisfaction scores.

Ian Ayres, in his book Super Crunchers, questions decisions made through intuition and advocates decisions made by optimal utilization of data. In the book, Ayres gives examples of how data driven decision making is impacting businesses, education, sports, government, etc. One such industry where a lot of data is collected but not necessarily used in decision making is the gaming industry. Despite Harrah’s being an analytics poster child, not a lot of gaming companies leverage analytics to its potential.

Casinos collect revenue and customer data from a variety of sources, such as player cards, slot machines, and gaming tables. They can also gather information from non-gaming sources: through call centers, surveys, hotels, restaurants, and events. Harrah’s is a pioneer in collecting detailed data on its customers’ activities. Harrah’s team sliced that data into finer segments ”identifying unique customer groups and targeting each group with pitch-perfect marketing strategies.” The results -  tremendous increase in customer loyalty that has turned Harrah’s from a relatively unremarkable player into an industry giant.

Recently, we worked on an engagement for one of the large gaming companies in the world, where the casino wasn’t able to capitalize the rich data it had on its players and slot machines. Direct marketing campaigns and slot machine floor layout optimization, were two big initiatives that were performed but without leveraging the rich information in the data. An analytics driven approach was needed to identify high potential players and optimize slot machine layout. In this post, we will talk about the approach we used to segment players and identify changing behavior patterns of players.

Though a lot of advanced concepts of customer analytics can be applied here, we used a basic RFM (Recency, Frequency, and Monetary) segmentation to effectively target players. RFM was tweaked to LFM (Latency, Frequency and Monetary) for a simple reason- unlike a website or a grocery store, where a person’s recent visit shows that he/she is likely to visit again in near future, a casino trip happens after some interval. So it makes more sense to calculate whether days between the consecutive trips are changing significantly. In an LFM cube the cutoff of each cell of cube can be determined using basic business rules based on frequency distributions.

One of the KPI that was tracked was average spend per trip, and due to this, some players with higher number of trips and medium spend are undervalued and under-targeted. The LFM cube gives us the visibility into such customers and helps us evaluate the true potential of such players. Such a cube can be run at regular intervals to capture the movement of players between the cells, and such movement is a good predictor of changing behaviors and can be an effective lever for targeting marketing campaigns.

In next post we shall talk about how analytics can be used around slot machines and how such inputs can be used to design the floor layout in a better manner.

Mobile phone operators are facing a rapidly saturating market, thus facing customer retention as the primary challenge.  Mobile operators, like almost all big retail operations, have their own predictive churn models. Moreover, mobile operators have the luxury of having a wider array of variables. The “Know Your Customer” questionnaire and the usage behavior of the mobile phone typically provide access to a few thousand variables.  Modelers frequently add some off-the-shelf segments (from providers such as Claritas or Acxiom) in the mix to make a better sense of and add more predictive power to the model.

But the large number of variables adds at least two complexities.  First, even after performing variable reduction, modelers usually are still left with a high number of components/ factors, a number high enough to make a concise business presentation difficult.  Second, the high number of variables also affects the actionability of model insights.  No single variable affects churn in isolation, so it is hard to determine the independent impact of individual variables in customer churn.

An alternative approach to comprehend and act on the model recommendations will be to bundle the variables in logical groups.  For example, the variables related to time can indicate when a customer is more prone to churn and can be grouped as a ‘when’ group of variables. This ‘when’ group will not necessarily tell us why a customer is more prone to attrition, and would rather tell us when she is more prone to attrition.  Similarly, the variables’ indication of issues faced by a customer (her interactions with call center might be a good source for this) such as handset problem or network issues can be grouped as ‘why’ variable.  The demographic variables that are significant in predicting churn help the modeler to profile the customers likelihood to churn and can be grouped as ‘who’ variables. An integrated view of these three groups is more palatable, and from a modeler’s perspective, it is easier to convey to the business ‘who’ are more likely to churn, ‘why’ they are more likely to churn and ‘when’ they are more likely to churn.

The above approach does not only make the models easier to comprehend, but also increases actionability by providing a structure to the model findings.

In our earlier post, we shared our thoughts on how data-driven personas can add the much needed value to attitudinal segmentation.The power of “personas” can be leveraged not just in traditional consumer marketing, but across other functional areas such as “franchisees”, “agencies”, “sales force”, etc.

We recently noted a similar parallel in the recruitment space.

There is little doubt on how recruiting continues to receive a keen attention from human resources function of every organization. ˜Who makes a good hire? “and ˜Who is best armed to succeed at the firm?” are questions that recruiters are constantly trying to answer. Recruitment is vital not only because it helps the organization meet its objective of talent acquisition but also because recruitment decisions impact the overall firm performance.

Given this context, we wanted to understand how an employee’s pre- joining profile (educational qualification, prior work experience, source of hiring, joining level) impacts her performance at the firm. For this, we applied rigorous data analytics techniques (decision tree algorithm) to

• Classify the employee pool into various performance based segments - high, average and low performers.
• Identify the variables that best discriminate one segment of employees from the other.

Based on our segmentation approach, we helped the recruiting team by creating personas to represent different employee segments. (For more thoughts on segmentation, click here). In this case, a persona is a profile sketch of the ˜desired hire” from the perspective of required skill sets, behavioral and attitudinal characteristics. Segmentation and creation of personas proved to be a vital exercise for the following reasons:

• The first step to effective recruiting is to fully understand the type of employees the firm wants to hire in terms of required skill sets, behavioral and cultural fit; personas help different stakeholders in the process including recruiters, interviewers, search firms, recruiting agents, share consistent understanding on the desired hire.
• Personas give a well rounded description of the employee by covering attitudinal and behavioral characteristics; this knowledge can be effectively leveraged by recruiters in answering questions like ˜how do we reach the potential candidate?”, ˜what kind of media should we use to advertise job postings?” and so on. This results in fast and speedy sourcing of candidates.
• Most importantly, personas are data driven and an outcome of rigorous analytics. To that extent, it simply shows what has worked and what has not worked for the firm in the past.

As the authors in ˜The Risky Business of Hiring Stars” rightly point out, hitching your wagon to the rising stars is not the solution to a recruiter’s woes. It is imperative for recruiters to devise specific tailor made recruitment strategies for different roles and profiles. Employee segmentation and persona creation is a key step in this direction.

Last but not the least, the prerequisite for the success of this analysis is the data itself. Firms need to start or continue investing in the right kind of performance management softwares that continuously track and maintain employee performance metrics.

This idea of “timeshare” as a “financial product” has been floating in our den for several weeks now. There are several definitions and models for timeshare. However, for the purpose of this post, let’s use a simpler definition. A timeshare “owner” is entitled to a defined vacation (pre-defined location and timing) every year for an extended duration of time. In effect, she gets property rights over “the vacation” by making an advance payment.

Let’s consider the table below:

Here, Ms. X buying an economy vacation might have ended up buying Vacation A. Let’s say, Vacation A allows 3 days and 4 nights of Deluxe Room stay at a Country Resort in Malibu. She is allowed to use any 3 day window between 4^th and 10^th January of every year (until 2020).

In 2008, however, she can exchange her Vacation A for Vacation B, which has the same price. Going further ahead, in 2010, her economy vacation of 2006 has become super premium and she can easily avail a longer version of Vacation B or C.

However, at any point, the ownership of vacation A allows Ms. X a chance to “exchange” the right over the room with another vacation of equivalent value. The pre-condition for this exchange is the “price.”

The price assignment, here, is done by a timeshare exchange, such as RCI, Interval, Marriott Timeshare, Starwood Vacation Ownership, etc. These exchanges factor the demand and supply for a particular vacation, location, quality of the property, length of vacation, seasonal variations in demand, etc., to arrive at the price of a vacation. The interesting thing is that this “price” is not constant, and depends on the “perceived” value of this “vacation” at a certain point in time.

Here, we introduce the financial twist. If vacation was tradable, then Ms. X would be able to make a $100 profit even after moving to what she would have considered a premium vacation in 2006, in lieu of her economy vacation of 2006. And suddenly, Ms. X would look like the smartest investor in town.

In the diagram above, Y-axis denotes the price of alternative vacations at three different time point.

Trading in vacations (quite similar to commodities trading, as some would point out) can help turn a timeshare into a financial product, and hence a product for a larger mass of people. This can be facilitated through a “financial timeshare exchange,” very similar to the NYSEs of the world.

However, this alternate financial market would require:

1.Data analytics capabilities to analyze customer’s behavioral preferences with respect to market forces of supply and demand

2.An understanding of the regulatory implications of completely transferring rights over a vacation, which can be facilitated only through the timeshare exchange

3.Educating customers to think of a timeshare as not just a lifestyle product, but also a financial product

George Olsen talks about the importance of personas as a valuable design tool in driving strategic and tactical level details.

Personas are fictitious characters that are created to represent different product user types and to aid tactical and strategic level decisioning. Typically, ad agencies have relied on qualitative methods such as interviews, focused group discussions, existing literature to develop a typical character sketch. However, a well designed persona is one which not only suffices an ad agency’s need but also addresses the interests of other stakeholders including marketing, sales, finance and HR teams.

Recently, our client, a financial services firm, wanted to understand the attitude and behavior of their users. They wanted to strengthen their brand architecture and develop the right kind of messaging for target audience. We expanded on our market segmentation approach and used the granular primary research data, company’s transaction data and third party segmentation schema (Claritas) data to do develop personas in a bottoms-up fashion. The data driven approach of building personas leveraged internal data, survey data, and a third party segmentation schema (Claritas P$YCLE).
We followed a four step approach to develop the personas:

1. Conducted a survey to capture the attitude and behavior of respondents towards financial issues and planning
2. Used a statistical technique called principal component analysis to reduce some 40 attitudinal and behavioral variables captured from the survey to two dimensions - expectation from the service provider and level of financial anxiety. We used cluster analysis to classify the respondent pool on these dimensions.
3. Built personas by mapping the clusters to the Claritas segments so as to capture information around media preferences and lifestyle behaviors. We enriched the personas with survey information including demographic indicators, behavior towards financial service providers, financial anxiety, as well as Claritas segments that helped highlight media, lifestyle, and income-producing asset ownership traits
4. Performed a “day in life” analysis to understand how a typical day for a persona would look like in his or her diary. This analysis explored the various lifestyle and media touch points which can be used for tactical marketing initiatives

Our approach to leveraging rapid market segmentation helped our client not only to understand the segments on the basis of attitudinal dimensions but also to appreciate the strategic and tactical initiatives that can be achieved with robust data driven personas.  This level of understanding of their customers helped them improve their segment specific marketing messaging and customizing the store experience for each segment.

As the US and the world economies encounter a downturn and firms look to scale back, Marketing is often one of the first places to face budget cuts.  Forrester reports that many companies expect to cut their marketing budgets by 3%.  But how do you maintain or grow your customer base and revenues when consumers are spending less and your message isn’t getting into the marketplace as loudly?

We think the use of online coupons deserves a harder look.  Emailing your customers and prospects with newsletters, product updates, and coupons is certainly nothing new, but it’s now well-positioned for even greater success:

• Companies are getting good at it. After dabbling in techniques like SEM and direct email, firms have gotten better at driving profitable growth from these methods, and many are increasing their focus on online advertising as a cheaper way to spend their marketing dollars. 
• Consumers want more of it. During these uncertain times, consumers plan to increase their use of coupons to save some money.  Sending these options straight to their inbox or mobile phone accomplishes that goal and positions you as a preferred provider.
• Consumers who use it are attractive prospects. Compared with consumers who only use offline coupons, Forrester reports that users of online coupon tend to have higher incomes, shop online, like to try new products, and influence peers.  Younger consumers also use coupons, and they can be a good avenue to get the word out about your product.
• More data is available to help you win at it. More firms sell marketing lists (or can help you run campaigns to get new lists), segmentation data helps you understand consumers’ preferences and desires, and syndicated data helps you understand purchase behavior.  Combining this data gives you incredible insight into consumers to tailor unique marketing messages.

You don’t just want to throw promotion dollars at existing customers to give them discounts on things they were already going to buy; rather, you likely want to use those dollars to deliver positive returns and achieve business goals - such as acquiring new customers, increasing market share, or increasing wallet share.  Doing this requires targeting offers to customers based on their stage of the customer life cycle:

1. Acquire. Coupons can be a good tool to help consumers overcome the risk associated with trying a new product; if a new product is cheaper than the one they normally use, the savings might be worth trying.  You can use them to attract entirely new customers to your firm, or to get your existing customers to try a new product line.  Targeting early adopters can also help generate buzz, as they will influence friends and family to buy the product as well.
2. Grow/Stimulate. Once you’ve acquired a customer, you want them to maintain or increase their purchases.  Two ways of stimulating usage are encouraging them to try a different variety (e.g., color, size, flavor) or showing them new uses for the same product (e.g., using Q-tips for craft projects in addition to hygiene).  In this stage, the focus should be on the marketing message, the coupon being used to help seal the deal and drive the customer to the store.
3. Manage. In this stage, your customers are steady-state users, and couponing may not be required to retain them. However, these consumers present a good opportunity to test new offers on an already loyal customer base and measure the response before using them on the general public.  You might test them using different demographics, layout, or wording, perhaps even running controlled experiments to determine which of two offers is more effective.  We’ve done some research on the use of Behavioral Economics to improve offer design, which might be helpful in performing this testing.
4. Reclaim. If customers reduce their consumption or begin to try competitors’ products, you can use targeted offers to reintroduce your product and retain them as customers.  However, depending on their needs and your product pipeline, you may otherwise opt to move back to the beginning of the life cycle and acquire them as customers of another of your products.

This strategy requires a high level of customer insight to understand preferences and stages in the life cycle.  You can gain this insight by applying segmentation schemes to your lists of customers and prospects, and by analyzing your customers’ history of purchases and coupon redemption.  Applying a rigorous testing approach will help you identify the most effective offers for each customer and stage.

Applying this framework to understanding your customers and targeting coupons will deliver several benefits, including:

1. Strong ROI potential.  Campaigns that are more effective and lower-cost, targeted at attractive customers, have a stronger potential to deliver a positive ROI.
2. Better data to analyze results. Results of online campaigns are easier to track and measure than traditional campaigns, particularly if your coupons lead customers to purchase from your own website.  Analyzing results from campaigns that involve multiple partners may require a different approach, as Vishal outlined in his earlier post on trade promotions.
3. Better customer relationships. You can use the insight you’ve gained about your customers’ behaviors, preferences, and purchase history to continually develop targeted offers.  This level of personalization will help you deliver the right offers to the right customers at the right time, and ensure that your promotion dollars are spent most effectively.

Trade promotions represent a significant part of the cost of consumer packaged goods (CPG), and an important part of a go-to-market strategy, it is the second highest expenditure after COGS and represents two thirds of marketing spend. Yet, there is little visibility into where this spending actually goes, or how effectively it increases revenues, expands market share, or creates brand awareness among consumers. The irony of the system stands affirmed with the fact that extra pressure generated over the supply chain for immediate deliveries is borne by the end consumers which has pushed the actual monetary benefits reaching the end consumers down to 50% over the years.

Trade promotions have taken various forms and attained different levels of complexities as the markets and available technology have matured over time:

1. Slotting allowance: reward paid to a retailer for the retailer’s shelf space
2. Case allowances: reward offered to a retailer when merchandise is purchased by the case; greater the number of cases, greater the discount
3. Account specific promotions: reward for specific big/small accounts (retailers)
4. MDF or market development fund: a lump-sum payment made once or twice per year which the retailer may (or may not) spend on marketing the product
5. Scan back promotions: reward for higher sales, where the retailers are required to submit paperwork as a proof of the sales, to qualify for the trade promotion allowance

Data at the most granular level is required for effective management of trade promotions, but most of the time it is not available or is highly fragmented and inconsistent (at times the most granular level data might be available in segregated laptops !). Combined with the differences in geography and final pricing at the retailer level, firms are most of times unable to break down trade promotions in sub-processes. Given the level of complexities involved, the market essentially lacks standard meaningful metrics to rate a promotion at an absolute or even at a relative level within the firm (*more than 70% of the trade promotions are not evaluated properly and less than 30 % are considered profitable). Eventually, many firms end up taking the outcomes of the promotions rather than the whole process as a proxy for the performance metrics, thereby making continuous improvement a challenge if not impossible.

One way we have approached the analysis of Trade Promotions is performing the following steps:

1. KPI Identification: Identifying the key parameters involved within the system will crucially determine the effectiveness of the whole procedure. Some common metrics include:
   • Time periods i.e. the average active period of a promotion and the time difference that may be involved to measure its effect.
   • The expected increase in the sales volume.
   • Consumer growth.
   • Discounts reaching the customer or customer pass-through.
   • Increase in the shelf space.
   • Increase in revenue/profit/profit margin for every dollar spent in trade promotion
   • Increase in the no. of cases sold per dollar spent in trade promotion
   • Increase in brand value over the specific demographics
2. Data Gathering and Quality Audits: Gathering the data from sources that may include invoice history, third party/syndicated data (e.g. IRI/Nielsen), retailer level sales data or POS data, and even individual laptops.
3. Data Preparation: Involving processes like missing value treatment, data anomaly treatment and creation of the final data set with maximum level of detail possible with respect to the KPIs.
4. Analytical Modeling: Applying analytical techniques to connect these disparate data sources to quantify KPIs
   • Linear Regression at the retailer-product level to attain coefficients (symbolizing the dependence of sales over the KPIs) of attributes for the time duration of specific promotions (which may be around 2-6 weeks).
   • Predicting the future values of the coefficients and product-sales using a beta algorithm based on Markov Chains and Monte Carlo simulation, developed by Diamond for this purpose.
5. Recommendation Developments: findings from the data modeling to determine most profitable promotions -e.g.
   • Retailer-product combination
   • Geography
   • Specific time of the year etc.

Although the lack of a centralized TPM system within CPG firms will be the biggest obstacle for advanced analytical analysis of the same, it’s still possible to gain valuable insights for better implementation and to improve the process on a whole.

*Sources: ACNielsen Survey of Trade Promotion Practices (2005), Cannondale Assoc’s, DCI Assessment

It appears that utilizing ATMs as a sales channel is not even a near future thing as we thought it was, it has already arrived.
Last Friday on my way home, I decided to withdraw some cash from an ATM of one of the larger private banks in India. When I was almost done with the transaction and was about to collect the money, there was a surprise waiting for me in the screen! The ATM offered me a reasonable amount of home loan and requested me to click a radio button written - if I want to avail or click a radio button with instruction of reminding me later if I want to think about it later or click - in case my answer was an outright no.
But then neither am I the only one, nor India the only country where it’s happening. One of our blog readers, a few continents away from my city, has already come across and availed one such offer from Chase!
Things surely are fast nowadays!

A few weeks ago, Shantanu wrote on recommendation engines and how user feedback and ratings can be a part of recommendations you provide to your customers.  But if you have ever looked through user recommendations while shopping online for a product, stock, or movie, you know that they aren’t all helpful.  Ideally, user ratings would accurately represent the population, but not all feedback is created equal, and there are some inherent challenges in these systems:

1. Not everyone will rate.  People may read the ratings when shopping for an item, but they won’t always come back to rate.  Unless a site offers an incentive for rating a product, a customer’s only real incentive for doing so is to talk about how much they love or hate it; moderates may be under-represented.
2. Ratings will be biased.  People’s individual biases produce variances in ratings, even if strict guidelines (think about employee performance reviews) are presented.  In addition, new raters tend to rate high.  Their average ratings decrease over time as they rate more items, presumably because they are exposed to more items and have a better sense of an item’s value relative to alternatives.
3. Ratings are averaged, masking the underlying data. Because people often only rate items they feel strongly about (love it or hate it), and an average of those extreme ratings may not truly represent the actual sentiment surrounding a product.  For example, a review of rankings on Amazon.com revealed that the reviews for the majority of the products have an asymmetric bimodal distribution. For these products, the mean of the online product reviews does not necessarily reveal the product’s true quality, resulting in misleading conclusions about the product’s future success.  In addition, established products are at a disadvantage against new ones.  Consider a product that has received five rankings, four and one giving it an average rating of 4.8. If a new product enters the space and receives one rating, it will be ranked higher than the other simply because it has fewer ratings. 
4. Ratings may be false.  Take the case of the Whole Foods CEO who posted disparaging comments about a competitor on a message board while talking up his own company, later stating “Sometimes I simply played ˜devil’s advocate” for the sheer fun of arguing.  Other visitors may post false comments to artificially affect the rating, and these are not always removed from the calculation.

As more products are marketed and sold online, feedback-based ranking systems are increasingly common components.  Amazon and eBay were among the first to use visitor ratings to rank products and sellers, and in February, Yahoo launched its Buzz service, which asks readers to click on their favorite stories, then uses those ratings to determine the most popular articles on the web.

Why is this important?  Because accurate product ratings help predict that product’s success, and higher product ratings lead to more sales - or, as Allen & Appelcline state, the value of individual items (most frequently goods) rise or fall based upon the largely subjective judgment of individual users. So what can you do to ensure the rating system on your own website accurately reflects your customers’ views and the value of your product?

We’ve seen some thought around using analytical techniques and Bayesian mathematics to create better product rankings.  Some of the solutions explored include:

1. Adjusting ratings based on known biases.  Since some people rate higher or lower than others, one approach is to assign users a “User Optimism” value based on their rating history, and adjust the product’s overall rating based on the raters’ optimism value.  Another approach is to remove all ratings from people who have only rated 1 or 2 items, helping to eliminate new-rater optimism or “drive-by” fraud.
2. Weighting a product’s ratings based on the number of ratings received.  When a product has a small number of ratings, these ratings should count less than those for a product rated many times.  To achieve this, you can add a “magic value” into the algorithm that calculates a product’s average ratings.  This “magic value” brings products with few ratings closer to the average ratings of all products, then reduces its effect as more ratings are received, allowing established products’ ratings to float freely and more closely reflect the average of its ratings.
3. Adjusting the algorithm to account for bimodal distributions.  To account for products that only have ratings at the extremes, one approach is to use a dual-point estimation model to more accurately reflect customers’ views of the product and predict the product’s success.
4. Analyzing a customer’s rating history for fraud.  Someone who rates several products on your site should have a predictable rating pattern.  You can analyze individual customers’ accounts to identify and remove anomalies that might skew product ratings.
5. Encouraging customers to leave text-based feedback.  When a person takes the time to write out a product review, and knows that her name will be attached to it, she generally does a better job in her rating.  These reviews tend to be closer to the average than those without.

Although your customers’ product ratings may be imperfect, they can still yield insights and value with the application of a good set of analytical techniques.