Analytical Engine

John Sviokla recently wrote in his Harvard Business blog about our first analytics product called DemandEstimator. He cites an example of a client situation in insurance industry where we have used the product to understand agent profitability, and shares some of his other ideas on how business executives can use DemandEstimator.

Please read his full post

Interesting things are taking place in pharmaceutical industry, which I believe has the potential to change pricing landscape of high cost drugs.

In June’07, the National Institute for Health and Clinical Excellence (NICE) of UK decided that Johnson and Johnson’s multiple myeloma drug, the one year treatment of which stands at around USD 48,000 a year, can be defrayed for if the company is willing to refund the cost in case of not showing a pre-decided level of cure.

This move towards sharing the risk of treatment outcome did not go unnoticed. Providers are all the more willing to share the risk of no-outcome with the drugs manufacturers. Biogen Idec, maker of Avonex for Multiple sclerosis drug that costs around $18,000 a year has made a dynamic pricing agreement with UK’s NICE. Under the plan, 5.000 multiple sclerosis patients are being followed for 10 years to see the effect of the drugs in slowing the progression of the disease. The prices of the drugs will be adjusted based on improvement of performance. Merck-Serono has offered to refund the primary care cost if its drug Erbitux if the patient does not respond within 6 weeks.

Although Europe seems to be leader in this development, but US companies are definitely taking note. United Healthcare have made an agreement with Genomic Health with United paying for the $3,460 genetic test to determine whether a woman with early-stage breast cancer would benefit from chemotherapy. If tests don’t have the intended impact of lowering chemotherapy United Healthcare will negotiate a lower price for the tests.

Essentially this development adds a new dimension to the pharmaceutical pricing, i.e, risk. Before pricing a drug, the point of paramount interest to both manufacturer and providers is what drugs are the ideal candidates for outcome-based pricing? The high cost drugs which are new in the market or have relatively low success rate seem to be the ideal target from the developments so far. The principal risk that companies should take into account is financial risk, which in turn will depend upon probability of no- outcome and amount of dollar exposure. Apart from regular variables such as physician’s ability, patient’s specific variables etc. there may be exogenous factors such as pollutions level for certain treatment which will affect the probability of outcome.

There are several interesting questions already. Will the companies have different refund policies for different locations? Will there be selection criteria? How will regulatory environment affect the risk-sharing?

We are at an interesting corner here. I will wait and watch how the pricing territory shapes up in the near future.

Market research is, but seldom treated as, something beyond reporting figures and displaying good looking charts. The intent of market research should be to provide in-depth insights and answer key questions around the business problem at hand. However, more often than not, post the survey execution, researchers and analysts end up fishing in the ‘numbers’ ocean, and with great difficulty, find their way to final insights and recommendations. What they leave the table with, often, are piece-meal insights that may or may not add up to strategic recommendations.

However, consulting as a profession requires quick and effective market research, most of which is conducted with specific end objectives in mind.
At Diamond, we extend the hypothesis driven approach (HDA) to conducting market research and survey analytics. HDA is the answer to most of the woes and worries faced by a market researcher

Let us use a simplistic scenario to explain HDA. Suppose, we want to conduct a study to understand the buying behavior of people towards personal computers and one of the hypothesis we want to test is that ‘price is an important attribute in purchasing a PC’.

This approach begins with hypothesis definition. In the example considered earlier, we want to test whether price is an important driver of purchasing a PC. This is followed by laying out sample analysis that would help prove or disprove the hypothesis, e.g. % respondents rating price as an important driver or average rank of price as a driver. Next, we would need to gather data on parameters such as relative importance of drivers, allocation of points between various drivers to support the analysis. The process till here would culminate into survey and questionnaire design (e.g. the exact question to be asked, the likert scale to be followed.) followed by survey execution (e.g. online vs. offline).
The beauty of the process lies in the fact that once the survey execution is complete, the collected responses can be directly fed into the sample analysis generated in the second step. The process closes with hypothesis validation and delivery of insights and recommendations to the end user.

One of the most common problems of retail sector is pricing a product. There are numerous similar products lying in the shelves that are at different life stages and have complex interrelationship with other products & customers. Providing an optimal price of an SKU in order to extract maximum customer surplus is a challenge in itself.

In our recent stint with a Europe based retail chain for electronics goods, we helped our client use analytics to better price cell-phone accessories. A lot of advanced sophisticated pricing models can be built and deployed; however, when efforts and incremental value of such models are weighed against the approach of using elasticity and bundling, the latter option seems to be an optimal choice to get some quick hits for the business. For the uninitiated, here is a quick review of elasticity and bundling:

— Price elasticity of demand: The measure of responsivenesses in the quantity demanded for a commodity as a result of change in price of the same commodity.
— Product bundling: A marketing strategy that involves offering several products for sale as one combined product

Price elasticity approach: We calculated the price elasticity, and accessories that are having extremely high or extremely low elasticity along with reasonable business impact (sales volume or margins or life-stage) are picked for analysis. With the help of gross margins, we came up with the best price change for such accessories. In order to evaluate the financial impact, we built a business case to find out the potential revenue impact of such price changes.

Product bundling approach: A basic market basket analysis can help find products customers like to buy together. However, bundle needs to be evaluated for elasticity as well as likelihood of success before taking it to the customer. In order to increase the sale of some of the underperforming accessories, such accessories can be strategically bundled with highly elastic products. Gross margin of the bundle can be used for driving the price of the bundle.

Both these methods fetch actionable recommendations which can be tested in a very short time-frame in a test environment. One of our recommendations led to an additional £100K revenue impact per week for selected SKUs. A similar approach can be used for other items to better understand the nature and the monetary potential of the products lying in a store.

We recently interviewed Bill Dorion, who is the Director of IT at PepsiAmericas. He has been a pioneer of championing and executing key analytics initiatives for PepsiAmericas. Bill has been in the Information Technology sector for 30 years, which includes 22 years in the CPG industry. Bill, in his current role, manages IT applications development team for strategic solutions selection, development and implementation.. Reproduced below, are some of the excerpts from our conversation with Bill.

Question: Can you tell us something about the type of problems you are solving in the organization using analytics?
Bill: Given our current strategic priorities, we are using analytics for reducing out of stock at retail locations. There are three key components that we are focusing on:
• Better Inventory forecasting (having the right product at the right delivering location)
• Divisional Dispatch & Voice Pick (having the ordered product picked and delivered correctly)
• Suggested Order Solution (ordering the right product and quantity at the right time to fill the customer demand)

Question: How do you quantify the value realized by an analytics project?
Bill: Analytics has several quantifiable and measurable benefits for us, as we have realized during the learning process:
• Reduced out of stocks at retail locations
• Reduced inventory levels are at the retailer back room
• Optimized delivery schedules suitable to better meet demand
• Less time for sales reps. in the ordering process and more time focusing on selling

Question: What are some of the key challenges that you face in analytics focused projects?
Bill: PepsiAmericas did not have a lot of experience in handling analytics projects which presented us with some challenges. The key learning for us was to understand that analytics projects have a strong R&D component inbuilt into them. While our organization is very comfortable with execution projects, the test and learn cycles inherent in an R&D process took some time to get used to. Another challenge for us was selecting the assistance of right resources that bridge the gap between business, statistics and common sense, and can communicate between these groups effectively.

Question: How do you see analytics growing within your organization and within the CPG industry?
Bill: If you ask me about the present, I see analytics growing in two main areas, optimizing the ordering process and helping in pricing and trade promotion spend

Question: Since selection of the right analytics resources is a key component for the success of the project, what do you look for in an ideal analyst that you hire into your team?
Bill: From my perspective, there are three key ingredients to a key analyst hire in our team. First and foremost, the analyst should fit into the culture of the team . He/She should be good at understanding business problem and incorporating it into the statistical tools to solve the problem. Last but certainly not the least, effective communication skills are very important. I talked about it earlier when I mentioned the need to bridge the gap between business, statistics and common sense through effective communication.

In addition to a very technical perception as a method of predicting a continuous variable, Linear Regression is also, and equally importantly, an analytical method to understand the business drivers of dependent (which is being predicted) variable. Because of its inherent nature, regression has been used to understand how a variable is influenced by the interaction of multiple variables. In this post, we will talk about a couple of such scenarios where we used linear regression in the context of strategy formulation.

Casino – Slot Machine Optimization

In our previous post we talked about how a simple LFM (Latency, Frequency, & Monetary) segmentation can be used for customer management across casino industry. In this post, we shall talk about how linear regression can be used for floor layout optimization of casino.

Recently, we worked on an engagement for one of the largest casinos in the world, where the casino wasn’t able to capitalize on the abundance of slot machines; moreover, the machines were spread across the floor without much thought or analysis behind why a machine should be placed at a certain location – and hence, were not tapping the right potential of either the machines or the location. However, even from a gut instinct, a lot of us who have been to a casino, know that most of the players have their own preferences when it comes to picking a machine – be it the corner vs. central isles, the red machines vs. the blue machines, the spinning wheels vs. the talking genies, and so on.

A simple profiling of slot machine performance across various variables like denomination of machine, jackpot, type of game, location of machine, etc. can fetch interesting results. However, all these variables need to be evaluated simultaneously to create a holistic picture.

Even though a lot of optimization concepts can be applied, a linear regression can be used to determine the drivers of performance and better understand how these drivers interact with each other. The drivers of performance help to determine an optimal floor layout, so that maximum returns can be obtained from the given slot machines on floor. We found that certain themes with certain colors drive the performance while for some types of machines it was progressive nature of machine payouts that drives money into them. Certain machines if placed around restaurants seem to do well, while others if placed around some attraction tend to perform well.

Sales Territory Prioritization - Distribution Performance:

For companies with retail distribution, it is critical that revenue is maximized while expanding business in hitherto untargeted geographies, and therein lies a need to prioritize geographies based on their expected worth. Such a strategy requires analysis for not just one factor but for multiple factors and an area where linear regressions is useful.

While working for a Financial Services firm, we helped our client prioritize territories where they did not have any presence. Based on the current product offering and existing market attributes, a regression model was built and untargeted territories were scored (prioritized). This not only helped to reduce the focus but also helped in designing a quick and efficient marketing strategy that targets the ‘right customers’ of such territories.

We found that, for some territories the major driver of revenue was the abundance of a certain segment of population, while for other territories, the key driver was the strong presence of a particular type of business in that territory. These drivers helped us tailor the marketing strategy in a customized manner by understanding the ‘needs’ of each territory.

This is not all by any means. At Diamond, we have effectively used regression in launching new products, managing churn, managing portfolios, etc. to not only come with a target(scored) population but also come up with precise recommendations based on the detailed study of drivers.

(There is no dictionary definition of P2P process, but often P2P is defined as a generic term used to encompass the payment for goods and services: The basic process of raising a purchase order, receiving the goods and paying the invoice is often put under P2P process .)

In a recently finished engagement with a manufacturing major, we leveraged Diamond’s procurement analytics solution to help the client identify dollar leakages and potential saving opportunities across their P2P process. This engagement had all the challenges associated with a large manufacturing setup viz. inbound and outbound from plants located across various locations, multiple vendors supplying the same raw material, a complex distribution and lack of unified IT systems. The list below can give you a feel of some the issues we identified –
1) Price variation/anomalies:
a. Procurement organization paying different price for the same raw material within the same time frame across different vendors
b. Price changes for certain non-contract items being anomalously higher (even in an increased demand scenario, where the organization might have had a chance to negotiate better prices)
2) Contract Adherence:
a. Shippers invoicing higher than the negotiated rates in purchase orders
b. Vendors charging prices which are not consistent with contract-negotiated prices
3) Freight Charge Variations:a. Different freight charges being charged for the same kind of delivery across supplies
b. Freight charges being consistently higher than industry standards
c. Variation in the freight charges for equal shipments from the same vendor Payment
4) Payment & Order Schedule:a. Payments made significantly prior to the negotiated deadlines leading to loss of revenue
b. Sub optimal volume discounts because of fragmented orders

Organizations are investing a lot of money and effort in IT solutions, designing data warehouse(s), and data collection methodology. These efforts, though, aimed at streamlining decision making processes, continue to exist in silos and are not creating the desired impact for the business. To help our client tackle this challenge, some of the things we tried to focus on, as part of the engagement were –

• Master Data Management: Integrating different data sources (General Ledger, Purchase Orders, Invoice, Vendor information etc.) and incorporating process information to come up with the final form of an analysis data mart
• Basic Data Profiling: Performing basic data validation, enriching/ cleansing and classifying required data, instead of building highly complex spend analyzers around existing solutions; The essence of all analytics done by a team should be the business benefit that can be derived, and not intellectual gratification
• Variance Reports: Leveraging the analysis data mart to create variance reports that help identify the variation in price/freights for a certain procured material
• Dashboards/Segmentation of the invoices : Developing a segmentation tool that helps identify where the money is invested/spent; Having a sense of where the dollars are going always helps businesses prioritize their spend rightly
• Payment segmentation: Identifying anomalies between the POs and Invoices, and keeping track of vendor issues such as reject rate/ quality, etc. can help renegotiate contracts, create appropriate procurement and penalty mechanisms for ongoing cost reduction
As can be seen, a significant part of our approach was about getting the basics right, and instituting basic analytics processes in place. Before moving on to advanced statistical concepts/techniques, we demonstrated the value of analytics through simplified dashboard and analytics reports,and showcased a potential saving of multimillion dollar to the client

In most organizations, customer satisfaction score is a nebulous measure and hard to map back to improvement opportunities (due to overall high scores across the board). Simplified customer satisfaction measures like net promoter scores are neither accurate nor actionable. Hence it was very refreshing to read John Aitchison’s post on how he is starting to think about customer satisfaction.

John proposes a new way to measure customer service and satisfaction similar to the new gymnastics scoring system, where you start with perfect score for execution and get deductions for mistakes.

His rationale is that people struggle with answering the positive abstractions of traditional customer satisfaction survey questions resulting in high scores and inconsistent scoring.

positives are ‘measured’, usually positive abstractions (e.g. the quality of the room) and these are added up in some manner. Working with such data is often rather unsatisfying, partly because the question measurement scale is often poor (most people give ratings at the high end), but partly I suspect that the data collection is focusing on the wrong problem.

People are not good with abstractions. And they are not good at telling you what is good about something (unless you are really skilled in the questioning), but can readily enough volunteer the specifics about what went wrong, how their expectations were disappointed.

Developing this core concept, the idea is to field a customer satisfaction survey where customers are asked to rate their a priori expectations from the service, identify the complaints or issues they had with the service, quality of the service and their overall satisfaction level of the service. It has been found that the gap between quality and a priori expectations is a strong indicator of overall satisfaction. This methodology is also actionable since the customers are only identifying issues they faced whose relative strength can be inferred using a regression analysis.

The trick is to find a way to ask and group the individual issues of the customers so that they retain their granularity and still can be used as variables in statistical modeling. One way that comes to mind, is creating an extensive and actionable issue root cause tree, similar to ones that call centers uses to code reasons for calling (at a much granular level) and then map all the individual issues to it. We have created such issues trees in the past and in our experience it is a useful tool to link operational improvement projects to customer issues. However we still need to explore how the issue tree can be used in context of a customer satisfaction survey.

This concept is worth further investigation, if you measure customer satisfaction or make decisions based on customer satisfaction scores.

Ian Ayres, in his book Super Crunchers, questions decisions made through intuition and advocates decisions made by optimal utilization of data. In the book, Ayres gives examples of how data driven decision making is impacting businesses, education, sports, government, etc. One such industry where a lot of data is collected but not necessarily used in decision making is the gaming industry. Despite Harrah’s being an analytics poster child, not a lot of gaming companies leverage analytics to its potential.

Casinos collect revenue and customer data from a variety of sources, such as player cards, slot machines, and gaming tables. They can also gather information from non-gaming sources: through call centers, surveys, hotels, restaurants, and events. Harrah’s is a pioneer in collecting detailed data on its customers’ activities. Harrah’s team sliced that data into finer segments ”identifying unique customer groups and targeting each group with pitch-perfect marketing strategies.” The results -  tremendous increase in customer loyalty that has turned Harrah’s from a relatively unremarkable player into an industry giant.

Recently, we worked on an engagement for one of the large gaming companies in the world, where the casino wasn’t able to capitalize the rich data it had on its players and slot machines. Direct marketing campaigns and slot machine floor layout optimization, were two big initiatives that were performed but without leveraging the rich information in the data. An analytics driven approach was needed to identify high potential players and optimize slot machine layout. In this post, we will talk about the approach we used to segment players and identify changing behavior patterns of players.

Though a lot of advanced concepts of customer analytics can be applied here, we used a basic RFM (Recency, Frequency, and Monetary) segmentation to effectively target players. RFM was tweaked to LFM (Latency, Frequency and Monetary) for a simple reason- unlike a website or a grocery store, where a person’s recent visit shows that he/she is likely to visit again in near future, a casino trip happens after some interval. So it makes more sense to calculate whether days between the consecutive trips are changing significantly. In an LFM cube the cutoff of each cell of cube can be determined using basic business rules based on frequency distributions.

One of the KPI that was tracked was average spend per trip, and due to this, some players with higher number of trips and medium spend are undervalued and under-targeted. The LFM cube gives us the visibility into such customers and helps us evaluate the true potential of such players. Such a cube can be run at regular intervals to capture the movement of players between the cells, and such movement is a good predictor of changing behaviors and can be an effective lever for targeting marketing campaigns.

In next post we shall talk about how analytics can be used around slot machines and how such inputs can be used to design the floor layout in a better manner.

Mobile phone operators are facing a rapidly saturating market, thus facing customer retention as the primary challenge.  Mobile operators, like almost all big retail operations, have their own predictive churn models. Moreover, mobile operators have the luxury of having a wider array of variables. The “Know Your Customer” questionnaire and the usage behavior of the mobile phone typically provide access to a few thousand variables.  Modelers frequently add some off-the-shelf segments (from providers such as Claritas or Acxiom) in the mix to make a better sense of and add more predictive power to the model.

But the large number of variables adds at least two complexities.  First, even after performing variable reduction, modelers usually are still left with a high number of components/ factors, a number high enough to make a concise business presentation difficult.  Second, the high number of variables also affects the actionability of model insights.  No single variable affects churn in isolation, so it is hard to determine the independent impact of individual variables in customer churn.

An alternative approach to comprehend and act on the model recommendations will be to bundle the variables in logical groups.  For example, the variables related to time can indicate when a customer is more prone to churn and can be grouped as a ‘when’ group of variables. This ‘when’ group will not necessarily tell us why a customer is more prone to attrition, and would rather tell us when she is more prone to attrition.  Similarly, the variables’ indication of issues faced by a customer (her interactions with call center might be a good source for this) such as handset problem or network issues can be grouped as ‘why’ variable.  The demographic variables that are significant in predicting churn help the modeler to profile the customers likelihood to churn and can be grouped as ‘who’ variables. An integrated view of these three groups is more palatable, and from a modeler’s perspective, it is easier to convey to the business ‘who’ are more likely to churn, ‘why’ they are more likely to churn and ‘when’ they are more likely to churn.

The above approach does not only make the models easier to comprehend, but also increases actionability by providing a structure to the model findings.