Analytical Engine

A few weeks ago, Shantanu wrote on recommendation engines and how user feedback and ratings can be a part of recommendations you provide to your customers.  But if you have ever looked through user recommendations while shopping online for a product, stock, or movie, you know that they aren’t all helpful.  Ideally, user ratings would accurately represent the population, but not all feedback is created equal, and there are some inherent challenges in these systems:

1. Not everyone will rate.  People may read the ratings when shopping for an item, but they won’t always come back to rate.  Unless a site offers an incentive for rating a product, a customer’s only real incentive for doing so is to talk about how much they love or hate it; moderates may be under-represented.
2. Ratings will be biased.  People’s individual biases produce variances in ratings, even if strict guidelines (think about employee performance reviews) are presented.  In addition, new raters tend to rate high.  Their average ratings decrease over time as they rate more items, presumably because they are exposed to more items and have a better sense of an item’s value relative to alternatives.
3. Ratings are averaged, masking the underlying data. Because people often only rate items they feel strongly about (love it or hate it), and an average of those extreme ratings may not truly represent the actual sentiment surrounding a product.  For example, a review of rankings on Amazon.com revealed that the reviews for the majority of the products have an asymmetric bimodal distribution. For these products, the mean of the online product reviews does not necessarily reveal the product’s true quality, resulting in misleading conclusions about the product’s future success.  In addition, established products are at a disadvantage against new ones.  Consider a product that has received five rankings, four and one giving it an average rating of 4.8. If a new product enters the space and receives one rating, it will be ranked higher than the other simply because it has fewer ratings. 
4. Ratings may be false.  Take the case of the Whole Foods CEO who posted disparaging comments about a competitor on a message board while talking up his own company, later stating “Sometimes I simply played ˜devil’s advocate” for the sheer fun of arguing.  Other visitors may post false comments to artificially affect the rating, and these are not always removed from the calculation.

As more products are marketed and sold online, feedback-based ranking systems are increasingly common components.  Amazon and eBay were among the first to use visitor ratings to rank products and sellers, and in February, Yahoo launched its Buzz service, which asks readers to click on their favorite stories, then uses those ratings to determine the most popular articles on the web.

Why is this important?  Because accurate product ratings help predict that product’s success, and higher product ratings lead to more sales - or, as Allen & Appelcline state, the value of individual items (most frequently goods) rise or fall based upon the largely subjective judgment of individual users. So what can you do to ensure the rating system on your own website accurately reflects your customers’ views and the value of your product?

We’ve seen some thought around using analytical techniques and Bayesian mathematics to create better product rankings.  Some of the solutions explored include:

1. Adjusting ratings based on known biases.  Since some people rate higher or lower than others, one approach is to assign users a “User Optimism” value based on their rating history, and adjust the product’s overall rating based on the raters’ optimism value.  Another approach is to remove all ratings from people who have only rated 1 or 2 items, helping to eliminate new-rater optimism or “drive-by” fraud.
2. Weighting a product’s ratings based on the number of ratings received.  When a product has a small number of ratings, these ratings should count less than those for a product rated many times.  To achieve this, you can add a “magic value” into the algorithm that calculates a product’s average ratings.  This “magic value” brings products with few ratings closer to the average ratings of all products, then reduces its effect as more ratings are received, allowing established products’ ratings to float freely and more closely reflect the average of its ratings.
3. Adjusting the algorithm to account for bimodal distributions.  To account for products that only have ratings at the extremes, one approach is to use a dual-point estimation model to more accurately reflect customers’ views of the product and predict the product’s success.
4. Analyzing a customer’s rating history for fraud.  Someone who rates several products on your site should have a predictable rating pattern.  You can analyze individual customers’ accounts to identify and remove anomalies that might skew product ratings.
5. Encouraging customers to leave text-based feedback.  When a person takes the time to write out a product review, and knows that her name will be attached to it, she generally does a better job in her rating.  These reviews tend to be closer to the average than those without.

Although your customers’ product ratings may be imperfect, they can still yield insights and value with the application of a good set of analytical techniques.

Some recent academic literature has shown that companies using the standard Net Present Value (NPV) approach to calculate Customer Lifetime Value (CLV) and allocating their marketing spending might be losing substantial profits.

The standard CLV approach calculates the net present value (NPV) of all the anticipated cash flows coming in (revenues) and going out (expenses) over the projected life time for a given customer. Customers with a positive NPV are aggressively marketed and the ones which do not have a positive NPV are dropped. This is a pretty logical choice at the point of calculation- but only then. The existing conventional structure of calculating CLV is static and has the potential to leave money on the table for the company; because it does not account for the fact the company has the flexibility of abandoning a customer at any future point in time. Flexibility means options and options have a value.  The paper lays down a methodology (based on dynamic programming) to explicitly estimate the value of such options in the CLV calculation.

The authors of the paper cite the example of a specialty catalog company. They took data of 12 years and around 100,000 customers, and the difference in CLV using the traditional versus real options was as much as 20% in some cases. In fact, certain customers who had negative CLV’s turned positive when the value of real options was considered.

Retailers, communication companies and consumer finance companies who frequently use CLV as a decision making metric, are strongly recommended to read this paper.

A set of whitepapers for the executive who wants to leverage the data to make smart marketing investments:

1. Identifying attractive markets and understanding your customers
2. Increasing online sales
3. Focusing on profitable distribution partners
4. Improve customer experience through call center data
5. Retaining your customers

Customers calling call centers don’t want long waits and often perceive them as longer than actual.

At the Intelligent Transportation Systems Lab of the University of Minnesota, an experiment was designed where highway driving conditions were simulated in a lab (a Saturn was peeled off and a 270 degree screen in front provided the video). It showed how people value time under different situations. While almost all the respondents remarked that they had actually ‘driven’ for a shorter time than the actual simulation, they also noted that they had waited on the ramp before entering the freeway for 30 seconds or more when the actual controlled waits ranged between 5 and 12 seconds.

A vital component of interaction between customers and service professionals is the time it takes to respond to the customer’s needs and requests or more importantly, the perception of such service times. Adaptive queuing theories can play an important role in improving perceptions of customer service. It is non-ideal to keep a customer waiting for a long time when their needs are fairly straightforward. And in such cases they value their waiting time even more than they actually waited for, in some cases, many times more. While such experiences may be fairly commonplace and their pain all too well-known, businesses, save for exceptions, rarely apply adaptive queuing theories to correctly predict and plan for servicing an individual. Disney engages customers in distractions while they wait for 45 minutes in a queue for a 3 minute ride. Situational elements like music, lighting, color, employee visibility, social interactions, visible queue movement, number of counters serving vs. idle, a visible clock nearby all influence perceived time. Companies can benefit by putting more thought into the psychological aspects of waiting times and queuing while trying to increase service levels and influence customer satisfaction.

Firing customers is not something companies prefer to do for profitability. Fierce wireless reports that Sprint recently fired some customers because of their frequent calls to customer care.

Ever wonder how many calls to care can actually get one fired from the wireless company, now that the precedent has been set? We did some back of the envelope analysis to understand it better.

Assuming that the fired customer paid an average bill of $60 per month and that Sprint has 30% margin and average cost per call of $4, it will take 5 calls in a month to make one a negative value customer. However the trick of all lifetime value calculations is to estimate future value of customers because you can be unprofitable in one month but go on to upgrade to a highest plan down the road. Since there is no reliable way to predict it, we will be conservative giving the benefit of the doubt to the customer. So to get fired, the customers must be making way more than 5 calls a month (probably more than 10-15) and that too for a few consecutive months; so that Sprint had no hope of making a profit in future from these customers.

While it is certainly a sound financial decision, it will be useful to determine whether these customers had anything in common (type of problem they were calling for, expectation setting at sales channel etc.) which led them to call more often.

Photo Credit: Pappalicious

In any customer facing business, the channels used to reach out to your customer as well as the channels your customer use to consume your products and services have a strong correlation with overall profitability. 

From the B2C perspective, advertisements, stores, online, mailers, charity, etc. are different ways of reminding customers about your offerings. From a C2B perspective, again, stores, online, home delivery orders, etc. are ways in which consumers consume your products. 

However, the key here is to understand which customer prefers which B2C and which C2B channel. Right-Targeting Customers is as important as Targeting Right Customers! Someone who spends 14 hours a day in front of his computer and has no time to go to a store 5 days a week may prefer a home delivery channel. On the other hand, a student in a college is only interested in the bargain channel, irrespective of the inconveniences, maybe. This report also mentions how retailers need to manage their investments across channel against the scale and timing of their expected return. I would go beyond Ron’s Right Channeling [read post] to include all aspects of targeting under the concept of Right-Targeting. Having said that, I agree that today’s world is about multi-channel customers, and the need of the hour is to optimize channel returns, rather than just channel re-alignment/phase-out.�

Targeting Right Customers- Its equally important is to understand how channel profitability gets affected if you are not targeting the right customer. For instance, Wal-Mart, even with its Everyday Low Prices (EDLP), must be making money on some products/ some SKUs and these would drive the overall positive profitability. However, what if your customers are not buying your profitable SKUs? What if the draw that brings them there is not luring them to buy more? What if there is no up sell/cross-sell/bundling that happens there? And suddenly the business realizes that channel profitability is coming under immense pressure! (One of our earlier posts tries to answer the question of market basket analysis and product bundling). 

Re-Targeting - And last but not the least– taking the difficult decision of phasing a channel out. If it’s not generating any returns (directly or indirectly), the business needs to reconsider the cost of the channel. But what about retaining those customers who are loyal to the channel and have helped you get some mileage out of it? They need to be Re-targeted with a new offering/communication. Traditional retargeting refers to targeting customers who did not convert the last time, though!

This report (PDF) here talks about issues to be kept in mind when moving customers from one channel to another. I am sure Kevin would want to talk about Multi-channel customers being better than single channel customers, and without refuting his argument I would classify my argument as being restricted to customers that are being phased out of a channel. This example demonstrates one case where the retailer has effectively leveraged multiple channels over a period of time by effective use of catalogs and internet channel. 

In Summary

• Right Targeting – Know your customer. Cater to their needs through their preferred channel. It increases the share of wallet, as well as longevity of the customer
• Targeting Right – Pick your customers for each channel-product combination well. Know what and how you are selling and who will buy it.
• Re-Targeting – If it’s imperative that you move away from a particular channel, think about retaining your loyal channel customers. A little caring goes a long way in creating customer loyalty.

The qualifying argument behind all this is that if you are thinking about channel optimization, it needs to be a more concerted discussion based on your strategy/business objective, your data, your needs and your concerns. What we know for sure is that strong data analytics is known to help not only Targeting Right (which is the most common application), but Right Targeting, as well as Re-Targeting.

Our client was losing revenue from 68% of potential customers who were sold the service by the call center channel but did not complete registration process, which meant that they could not be billed.

One of the initial analysis we did was plotted the percentage of ‘sold but not active’ customers against the number service agents and found that a small number of the agents were disproportionately responsible for making the sales.

This led us to investigate the efficacy of the sales process itself and whether some agents were abusing the system for commissions. To answer the former, we did a customer survey while for the latter we started generating an agent-level report on a weekly basis. The report was able to identify the precise subset of agents who were trying to game the system and our client was able to take corrective action.

As the example illustrates, agent-level reporting is a very actionable tool and can be used for:

Metric Tracking:
Strategic metrics like efficacy of saves, sales quality etc. should be tracked at an agent level

Auditing:
For companies outsourcing a large portion of their call center operations, such a report can serve as an auditing tool

Compensation design and reporting:
It also can help to design and report a performance based incentive structure for agents

After porting my wireless number to one of the GSM cell-phone carriers, I called the customer care to activate my international roaming for my trip to India. It was a 40 minute long call, and the customer care agent explained that since I had been a customer for less than 90 days, he would need his manager’s approval to activate the service which would take up to 5 business days. When I landed at the Mumbai airport, the service had not been activated and 10 days later it is still the case.

The carrier not only lost potential revenue, incurred cost of a long care call but also caused customer dissatisfaction.

Is there any way for companies to avoid such one size fits all rules trap?

One potential way is to use predictive analytics to design the operational rules and automate front line decisions, to increase both revenue and satisfaction.

Expensive calls to customer care can be avoided by better managing core business processes. Most service providers are concerned with non-value added call volumes because they represent a significant cost to the business and offer little true opportunity for incremental sales. Often non-value added calls are triggered by problems or inefficiencies throughout complex operating environments, for example in claims processing. In our work we have found a stunning relationship between the “days to pay” intervals at several different types of insurance companies. As the chart below demonstrates, the probability of a claim generating a call to customer service increases dramatically as the claim ages. In this case claims that age beyond 40 days are basically guaranteed to generate at least one call, thus driving up costs needlessly.

So what? In this case, the company was tracking average claims “days of mail on hand” and believed that there were roughly four or five days of claims inventory to be processed. This faulty and misleading metric was well within the early part of the curve, where a call to care is highly unlikely. An accurate aging of the claims inventory revealed that the average payment interval was almost 17 days! This was well beyond historical norms and the operating assumptions that were used to forecast call volumes. By pinpointing the processing areas that had excessive aged inventories (using further detailed analysis of electronic data) we were able to devise a simple plan to reduce pockets of aged claims, put more accurate measures in place, and stanch call volumes.

Will ‘support tagging’ reverse the customer support model as we know it? As customers start going online to share their views about products and services and research their next purchase, companies have little to identify and understand such large amount of unstructured information online which relates to their products and services, much less to do anything about it.

And that is where the concept of  ‘support tagging’ or ‘beacon’ comes in. There have been recent discussions (1, 2 and 3) on the concept with some suggestions on how the model will work. The basic idea is that organizations will publish and publicize their high level reason code trackers or ‘tags’, they use classify their customer interactions – to be used by customers when writing any blog comment about the company’s products.

• for example, “office2007question” would have been a good tag the MS could promote with its Office 2007 product
• support tags, like “support:typepad” or “feature-request:iCal”, case experience tags, like “user-testimonial:nokia+n90″

These tags will make it easier for the company to search for customer feedback/support issues and potentially even address customer issues, right where the problem has been stated. In some ways, it reverses the customer support model, with companies seeking out customer issues and solving them rather than customers following company’s procedures to state their complaints.

A very vital secondary benefit will be getting customer feedback from a market research perspective. ‘Beacons’ will be a good addition to call monitoring and targeted surveys as effective mechanisms to capture customer feedback and link them to company’s rich interaction and transaction databases  (through data analytics) to identify and prioritize opportunities.