What to Do When People Draw Different Conclusions From the Same Data

Walter Frick writes for HBR:

That famous line from statistician William Edwards Deming has become a mantra for data-driven companies, because it points to the promise of finding objective answers. But in practice, as every analyst knows, interpreting data is a messy, subjective business. Ask two data scientists to look into the same question, and you’re liable to get two completely different answers, even if they’re both working with the same dataset.

So much for objectivity.

But several academics argue there is a better way. What if data analysis were crowdsourced, with multiple analysts working on the same problem and with the same data? Sure, the result might be a range of answers, rather than just one. But it would also mean more confidence that the results weren’t being influenced by any single analyst’s biases. Raphael Silberzahn of IESE Business School, Eric Luis Uhlmann of INSEAD, Dan Martin of the University of Virginia, and Brian Nosek of the University of Virginia and Center for Open Science are pursuing several research projects that explore this idea. And a paper released earlier this year gives an indication of how it might work.

I believe it is best practice to have multiple analysts look at a problem to at least devise what their methodology would be for a certain problem.  In fact, I always like to take a crack myself when the problem is particularly difficult, just so I have an idea of what the data looks like and how certain variables are influencing the results.  

I think too many executives are unwilling to dig into the data and work with a problem.  I believe it is very important to have a deep understanding of the data issues so, as an executive, you can make better decisions on how to guide the team.  Many times the answer is not a deployable model, but a data mining exercise that will glean some testable hypothesis.  

Though most companies don’t have 60 analysts to throw at every problem, the same general approach to analysis could be used in smaller teams. For instance, rather than working together from the beginning of a project, two analysts could each propose a method or multiple methods, then compare notes. Then each one could go off and do her own analysis, and compare her results with her partner’s. In some cases, this could lead to the decision to trust one method over the other; in others, it could lead to the decision to average the results together when reporting back to the rest of the company.

“What this may help [to do] is to identify blind spots from management,” said Raphael Silberzahn, one of the initiators of the research. “By engaging in crowdsourcing inside the company we may balance the influence of different groups.”

I do believe in internal "crowdsourcing".  The minute tough problems start to be outsourced, the company loses the great insight their analysts and business owners have that can bring insight tot he data that many analysts outside of the company could never understand.  I truly believe analytics is art and science, but too many times the art is under appreciated.  

When it Comes to Data, Small is the New Big

A great example of taking the data you already have by Anthony Smith:

Customer data is a commonly unrecognized superpower. Companies that provide software as a service (SaaS) are especially likely to have massive amounts of uncategorized, unmanaged customer data, creating a well of potential that largely goes unused. This data, when analyzed properly, can provide companies with unlimited opportunities to improve product functionality, increase customer satisfaction and stimulate business growth.

As I wrote in Data + Action = Insight and Back Again, big data is not necessarily the next key driver for a business.  The next key driver may be harnessing the data you currently have and using that to glean more insight.  Once you have this insight, you can develop new strategies and tactics to deliver targeted content and create great customer experiences, which in turn lead to increased revenue.

Anthony talks about how he did this with his company and I believe so may companies can do this too.  Make sure you have exhausted all current data possibilities before leaping into "big data".  The data you have may be the next key driver for your business.

Busy is not a Strategy

One of my favorite people once taught me the mantra of "Busy is not a Strategy".  So many businesses use the wrong metrics or KPI’s when measuring success of the business.  For brick and mortar companies, their eyeballs tend to deceive them and they use that as their main metric (we were so busy).  For other industries it is market share.  How many widgets can we sell.  The problem can be using the wrong KPI’s along with having the wrong culture can lead to an unprofitable business.

I have implemented the “Busy is not a Strategy” with resounding success before.  We had a casino/hotel in a declining market that had 1,800 rooms.  They were moderately successful considering their location, but they were using the wrong metrics.  Their KPI’s were hotel occupancy and casino revenues.  Now anyone who knows the casino/hotel business is going to ask, what is wrong with those metrics?  They had good casino revenues for the market and an occupancy of 87%.  Most anyone would love these numbers.  Plus, they were really busy.

When we took over the business strategy of the property we saw to get these impressive numbers, there were a lot of giveaways and very low hotel room rates.  To drive the wrong metrics, they were servicing a large number of unprofitable guests.  The belief was if the hotel is full, more profits would eventually flow to the bottomline.  There was just one problem, the other centers of business were not large profit centers and the customers coming in at very low hotel rates did not gamble, because they didn’t have a lot of money.  

To increase profits, we decided we were not going to busy, we would focus our attention on the best customers and try to drive more frequency from these guests while sacrificing the low-end of the business.  This resulted in decreased occupancy and decreased casino revenues.  Uh oh.  Hotel occupancy went down to 44% and casino revenues were down 10%.  The operators were crying “the business is being ruined”.  Even competitors were coming over and asking the operators “are you going to be able to remain open until the end of the year”.  There was pure panic.  That was until the financials came out.  EBITDA was up 100% for the quarter.

By focusing on the best and most profitable customers, this property saw increases where it mattered most, the bottomline.  How did this happen?  The expenses to drive the KPI’s that were important to this property were astronomical.  They were essentially competing for market share instead of profit.  What happened through time, is the best customers started to come more often as that was the new focus of the property.  Casino revenues started to increase through time to levels much higher than before the strategy change, however occupancy remained at 44%.  They did this by focusing on:

• Increase frequency of their top tier from the players club
• Increase hotel room rate
• Target giveaways to the more profitable sector of the database
• Increase customer satisfaction of the best customers

This is very similar to what I see is happening in the phone industry.  There are many manufacturers and most of them are focusing on “Busy” as a strategy.  Now the metrics for busy in this industry are phones sold and market share.  Android accounts for approximately 80% of the worldwide market share for phones sold.  Yet when it comes to profit, that metric is almost reversed.  In fact it is a lot less than 20% in the last quarter.  So how can this be?

The phone manufacturers are selling basically the same thing.  They run Android software that they manipulate in small ways, but all the apps are compatible with their competitors.  This creates an experience that cannot be differentiated in any way but price.  This is the same thing that happened in the PC industry.  All manufacturers ran the same operating system, Windows, and they had to compete on price which forced them to make deals of adding bloatware onto their machines that destroyed the customer experience.  This is where the phone industry is heading.  When price is the main differentiator, businesses eventually will go out of business unless they can outlast the competition.  

So these OEM’s sell many millions of phones to increase market share which leads to…  To what?  I don’t know.  From what I have read these manufacturers have a decent amount of customers that are buying new phones, but they are buying them for the price.  So the manufacturer sold an unprofitable phone so they can gain a customer who will buy another unprofitable phone.  That doesn't sound like a sustainable strategy.  There is nothing that differentiates the experience of the customer enough to make that return customer more profitable.  It is a vicious cycle.  

The only company that is running a different strategy is Apple.  Apple is making almost all of the profit in the phone industry by having a differentiated product that is customer focused.  Apple is doing the same thing in the PC industry, their Macs account for about 10% of the market, but more than 50% of the profits.  Apple has been able to run the “Busy is not a Strategy” strategy to ultimate success.  Sure Apple sells a lot of phones and they would like to sell more, but these sales are the outcome of their strategy, not the focus.  Apple has a culture that is design focused which leads to a product that has a better customer experience.  

Apple is dominating the phone industry because they do not bow down to the marketshare gods.  They focus on the customer first through their design culture.  They make profitable, differentiated products which bring in the majority of the profits in the industry, which then allows them to spend more money on R&D to create more products and services to keep their customers in the ecosystem.  These customers buy new phones at a nice profit which creates a beautiful cycle.  All because Apple is NOT implementing “Busy as a Strategy” 

Managing Your Mission-Critical Knowledge - HBR

Martin Ihrig and Ian MacMillan for HBR:

Tantalizing as the promise of big data is, an undue focus on it may cause companies to neglect something even more important—the proper management of all their strategic knowledge assets: core competencies, areas of expertise, intellectual property, and deep pools of talent. We contend that in the absence of a clear understanding of the knowledge drivers of an organization’s success, the real value of big data will never materialize.

This continues to be a theme I am seeing with "big data" and I wrote about here.  This applies to "regular ole" data also, the key is to apply the knowledge and insight of the operation with the data to create actionable strategies and tactics.  Data analytics, big and small, is a combination of Art and Science.

This is a fascinating paper and one that I would recommend reading.  It attempts to identify company knowledge to find strengths and opportunities within that base.  It also attempts to spread that knowledge and expertise throughout the company to different divisions, which would assist on where to use that knowledge for the most profitable or strategic initiatives.  Very high level and difficult work, but very valuable.