Transitioning to a Data-Driven Organization

Introduction

I gave a talk at the CDO Summit in Singapore on 18 June and thought it may be worthwhile to address some of the points and provide additional detail on the slides I used.

Sections below are listed by slide content and provide extra explanation.

Kardashev Scale

This is simply applying the Kardashev Scale of technological advancement to the advancement of data usage in organizations. For more detail see the article I wrote on the Kardashev Scale of Data Maturity.

Culture

The main point of the talk is that no matter the data maturity of the organization, and no matter what kind of data strategy you have developed, the culture surrounding data (and the company culture more broadly) will be the deciding factor in whether or not the organization continues to progress in the usage of data. To use the phrase often attributed to Peter Drucker:

…culture eats strategy for breakfast

Consider what this means for your organization. Does it have the right culture? If not, your data strategy probably isn’t worth much.

Defend forecasts against actuals

This topic is one that crushes the hearts of many people, and the question “When was the last time you had to defend forecasts against actuals?” got quite a few laughs and smiles at the conference. The main point is that you can use that question as a proxy for the data maturity in an organization.

Do people expect the forecasts to be absolutely correct? Is there ever discussion or acceptance of error bars around forecasts? Forecasts are, after all, nothing but a guess, so they should be treated as such. If your organization is having intense debates or asking for extensive justification about why any forecasts differ from actuals, then the culture in the organization simply is not ready to advance in terms of data usage. Yes, certain things in businesses can be forecasted more accurately than others, and yes businesses with a long history of stable trends may be able to forecast accurately, but that doesn’t change the fact that a forecast will necessarily have some kind of error. Unless your business has those kinds of ultra-stable histories, having pointless debates when forecasts are within error bounds is downright toxic. If you need to produce some kind of forecast to satisfy investors or other people, it’s best to make an educated guess based on what kinds of business developments you expect in the pipeline, but that’s definitely not a Data Science problem, it’s a BI/FP&A problem.

Some businesses deserve credit for trying to get around this problem with approaches like rolling forecasts (e.g. quarterly) which are always updated. This resolves the problem in a great way because there is no budgeting at that point, only the rolling quarterly forecasts. See Beyond Budgeting for more details.

The problem with painting a picture of data

There were quite a few presenters at the conference who claimed that one of the primary functions of a CDO is to paint a picture of the data, to be a data champion, to help people understand all the great things which can be done with data, and so on. While this is somewhat true, and may be very much the case in some industries, I’ve found that the bigger problem is one of inflated expectations. In other words, the picture has already been painted and it’s very different than the reality.

This is extremely difficult to address, especially since a natural reaction of some non-technical people is to assume that the CDO or person trying to temper expectations is actually incompetent. If your organization is going through data or technical leaders one after the other, the issue of overly-inflated expectations from CXOs or board members may be the root cause.

The underlying issue is that people who don’t really know much about the details of implementing data programs and transforming organizations to use data more effectively are being bombarded with messaging from vendors and other non-technical people which states that (1) the company’s data is worth a ton of money and that (2) the vendor can help the company to extract the value from this data in a very short time.

Often, the first point is wrong. Sometimes data simply is not valuable. However, the second point is almost always wrong. Most vendors cannot help you solve your data problems since the problems are usually cultural issues which a vendor cannot change, or technical in a way that’s specific to your organization and how it has developed over time. The typical outcome from non-technical people talking with vendors is that the technical people in the organization will be frustrated because they must again tell non-technical people that what the vendor has promised as possible really isn’t. This causes a non-technical person to lose trust in technical colleagues, everyone to lose trust in the vendors, and results in a sour mood all around.

My suggestion is that non-technical people do not talk to vendors and do not forward them on to technical staff. One job of technical staff is to stay abreast of the solutions available, and the only reason the vendors are talking to non-technical people is because non-technical people sign the checks or because non-technical people don’t have the knowledge to fully understand the problem, therefore being easier to deceive. It’s better to just save everyone the trouble and frustration and not start those conversations in the first place.

A huge amount of time in the data space is wasted on trying to reduce other people’s inflated expectations. Please don’t contribute to this by painting unrealistic pictures of data and its usefulness.

The devil is in the details

As mentioned above, there are vendors selling all manner of solutions, most of which are not appropriate for your business. The details about whether or not a solution will work for your business is the responsibility of the technical people in the business and should be treated as such.

I’ve seen many businesses get sidetracked on projects, some of which have gone so far over time and over budget that the business has gone bankrupt, simply because a non-technical person was making decisions about technical products or projects. This is often due to being promised things by a vendor.

I have personally been involved in the cleanup and aftermath of a non-technical Director of Product telling a technical team that they must use Hadoop to solve a certain problem because the business needs to work with Big Data. The result was a 60 node cluster which was used for, among other things, processing a 360kb (yes, kilobytes) job.

In short, just because a vendor is telling you what you want to hear, doesn’t make it true, and non-technical people dictating technical details of projects is a huge red flag.

Don’t sell to people who aren’t buying

Taking a larger view of the whole topic for a moment, what is sometimes happening to data leaders in organizations is that they are trying to sell a topic or a dream to people who aren’t interested in buying. If someone isn’t open-minded about the potential of data in your organization, and you have to paint an unrealistic picture in order to convince them, then you’re contributing to the problems mentioned above. In addition, you’re also wasting your own time and energy and doing harm to the organization.

If you feel like you’re having to sell to people who aren’t buying, your best bet is to leave and go to an organization which values your abilities. There are plenty of organizations who understand they need to advance their data capabilities, and it won’t be you trying to sell that story to them, but them trying to sell you on why they provide you with the best opportunity to use and grow your data expertise.

Kardashev Scale (data)

This references the Kardashev Scale of Data Maturity post which I wrote previously.

From Type 2 to Type 3

This is an overview of a previous post I wrote on moving from Type 2 to Type 3 data organizations.

Conclusion

The CDO role should focus on real problems in the business (e.g., fraud detection, recommender systems, data quality, etc.) and not get bogged down in useless debates about forecasts versus actuals and so on. There can be a BI or FP&A team who handle those topics, and the policies and procedures for baseline business reporting are already settled business. It’s simply a matter of execution. You probably don’t need a CDO for that, and if you bring one in thinking that they will be a magic bullet then everyone will be frustrated. Even worse, the CDO will probably become the whipping boy for any data problems of any kind. Simply having a CDO does not eliminate data problems in an organization, it only puts the organization on a path to minimizing data problems and starting to have more data focus in products and services.

In the end, the focus must be on people more the processes (hello Agile Manifesto) and there must be an understanding that cultural transformation and enablement of people to use data more effectively in their daily work is far more valuable to the business and its long-term health than some quick wins gained from addressing the low-hanging fruit. The difficult part is that such changes are hard to measure and therefore may rarely, if ever, get the credit they deserve.

Good luck.