The Difficulty Of Obtaining Good Data

Are you also annoyed by the emails you get after you have been in a hotel or booked a rental car. For the hundredth time you are asked: “How satisfied were you with our service? Please help us by answering a little survey.” Oh, dear. It gets bad when the “small survey” leads to a never ending chain of questions and at the very end the hint to write a second rating on Tripadvisor. I’m already thinking about creating a blacklist of companies that send out surveys. Why? Because my effort is high, but a benefit for me is not visible.

But that is exactly my job now. Getting data from customers who use our software. And this data, this feedback is important. It’s how we can ensure that we continue to develop our solutions in the right direction. It’s all about collecting data and providing benefits for customers.

Automated Or Manual Data Collection

In many cases data is collected automatically. When you install software, you will be asked if you want to provide data to the developers. Do you allow this? Privacy experts often advise against it. However, I do allow it in individual cases if a software is important to me and I trust the manufacturer. Whether this trust is justified is a completely different question.

But how relevant is such automated data collection? Especially in the B2B environment, I hear time and again that no data collection is allowed in production systems, but only in test systems. But then the collected data can be misleading. Whatever you do when data is collected automatically, you should check exactly how relevant the data is.

You can force the collection of data from production systems, for example by linking it to support contracts. Only if data is sent, there will be good support. More and more companies are making this a requirement. Via the Internet of Things, machine parts deliver information to servers to improve maintenance. I think this is very interesting, but in the data center environment I still see a lot of skepticism among customers.

Sending email surveys is also part of automated data collection for me. But the feedback rate can be low. The questions must be asked in a way that prevents misunderstandings in interpretation. We use email surveys in part because it scales well. But at the same time the data quality suffers because relatively few answers come in, and sometimes the answers do not fit the questions.

I currently collect my data mostly manually by interviews. This is more complex, but the quality of the data is extremely high. The biggest hurdle here is to find the right people who know how to use our software and who are willing to answer questions. We try to scale this up through our sales teams and product champions (our fans). You have to remember that it is a give and take. Just asking for information without delivering anything – that’s not good. We directly provide special technical information and tips, and indirectly provide better software once the feedback has been implemented. In this partnership we are making progress.

Connecting The Worlds

We plan to link the automatic and manual data collection. Automatically, as much relevant technical information as possible should be collected. This data can then be supplemented manually via ratings or information on priorities on the customer side. Let’s see where this journey takes us.

Soil Samples, Power Generation, and IT

Sometimes I am surprised how seemingly completely different areas are closely connected. When I built the first dashboards for my data to use our solutions, I saw that the quality of the data had room for improvement. Some data was missing or outdated. And yet I was already able to present insights and results for our product management.

Last week, I spoke to a friend of mine who works as a chemical assistant in a government agency and analyses soil samples. She also collects data and provides dashboards and overviews. Her data is also incomplete or incorrect. But she too can generate valuable insights.

This was followed by a conversation with another friend who collects data on electricity generation at an energy producer and prepares it for reports to the management. He, too, is struggling with data quality.
I find it very exciting that all these different departments end up having the same challenges. And the same two strategies always emerge:

  1. show the best you can with the data you have.
  2. work on the quality of your data.

But don’t try to boil the ocean. You will never have 100% correct data. Make sure that the errors don’t twist the message of your dashboards. I work with two views of our data: An internal view that reveals gaps in quality, and an external view that helps our stakeholders make decisions. In doing so, we have to provide interpretation aids as long as there are gaps in the quality of the data.

And most importanty, we need t clearly state the quality of the data when we build dashboards.

I’ll Be A Kind Of Data Scientist

I have recently started working as a “Senior Customer Adoption Engineer”. This is a kind of data scientist who helps to gain information about how modern technologies are used. So better decisions can be made about future software developments. This in turn benefits our customers. I am very happy about this new task. I think it is very important. Let me explain why I think so.

Data can save lives

When we hear about large amounts of data, this is often accompanied by concerns about our privacy. But as is so often the case, there is another side to it. Large amounts of data can help us make vital decisions. This is particularly visible during the current corona pandemic. The more information we have about the status of the infections, the more accurate and reliable this data is, the better decisions we can make. Decisions that can save many lives.

That’s why scientists are working feverishly to increase testing capacities for the corona virus. That’s why politicians and decision-makers in Germany and many other countries listen to scientists. The scientists’ recommendations can be made on the basis of a good database.

Data in product developments and new technologies

My employer VMware delivers new technologies to better create, run and use applications. Product managers and business units must make decisions about what features are needed, or what improvements are important or needed quickly. Some decisions are made on instinct, but they are often based on existing data.

There are business figures such as revenue, licenses sold per solution, pipeline, and other relatively easy to obtain information. But there is also another side: What are customers doing with the technologies? How are they used? What benefits are particularly important, what are the difficulties? We have a lot of customers, so we use analysis of support requests, but also anonymously sent information from our products, if the customers allow it. You can get interesting data through these automated channels.

But really valuable are the data that reflect the subjective perception of the users. How well do the new technologies and solutions perform the tasks for which they are intended? I am now working on improving this kind of research in my new team. There’s a wonderful book on this topic that I would recommend if you are interested in gaining customer information: “Lean Customer Development” by Cindy Alvarez. The subtitle is “Build Products Your Customers Will Buy”, and it’s full of insights and practical tips on how to approach customers, be it via email, phone interviews or on-site interviews. The book is valuable for start-ups and also for established companies.

I am working on ways to provide useful information that generates valuable insights. Our customers should benefit from this because they get even better solutions, but it should also make the work of our product management easier. It should help them to make the right decisions.

The Eel and The Discipline of Small Steps

Have you ever tried to hold on to a live eel? You’ll hardly ever succeed. I grew up in Northern Germany at the Steinhuder Meer, and there are eels there. With my school class, I went to an eel smokehouse once, and we were allowed to try to hold an eel. It slips through your fingers. Nobody could hold it for more than a few seconds.

Sometimes it seems to me that the business value of a new technical solution is like an eel. You’ve invested millions in new software or services, and in the end you’re not sure whether this investment has delivered measurable added value for your own company. This seems to be a trend across all industries, but especially in modern IT such as cloud computing, IT departments have a hard time. Vendors are reacting with new roles such as Customer Success Manager. A search on LinkedIn for this job title yields 65,671 hits today. These people help customers to realize the added value of a solution.

In an ideal world, a product delivers its business value after installation and everyone is happy. But the world is not ideal. Especially solutions that involve change of operational processes, that are supposed to deliver particularly high added value, require a change in user behavior. That starts at the Apple Retail Store, where you can get a demonstration of how to make the transition to Apple products work. But this is even more true in large companies. This is often referred to as operational transformation, the change in IT operations.

That’s why I’m a big fan of small steps: Think big and start small. If the value of a great idea is visible in a first implementation after a short time, then the IT manager can provide management with more reliable predictions about future business value.

Look for a concrete use case that is close to the business. Define how you want to measure success. Pay special attention to how you want to measure the success of a business transformation. And don’t wait too long until the first milestone is reached.

When there is a special relationship of trust between customer and supplier, sometimes very large projects are initiated and new investments are made before the previous project has delivered measurable business value. This can work, but in the long run it is a risk for both sides. Think about the discipline of small steps.

Like a Broken Marriage

There are cases where I think IT and business are like a broken marriage, and my work is that of a family therapist. What makes me think so?

Well, at a meeting of IT specialists I once asked which of the IT experts has the pressure to provide infrastructure faster. No one had come forward. They all said their work was okay.

A week later at the Machine Learning Conference I asked a few people where they run their applications. They said in the public cloud. When I asked if they were considering doing the same in their own data center, they gave me a big look: “I would never ask my own IT department if they were running machine learning applications. They are way too slow to deploy.”

No wonder IT staff feel no pressure, that business doesn’t even ask for faster deployment because they have given up.

It’s like in a marriage where the spouses have given up communicating. If you want to solve this, it’s hard work.

Now there are certainly IT departments that are working well with their customers in their respective business areas. But, dear IT people, are you sure that you know all the requirements of the business units? Are they still talking to you, or have they given up? It might be a good idea to validate assumptions explicitly. Maybe there is still unused potential for improvement. And dear business departments, have you asked your IT department lately if they could react faster? Perhaps you have overlooked potential in your own company?

If IT reacts too slowly to business requirements, then it has very little to do with technology, it’s all about processes and team structures. And above all it is about communication. Maybe you are getting help to get communication back on track.