Data-Driven Work Cultures: NTT DATA’s Theresa Kushner On How To Effectively Leverage Data To Take Your Company To The Next Level

As part of our series about “How To Effectively Leverage Data To Take Your Company To The Next Level”, I had the pleasure of interviewing Theresa Kushner.

Theresa Kushner is the Data and Analytics Practice Lead at NTT DATA Services. In her role, she helps companies gain value from data and information, specifically how companies apply AI, ML Ops and overall governance to their everyday challenges. For more than 25 years, she has led companies in recognizing, managing, and using the information or data that has exploded exponentially. Theresa has also co-authored two books on data and its use in business.

Thank you so much for joining us in this interview series. Before we dive in, our readers would love to “get to know you” a bit better. Can you tell us a bit about your ‘backstory’ and how you got started?

Thank you for the opportunity to join this series. As for my background, I came in through the back door. I am a journalist by education and swore to my computer-loving dad that I would never go into computers. So much for that. I started my career at Texas Instruments in the consumer products division where I could use my writing skills developing merchandising materials for children’s learning aids. I got hooked on computer technology and marketing. When I left TI and joined IBM, I was still in marketing. It was a stint in Paris as the Director of Small and Medium Business Marketing for Europe, the Middle East, and Africa where I came face-to-face with the reality of data. There wasn’t any with which to market to our customers. We didn’t even know who our customers were because we sold through partners and never asked for the customers’ information. Thus began my life’s work.

I built a marketing database with IT at IBM, then moved to Cisco to do the same. During that time, I co-wrote a book with my IBM IT partner, Maria Villar. We were trying to educate the business on the value of data. The book landed me a job as the VP of Data Governance for VMware. There I built an enterprise information group that managed data, developed reports, and took on projects that only AI and machine learning could solve. In 2015, I combined my data background with the marketing expertise of Ruth Stevens to co-author another book on B2B data-driven marketing. In 2017, I went to Dell as the SVP of the Performance Analytics Group. Then in 2019 — to increase my understanding of how customers were really using data and AI — I began my work with NTT DATA as the solutions lead for data and analytics. My job at NTT DATA is to design solutions that customers find compelling and want to purchase. In other words, solutions that solve their problems or at least help them meet their challenges.

Can you share a story about the funniest mistake you made when you were first starting? Can you tell us what lessons or ‘takeaways’ you learned from that?

Data stories are usually not that funny. But I do recall a situation that was not so funny to our customer but taught my data team and I a very valuable lesson. We had a database that had names, email addresses, and sometimes phone numbers of customers, prospects, and partners. We used the source for mailings, specifically for our largest customer meeting. This large mailing happened annually and was attended by most of our customers and prospects. I oversaw the quality of the data but had been with the company only a month when the first mailing went out. We promptly got a phone call from the CEO of one of our oldest customers. She had received her invitation to be part of the executive attendees at the event, but it was addressed to Mickey Mouse at her address. What we learned immediately, and the hard way, was that data entry needed to be policed. We could fix the data after it was entered, but the cost of fixing data when you start to use it is about 10 times the cost of getting it entered correctly in the first place.

Is there a particular book, podcast, or film that made a significant impact on you? Can you share a story or explain why it resonated with you so much?

One film that has stayed with me since I took this job at NTT DATA is Steven Spielberg’s AI. It made a big impression on me at the time because of the ability of the main character — driven by AI — to keep learning. We are entering an age when we have systems that do exactly that. We don’t yet have systems that mimic human thought or emotions, but we are no longer centuries away from that promise. I think of that movie every time I talk to executives about AI. I like to remind them that AI algorithms are like children. They learn from what is around them. For AI systems, they learn from the data that is available. If that data is of poor quality, biased, racist, or worse, the algorithms — like children — will learn. it.

Are you working on any new, exciting projects now? How do you think that might help people?

I’m currently working on three new projects that keep me up at night! The first relates to digital humans. NTT DATA has a group of exceptional engineers in Denmark who are actively working on developing digital humans who can remember, respond to emotion, and provide information. The digital concierges will find their way to countless service desks as the service industry shrinks further from the Great Resignation. Digital humans will begin to fill in for the work of regular humans. You see that pattern today in Amazon as pickers work side by side with robots.

The second project is all things digital. At NTT DATA Services, we are launching a new program to better support our customers join the Digital Transformation movement. The pandemic was a wake-up call for customers to continue investing or start investing in digital transformation. Everything from going digital in the workplace to going online for your customers took on a new urgency with COVID lockdowns. Our program, called Digital Success, is designed to help customers begin or accelerate their digital journey. Of course, we do that by helping them get a good data foundation first.

Finally, my third project focuses on scholarships for women in big data. Personally, my excitement stems from the fact that we, as board members of WiBD, are creating tangible opportunities for women who want to enter the world of data, but may not have had the means or resources to do so before. Equity is the future of the workplace, and we pride ourselves on taking proactive measures to bring everyone into the room.

Thank you for all that. Let’s now turn to the main focus of our discussion about empowering organizations to be more “data-driven.” My work centers on the value of data visualization and data collaboration at all levels of an organization, so I’m particularly interested in this topic. For the benefit of our readers, can you help explain what exactly it means to be data-driven? On a practical level, what does it look like to use data to make decisions?

On a practical level, being “data driven” is a mindset. By that, I mean it is a way of problem solving that requires facts or data. It means that, before you make a decision, you consider the data or information you have about the problem and the proposed solution. You weigh the options and then you act. That seems very simple, but it’s very hard to do consistently within an organization because of one reason — we’re all human and we have biases.

It’s hard to be data driven, especially when the data tells you something that is contrary to what you believe. The more engrained your belief, the harder it is to be data driven.

Being data driven starts with the belief that data or information can provide a solid platform on which to make a decision. Let’s take, for example, the problem of which employees may be looking for a new position. Since the resignation rate has reached new heights in the US, this could be a problem for which AI offers a solution. In fact, Ginni Rometty, the former CEO of IBM, initiated just such a program. I’ll tell you what happened to that program in a moment.

First, you need to determine if you have data that could indicate that someone is looking to leave their job. Some examples of a few data pieces that could be used include:

1. Number of “vacation” days or absences in the last 3 month
2. Employee-expressed disappointment in a performance review
3. Employee request for salary information
4. Change in working habits — staying late on the job, leaving early, long lunches

How many of these pieces of data would you be able to accumulate from your working environment?

If you are like most companies, you probably do not have a lot of the information above for each employee. Data that might be available and helpful may also have privacy or risk issues associated with it. Even if you have the data and can produce an AI algorithm that can provide indicators of which employees are at risk, you could face the same issue Ginni had at IBM. Ginni couldn’t get the managers to use the information. Months after the algorithm was produced, IBM had to conduct extensive workshops to help managers HOW to use the information.

Using data is the key to being “data driven.”

Which companies can most benefit from tools that empower data collaboration?

The companies who benefit the most from data collaboration are those that put the customer first. But let’s start with what we mean by data collaboration, especially in the context of customer engagement, experience, and retention.

Data collaboration is the act of ensuring that data held in one part of a company is shared and made available to other parts. That seems like a reasonable statement; however, most large companies find that their data supports the same silos as their organizations. For example, financial data may not always be available to marketing and service data is often not associated with sales or marketing data. Then, there are partners who may support product sales or sell products on a company’s behalf. In the latter case, the customer data may not be shared at all with the supplier for fear of cutting out the reseller or distributor.

Tools like Snowflake can help coordinate data collaboration without risking loss of privacy or information. This affords companies the opportunity to explore other datasets outside the confines of their data infrastructure. For example, adding weather data to a system that controls traffic lights in a Smart City.

At NTT DATA, we have an AI application called Advocate AI for imaging. The AI application reads MRI scans and can detect anomalies faster. It removes the physician reporting variability that arises when two physicians read the same image. Advocate AI brings curated multi-site data access to clinical trial data and was designed to speed up the analysis of this data. The real value of Advocate AI is helping physicians make better decisions that help the patients live longer. The companies that benefit the most from data collaboration are those that use this collaborative power for the benefit of the customer.

We’d love to hear about your experiences using data to drive decisions. In your experience, how has data analytics and data collaboration helped improve operations, processes, and customer experiences? We’d love to hear some stories if possible.

I have two stories to tell that demonstrate the power of making decisions with data. Recently, we had the opportunity to respond to an RFP for a customer that we’d had for several years. We have been doing their service desk work as a managed service and this opportunity was for expanding into other areas of the customer’s business. The data side of this story lies in how we helped the customer understand what was happening in their business.

Since we have data on the customer interactions, we used it and did a deep dive on it for the customer. Although some of this information had been made available to the client via dashboards, no one had ever explored the unstructured data found in chatbot conversations and, in addition, no one had made the connection of the chatbot data to service tickets. By doing a word and sentiment analysis of the chatbot conversations, we discovered that 75% of the chatbot conversations internally were driven by vacation and holiday issues. Correcting this by providing information to employees on these topics allowed the client to cancel many pending tickets.

Another experience that was very impactful happened a few years ago. My team was generating a market analysis of our company’s customers, concentrating on which products were being purchased most often by which customers. The product marketing teams just knew that we were selling more of Widget A than Widget B. They were sure of it. Unfortunately, what we found by looking at the data was that Widget B was losing customers slowly over time. We didn’t learn that until we started looking at number of units instead of overall revenue for the product. Because the cost of the product had been rising as well, the loss of product sales didn’t immediately show. Data here made a big difference and the product team immediately started looking for ways to increase their market share.

Has the shift towards becoming more data-driven been challenging for some teams or organizations from your vantage point? What are the challenges? How can organizations solve these challenges?

At NTT DATA, we first encourage our customers to look at their overall business strategy and decide where data and AI can be best applied. Making that connection to strategy is important. And it’s not good enough to have a data strategy that is unconnected to the business. The shift to becoming more data driven lies in the exercise of drawing these connections. There are, however, many challenges to this quest, including:

• Lack of leadership. The quest to become data-driven needs to start at the top with a leader who understands the value of data. At VMware during my first meeting with Pat Gelsinger, he asked me why he didn’t have a sales report that everyone agreed had the right numbers. “And why do I get six sheets of paper instead of being able to click on my iPad?” His vision of what he needed helped us build an executive dashboard (that ran on his iPad) and gave him and the executive team the same set of agreed-on numbers. With the introduction of this report, executive staff meeting time allotted to sales reviews was cut in half. Everyone had the same numbers, and the team could address what actions to take instead of which numbers were right.
• Insufficient funding. Data projects are often seen as “fix it” projects. This usually means that the records in the data warehouse or data lake undergo some cleaning and processing. For three months after, the data looks “clean.” These are usually expensive undertakings and because the processes that created the bad data are not fixed, the problem comes back. Funding for data projects should be ongoing, funding a team who can transform the data and the processes creating it, managing it, and using it.
• Siloed thinking within siloed organizations. Leaders in finance, operations, HR, and sales often see the need for good data faster than IT. They are, after all, the users. However, each of these organizations can and will make the case that they need control over their own data and that, because of privacy issues or cited ills, they need their own databases. Not only does this approach hurt each organization, but it also keeps the company from serving their customers. Bringing these organizations together to decide what data is important to support the company strategy usually helps break down these siloes, but this is only possible if the company adopts a collaborative mindset as well.
• Failure to articulate desired outcomes. This problem arises often with AI projects. Too often we believe that AI is the magic bullet, that it can solve previously unsolvable problems. Sometimes it can, but articulating an outcome that the user sees as valuable is key to this magic. For example, if an AI project is going to provide a manager with information as to the productivity of her call agents, then the manager must be required to articulate what productivity means, how measuring productivity will change because of the AI project and what will happen with the productivity information once the manager receives it. Will she dismiss low performing agents? Will high performing agents be given more desirable jobs? AI projects should not be undertaken without full consideration of the outcomes and usage.

At NTT DATA, our tagline is Trusted Data Advisor, because at the heart of every data effort is Trust. Users often say, “that data is bad” which is code for “I don’t trust it.” Trust is an important element in business and THE most important element for data, built over time with actions that are predictable and understandable.

Ok. Thank you. Here is the primary question of our discussion. Based on your experience and success, what are “Five Ways a Company Can Effectively Leverage Data to Take It To The Next Level”? Please share a story or an example for each.

• Use it to better understand your customers. My own journey into data began with wanting to know more about my customers. In today’s digital age, you can know a lot about your customers from their transactions, their website visits, their every move on the internet. Combining that data to help you better understand a customer journey can be worth something to your company. At Cisco, we developed a predictive model that was used with our sales teams to give them insight into what product the customer would most likely purchase next. Through that one program, we generated $984M of incremental revenue in the first year.
• Concentrate on key processes, data and THEN systems to support both. Technology comes last. Streamlining processes helps you to better understand what should be happening with your systems. A clear business process also helps when evaluating software or technology. Understanding the processes gives meaning to the data and how it moves from one process to the next. Many large ERP and/or CRM installations make the mistake of selecting a software application BEFORE the processes are clearly understood. This results in modifications to the software that are expensive and time consuming. It also prohibits updates to the systems that are so valuable over time.
• Employ data collaboration. Being able to gain agreement on data meanings and usage is important to becoming a data driven organizations. Often organizations have different meanings for the same pieces of data. Two organizations I supported in the past were constantly at war over sales discounts. The number from sales never agreed with the one from finance. Then one day, a very smart data analyst on the team sat down and asked each organization to define the data elements that made up the sales discount number and the timing associated with gathering the information. Surprise, there were differences.
• Make everyone in the company data and AI literate. The Data Literacy Project estimates that only one-third of us can confidently understand, analyze, and argue with data. Imagine if your company had only one-third of its employees who could read. Understanding data is as important in today’s world as understanding how to read. Now, ask yourself, what percentage of your employees are AI literate? How many can articulate what an AI project can do, how it is managed, how the data is selected, how the output is reviewed and monitored? That’s AI literacy. Both literacies are essential to moving to the next level of data maturity.
• Reward publicly and often those who demonstrate the mindset towards data driven. One of the tenets of managing a transformation lies in rewarding the right actions — both good and bad. At Dell, we were beginning to dive deeper into AI and wanted to prove that if we worked together across the organization, we could deliver valuable projects. One of the ways to get the attention of the data scientists and their managers who were involved in projects across dozens of organizations was to provide them with a forum for showing off their work. The AI forum we created was a showcase and through it we discovered that some teams were working in isolation on the same projects. But overall, this forum gave us the opportunity to highlight good work that was the basis for being a data driven organization.

The name of this series is “Data-Driven Work Cultures”. Changing a culture is hard. What would you suggest is needed to change a work culture to become more Data Driven?

Two things that are most important when creating a data driven work culture are Mindset and Leadership.

Mindset begins with everyone in the company having a growth mindset that allows them to be curious. Curiosity then leads them to question why processes work the way they do, or what is the best way to accomplish a task or when should you introduce a new product, service, or benefit? Mindset leaves them open to new possibilities and changing a culture must begin with possibilities.

Second, leaders count in this game. There is a group of leaders called the Data Leaders that has published the Data Manifesto. This international, volunteer data management group has recognized the issues with data and started a grassroots effort to help business managers and executives understand the value of data and treat it as an asset. Their manifest begins with the statement, “Your organization’s best opportunity for organic growth lies in data. Data offers enormous untapped potential to create, competitive advantage, new wealth and jobs; improve healthcare; keep us all safer; and otherwise improve the human condition.” I have the pleasure of sitting on the board with these leaders who are out to change the world by being both supporters and provocateurs of business executives today. Movements — whether political or social or commercial — all start with one person or one group who has a passion. Changing a culture must begin with a movement.

The future of work has recently become very fluid. Based on your experience, how do you think the needs for data will evolve and change over the next five years?

My crystal ball tells me there will be two tales to tell in five years. The Yin and Yang of data — one good, one not so good.

First, the more data we get, the more we will want. Not only are the volumes of our data increasing exponentially, but the variety is as well. Structured data from company applications is probably the least of our volumes today. Data from the internet, data from things, from images and videos are all causing this upheaval. But still, we will want more of it. We will want it faster, some even real-time. And, we will want inferences drawn from this data and have those inferences readily available at our fingertips, on our phones and on our wearables. We will want new ways to manage this deluge of information because we will start to monetize it, sell it to others, and profit from our data management. Data will become so valuable that you will begin to see a commercial infrastructure — a new industry — developing to monetize data. AI will provide new capabilities for this kind of data management by processing data faster and applying it to problems more readily. That’s the positive side.

Second, we will begin to see the downside of having lots and lots of data. Data, and the AI applications using it, will be legislated more. We’re beginning to see states pass laws today aimed at ensuring that AI applications remain unbiased and trustworthy. Those statutes will increase state by state and maybe even find a voice in Congress. And because of our monetization efforts, data may become so valuable that the ransoms paid for it will make the Wall Street Journal. Data security will remain an issue and only get worse as quantum computing begins to deliver on the promise of breaking down blockchain and other encryptions.

Does your organization have any exciting goals for the near future? What challenges will you need to tackle to reach them? How do you think data analytics can best help you?

We have a program for Data and Intelligence that will help prepare our customers for digital success. It’s called the Digital Success Program and it’s a worldwide effort concentrating on helping customers begin to reap the value from data and AI. Our goal for the program next year is to identify a handful of high-profile accounts that want to begin or continue their digital transformation journey. We will need to tackle challenges such as ensuring the customers have a well define project to which AI or Machine Learning can be applied and that the results can be used and understood. We may also have to tackle data issues. For example, once the problem is identified, we will have to ascertain if the data to help solve it is available, accessible, of the proper quality, and unbiased in any way. Developing the algorithm will be fun, but one of the issues we often encounter with clients is that they don’t understand that developing an AI algorithm is much different than developing a software application. AI is much more trial-and-error and clients often get frustrated with the experimentation that data scientists must go through. So, we will need to do some good AI and data literacy training to manage client expectations.

Using our own data to help us manage with our clients is one of our goals for FY22. This means that we are looking at information that can help us identify the right people within an account and their propensity to engage with us.

How can our readers further follow your work? Best way to follow is LinkedIn.

To communicate directly with me, you can email me at theresa.kushner@nttdata. You can also follow me on LinkedIn and Twitter.