TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Empowering Everyone to Make Decisions with Confidence

Focusing on these five points can help your enterprise democratize its data science successfully.

• By Gerrit Kazmaier
• April 2, 2019

We live in an era in which people, places, and things are more digitally active than ever. As a result, data volumes are growing at a phenomenal rate. In a report from last November, IDC predicted that the global datasphere will grow from 33 zettabytes in 2018 to 175 by 2025. The size and complexity of data has outgrown humans' ability to draw meaningful conclusions from it. The challenge lies in making this data "human-readable" to extract useful information to support decision making.

For Further Reading: How and Why Your Enterprise Should Democratize Data Science The Care and Feeding of the Citizen Data Scientist The Rise of Automated Machine Learning

To do this, organizations have turned to data scientists to help translate complex data into something useful for business optimization. However, the complex combination of skills and experience needed to do this job well means that demand for these specialists is significantly outstripping supply. Research company Gartner predicts that "through 2020, 80% of AI projects will remain alchemy, run by wizards whose talents will not scale in the organization."

To compensate for the shortage of data scientists and strengthen trust in the role and the benefits of machine learning, companies are increasingly looking for ways to improve the skills of their employees and bring analytics and data science to a greater part of their workforce.

What should these companies do? Here are five focal points that can help accelerate data science democratization and success.

1. Understand that good data is better than more data

Even the best predictive model won't produce reliable results if its input data is of low quality. Poor outcomes will lead to, at best, frustration and, at worst, a complete rejection of the technology.

As a golden rule, prioritize quality over quantity. Good data is better than more data. To know if your data is reliable -- complete, accurate, and consistent -- you need to understand where and how it came together.

2. Prepare for the necessary cultural shift

Data literacy is clearly an essential skill in the digital age. It's true for everyone in your organization, from the boardroom to the shop floor. Improving employee skills so workers can use, understand, analyze, and especially question data also requires a cultural shift across your entire enterprise.

Workers can fear change or even be unwilling to change. Employees might need to be reassured that the workplace changes are not a matter of artificial intelligence (AI) replacing their jobs but supporting them in their current jobs. Employees may be concerned about having to work with new technology and feel underqualified for the new challenge. Understanding the benefits of AI to specific roles and being prepared for change is critical. Once the benefits are evident and the resulting changes begin to be more obvious, your enterprise needs open channels of communication for feedback that both celebrates the successes and addresses any challenges.

3. Learn by (unintentionally) doing

Although training courses play an important role in improving skills, and guided, self-service learning options such as on-demand webinars, curated blogs, and video playlists offer flexibility, ultimately confidence in new technology comes from first-hand experience.

Just as no one needs to be an electrical engineer to use electricity, no one should be required to be a data scientist to experience the benefits of AI. The goal should not be to turn a company's marketing professionals or accountants into data scientists but rather to augment their roles with intuitive, easy-to-use technology. The technology should feel like a natural (and welcome) tool for the job, not an additional task to be mastered.

Exposing users across the organization to AI through augmented analytics will help drive the culture change required for data science acceptance. Intelligent technologies can proactively point out what an enterprise's data means, drawing attention to any unusual patterns and helping to reduce bias in a worker's interpretation of data. Natural language processing allows users to find answers through a search-engine-like experience.

These technologies offer instant gratification -- a new insight that requires little effort from users. It might be just a small machine-generated alert that suggests there is something more to learn about the forecast or the report they are looking at, but once users have experienced it, found that its prediction was correct, and benefitted from its insights, their confidence in the technology naturally grows. The result is a healthy curiosity to see and do even more.

4. Start small and scale up

Just as an ostensibly minor feature can kindle more interest in AI-driven (and AI-enabled) insights, a relatively small but successful project can become a lighthouse project to be scaled across an entire enterprise.

The key is to start small, with a concrete use case and a clear overview of the goals to be achieved. For such initial projects, it's crucial to have the right employees in the team. As explained by André Sionek's "Do you really need a data scientist?", someone with a scientific mindset to identify problems and test hypotheses is more valuable than someone who deeply understands algorithms.

Find the users in your organization who are curious, self-motivated, and comfortable with uncertainty: someone analytical who is willing to try and fail -- frequently. These traits, combined with a good foundational knowledge of your business, will help you succeed. Your data champions will act as change agents in your organization, helping to decipher the mystery of data science for their colleagues and act as an internal reference to similar projects.

5. Strike the right balance between automation and explainability

Data science techniques are time-consuming and complex. Fully automating the entire process will lead to black-box outcomes and a lack of trust, while full control over all steps in the process can only be achieved by a data science expert.

Depending on your use case, buying pre-trained AI models may suit your needs, but to scale data science across your organization, the solution you choose needs to automate complex processes while informing users about what is happening.

The right solution will guide users through the data science process, providing transparency about accuracy, reliability, and what factors were considered during model creation. In addition, users unfamiliar with data science need to understand why steps in the process may fail and how to rectify them. For example, more data may need to be added, or perhaps no significant pattern was detected. Solutions that offer prescriptive corrections will help users remove roadblocks and motivate them to try again. Over time, users will inherently adopt these best practices.

Although data science is based on mathematics, there is never a single right answer; there are many ways to approach a business problem or question. Data science generates additional insights to help business users make decisions with confidence. The key is to find the right solution that enables everyone -- regardless of their prior experience with such tools -- to learn through experimentation and continuously improve. Companies need to invest in creating a culture of trust between machine learning models and business users, and ultimately, confidence will come as a byproduct of consistently delivering information that the business deems valuable.