6 Characteristics of Machine Learning Organizational Maturity
Whenever you think of business intelligence, think machine learning. Here's what an enterprise with a mature ML environment looks like.
- By William McKnight
- April 28, 2020
The distribution of companies across maturity levels in most anything is always highly skewed toward the low end. It is no different for machine learning, but this shouldn't matter to any organization aspiring to sustained success. The key to that success is to become a mature machine learning operator (MMLO).
What does such an enterprise look like? What characteristics does it exhibit that indicate its sophisticated use of machine learning is part of its business success?
1. Data scientists are valued.
In terms of strategy, the MMLO has already justified the use of (and hired) a data scientist. It has the data environment ready so that data scientist can be effective. After seeing the benefits produced, the enterprise has gone further and hired additional data scientists.
When a new data scientist is brought into the organization, the documentation and defined business goals (that adhere to reasonable, established constructs in the data environment) enable these new scientists to come up to speed in weeks, not quarters.
2. ML is a normal part of every project.
In the MMLO, the project specification process has a checkpoint to make sure that ML is appropriately considered for major projects. People with ML knowledge and leadership skills will be part of every project or every architectural review. It would be easy to maintain the status quo, use long-familiar working processes, and continue to use only legacy approaches, but the mature ML organization knows it must specifically include ML today.
3. Models and data are actively managed.
The MMLO catalogs all models along their life cycle so that models can be reused and leveraged rather than being "one and done." The data environment is mature as well, with enterprise data cataloged, accessible, performing according to expectations, and well-managed. This means all enterprise data -- and relevant external data -- is captured and utilized. It means there are data warehouse and data lake infrastructures and a data catalog over the top capturing the location of information. It means the commitment to the cloud is real and the data governance program is pervasive across major subject areas throughout the enterprise.
In these companies, data is recognized as a discipline. These organizations employ a chief data officer, and data is not an afterthought or secondary to applications.
4. It takes ML configuration and transparency seriously.
Mistakes in ML configuration can be costly, leading to wasted efforts and wasted computing resources. Mistakes can create production issues. In a mature ML shop, manual errors are uncommon, as are omissions and oversights in the model that lead to waste. ML systems in these shops are transparent, with special focus on those cases that have the potential to result in loss, harm, or damage to the company.
Their models are predictable and consistent, with auditable and reproducible outcomes. The MMLO understands it is important to be able to rerun experiments and get similar results. Unused and redundant settings are detectable in the environments of the mature ML operator.
5. Testing and model maintenance are needed for well-functioning systems.
Mature ML operators take their processes to a significantly higher level. Models have access restrictions. Code naturally should be tested, but it's clear to the mature ML operator that some amount of data testing is also critical to a well-functioning system. Mature ML operators perform data testing that monitors changes in the distribution of the data.
The mature environment enables current objectives to reuse existing models. Rather than start from scratch, the MMLO can add features to distinguish a new model. ML processes include the use of a repository for models and robust model packaging, deployment, serving, and monitoring.
6. Ethics is not an afterthought.
A MMLO's AI programs have incorporated ethical frameworks and ensure that ethics and security are paramount. For example, though they may not have perfected it yet, these programs have taken steps to remove the potential for the malicious use of ML, which could include cyber attacks, deployment of physically destructive force, deep privacy invasion, or the application of undue influence.
A Final Word
No two shops will have the exact same ML journey. All begin at different points and take a circuitous route to maturity. However, for most enterprises, the level of ML described here, the mature machine learning operator, will be required in the near future for company sustainability. Steps should be taken now to advance machine learning maturity in your organization.
About the Author
McKnight Consulting Group is led by William McKnight. He serves as strategist, lead enterprise information architect, and program manager for sites worldwide utilizing the disciplines of data warehousing, master data management, business intelligence, and big data. Many of his clients have gone public with their success stories. McKnight has published hundreds of articles and white papers and given hundreds of international keynotes and public seminars. His teams’ implementations from both IT and consultant positions have won awards for best practices. William is a former IT VP of a Fortune 50 company and a former engineer of DB2 at IBM, and holds an MBA. He is author of the book Information Management: Strategies for Gaining a Competitive Advantage with Data.