TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Why Digital Transformation Will Become More Critical in 2021

These three data-focused trends will help enterprises advance from data to knowledge to insight.

• By Kendall Clark
• January 12, 2021

Organizations of all sizes have demonstrated unprecedented resilience in light of COVID-19 and have placed an even greater focus and investment on digital transformation. Despite this renewed attention, fundamental data challenges remain a primary obstacle. In 2021, even as we expect the pandemic to come under control, organizations can expect to see further intensification and specialization of the following three technology trends.

For Further Reading: Leveraging Data Virtualization for Digital Transformation Five Database Requirements for Digital Transformation How Process Mining Can Aid Your Digital Transformation

Trend #1: Data finally becomes "machine-understandable"

The reality of digital transformation is that most data-driven efforts are doomed to fail, primarily because data is not machine-understandable. Unlike humans, machines cannot handle irregularity, variance, or strange distributions by default. Human decision making is based on contextual intelligence and powerful mechanisms of meaning construal, and to successfully automate to higher and higher degrees of fidelity, machines need to know more of what we know.

In 2021, organizations will adopt modern data integration approaches such as an enterprise knowledge graph (EKG) that fuse data management and knowledge management techniques. These systems will allow enterprises to discover hidden facts and relationships through inference mechanisms that would otherwise be unable to catch on at large scale without the accelerating power of machines. EKGs make knowledge not just machine-readable but machine-understandable, and they do this by capturing and encoding real-world context from disconnected data sources on a specific topic, person, project, or process.

Moving from data to knowledge and achieving higher degrees of automation requires insight into, and leverage over, the relationships between things in the world. As a result, older ways of managing data that rely on rigid, tabular, and storage-based techniques will increasingly be seen as inadequate in 2021 and beyond.

Trend #2: Semantic graphs fuel the new data integration landscape

Relational systems were never designed to support connection- and relationship-rich data landscapes with rapidly changing requirements and meaning. In fact, data integration as a whole is an artifact of where data management was 20 years ago, which is to say that relational systems were never intended to manage large-scale, radically heterogeneous information systems.

To manage data algorithmically, you must first represent what is actually meaningful about the data in ways that are accessible to algorithmic design. In one of the grander rhetorical ironies of IT, the relational data model just isn't very good at representing complex, real-world relationships and connections.

To do that realistically, the rules of the game have to be changed. Semantic graph data models -- one of the fastest-growing new technologies -- is the most natural way to represent data that is natively stored in other structures and will become increasingly important in the new year. In 2021, organizations will look to semantic graphs to connect data from structured, semistructured, and unstructured sources to gain a full picture of connected enterprise data and to understand the relationships and nuances that exist.

Trend #3: Query-answering drives the next-generation data fabric

Data fabrics are heralded for their ability to weave together existing data management systems, with the result being applications enriched by greater accessibility to the data that matters. Data fabric systems will continue to emerge in 2021 and will earn their spot as the next step forward in the maturation of the data management space.

Most of us will recall how data lakes once held the promise of centralizing an enterprise's data assets. Although they have achieved that goal to some degree, it's not the crucial goal that the enterprise needed because physical colocation is not the same as data connectedness. If it were, none of us would be forced to spend nearly as much time as we do searching our laptops for data that we cannot find. An enterprise-grade file system is necessary but is not itself sufficient to transform an enterprise digitally.

Data warehouses, which are an even older technology than data lakes, of course, are even less capable than data lakes because they really only deal well with structured data to begin with, leaving semi-structured and unstructured data silos completely disconnected. Even a data warehouse in the cloud is limited to the capacity of the relational model to represent data despite the business triumph achieved. Data catalogs have since emerged to provide an inventory of the bewildering diversity of their data landscapes only to be faced with the next great challenge: how to get business answers from a catalog of data sources.

In 2021, companies that have lived through these eras of data management will make data usable and reusable at enterprise scale. By applying powerful query-answering services in data fabric systems that connect all the disparate parts of the fragmented data landscape, they will finally be able to answer questions, derive insights, and make data truly actionable.

Closing Thoughts

We can only wake up to new possibilities if the old ways really change. The 2020 worldwide pandemic may be the first truly global event in all of human history to force everyone at once to think differently given its reach and impact. Realistically, the shocks of COVID-19 will be felt for years to come, forcing enterprise software and data management systems, and organizations as a whole for that matter, to adapt and innovate.