Modernizing Data Management with Karen Lopez
Karen Lopez, data evangelist with InfoAdvisors, discusses modernizing data management practices, including the biggest challenges organizations face and the expanding role of data governance.
- By Upside Staff
- March 15, 2024
In this “Speaking of Data” podcast, Karen Lopez explored the ins and outs of modernizing an organization’s data management practices, including the challenges faced and the expanding role of data governance. Lopez is a data evangelist and senior project manager with consulting firm InfoAdvisors and will be teaching at the upcoming TDWI Modern Data Leader’s Summit. [Editor’s note: Speaker quotations have been edited for length and clarity.]
“One thing I’ve found working in the data industry is that there’s always something new coming over the horizon,” Lopez began. “Even so, we can still find ourselves suffering from the same struggles I was working on 35 years ago.”
However, she pointed out, although relational databases were the core of everything until about 10 years ago, at that time there was an explosion of other types of databases and data stores -- a fact that makes the addition of the word “modern” much more meaningful than it otherwise might have been. “There are just so many more opportunities for new approaches to data management now,” she added.
“I’m usually more of a skeptic when I see ‘modern’ in front of anything,” Lopez said. “There are certain standards, principles, and practices that work even in this new environment. It usually takes someone with a lot of hard-won experience to be able to tell whether one of these new systems or tools is trustworthy. Some of these things may be really exciting, but they just don’t catch on. For example, maybe they’re not scalable or they don’t meet the cost-benefit test -- there are plenty of reasons.”
Lopez noted that, unlike many other professions, the data management industry often allows vendors to set the standards and practitioners just react. As a result, one of the key challenges data managers face is combatting vendor lock-in while still ensuring that the various platforms and tools they use will work well together, she explained.
“If you’re using five different products, how can you share your data among them -- especially classified or sensitive data -- without excessive risk? How are you going to get these tools to help you do the right thing?” Lopez asked. “That’s been my challenge. Plus, now vendors are adding AI to absolutely everything, which is like throwing 1,500 more people on a project and trying to figure out how they’re going to work together.”
She explained that the most mature organizations -- the ones that have best solved these problems -- are the ones that have good accountability and governance practices as part of their data management processes. They don’t have that many “cowboy developers,” as Lopez called them -- independent, siloed workers who acquire a one-off solution and use it for their own projects without regard to the organization’s overall standards and practices or existing tools.
“I was on a project recently where everyone was doing their own data inventories using their own classifications and were making their own decisions about whether a document contained sensitive information,” she said. “There was no one person who was responsible for rationalizing those things and bringing everyone to a common understanding.”
Simplifying Tasks
Some recent developments may hold the promise of simplifying these tasks, Lopez said, such as data meshes, data fabrics, and data vaults.
“Of course, they’re not the same things, but they can really be used to complement each other,” she explained. “For example, the data mesh focuses more on the people and processes problems, where the data fabric focuses more on managing the various types and locations of data that may be present.”
This brought the conversation to the subject of data governance.
“It’s interesting that now we have people selling data governance tools and vendors are building governance features into their basic databases,” Lopez said. “It seems that data center and infrastructure vendors actually care about the quality and safety of their clients’ data, so that’s really exciting. It really sets the stage for us to accomplish those bigger projects that will provide extra value to the organization.”
Lopez was also excited by the news of generative AI tools that discover, document, classify, and summarize the existing data and projects in an organization, as well as the natural language querying capabilities that allow users to direct their questions at specialized data sets.
“Where many organizations are facing challenges as they try to modernize is with the people who were experts in the old way of doing things and who aren’t interested in learning something new,” she added. “I live on the leading edge of things where I get to play with all the new stuff and see where it will be useful, but some people just aren’t there. Sometimes that skepticism can carry over into the rest of the project.” In the end, she said, every great technological innovation still requires a combination of old and new skills because there will be different use cases, different workloads, and different performance needs.
For more information, see the Modern Data Leader’s Summit at TDWI.org.