Governed Data Discovery: Blending Self-Service and Standards
What happens when business units cut their tether to the IT department? In the short run, business users are ecstatic, but without governance strategy, mismanaged self-service can undermine data consistency.
- By Wayne Eckerson
- July 28, 2016
Self-service is all the rage these days. Business users no longer want to wait for corporate business intelligence (BI) teams to develop reports and dashboards or build out a data warehouse before they gain access to data. Today, there are a plethora of self-service BI and data preparation tools that business units can purchase and implement without assistance from the corporate information technology (IT) department.
Unintended Downsides of Self Service
What happens when business units and their users cut their tether to the IT department? In the short run, business users are ecstatic: they get the information they want, when they want it, and how they want it. However, without an overarching governance strategy, the business units gradually produce spreadmarts and reporting shadow systems that undermine the data consistency required to align an organization.
In the worst case -- which is almost every client I work with -- self-service BI creates an environment in which no one trusts the data that anyone else produces. Because everyone creates reports using their own data and calculates metrics uniquely, there is widespread distrust of the data. Self-service creates a proverbial Tower of Babel where everyone is empowered with data but no one can communicate.
Moreover, many report shadow systems are brittle, pulled together with information duct tape and bailing wire and held together by one or two people upon whom the business unit is completely dependent. If they get run over by a bus -- or leave the company -- their reporting system quickly falls apart. Many of these renegade systems eventually hit the wall as they grow in popularity because they weren't designed for scalability, availability, reliability, or security, and they can't handle complex data scenarios.
At this point, many business units come crawling back to IT and request that it fix their reporting environment or manage it permanently. A vindictive IT team might take this opportunity to say "I told you so" and re-impose the draconian controls and plodding processes that caused business units to revolt in the first place. Hopefully, instead everyone has learned a useful lesson: business and IT must work cooperatively together to deliver both standards and self-service, governance and agility, control and speed. This is easy to say but hard to do.
Merging Top-Down and Bottom-Up BI
BI is fueled by two opposing forces: top-down and bottom-up BI.
In top-down BI, the corporate IT group imposes standards on the delivery of data and reports to ensure a single version of truth. Its output answers predefined questions and "runs" the business. Conversely, in bottom-up BI, business unit analysts create their own reports with custom data sets that they acquire from a variety of systems, including the data warehouse. Their output answers new and frequently unanticipated questions and often "changes" the business.
In most organizations, these two camps are at war. I spend most of my consulting hours brokering truces between them.
The top-down camp emphasizes centralization, economies of scale, governance, standards, and control to ensure the delivery of accurate, consistent data throughout the organization. In contrast, bottom-up BI focuses on speed, flexibility, and freedom so business units can meet local needs quickly.
From 2000 to 2010, the BI field was dominated by top-down approaches and tooling because it enabled businesses to deliver a single version of truth and align the business around a common set of metrics for understanding and measuring business activity. Starting in 2010, the balance of power shifted to the bottom-up camp with the rise of big data, advanced analytics, and self-service everything. Finally, business units had the tools and techniques to harvest their own data, and they turned their back on corporate IT.
Today the pendulum is swinging back again. Business units and corporate executives are seeing the limits of unbridled self-service and local information autonomy. The mantra today is "governed self-service" -- that is, let's provide business units with the ability to service their own information needs but within a governed framework managed centrally by the corporate BI team.
How to Have Your Cake and Eat It Too!
The techniques for blending governance and self-service are outlined in a new report from Eckerson Group titled "Governed Data Discovery: Balancing Flexibility and Standards." (Editor's note: registration required to access linked report.) To succeed with BI, the report says that organizations need to develop a comprehensive strategy that unites the two worlds of BI across three architectural vectors: organizational, informational, and technological.
Organizational. The ideal organizational architecture federates a BI team across corporate and business unit staff, leveraging matrixed lines of reporting and various cross-functional governance committees to knit the two worlds together.
Informational. The proper data and analytics architecture aligns people and processes with appropriate data structures and tools and carves out sandboxes for business analysts to access, merge, and analyze data without creating data silos that undermine data consistency.
Technological. Today's BI tools have morphed into analytical platforms that support self-service data exploration and report creation within a governed environment that ensures data standards.
Yin and Yang
Our industry is quickly discovering that self-service is not the panacea it once seemed. Unless an organization wraps self-service within a governed framework, it will propagate data chaos.
Rather than flip-flopping between top-down and bottom-up approaches to information delivery, an organization needs to balance these opposing impulses and blend freedom and control, flexibility and standards. In short, they need to create a unified organizational and technical environment that reconciles top-down and bottom-up BI.
About the Author
Wayne Eckerson is an internationally recognized thought leader in the business intelligence and analytics field. He is a sought-after consultant and noted speaker who thinks critically, writes clearly, and presents persuasively about complex topics.
Eckerson has shared his insights and advice with a wide range of companies. Recent clients include AAA, Air Canada, Carlsberg, Boston Beer, New Balance, Avalon Bay Communities, and TE Connectivity. He has also conducted many groundbreaking research studies, chaired numerous conferences, and written two widely read books: The Secrets of Analytical Leaders: Insights from Information Insiders and Performance Dashboards: Measuring, Monitoring, and Managing Your Business.
Eckerson is founder and principal consultant of Eckerson Group, LLC, a business-technology consulting firm that helps business leaders use data and technology to drive better insights and actions. His team of senior researchers and consultants provide cutting-edge information and advice on business intelligence, analytics, performance management, data governance, data warehousing, and big data. They work closely with organizations that want to assess their current capabilities and develop a strategy for turning data into insights and action. He can be reached at wayne@eckerson.com.