HomeLandscapeAbout me

Data Governance From an Engineering Perspective

By Valdas Maksimavicius
Published in Data Governance
August 06, 2020
2 min read
Data Governance From an Engineering Perspective

Data governance is a combination of physical systems, data models and business processes. Learn the core concepts by exploring available libraries and tools.

This article is the first in the wider series about Data Governance and Metadata. In them, I write about what I’ve learned on data platforms, how I think of it, and how I use that knowledge. I plan to release other posts in the future.


Why should you read it?

1. No marketing fluff

The majority of freely available content about the data governance is vendor specific. Understandably, Informatica, Collibra, Alation, and other vendors, seek to create more demand and praise their features.

As a result, IMHO, many metadata management and data governance aspects are exaggerated and made more complex that it should be.

2. Focus on practical aspects

Secondly, research and advisory companies, like Gartner or McKinsey, publish many data governance articles too.

The issue there - too high level and disconnect from the technology capabilities.

In these series, I want to focus primarily on explaining the concepts by using a specific library or a tool.

3. Don’t be boring

Topics like privacy, governance, or security are very formal and get boring fast. I hope you don’t mind a funny picture or meme.  

The evolution of Big Data
The evolution of Big Data

How to approach a wide topic like data governance?

I use the below Venn diagram as a starting point. I used Willem Koenders’ LinkedIn post as an inspiration.

Data Governance as an intersection of physical systems, data models and business processes
Data Governance as an intersection of physical systems, data models and business processes

The physical systems cover data engineering and technical aspects. 

The pysical systems in data governance
The pysical systems in data governance

The data models describe conceptual and data modelling techniques to bring the most value to the business. 

Data models in data governance
Data models in data governance

Last but not least, the business processes & compliance focus on an organization, its processes, privacy and industry regulations.

Business processes in data governance
Business processes in data governance

Introduction to data governance for engineers

Here’s my video about the topic recorded at Big Data Conference 2020


Table of contents:

  1. Introduction
    1. Data Governance From an Engineering Perspective (this post)
    2. The Alter Ego of Data
    3. Tools in the Data Management Zoo
    4. With Data Comes Responsibility
  2. Physical system
    1. Databases and storages
    2. Integration & synchronization
    3. Security
    4. Infrastructure
    5. Automation & orchestration
    6. Consumption
    7. DevOps
  3. Data models
    1. Data modelling techniques
    2. Data catalog
    3. Business glossary
    4. Data exploration
    5. Master data management
    6. Data quality
    7. Visualizations
    8. ML models
    9. Compliance
    10. Roles and responsibilities
    11. Data quality
  4. Business processes & Compliance
    1. Business processes
    2. Industry specific regulations
    3. Budget, ROI
    4. Privacy regulations
    5. Business and technology architecture
    6. Roadmap
    7. Data governance models
    8. Organization structure

Tags

#data governance#data catalog#business glossary

The latest set of Azure Data Platform best practices - April 2021

A simple blog post evolved to 25+ page guide with 75+ different recommendations.
Download Document
Previous Article
With Data Comes Responsibility
Valdas Maksimavicius

Valdas Maksimavicius

IT Architect | Microsoft Data Platform MVP

Resources

ADVERTISE WITH US

Topics

Data Architecture
Data Governance
Miscellaneous
Data Engineering

Related Posts

Apache Ranger Evaluation for Cloud Migration and Adoption Readiness
May 24, 2021
15 min
© 2021, All Rights Reserved.

Quick Links

About mePrivacyContactLandscape

Social Media