Tag Archives: ISO 11179

Enterprise Data Management – Where to Start?

3 Jul

What is Enterprise Data Management? In some organizations, this is an easy question to answer. However, in others – especially those with an analytical mission – it is much harder. Often the function is put under the Enterprise Architecture team.  One often hears that “the Enterprise data folks just do not get it”. As one executive in a large financial organization put it: “EA is where rubber hits the air”. So how do we define a role for the data function within an Enterprise Architecture  team?

This post is not about how to organize effectively in order to align with the business units to show business impact – although a worthy topic. This post is about suggesting a role for the enterprise data management team when that team is organized under the CIO within the Enterprise Architecture  function of the organization. In order for the data function to be deemed valuable to the business stakeholders it must be understandable, actionable and  tied to the business objectives.

Data is everywhere. When we talk about “Enterprise Data Management” the temptation is for managers to say – well that means we manage all data in all locations. As enterprise data managers, we must know all about everything! Really? Have you ever seen this work? This leads to the top down mandate of the “canonical” approach where the objective is a single standard, a single canonical model – a single ring to rule them all! This rarely works well (if at all). Business requirements, analytical activity, market trends, and evolutions in technology all lead to a core business requirement for flexibility. Additionally, there is a fundamental need to recognize that the “ground truth” is almost always with the business side of the house – and “truth” is often a shifting concept in the real world. This is part of the reason why the Enterprise Data Warehouse (EDW) “single version of the truth” is problematic for analytical and BI staff and for the rise of Hadoop as a more flexible environment. As an analytical or BI person, my version of the truth – or the right data – depends on the context of a particular decision.

So how do we focus the EDM team on what makes sense? The graphic in this excellent article on risk architecture caught my eye. I have modified it a bit to identify some core activities that I see as foundational for organizations seeking to mature their data management in general, and specifically, the integration of the enterprise architecture team in the data management process. The original graphic is attributed to Naomi Clarke currently at Credit Suisse.

EDM

Based on the above, the role of EDM is simple; manage data assets to expose those attributes that are needed to answer key business questions about data assets:

  • What are they?
  • Where are they?
  • What has happened to them?
  • What are they related to?

To do this, one needs a data management “hub”. I call it a Hub as this provides flexibility for discussion purposes. Some would call it a Managed Metadata Environment (MME), others perhaps a Metadata Registry. Regardless, the goal is the metadata ecosystem that can support key functions related to governance, curation, quality, usability and discoverability. This view suggests the following regarding the roles of the EDM team – especially when it is organized within the EA Team:

  1. The Team needs to only manage three inputs: lineage metadata; definitions, and the physical location (what, where and change). The way the organization creates those three inputs is part of an overall data strategy, but not something the EDM team drives – these are driven by the business. By focusing in this way, the Enterprise team leaves it up to the business or operational components to determine the optimal approaches.
  2. Definitions are aligned to business terms and to “Concept Systems” (as defined in the ISO 11179 Specs). This enables discoverability and complex search approaches based on an understanding of semantic equivalence.
  3. Data Assets can be classified within the context of an enterprise  data reference model (DRM). In most organizations, this supports the governance process. However, in government organizations, the DRM is also used to align policy and strategy objectives to IT activities. See Federal Enterprise Architecture framework for how this works in the US.
  4. Capabilities to support governance functions must be provided: vocabulary management tools with the capability to curate and link ontologies, taxonomies, controlled vocabularies, etc, data quality tools, and governance tools.

If one can limit the INITIAL scope of the EDM team to these items, it is much easier to tie enterprise activities to the business needs, and provide a set of capabilities that address challenges of high value to the organization: search, discoverability and integration. Evolving the role once these benefits are established is a much easier task.

Tags: , , , , , , , ,

A good baseline presentation on creating semantic data repositories

8 Jul

I have been meaning to give a shout out to Dan MCCreary http://www.danmccreary.com/ for some time now, and am only just getting around to it. What caught my attention was a presentation Dan gave at the 2006 Semantic Technology Conference. The presentation walked the audience through the process of creating a well-structured data repository enabling players within the Minnesota education system to ask complex across all of the data available. While the presentation is rather dated, it is one of the better presentations I have seen that takes things back to a level most people can understand. Additionally, I don’t see that the state of the industry has changed that much in terms of tools to support the creation well-structured data repositories that support broad “knowledge management” goals.

Dan focuses on metadata as the means to achieve the goals of the participants: Minnesota Department of Education; the Wisconsin Department of Public Instruction and the Michigan Department of Education. The goal is to create a semantic understanding of the data available. Semantic is one of those words that means many things to different people. Once you combine “semantic” with “knowledge management” or ”knowledge discovery”, everyone seems to have a different opinion as to what you are talking about – IT and business folks alike. It becomes very important to level set your audience. For the purposes of this discussion, creating a semantic repository involves:

  • Enhancing data with metadata that focuses on describing the data asset as a whole, and with respect to its component content parts;
  • Creating descriptive and content metadata that describes the data in specific terms, and in more general “conceptual” terms.
  • Creating a description of the data that reveals connections to other concepts; related terms; multiple standards.
  • Creating highly linked data that is organized through taxonomies and ontologies to reveal links that are perhaps not obvious to users.

The desired end state is the ability for a broad range of end users to query disparate data sources and understand how available date can be used in their particular context.

Within the referenced deck (http://www.danmccreary.com/presentations/semweb2006/), there are a number of sections worth drawing your attention to:

ISO 11179 (Starting on page 15). Most people I speak to that know about ISO 11179 (http://www.iso.org/iso/home/search.htm?qt=11179&published=on&active_tab=standards&sort_by=rel ) are somewhat skeptical as the spec is a heavy on as written. Once you get your head around all the good ideas that have gone onto the specification, the question becomes how much of it do you adopt? The idea behind the spec is that each piece of metadata has its own metadata that allows a user to know exactly how that metadata is to be used. Dan presents the 11179 concept within his discussion of the National Information Exchange Model (NIEMS https://www.niem.gov/Pages/default.aspx) which brings us to another interesting point…

Mapping to Standards (Page 16). I like the way that the process of mapping to standards is presented. In this instance, the Educational System is only trying to map to one standard. However, you can imagine an environment where there are multiple standards of interest. In this instance, the mapping approach is organized around the OWL “SameAs” terminology (Web Ontology Language http://www.w3.org/TR/owl-features/). It does not say it in the presentation, but I am assuming that this is taken from the SKOS Simple Knowledge Organization System standards (http://www.w3.org/2004/02/skos/). As you classify data assets, there will be multiple mapping exercises where the mapping will not be exact. The linking concepts defined within SKOS will allow you to relate items that are not precise or consistent in their relationships.

Demonstrating that the data problem grows faster than the number of data sources (Page 17). This sounds so simple. However, I am still surprised when I hear people say – “we have the data mapped between the sources I want, so no need to spend time mapping to standards”. Well ok that works if you have a small number of static data sets, but … that happens rarely. In the presentation Dan refers to this as the O(N2) problem versus the O(N) problem. I have included his graphics as these tell the story better than words.

network Problem

Tags: , , , , ,