Here is a more detailed look at Dan Gillman´s “20 years of progress” paper from the 2010 UNECE/METIS meeting.
1. “However, a desire to build much larger systems met with new challenges, and in 1999 the idea that building “cathedrals” should not be an initial goal – introduced by Jostein Ryssevik – was an important outcome. The notion of building a system up step by step is an old strategy, but it required repeating”
An important outcome? The projects that followed in the beginning of 2000 attempted exactly that. Metanet and COSMOS are two examples, with ambitions for common or reference models. IPIS is another. In fact, Dan Gillman´s CMR was one of the earliest attempts at building cathedrals and he has remained devoted to that approach all this time. In other words, this statement has nothing to do with historical truth. It is an example of rewriting history to fit the political needs of the present. It has been inserted to justify later failures and current confusions.
2. “Many offices also offer good metadata to go along with them. Now, the challenges are around services, software designed to work on the Web to find, manipulate, and integrate data from different sources. The change is toward more standardization and building automatic means for finding and manipulating data…”
This is glib, indeed. NSOs are far from offering good metadata. The current focus on exchange and standards has no connection to the existence of metadata in NSOs. To the contrary, it is the failure of several international and national metadata projects that have created the buzz surrounding projects such as SDMX. There is also no real standardisation going on. In fact, the opposite has happened. The focus on developing truly standard models for NSO systems has been lost. And the focus on “active” metadata is not a step in a general development, it is a result of the failed implementation of passive metadata.
3. “But, this requires significant upgrades to data and metadata systems, such as using standards for interoperability and designing “computable” terminology systems (ontologies)” [the sematic web]
Here is another example of self-promotion. Gillman attempts to further his own ideas from ISO/IEC 11179 by linking them to the requirements for the semantic web. His technique is to insert his own terms from 11179 (interoperability and terminology). I know very little about the semantic web. But the same thing was said about XML, and nothing has happened there yet. So, why should the semantic web be different? The truth of the matter is, again, that the core problems of metadata remain unsolved. NSOs are still crying out for truly standard models that are implemented successfully because they support NSO users in their daily work. Without a successful capture of metadata, there will not be any success, regardless of the latest technical fad.
4. “By the year 2000, new standards efforts such as the DDI started focusing on the survey life-cycle. In the last few years, Part C of the CMF, and the Generic Statistical Business Process Model in particular, is a much stronger focus for METIS and the Steering Group. Part A of the CMF also reflects the need for a strong internal focus.”
This is an example of what the Soviets called “the death of history”. DDI is named, because of its present interest, but other projects are striken from the historical record, such as SCBDOK, Metanet, IPIS, earlier UN standards, the Latvian system, etc.
5. Then we have the greatest self-promotion of all. “Terminology” is cast as a theme of its own. Why is not models a theme of its own? Instead, “formalism” is cast as a separate theme. This shows how Gillman is condescending in his view of real standard models, and how he views his own ideas about “terminology” as more important than real solutions to modelling problems. He is afraid of discussing real issues regarding real models.
“The importance of terminology was recognized early as one of the main outputs of METIS. Early work focused on the development of a terminology for metadata systems, edited by Dusan Prazenka then Daniel Gillman, (published by the UNECE secretariat in 1999) and models for handling classification systems (a form of a terminological system), which evolved into the Neuchâtel Classification model (published in 2000 and available in the CMF web site through http://www.unece.org/stats)”
In this quote we find all the cardinal sins at the same time. Gillman promotes his own early work, then attempts, with slippery language, to tie this work to the Neuchatel model for classifications. CLASET is not mentioned. Then he refers to classifications as a form of terminology, to make his own ideas about terminology seem more general and valid than they are, and, finally, he includes a link, in order to promote Neuchatel.
6. “In general, terminology management consists of defining and relating the concepts underlying terms that are used in an office and gathering together all the terms used for one concept. There are a wide variety of possible uses and applications of this technique, and statistical offices have long recognized the importance of this. Classifications, thesauri, ontologies, nomenclatures, glossaries, and dictionaries are all examples; and most if not all of these kinds of structures are in use in statistical offices”
Here is Gillman´s defintion of terminology work. Note that he again attempts to include classifications. He has to do this, in order to be able to claim that the importance of terminology work has been evident to NSOs and that such systems are widespread. But, classification databases have very little to do with ISO/IEC 11179 type models, and a central registry does in fact not need to collect all terms for a concepts. This is one of the mayor fallacies of 11179. If you have a central repository, why bother about local names?
To further these false claims about terminology in NSOs he includes a discussion of multi-lingual requirements. But this also has little to do with terminology management as handled by 11179. Of course, several offices are multi-lingual, but in terms of metadata systems, this is primarily a technical aspect of system design.
7. “In the 2000s, the same trend continued (MetaNet, Neuchâtel variables, ISO/IEC 11179, SDMX, DDI, and others). ISO/IEC 11179 and DDI were developed as standards, and SDMX became one. SDMX and DDI were both based on XML and complied with ISO/IEC 11179”
Here is just another example of the selective mentioning of standards. Why is not the Latvian system mentioned?
In this section, Gillman also turns to outright polemics. His claim is that there is something called “mathematical” formalism, and that this is no longer interesting, and not very practical. This is a false description of reality. Gillman´s purpose with this false description is to discredit the claim that theroetical insights are fundamental to the solution of ISOS modelling problems. In other words, it is a polemic directed towards the challenge from this blog and key insights.
Here is a quote:
“The CMF Part B (Concepts, Standards, and Models) and Part C (Statistical Life-Cycle and Generic Statistical Business Process Model) are further efforts to advance the work in this area. Again, the formality of this work is less than the mathematical precision required by some, but it has a practical value we will visit in the next section”
None of these “efforts” have advanced anything in this area.
8. “The real focus of METIS is to help statistical offices design, build, and use statistical metadata systems. Even though theory, formalism, concepts, standards, and models are part of the discussion, they are not the final product of the work”
Here is some honesty. But compare this with the choice of themes in the paper. Why is the paper not organised around systems and models as core themes? As opposed to “terminology” and “formalism”? As we have seen, this is because the two latter serve Gillman´s self-promotional activities and sly polemics. A focus on the two former would, on the other hand, reveal difficult truths.
Then the bare knuckles are shown. Gillman has nothing to say about neither impementation nor systems. This section, titled “implementation”, instead contains propaganda for the UNECE/METIS framework, especially part D and part A. But part A is not a METIS history of implementation and systems. And part D contains more general case studies, that are not very specific about NSO systems.
Why does Gillman have almost nothing to say about NSO systems, in this section?
“Over time, the complexity of the implementations in statistical offices has increased. Whereas, as we discussed in section IV-B, early implementations were more focused on data sets or single surveys, the corporate approach is much more common today. This migration over time is evidence that the lesson about not building cathedrals was learned. A slow approach is much more effective. In fact, it is evidence of the idea we are trying to convey in paragraph 43”
Once again he is arguing his own false point about developing cathedrals. What has happened is that NSOs generally have failed to develop the systems that they need, so they end up with very simple or overly complex systems, instead of real solutions. These failures are glossed over as a leason learned about strategy. Nowhere in this is the real lesson mentioned, the one about addressing the big picture and then developing truly standard core models. In fact, the term “core model” is not mentioned at all. Nore that the framework was supposed to be a description of system requirements.
9. Here is the grand finale:
“With the new effort to incorporate ideas from the Semantic Web, corporate implementations will get more complicated. Given the careful approach taken so far by statistical offices in developing their statistical metadata systems, even if these efforts don’t succeed, the overall systems will not fail. Seeing that this is the case, we conclude that METIS has been a great success so far. Let us ensure the next 20 years be as successful”
Ten dollars to anyone who understands what is being said here! Efforts may not succeed but systems will not fail? Which systems? What are their relation to this particular effort? Why should systems become more complex? Why is this supposed complexity not a problem, if the careful approach is used?
None of this is given a clear answer in Gillman´s paper.
So, what is this paper really all about?
My guess is that ISO/IEC 11179 has become a target of increasing criticisms due to its overly complex model. What Gillman is trying to do is argue (1) that this complexity will be needed in the future and (2) that it can be handled if metadata systems are developed step by step.
Maybe an important source of these criticisms has been my observation that Dan Gillman´s flagship “terminology” model ISO/IEC 11179 has been a resounding failure in the private sector? Maybe one or two NSOs are now also regretting the day they decided to implement 11179?