SAB, DDI and SDMX

In the latest SAB newsletter (October 2011) Arofan Gregory has an item about the “business case” for the use of both DDI and SDMX.

Do you think that his text contains a business case?

Do you think that it contains a solution to the DDI/SDMX integration issue?

Of course not. That would be doing serious work. Attempts in that direction would also reveal that Mr Gregory does not have a clue about what he is doing.

Instead, the only thing we learn from the article is that NSOs may need both DDI and SDMX, because they have been developed for different and only partly overlapping purposes. To cap we are informed that there is such a thing as the DDI/SDMX dialogue (but not whay they have (not) achieved).

This is the wagging-the-dog syndromed all over again. There is DDI, which grew slowly and obliquely into the NSO metadata sphere. Then there is SDMX, which was a one off consultant-cowboy initiative that nobody really understood.

So, you take parts from two different bodies and then try to put them together in a new body. Good luck with that! In fact, to continue the metaphor, they do not even know what a real body looks like nor what parts of such a body that are needed and what parts DDI and SDMX cover.

Here is a thought! What if the solution really is one model, which neither is like DDI nor SDMX. Then what? (I know this to be the case, because I know what the solution is).

Then there are organisational issues, discussed by Martin Vocsan in the editorial.

Sure, the organizational structures need focusing. There is currently more organizational bodies “working” with these issues than there are high level objects in the GSIM model. (Here is the SAB page that shows an overview of the current organizationl structure). However, such a focus is hardly possible before there is a focus for content issues. What exactly are all these groups trying to do, and in what order?

The simple truth is that they just do not know.

Advertisements

Progressing considerably?

SDMX now has an “action plan”. Wow! I love the word “action”!

According to this plan, SDMX has been “progressing considerably” during the last ten years. Great!

I then attempt to read what the progress has consisted of, but I all I find are things that only make bureaucrats happy.  The bulk of the “progress” listed in this “action plan” consists of organizational developments, infra structure and official recognition. Not real results!

Also, out of four headings, there is only one that really discusses “progress”. That is the heading “implementation”:

“Transition from GESMES/TS to SDMX in 2003”

Sure! But this tells us very little about real implementation. It may just refer to a formal decission. In any case, just replacing GESMES is of little interest, in the bigger picture.

“SDMX web services adopted by a growing number of statistical organisations”

Really, and what does that mean, in plain English? “Adopt”? “Web services”? “Growing” number? Does the latter mean that there are now two NSOs instead of one? Wow, that would be a 100% increase! But, is it considerable progress?

“continuous increase of statistical domains creating and using SDMX compliant data structure and metadata structure definitions”

A “continuous increase”. That sounds impressive! So, one became two and two then became three? How many are there now? Three instead of one? How many NSOs use each domain?

I suppose that these questions are a bit too impertinent. This is bureaucracy-land. In this land there is a statute that regulates clarity. Clarity is forbidden most of the time, because it can be embarassing to civil servants.

Well, I am not entirely fair! This is what they also had to say:

“The SDMX Global Conference in May 2011 confirmed that SDMX is broadly used by statistical organisations along the statistical business processes. Many statistical organisations around the world consider SDMX as useful or even very useful”.

Great! They quote themselves. That is another good way of avoiding uncomfortable truths. (The survey presented at the 2011 conference has allready been reviewed in detail on this blog).

There is also one or two words about implementation in ISOs – not NSOs!

Do you know what I think?

If the SDMX consortium have hard core statistics to prove their supposed “progress” in relation to NSOs they would be pushing them ad nauseam. We would here about 30% this and 50% this, etc.

The very fact that they are not doing this is what suggests the true nature of ten year of SDMX “progress”.

By the way, the “action plan” is directed towards international agencies, and has little or nothing to do with NSOs. It would seem that these international organisations are going ahead, even if they have a lack of support from NSOs.

There is nothing like prestige to fuel irrational courses of action…

A result of this is that a future failure for SDMX will be all the more costly.

It is the NSO business case, stupid!

Caught in the middle?

In an earlier post I reviewed Trevor Fletcher’s presentation of the 2011 global SDMX survey.

There is reason to return to that presentation. It was not clear who the respondents were. Were they stary-eyed and self-interested “metadata” experts?

Regarding possible obstacles to SDMX, I noted that lack of subject matter support, lack of DSDs and MSDs, and lack of top management support all had high and more or less even ratings.

Let us translate this to plain english:

Nodobody, except the self-interested metadata experts, is enthusiastic about SDMX, and these experts have not been able to make SDMX work properly!

Bo Sundgren about SDMX

Here is an interesting comment by Bo Sundgren about SDMX. For improved public access to NSO data and metadata he wants to explore:

“relevant existing and/or emerging standards, such as SDMX, http://www.sdmx.org, developed by a consortium of seven international organisations”

“however”, he continues:

“these standards have not been developed with citizens and entrepreneurs in focus, and they have to be further developed for these purposes, especially as regards simplicity and richness of user-oriented metadata”

Really?

DSDs and MSDs before SDMX?

It is a great idea. To replace Excel spreasheets containing data – and assorted documents containing metadata – with one standard XML-based technical and content format for exchange.

In current SDMX terminology this means agreeing on a DSD and an MSD. In current SDMX strategy papers the key to success for SDMX is also described as more DSDs.

Now, I wonder, what existed before SDMX? How did OECD, Eurostat, etc, know which data and metadata they wanted? And how did they manage to unify the information that they received?

It seems to me that there must have been standards already in place, to which the various contributions were aligned. If not, why not? If so, why are they now having problems agreeing on DSDs and MSDs?

Do you get my drift?

There is something terribly wrong here. The ISOs have been struggling to get SDMX moving, and a constant road block seems to have been a lack of DSDs. Now it is supposedly a key to success.

But, if there had been research done before the SDMX project started, then one would have known to what extent it was possible to at all align statistics between NSOs.

Maybe this is simply not possible, in very many cases, and ISOs have mostly been ad libing based on whatever the NSIs cared to send to them?

I do not know, but I smell something terribly wrong here. Is this why there are not more DSDs, because there is no realistic base for this type of unification of national statistical data?

I would like to see a special task force that investigates the basis for a DSD and MSD-driven international standarisation process.

This leaves us with an additional observation. The Dutch do not even seem to believe in SDMX for data exchange, but they do believe that an SDMX registry for standards, including DSDs, would be a killer app.

Well, could not the Dutch then please propose a large number of new possible DSDs?

Trevor Fletcher´s sleight-of-hand

SDMX and Trevor Fletcher have published results from their 2011 global survey.

One finding is that respondents have shifted from seeing SDMX as very useful to just useful. SDMX believes that this is normal, and due to an initial IT-hype. This amounts to a recognition that there is little rationality in the way that IT-systems initially are perceived and evaluated. What will happen in a years time? Will there be a shift from useful to not so useful?

Fletcher believes that the perception of SDMX will bounce back. But, there is no reason to rely on statistics in the individual case. “Enlightenment” may well lead to a rejection of SDMX.

Those who believe that SDMX is extremely or very useful are now a minority, 48%. In reality this means 27% of the total number, since only 58% responded. In other words, a clear minority. We should then note that there is only one negative option in the way this question was asked!

Only 35% use SDMX now. This amounts to 24% of the total, a definite minority. However, it is not clear exactly what is meant by using SDMX. Does this include trials? Why have they not instead asked if they have implemented SDMX and in what way?

In any case, those that use it or plan to use it are an seemingly overwhelming 85% (of  58%), But, this also amounts to a minority of some 46%.

It would seem that the statistical community is divided and that the SDMX enthusiasts are a minority.

The main challenges graph is difficult to interpret. That one challenge is slightly higher rated does not tell us anything about the make or break nature of the remaining challenges. Fletcher suggests that a lack of resources as a main challenge is a result of the financial crisis. This could explain the response, but if there is a business case for SDMX, this should not matter. There should be a positive bottom line. This said, it is interesting to note the even distribution between several challenges. Most interesting is the combination of lack of subject matter support, lack of DSDs and MSDs, and lack of top management support. These challenges are both highly and more or less equally ranked. This would suggest that the cards are stacked against SDMX.

However, the main problem with this question is of course the sample. Who have they asked? The usual suspects? Central methods and IT-staff, that all have a vested interest and no practical or economic responsabilities? What if they had asked top management about challenges? Or subject matter statisticians directly? That would probably result in a completely different set of answers to what the challenges really consist of. Is it bold to guess: a lack of a clear business case, an overly complex solution, and no clear benefit but possible disadvantages to operations?

Trevor Fletcher´s claim that “Challenges to implementation are mainly due to lack of resources (human & financial) – particularly after period of international financial crisis and budgetary constraint”, is in any case not supported by the survey results. That is what happens when there is – intentionally – poor survey design and no qualitative analysis.

The solution, according to SDMX? More marketing! Have we heard that before? Then a global DSD registry. Sure, but you have been working on that for five years now. What is stopping you?

When we evaluate this survey, we should remember that the main function of SDMX as a replacement for GESMES has assured it a relatively broad base from the start, especially among central banks. The fact that many NSIs use or plan to use SDMX does not really tell us anything about SDMX beyond the fact that it replaces GESMES.

Well, it gets better. Allmost half of the respondents are central banks, rather than NSOs! So, what is the NSO situation? For all we know, it could be the central banks that are propping up the statistics.

Why has Trevor Fletcher not published separate NSO statistics?

So, let us be the devil´s advocate! What do these results really say? Here is the worst case scenario. Only some 10-15% of NSOs are enthusiastic about SDMX, and they have only used SDMX for trials, so far. The result of these trials may very well be the same as for Statistics Netherlands. There is no business case, and this is reflected in subject matter resistance and top management scepticism.

For all we know, with all the money spent, SDMX will be little more than a replacement for GESMES, with central banks as the main user community.

P.S. If it is not too impertinent to ask: When will the SDMX global survey be published using SDMX technology? D.S.

Macedonia is also honest about SDMX

In the contributions to the 2011 CES plenary session, we find a report from Macedonia. Like the Norweigians they are not entirely convinced about its merits for NSOs internally:

“Although SDMX implementation is considered as strategic, advantages are not immediately obvious”

“In NSOs, domain experts are not yet convinced of the benefits of SDMX and consider it an additional burden”

However, we also find those who are more upbeat. The Swiss, for example, claim tremendous benefits. We also remember Mexico.

How is it that different NSOs can reach such different conclusions?

I think one has to take in account the self-interests of those who manage these projects and write the reports. In short, reports of these type are not always honest.

However, the answer could also be that there is a lot of abstract thought and speculation going on. They are optimistic because they lack experience of metadata projects. Here is a really good and healthy attitude, from Macedonia:

“To make the complexity of implementation manageable, future capacity building activities should be focused more on gaining knowledge for the implementation strategy in order to have a clear view of the larger SDMX picture and steps should be defined how to gradually build the environment. This approach is supposed to downsize the complexity of the issue, but also to justify used resources, to see if benefits from an efficiency point of view and re-usability of metadata are really achieved”

Note the passage “are really achieved”. They realise that they do not know this yet. That is maturity and responsibility for you.

And, how about the political processes?

“There is an urgent need for the international organizations to better harmonize their own requirements for Data Structure Definitions (DSDs)”

Gee, I would like to know why that is not really working out? How many years is it?

Here is my brief analysis:

1. Standardisation takes time, and may not always be rational.

2. NSOs first need to standardize their internal processes, so the metadata is already there in the right format.

3. To do this, they need much better models and systems. The push towards DDI and SDMX is just because they are there, not because they are good.

These are the reasons that things will not work out this time around, either. We are back where we started. We first need to solve basic NSO metadata modelling issues in a truly standardised way.

I know how to do it. Does anybody else?