The Integrity of Official Statistics – What’s Needed

National Research Council of the National Academies, Principles and Practices for a Federal Statistical Agency, Source: http://bit.ly/2jsKL4l

Americans routinely turn to numbers for evaluating the state of the economy and social conditions. In particular, we rely on the accuracy and integrity of official statistics. While U.S. official statistics are not perfect, they are widely considered to be impartial measures that are prepared in a professional and objective manner. The accuracy of the U.S. data contrasts with the data that’s been disseminated by some other countries, such as Greece and Argentina, where it was learned that they had been doctored by politicians.

With each new U.S. administration—but especially for a Trump administration that eschews prior government service—political appointees with little experience with official statistics are appointed to positions with responsibility for overseeing those statistics. The National Research Council of the National Academies publishes a great guide on how a federal statistical agency should work, Principles and Practices for a Federal Statistical Agency. To maintain the integrity of official statistics, political leaders need to understand and commit to support these principles.

The guide identifies four fundamental principles for producing high-quality statistics:

  1. Relevance to policy issues – Statistical agencies should design their statistical products to meet the needs of a broad spectrum of users.
  2. Credibility among data users – Agencies should disseminate their data on an equal basis to all, be open about their data sources and their limitations, and provide full documentation of their methodologies.
  3. Trust among data providers – Data providers, such as survey respondents, need to be able to trust that the statistical agency will honor its pledge to protect the confidentiality of the data and that it will not be used non-statistical purposes, such as law enforcement or regulation.
  4. Independence from political and other undue external influence – Agencies must avoid any appearance that their data compilation and dissemination processes could be manipulated for political purposes.

The guide also discusses 13 critical practices:

  1. a clearly defined and well-accepted mission,

  2. necessary authority to protect independence,

  3. continual development of more useful data,

  4. openness about sources and limitations of the data provided,

  5. wide dissemination of data,

  6. cooperation with data users,

  7. respect for the privacy and autonomy of data providers,

  8. protection of the confidentiality of data providers’ information,

  9. commitment to quality and professional standards of practice,

  10. an active research program,

  11. professional advancement of staff,

  12. a strong internal and external evaluation program, and

  13. coordination and cooperation with other statistical agencies.

In subsequent posts I plan to delve more deeply into two of these practices, protection of the confidentiality of the data, and protection of independence.

From my own experience of 32 years in the federal statistical system, these principles have been adhered to under administrations from both parties. From 1997 to 2016 I oversaw BEA’s GDP and national accounts statistics and never experienced any political interference in the estimation or dissemination of those statistics. I don’t think the fact that official statistics are compiled with this degree of independence is as well known as it should be; for example, I’ve talked to corporate accountants who find it surprising. The federal government has strong norms of statistical independence and the statistical agencies rely on OMB statistical policy directives that restrict pre-release access to statistics.

I remember many years ago talking to a very partisan reporter after our corporate profits estimates were revised down following a change in administration. He wanted me to say that the Under Secretary from the previous administration had asked us to boost the estimates prior to the election, but it simply wasn’t true. The actual reason for the large initial overestimate was that BEA’s early profits estimates were based in part on accounting data from companies like Enron that were cooking their books. The reporter had to change the punchline of his story, so it instead ended with a paragraph accusing us of incompetence. While the article stung, I was proud that his search for political interference had come up empty.

6 Comments

  1. Richard Penny

    Not a comment, but more a request. I would be interested in your view on the trade-off between timeliness and accuracy as many users of official statistics tend to neglect this.

    Reply
    • Brent Moulton

      Richard,

      Thanks. That’s a great suggestion – something I’ve given a lot of thought to over the years. I’ll see if I can work up a post on that question sometime in the next week or two.

      Reply
  2. Tom G

    Thanks for highlighting the very important limitation of statistics collections — if the sources of data are cooking the books, all statistics including that data will be wrong.

    Reply
    • Brent Moulton

      It’s hard to say until the details of the budget come out. For example, right now I don’t think we know how much of the proposed budget cuts apply to the statistical agencies in general, or how much is proposed for each agency.

      However, the total magnitude of the cuts appear to be huge. The numbers cited in this report suggest the administration is proposing roughly a 10% cut to non-defense discretionary spending in fiscal year 2018. The statistical agencies are, of course, part of non-defense discretionary spending, so they most likely will be included in those proposed cuts. Most of the stastistical agencies’ budgets go directly to data collection, compilation, and dissemination, so their budget cuts would undoubtedly mean fewer data collections and less data availability.

      Reply

Cancel reply Cancel reply

Your email address will not be published.