Editor’s Note: Civil Beat’s participation in the State Integrity Investigation, a 50-state assessment of government transparency and anti-corruption efforts, was the subject of a recent post on Ian Lind’s blog. It has been reprinted here with his permission, along with Civil Beat’s response.

Civil Beat spent a lot of time and put a good deal of resources into the State Integrity Investigation done in cooperation with several national groups, including the Center for Public Integrity.

I was one of those consulted along the way about several areas of investigation, and I came away from the experience doubtful about the methodology (a standardized interview protocol with sometimes strange and extremely repetitive questions) and unsure about their ability to extract correct answers.

The results have been trickled out, and I’m just now trying to go back and check them out.

Some of the “findings” appear to me to be plainly wrong. Take the case of this category, “civil service management.”

Hawaii was given an overall grade of “D”.

You have to drill down through several levels of questions to see what the “scores” are based on.

For example, Civil Beat gave Hawaii a 50% score on this question: “Are there regulations for the state civil service encompassing, at least, the managerial and professional staff?”

Why only 50%? Well, it was based on answers to four separate questions. Hawaii received a 100% grade on two, and zero on the other two.

But one of these scores appears to be plainly incorrect.

Hawaii was given a zero rating for failing to meet this criteria:

171: In law, there are regulations to prevent nepotism (favorable treatment of family members), cronyism (favorable treatment of friends and colleagues), or patronage (favorable treatment of those who reward their superiors) within the civil service.

Civil Beat explained it this way:

The state law specifies equal opportunity for all under the civil service provision. It does not specifically use the terms “nepotism,” “cronyism” and “patronage,” but the equal opportunity principle may apply. See below.

Whether or not those specific terms are used, Hawaii clearly prohibits favorable treatment of family members, friends, and others.

Those provisions are prominent parts of Chapter 84, “Standards of Conduct,” and apply to all state employees and officials, whether covered by civil service or in exempt positions.

This is usually referred to as our ethics law, but it actually appears among various provisions regulating actions of all public officers and employees.

So it seems to me that Hawaii didn’t deserve a zero in this case.

And it’s just one example where the overall effort fell short.

Take another question: “Are the regulations addressing conflicts of interest for civil servants effective?”

Here Hawaii was giving failing grades of just 25% for regulation of post-government employment and regulation of gifts.

Here the scoring seems to be far too subjective. For example, the low rating on regulation of gifts appears to be based on reports of formal investigations into alleged improper gifts.

Reported gift violations triggered enforcement action. Sounds like the law was working.

So should enforcement actions like this be seen as positives that would boost our score or, as Civil Beat decided, as negatives that lowered our score?

CB went one way, I would have gone the other.

The examples go on, even in this one category.

Here are two related criteria.

First: “In law, citizens can access the asset disclosure records of senior state civil servants.”

Hawaii was given a 100% score, based on the statute requiring that directors and deputy directors, along with members of certain specific boards, must file public financial disclosures.

Fair enough.

Then here’s the following criteria: “In practice, citizens can access the asset disclosure records of senior state civil servants within a reasonable time period.”

Hawaii was give a zero in this category, even though those required public disclosures (the ones that earned the 100% score above) are available online and are posted relatively promptly.

Based on this sample, I have to take the findings of the whole project with a big grain of salt. Interesting, but not necessarily reliable. And drilling down through the layers of questions and subjective evaluations isn’t quick or simple.

And so it goes on this Saturday morning.

Civil Beat Assistant Editor Sara Lin Responds

Hi Ian,

Thanks for your questions about the Integrity Investigation. I’m glad we’re talking about the Accountability Project.

One thing I want to make clear is that this wasn’t “Civil Beat’s integrity investigation.” The State Integrity Investigation was a national effort that analyzed laws in all 50 states using standardized criteria. The reporting and the scores were subject to multiple layers of scrutiny. And as the person who was Civil Beat’s point person on the project, I can attest to the fact that there were back-and-forth discussions on some of our answers that resulted in scores being adjusted before the final report.

On scoring, you write that it seems to be subjective. I don’t think that’s a fair statement. Civil Beat’s conclusions went through three discrete layers of review, one of which was a blind peer review.

I’d like to point you to the State Integrity Investigation’s methodology page http://www.stateintegrity.org/methodology:

“Editors worked with the state reporters to ensure that data was sourced appropriately and scored against the established criteria. The data were then blindly reviewed by a peer reviewer for each state who was asked to flag indicators that appeared inaccurate, inconsistent, biased, or otherwise deserving of correction.”

The methodology page is also helpful in explaining how editors determined what questions to ask. We appreciated your participation in the project. Some of the questions may have felt repetitive to you, but they were thoughtfully determined.

Here again I’ll quote from the methodology page: “To identify the project’s Corruption Risk Indicators, staff from Global Integrity and the Center for Public Integrity contacted nearly 100 state-level organizations working in the areas of good government and public sector reform around the country. … The outcome was a list of questions, rooted in the reality of state government in the US, that these stakeholders identified as mattering most when it came to assessing the core risks of corruption in their states.”

You raise questions about our findings regarding the Civil Service Management category. Specifically, you question the 50 percent score for regulations “for the state civil service encompassing, at least, the managerial and professional staff.”

Part of that grade was based on whether Hawaii has an anti-nepotism law. We say Hawaii does not and gave the state a zero grade. You disagree and say Hawaii “clearly prohibits favorable treatment of family members, friends, and others.”

If you’re correct, then why did the State Ethics Commission attempt to pass a standalone anti-nepotism law in 2011? It would have been Hawaii’s first.

Here’s the article we wrote in March 2011 about the bill’s demise: http://www.civilbeat.org/articles/2011/03/22/9804-hawaii-anti-nepotism-bill-facing-long-odds-in-house/

As we wrote then: “Hawaii is among a minority of states without a specific anti-nepotism law, according to the National Conference of State Legislatures. About half of all states specifically ban nepotism either in their constitutions or via statute.”

The proposed bill came in response to claims that some state workers were abusing their positions. In December 2010, reports surfaced that Principal Diana Oshiro of the Myron B. Thompson Academy gave three of the top four administrative positions at the public charter school to family members.

You may think we have clear laws, but Ethics Commission Executive Director Les Kondo told Civil Beat at the time that the original draft of SB 994 was narrowly tailored to provide lawmakers with clear boundaries when it comes to hiring or appointing family members to state jobs.

So clearly the state’s top ethics official doesn’t agree with you.

You then challenge the score on the question “Are the regulations addressing conflicts of interest for civil servants effective?” You say the score is “subjective.”

There is no regulation of post-government employment. Rep. Karl Rhoads, the committee chairman in charge of public employment issues, admitted as much himself. He said it is “almost impossible” to enforce.

You are much more sanguine with gift enforcement. We know for a fact there is no auditing of gift reports and that the ethics commission relies on complaints to enforce the law.

You may have gone the other way on the score. The point of this exercise is to get people talking and thinking about public transparency issues. But I don’t see how you can justify your view that the law is working.

Next, you raise a question about whether Hawaii law gives citizens access to the asset disclosure records of senior civil servants.

We wrote that the score depended on what your definition of senior civil servant is.

You raise a good question about why Hawaii gets a 100% in the “in law” category, but a zero in practice.

The law is good.

But in practice, we stand by our zero. Here’s why: the Ethics Commission receives 1,800 financial disclosure filings each year from state employees. Just 160 of those are made public.

You seem to think that asset disclosure records are “available online and are posted relatively promptly.”

But that’s not true.

Take the Cabinet-level appointees of Gov. Neil Abercrombie.

More than two months after his administration had taken office, Michael Levine wrote in February 2011: “A review of the commission’s website shows that disclosure filings for other Cabinet-level appointees — Chief of Staff Amy Asselbaye, Attorney General David Louie, Commerce and Consumer Affairs Director Kealii Lopez, Hawaiian Home Lands Director Alapaki Nahale-a, Public Safety Director Jodie Maesaka-Hirata and Transportation Director Glenn Okimoto, just to name a few — aren’t yet available for public review. They may well have been submitted, but not posted online.”

We discovered this as we were trying to verify the income of William Aila, Abercrombie’s controversial pick to head the Department of Land and Natural Resources.

Questions had been raised about potential conflicts of interest regarding Aila’s income from fishing. When we looked for his financial disclosure, it wasn’t online. The first excuse was that the Ethics Commission had it on hand, but they were too busy to post it online.

Eventually the commission discovered it had never received it and only got it from him after Civil Beat asked for it — more than a month after it was due. (It turned out that he doesn’t make much money from his fishing operations.)

Here’s the full article: http://www.civilbeat.org/posts/2011/02/17/9027-case-study-in-hawaii-ethics-enforcement/

We appreciate your critical reading of our work. However, is it really fair to reach a sweeping conclusion about the project based on the answers to just a few questions in a survey that involved 330 questions? Especially when there are flaws in your analysis of the few questions you did look at.

Sara Lin
Assistant Editor of Civil Beat