Reach is not impact, and the gap is where credibility goes
Eight measurement levels, and why reporting at the wrong one is the most common way a civic programme loses its funder.
Civiq.plus research · Impact measurement
A programme runs sixty community meetings, reaches eleven thousand people, and reports improved civic satisfaction. Somewhere between the second and third clause, a reach number became an opinion claim, and nobody in the chain decided to make that leap. It happened because the reporting format had one row.
The levels are different kinds of statement
Civiq.plus separates measurement into eight levels, and the separation is enforced in how results are stored rather than left to the writer:
- Operational — tasks completed, events executed, issues resolved
- Reach — unique reach, community delivery, attendance, exposure
- Attention — watch time, completion, replay, retention
- Engagement quality — substantive comments, meaningful shares, survey participation
- Narrative — topic association, velocity, risk, question frequency
- Opinion — mood and perception movement, with confidence
- Service outcome — issue resolution and recurrence
- Attribution — an experimental or quasi-experimental estimate
The first four are things you did. The last four are things that changed. Almost every over-claim in civic reporting is a quiet promotion from the first group to the second.
Attribution has entry requirements
The platform will not label something an attributed effect without a comparison method. An impact study records its intervention, pre and post windows, baseline metrics, the control or comparison method, known confounders, the estimated effect with a confidence interval, a confidence grade and whether an analyst has reviewed it.
If those fields are not populated, the result is available as a before-and-after observation, which is a legitimate and useful thing. It is simply not the same thing, and it does not get to borrow the stronger word.
What this does to a board meeting
It makes them slower and considerably more useful. The conversation shifts from whether the programme worked to which level we can actually speak at, and that question has an answer that everyone in the room can check.
