Product

Designing for editorial autonomy: why MediaMeter never auto-publishes

The product constraint that shapes everything: AI advises, humans decide. Here is what that means in practice.

Every AI product makes an implicit claim about who is in charge. A recommendation engine claims that relevance can be computed. A moderation system claims that harm can be classified. An assessment tool claims that quality can be measured. Those claims are not neutral, and the interface decides how much authority they are given.

MediaMeter’s central design constraint is simple: AI advises, humans decide. That constraint is easy to put on a marketing page and hard to honour in a product. This piece explains what it changes.

No auto-publishing, ever

MediaMeter does not publish, schedule, suppress, rank, or unpublish content. It has no write access to a customer’s CMS by default. The integration reads drafts and returns findings; the editor acts.

This is a deliberate limitation. An auto-publish feature would be easy to build and attractive to sell. It would also move the accountability for a wrong judgement from the newsroom to the tool, while leaving the legal and reputational consequences with the newsroom. That is a bad trade.

Review is a role, not a button

“Human in the loop” often means a person clicks approve on a queue. That is not oversight; it is a liability shield. Real review requires time, information, authority, and a record.

We specify four things for high-stakes findings:

  • Time: the reviewer can see how long a review took and flag unrealistic queues.
  • Information: the evidence, rejected sources, model version, and confidence are visible.
  • Authority: the reviewer can override without a second approval, and the override stands.
  • Record: the decision, reason, and reviewer identity are stored with the finding.

Disagreement is data

When a reviewer overrides the system, that is not noise to be suppressed. It is the most useful signal we have. Overrides are sampled, categorised, and used to recalibrate the model and the criteria.

A system that hides its disagreement rate is asking for trust it has not earned. We intend to publish override rates by dimension and content type.

No scores without evidence

A score is a compression of evidence. If the evidence is not available, the score should not be either. Every dimension returns references: a quoted sentence, a linked source, a detection signal. If the system cannot find support, it lowers confidence and says so.

This makes the product less impressive in a demo. A dashboard full of numbers looks more capable than a list of caveats. It is also less likely to mislead an editor under deadline.

No ranking of journalists

It is technically possible to rank reporters by average assessment score. We will not build it. Ranking individuals by a noisy, context-dependent metric invites gaming and punishes difficult reporting. Desk-level and format-level trends are useful; individual league tables are not.

No training on customer content

Customer content is processed to provide the service, not to train third-party models. Any use for internal evaluation requires explicit, revocable consent, and the resulting model card discloses it. This is a contractual commitment, not just a policy page.

The commercial tension

Autonomy is less profitable than automation. A tool that decides is easier to sell than a tool that advises. We think the market for trustworthy decision support is real, but we are not naive about the incentive.

The constraint also limits growth: every customer integration needs editorial buy-in, not just a procurement signature. That is slower. It is the right kind of slow.

What autonomy looks like in the interface

The default view is not a score. It is a short list of findings with evidence and a recommended action. The editor can accept, dismiss with a reason, or escalate. Nothing is pre-selected. The system does not nudge toward publication.

When confidence is low, the interface says so prominently. When the system is unsure, the correct experience is uncertainty, not a confident guess with a small disclaimer.

A test for any editorial AI

Ask: if the model is wrong, who finds out, how, and what happens? If the answer is “the reader, after publication,” the design has failed. If the answer is “a named editor, before publication, with the evidence in front of them,” the design has a chance.

That is the standard we are trying to meet. It is not a feature list. It is a constraint on what we are willing to build.

MediaMeter Editorial

Written by the MediaMeter editorial team. We publish our methods, limitations, and corrections.

https://mediameter.eu/resources/blog/designing-for-editorial-autonomy/

Related reading

Newsletter

Monthly notes on journalism quality.

One email a month. Methods, limitations, and what we are learning. No product spam.