Quality is not one number. MediaMeter scores seven dimensions separately, weights them by content type, and shows the evidence behind each judgement.
The seven dimensions
Accuracy
Claims match the best available evidence at the time of assessment.
- Claim-level verification
- Correction history
- Precision of numbers and names
Evidence
Key assertions are traceable to named, checkable sources.
- Source proximity
- Document trail
- On-record attribution
Sourcing
The range of voices and documents is appropriate to the story.
- Source diversity
- Primary vs. secondary
- Conflict-of-interest disclosure
Fairness
Competing positions are represented without loaded framing.
- Right of reply
- Balanced characterization
- No selective quotation
Context
Readers get the background needed to interpret the claim correctly.
- Temporal context
- Base rates
- Known caveats
Clarity
Language is precise enough that reasonable readers reach the same meaning.
- Headline-body alignment
- Defined terms
- No false precision
Accountability
The work shows who is responsible and how to challenge it.
- Byline and dateline
- Corrections policy
- Contact route
Type-aware by design
A breaking-news brief and a 4,000-word investigation should not be judged by the same weights. MediaMeter specifies 54 content types — news, analysis, explainer, interview, review, obituary, live blog, newsletter, and more — each with its own dimension weights and minimum evidence thresholds.
Type detection is a suggestion, not a verdict. Editors can override the type, and the assessment is recalculated with the new weights. The override is logged.
Evidence for every score
Every dimension returns three things: a score, a confidence interval, and a set of evidence references. A reference can be a quoted sentence, a linked source, a named entity match, or a detection signal. If the system cannot find evidence, it says so and lowers confidence rather than inventing support.
A sample finding reads: “Evidence: 62/100 (confidence 0.71). Two of five key assertions cite unnamed sources; the budget figure is attributed to a document that is not linked.” This is a format example, not a real assessment.
Confidence and calibration
Confidence intervals are published alongside scores because point estimates imply precision the system does not have. Calibration is measured against a held-out set of human-rated articles, reported by dimension and content type, and re-run whenever a model changes.
Explainable, auditable, appealable
- Explainable: each score names the criteria that moved it.
- Auditable: model version, prompt version, and reviewer decisions are stored with the assessment.
- Appealable: publishers can contest a finding; appeals and outcomes are recorded in the editorial standards process.
Technical specifications
Model versions and evaluation protocol
Assessment uses a versioned ensemble. Each release ships with a model card, an evaluation report, and a changelog entry. Evaluation covers per-dimension agreement with trained human raters, calibration error, and performance by language and content type.
What the system does not do
It does not determine truth, intent, or legal liability. It does not score outlets or journalists. It does not publish, suppress, or rank content. It produces findings for editorial review.
Data handling
Content is processed for assessment only. It is not used to train third-party models. Retention is configurable; the default is 90 days for content and 7 years for audit metadata. EU-only residency is a planned option.