Data changelog

What is new

QOF does not measure every condition against everybody on a GP practice list. For nine conditions it divides the disease register by an eligible-age list: asthma 6+, rheumatoid arthritis 16+, diabetes 17+, CKD/depression/epilepsy/non-diabetic hyperglycaemia/obesity 18+, and osteoporosis 50+ (with obesity using 16+ in output year 2015). The earlier ADI rate deliberately divided those registers by all residents to express whole-population burden. The new rate uses the eligible resident population instead, making it comparable in definition with QOF's published prevalence.

Each affected condition now has an additional three-column group. For diabetes, for example:

DM_qof_afflicted       modelled eligible-age resident count
DM_qof_afflicted_pop   covered resident population aged 17+
DM_qof_afflicted_rate  DM_qof_afflicted / DM_qof_afflicted_pop

The original DM_afflicted, DM_afflicted_pop, and DM_afflicted_rate columns remain unchanged. The two triples are alternative representations of the same disease estimate and must not be added or averaged.

Exact eligible-age resident populations come from the single-year-of-age dimension of the same Nomis datasets already used for ADI population: NM_2010_1 for LSOA 2011 and NM_2014_1 for LSOA 2021. The pipeline composes 6+, 16+, 17+, 18+, and 50+ bands from published ages and applies the same geography conversion, publication-year population and 2025 carry-forward policy as the all-age data.

Availability

AST remains blank until QOF changed its denominator from all ages to 6+ in 2020-21; NDH begins when QOF introduced that register in the same year. These are intentional source-definition boundaries, not missing data. The new columns do not break or overwrite the original 2014--2025 series.

The two rates are not interchangeable

The original rate is a share of all residents. The new QOF rate is a share of the eligible-age resident population. For England osteoporosis in 2024-25:

The new value is 2.67 times the all-resident value and is within 0.26% of the NHS published rate (ratio 1.0026). All nine new England rates for 2024-25 are within approximately 0.3--2% of NHS published prevalence. This does not make the old value erroneous: it answers a different question. Analysts must choose the denominator that matches their interpretation and must label it.

Age-restricted is not age-standardised. The new denominator excludes residents below the eligibility cut-off; it does not reweight the eligible residents to a common age distribution. The new rate is not “adjusted for age” and does not support an inference that differences between places or years are independent of age structure.

Deprivation-gradient diagnostic

We independently recomputed Pearson correlations at LSOA level between QOF 2022-23 health rates (output year 2023) and the calendar-year 2023 Claimant Count rate across 33,749 LSOAs:

All nine associations moved in the same direction: more positive, or less negative, when the ineligible population was removed. The individual changes are modest, but their consistency shows that the all-age denominator attenuated the ADI health--deprivation association in this 2022-23 diagnostic. It does not show that the eligible-age rate is age-standardised or establish a causal relationship.

What council analysts should do

1. If an existing analysis uses only the original columns, no recalculation is required. Preserve its whole-population interpretation; the earlier 2014--2025 values have not changed.
2. Use the new *_qof_afflicted_rate when comparing with NHS/QOF prevalence or when the analytical question is prevalence among people eligible for that QOF register.
3. Use each new rate's adjacent *_qof_afflicted_pop denominator. It is an eligible-age population and is deliberately different from the row's all-age pop.
4. Rerun rankings, correlations and models if you switch definitions. Osteoporosis changes by a factor of 2.67 nationally, and local changes also reflect population age structure.
5. Do not splice across unavailable years. The eligible-age view starts in 2015 for seven conditions and 2021 for AST and NDH; 2014 blanks must not be imputed from the all-age rate.
6. Describe the measure as age-restricted, never age-standardised or age-adjusted.

What changed

Crime: replace every analysis of the former headline

The downloadable and site headline is now Police-recorded street crime (excludes ASB):

recorded_count       sum of 13 police-recorded crime categories
recorded_count_pop   population covered by those categories
recorded_count_rate  recorded_count / recorded_count_pop

Anti-social behaviour has its own count, coverage population and rate and must not be added to recorded_count when describing recorded crime. This is a definition correction, not a cosmetic rename. Previously published headline values cannot be interpreted as a recorded-crime series, because they combined the main police-recorded crime collection with ASB incidents governed under the National Standard for Incident Recording.

The input treatment also changed:

• British Transport Police is excluded entirely: its relevant exposure is rail passengers, not the resident population used by ADI.
• A territorial force-year is accepted only if all 12 monthly street files are present and non-empty and at least 90% of potentially English records have an English LSOA code.
• An identified incident repeated exactly in the source is deduplicated. Records without an incident ID, notably ASB, are not deduplicated merely because their anonymised fields match.
• When a force-year fails, its whole inferred LAD/LSOA footprint is blank. Region and England values remain available over the smaller metric-specific covered population; they may therefore cover a different footprint in adjacent years.

Current unavailable force footprints are Avon and Somerset (2016-2019 and 2025), Staffordshire (2018), Lancashire, Thames Valley and Suffolk (2019), Greater Manchester (2019-2025), and Gloucestershire (2020-2022). Exact LAD counts are:

These blanks supersede earlier guidance that singled out Devon & Cornwall and City of London in 2022; the revision-aware source refresh changed which force-years meet the explicit tests. Do not copy an old missingness mask into a new analysis.

Population and geographic conversion

LSOA 2011 source data is converted to LSOA 2021 before publication. Split-LSOA weights are now calculated from the same year's LSOA 2021 population that is published in pop; previously, a later population frame could split an earlier year's count. Rates and trends can therefore change in every year.

The LSOA-level source is available through mid-2024. ADI explicitly carries that estimate forward for 2025 and aggregates the same LSOA base to LAD, Region and England, rather than mixing in newer upper-geography estimates. Population fetching now verifies the year actually returned: an allowed one-year carry-forward is explicit, while a stale, mislabelled, or more-than-one-year substitute raises PopulationVintageError instead of silently entering the release.

Six complex many-to-many boundary-change LSOAs remain excluded. The release contains 33,749 of England's 33,755 LSOA 2021 areas, so pop is the ONS population summed over included LSOAs rather than necessarily the complete official total for an affected LAD.

Why counts and denominators behave differently by domain

Every ordinary metric is a three-column group:

<count>        metric count or modelled count
<count>_pop    population covered by that count
<count>_rate   <count> / <count>_pop

Employment and crime values are source counts. Introducing the target-year population cannot change how many claimants or incidents were counted, so their counts are retained and their rates are recomputed against the new denominator.

Health *_afflicted values are different: they are modelled as a QOF-weighted prevalence rate multiplied by ONS resident population. The prevalence rate is the estimated health quantity. When the population base changes, ADI therefore holds the rate fixed and re-derives *_afflicted. Keeping the old modelled count would invent a change in prevalence with no new health measurement. A revised health count can consequently reflect a population revision even when the estimated prevalence is unchanged, and it is not comparable to a raw QOF register count.

At higher geographies, a smaller <count>_pop means some LSOAs had no usable value. Sum counts and their matching coverage populations separately, then divide. Never replace a blank with zero or divide a partially covered count by full pop.

Health: coverage, assumptions and restored history

• QOF practice weights now renormalise over practices with a usable disease register and positive list size. A practice missing from a publication is no longer treated as having zero prevalence.
• Disease estimates below 80% disease-specific QOF registration coverage are withheld. The threshold is applied before short interior gaps are interpolated.
• qof_coverage reports the overall share of GP registrations at practices included in that year's QOF publication with a usable list. Disease-specific coverage can be lower.
• registration_coverage reports total GP registrations divided by ONS residents. It can exceed 1 because they are different administrative measures and is reported rather than thresholded.
• Two representativeness assumptions are unavoidable: a practice's overall QOF prevalence is applied uniformly to its patients in every LSOA it serves, and GP registrations attributed to an LSOA are assumed to represent all residents there. The coverage columns reveal how much evidence is represented but cannot verify either assumption.
• Leading and trailing gaps are left blank; only one- or two-year interior gaps bracketed by observations are interpolated.
• A disease register larger than its practice list is arithmetically impossible and is rejected. Estimated prevalence outside [0, 1] is likewise rejected to missing, never clamped.
• Eight implausible LSOA epilepsy values in 2016 and seven heart-failure values in 2021 are rejected at publication. Depression in output year 2024 and osteoporosis in 2015 have known one-year source-basis anomalies and are replaced from adjacent LSOA rates only where both anchors exist.
• CVDPP is published for output years 2014-2020 (QOF 2013-14 through 2019-20) and blank from 2021. Its England series has a sharp 2014→2015 break, and Dartford's 2019 value remains a known local anomaly; do not treat the seven-year window as automatically homogeneous.
• The one-year SMOK and THY source groups remain excluded. Together with restored CVDPP, the download and site contain 22 health metrics.

Health output year 2021 means QOF 2020-21 (April 2020 to March 2021). Employment and crime year 2021 mean calendar year 2021. A join on the numeric label therefore does not align identical periods.

Download bundles and site delivery

Four archives are published: adi-england.zip, adi-region.zip, adi-lad.zip, and adi-lsoa.zip. Each contains long-by-year employment, crime and health CSVs, a data dictionary, a geography file, and a README. Tables are keyed by code, name, and year, with 12 rows per area: 12 England rows, 108 Region rows, 3,552 LAD rows, or 404,988 LSOA rows.

The crime CSV now includes the derived recorded_count triple as well as all 13 constituent crime categories and the separate ASB series. The health CSV includes 22 metrics plus registration_coverage and qof_coverage. Counts retain sufficient decimal precision to reproduce the published eight-decimal rates.

Bundle metadata reports compressed on-disk sizes in decimal KB/MB and extracted sizes in binary KiB/MiB, so the number and unit agree. Known routes, the home, Explorer, Area, Compare, Trends, ADI-vs-IMD, Download and About pages, are prerendered as real route HTML. Crawlers and visitors without JavaScript therefore receive page content rather than only the application shell.

See METHODOLOGY.md and each archive's data dictionary for full definitions.

Intentional blank coverage

Trend-direction restatement

Method

The audit compared the 4,440 rate values in the 370 area-metric series common to both releases (10 England/Region geographies × 37 metrics × 12 years). CVDPP is documented separately because it was not present on the b152edf public surface. The baseline was reconstructed with b152edf aggregation and publication corrections. The current side uses the reprocessed store outputs plus the current publication-stage health corrections; its headline crime total is the corrected 13-category recorded-crime measure, while b152edf used the mixed 14-series headline.

A reversal means two finite adjacent-year differences have strictly opposite signs. A transition involving a blank or exact zero difference is not counted. This is a release-impact inventory, not a claim of statistical significance.

There are 194 reversed adjacent-year comparisons across 145 distinct area-metric series:

Of the 194 comparisons, 123 have an absolute old or current movement of at least 0.1 incidents per 1,000 for crime or 0.01 percentage points for employment/health. There are 14 reversals at the 2020→2021 boundary and 7 at 2024→2025. The 2025 cases are not caused by a new LSOA population estimate: 2025 deliberately repeats the mid-2024 base.

The largest changed directions are concentrated in the redefined crime headline:

Those headline differences combine a corrected definition, refreshed source archives, BTP exclusion and force-coverage rejection. Do not interpret them as the effect of any single processing change.

What analysts should rerun

1. Replace earlier extracts rather than appending 2025. Corrections affect the complete history.
2. Rebuild every crime headline. Use recorded_count/recorded_count_rate for recorded crime and analyse ASB separately. Do not compare a former 14-series “total” directly with the new 13-category series.
3. Rebuild crime time comparisons and area rankings with coverage checks. Force exclusions change both values and which resident population is represented. Do not impute blank force footprints as zero.
4. Update denominator code. Use each <count>_pop; sum count and coverage population separately before division. Do not average local rates.
5. Rerun every historical trend, rank, threshold, regression and chart. There are 194 changed England/Region adjacent-year directions, and local results can change even where an aggregate direction does not.
6. Treat health counts as modelled resident estimates. A population revision can change *_afflicted while prevalence remains fixed. Do not reconcile these values as if they were raw QOF register counts.
7. Use the new coverage indicators. Inspect registration_coverage and qof_coverage, especially for border, transient, or low-registration areas. Neither tests whether a practice's prevalence is uniform across its served LSOAs or whether represented registrations resemble all residents.
8. Restore or remove health metrics deliberately. CVDPP is available only for 2014-2020 and has known comparability concerns; SMOK and THY remain absent.
9. Align periods explicitly. Health uses QOF financial years labelled by ending year; employment and crime use calendar years.
10. Refresh parsers and metadata. Account for the recorded_count triple, 22 health metrics, two health coverage columns, dictionaries/geography files, and intentional blanks.

Validation and residual issues

The extended whole-series validator was rerun against the current store/outputs/default:

uv run python scripts/validate_outputs.py
SUMMARY: 24 BLOCKER, 104 WARN, 1 INFO

These are raw-pipeline findings, before publication-stage health corrections:

No new blocker class was found. This is not an unconditional quality guarantee: the Dartford CVDPP value and the five LAD health anomalies remain known concerns. The validator found no rate/count/coverage-population identity failure, incoherent partial triple, area-set loss, additivity failure or split-family inconsistency. A separate extremes review continues to flag the documented Forest of Dean 010C Claimant Count spike; no new release-blocking pattern was identified.

Complete reversal table

Positive values mean an increase and negative values a decrease. Crime changes are incidents per 1,000 people in the metric's covered population; employment and health changes are percentage points. Health intervals use QOF financial-year labels. The values are movements within each release, not the difference between releases in a single year.

Claimant Count (2)

Crime (102)

Health (90)