We tested the flag. Here is everything it did and did not do.

In plain terms: about one in eight flagged homes lost the rating inside 4 years, against about one in eleven of the clean ones. That is the whole of it. It is a real difference and it is a difference in odds, not a forecast about any particular home. Working a flagged list, most of your calls will still be to homes that were never going to be downgraded. There will just be fewer of them than if you worked the register in alphabetical order.

The outcome being counted is the commercial one. Of the 106 falls in the flagged arm, 98 went to Requires improvement and 8 to Inadequate. Not one is an Outstanding home slipping quietly to Good, and that is structural rather than lucky. The flagged arm contains 0, 0, 0 Outstanding homes across the three windows, because CQC does not rate a home Outstanding overall while marking its Well-led below Good. Every fall this flag can ever produce is a Good home losing Good.

The control arm is not built that way. It holds 638 Outstanding homes in the 4-year window and 109 of its 996 falls are Outstanding homes dropping to Good, which is not the outcome a consultant sells against. Count only falls that land in Requires improvement or Inadequate and the risk ratio goes up rather than down, to 1.61 [1.34-1.95]. The headline uses the smaller number.

The sample. Every row in CQC's 01 July 2022 Latest ratings.ods file marked as a care home, taking the whole-location rating rather than a per-service breakdown, in the 9 English regions, carrying an overall rating that can be ranked. That is 14,583 homes out of the 14,603 in the file. The 20 excluded are 16 with no overall row and 4 carrying an overall value that is not one of the four rating words. Those counts are printed in the results table for each window so the filter can be checked rather than believed.

The arms. Flagged is Good or Outstanding overall with Well-led at Requires improvement or Inadequate, which is the FirstFlag test exactly as the method page states it. Clean is Good or Outstanding overall with Well-led at Good or Outstanding, which is to say Well-led was not marked down. Every Good or Outstanding home in the baseline file is in exactly one of those two arms and 0 are in neither. A third arm, used further down, is Good overall with Well-led Good or Outstanding and exactly one of the other four key questions at Requires improvement.

A fall. Ratings are ordered Outstanding, Good, Requires improvement, Inadequate. A home fell if its overall rating in the 1 July 2026 file sits lower in that order than it did at baseline. A re-publication at the same level is not a fall. A re-worded judgement is not a fall. The single assessment framework introduced values that are not ratings at all, and those are counted in their own column and never as a fall.

What was fixed in advance, and what cannot be shown. Three baselines, 1 July 2022, 3 July 2023, 1 July 2024, all three reported below. They are July anniversaries of the one endpoint file and they were set in the extract step before the join ran. That is a design description, not a registration. There is no timestamped artefact on this site or in the repository that proves the windows were chosen before the results were seen, so this page does not claim there is one. An earlier version did.

What can be checked instead: no window was dropped, and no comparison was run and left out. The results table counts every test in it, including the ones that go against us, and the count is at the foot of that file. The 4-year window carries the headline because it is the only one long enough for a home's next inspection to have plausibly happened. Take that as a stated reason and read the other two rows yourself.

Both of these would produce the gap above without any home actually being worse run. Both were tested before the result was written up.

A fall needs two separate things to happen. CQC has to come back, and the rating has to drop when it does. Splitting the age-adjusted effect across those two channels says which one is carrying it, and the answer is not the same at two years as it is at four.

Risk ratios, flagged against clean, age-adjusted, split on the log scale.

Read plainly, that says over 4 years the flag is mostly telling you the home is genuinely more likely to be marked down. Over the shortest window it is mostly telling you CQC is coming back to this home sooner. Both are useful to a consultant and they are not the same product, so we are not going to describe one as the other.

Percentages are the unconditional probability of a fall. Intervals are 95%. The last column is the whole flagged arm against control B, one estimator in all three rows.

The three windows share most of their homes. Overlap between the flagged arms, measured on CQC location ids, runs from 0.51 to 0.78 by Jaccard: 634 homes shared between Jul 2022 and Jul 2023, 555 homes shared between Jul 2022 and Jul 2024, 727 homes shared between Jul 2023 and Jul 2024. So these are three views of one cohort, not three independent replications. They must never be pooled, and three intervals excluding 1 is not three times the evidence of one. They are printed together because a single window would be a choice, and the choice would be ours.

Shared membership understates it. What the three rows really share is the falls themselves. Added up across the windows the flagged arm produced 241 falls, and those come from 130 distinct homes: 62 of the 76 falls in the Jul 2023 window are homes that also fell in the Jul 2022 one, 38 of the 59 falls in the Jul 2024 window are homes that also fell in the Jul 2022 one, 49 of the 59 falls in the Jul 2024 window are homes that also fell in the Jul 2023 one. Read the table as one set of events counted up to three times, not as a result that replicated.

The last column is the flag measured against homes whose single below-Good key question is one of the other four. It is the comparison that decides whether Well-led is special. Both arms are the full arms, so the numerator is the same flagged arm as every other column here; the version that also strips the flagged arm down to homes carrying exactly one below-Good key question is in the results table, and it does not change the story. The direction reverses across the windows, which is the first entry in the section below.

One more thing before those entries, because it changes how every interval on this page should be read. The results table runs 70 distinct risk ratios with a significance value, plus 27 age-adjusted estimates reported as intervals. No correction for multiple comparisons is applied to any single line, so a Bonferroni threshold across them is 0.05 over 70, or 7.14e-4. The headline sits at 1.5e-4 and clears it, along with 8 other comparisons, most of which are that same contrast restated or stress-tested and are therefore not extra evidence for it. The two shorter windows do not clear it. Neither does any single domain comparison below, nor the flagged arm against control B in any window. Those are directions worth a purpose-built study, not findings to sell, and this page will not present them as more than that.

The comparison above pools four key questions into one control arm, which is convenient and which hides something. Below is the same cohort unpooled: Good overall, exactly one key question at Requires improvement, none Inadequate, split by which key question it is. The number of marks is held at one, so the only thing varying is which one.

Unconditional probability of a fall by the 1 July 2026 file, with the numerator and denominator of every cell.

Caring is the highest of the five in all three windows, and it is not close in the shortest one. Against homes whose single mark is Well-led, the risk ratios for Caring are 1.84 [0.94-3.59] in the Jul 2022 window, 2.34 [1.12-4.92] in the Jul 2023 window, 3.40 [1.52-7.60] in the Jul 2024 window. 2 of those three intervals exclude 1.

Now the caveats, which are real and which are not a way of taking it back. The Caring cells are 29, 28, 21 homes. Not one of those risk ratios clears the multiplicity threshold set out above. Almost no home in England carries a Caring mark on its own, which is why the cells are that size, and a rate computed on 29 homes moves a long way if two of them go the other direction.

It is printed at full size anyway. Further down this page there is a small-n null that happens to suit us, the 9 homes carrying two marks, and it is used to decline a claim we would otherwise like to make. A page that prints the small cell when the small cell is convenient and pools the small cell away when it is not is doing the opposite of what this page claims to do. So: on this evidence the sharpest single key question is Caring, at 24.1% against Well-led at 13.1% over 4 years, and it is not the one we filter on.

Density is only half of what a list has to answer. The other half is reach: not how concentrated the list is in future falls, but how many of them it contains at all. The pool for the 1 July 2024 window is 11,365 English care homes rated Good or Outstanding overall, of which 584 went on to fall.

Our list is 832 homes and holds 10.1% of the falls in the pool. Every Good-rated home with any key question below Good is 1,741 homes and holds 22.6%, at a slightly better lift as well. The middle row beat us too: homes whose one below-Good key question is not Well-led are more numerous, hold more of the falls than we do, and carry the highest lift in the table. If the number you care about is how many of the sector's coming downgrades you can get in front of, the wider filter wins in this window, and it is free.

Start from what the backtest actually establishes, which is narrower than either the old pitch or the objection above. A home rated Good or Outstanding overall that already carries a below-Good key question is more likely to be downgraded than one that carries none. That is the finding. It does not identify which key question, and on this evidence it is not Well-led in particular.

Probability of a fall by the number of key questions below Good at baseline, Good and Outstanding homes only. The last column is a headcount rather than a rate: too few homes carry two below-Good key questions for a percentage to mean anything.

So the risk is real whichever key question is carrying the mark, and choosing Well-led is therefore not a claim about prediction. It is a claim about what a consultant can sell and then deliver, which is a different question and one the backtest was never going to answer.

Well-led is the domain a governance engagement moves. Audit trails, records, statutory notifications, who is accountable for what, how long the manager has been in post, whether anybody read the last action plan back and closed it out. That is scoped work with a document trail at the end of it. It can start in a week, it can finish inside one engagement, and the evidence of it is written down where the next inspector will look.

A Safe mark is usually staffing levels or medicines competency. Fixing it means recruiting in a market that has no staff in it, or retraining, or spending money on the building. A Caring mark is culture, which is the slowest thing in a home to shift and the hardest to evidence when it has shifted. Both are real work. Neither is a proposal a consultant can scope on a first call and close in a fortnight.

That cuts both ways and the Caring table above is where it bites. Caring is the sharpest single mark in this cohort and it is also the one a consultant can do least about inside one engagement, so the domain that predicts best is the domain that sells worst. We are not going to resolve that by pretending the table says something else.

That is the whole argument, and notice that it does not need Well-led to be the sharpest signal. It is not the sharpest signal. It is the one where the phone call has somewhere to go, sitting on top of a risk that is real either way.

If what you want is the widest net rather than the most workable one, take the wider filter. It is one pass over CQC's monthly spreadsheet, it costs nothing, and the file is linked below. Saying that here is cheaper than you working it out in month three.

This site used to imply that Well-led goes first, and that the flag was a warning the other domains do not give. That was an assumption nobody had tested. It has now been tested and it did not survive, so it is gone from the copy. The comparisons that killed it are the last column of the three-window table and the five-key-question table under it.

One split in this dataset changes the size of the effect more than any other, and it was not on this page until now. Take the same two arms and separate them on how old the baseline Well-led judgement already was. A mark published in the last two years is an inspector's current finding. An older one may already have been worked through, with the home carrying a label that no longer describes it.

Risk ratios, flagged against clean, computed inside each stratum. Both arms split on the same cut. Intervals are 95%.

In the 4-year window the recent-mark stratum is 43 falls out of 208 flagged homes against 169 out of 1,788 clean ones, a risk ratio of 2.19 [1.62-2.96]. The older-mark stratum is 1.19 [0.93-1.52], and its interval touches 1 in all three windows. The whole of the headline effect is concentrated in homes whose Well-led mark is still fresh.

Only the 4-year recent-mark figure clears the multiplicity threshold above, so this is one cut on one dataset and not a law. It is published because it is the most useful thing in the study for anyone actually working a list: a flagged home whose Well-led mark went up last year is a different prospect from one whose mark has sat there since before the pandemic, and nothing on this site said so before today.

Nothing here is causal. Two groups of homes differed on a public record at a baseline date and were followed forward. Nobody assigned anything, so the study cannot say the Well-led mark caused the downgrade, only that homes carrying it were downgraded more often.

And the base rate is the base rate. Over 4 years, 697 of the 803 flagged homes did not lose their overall rating: roughly seven in eight. The flag moves the odds from about one in eleven to about one in eight. That is a targeting instrument. Anyone who opens a call as though it were a prophecy will be corrected by the person who picks up, and deservedly.

None of this needs to be taken on trust. The results table every figure on this page is drawn from is here, so are the scripts that produced it, and so is the public archive they read.

The spreadsheets are CQC's own, published under the Open Government Licence. Three baselines and one endpoint, plus the closure file the disappearances were reconciled against. The baselines come out of the archived monthly folder linked above. The other two are current downloads from CQC.

• 01 July 2022 Latest ratings.ods
• 03 July 2023 Latest ratings.ods
• 01 July 2024 Latest ratings.ods
• 01_July_2026_Latest_ratings.ods
• 01_July_2026_Deactivated_Locations.ods

To re-run it: pull the four spreadsheets, run extract.mjs over each one, then analyse.mjs, and it writes the same table. To check a single home rather than the whole cohort, verify.mjs prints its raw rows out of all four snapshots so you can follow one location id through the join by hand. If a number here does not survive that, we want to hear about it, and the number changes.

You can build the flag list yourself out of the same monthly file, and the method page sets out, item by item, what it will not come with. Those items are what we charge for. This page is only here to tell you what the sorting underneath them is worth, before you decide whether the rest is.