Rating puzzle difficulty by decisions
Grid size is the obvious difficulty proxy and it is wrong. Measuring the branching factor of the solve gives a rating that survives contact with players.
The walk and the tiers, ready to adaptGist · IndieCoreDev/02835f475522e68083df2567f1a562b5Part 3 of 10 on building a word-block puzzle engine. Part 2: generate levels offline.
Every puzzle game has to answer "how hard is this level" before a player ever plays it, and the proxies within easy reach are all bad. Grid size is bad. Piece count is bad. Number of words is worse than either. I shipped a first pass ordered by grid size and it produced a curve that went sideways for eighty levels and then fell off a cliff.
The rating I use now measures the one thing that actually varies: how much the player has to work out.
The insight that made it measurable
The player never sees the answer. The blocks carry letters, but they cannot be matched against anything — there is no ghost image of the finished board. So the board is a fitting problem, and its difficulty is how often the fit is forced and how often it is a guess.
That suggests a procedure, and the procedure is what a person actually does with a jigsaw:
Fill the topmost-leftmost empty square. Count how many blocks still in hand could go there.
One means the board placed the block for you and cost you nothing. Five means a real decision, and a wrong answer that will have to be undone later.
// Rating a fitting puzzle by the decisions it forces.
//
// The player never sees the answer, so the board is a fitting problem and its
// difficulty is how much has to be worked out. That suggests a procedure, and
// the procedure is what a person actually does with a jigsaw:
//
// fill the topmost-leftmost empty cell,
// count how many pieces still in hand could go there.
//
// One means the board placed the piece for you and cost you nothing. Five means
// a real decision, and a wrong answer that will have to be undone.
//
// Everything downstream is arithmetic on the list this returns: the difficulty
// rating, the coin reward, and the two power-ups (place a piece / flag a wrong
// one) which are this same walk with a different question asked of it.
//
// Cost: at most ~30 pieces per board, so a walk is a few thousand cell tests.
// Well under a frame on a phone.
/** Occupied cells of a piece, relative to its own bounding box. */
export const pieceCells = (piece) => {
const out = [];
piece.shape.forEach((line, r) => {
[...line].forEach((ch, c) => {
if (ch !== '.') out.push({ r, c, ch });
});
});
return out;
};
// Memoised onto the piece. Boards arrive from two places — a script reading the
// pack off disk, and a fetch in the browser — and only the first had any reason
// to decorate them. Going through this means a board works the same whichever
// door it came in.
const cellsOf = (piece) => {
if (!piece._cells) piece._cells = pieceCells(piece);
return piece._cells;
};
/**
* Walk a board and return the branching factor at each step.
*
* `preplaced` models a power-up that puts a piece down for the player.
* `revealedRows` / `revealedCols` model a hint that shows true letters: a
* candidate piece must then match every revealed letter it would cover.
*/
export const walk = (board, { preplaced = [], revealedRows = null, revealedCols = null } = {}) => {
const R = board.rows;
const C = board.cols;
const occ = Array.from({ length: R }, () => new Array(C).fill(null));
const hand = new Map();
for (const p of board.pieces) hand.set(p.id, p);
const put = (p) => {
const [hr, hc] = p.home;
for (const { r, c } of cellsOf(p)) occ[hr + r][hc + c] = p.id;
hand.delete(p.id);
};
for (const id of preplaced) if (hand.has(id)) put(hand.get(id));
const known = (r, c) => Boolean(revealedRows?.has(r)) || Boolean(revealedCols?.has(c));
const steps = [];
while (hand.size) {
// The topmost-leftmost empty cell is the next thing to fill.
let tr = -1;
let tc = -1;
outer: for (let r = 0; r < R; r++) {
for (let c = 0; c < C; c++) {
if (occ[r][c] === null) { tr = r; tc = c; break outer; }
}
}
if (tr < 0) break;
// How many pieces in hand could legally cover that cell, at any offset.
let choices = 0;
for (const p of hand.values()) {
let fits = false;
for (const anchor of cellsOf(p)) {
const or = tr - anchor.r;
const oc = tc - anchor.c;
if (or < 0 || oc < 0 || or + p.h > R || oc + p.w > C) continue;
let ok = true;
for (const { r, c, ch } of cellsOf(p)) {
if (occ[or + r][oc + c] !== null) { ok = false; break; }
if (known(or + r, oc + c) && board.solution[or + r][oc + c] !== ch) { ok = false; break; }
}
if (ok) { fits = true; break; }
}
if (fits) choices++;
}
steps.push({ choices, at: [tr, tc] });
// Then place the piece that genuinely belongs there and move on.
let truth = null;
for (const p of hand.values()) {
const [hr, hc] = p.home;
if (cellsOf(p).some(({ r, c }) => hr + r === tr && hc + c === tc)) { truth = p; break; }
}
if (!truth) break;
put(truth);
}
return steps;
};
Walk the whole board that way and you get a list of branching factors, one per step. Everything else is arithmetic on that list.
board pieces forced steps average choice worst step
8x8 13 38% 3.5 11
10x10 17 35% 3.9 12
17x10 25 32% 5.6 18
Three numbers ship on every board — plus a fourth the economy needs, which is the same list summed:
// The numbers the walk is for.
//
// `walk()` returns one branching factor per step. Everything downstream is
// arithmetic on that list — three numbers ship on every board and are checked
// against the generator's own figures at build time, and the fourth is the one
// the economy needs.
/**
* The three numbers that ship on every board, plus the one the economy needs.
*
* `decisions` is the sum of (choices - 1) over the whole solve — the total
* count of real choices the board forced. It runs 7.5 on the easiest tier to
* 96.7 on the hardest, a 13x spread, which is why nothing in the economy is a
* flat rate.
*/
export const summarise = (steps) => {
if (!steps.length) return { steps: 0, forced: 0, avg: 0, worst: 0, decisions: 0 };
const n = steps.map((s) => s.choices);
return {
steps: n.length,
forced: n.filter((x) => x === 1).length / n.length,
avg: n.reduce((a, b) => a + b, 0) / n.length,
worst: Math.max(...n),
decisions: n.reduce((a, b) => a + (b - 1), 0),
};
};
Why the walk had to be exact
It would be tempting to approximate — sample a few steps, or estimate from piece shapes. I did not, for a reason that only became obvious later: the same walk runs on the device.
Two power-ups need it. One places a block for the player, which means choosing which block placement helps most; the other flags a block that is currently wrong. Both are the walk with a different question asked of it. And the economy (part 8) prices a level off the total decision load, which is the same list summed.
So the walk exists three times — in the offline rater, in an independent verifier, and in the engine at runtime — and a test asserts that the runtime one reproduces the numbers the rater shipped, board for board, or the build fails. Approximation would have made that impossible to check.
The cost on device is nothing to worry about: at most 30 blocks per board, so a walk is a few thousand cell tests, and choosing the best anchor is thirty of those. Well under a frame, and it only runs when someone taps a power-up.
One number hides too much
overall: 7.2 tells a player nothing, and it told me nothing either — two boards rating the
same felt completely different to play. So the rating carries four dimensions, each a percentile
within the library, on one 0–10 scale:
| dimension | what it measures | weight |
|---|---|---|
deduction |
how often a genuine choice has to be made | 0.35 |
peak |
the worst single decision on the board | 0.25 |
scale |
how much there is to place at all | 0.25 |
sameness |
blocks sharing an outline, which the eye cannot separate | 0.15 |
sameness is the one I would not have thought of from first principles. A board where six
blocks are the same silhouette is harder in a way none of the other three catch, because the
difficulty is not deduction — it is that you cannot see the difference and have to try them.
What is deliberately not in the list matters as much. The reference model this follows has dimensions for precision, reaction, memory, time pressure and randomness. This game has no timer, no dexterity and no luck, so those are left out rather than filled with zeroes. A dimension that is always 0 is a column of noise that dilutes every weight next to it.
Ratings must overlap across sizes
Here is the test that told me the model worked. If difficulty were really about size, ratings would form disjoint bands: all the 8×8s below all the 10×10s below all the 13×13s.
They do not. Across the library ratings run 0.6 to 9.3, and a 10×10 spans 4.0 to 7.7 — it meets the 8×8s below it and the 13×13s above it. There are 10×10 boards harder than most 13×13s, which matches how they play and is invisible to any size-based proxy.
That overlap is also what makes a smooth campaign possible at all. If every size were its own band you could only ramp difficulty by ramping size, and the player would watch the board grow rather than the puzzle deepen.
Turning a number into something a player can read
Nobody reads "6.4 / 10, deduction 9.2" and knows what they are in for. So there is exactly one module that turns the model into player-facing language, and every difficulty surface in the game — the level grid, the briefing card, the in-game banner — reads from it. Three surfaces computing their own tiers is three surfaces that eventually disagree.
The tier boundaries are the pack's own quartiles, not invented round numbers:
// Turning a 0-10 rating into something a player can read.
//
// Nobody reads "6.4 / 10, deduction 9.2" and knows what they are in for. This
// is the one place the model becomes language, and every difficulty surface in
// the game reads from here — the level grid, the briefing card, the in-game
// banner. Three surfaces computing their own tiers is three surfaces that
// eventually disagree.
//
// The words are deliberately NOT here. A tier carries a `key` and the caller
// looks it up in the string catalogue, which is what lets the model ship in
// seventeen languages without seventeen copies of the quartiles.
/**
* Tier boundaries are the pack's own quartiles, not invented round numbers:
* p25 / p50 / p75 / p90 of the shipped library.
*
* That splits 392 boards 23 / 25 / 24 / 15 / 12 percent, so the top tier stays
* rare enough to mean something when one turns up. Round numbers put 60% of the
* library in a single tier.
*
* Rebuild the pack and these want re-measuring — nothing else will tell you.
*/
export const TIER_BOUNDS = [2.9, 5.1, 6.9, 8.0];
export const TIERS = [
{ key: 'easy', level: 1, color: '#10b981', deep: '#047857', ink: '#ffffff' },
{ key: 'steady', level: 2, color: '#38bdf8', deep: '#0369a1', ink: '#062f4a' },
{ key: 'tricky', level: 3, color: '#f59e0b', deep: '#b45309', ink: '#3b2503' },
{ key: 'hard', level: 4, color: '#f43f5e', deep: '#9f1239', ink: '#ffffff' },
{ key: 'expert', level: 5, color: '#a855f7', deep: '#6b21a8', ink: '#ffffff' },
];
/** The tier a 0-10 rating falls in. Unrated boards fall back to the middle. */
export const tierFor = (overall) => {
if (typeof overall !== 'number' || Number.isNaN(overall)) return TIERS[1];
let i = 0;
while (i < TIER_BOUNDS.length && overall >= TIER_BOUNDS[i]) i += 1;
return TIERS[i];
};
/**
* The four dimensions behind the single number, each a percentile within the
* library, all on one 0-10 scale.
*
* `sameness` is the one that is not obvious from first principles: a board
* where six pieces share an outline is harder in a way the other three cannot
* catch, because the difficulty is not deduction — it is that you cannot see
* the difference and have to try them.
*
* What is left out matters as much. The reference model also lists precision,
* reaction, memory, time pressure and randomness. This game has no timer, no
* dexterity and no luck, so those are omitted rather than filled with zeroes —
* a dimension that is always 0 is a column of noise diluting every weight next
* to it.
*/
export const DIMENSION_WEIGHTS = {
deduction: 0.35, // how often a genuine choice has to be made
peak: 0.25, // the worst single decision on the board
scale: 0.25, // how much there is to place at all
sameness: 0.15, // pieces sharing an outline
};
/**
* The rating is a prior, not a truth.
*
* Every board ships `confidence: 0.2` and `sampleCount: 0` because there are no
* players yet. This measures the *board*, which is not the same as measuring
* the difficulty a person experiences.
*
* The seam is left open deliberately: when telemetry arrives, blend towards
* observed difficulty and raise confidence with the sample count, rate-limited
* so a handful of unusual players cannot move a rating far. Consumers already
* read `confidence`, so the day it changes nothing downstream is rewritten.
*/
export const isProvisional = (difficulty) => (difficulty?.confidence ?? 0) < 0.5;
p25, p50, p75, p90. That splits 392 boards 23 / 25 / 24 / 15 / 12 percent across five tiers, which keeps the top tier rare enough that seeing one means something. Round numbers would have put 60% of the library in one tier.
Those constants are pack-specific and there is a report script that prints the current split, because rebuilding the pack moves the quartiles and nothing else will tell you.
The words themselves are not in that module — a tier carries a key and the caller looks it up in the string catalogue. That split is what lets the model ship in seventeen languages without seventeen copies of the quartiles in them.
The rating is a prior, not a truth
The most important line in the model is the one that admits what it does not know. Every board ships with:
"confidence": 0.2,
"sampleCount": 0
There are no players yet. The rating is a measurement of the board, which is not the same thing as a measurement of the difficulty a person experiences, and pretending otherwise is how you end up defending a number against your own players.
The seam for fixing that is deliberately left open: when telemetry arrives, blend toward
observed difficulty and raise confidence with the sample count, with a rate limit so a handful
of unusual players cannot move a rating far. None of that is built. But confidence is in the
data and every consumer reads it, so the day it changes nothing downstream has to be rewritten.
What I would tell someone starting this
Find the thing the player is actually doing, and count it. Not the thing you can measure easily — the thing that is hard. For this game it was branching factor; for a match-3 it might be the number of boards reachable in one move; for a platformer, the length of the longest sequence with no checkpoint.
And write the second implementation. The verifier that re-derives the ratings independently has paid for itself twice, and both times the bug was a shared assumption that a unit test of the first implementation was structurally incapable of finding.
Next: part 4, difficulty rates a stage, progression places it — why sorting your levels by difficulty produces a terrible campaign.