Tuning a sonar mechanic with simulated players

Echo Bat is a one-touch cave flyer. The cave is pitch black, and the bat sees by sonar: a ring goes out, lights the rock it reaches, and fades. On paper that is the whole game. You fly on what the last ping showed you.

After the first round of art and tuning I flew a build on the emulator, and my verdict fit on one line: it has become Flappy Bird, every obstacle is always visible. The darkness was decoration.

This post is how I measured that, and the three changes to the sonar mechanic that made the dark matter without making it unfair. The fairness numbers come from simulated players flying the game's real rules: about 12,000 runs in the first playtest, sweeps of a few hundred runs per variant, and 3,000 to confirm each change. The emulator checked what the screen actually showed.

A player that only knows what the sonar showed

Whether darkness matters is a question about information: what did the player know, and when did they know it? So the bot has to be limited to what the sonar has lit. Mine reacts after a human delay, aims with some error, and learns where a gap is only once an echo has lit the rock around it past a threshold. Until then it holds the middle of the cave.

This is a simplified version, not the game's code:

// Simplified: a player who only knows what the sonar has shown it.
function makePlayer({ lag, aimNoise, seeAbove = 0.15 }) {
  const known = new Map(); // obstacle id -> { aim, seenAt }
  return {
    look(run) {
      for (const ob of obstaclesAhead(run)) {
        if (known.has(ob.id)) continue;
        if (lightAt(run, ob.gapEdge) > seeAbove) {
          known.set(ob.id, { aim: ob.gapMiddle + gauss() * aimNoise, seenAt: run.time });
        }
      }
    },
    aim(run) {
      const next = nextObstacle(run);
      return known.get(next?.id)?.aim ?? run.caveMiddle; // unseen: hold the middle
    },
    sawKiller(run) {
      return known.has(obstacleAt(run, run.batX)?.id);
    },
  };
}

Three skill levels: casual (0.26 s reaction, aim off by up to about 22 units), average (0.2 s, 14) and sharp (0.15 s, 8). Each run records two things beyond the score:

• Lead time for each gap: how long before the bat reaches it the gap was first seen. Under 0.5 s counts as a late reveal, a gap you could not reasonably react to.
• Whether the rock that killed it had ever been seen.

The first playtest was 1,500 runs per skill. The rock that killed the bot had been seen first in 100% of them, at all three skill levels. A gap was first seen a median of 2.0 s before the bat reached it. Every flap pinged, and the bot flapped about 1.8 times a second, so the cave was simply never dark.

An unseen-death rate of 0% looks like good news. Here it was the symptom: nothing the dark hid ever mattered.

Shortening the echo trades one problem for another

The obvious fix is less light. I swept echo reach against a recharge time, so a flap only pings when the sonar is charged:

Echo reach, recharge	Median score	Killed by a rock never seen	Gap first seen (median)
420, every flap (as it was)	34	0%	1.96 s
220, 1.0 s	23	0%	1.09 s
220, 1.3 s	17	1%	0.73 s
180, 1.3 s	14	10%	0.68 s
180, 1.6 s, faster fade	9	32%	0.64 s

There is no comfortable row. Either the dark changes nothing, or it starts killing players with rock they never had a chance to see. A shorter echo alone barely helped either: in an earlier sweep, reach 260 with a 1.2 s recharge left a gap first seen under 0.35 s ahead in only 10% of runs. Once a gap has been seen, the player remembers it.

I shipped the recharge anyway, as a first step: 0.9 s at the start, slowing to 1.4 s by score 40, with a ring round the bat that fills as it charges. Eating a moth recharged it at once, which made the snacks worth something in sight. On the emulator the next gap now showed about 0.7 s ahead instead of 2 s, and the average bot's median dropped from 34 to about 14–17.

The ping that came too late

Playing it, I noticed the wave sometimes arrived with no time left to react. The simulator agreed. The sonar only pinged on a flap, so if it finished charging just after you flapped, the echo waited for your next one. By then the gap was close.

With the average player, 8.2% of gaps were revealed less than 0.5 s ahead, and 75% of runs had at least one. The worst 10% of gaps were seen only 0.55 s ahead.

Pinging by distance instead of by tap

I compared ways of firing the ping, 700 runs each with the average player. These were quick variants patched onto the rules from outside, and the last row used an approximation of the game's light, so I read the table as a ranking, not as final numbers:

Ping rule	Gaps seen late	Runs with one	Worst 10% seen ahead	Next gap dark
On a flap once charged	8.2%	75%	0.55 s	—
By itself once charged	2.6%	38%	0.71 s	—
Every 200 units flown	3.1%	38%	0.62 s	80%
Every 170 units flown	1.7%	27%	0.71 s	76%
Every 150 units flown	1.6%	29%	0.76 s	73%
Every 170, reach 240	0.8%	20%	0.92 s	67%

"Next gap dark" is the share of frames where the gap ahead was not lit enough to see. That column is the one that says whether the darkness survived the fix. A third of the time lit is still a dark cave.

What shipped: the bat pings by itself every 170 units it flies, a moth sends one extra ping, and a tap only flaps. The ring round the bat became a countdown to the next ping. Because the beat is tied to distance rather than time, the spacing between pings in the cave stays the same as the game speeds up.

On the real rules, 1,000 runs per skill:

	Casual	Average	Sharp
Gaps seen late	0.0%	0.0%	0.0%
Runs with a late gap	0%	0%	1%
Worst 10% seen ahead	1.04 s	0.94 s	0.84 s
Median score	19	32	45

The average player's median went from 20 to 32, and nobody was killed by a rock they had not seen. What I gave up is the choice of when to ping. That was a real part of the original idea, and I traded it for not punishing players for the timing of a recharge they could not see.

Letting the wave bend round rock

The next complaint was mine too: the waves did not act like waves. An echo was a circle that grew and lit whatever it overlapped, straight through rock. So I rebuilt it closer to sound. Rock stops the wavefront and lights only the face it strikes. The call is strongest straight ahead and weaker behind. Rock sends a ripple back toward the bat.

Stopping the wave at rock means the cave behind a rock is in an acoustic shadow. With fully black shadows, the unseen-rock deaths came straight back:

Killed by a rock never seen	Casual	Average	Sharp
Black shadows	1.8%	3.6%	9.7%
Shadows lit at 0.45 of the wave	0.3%	0.8%	2.0%

The second row is diffraction. Real sound bends round an edge, so the cave behind a rock gets a weaker wave instead of none, and what lies there is dim rather than invisible. I set that bent wave to 0.45 of the full strength. Late reveals came out at 0.4%, 0.7% and 1.5%, up from zero but still rare, with medians of 19, 30 and 38. A little mystery lives behind rocks now, and it rarely kills you.

I do not have an explanation for why sharp players die to unseen rock more often than casual ones, and I have not tested one, so I will not offer one.

At this point the light the bot reads and the light the screen draws came from the same function in the rules. That was deliberate. If the drawing had its own idea of what is lit, every number above would describe a different game from the one on the phone.

Building the cave two screens ahead

Rock used to be created about 120 units past the right edge, while the wave carried up to 900. So waves crossed empty space where the next obstacles should have been. Now at least two screens of cave always exist ahead of the bat, built a whole screen at a time. Waves hit rock before it comes into view, and their echoes come back from beyond the edge.

Batching also cut the work. Over a minute of flight, a wave's shape (where rock stops it) was recalculated 268 times instead of 454, 167 ms of work instead of 306 ms. On 1,000 runs per skill, unseen-rock deaths came out at 0.6%, 0.3% and 3.0%. The sharp figure is a point higher than before; I have not checked whether that is noise. Measuring this change is also where a git stash lost a staged file.

When the rules say dark and the screen does not

A few days later, with painted rock art in, the screen was still showing rock between waves. The rules said dark. The drawing disagreed, and the brightness came from three places:

• the background, drawn at a constant 30%;
• the rules' "never quite invisible" base light of 3%, multiplied by a 3× reveal gain in the drawing, so rock always showed at about 9%;
• the afterglow. The rules keep 14% of light on rock after a wave passes, fading over about a second, so a player can fly from memory. The same 3× gain turned that into rock about 40% visible. With a ping every 170 units, the tunnel never went dark.

I moved "how much the player sees without a wave" into one small module, which both the background and the rock read. The level below which nothing shows is tied to the rules' afterglow constant instead of being a number of its own. And there is a test that lights rock with afterglow only and asserts nothing is revealed.

That test failed the first time it ran. My earlier threshold sat below the afterglow's peak, so rock right next to the bat would have stayed visible. That was the leak I had been staring at.

On the emulator, rock areas then measured about 2 out of 255 between waves and about 6 inside one. I tried the no-wave level at 20%, 5% and 1.5% before settling on 2%. Later the reveal got its own trail, half a second behind the wavefront and eased so it fades without a visible step, and the afterglow stopped playing any part in what is drawn.

None of that touched the rules, so the bot plays exactly as before. It also means the fairness numbers above describe the rules' light. I have not re-run them against the drawn picture.

What I would do again

Measure two numbers, not one. "Killed by something unseen" catches a mechanic that is unfair, and "how often the next obstacle is dark" catches one that does nothing. On Echo Bat the first number read 0% for days while the mechanic was not working at all.

Give the bot only what the player has. A bot that reads rock positions directly would have scored the same in a lit cave and a black one, and told me nothing about darkness.

Keep one definition of "lit". The rules, the bot and the screen each had a chance to disagree, and the screen did. One module and one test tied to the afterglow constant closed it.

The limit is the obvious one. These are simulated players with a fixed reaction time and aim error. They say a gap was visible 0.84 s ahead. They cannot say whether a person who has never played will notice it.