the symptom, in your words

"Decimated heightfield cliffs render as sawtooth teeth — snap the above-gap cluster"

✓ verified on 26.07.15a
lane Writing gameplayposted

▸ SYMPTOM

A decimated heightfield mesher (one that reduces a dense height grid to a coarser triangle mesh) renders carved cliffs as an alternating sawtooth — the crest reads as a row of teeth instead of a clean lip. The two obvious fixes disappoint:

  • Forcing consistent diagonals cleans the mid-wall but leaves the crest serrated.
  • A majority-vote corner snap (snap each mixed corner to whichever height most of its samples agree on) does not fix it either.

▸ CAUSE

There are two stacked artifacts, and only probing the actual per-corner numbers separates them:

  1. Mid-wall: the "pick the smaller-delta diagonal" rule flips triangle orientation block-to-block when both diagonals carry similar (large) deltas across a steep wall. The fix is to force one diagonal on wall-relief blocks so the orientation stops flipping.

  2. Crest: the cliff line runs diagonally through the cell lattice, so the mixed 4-cell corner column drifts about one cell every two to three rows. Corner averaging then smears a 45° ramp along several block rows of crest — that smear is the sawtooth fringe.

A majority vote fails on the crest specifically because on a one-to-two-cell-wide carved wall the transition cells carry intermediate heights: mixed corners are mostly 2–2 splits, so the majority never fires; and where it does fire it snaps in alternating directions, which is the sawtooth all over again.

▸ FIX

Replace the corner average with a gap-cluster snap:

  1. Sort the four corner height samples.
  2. Find the largest gap between consecutive sorted samples.
  3. If that gap exceeds wallSlope × cellSpan, take the mean of the cluster above the gap (not the mean of all four).

This is a uniform plateau bias: it dilates the lip by at most one cell and keeps the lip polyline level, which matches a plateau-first, faceted silhouette. To avoid flicker on near-threshold graded ramps, blend the snap in over the interval [gap, 2 × gap] so a corner right at the threshold can't oscillate between "snapped" and "averaged" frame to frame.

▸ WHY IT WORKS

Averaging pulls the lip toward the mean of a diagonally-drifting corner column, which is exactly what smears the 45° ramp. Taking the mean of only the cluster above the largest gap keeps every crest corner pinned to the top surface, so the lip stays a level polyline instead of a graded ramp — the teeth collapse into a clean edge.

Corollary — faceted styles only. The snap suits a faceted (flat-shaded) mesh. On a smooth-shaded mesh the residual cluster wobble on graded ramps punches a new grazing-angle sawtooth into the skyline, so gate the snap behind a per-style flag rather than applying it everywhere.

Verified on engine 26.07.15a: seen in a real project.
s&box moves fast; an undated fix is a liability. Spot a stale detail?
changelog
  • Published

Want to know when new guides or fixes drop? Join the community to help build this out. Report gotchas, flag outdated fixes, or just lurk.

Join the Discord