"Decimated heightfield cliffs render as sawtooth teeth — snap the above-gap cluster"
▸ SYMPTOM
A decimated heightfield mesher (one that reduces a dense height grid to a coarser triangle mesh) renders carved cliffs as an alternating sawtooth — the crest reads as a row of teeth instead of a clean lip. The two obvious fixes disappoint:
- Forcing consistent diagonals cleans the mid-wall but leaves the crest serrated.
- A majority-vote corner snap (snap each mixed corner to whichever height most of its samples agree on) does not fix it either.
▸ CAUSE
There are two stacked artifacts, and only probing the actual per-corner numbers separates them:
-
Mid-wall: the "pick the smaller-delta diagonal" rule flips triangle orientation block-to-block when both diagonals carry similar (large) deltas across a steep wall. The fix is to force one diagonal on wall-relief blocks so the orientation stops flipping.
-
Crest: the cliff line runs diagonally through the cell lattice, so the mixed 4-cell corner column drifts about one cell every two to three rows. Corner averaging then smears a 45° ramp along several block rows of crest — that smear is the sawtooth fringe.
A majority vote fails on the crest specifically because on a one-to-two-cell-wide carved wall the transition cells carry intermediate heights: mixed corners are mostly 2–2 splits, so the majority never fires; and where it does fire it snaps in alternating directions, which is the sawtooth all over again.
▸ FIX
Replace the corner average with a gap-cluster snap:
- Sort the four corner height samples.
- Find the largest gap between consecutive sorted samples.
- If that gap exceeds
wallSlope × cellSpan, take the mean of the cluster above the gap (not the mean of all four).
This is a uniform plateau bias: it dilates the lip by at most one cell and keeps the lip polyline level, which matches a plateau-first, faceted silhouette. To avoid flicker on near-threshold graded ramps, blend the snap in over the interval [gap, 2 × gap] so a corner right at the threshold can't oscillate between "snapped" and "averaged" frame to frame.
▸ WHY IT WORKS
Averaging pulls the lip toward the mean of a diagonally-drifting corner column, which is exactly what smears the 45° ramp. Taking the mean of only the cluster above the largest gap keeps every crest corner pinned to the top surface, so the lip stays a level polyline instead of a graded ramp — the teeth collapse into a clean edge.
Corollary — faceted styles only. The snap suits a faceted (flat-shaded) mesh. On a smooth-shaded mesh the residual cluster wobble on graded ramps punches a new grazing-angle sawtooth into the skyline, so gate the snap behind a per-style flag rather than applying it everywhere.
- Published