The Gamma Trap: Why Averaging Colors Goes Wrong

Here is a small experiment. Make an image of a fine black-and-white checkerboard, one pixel per square. Shrink it down until the squares are no longer resolvable. What color should it be?

Half the light is coming from white squares and half from black ones, so the answer should be a mid-gray — the same gray you'd get from a surface reflecting 50% of the light. What most image tools actually produce is a gray around #808080, which reflects roughly 22%. It looks obviously too dark next to the real thing.

Nothing is broken. This is what happens when you do arithmetic on values that aren't linear in the quantity you're averaging.

What the number 128 actually means

An 8-bit channel gives 256 possible values, and they have to be distributed across the range from black to white. Distributing them evenly by light intensity would be wasteful, because human brightness perception is strongly non-linear: we can distinguish many more steps in the darks than in the highlights. A linear encoding would waste most of its codes on bright tones where the differences are invisible, and give the shadows so few that gradients would band visibly.

So sRGB — the color space essentially every image file and every display assumes by default — stores values through a transfer function. Encoding applies roughly a power of 1/2.2; the display applies roughly a power of 2.2 to undo it. The round trip is correct, so images look right, and the encoded values are distributed in a way that's efficient for perception.

The consequence is that the stored number is perceptually linear, not physically linear. A value of 128 is about halfway up the perceptual scale, and it emits about 22% of the light of 255. Both statements are true and they're what makes this confusing.

That's fine as long as you only display the values. It stops being fine the moment you average them, because averaging is a physical operation on light.

Where this shows up

Resizing. Every resampling filter is a weighted average of neighboring pixels. Averaging encoded values instead of light means the result is biased dark, and the bias grows with local contrast. Shrink a photo of a tree against a bright sky and the fine branches darken more than they should; shrink a flat blue wall and nothing visible happens. This is why some tools produce noticeably brighter, more faithful thumbnails than others from the same source — one of them converts to linear light before filtering. It's the same mechanism described in how image resizing works, operating one layer down.

Alpha compositing. Blending a foreground over a background with result = fg × α + bg × (1 − α) is a weighted average. Done on encoded values, semi-transparent edges come out too dark. Anti-aliased text is the classic victim: the partially covered pixels along each glyph edge get darkened, so white text on a dark background looks thinner and dark text on white looks heavier than the font intends. On low-resolution displays this is visible enough that font rendering engines compensate for it explicitly.

Blurring and glows. A blur is a weighted average over a neighborhood. In encoded space, a bright highlight blurred against a dark background loses energy — the glow looks weak and muddy rather than bright and soft. Rendering engines that do lighting in linear space produce noticeably different bloom for exactly this reason.

Grayscale conversion. Collapsing RGB to a single channel is a weighted sum, usually with luma coefficients around 0.21 R + 0.72 G + 0.07 B. Those weights describe how much each primary contributes to perceived luminance — a physical quantity — so applying them to encoded values is another instance of the same mistake. It's not a catastrophic one, and it's why the result still looks plausible, but it isn't the luminance it claims to be. There's more on why the weights are so lopsided in grayscale is not just removing color.

Doing it correctly

The procedure is always the same three steps:

1. Decode each channel from sRGB to linear. Approximately linear = (v/255)^2.2; exactly, sRGB uses a small linear segment near zero and a power of 2.4 above it, which matters for very dark values.
2. Do the arithmetic — average, blend, blur, sum — in linear space.
3. Encode back to sRGB for storage or display.

Redo the checkerboard this way. Black is 0.0 linear, white is 1.0, the average is 0.5, and encoding 0.5 back to sRGB gives about 188 — a distinctly lighter gray than 128, and the one that actually matches a 50% reflective surface.

Two practical notes. First, don't do this in 8 bits. Linear values in the shadows need more precision than 8 bits provide; converting to linear, rounding to 8 bits, and converting back visibly bands the darks. Work in 16-bit integers or floats. Second, this only applies to values representing light. An alpha channel is a coverage fraction and is already linear; a normal map or a depth buffer stores geometry, not brightness. Applying a gamma transform to those is its own bug.

When to leave it alone

Perceptually-encoded values aren't a mistake to be corrected everywhere. They're the right space for some operations precisely because they track perception:

• Gradients between two colors often look more even when interpolated in a perceptual space; a linear-light gradient from black to white spends a lot of its length looking almost white.
• Palette extraction and color quantization are trying to group colors the way a viewer would, so clustering in a perceptual space (LAB or OKLab, better still than raw sRGB) gives more useful results than clustering in linear light. A color palette extractor that works in linear RGB tends to return colors that feel wrong even though they're physically defensible.
• Contrast and accessibility ratios are defined on relative luminance with a specific formula — follow the spec, not your instinct.

The distinction is simple once stated: if you're simulating what light does — combining it, spreading it, mixing it — work in linear space. If you're modeling what a person sees — sorting, clustering, interpolating for appearance — work in a perceptual space, and prefer a properly uniform one over sRGB.

Most image bugs in this family come from doing neither deliberately: reaching for whatever numbers are in the buffer and treating them as though they mean light, because they're stored as plain numbers and nothing warns you otherwise.