How Image Resizing Actually Works

An image is a grid of measurements. A 4000×3000 photo holds twelve million samples of the light that fell on a sensor, arranged on a fixed lattice. When you resize it to 800×600, the new grid's sample points don't line up with the old ones — a pixel at position 137 in the output sits somewhere between pixels 685 and 686 in the input. Resizing is the problem of deciding what value belongs at a coordinate where nothing was ever measured. That problem has several answers, and they look different.

Upscaling: guessing between known points

Start with the easier direction. Enlarging means the output grid is denser than the input, so every output pixel falls between input samples and needs an interpolated value.

Nearest neighbor rounds to the closest input pixel and copies it. It's the fastest option and the only one that never invents a color that wasn't already in the image. That makes it correct — not merely acceptable — for pixel art and for anything where exact color values carry meaning, like an indexed screenshot or a sprite sheet. Enlarge a photo with it and you get visible square blocks, because that's literally what it produces: each input pixel becomes a solid rectangle.

Bilinear takes the four surrounding input pixels and blends them by distance, weighting each by how close it is along both axes. Every output pixel is a weighted average, so the blockiness disappears — replaced by softness, because averaging is a blur. Edges that were one pixel wide become two or three pixels of gradient.

Bicubic uses a 4×4 neighborhood and fits a cubic curve through it instead of a straight line. The curve's slope at each sample carries information a linear blend throws away, so edges stay noticeably crisper. The cost is that the cubic kernel has negative lobes: near a sharp light-to-dark transition it can produce values slightly darker than anything in the original, which reads as a faint dark halo hugging the edge. That overshoot is usually a feature — it's the same trick as an unsharp mask — but it's why bicubic output can look subtly "contrasty" compared to bilinear.

Lanczos extends the idea further, using a windowed sinc function over a larger neighborhood (typically 6×6 or 8×8). Signal theory says sinc is the mathematically ideal reconstruction filter for a band-limited signal; windowing it to a finite width is the practical compromise. Lanczos preserves fine detail better than bicubic and overshoots slightly more, which is why it's the default in many command-line tools and why it can make already-sharp images look over-processed.

There is no algorithm that recovers detail the original didn't capture. Upscaling redistributes existing information across more pixels. The differences between these methods are entirely about how the existing information is spread, not about adding any.

Downscaling: the part everyone gets wrong

Shrinking looks like the easy direction — you're throwing information away, and how hard can that be? — but it's where most bad output comes from, and the reason is aliasing.

Suppose you shrink a 4000-pixel-wide image to 400 pixels using nearest neighbor. The algorithm samples every tenth pixel and discards the other nine. If the image contains a fine repeating pattern — a striped shirt, roof tiles, a distant fence, text — the sampling grid and the pattern interfere. Stripes that repeat every 9 pixels, sampled every 10, produce a slow-moving beat pattern that wasn't in the photograph. That's a moiré artifact, and it's not a rendering bug; it's the Nyquist–Shannon sampling theorem asserting itself. A grid of 400 samples cannot represent detail finer than about 200 cycles across the image. Detail above that limit doesn't vanish gracefully — it folds back down and masquerades as low-frequency structure.

The fix is to remove the detail before sampling. A correct downscale blurs the image to below the new Nyquist limit first, then samples. In practice good implementations do both steps at once by making the filter kernel scale with the reduction factor: shrinking by 10× means each output pixel averages roughly a 10-pixel-wide neighborhood rather than a fixed 4×4. This is why "box filter" or "area averaging" — literally averaging every input pixel that falls inside an output pixel's footprint — is a solid default for large reductions, and why a naive bicubic with a fixed kernel can still alias badly when shrinking by a large factor.

This is also why halving repeatedly often beats a single big jump in naive implementations: each 2× step keeps the fixed kernel wide enough relative to the reduction. It's a workaround for a filter that doesn't scale, not a superior technique in itself.

The gamma problem hiding underneath

There's a second correctness issue that affects both directions and is invisible until you look for it.

Pixel values in a normal image file are not proportional to light. sRGB stores them through a transfer curve roughly equivalent to raising the value to the power 1/2.2, which allocates more of the available codes to dark tones where the eye is more sensitive. That's a good encoding choice and a terrible thing to do arithmetic on.

Averaging two pixels means averaging light. But averaging their encoded values isn't the same operation. Take pure black (0) and pure white (255): the average encoded value is 128, which in sRGB represents about 22% of the light of white — not 50%. A checkerboard of black and white squares, downscaled naively, comes out visibly darker than the mid-gray it should be.

The correct procedure is to convert to linear light, resample there, and convert back. Many image pipelines skip it, which is why shrinking a detailed image sometimes darkens it in a way that's hard to name. The effect is strongest on high-contrast fine detail — exactly the content that also aliases.

What this means in practice

A few rules fall out of the mechanics:

• Pixel art, screenshots, and indexed images: nearest neighbor, and only at integer scale factors. Any smoothing filter destroys the property that makes them what they are.
• Photos, enlarging: bicubic or Lanczos. Don't expect miracles; you're spreading detail, not creating it.
• Photos, shrinking: a filter whose kernel scales with the reduction. If you have a choice, prefer area averaging or Lanczos over fixed-kernel bilinear.
• Shrink once, from the original. Every resize is lossy in the information-theoretic sense, and if the format is also lossy, each save compounds it. Going 4000 → 1200 → 800 is worse than 4000 → 800.
• Resize before compressing, not after. Removing high-frequency detail first gives the compressor a much easier signal to encode, which is why an image resizer plus modest compression usually beats aggressive compression at full size for the same target file size.

The uncomfortable summary is that resizing has no neutral setting. Every method makes a specific trade between sharpness, ringing, and aliasing, and the "best" one depends entirely on whether your image is a photograph or a grid of deliberate pixels. What you can avoid is the failure mode nobody chooses: a naive downscale that aliases fine detail into patterns the camera never saw.