Type-Safe Computer Vision · Part 1 of 3

The Blue Face Bug

Your camera says blue-green-red. Your model says red-green-blue. The bytes in between say nothing at all, and no one has to notice.

Thomas Stephan · 11 September 2026 · Series index

The same three bytes, twice

Here is a pixel. Three bytes, and nothing else:

Nothing in those three bytes says which channel comes first. The order is a convention, and computer vision has two common ones. Most of the world (PNG, JPEG decoders, browsers, GPUs, PyTorch, your screen) speaks RGB. OpenCV decided in the early 2000s to store images as BGR, because the Windows bitmaps and cameras of that era did. Both conventions are fine. The bug lives in the handoff.

The bug you have already written

Every vision developer writes this program once:

# load with OpenCV (returns BGR), show with matplotlib (expects RGB)
img = cv2.imread("portrait.jpg")
plt.imshow(img)      # ← everyone's face is now blue

And every vision developer gets lucky, because this version of the bug is loud. A blue face is obvious. You laugh, you add cv2.cvtColor(img, cv2.COLOR_BGR2RGB), you move on.

what the file contains

what your code displayed

Now remove the human. Feed that swapped image into a neural network, a color-based blob detector, a skin-tone filter, a white-balance routine. None of them crash. None of them warn. They return answers that look exactly like correct ones: a classifier trained on RGB quietly loses accuracy on BGR input, a "find the red marker" function finds blue markers, and nothing in any log file will ever tell you why.

A blue face on screen is a funny bug. A blue face inside a tensor is a silent one.

Why this keeps happening

Search any programming forum for cvtColor BGR2RGB and you will find thousands of hits: matplotlib displays, PIL round-trips, model inference gone wrong, camera SDKs handing over BGRA buffers that got pasted into RGB pipelines. This bug is more than twenty years old, perfectly understood, and still written every day. Why?

Because in almost every vision library, an image is just an array of numbers. A NumPy array of shape (H, W, 3). A cv::Mat of type CV_8UC3. A bare pointer from a camera SDK. The channel order lives in documentation, in variable names, in the developer’s head, everywhere except in the data itself. The type system, the one tool that checks your assumptions on every single line, has been told nothing. It is color-blind, so it checks nothing.

There is no flawed logic anywhere. Every function did exactly what it should. The mistake was that two parts of the program disagreed about what the bytes meant, and had no way to find out. That is not a logic bug. It is a type bug, forced to masquerade as a runtime mystery because the types were too weak to catch it.

Making the compiler care about channel order

Here is the same situation in fovea, a Rust computer-vision library where the pixel type records its own layout. A BGR buffer from a camera SDK is an Image<Bgr8>, not an array you have to remember things about:

use fovea::pixel::{Bgr8, Rgb8};
use fovea::transform::{ColorSwap, convert_image};

// The camera SDK documents its buffer as BGR, expressed in the type.
let frame: Image<Bgr8> = Image::from_raw_bytes(w, h, sdk_buffer)?;

// A function that needs RGB *says* it needs RGB:
fn detect_marker(img: &Image<Rgb8>) -> Option<Point> { /* … */ }

detect_marker(&frame);
// error[E0308]: expected `&Image<Rgb8>`, found `&Image<Bgr8>`

The bug that used to hide inside a tensor now fails immediately, before the program exists. And the fix is a conversion with a name, written where the conversion actually happens:

// Swap R ↔ B, on purpose, visibly, exactly once:
let rgb: Image<Rgb8> = convert_image(&frame, ColorSwap);

detect_marker(&rgb);   // ✓ compiles, and is actually correct

Two details matter here. First, Bgr8 and Rgb8 have identical memory layouts: three bytes, no padding. The wrapper costs nothing at runtime; taking the SDK’s buffer is zero-copy. The only thing that changed is that the meaning of the bytes is now written down where the compiler can read it. Second, the conversion is named. Six months from now, ColorSwap in a diff tells you precisely what crossed the boundary and why. A silent reinterpretation never can.

The stakes are going up

Computer vision is moving from demos into factories, vehicles, laboratories, and medical devices. At the same time, vision code is increasingly written by coding agents at a speed no human reviewer can match. Both trends point the same way: the class of bug that is invisible in the data and absent from the logs is exactly the class we can least afford. Channel order is the simplest possible example there is: one swap, two conventions. And the field has still not managed to stop writing it.

The fix is not more care. Care doesn’t scale. The fix is to write the meaning into the type, so that every mismatch (human-written or agent-written) dies at compile time, loudly, with a line number.

Next up: a bug that is subtler than a blue face, hides in nearly every resize call ever written, and makes your images physically wrong in a way most people have stared at for years without seeing. It involves the number 128, which turns out not to be the middle of anything.