imageio

Documentation

Overview

Package imageio decodes images and puts them in the shape a vision encoder reads: planar RGB, one float per channel per pixel.

It knows nothing about any model. What size to resize to is a question the model answers; this package answers how.

Index

• type Image
  • func Decode(r io.Reader) (*Image, error)
  • func (im *Image) PadInto(w, h int, colour [3]uint8) *Image
  • func (im *Image) PlanarRGB(dst []float32)
  • func (im *Image) ResizeBilinear(w, h int) *Image

Types

type Image

type Image struct {
	W, H int
	Pix  []uint8 // W*H*3
}

Image is eight-bit RGB, interleaved, no alpha and no stride: a decoder's output flattened to the one layout everything below wants.

func Decode

func Decode(r io.Reader) (*Image, error)

Decode reads PNG, JPEG, GIF or WebP. Anything with transparency is composited on white, because a model shown a checkerboard of alpha would describe the checkerboard.

func (*Image) PadInto

func (im *Image) PadInto(w, h int, colour [3]uint8) *Image

PadInto centres this image on a canvas of w by h filled with one colour, and is what a resize that preserves the aspect ratio exactly needs: the shape is kept and the difference becomes a border. The offsets round down, as the reference's integer division does.

func (*Image) PlanarRGB

func (im *Image) PlanarRGB(dst []float32)

PlanarRGB writes the whole red plane, then the green, then the blue, scaled to 0..1. That is the layout ggml's convolution reads.

func (*Image) ResizeBilinear

func (im *Image) ResizeBilinear(w, h int) *Image

ResizeBilinear is the resampling clip.cpp performs, and is written to agree with it rather than to be the best resampling available.

Two details are the whole point. The grid is corner-aligned — output pixel i reads source position i*(src-1)/(dst-1), so the first and last output pixels are exactly the first and last input ones — and the result is truncated rather than rounded, because the reference casts a float to a byte. Neither is what a graphics library would do, and both are what the model was fed.