media

README

go-media

This module provides an interface for media services, including:

• Bindings in golang for FFmpeg 7.1;
• Opening media files, devices and network sockets for reading and writing;
• Retrieving metadata and artwork from audio and video media;
• Re-multiplexing media files from one format to another;
• Fingerprinting audio files to identify music.

Current Status

This module is currently in development and subject to change. If there are any specific features you are interested in, please see below "Contributing & Distribution" below.

Requirements

If you're building for docker, then you can simply run the following command. This creates a docker image with all the dependencies installed.

DOCKER_REGISTRY=docker.io/user make docker

However, it's more likely that you want to build the bindings. To do so, compile the FFmpeg libraries first:

# Debian/Ubuntu
apt install libfreetype-dev libmp3lame-dev libopus-dev libvorbis-dev libvpx-dev libx264-dev libx265-dev libnuma-dev
git clone github.com/mutablelogic/go-media
cd go-media
make ffmpeg

# Fedora
dnf install freetype-devel lame-devel opus-devel libvorbis-devel libvpx-devel x264-devel x265-devel numactl-devel
git clone github.com/mutablelogic/go-media
cd go-media
make ffmpeg

# Homebrew
brew install freetype lame opus libvorbis libvpx x264 x265
git clone github.com/mutablelogic/go-media
cd go-media
make ffmpeg

This will place the static libraries in the build/install folder which you can refer to when compiling your golang code. For example, here's a typical compile or run command on a Mac:

PKG_CONFIG_PATH="${PWD}/build/install/lib/pkgconfig" \
  LD_LIBRARY_PATH="/opt/homebrew/lib" \
  CGO_LDFLAGS_ALLOW="-(W|D).*" \
  go build -o build/media ./cmd/media

Demultiplexing

package main

import (
  media "github.com/mutablelogic/go-media"
)

func main() {
  manager, err := media.NewManager()
  if err != nil {
    log.Fatal(err)
  }

  // Open a media file for reading. The format of the file is guessed.
  // Alteratively, you can pass a format as the second argument. Further optional
  // arguments can be used to set the format options.
  file, err := manager.Open(os.Args[1], nil)
  if err != nil {
    log.Fatal(err)
  }
  defer file.Close()

  // Choose which streams to demultiplex - pass the stream parameters
  // to the decoder. If you don't want to resample or reformat the streams,
  // then you can pass nil as the function and all streams will be demultiplexed.
  decoder, err := file.Decoder(func (stream media.Stream) (media.Parameters, error) {
    return stream.Parameters(), nil
  }
  if err != nil {
    log.Fatal(err)
  }

  // Demuliplex the stream and receive the packets. If you don't want to
  // process the packets yourself, then you can pass nil as the function
  if err := decoder.Demux(context.Background(), func(_ media.Packet) error {
    // Each packet is specific to a stream. It can be processed here
    // to receive audio or video frames, then resize or resample them,
    // for example. Alternatively, you can pass the packet to an encoder
    // to remultiplex the streams without processing them.
    return nil
  }); err != nil {
    log.Fatal(err)  
  })
}

Decoding - Video

This example shows you how to decode video frames from a media file into images, and encode those images to JPEG format.

package main

import (
  "context"
  "fmt"
  "image/jpeg"
  "io"
  "log"
  "os"
  "path/filepath"

  // Packages
  ffmpeg "github.com/mutablelogic/go-media/pkg/ffmpeg"

  // Namespace imports
  . "github.com/mutablelogic/go-media"
)

func main() {
  // Open a media file for reading. The format of the file is guessed.
  input, err := ffmpeg.Open(os.Args[1])
  if err != nil {
    log.Fatal(err)
  }

  // Make a map function which can be used to decode the streams and set
  // the parameters we want from the decode. The audio and video streams
  // are resampled and resized to fit the parameters we pass back the decoder.
  mapfunc := func(stream int, par *ffmpeg.Par) (*ffmpeg.Par, error) {
    if par.Type() == VIDEO {
      // Convert frame to yuv420p to frame size and rate as source
      return ffmpeg.VideoPar("yuv420p", par.WidthHeight(), par.FrameRate()), nil
    }
    // Ignore other streams
    return nil, nil
  }

  // Make a folder where we're going to store the thumbnails
  tmp, err := os.MkdirTemp("", "decode")
  if err != nil {
    log.Fatal(err)
  }

  // Decode the streams and receive the video frame
  // If the map function is nil, the frames are copied. In this example,
  // we get a yuv420p frame at the same size as the original.
  n := 0
  err = input.Decode(context.Background(), mapfunc, func(stream int, frame *ffmpeg.Frame) error {
    // Write the frame to a file
    w, err := os.Create(filepath.Join(tmp, fmt.Sprintf("frame-%d-%d.jpg", stream, n)))
    if err != nil {
      return err
    }
    defer w.Close()

    // Convert to an image and encode a JPEG
    if image, err := frame.Image(); err != nil {
      return err
    } else if err := jpeg.Encode(w, image, nil); err != nil {
      return err
    } else {
      log.Println("Wrote:", w.Name())
    }

    // End after 10 frames
    n++
    if n >= 10 {
      return io.EOF
    }
    return nil
  })
  if err != nil {
    log.Fatal(err)
  }
}

Encoding - Audio and Video

This example shows you how to encode video and audio frames into a media file. It creates a testcard signal overlayed with a timestamp, and a 1KHz tone at -5dB

package main

import (
  "fmt"
  "io"
  "log"
  "os"

  // Packages
  ffmpeg "github.com/mutablelogic/go-media/pkg/ffmpeg"
  generator "github.com/mutablelogic/go-media/pkg/generator"
)

// This example encodes an audio an video stream to a file
func main() {
  // Create a new file with an audio and video stream
  file, err := ffmpeg.Create(os.Args[1],
    ffmpeg.OptStream(1, ffmpeg.VideoPar("yuv420p", "1024x720", 30)),
    ffmpeg.OptStream(2, ffmpeg.AudioPar("fltp", "mono", 22050)),
  )
  if err != nil {
    log.Fatal(err)
  }
  defer file.Close()

  // Make an video generator which can generate frames with the same
  // parameters as the video stream
  video, err := generator.NewEBU(file.Stream(1).Par())
  if err != nil {
    log.Fatal(err)
  }
  defer video.Close()

  // Make an audio generator which can generate a 1KHz tone
  // at -5dB with the same parameters as the audio stream
  audio, err := generator.NewSine(1000, -5, file.Stream(2).Par())
  if err != nil {
    log.Fatal(err)
  }
  defer audio.Close()

  // Write 90 seconds, passing video and audio frames to the encoder
  // and returning io.EOF when the duration is reached
  duration := float64(90)
  err = file.Encode(func(stream int) (*ffmpeg.Frame, error) {
    var frame *ffmpeg.Frame
    switch stream {
    case 1:
      frame = video.Frame()
    case 2:
      frame = audio.Frame()
    }
    if frame != nil && frame.Ts() < duration {
      return frame, nil
    }
    return nil, io.EOF
  }, nil)
  if err != nil {
    log.Fatal(err)
  }
}

Multiplexing

TODO

Retrieving Metadata and Artwork from a media file

Here is an example of opening a media file and retrieving metadata and artwork. You have to read the artwork separately from the metadata.

package main

import (
  "log"
  "os"

  // Packages
  ffmpeg "github.com/mutablelogic/go-media/pkg/ffmpeg"
)

func main() {
  // Open a media file for reading. The format of the file is guessed.
  reader, err := ffmpeg.Open(os.Args[1])
  if err != nil {
    log.Fatal(err)
  }
  defer reader.Close()

  // Retrieve all the metadata from the file, and display it. If you pass
  // keys to the Metadata function, then only entries with those keys will be
  // returned.
  for _, metadata := range reader.Metadata() {
    log.Print(metadata.Key(), " => ", metadata.Value())
  }

  // Retrieve artwork by using the MetaArtwork key. The value is of type []byte.
  // which needs to be converted to an image. 
  for _, artwork := range reader.Metadata(ffmpeg.MetaArtwork) {
    mimetype := artwork.Value()
    if mimetype != "" {
      // Retrieve the data using the metadata.Bytes() method
      log.Print("We got some artwork of mimetype ", mimetype)
    }
  }
}

Audio Fingerprinting

You can programmatically fingerprint audio files, compare fingerprints and identify music using the following packages:

• sys/chromaprint provides the implementation of the lower-level function calls to chromaprint. The documentation is here
• pkg/chromaprint provides the higher-level API for fingerprinting and identifying music. The documentation is here.

You'll need an API key in order to use the AcoustID service. You can get a key here.

Contributing & Distribution

This module is currently in development and subject to change.

Please do file feature requests and bugs here. The license is Apache 2 so feel free to redistribute. Redistributions in either source code or binary form must reproduce the copyright notice, and please link back to this repository for more information:

go-media
https://github.com/mutablelogic/go-media/
Copyright (c) 2021-2025 David Thorpe, All rights reserved.

This software links to shared libraries of FFmpeg licensed under the LGPLv2.1.

References

• https://ffmpeg.org/doxygen/7.0/index.html
• https://pkg.go.dev/github.com/mutablelogic/go-media

Documentation

Overview

This is a package for reading, writing and inspecting media files. In order to operate on media, call NewManager() and then use the manager functions to determine capabilities and manage media files and devices.

This is a package for reading, writing and inspecting media files. In order to operate on media, call NewManager() and then use the manager functions to determine capabilities and manage media files and devices.

Index

• type Err
  • func (code Err) Code() uint
  • func (code Err) Error() string
  • func (code Err) With(args ...interface{}) error
  • func (code Err) Withf(format string, args ...interface{}) error
• type Format
• type Frame
• type FrameFunc
• type Manager
• type MapFunc
• type Media
• type Metadata
• type Par
• type Type
  • func (t Type) FlagString() string
  • func (t Type) Is(u Type) bool
  • func (t Type) MarshalJSON() ([]byte, error)
  • func (t Type) String() string

Types

type Err

type Err uint

const (
	ErrBadParameter   Err = http.StatusBadRequest
	ErrInternalError  Err = http.StatusInternalServerError
	ErrNotImplemented Err = http.StatusNotImplemented
)

func (Err) Code

func (code Err) Code() uint

func (Err) Error

func (code Err) Error() string

func (Err) With

func (code Err) With(args ...interface{}) error

func (Err) Withf

func (code Err) Withf(format string, args ...interface{}) error

type Format

type Format interface {
	// The type of the format, which can be combinations of
	// INPUT, OUTPUT, DEVICE, AUDIO, VIDEO and SUBTITLE
	Type() Type

	// The unique name that the format can be referenced as
	Name() string

	// Description of the format
	Description() string
}

A container format for a media file

type Frame

type Frame interface{}

A frame of decoded data

type FrameFunc

type FrameFunc func(int, Frame) error

FrameFunc is a function which is called to send a frame after decoding. It should return nil to continue decoding or io.EOF to stop.

type Manager

type Manager interface {
	// Open a media file or device for reading, from a path or url.
	// If a format is specified, then the format will be used to open
	// the file. Close the media object when done.
	Open(string, Format, ...string) (Media, error)

	// Open a media stream for reading.  If a format is
	// specified, then the format will be used to open the file. Close the
	// media object when done. It is the responsibility of the caller to
	// also close the reader when done.
	Read(io.Reader, Format, ...string) (Media, error)

	// Create a media file or device for writing, from a path. If a format is
	// specified, then the format will be used to create the file or else
	// the format is guessed from the path. If no parameters are provided,
	// then the default parameters for the format are used.
	Create(string, Format, []Metadata, ...Par) (Media, error)

	// Return supported input and output container formats which match any filter,
	// which can be a name, extension (with preceeding period) or mimetype. The Type
	// can be a combination of DEVICE, INPUT, OUTPUT or ANY to select the right kind of
	// format
	Formats(Type, ...string) []Format

	// Return all supported sample formats
	SampleFormats() []Metadata

	// Return all supported pixel formats
	PixelFormats() []Metadata

	// Return standard channel layouts which can be used for audio,
	// with the number of channels provided. If no channels are provided,
	// then all standard channel layouts are returned.
	ChannelLayouts() []Metadata

	// Return all supported codecs, of a specific type or all
	// if ANY is used. If any names is provided, then only the codecs
	// with those names are returned.
	Codecs(Type, ...string) []Metadata

	// Log error messages with arguments
	Errorf(string, ...any)

	// Log warning messages with arguments
	Warningf(string, ...any)

	// Log info messages  with arguments
	Infof(string, ...any)

	// Decode an input stream, determining the streams to be decoded
	// and the function to accept the decoded frames. If MapFunc is nil,
	// all streams are passed through (demultiplexing).
	Decode(context.Context, Media, MapFunc, FrameFunc) error
}

Manager represents a manager for media formats and devices. Create a new manager object using the NewManager function.

	  import "github.com/mutablelogic/go-media/pkg/ffmpeg"

	  manager, err := ffmpeg.NewManager()
	  if err != nil {
		...
      }

Various options are available to control the manager, for logging and affecting decoding, that can be applied when creating the manager by passing them as arguments.

Only one manager can be created. If NewManager is called a second time, the previously created manager is returned, but any new options are applied.

type MapFunc

type MapFunc func(int, Par) (Par, error)

MapFunc return parameters if a stream should be decoded, resampled (for audio streams) or resized (for video streams). Return nil if you want to ignore the stream, or pass back the stream parameters if you want to copy the stream without any changes.

type Media

type Media interface {
	io.Closer

	// The type of the format, which can be combinations of
	// INPUT or OUTPUT
	Type() Type
}

A container format for a media file, reader, device or network stream

type Metadata

type Metadata interface {
	// Return the metadata key
	Key() string

	// Return the value as a string. Returns the mimetype
	// if the value is a byte slice, and the mimetype can be
	// detected.
	Value() string

	// Returns the value as a byte slice
	Bytes() []byte

	// Returns the value as an image
	Image() image.Image

	// Returns the value as an interface
	Any() any
}

Metadata is a key/value pair which can be used to describe a media object or other metadata. The value can be retrieved as a string value, data, or other type. If the value is a byte slice, then it can also be retrieved as an image (for artwork)

type Par

type Par interface{}

Parameters for a stream or frame

type Type

type Type int

Type of codec, device, format or stream

const (
	NONE     Type = 0           // Type is not defined
	VIDEO    Type = (1 << iota) // Type is video
	AUDIO                       // Type is audio
	SUBTITLE                    // Type is subtitle
	DATA                        // Type is data
	UNKNOWN                     // Type is unknown
	INPUT                       // Type is input format
	OUTPUT                      // Type is output format
	DEVICE                      // Type is input or output device

	ANY = NONE // Type is any (used for filtering)
)

func (Type) FlagString

func (t Type) FlagString() string

Return a flag as a string

func (Type) Is

func (t Type) Is(u Type) bool

Returns true if the type matches a set of flags

func (Type) MarshalJSON

func (t Type) MarshalJSON() ([]byte, error)

Return the type as a string

func (Type) String

func (t Type) String() string

Return the type as a string