media

package
v1.26.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 22, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Overview

Package media provides unified multi-provider image and video generation.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrNoModel is returned when no model is specified.
	ErrNoModel = errors.New("media: no model specified")

	// ErrProviderNotFound is returned when no provider matches the model name.
	ErrProviderNotFound = errors.New("media: no provider found for model")

	// ErrEditNotSupported is returned when a provider does not support editing.
	ErrEditNotSupported = errors.New("media: provider does not support image editing")

	// ErrTimeout is returned when generation exceeds the timeout.
	ErrTimeout = errors.New("media: generation timed out")

	// ErrNoResult is returned when a provider returns no results.
	ErrNoResult = errors.New("media: provider returned no results")
)

Functions

func AudioExtensionFromMIME added in v1.7.0 

func AudioExtensionFromMIME(mime string) string

AudioExtensionFromMIME returns a suitable file extension for a MIME type.

func ConvertImage added in v1.5.0 

func ConvertImage(data []byte, target Format) ([]byte, error)

ConvertImage re-encodes image data to the target format. Supports PNG and JPEG targets. Returns an error for WebP target (no Go stdlib encoder). If source format matches target, returns data unchanged.

func DetectAudioMIMEFromBytes added in v1.7.0 

func DetectAudioMIMEFromBytes(data []byte) string

DetectAudioMIMEFromBytes returns the MIME type based on common audio magic bytes. It falls back to audio/mpeg because MP3 is the most common compressed input.

func DetectMIMEFromBytes added in v1.5.0 

func DetectMIMEFromBytes(data []byte) string

DetectMIMEFromBytes returns the MIME type based on magic bytes.

func PCMToWAV added in v1.7.0 

func PCMToWAV(pcm []byte, sampleRate, channels, bitsPerSample int) []byte

PCMToWAV wraps signed 16-bit little-endian PCM data in a WAV container.

func RegisterImage added in v1.5.0 

func RegisterImage(entry ImageProviderEntry)

RegisterImage adds an image provider entry to the default registry.

func RegisterTextToSpeech added in v1.7.0 

func RegisterTextToSpeech(entry TextToSpeechProviderEntry)

RegisterTextToSpeech adds a text-to-speech provider entry to the default registry.

func RegisterTranscription added in v1.7.0 

func RegisterTranscription(entry TranscriptionProviderEntry)

RegisterTranscription adds a transcription provider entry to the default registry.

func RegisterVideo added in v1.5.0 

func RegisterVideo(entry VideoProviderEntry)

RegisterVideo adds a video provider entry to the default registry.

func SlugifyPrompt added in v1.5.0 

func SlugifyPrompt(prompt string, maxLen int) string

SlugifyPrompt generates a short filename-safe slug from a prompt string.

func StandardImageDimensions added in v1.5.0 

func StandardImageDimensions(ar AspectRatio) (width, height int)

StandardImageDimensions returns default pixel dimensions for an aspect ratio. Returns (1024, 1024) for unrecognized ratios.

func StandardVideoDimensions added in v1.5.0 

func StandardVideoDimensions(ar AspectRatio) (width, height int)

StandardVideoDimensions returns default pixel dimensions for video at the given aspect ratio. Returns (1920, 1080) for unrecognized ratios.

func UniquePath added in v1.5.0 

func UniquePath(path string) string

UniquePath returns path unchanged if it does not exist. If it does exist, a numeric suffix is inserted before the extension (e.g., "photo1.png", "photo2.png") until an available name is found.

func ValidateAudioFormat added in v1.7.0 

func ValidateAudioFormat(f AudioFormat) error

ValidateAudioFormat returns an error if the format is not recognized.

func ValidateFormat added in v1.5.0 

func ValidateFormat(f Format) error

ValidateFormat returns an error if the format is not a recognized value.

Types

type AspectRatio added in v1.5.0 

type AspectRatio string

AspectRatio represents the aspect ratio for generated media. 

const (
	// AspectAuto lets the provider choose a default aspect ratio.
	AspectAuto AspectRatio = ""

	Aspect1x1  AspectRatio = "1:1"
	Aspect16x9 AspectRatio = "16:9"
	Aspect9x16 AspectRatio = "9:16"
	Aspect4x3  AspectRatio = "4:3"
	Aspect3x4  AspectRatio = "3:4"
	Aspect4x1  AspectRatio = "4:1"
	Aspect1x4  AspectRatio = "1:4"
)

func (AspectRatio) String added in v1.5.0 

func (ar AspectRatio) String() string

String returns the string representation of the aspect ratio.

type AudioFormat added in v1.7.0 

type AudioFormat string

AudioFormat represents an audio output format. 

const (
	AudioFormatMP3  AudioFormat = "mp3"
	AudioFormatOpus AudioFormat = "opus"
	AudioFormatAAC  AudioFormat = "aac"
	AudioFormatFLAC AudioFormat = "flac"
	AudioFormatWAV  AudioFormat = "wav"
	AudioFormatPCM  AudioFormat = "pcm"
)

func AudioFormatFromMIME added in v1.7.0 

func AudioFormatFromMIME(mime string) AudioFormat

AudioFormatFromMIME returns the AudioFormat corresponding to a MIME type.

func (AudioFormat) FileExtension added in v1.7.0 

func (f AudioFormat) FileExtension() string

FileExtension returns the file extension (with dot) for the audio format.

func (AudioFormat) MIMEType added in v1.7.0 

func (f AudioFormat) MIMEType() string

MIMEType returns the MIME type string for the audio format.

func (AudioFormat) String added in v1.7.0 

func (f AudioFormat) String() string

String returns the string representation.

type AudioResult added in v1.7.0 

type AudioResult struct {
	// Data is the raw audio bytes.
	Data []byte

	// Model is the model that generated this audio.
	Model string

	// Format is the audio container or codec format.
	Format AudioFormat

	// MimeType is the MIME type of the audio.
	MimeType string

	// Duration is the audio duration when known.
	Duration time.Duration

	// Metadata contains provider-specific metadata.
	Metadata map[string]any
}

AudioResult is the output of a text-to-speech operation.

func TextToSpeech added in v1.7.0 

func TextToSpeech(ctx context.Context, text string, opts ...Option) (*AudioResult, error)

TextToSpeech generates spoken audio from text.

func (*AudioResult) SetAudioFormat added in v1.7.0 

func (r *AudioResult) SetAudioFormat(mimeType string)

SetAudioFormat sets format fields from a MIME type.

func (*AudioResult) WriteTo added in v1.7.0 

func (r *AudioResult) WriteTo(path string) (string, error)

WriteTo writes the audio data to the given file path. If the path has no extension, the format's extension is appended. If the path already exists, a numeric suffix is appended to avoid overwriting.

type Config added in v1.5.0 

type Config struct {
	// Model is the model to use for generation.
	Model string

	// Models is used for fan-out generation across multiple models.
	Models []string

	// AspectRatio controls the output dimensions.
	AspectRatio AspectRatio

	// OutputFormat requests a specific image format (png, jpeg, webp).
	OutputFormat Format

	// AudioFormat requests a specific audio output format.
	AudioFormat AudioFormat

	// AudioMIMEType hints the MIME type of input audio bytes.
	AudioMIMEType string

	// Voice selects the voice for text-to-speech.
	Voice string

	// VoiceInstructions controls voice style for text-to-speech when supported.
	VoiceInstructions string

	// SpeechSpeed controls generated speech speed when supported.
	SpeechSpeed *float64

	// Language sets the transcription or speech language when supported.
	Language string

	// TranscriptionPrompt gives transcription models context.
	TranscriptionPrompt string

	// Count is the number of images to generate per model.
	Count int

	// ReferenceImages are input images for editing operations.
	ReferenceImages [][]byte

	// Duration is the target video duration.
	Duration time.Duration

	// Timeout is the maximum time to wait for generation.
	// Defaults to 5 minutes for images, 15 minutes for video.
	Timeout time.Duration
}

Config holds the resolved options for a media generation call.

func (*Config) Apply added in v1.5.0 

func (c *Config) Apply(opts ...Option)

Apply applies all options and sets defaults for unset fields.

type Format added in v1.5.0 

type Format string

Format represents an image output format. 

const (
	FormatPNG  Format = "png"
	FormatJPEG Format = "jpeg"
	FormatWebP Format = "webp"
)

func DetectFormat added in v1.5.0 

func DetectFormat(data []byte) Format

DetectFormat inspects magic bytes and returns the image format. Returns FormatPNG if the format cannot be determined.

func FormatFromMIME added in v1.5.0 

func FormatFromMIME(mime string) Format

FormatFromMIME returns the Format corresponding to a MIME type string.

func (Format) FileExtension added in v1.5.0 

func (f Format) FileExtension() string

FileExtension returns the file extension (with dot) for the format.

func (Format) MIMEType added in v1.5.0 

func (f Format) MIMEType() string

MIMEType returns the MIME type string for the format.

func (Format) String added in v1.5.0 

func (f Format) String() string

String returns the string representation.

type ImageEditor added in v1.5.0 

type ImageEditor interface {
	// EditImage edits reference images according to the prompt.
	// Reference images are passed via config.ReferenceImages.
	EditImage(ctx context.Context, prompt string, config *Config) ([]*ImageResult, error)
}

ImageEditor edits images using a text prompt and reference images. Providers that support editing implement this in addition to ImageProvider.

type ImageProvider added in v1.5.0 

type ImageProvider interface {
	// GenerateImage generates one or more images from a prompt.
	// The number of images is controlled by config.Count.
	GenerateImage(ctx context.Context, prompt string, config *Config) ([]*ImageResult, error)
}

ImageProvider generates images from text prompts.

type ImageProviderEntry added in v1.5.0 

type ImageProviderEntry struct {
	Name    string
	Match   ModelMatcher
	Factory ImageProviderFactory
}

ImageProviderEntry pairs a matcher with its factory.

type ImageProviderFactory added in v1.5.0 

type ImageProviderFactory func(model string) ImageProvider

ImageProviderFactory creates an ImageProvider for a given model name.

type ImageResult added in v1.5.0 

type ImageResult struct {
	// Data is the raw image bytes.
	Data []byte

	// Model is the model that generated this image.
	Model string

	// Format is the detected image format (png, jpeg, webp).
	Format Format

	// MimeType is the MIME type of the image.
	MimeType string

	// Width is the image width in pixels.
	Width int

	// Height is the image height in pixels.
	Height int

	// Metadata contains provider-specific metadata.
	Metadata map[string]any

	// Err is non-nil if this result represents a provider failure
	// during fan-out generation. Other fields may be empty.
	Err error
}

ImageResult is the output of an image generation or edit operation.

func EditImage added in v1.5.0 

func EditImage(ctx context.Context, prompt string, opts ...Option) (*ImageResult, error)

EditImage edits a reference image using a text prompt. Requires WithReferenceImage(). Returns ErrEditNotSupported if the provider does not implement image editing.

func GenerateImage added in v1.5.0 

func GenerateImage(ctx context.Context, prompt string, opts ...Option) (*ImageResult, error)

GenerateImage generates an image using a single provider. Returns the first result when config.Count is 1, or call with WithCount(n) to generate multiple images.

func GenerateImageBatch added in v1.5.0 

func GenerateImageBatch(ctx context.Context, prompt string, opts ...Option) ([]*ImageResult, error)

GenerateImageBatch generates multiple images using a single provider. Use WithCount(n) to control how many images are generated.

func GenerateImages added in v1.5.0 

func GenerateImages(ctx context.Context, prompt string, opts ...Option) ([]*ImageResult, error)

GenerateImages fans out the same prompt to multiple models concurrently. Requires WithModels(). Returns one result per model. Individual provider failures are captured in ImageResult.Err rather than failing the entire call.

func (*ImageResult) WriteTo added in v1.5.0 

func (r *ImageResult) WriteTo(path string) (string, error)

WriteTo writes the image data to the given file path. If the path has no extension, the format's extension is appended. If the path already exists, a numeric suffix is appended to avoid overwriting (e.g., "photo1.png", "photo2.png").

type ModelMatcher added in v1.5.0 

type ModelMatcher func(model string) bool

ModelMatcher determines if a model name matches a provider.

func PrefixMatcher added in v1.5.0 

func PrefixMatcher(prefix string) ModelMatcher

PrefixMatcher returns a matcher that checks for a case-insensitive prefix.

func PrefixesMatcher added in v1.5.0 

func PrefixesMatcher(prefixes ...string) ModelMatcher

PrefixesMatcher returns a matcher that checks for any of the given prefixes (case-insensitive).

type Option added in v1.5.0 

type Option func(*Config)

Option configures a media generation call.

func WithAspectRatio added in v1.5.0 

func WithAspectRatio(ar AspectRatio) Option

WithAspectRatio sets the aspect ratio.

func WithAudioFormat added in v1.7.0 

func WithAudioFormat(f AudioFormat) Option

WithAudioFormat sets the desired audio output format.

func WithAudioMIMEType added in v1.7.0 

func WithAudioMIMEType(mimeType string) Option

WithAudioMIMEType sets the MIME type for input audio bytes.

func WithCount added in v1.5.0 

func WithCount(n int) Option

WithCount sets the number of images to generate.

func WithDuration added in v1.5.0 

func WithDuration(d time.Duration) Option

WithDuration sets the target video duration.

func WithLanguage added in v1.7.0 

func WithLanguage(language string) Option

WithLanguage sets the language for transcription or text-to-speech.

func WithModel added in v1.5.0 

func WithModel(model string) Option

WithModel sets the model for generation.

func WithModels added in v1.5.0 

func WithModels(models ...string) Option

WithModels sets multiple models for fan-out generation.

func WithOutputFormat added in v1.5.0 

func WithOutputFormat(f Format) Option

WithOutputFormat sets the desired output format.

func WithReferenceImage added in v1.5.0 

func WithReferenceImage(data []byte) Option

WithReferenceImage adds a reference image for editing operations.

func WithSpeechSpeed added in v1.7.0 

func WithSpeechSpeed(speed float64) Option

WithSpeechSpeed sets the generated speech speed when supported.

func WithTimeout added in v1.5.0 

func WithTimeout(d time.Duration) Option

WithTimeout sets the maximum time to wait for generation.

func WithTranscriptionPrompt added in v1.7.0 

func WithTranscriptionPrompt(prompt string) Option

WithTranscriptionPrompt gives transcription models context.

func WithVoice added in v1.7.0 

func WithVoice(voice string) Option

WithVoice sets the voice for text-to-speech.

func WithVoiceInstructions added in v1.7.0 

func WithVoiceInstructions(instructions string) Option

WithVoiceInstructions sets style instructions for text-to-speech.

type Registry added in v1.5.0 

type Registry struct {
	// contains filtered or unexported fields
}

Registry manages model-to-provider mappings for media generation.

func DefaultRegistry added in v1.5.0 

func DefaultRegistry() *Registry

DefaultRegistry returns the default global registry.

func (*Registry) ImageEntries added in v1.5.0 

func (r *Registry) ImageEntries() []ImageProviderEntry

ImageEntries returns a copy of all registered image provider entries.

func (*Registry) RegisterImage added in v1.5.0 

func (r *Registry) RegisterImage(entry ImageProviderEntry)

RegisterImage adds an image provider entry to the registry.

func (*Registry) RegisterTextToSpeech added in v1.7.0 

func (r *Registry) RegisterTextToSpeech(entry TextToSpeechProviderEntry)

RegisterTextToSpeech adds a text-to-speech provider entry to the registry.

func (*Registry) RegisterTranscription added in v1.7.0 

func (r *Registry) RegisterTranscription(entry TranscriptionProviderEntry)

RegisterTranscription adds a transcription provider entry to the registry.

func (*Registry) RegisterVideo added in v1.5.0 

func (r *Registry) RegisterVideo(entry VideoProviderEntry)

RegisterVideo adds a video provider entry to the registry.

func (*Registry) ResolveImage added in v1.5.0 

func (r *Registry) ResolveImage(model string) (ImageProvider, error)

ResolveImage returns an ImageProvider for the given model name.

func (*Registry) ResolveTextToSpeech added in v1.7.0 

func (r *Registry) ResolveTextToSpeech(model string) (TextToSpeechProvider, error)

ResolveTextToSpeech returns a TextToSpeechProvider for the given model name.

func (*Registry) ResolveTranscription added in v1.7.0 

func (r *Registry) ResolveTranscription(model string) (TranscriptionProvider, error)

ResolveTranscription returns a TranscriptionProvider for the given model name.

func (*Registry) ResolveVideo added in v1.5.0 

func (r *Registry) ResolveVideo(model string) (VideoProvider, error)

ResolveVideo returns a VideoProvider for the given model name.

func (*Registry) TextToSpeechEntries added in v1.7.0 

func (r *Registry) TextToSpeechEntries() []TextToSpeechProviderEntry

TextToSpeechEntries returns a copy of all registered text-to-speech provider entries.

func (*Registry) TranscriptionEntries added in v1.7.0 

func (r *Registry) TranscriptionEntries() []TranscriptionProviderEntry

TranscriptionEntries returns a copy of all registered transcription provider entries.

func (*Registry) VideoEntries added in v1.5.0 

func (r *Registry) VideoEntries() []VideoProviderEntry

VideoEntries returns a copy of all registered video provider entries.

type TextToSpeechProvider added in v1.7.0 

type TextToSpeechProvider interface {
	// TextToSpeech generates audio from text.
	TextToSpeech(ctx context.Context, text string, config *Config) (*AudioResult, error)
}

TextToSpeechProvider generates spoken audio from text.

type TextToSpeechProviderEntry added in v1.7.0 

type TextToSpeechProviderEntry struct {
	Name    string
	Match   ModelMatcher
	Factory TextToSpeechProviderFactory
}

TextToSpeechProviderEntry pairs a matcher with its factory.

type TextToSpeechProviderFactory added in v1.7.0 

type TextToSpeechProviderFactory func(model string) TextToSpeechProvider

TextToSpeechProviderFactory creates a TextToSpeechProvider for a given model name.

type TranscriptionProvider added in v1.7.0 

type TranscriptionProvider interface {
	// Transcribe transcribes audio bytes into text.
	Transcribe(ctx context.Context, audio []byte, config *Config) (*TranscriptionResult, error)
}

TranscriptionProvider transcribes speech audio into text.

type TranscriptionProviderEntry added in v1.7.0 

type TranscriptionProviderEntry struct {
	Name    string
	Match   ModelMatcher
	Factory TranscriptionProviderFactory
}

TranscriptionProviderEntry pairs a matcher with its factory.

type TranscriptionProviderFactory added in v1.7.0 

type TranscriptionProviderFactory func(model string) TranscriptionProvider

TranscriptionProviderFactory creates a TranscriptionProvider for a given model name.

type TranscriptionResult added in v1.7.0 

type TranscriptionResult struct {
	// Text is the recognized transcript.
	Text string

	// Model is the model that transcribed the audio.
	Model string

	// Language is the recognized or requested language when known.
	Language string

	// Duration is the input audio duration when known.
	Duration time.Duration

	// Metadata contains provider-specific metadata.
	Metadata map[string]any
}

TranscriptionResult is the output of a transcription operation.

func Transcribe added in v1.7.0 

func Transcribe(ctx context.Context, audio []byte, opts ...Option) (*TranscriptionResult, error)

Transcribe transcribes speech audio bytes into text.

type VideoProvider added in v1.5.0 

type VideoProvider interface {
	// GenerateVideo generates a video from a prompt.
	// The call blocks until generation is complete or ctx is cancelled.
	GenerateVideo(ctx context.Context, prompt string, config *Config) (*VideoResult, error)
}

VideoProvider generates videos from text prompts.

type VideoProviderEntry added in v1.5.0 

type VideoProviderEntry struct {
	Name    string
	Match   ModelMatcher
	Factory VideoProviderFactory
}

VideoProviderEntry pairs a matcher with its factory.

type VideoProviderFactory added in v1.5.0 

type VideoProviderFactory func(model string) VideoProvider

VideoProviderFactory creates a VideoProvider for a given model name.

type VideoResult added in v1.5.0 

type VideoResult struct {
	// Data is the raw video bytes.
	Data []byte

	// Model is the model that generated this video.
	Model string

	// Format is the video container format (mp4, webm).
	Format string

	// MimeType is the MIME type of the video.
	MimeType string

	// Width is the video width in pixels.
	Width int

	// Height is the video height in pixels.
	Height int

	// Duration is the video duration.
	Duration time.Duration

	// AspectRatio is the video aspect ratio.
	AspectRatio AspectRatio

	// Metadata contains provider-specific metadata.
	Metadata map[string]any
}

VideoResult is the output of a video generation operation.

func GenerateVideo added in v1.5.0 

func GenerateVideo(ctx context.Context, prompt string, opts ...Option) (*VideoResult, error)

GenerateVideo generates a video from a text prompt. Blocks until generation is complete or the context is cancelled.

func (*VideoResult) SetVideoFormat added in v1.5.0 

func (r *VideoResult) SetVideoFormat(mimeType string)

SetVideoFormat sets format fields from a MIME type.

func (*VideoResult) WriteTo added in v1.5.0 

func (r *VideoResult) WriteTo(path string) (string, error)

WriteTo writes the video data to the given file path. If the path has no extension, ".mp4" is appended. If the path already exists, a numeric suffix is appended to avoid overwriting (e.g., "clip1.mp4", "clip2.mp4").

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.