goshl

package module
v0.0.11 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 20, 2026 License: GPL-3.0 Imports: 16 Imported by: 0

README

Goshl

Pronounced "goshul"

An HLS transcoding library in Go.

What it does

goshl takes a video file and serves it as an HLS stream. It:

  • Probes the source to get resolution, duration, audio tracks, and subtitles
  • Generates master and variant playlists
  • Transcodes segments on demand (not upfront)
  • Caches segments after transcoding
  • Generates multiple quality levels based on source resolution
  • Extracts thumbnail sprites for seek previews
  • Extracts subtitles to WebVTT

Segments are only transcoded when a client requests them. A 2-hour video doesn't need to finish transcoding before playback can start.

Install

go get github.com/eleven-am/goshl

Requires ffmpeg and ffprobe in PATH.

Quick start

controller := goshl.NewController(goshl.Options{
    Storage:     myStorage,
    Coordinator: myCoordinator,
    PathGen:     myPathGen,
})

if err := controller.Start(ctx); err != nil {
    log.Fatal(err)
}
defer controller.Stop()

Interfaces

You need to implement three interfaces:

Storage

Handles persistence of segments, metadata, sprites, and subtitles.

type Storage interface {
    MetadataExists(ctx context.Context, sourceURL string) (bool, error)
    GetMetadata(ctx context.Context, sourceURL string) ([]byte, error)
    SetMetadata(ctx context.Context, sourceURL string, data []byte) error

    WriteSegment(ctx context.Context, info SegmentData, data []byte) error
    ReadSegment(ctx context.Context, info SegmentData) ([]byte, error)
    SegmentExists(ctx context.Context, info SegmentData) (bool, error)

    WriteSprite(ctx context.Context, sourceURL string, index int, data []byte) error
    ReadSprite(ctx context.Context, sourceURL string, index int) ([]byte, error)
    SpriteExists(ctx context.Context, sourceURL string, index int) (bool, error)

    WriteSpriteVTT(ctx context.Context, sourceURL string, data []byte) error
    ReadSpriteVTT(ctx context.Context, sourceURL string) ([]byte, error)
    SpriteVTTExists(ctx context.Context, sourceURL string) (bool, error)

    WriteSubtitleVTT(ctx context.Context, sourceURL string, lang string, data []byte) error
    ReadSubtitleVTT(ctx context.Context, sourceURL string, lang string) ([]byte, error)
    SubtitleVTTExists(ctx context.Context, sourceURL string, lang string) (bool, error)

    WritePreview(ctx context.Context, sourceURL string, data []byte) error
    ReadPreview(ctx context.Context, sourceURL string) ([]byte, error)
    PreviewExists(ctx context.Context, sourceURL string) (bool, error)

    WriteThumbnail(ctx context.Context, sourceURL string, data []byte) error
    ReadThumbnail(ctx context.Context, sourceURL string) ([]byte, error)
    ThumbnailExists(ctx context.Context, sourceURL string) (bool, error)
}
Coordinator

Manages job distribution and segment notifications. For single-instance deployments, an in-memory implementation works. For distributed setups, use something like Redis.

type Coordinator interface {
    Enqueue(ctx context.Context, job Job) error
    Subscribe(ctx context.Context, streamType StreamType) (<-chan Job, error)
    Ack(ctx context.Context, jobID string) error

    NotifySegment(ctx context.Context, info SegmentData, status SegmentStatus) error
    WaitSegment(ctx context.Context, info SegmentData) (<-chan SegmentStatus, error)

    AcquireLock(ctx context.Context, key string, ttl time.Duration) (token string, err error)
    ReleaseLock(ctx context.Context, key string, token string) error

    Close()
}
PathGenerator

Generates URLs that get embedded in playlists. These URLs should route back to your HTTP handlers.

type PathGenerator interface {
    MasterPlaylist(sourceURL string) string
    VariantPlaylist(sourceURL string, rendition string, streamType StreamType) string
    Segment(sourceURL string, rendition string, streamType StreamType, index int) string
    SpriteVTT(sourceURL string) string
    Sprite(sourceURL string, index int) string
    SubtitleVTT(sourceURL string, lang string) string
    Preview(sourceURL string) string
    Thumbnail(sourceURL string) string
}

Controller methods

// Returns master playlist with available renditions
playlist, err := controller.MasterPlaylist(ctx, "file:///path/to/video.mp4")

// Returns variant playlist for a specific rendition
playlist, err := controller.VariantPlaylist(ctx, sourceURL, goshl.StreamVideo, "720p")

// Returns segment data (transcodes on first request, cached after)
data, err := controller.Segment(ctx, sourceURL, goshl.StreamVideo, "720p", 0)

// Returns WebVTT file for thumbnail sprites
vtt, err := controller.SpriteVTT(ctx, sourceURL)

// Returns sprite sheet image
sprite, err := controller.Sprite(ctx, sourceURL, 0)

// Returns subtitles in WebVTT format
subs, err := controller.SubtitleVTT(ctx, sourceURL, "en")

Options

goshl.Options{
    Storage:        myStorage,          // required
    Coordinator:    myCoordinator,      // required
    PathGen:        myPathGen,          // required

    HWAccel:        false,              // auto-detect GPU encoding if available
    HWAccelMode:    goshl.AccelNone,    // explicitly choose none, cuda, videotoolbox, vaapi, or qsv
    SegmentTimeout: 30 * time.Second,   // max wait for segment transcoding
    TargetDuration: 6.0,                // target segment duration in seconds
    SegmentsPerJob: 4,                  // segments per transcoding job
    VideoPoolSize:  2,                  // video transcoding workers
    AudioPoolSize:  4,                  // audio transcoding workers
}

Hardware acceleration

Set HWAccel: true to auto-detect GPU encoding. Supported accelerators are NVIDIA CUDA/NVENC, Apple VideoToolbox, VAAPI, and Intel QSV. If detection fails, goshl uses software encoding and exposes the detection error through controller.HWAccelConfig().

Set HWAccelMode when the caller already knows the desired accelerator:

controller := goshl.NewController(goshl.Options{
    Storage:     myStorage,
    Coordinator: myCoordinator,
    PathGen:     myPathGen,
    HWAccelMode: goshl.AccelCUDA,
})

Sprite generation also uses hardware decode flags for GPU modes where frames can be downloaded back into the CPU filter chain safely.

cfg := controller.HWAccelConfig()
log.Printf("accelerator=%s requested=%s available=%v error=%s", cfg.Accelerator, cfg.Requested, cfg.Available, cfg.Error)

Author

Roy Ossai

License

GPL-3.0

Documentation

Overview

Package goshl provides an HLS transcoding service for on-demand video streaming.

goshl handles the complete HLS workflow including media probing, adaptive bitrate transcoding, playlist generation, and segment delivery. It supports hardware acceleration (NVIDIA NVENC, Apple VideoToolbox) and provides a distributed architecture through pluggable Storage and Coordinator interfaces.

Architecture

The library is built around three core interfaces that must be implemented:

  • Storage: Persists metadata, segments, sprites, and subtitles
  • Coordinator: Manages job queues and segment readiness notifications
  • PathGenerator: Generates URLs for playlists, segments, and assets

Basic Usage 

controller := goshl.NewController(goshl.Options{
    Storage:     myStorage,
    Coordinator: myCoordinator,
    PathGen:     myPathGenerator,
    HWAccel:     true, // Enable hardware acceleration
})

// Start the transcoding worker pools
if err := controller.Start(ctx); err != nil {
    log.Fatal(err)
}
defer controller.Stop()

// Generate master playlist for a media file
playlist, err := controller.MasterPlaylist(ctx, "file:///path/to/video.mp4")

HLS Workflow

When a client requests content:

  1. MasterPlaylist probes the source and returns available renditions
  2. Client selects a variant and requests VariantPlaylist
  3. Client requests individual Segments, which triggers transcoding if needed
  4. Transcoded segments are cached in Storage for subsequent requests

Segment-on-Demand

Segments are transcoded lazily when first requested. The Segment method:

  • Returns immediately if the segment exists in storage
  • Otherwise, enqueues a transcoding job and waits for completion
  • Jobs transcode multiple segments at once (configurable via SegmentsPerJob)

Index

Constants

View Source 
const (
	// StreamVideo represents a video stream.
	StreamVideo = domain.StreamVideo

	// StreamAudio represents an audio stream.
	StreamAudio = domain.StreamAudio

	// AccelNone disables hardware acceleration and uses software encoding.
	AccelNone = domain.AccelNone

	// AccelCUDA selects NVIDIA CUDA/NVENC acceleration.
	AccelCUDA = domain.AccelCUDA

	// AccelVideoToolbox selects Apple VideoToolbox acceleration.
	AccelVideoToolbox = domain.AccelVideoToolbox

	// AccelVAAPI selects VAAPI acceleration.
	AccelVAAPI = domain.AccelVAAPI

	// AccelQSV selects Intel Quick Sync Video acceleration.
	AccelQSV = domain.AccelQSV
)

Variables

View Source 
var (
	// ErrLockHeld is returned by Coordinator.AcquireLock when the lock is held by another instance.
	ErrLockHeld = domain.ErrLockHeld
)

Functions

This section is empty.

Types

type Accelerator added in v0.0.10 

type Accelerator = domain.Accelerator

Accelerator identifies a hardware acceleration backend.

type AudioStream added in v0.0.6 

type AudioStream = domain.AudioStream

AudioStream contains audio stream information.

type Controller 

type Controller struct {
	// contains filtered or unexported fields
}

Controller is the main entry point for HLS transcoding operations. It coordinates media probing, playlist generation, and on-demand transcoding.

A Controller must be started with Start before processing requests, and stopped with Stop when shutting down to ensure clean worker termination.

func NewController 

func NewController(opts Options) *Controller

NewController creates a new Controller with the given options. It panics if required options (Storage, Coordinator, PathGen) are nil.

The controller is not started automatically; call Start to begin processing transcoding jobs.

func (*Controller) HWAccelConfig added in v0.0.10 

func (c *Controller) HWAccelConfig() domain.HWAccelConfig

HWAccelConfig returns the hardware acceleration configuration selected by the controller.

func (*Controller) MasterPlaylist 

func (c *Controller) MasterPlaylist(ctx context.Context, sourceURL string) (string, error)

MasterPlaylist returns the HLS master playlist for a media source.

The playlist advertises all available video renditions (based on source resolution and bitrate) and audio tracks. On first call for a source, it probes the media file using ffprobe and caches the metadata.

The returned string is a complete M3U8 playlist ready to serve to clients.

func (*Controller) Metadata added in v0.0.6 

func (c *Controller) Metadata(ctx context.Context, sourceURL string) (*Metadata, error)

func (*Controller) Preview added in v0.0.4 

func (c *Controller) Preview(ctx context.Context, sourceURL string, config *PreviewConfig) ([]byte, error)

Preview returns an animated GIF preview of the video.

The preview is generated by sampling frames from sprite sheets and combining them into an animated GIF. This provides a quick visual overview of the video content suitable for hover previews.

If config is nil, DefaultPreviewConfig is used. The generated preview is cached in storage for subsequent requests.

Returns GIF image data.

func (*Controller) Segment 

func (c *Controller) Segment(ctx context.Context, sourceURL string, streamType StreamType, renditionName string, index int) ([]byte, error)

Segment returns a transcoded media segment.

If the segment exists in storage, it returns immediately. Otherwise, it:

  1. Acquires a distributed lock to prevent duplicate transcoding
  2. Double-checks segment existence after acquiring lock
  3. Subscribes to segment readiness notifications via Coordinator
  4. Enqueues a transcoding job covering this segment and nearby segments
  5. Waits for the worker to complete transcoding (up to SegmentTimeout)
  6. Returns the transcoded segment data

Parameters:

  • sourceURL: The media source URL
  • streamType: Either StreamVideo or StreamAudio
  • renditionName: The rendition identifier (e.g., "1080p", "aac_stereo")
  • index: Zero-based segment index

Returns the raw MPEG-TS segment data, or an error if transcoding fails or times out.

func (*Controller) Sprite 

func (c *Controller) Sprite(ctx context.Context, sourceURL string, index int) ([]byte, error)

Sprite returns a sprite sheet image containing multiple video thumbnails.

Each sprite sheet contains a grid of thumbnails extracted from the video. The index parameter selects which sprite sheet to return (videos are divided into multiple sheets based on duration).

Returns JPEG image data.

func (*Controller) SpriteVTT 

func (c *Controller) SpriteVTT(ctx context.Context, sourceURL string) ([]byte, error)

SpriteVTT returns a WebVTT file mapping timestamps to thumbnail sprite images.

The VTT file references sprite sheet images (containing multiple thumbnails) with spatial coordinates for each timestamp. This enables video preview thumbnails during seek operations.

Sprite sheets and VTT data are generated on first request and cached.

func (*Controller) Start 

func (c *Controller) Start(ctx context.Context) error

Start initializes the video and audio transcoding worker pools. It subscribes to the Coordinator for incoming jobs and begins processing.

Start must be called before any transcoding can occur. The provided context controls the lifetime of background workers; canceling it triggers shutdown.

Returns an error if the worker pools fail to subscribe to the Coordinator.

func (*Controller) Stop 

func (c *Controller) Stop()

Stop gracefully shuts down the transcoding worker pools. It waits for any in-progress transcoding jobs to complete before returning. Always call Stop when shutting down to prevent resource leaks.

func (*Controller) SubtitleVTT 

func (c *Controller) SubtitleVTT(ctx context.Context, sourceURL string, lang string) ([]byte, error)

SubtitleVTT extracts and returns subtitles in WebVTT format.

The lang parameter specifies the subtitle track language code (e.g., "en", "es"). If the requested language is not found in the source media, an error is returned.

Subtitles are extracted from the source on first request and cached.

func (*Controller) Thumbnail added in v0.0.5 

func (c *Controller) Thumbnail(ctx context.Context, sourceURL string, config *ThumbnailConfig) ([]byte, error)

func (*Controller) VariantPlaylist 

func (c *Controller) VariantPlaylist(ctx context.Context, sourceURL string, streamType StreamType, renditionName string) (string, error)

VariantPlaylist returns the HLS media playlist for a specific rendition.

Parameters:

  • sourceURL: The media source URL
  • streamType: Either StreamVideo or StreamAudio
  • renditionName: The rendition identifier (e.g., "1080p", "720p", "aac_stereo")

The playlist contains segment references with durations calculated from the source keyframe positions. Segment URLs are generated via PathGenerator.

type Coordinator 

type Coordinator = domain.Coordinator

Coordinator manages the distributed transcoding workflow. It handles job queuing across worker pools and notifies waiting clients when segments become available. For single-instance deployments, an in-memory implementation suffices. For distributed systems, consider Redis or a message queue.

type Job added in v0.0.2 

type Job = domain.Job

Job represents a transcoding task for a range of segments.

type Metadata added in v0.0.6 

type Metadata = domain.Metadata

Metadata contains probed information about a media source.

type Options 

type Options struct {
	// Storage is required. Handles persistence of metadata, segments, and assets.
	Storage Storage

	// Coordinator is required. Manages job distribution and segment notifications.
	Coordinator Coordinator

	// PathGen is required. Generates URLs for playlists and segments.
	PathGen PathGenerator

	// HWAccel enables hardware-accelerated encoding when available.
	// Automatically detects NVENC (NVIDIA) or VideoToolbox (Apple).
	// Falls back to software encoding if no hardware support is found.
	HWAccel bool

	// HWAccelMode explicitly selects a hardware accelerator.
	// When set, it takes precedence over HWAccel auto-detection.
	HWAccelMode Accelerator

	// SegmentTimeout is the maximum time to wait for a segment to be transcoded.
	// Default: 30 seconds.
	SegmentTimeout time.Duration

	// TargetDuration is the target HLS segment duration in seconds.
	// Actual duration varies based on keyframe positions.
	// Default: 6.0 seconds.
	TargetDuration float64

	// SegmentsPerJob is the number of segments transcoded per job.
	// Higher values improve throughput but increase latency for first segment.
	// Default: 4.
	SegmentsPerJob int

	// VideoPoolSize is the number of concurrent video transcoding workers.
	// Default: 2.
	VideoPoolSize int

	// AudioPoolSize is the number of concurrent audio transcoding workers.
	// Default: 4.
	AudioPoolSize int

	// DefaultPreviewConfig sets default parameters for animated preview generation.
	// Defaults: MaxFrames=40, Frameduration=250ms, Width=480.
	DefaultPreviewConfig PreviewConfig

	// DefaultThumbnailConfig sets default parameters for thumbnail generation.
	// Defaults: Position=0.1 (10% into video), Width=480.
	DefaultThumbnailConfig ThumbnailConfig
}

Options configures the Controller behavior and dependencies.

type PathGenerator 

type PathGenerator = domain.PathGenerator

PathGenerator creates URLs for HLS resources. These URLs are embedded in playlists and must be routable back to the appropriate Controller methods.

type PreviewConfig added in v0.0.4 

type PreviewConfig = domain.PreviewConfig

PreviewConfig configures animated preview generation parameters.

type SegmentData added in v0.0.2 

type SegmentData = domain.SegmentData

SegmentData encapsulates information about a specific media segment.

type SegmentStatus added in v0.0.3 

type SegmentStatus = domain.SegmentStatus

SegmentStatus represents the processing state of a segment.

type Storage 

type Storage = domain.Storage

Storage defines the interface for persisting transcoded content and metadata. Implementations should handle concurrent access and may use any backing store (filesystem, S3, Redis, etc.).

type StreamType 

type StreamType = domain.StreamType

StreamType identifies the type of media stream (video or audio).

type SubtitleStream added in v0.0.6 

type SubtitleStream = domain.SubtitleStream

SubtitleStream contains subtitle stream information.

type ThumbnailConfig added in v0.0.5 

type ThumbnailConfig = domain.ThumbnailConfig

ThumbnailConfig configures thumbnail generation parameters.

type VideoStream added in v0.0.6 

type VideoStream = domain.VideoStream

VideoStream contains video stream information.

Source Files

View all Source files

Directories

Path Synopsis
internal
domain
ffmpeg
hwaccel
mediaexec
misc
playlist
probe
rendition
segment
transcode

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.