omniavatar

README

OmniAvatar

Batteries-included package for real-time AI avatars. Provides provider implementations for HeyGen, Tavus, and bitHuman.

For core interfaces only (no provider dependencies), see omniavatar-core.

Quick Start

import (
    "github.com/plexusone/omniavatar"
    _ "github.com/plexusone/omniavatar/providers/all"
)

func main() {
    provider, err := omniavatar.GetAvatarProvider("heygen",
        omniavatar.WithAPIKey(os.Getenv("HEYGEN_API_KEY")),
        omniavatar.WithExtension("avatar_id", avatarID),
        omniavatar.WithExtension("sandbox", true))
    if err != nil {
        log.Fatal(err)
    }

    session, err := provider.CreateSession(avatar.SessionConfig{
        AudioConfig: avatar.DefaultAudioConfig(),
    })
    if err != nil {
        log.Fatal(err)
    }

    // Start with LiveKit
    err = session.Start(ctx, &omniavatar.LiveKitStartOptions{
        Room:             room,
        AgentIdentity:    "agent-123",
        LiveKitURL:       os.Getenv("LIVEKIT_URL"),
        LiveKitAPIKey:    os.Getenv("LIVEKIT_API_KEY"),
        LiveKitAPISecret: os.Getenv("LIVEKIT_API_SECRET"),
    })
}

Architecture

omniavatar-core/              # Core interfaces (no provider deps)
├── avatar/
│   ├── provider.go           # Provider interface
│   ├── session.go            # Session interface
│   └── audio.go              # AudioDestination interface
└── registry/
    └── registry.go           # Factory types

omniavatar/                   # Provider implementations (this package)
├── registry.go               # Global registry
├── token.go                  # LiveKit token generation
├── start_options.go          # LiveKitStartOptions
└── providers/
    ├── heygen/               # HeyGen LiveAvatar
    ├── tavus/                # Tavus Conversational Video
    ├── bithuman/             # bitHuman Real-time Avatars
    └── all/                  # Convenience import

Provider Registry

Priority System

Providers register with a priority level:

Priority	Constant	Description
0	PriorityThin	Minimal implementations
10	PriorityThick	Full SDK implementations

Higher priority providers override lower priority registrations for the same name.

Auto-Registration

Providers auto-register via init() when imported:

// Import specific provider
import _ "github.com/plexusone/omniavatar/providers/heygen"

// Or import all providers
import _ "github.com/plexusone/omniavatar/providers/all"

Registry Functions

// Get a provider by name
provider, err := omniavatar.GetAvatarProvider("heygen", opts...)

// List all registered providers
names := omniavatar.ListAvatarProviders()

// Check if provider is registered
if omniavatar.HasAvatarProvider("heygen") {
    // ...
}

Supported Providers

HeyGen LiveAvatar

Real-time avatar with lip-sync using HeyGen's LITE mode.

provider, err := omniavatar.GetAvatarProvider("heygen",
    omniavatar.WithAPIKey(os.Getenv("HEYGEN_API_KEY")),
    omniavatar.WithExtension("avatar_id", "josh_lite3_20230714"),
    omniavatar.WithExtension("sandbox", true),           // 60s limit, no credits
    omniavatar.WithExtension("video_quality", "high"),   // very_high, high, medium, low
)

Option	Description
avatar_id	Avatar UUID (required)
sandbox	Enable sandbox mode (recommended for dev)
video_quality	Video quality preset

Tavus Conversational Video

Real-time avatar using Tavus PAL (Personalized AI Likeness).

provider, err := omniavatar.GetAvatarProvider("tavus",
    omniavatar.WithAPIKey(os.Getenv("TAVUS_API_KEY")),
    omniavatar.WithExtension("pal_id", "pal_xxx"),  // Optional
    omniavatar.WithExtension("face_id", "face_xxx"), // Optional
)

Option	Description
pal_id	PAL ID (optional, uses stock avatar if not set)
face_id	Face override (optional)

bitHuman Real-time Avatars

Ultra-low latency avatars using bitHuman.

provider, err := omniavatar.GetAvatarProvider("bithuman",
    omniavatar.WithAPIKey(os.Getenv("BITHUMAN_API_KEY")),
    omniavatar.WithExtension("agent_id", "agent_xxx"),
)

Option	Description
agent_id	bitHuman agent ID (required)

Session Lifecycle

1. Get Provider    → omniavatar.GetAvatarProvider("heygen", opts...)
2. Create Session  → provider.CreateSession(cfg)
3. Start           → session.Start(ctx, &LiveKitStartOptions{...})
4. Wait for Join   → session.WaitForJoin(ctx, 30*time.Second)
5. Stream Audio    → session.AudioOutput().CaptureFrame(ctx, pcm)
6. Close           → session.Close(ctx)

LiveKit Integration

Token Generation

Generate tokens for avatar participants to join LiveKit rooms:

token, err := omniavatar.GenerateAvatarToken(omniavatar.TokenOptions{
    APIKey:        os.Getenv("LIVEKIT_API_KEY"),
    APISecret:     os.Getenv("LIVEKIT_API_SECRET"),
    RoomName:      "my-room",
    Identity:      "avatar-heygen-abc123",
    Provider:      "heygen",
    AgentIdentity: "agent-123",
    TTL:           time.Hour,
})

Start Options

type LiveKitStartOptions struct {
    Room             *lksdk.Room  // LiveKit room reference
    AgentIdentity    string       // Agent's participant identity
    LiveKitURL       string       // LiveKit server URL
    LiveKitAPIKey    string       // API key for token generation
    LiveKitAPISecret string       // API secret for token generation
}

Audio Format

Default audio configuration:

Parameter	Value
Sample Rate	24000 Hz
Channels	1 (mono)
Encoding	PCM16 (linear16)

Provider Comparison

Provider	Latency	Video Quality	Voice Cloning
HeyGen	~500ms	Excellent	Yes
Tavus	~300ms	Excellent	Yes (via PAL)
bitHuman	~200ms	Good	No

Resources

• omniavatar-core - Core interfaces
• HeyGen LiveAvatar
• Tavus
• bitHuman

Documentation

Overview

Package omniavatar provides a unified, provider-agnostic interface for real-time AI avatars.

This is the batteries-included package that imports all providers. For a minimal dependency footprint, use github.com/plexusone/omniavatar-core instead.

Quick Start

import (
    "github.com/plexusone/omniavatar"
    _ "github.com/plexusone/omniavatar/providers/all"
)

func main() {
    provider, err := omniavatar.GetAvatarProvider("heygen",
        omniavatar.WithAPIKey(os.Getenv("HEYGEN_API_KEY")),
        omniavatar.WithExtension("avatar_id", avatarID),
        omniavatar.WithExtension("sandbox", true))
    if err != nil {
        log.Fatal(err)
    }

    session, err := provider.CreateSession(avatar.SessionConfig{
        AudioConfig: avatar.DefaultAudioConfig(),
    })
    if err != nil {
        log.Fatal(err)
    }

    // Use session with LiveKit or other platform adapters
}

Available Providers

• heygen: HeyGen LiveAvatar (LITE mode)
• tavus: Tavus Conversational Video
• bithuman: bitHuman Real-time Avatars

Architecture

omniavatar follows the same pattern as omnivoice:

• omniavatar-core: Core interfaces with no provider dependencies
• omniavatar: Provider implementations with auto-registration
• omniavatar/providers/all: Convenience import for all providers

Index

• Constants
• Variables
• func GenerateAvatarToken(opts TokenOptions) (string, error)
• func GetAvatarProvider(name string, opts ...ProviderOption) (avatar.Provider, error)
• func GetAvatarProviderPriority(name string) int
• func HasAvatarProvider(name string) bool
• func ListAvatarProviders() []string
• func RegisterAvatarProvider(name string, factory ProviderFactory, priority int)
• type AvatarMetadata
  • func DefaultAvatarMetadata(provider, agentIdentity string) AvatarMetadata
• type LiveKitStartOptions
  • func (o *LiveKitStartOptions) Validate() error
• type ProviderConfig
• type ProviderFactory
• type ProviderOption
• type TokenOptions

Constants

const (
	// PriorityThin is the priority for thin (stdlib-only) provider implementations.
	// These have no external dependencies beyond the standard library.
	PriorityThin = 0

	// PriorityThick is the priority for thick (official SDK) provider implementations.
	// These use official provider SDKs for full feature support.
	PriorityThick = 10
)

Priority constants for provider registration. Higher priority values override lower priority registrations.

Variables

var (
	WithAPIKey    = registry.WithAPIKey
	WithBaseURL   = registry.WithBaseURL
	WithExtension = registry.WithExtension
)

Re-export option functions from omniavatar-core/registry.

Functions

func GenerateAvatarToken

func GenerateAvatarToken(opts TokenOptions) (string, error)

GenerateAvatarToken creates a JWT token for an avatar to join a room.

The token includes the special "lk.publish_on_behalf" attribute that allows the avatar participant to publish tracks that appear in the UI as if they came from the agent participant.

func GetAvatarProvider

func GetAvatarProvider(name string, opts ...ProviderOption) (avatar.Provider, error)

GetAvatarProvider creates an avatar provider instance from the registry. Returns an error if the provider is not registered or if creation fails.

Example:

provider, err := omniavatar.GetAvatarProvider("heygen",
    omniavatar.WithAPIKey(os.Getenv("HEYGEN_API_KEY")),
    omniavatar.WithExtension("avatar_id", avatarID))

func GetAvatarProviderPriority

func GetAvatarProviderPriority(name string) int

GetAvatarProviderPriority returns the priority of the registered avatar provider. Returns -1 if the provider is not registered.

func HasAvatarProvider

func HasAvatarProvider(name string) bool

HasAvatarProvider returns true if an avatar provider with the given name is registered.

func ListAvatarProviders

func ListAvatarProviders() []string

ListAvatarProviders returns a list of all registered avatar provider names.

func RegisterAvatarProvider

func RegisterAvatarProvider(name string, factory ProviderFactory, priority int)

RegisterAvatarProvider registers an avatar provider factory with the given name and priority. Higher priority values override lower priority registrations.

Example:

// In omniavatar/providers/heygen/register.go (thick, priority 10)
func init() {
    omniavatar.RegisterAvatarProvider("heygen", NewProviderFromConfig, omniavatar.PriorityThick)
}

Types

type AvatarMetadata

type AvatarMetadata struct {
	// Kind identifies this as an avatar participant.
	Kind string `json:"kind"`

	// Provider is the avatar provider name.
	Provider string `json:"provider,omitempty"`

	// AgentIdentity is the identity of the agent this avatar represents.
	AgentIdentity string `json:"agent_identity,omitempty"`
}

AvatarMetadata is the standard metadata structure for avatar participants.

func DefaultAvatarMetadata

func DefaultAvatarMetadata(provider, agentIdentity string) AvatarMetadata

DefaultAvatarMetadata returns the default metadata for avatar participants.

type LiveKitStartOptions

type LiveKitStartOptions struct {
	// Room is the LiveKit room the agent has joined.
	// Required.
	Room *lksdk.Room

	// AgentIdentity is the identity of the agent participant.
	// The avatar will publish tracks on behalf of this identity
	// using the lk.publish_on_behalf attribute.
	// Required.
	AgentIdentity string

	// LiveKitURL is the LiveKit server URL for the avatar to connect to.
	// This should match the URL the agent connected to.
	// Required.
	LiveKitURL string

	// LiveKitAPIKey is used to generate tokens for the avatar.
	// Required.
	LiveKitAPIKey string

	// LiveKitAPISecret is used to generate tokens for the avatar.
	// Required.
	LiveKitAPISecret string

	// Callbacks configures optional event callbacks.
	Callbacks *avatar.SessionCallbacks

	// AudioDestination is the audio output for streaming TTS audio.
	// If provided, the session will use this instead of creating its own.
	// Optional.
	AudioDestination avatar.AudioDestination
}

LiveKitStartOptions contains LiveKit-specific start options for avatar sessions.

This is passed to Session.Start() when integrating with LiveKit.

func (*LiveKitStartOptions) Validate

func (o *LiveKitStartOptions) Validate() error

Validate checks that all required fields are set.

type ProviderConfig

type ProviderConfig = registry.ProviderConfig

ProviderConfig holds common configuration options for creating providers.

type ProviderFactory

type ProviderFactory = registry.ProviderFactory

ProviderFactory creates a Provider from configuration.

type ProviderOption

type ProviderOption = registry.ProviderOption

ProviderOption configures a ProviderConfig.

type TokenOptions

type TokenOptions struct {
	// APIKey is the LiveKit API key.
	// Required.
	APIKey string

	// APISecret is the LiveKit API secret.
	// Required.
	APISecret string

	// RoomName is the room the avatar will join.
	// Required.
	RoomName string

	// AvatarIdentity is the participant identity for the avatar.
	// Required.
	AvatarIdentity string

	// AvatarName is the display name for the avatar participant.
	// Optional, defaults to AvatarIdentity.
	AvatarName string

	// PublishOnBehalf is the identity of the agent participant.
	// The avatar will publish tracks that appear as if they're from this participant.
	// Required.
	PublishOnBehalf string

	// TTL is the token validity duration.
	// Default: 5 minutes
	TTL time.Duration

	// Metadata is optional participant metadata.
	Metadata string
}

TokenOptions configures avatar token generation.