convoai

README

Conversational AI Engine Service

English | 简体中文

Service Overview

Agora's Conversational AI Engine redefines human-computer interaction, breaking through traditional text interactions to achieve highly realistic, natural, and smooth real-time voice conversations, allowing AI to truly "speak." It is suitable for innovative scenarios such as intelligent assistants, emotional companionship, spoken language training, intelligent customer service, smart hardware, and immersive game NPCs.

Environment Setup

• Obtain Agora App ID -------- Agora Console

  • Click Create Application

    

  • Select the type of application you want to create

    

• Obtain App Certificate ----- Agora Console

  In the project management page of the Agora Console, find your project and click Configure. Click the copy icon under Primary Certificate to obtain the App Certificate for your project.

• Enable Conversational AI Engine Service ----- Enable Service

  

API Definition

For more api details, please refer to the API Documentation

API Call Examples

Initialize Conversational AI Engine Client

    const (
        appId                 = "<your appId>"
        username              = "<the username of basic auth credential>"
        password              = "<the password of basic auth credential>"
	)
	// Initialize Conversational AI Config
	config := &convoai.Config{
		AppID:      appId,
		Credential: auth.NewBasicAuthCredential(username, password),
		// Specify the region where the server is located. Options include CN, EU, AP, US.
		// The client will automatically switch to use the best domain based on the configured region.
		DomainArea: domain.US,
		// Specify the log output level. Options include DebugLevel, InfoLevel, WarningLevel, ErrLevel.
		// To disable log output, set logger to DiscardLogger.
		Logger: agoraLogger.NewDefaultLogger(agoraLogger.DebugLevel),
		ServiceRegion: convoai.GlobalServiceRegion,
	}

	// Initialize the Conversational AI service client
	convoaiClient, err := convoai.NewClient(config)
	if err != nil {
		log.Fatalln(err)
	}

Create Conversational Agent

Create a Conversational AI agent instance and join an RTC channel.

Parameters to set: LLM, TTS, and Agent related parameters.

Call the Join method to create a conversational agent, using Microsoft TTS as an example:

    const (
		appId                 = "<your appId>"
        cname                 = "<your cname>"
        agentRtcUid           = "<your agent rtc uid>"
        username              = "<the username of basic auth credential>"
        password              = "<the password of basic auth credential>"
        agentRtcToken         = "<your agent rtc token>"
        llmURL                = "<your LLM URL>"
        llmAPIKey             = "<your LLM API Key>"
        llmModel              = "<your LLM model>"
		ttsMicrosoftKey       = "<your microsoft tts key>"
		ttsMicrosoftRegion    = "<your microsoft tts region>"
		ttsMicrosoftVoiceName = "<your microsoft tts voice name>"
    )
	// Start agent
	name := appId + ":" + cname
	joinResp, err := convoaiClient.Join(context.Background(), name, &req.JoinPropertiesReqBody{
		Token:           agentRtcToken,
		Channel:         cname,
		AgentRtcUId:     agentRtcUid,
		RemoteRtcUIds:   []string{"*"},
		EnableStringUId: agoraUtils.Ptr(false),
		IdleTimeout:     agoraUtils.Ptr(120),
		LLM: &req.JoinPropertiesCustomLLMBody{
			Url:    llmURL,
			APIKey: llmAPIKey,
			SystemMessages: []map[string]any{
				{
					"role":    "system",
					"content": "You are a helpful chatbot.",
				},
			},
			Params: map[string]any{
				"model":      llmModel,
				"max_tokens": 1024,
			},
			MaxHistory:      agoraUtils.Ptr(30),
			GreetingMessage: "Hello, how can I help you?",
		},
		TTS: &req.JoinPropertiesTTSBody{
			Vendor: req.MicrosoftTTSVendor,
			Params: &req.TTSMicrosoftVendorParams{
				Key:        ttsMicrosoftKey,
				Region:     ttsMicrosoftRegion,
				VoiceName:  ttsMicrosoftVoiceName,
				Speed:      1.0,
				Volume:     70,
				SampleRate: 24000,
			},
		},
	})
	if err != nil {
		log.Fatalln(err)
	}

	if joinResp.IsSuccess() {
		log.Printf("Join success:%+v", joinResp)
	} else {
		log.Printf("Join failed:%+v", joinResp)
	}

Stop Conversational Agent

Stop the conversational agent and leave the RTC channel.

Parameters to set:

• AgentId returned by the Join interface

    // Leave agent
    leaveResp, err := convoaiClient.Leave(ctx, agentId)
	if err != nil {
		log.Fatalln(err)
	}

	if leaveResp.IsSuccess() {
		log.Printf("Leave success:%+v", leaveResp)
	} else {
		log.Printf("Leave failed:%+v", leaveResp)
	}

Update Agent Configuration

Currently, only the Token information of a running conversational agent can be updated.

Parameters to set:

• AgentId returned by the Join interface
• Token to be updated

    // Update agent
	updateResp, err := convoaiClient.Update(ctx, agentId, &req.UpdateReqBody{
		Token: updateToken,
	})
	if err != nil {
		log.Fatalln(err)
	}

	if updateResp.IsSuccess() {
		log.Printf("Update success:%+v", updateResp)
	} else {
		log.Printf("Update failed:%+v", updateResp)
	}

Query Agent Status

Query the status of the conversational agent.

Parameters to set:

• AgentId returned by the Join interface

    // Query agent
	queryResp, err := convoaiClient.Query(ctx, agentId)
	if err != nil {
		log.Fatalln(err)
		return
	}

	if queryResp.IsSuccess() {
		log.Printf("Query success:%+v", queryResp)
	} else {
		log.Printf("Query failed:%+v", queryResp)
	}

Retrieves a list of agents

Retrieves a list of agents that meet the specified criteria.

Parameters to set:

• AgentId returned by the Join interface

     // List agent
	listResp, err := convoaiClient.List(ctx,
		req.WithState(2),
		req.WithLimit(10),
	)
	if err != nil {
		log.Fatalln(err)
	}

	if listResp.IsSuccess() {
		log.Printf("List success:%+v", listResp)
	} else {
		log.Printf("List failed:%+v", listResp)
	}

Error Codes and Response Status Codes Handling

For specific business response codes, please refer to the Business Response Codes documentation.

Documentation

Index

• Variables
• type Client
  • func NewClient(config *Config) (*Client, error)
  • func (c *Client) GetHistory(ctx context.Context, agentId string) (*resp.HistoryResp, error)
  • func (c *Client) Interrupt(ctx context.Context, agentId string) (*resp.InterruptResp, error)
  • func (c *Client) Join(ctx context.Context, name string, payload *req.JoinPropertiesReqBody) (*resp.JoinResp, error)
  • func (c *Client) Leave(ctx context.Context, agentId string) (*resp.LeaveResp, error)
  • func (c *Client) List(ctx context.Context, options ...req.ListOption) (*resp.ListResp, error)
  • func (c *Client) Query(ctx context.Context, agentId string) (*resp.QueryResp, error)
  • func (c *Client) Speak(ctx context.Context, agentId string, payload *req.SpeakBody) (*resp.SpeakResp, error)
  • func (c *Client) Update(ctx context.Context, agentId string, payload *req.UpdateReqBody) (*resp.UpdateResp, error)
• type Config
• type ServiceRegion

Variables

var RetryCount = 3

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

func NewClient

func NewClient(config *Config) (*Client, error)

NewClient

@brief Creates a Conversational AI engine client with the specified configuration

@param config Configuration of the Conversational AI engine client. See Config for details.

@return Returns the Conversational AI engine client.

@return Returns an error object. If the request fails, the error object is not nil and contains error information.

@since v0.7.0

func (*Client) GetHistory

func (c *Client) GetHistory(ctx context.Context, agentId string) (*resp.HistoryResp, error)

GetHistory

@brief Acquires the short-term memory of the specified agent instance

@since v0.9.0

@example Use this method to acquire the short-term memory of the specified agent instance.

@param ctx Context to control the request lifecycle.

@param agentId Agent ID.

@return Returns the response *HistoryResp. See api.HistoryResp for details.

@return Returns an error object. If the request fails, the error object is not nil and contains error information.

func (*Client) Interrupt

func (c *Client) Interrupt(ctx context.Context, agentId string) (*resp.InterruptResp, error)

Interrupt

@brief Interrupts the specified agent instance

@since v0.9.0

@example Use this method to interrupt the specified agent instance.

@param ctx Context to control the request lifecycle.

@param agentId Agent ID.

@return Returns the response *InterruptResp. See api.InterruptResp for details.

@return Returns an error object. If the request fails, the error object is not nil and contains error information.

func (*Client) Join

func (c *Client) Join(ctx context.Context, name string, payload *req.JoinPropertiesReqBody) (*resp.JoinResp, error)

Join @brief Creates an agent instance and joins the specified RTC channel

@since v0.7.0

@example Use this to create an agent instance in an RTC channel.

@post After successful execution, the agent will join the specified channel. You can perform subsequent operations using the returned agent ID.

@param ctx Context to control the request lifecycle.

@param name Unique identifier for the agent. The same identifier cannot be used repeatedly.

@param propertiesBody Configuration properties of the agent, including channel information, token, LLM settings, TTS settings, etc. See api.JoinPropertiesReqBody for details.

@return Returns the response *JoinResp. See api.JoinResp for details.

@return Returns an error object. If the request fails, the error object is not nil and contains error information.

func (*Client) Leave

func (c *Client) Leave(ctx context.Context, agentId string) (*resp.LeaveResp, error)

Leave

@brief Stops the specified agent instance and leaves the RTC channel

@since v0.7.0

@example Use this to stop an agent instance.

@post After successful execution, the agent will be stopped and leave the RTC channel

@note Ensure the agent ID has been obtained by calling the Join API before using this method.

@param ctx Context to control the request lifecycle.

@param agentId Agent ID.

@return Returns the response *LeaveResp. See api.LeaveResp for details.

@return Returns an error object. If the request fails, the error object is not nil and contains error information.

func (*Client) List

func (c *Client) List(ctx context.Context, options ...req.ListOption) (*resp.ListResp, error)

List @brief Retrieves a list of agents that meet the specified criteria

@since v0.7.0

@example Use this to get a list of agents that meet the specified criteria.

@post After successful execution, a list of agents that meet the specified criteria will be retrieved.

@param ctx Context to control the request lifecycle.

@param ListOption Query parameters. See api.ListOption for details.

@return Returns the response *ListResp. See api.ListResp for details.

@return Returns an error object. If the request fails, the error object is not nil and contains error information.

func (*Client) Query

func (c *Client) Query(ctx context.Context, agentId string) (*resp.QueryResp, error)

Query

@brief Query the current status of the specified agent instance

@since v0.7.0

@example Use this to get the current status of the specified agent instance.

@post After successful execution, the current status of the specified agent instance will be retrieved.

@note Ensure the agent ID has been obtained by calling the Join API before using this method.

@param ctx Context to control the request lifecycle.

@param agentId Agent ID.

@return Returns the response *QueryResp. See api.QueryResp for details.

@return Returns an error object. If the request fails, the error object is not nil and contains error information.

func (*Client) Speak

func (c *Client) Speak(ctx context.Context, agentId string, payload *req.SpeakBody) (*resp.SpeakResp, error)

Speak

@brief Speaks a custom message for the specified agent instance

@since v0.9.0

@param ctx Context to control the request lifecycle.

@param agentId Agent ID.

@param payload Request body for the specified agent to speak a custom message. See api.SpeakBody for details.

@return Returns the response *SpeakResp. See api.SpeakResp for details.

@return Returns an error object. If the request fails, the error object is not nil and contains error information.

func (*Client) Update

func (c *Client) Update(ctx context.Context, agentId string, payload *req.UpdateReqBody) (*resp.UpdateResp, error)

Update

@brief Adjusts the agent's parameters at runtime

@since v0.7.0

@example Use this to adjust the agent's parameters at runtime.

@post After successful execution, the agent's parameters will be adjusted.

@note Ensure the agent ID has been obtained by calling the Join API before using this method.

@param ctx Context to control the request lifecycle.

@param agentId Agent ID.

@param payload Parameters to be adjusted. See api.UpdateReqBody for details.

@return Returns the response *UpdateResp. See api.UpdateResp for details.

@return Returns an error object. If the request fails, the error object is not nil and contains error information.

type Config

type Config struct {
	// Agora AppID
	AppID string
	// Timeout for HTTP requests
	HttpTimeout time.Duration
	// Credential for accessing the Agora service.
	//
	// Available credential types:
	//
	//  - BasicAuthCredential: See auth.NewBasicAuthCredential for details
	Credential auth.Credential

	// Domain area for the REST Client. See domain.Area for details.
	DomainArea domain.Area

	// Logger for the REST Client
	//
	// Implement the log.Logger interface in your project to output REST Client logs to your logging component.
	//
	// Alternatively, you can use the default logging component. See log.NewDefaultLogger for details.
	Logger log.Logger

	// Service version. See ServiceRegion for details.
	ServiceRegion ServiceRegion
}

@brief Defines the configuration for the Conversational AI engine client

@since v0.7.0

type ServiceRegion

type ServiceRegion int

@brief ServiceRegion represents the region of the Conversational AI engine service

@note The service in Chinese mainland and the global region are two different services

@since v0.7.0

const (
	UnknownServiceRegion ServiceRegion = iota
	// ChineseMainlandServiceRegion represents the Conversational AI engine service in Chinese mainland
	ChineseMainlandServiceRegion
	// GlobalServiceRegion represents the Conversational AI engine service in the global region, except Chinese mainland
	GlobalServiceRegion
)