client

package module
v0.32.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 4, 2024 License: Apache-2.0 Imports: 15 Imported by: 12

README

Prediction Guard Go Client

CircleCI Go Report Card go.mod Go version

Copyright 2024 Prediction Guard bill@predictionguard.com

Description

This Module provides functionality developed to simplify interfacing with Prediction Guard API in Go.

Requirements

To access the API, contact us here to get an enterprise access token. You will need this access token to continue.

Usage
package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"time"

	"github.com/predictionguard/go-client"
)

func main() {
	if err := run(); err != nil {
		log.Fatalln(err)
	}
}

func run() error {
	host := "https://api.predictionguard.com"
	apiKey := os.Getenv("PGKEY")

	logger := func(ctx context.Context, msg string, v ...any) {
		s := fmt.Sprintf("msg: %s", msg)
		for i := 0; i < len(v); i = i + 2 {
			s = s + fmt.Sprintf(", %s: %v", v[i], v[i+1])
		}
		log.Println(s)
	}

	cln := client.New(logger, host, apiKey)

	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
	defer cancel()

	input := client.ChatInput{
		Model: client.Models.NeuralChat7B,
		Messages: []client.ChatInputMessage{
			{
				Role:    client.Roles.User,
				Content: "How do you feel about the world in general",
			},
		},
		MaxTokens:   1000,
		Temperature: 0.1,
		TopP:        0.1,
		Options: &client.ChatInputOptions{
			Factuality:       true,
			Toxicity:         true,
			PII:              client.PIIs.Replace,
			PIIReplaceMethod: client.ReplaceMethods.Random,
		},
	}

	resp, err := cln.Chat(ctx, input)
	if err != nil {
		return fmt.Errorf("ERROR: %w", err)
	}

	fmt.Println(resp.Choices[0].Message.Content)

	return nil
}

Take a look at the examples directory for more examples.

Docs

You can find the SDK and Prediction Guard docs using these links.

SDK Docs

PG API Docs

Getting started

Once you have your api key you can use the makefile to run curl commands for the different api endpoints. For example, make curl-injection will connect to the injection endpoint and return the injection response. The makefile also allows you to run the different examples such as make go-injection to run the Go injection example.

Licensing
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Copyright 2024 Prediction Guard

Documentation

Overview

Package client provides support to access the Prediction Guard API service.

Index

Examples

Constants

This section is empty.

Variables

View Source 
var Capabilities = capabilitySet{
	ChatCompletion:     newCapability("chat-completion"),
	ChatWithImage:      newCapability("chat-with-image"),
	Completion:         newCapability("completion"),
	Embedding:          newCapability("embedding"),
	EmbeddingWithImage: newCapability("embedding-with-image"),
	Tokenize:           newCapability("tokenize"),
}

Capabilities represents the set of model capabilities.

View Source 
var Directions = directionSet{
	Right: newDirection("Right"),
	Left:  newDirection("Left"),
}

Directions represents the set of directions.

View Source 
var ErrUnauthorized = errors.New("api understands the request but refuses to authorize it")

ErrUnauthorized represent a situation where authentication fails.

View Source 
var Languages = languageSet{
	Afrikanns:  newLanguage("afr"),
	Amharic:    newLanguage("amh"),
	Arabic:     newLanguage("ara"),
	Armenian:   newLanguage("hye"),
	Azerbaijan: newLanguage("aze"),
	Basque:     newLanguage("eus"),
	Belarusian: newLanguage("bel"),
	Bengali:    newLanguage("ben"),
	Bosnian:    newLanguage("bos"),
	Catalan:    newLanguage("cat"),
	Chechen:    newLanguage("che"),
	Cherokee:   newLanguage("chr"),
	Chinese:    newLanguage("zho"),
	Croatian:   newLanguage("hrv"),
	Czech:      newLanguage("ces"),
	Danish:     newLanguage("dan"),
	Dutch:      newLanguage("nld"),
	English:    newLanguage("eng"),
	Estonian:   newLanguage("est"),
	Fijian:     newLanguage("fij"),
	Filipino:   newLanguage("fil"),
	Finnish:    newLanguage("fin"),
	French:     newLanguage("fra"),
	Galician:   newLanguage("glg"),
	Georgian:   newLanguage("kat"),
	German:     newLanguage("deu"),
	Greek:      newLanguage("ell"),
	Gujarati:   newLanguage("guj"),
	Haitian:    newLanguage("hat"),
	Hebrew:     newLanguage("heb"),
	Hindi:      newLanguage("hin"),
	Hungarian:  newLanguage("hun"),
	Icelandic:  newLanguage("isl"),
	Indonesian: newLanguage("ind"),
	Irish:      newLanguage("gle"),
	Italian:    newLanguage("ita"),
	Japanese:   newLanguage("jpn"),
	Kannada:    newLanguage("kan"),
	Kazakh:     newLanguage("kaz"),
	Korean:     newLanguage("kor"),
	Latvian:    newLanguage("lav"),
	Lithuanian: newLanguage("lit"),
	Macedonian: newLanguage("mkd"),
	Malay1:     newLanguage("msa"),
	Malay2:     newLanguage("zlm"),
	Malayalam:  newLanguage("mal"),
	Maltese:    newLanguage("mlt"),
	Marathi:    newLanguage("mar"),
	Nepali:     newLanguage("nep"),
	Norwegian:  newLanguage("nor"),
	Persian:    newLanguage("fas"),
	Polish:     newLanguage("pol"),
	Portuguese: newLanguage("por"),
	Romanian:   newLanguage("ron"),
	Russian:    newLanguage("rus"),
	Samoan:     newLanguage("smo"),
	Serbian:    newLanguage("srp"),
	Slovak:     newLanguage("slk"),
	Slovenian:  newLanguage("slv"),
	Slavonic:   newLanguage("chu"),
	Spanish:    newLanguage("spa"),
	Swahili:    newLanguage("swh"),
	Swedish:    newLanguage("swe"),
	Tamil:      newLanguage("tam"),
	Telugu:     newLanguage("tel"),
	Thai:       newLanguage("tha"),
	Turkish:    newLanguage("tur"),
	Ukrainian:  newLanguage("ukr"),
	Urdu:       newLanguage("urd"),
	Welsh:      newLanguage("cym"),
	Vietnamese: newLanguage("vie"),
}

Languages represents the set of languages that can be used.

View Source 
var PIIs = piiSet{
	Block:   newPII("block"),
	Replace: newPII("replace"),
}

PIIs represents the set of PIIs that can be used.

View Source 
var ReplaceMethods = replaceMethodSet{
	Random:   newReplaceMethod("random"),
	Fake:     newReplaceMethod("fake"),
	Category: newReplaceMethod("category"),
	Mask:     newReplaceMethod("mask"),
}

ReplaceMethods represents the set of replace methods that can be used.

View Source 
var Roles = roleSet{
	Assistant: newRole("assistant"),
	User:      newRole("user"),
	System:    newRole("system"),
}

Roles represents the set of roles that can be used.

Functions

func Ptr added in v0.15.0 

func Ptr[T any](t T) *T

Ptr converts any value into a pointer of that value.

func WithClient 

func WithClient(http *http.Client) func(cln *Client)

WithClient adds a custom client for processing requests. It's recommend to not use the default client and provide your own.

Types

type Base64Encoder added in v0.7.0 

type Base64Encoder interface {
	EncodeBase64(ctx context.Context) (string, error)
}

Base64Encoder defines a method that can read a data source and returns a base64 encoded string.

type Capability added in v0.28.0 

type Capability struct {
	// contains filtered or unexported fields
}

Capability represents a capability in the system.

func (Capability) Equal added in v0.28.0 

func (c Capability) Equal(c2 Capability) bool

Equal provides support for the go-cmp package and testing.

func (Capability) MarshalText added in v0.28.0 

func (c Capability) MarshalText() ([]byte, error)

MarshalText implement the marshal interface for JSON conversions.

func (Capability) String added in v0.28.0 

func (c Capability) String() string

String returns the name of the capability.

func (*Capability) UnmarshalText added in v0.28.0 

func (c *Capability) UnmarshalText(data []byte) error

UnmarshalText implement the unmarshal interface for JSON conversions.

type Chat 

type Chat struct {
	ID      string       `json:"id"`
	Object  string       `json:"object"`
	Created Time         `json:"created"`
	Model   string       `json:"model"`
	Choices []ChatChoice `json:"choices"`
}

Chat represents the result for the chat call.

type ChatChoice added in v0.3.0 

type ChatChoice struct {
	Index   int         `json:"index"`
	Message ChatMessage `json:"message"`
}

ChatChoice represents a choice for the chat call.

type ChatInput added in v0.7.0 

type ChatInput struct {
	Model           string
	Message         string
	MaxTokens       *int
	Temperature     *float32
	TopP            *float64
	TopK            *int
	InputExtension  *InputExtension
	OutputExtension *OutputExtension
}

ChatInput represents the full potential input options for chat.

func (ChatInput) ChatInputType added in v0.23.0 

func (ChatInput) ChatInputType()

ChatInputType implements the ChatInputTypes interface.

type ChatInputMessage added in v0.8.0 

type ChatInputMessage struct {
	Role    Role
	Content string
}

ChatInputMessage represents a role and content related to a chat.

type ChatInputMulti added in v0.23.0 

type ChatInputMulti struct {
	Model           string
	Messages        []ChatInputMessage
	MaxTokens       *int
	Temperature     *float32
	TopP            *float64
	TopK            *int
	InputExtension  *InputExtension
	OutputExtension *OutputExtension
}

ChatInputMulti represents the full potential input options for chat.

func (ChatInputMulti) ChatInputType added in v0.23.0 

func (ChatInputMulti) ChatInputType()

ChatInputType implements the ChatInputTypes interface.

type ChatInputTypes added in v0.23.0 

type ChatInputTypes interface {
	ChatInputType()
}

ChatInputTypes defines behavior any chat input type must implement. The method doesn't need to do anything, it just needs to exist.

type ChatMessage 

type ChatMessage struct {
	Role    Role   `json:"role"`
	Content string `json:"content"`
}

ChatMessage represents the role of the sender and the response.

type ChatSSE 

type ChatSSE struct {
	ID      string          `json:"id"`
	Object  string          `json:"object"`
	Created Time            `json:"created"`
	Model   string          `json:"model"`
	Choices []ChatSSEChoice `json:"choices"`
	Error   string          `json:"error"`
}

ChatSSE represents the result for the sse call.

type ChatSSEChoice 

type ChatSSEChoice struct {
	Index        int          `json:"index"`
	Delta        ChatSSEDelta `json:"delta"`
	Text         string       `json:"generated_text"`
	Probs        float32      `json:"logprobs"`
	FinishReason string       `json:"finish_reason"`
}

ChatSSEChoice represents a choice for the sse call.

type ChatSSEDelta 

type ChatSSEDelta struct {
	Content string `json:"content"`
}

ChatSSEDelta represents content for the sse call.

type ChatSSEInput added in v0.9.0 

type ChatSSEInput struct {
	Model          string
	Messages       []ChatInputMessage
	MaxTokens      *int
	Temperature    *float32
	TopP           *float64
	TopK           *int
	InputExtension *InputExtension
}

ChatSSEInput represents the full potential input options for SSE chat.

type ChatVision added in v0.7.0 

type ChatVision struct {
	ID      string             `json:"id"`
	Object  string             `json:"object"`
	Created Time               `json:"created"`
	Model   string             `json:"model"`
	Choices []ChatVisionChoice `json:"choices"`
}

ChatVision represents the result for the vision call.

type ChatVisionChoice added in v0.7.0 

type ChatVisionChoice struct {
	Index   int               `json:"index"`
	Message ChatVisionMessage `json:"message"`
}

ChatVisionChoice represents a choice for the vision call.

type ChatVisionInput added in v0.9.0 

type ChatVisionInput struct {
	Model           string
	Role            Role
	Question        string
	Image           Base64Encoder
	MaxTokens       int
	Temperature     *float32
	TopP            *float64
	TopK            *int
	InputExtension  *InputExtension
	OutputExtension *OutputExtension
}

ChatVisionInput represents the full potential input options for vision chat.

type ChatVisionMessage added in v0.7.0 

type ChatVisionMessage struct {
	Role    Role   `json:"role"`
	Content string `json:"content"`
}

ChatVisionMessage represents content for the vision call.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client represents a client that can talk to the PG API service.

func New 

func New(log Logger, host string, apiKey string, options ...func(cln *Client)) *Client

New constructs a Client that can be used to talk with the PG API service.

func (*Client) Capability added in v0.28.0 

func (cln *Client) Capability(ctx context.Context, capability Capability) (ModelResponse, error)

Capability returns the set of models for the specified capability.

Example 
// examples/capability/main.go

host := "https://api.predictionguard.com"
apiKey := os.Getenv("PREDICTIONGUARD_API_KEY")

logger := func(ctx context.Context, msg string, v ...any) {
	s := fmt.Sprintf("msg: %s", msg)
	for i := 0; i < len(v); i = i + 2 {
		s = s + fmt.Sprintf(", %s: %v", v[i], v[i+1])
	}
	log.Println(s)
}

cln := client.New(logger, host, apiKey)

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

resp, err := cln.Capability(ctx, client.Capabilities.ChatCompletion)
if err != nil {
	log.Fatalln("ERROR:", err)
}

fmt.Println(resp)

Output:

func (*Client) Chat 

func (cln *Client) Chat(ctx context.Context, input ChatInputTypes) (Chat, error)

Chat generate chat completions based on a conversation history.

Example 
// examples/chat/basic/main.go

host := "https://api.predictionguard.com"
apiKey := os.Getenv("PREDICTIONGUARD_API_KEY")

logger := func(ctx context.Context, msg string, v ...any) {
	s := fmt.Sprintf("msg: %s", msg)
	for i := 0; i < len(v); i = i + 2 {
		s = s + fmt.Sprintf(", %s: %v", v[i], v[i+1])
	}
	log.Println(s)
}

cln := client.New(logger, host, apiKey)

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

input := client.ChatInput{
	Model:       "neural-chat-7b-v3-3",
	Message:     "How do you feel about the world in general",
	MaxTokens:   client.Ptr(1000),
	Temperature: client.Ptr[float32](0.1),
	TopP:        client.Ptr(0.1),
	InputExtension: &client.InputExtension{
		PII:              client.PIIs.Replace,
		PIIReplaceMethod: client.ReplaceMethods.Random,
	},
	OutputExtension: &client.OutputExtension{
		Factuality: true,
		Toxicity:   true,
	},
}

resp, err := cln.Chat(ctx, input)
if err != nil {
	log.Fatalln("ERROR:", err)
}

fmt.Println(resp.Choices[0].Message.Content)

inputMulti := client.ChatInputMulti{
	Model: "neural-chat-7b-v3-3",
	Messages: []client.ChatInputMessage{
		{
			Role:    client.Roles.User,
			Content: "How do you feel about the world in general",
		},
	},
	MaxTokens:   client.Ptr(1000),
	Temperature: client.Ptr[float32](0.1),
	TopP:        client.Ptr(0.1),
	InputExtension: &client.InputExtension{
		PII:              client.PIIs.Replace,
		PIIReplaceMethod: client.ReplaceMethods.Random,
	},
	OutputExtension: &client.OutputExtension{
		Factuality: true,
		Toxicity:   true,
	},
}

resp, err = cln.Chat(ctx, inputMulti)
if err != nil {
	log.Fatalln("ERROR:", err)
}

fmt.Println(resp.Choices[0].Message.Content)

Output:

func (*Client) ChatSSE 

func (cln *Client) ChatSSE(ctx context.Context, input ChatSSEInput, ch chan ChatSSE) error

ChatSSE generate chat completions based on a conversation history.

Example 
// examples/chat/sse/main.go

host := "https://api.predictionguard.com"
apiKey := os.Getenv("PREDICTIONGUARD_API_KEY")

logger := func(ctx context.Context, msg string, v ...any) {
	s := fmt.Sprintf("msg: %s", msg)
	for i := 0; i < len(v); i = i + 2 {
		s = s + fmt.Sprintf(", %s: %v", v[i], v[i+1])
	}
	log.Println(s)
}

cln := client.New(logger, host, apiKey)

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

input := client.ChatSSEInput{
	Model: "neural-chat-7b-v3-3",
	Messages: []client.ChatInputMessage{
		{
			Role:    client.Roles.User,
			Content: "How do you feel about the world in general",
		},
	},
	MaxTokens:   client.Ptr(1000),
	Temperature: client.Ptr[float32](0.1),
	TopP:        client.Ptr(0.1),
}

ch := make(chan client.ChatSSE, 100)

err := cln.ChatSSE(ctx, input, ch)
if err != nil {
	log.Fatalln("ERROR:", err)
}

for resp := range ch {
	for _, choice := range resp.Choices {
		fmt.Print(choice.Delta.Content)
	}
}

Output:

func (*Client) ChatVision added in v0.7.0 

func (cln *Client) ChatVision(ctx context.Context, input ChatVisionInput) (ChatVision, error)

ChatVision generate chat completions based on a question and an image.

Example 
// examples/chat/vision/main.go

host := "https://api.predictionguard.com"
apiKey := os.Getenv("PREDICTIONGUARD_API_KEY")

logger := func(ctx context.Context, msg string, v ...any) {
	s := fmt.Sprintf("msg: %s", msg)
	for i := 0; i < len(v); i = i + 2 {
		s = s + fmt.Sprintf(", %s: %v", v[i], v[i+1])
	}
	log.Println(s)
}

cln := client.New(logger, host, apiKey)

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

image, err := client.NewImageNetwork("https://pbs.twimg.com/profile_images/1571574401107169282/ylAgz_f5_400x400.jpg")
if err != nil {
	log.Fatalln("ERROR:", err)
}

input := client.ChatVisionInput{
	Model:       "llava-1.5-7b-hf",
	Role:        client.Roles.User,
	Question:    "Is there a deer in this picture?",
	Image:       image,
	MaxTokens:   1000,
	Temperature: client.Ptr[float32](0.1),
	TopP:        client.Ptr(0.1),
}

resp, err := cln.ChatVision(ctx, input)
if err != nil {
	log.Fatalln("ERROR:", err)
}

for i, choice := range resp.Choices {
	fmt.Printf("choice %d: %s\n", i, choice.Message.Content)
}

Output:

func (*Client) Completions 

func (cln *Client) Completions(ctx context.Context, input CompletionInput) (Completion, error)

Completions retrieve text completions based on the provided input.

Example 
// examples/completion/main.go

host := "https://api.predictionguard.com"
apiKey := os.Getenv("PREDICTIONGUARD_API_KEY")

logger := func(ctx context.Context, msg string, v ...any) {
	s := fmt.Sprintf("msg: %s", msg)
	for i := 0; i < len(v); i = i + 2 {
		s = s + fmt.Sprintf(", %s: %v", v[i], v[i+1])
	}
	log.Println(s)
}

cln := client.New(logger, host, apiKey)

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

input := client.CompletionInput{
	Model:       "neural-chat-7b-v3-3",
	Prompt:      "Will I lose my hair",
	MaxTokens:   1000,
	Temperature: client.Ptr[float32](0.1),
	TopP:        client.Ptr(0.1),
}

resp, err := cln.Completions(ctx, input)
if err != nil {
	log.Fatalln("ERROR:", err)
}

fmt.Println(resp.Choices[0].Text)

Output:

func (*Client) Embedding added in v0.7.0 

func (cln *Client) Embedding(ctx context.Context, model string, input EmbeddingInputTypes) (Embedding, error)

Embedding converts text, text + image, and image to a numerical representation that is useful for search and retrieval. When you have both text and image, the use case would be like a video frame plus the transcription or an image plus a caption. The response should include the output vector.

Example 
// examples/embedding/basic/main.go

host := "https://api.predictionguard.com"
apiKey := os.Getenv("PREDICTIONGUARD_API_KEY")

logger := func(ctx context.Context, msg string, v ...any) {
	s := fmt.Sprintf("msg: %s", msg)
	for i := 0; i < len(v); i = i + 2 {
		s = s + fmt.Sprintf(", %s: %v", v[i], v[i+1])
	}
	log.Println(s)
}

cln := client.New(logger, host, apiKey)

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

image, err := client.NewImageNetwork("https://pbs.twimg.com/profile_images/1571574401107169282/ylAgz_f5_400x400.jpg")
if err != nil {
	log.Fatalln("ERROR: %w", err)
}

input := client.EmbeddingInputs{
	{
		Text:  "This is Bill Kennedy, a decent Go developer.",
		Image: image,
	},
}

resp, err := cln.Embedding(ctx, "bridgetower-large-itm-mlm-itc", input)
if err != nil {
	log.Fatalln("ERROR:", err)
}

for _, data := range resp.Data {
	fmt.Print(data.Embedding)
}

Output:

func (*Client) EmbeddingWithTruncate added in v0.30.0 

func (cln *Client) EmbeddingWithTruncate(ctx context.Context, model string, input EmbeddingInputTypes, direction Direction) (Embedding, error)

EmbeddingWithTruncate behaves like Embedding but provides an option to set a truncation direction for models that support truncation.

Example 
// examples/embedding/truncate/main.go

host := "https://api.predictionguard.com"
apiKey := os.Getenv("PREDICTIONGUARD_API_KEY")

logger := func(ctx context.Context, msg string, v ...any) {
	s := fmt.Sprintf("msg: %s", msg)
	for i := 0; i < len(v); i = i + 2 {
		s = s + fmt.Sprintf(", %s: %v", v[i], v[i+1])
	}
	log.Println(s)
}

cln := client.New(logger, host, apiKey)

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

input := client.EmbeddingInputs{
	{
		Text: "This is Bill Kennedy, a decent Go developer.",
	},
}

resp, err := cln.EmbeddingWithTruncate(ctx, "multilingual-e5-large-instruct", input, client.Directions.Right)
if err != nil {
	log.Fatalln("ERROR:", err)
}

for _, data := range resp.Data {
	fmt.Print(data.Embedding)
}

Output:

func (*Client) Factuality 

func (cln *Client) Factuality(ctx context.Context, reference string, text string) (Factuality, error)

Factuality checks the factuality of a given text compared to a reference.

Example 
// examples/factuality/main.go

host := "https://api.predictionguard.com"
apiKey := os.Getenv("PREDICTIONGUARD_API_KEY")

logger := func(ctx context.Context, msg string, v ...any) {
	s := fmt.Sprintf("msg: %s", msg)
	for i := 0; i < len(v); i = i + 2 {
		s = s + fmt.Sprintf(", %s: %v", v[i], v[i+1])
	}
	log.Println(s)
}

cln := client.New(logger, host, apiKey)

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

fact := "The President shall receive in full for his services during the term for which he shall have been elected compensation in the aggregate amount of 400,000 a year, to be paid monthly, and in addition an expense allowance of 50,000 to assist in defraying expenses relating to or resulting from the discharge of his official duties. Any unused amount of such expense allowance shall revert to the Treasury pursuant to section 1552 of title 31, United States Code. No amount of such expense allowance shall be included in the gross income of the President. He shall be entitled also to the use of the furniture and other effects belonging to the United States and kept in the Executive Residence at the White House."
text := "The president of the united states can take a salary of one million dollars"

resp, err := cln.Factuality(ctx, fact, text)
if err != nil {
	log.Fatalln("ERROR:", err)
}

log.Println(resp.Checks[0])

Output:

func (*Client) HealthCheck 

func (cln *Client) HealthCheck(ctx context.Context) (string, error)

HealthCheck validates the PG API Service is available.

func (*Client) Injection 

func (cln *Client) Injection(ctx context.Context, prompt string) (Injection, error)

Injection detects potential prompt injection attacks in a given prompt.

Example 
// examples/injection/main.go

host := "https://api.predictionguard.com"
apiKey := os.Getenv("PREDICTIONGUARD_API_KEY")

logger := func(ctx context.Context, msg string, v ...any) {
	s := fmt.Sprintf("msg: %s", msg)
	for i := 0; i < len(v); i = i + 2 {
		s = s + fmt.Sprintf(", %s: %v", v[i], v[i+1])
	}
	log.Println(s)
}

cln := client.New(logger, host, apiKey)

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

prompt := "A short poem may be a stylistic choice or it may be that you have said what you intended to say in a more concise way."

resp, err := cln.Injection(ctx, prompt)
if err != nil {
	log.Fatalln("ERROR:", err)
}

fmt.Println(resp.Checks[0].Probability)

Output:

func (*Client) ReplacePII added in v0.11.0 

func (cln *Client) ReplacePII(ctx context.Context, prompt string, method ReplaceMethod) (ReplacePII, error)

ReplacePII replaces personal information such as names, SSNs, and emails in a given text.

Example 
// examples/ReplacePII/main.go

host := "https://api.predictionguard.com"
apiKey := os.Getenv("PREDICTIONGUARD_API_KEY")

logger := func(ctx context.Context, msg string, v ...any) {
	s := fmt.Sprintf("msg: %s", msg)
	for i := 0; i < len(v); i = i + 2 {
		s = s + fmt.Sprintf(", %s: %v", v[i], v[i+1])
	}
	log.Println(s)
}

cln := client.New(logger, host, apiKey)

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

prompt := "My email is bill@ardanlabs.com and my number is 954-123-4567."

resp, err := cln.ReplacePII(ctx, prompt, client.ReplaceMethods.Mask)
if err != nil {
	log.Fatalln("ERROR:", err)
}

fmt.Println(resp.Checks[0].NewPrompt)

Output:

func (*Client) Tokenize added in v0.31.0 

func (cln *Client) Tokenize(ctx context.Context, input TokenizeInput) (Tokenize, error)

Tokenize provides the set of tokens the model server calculates.

Example 
// examples/tokenize/main.go

host := "https://api.predictionguard.com"
apiKey := os.Getenv("PREDICTIONGUARD_API_KEY")

logger := func(ctx context.Context, msg string, v ...any) {
	s := fmt.Sprintf("msg: %s", msg)
	for i := 0; i < len(v); i = i + 2 {
		s = s + fmt.Sprintf(", %s: %v", v[i], v[i+1])
	}
	log.Println(s)
}

cln := client.New(logger, host, apiKey)

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

input := client.TokenizeInput{
	Model: "Hermes-2-Pro-Mistral-7B",
	Input: "how many tokens exist for this sentence.",
}

resp, err := cln.Tokenize(ctx, input)
if err != nil {
	log.Fatalln("ERROR:", err)
}

fmt.Println(resp.Data)

Output:

func (*Client) Toxicity 

func (cln *Client) Toxicity(ctx context.Context, text string) (Toxicity, error)

Toxicity checks the toxicity of a given text.

Example 
// examples/toxicity/main.go

host := "https://api.predictionguard.com"
apiKey := os.Getenv("PREDICTIONGUARD_API_KEY")

logger := func(ctx context.Context, msg string, v ...any) {
	s := fmt.Sprintf("msg: %s", msg)
	for i := 0; i < len(v); i = i + 2 {
		s = s + fmt.Sprintf(", %s: %v", v[i], v[i+1])
	}
	log.Println(s)
}

cln := client.New(logger, host, apiKey)

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

text := "Every flight I have is late and I am very angry. I want to hurt someone."

resp, err := cln.Toxicity(ctx, text)
if err != nil {
	log.Fatalln("ERROR:", err)
}

fmt.Println(resp.Checks[0].Score)

Output:

func (*Client) Translate 

func (cln *Client) Translate(ctx context.Context, text string, source Language, target Language, useThirdPartyEngine bool) (Translate, error)

Translate converts text from one language to another.

Example 
host := "https://api.predictionguard.com"
apiKey := os.Getenv("PREDICTIONGUARD_API_KEY")

logger := func(ctx context.Context, msg string, v ...any) {
	s := fmt.Sprintf("msg: %s", msg)
	for i := 0; i < len(v); i = i + 2 {
		s = s + fmt.Sprintf(", %s: %v", v[i], v[i+1])
	}
	log.Println(s)
}

cln := client.New(logger, host, apiKey)

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

text := "The rain in Spain stays mainly in the plain"

resp, err := cln.Translate(ctx, text, client.Languages.English, client.Languages.Spanish, false)
if err != nil {
	log.Fatalln("ERROR:", err)
}

fmt.Println(resp.BestTranslation)

Output:

type Completion 

type Completion struct {
	ID      string             `json:"id"`
	Object  string             `json:"object"`
	Created Time               `json:"created"`
	Model   string             `json:"model"`
	Choices []CompletionChoice `json:"choices"`
}

Completion represents the result for the completion call.

type CompletionChoice 

type CompletionChoice struct {
	Index int    `json:"index"`
	Text  string `json:"text"`
}

CompletionChoice represents a choice for the completion call.

type CompletionInput added in v0.9.0 

type CompletionInput struct {
	Model           string
	Prompt          string
	MaxTokens       int
	Temperature     *float32
	TopP            *float64
	TopK            *int
	InputExtension  *InputExtension
	OutputExtension *OutputExtension
}

CompletionInput represents the full potential input options for completion.

type Direction added in v0.30.0 

type Direction struct {
	// contains filtered or unexported fields
}

Direction represents a direction in the system.

func (Direction) Equal added in v0.30.0 

func (d Direction) Equal(d2 Direction) bool

Equal provides support for the go-cmp package and testing.

func (Direction) MarshalText added in v0.30.0 

func (d Direction) MarshalText() ([]byte, error)

MarshalText implement the marshal interface for JSON conversions.

func (Direction) String added in v0.30.0 

func (d Direction) String() string

String returns the name of the direction.

func (*Direction) UnmarshalText added in v0.30.0 

func (d *Direction) UnmarshalText(data []byte) error

UnmarshalText implement the unmarshal interface for JSON conversions.

type Embedding added in v0.7.0 

type Embedding struct {
	ID      string          `json:"id"`
	Object  string          `json:"object"`
	Created Time            `json:"created"`
	Model   string          `json:"model"`
	Data    []EmbeddingData `json:"data"`
}

Embedding represents the result for the embedding call.

type EmbeddingData added in v0.7.0 

type EmbeddingData struct {
	Index     int       `json:"index"`
	Object    string    `json:"object"`
	Embedding []float64 `json:"embedding"`
}

EmbeddingData represents the vector data points.

type EmbeddingInput added in v0.7.0 

type EmbeddingInput struct {
	Text  string
	Image Base64Encoder
}

EmbeddingInput represents text and an image for embedding.

type EmbeddingInputTypes added in v0.31.0 

type EmbeddingInputTypes interface {
	EmbedInputType()
}

EmbeddingInputTypes defines behavior any embedding input type must implement. The method doesn't need to do anything, it just needs to exist.

type EmbeddingInputs added in v0.31.0 

type EmbeddingInputs []EmbeddingInput

EmbeddingInputs represents a collection of EmbeddingInput.

func (EmbeddingInputs) EmbedInputType added in v0.31.0 

func (EmbeddingInputs) EmbedInputType()

EmbedInputType implements the EmbeddingInputTypes interface.

type EmbeddingIntInputs added in v0.31.0 

type EmbeddingIntInputs [][]int

EmbeddingIntInputs represents the input to generate embeddings.

func (EmbeddingIntInputs) EmbedInputType added in v0.31.0 

func (EmbeddingIntInputs) EmbedInputType()

EmbedInputType implements the EmbeddingInputTypes interface.

type Error 

type Error struct {
	Message string `json:"error"`
}

Error represents an error in the system.

func (*Error) Error 

func (err *Error) Error() string

Error implements the error interface.

type Factuality 

type Factuality struct {
	ID      string            `json:"id"`
	Object  string            `json:"object"`
	Created Time              `json:"created"`
	Checks  []FactualityCheck `json:"checks"`
}

Factuality represents the result for the factuality call.

type FactualityCheck 

type FactualityCheck struct {
	Score float64 `json:"score"`
	Index int     `json:"index"`
}

FactualityCheck represents the result for the factuality call.

type ImageBase64 added in v0.7.0 

type ImageBase64 struct {
	// contains filtered or unexported fields
}

ImageBase64 represents image data that is already in base64.

func NewImageBase64 added in v0.7.0 

func NewImageBase64(base64 string) ImageBase64

NewImageBase64 constructs a ImageBase64 with an encoded image.

func (ImageBase64) EncodeBase64 added in v0.7.0 

func (img ImageBase64) EncodeBase64(ctx context.Context) (string, error)

EncodeBase64 returns the base64 image provided during construction.

type ImageFile added in v0.7.0 

type ImageFile struct {
	// contains filtered or unexported fields
}

ImageFile represents image data that will be read from a file.

func NewImageFile added in v0.7.0 

func NewImageFile(imagePath string) (ImageFile, error)

NewImageFile constructs a ImageFile that can read an image from disk.

func (ImageFile) EncodeBase64 added in v0.7.0 

func (img ImageFile) EncodeBase64(ctx context.Context) (string, error)

EncodeBase64 reads the specified image from disk and converts the image to a base64 string.

type ImageNetwork added in v0.7.0 

type ImageNetwork struct {
	// contains filtered or unexported fields
}

ImageNetwork represents image data that will be read from the network.

func NewImageNetwork added in v0.7.0 

func NewImageNetwork(imageURL string) (ImageNetwork, error)

NewImageNetwork constructs a ImageNetwork that can read an image from the network.

func (ImageNetwork) EncodeBase64 added in v0.7.0 

func (img ImageNetwork) EncodeBase64(ctx context.Context) (string, error)

EncodeBase64 reads the specified image from the network and converts the image to a base64 string.

type Injection 

type Injection struct {
	ID      string           `json:"id"`
	Object  string           `json:"object"`
	Created Time             `json:"created"`
	Checks  []InjectionCheck `json:"checks"`
}

Injection represents the result for the injection call.

type InjectionCheck 

type InjectionCheck struct {
	Probability float64 `json:"probability"`
	Index       int     `json:"index"`
	Status      string  `json:"status"`
}

InjectionCheck represents the result for the injection call.

type InputExtension added in v0.25.0 

type InputExtension struct {
	BlockPromptInjection bool
	PII                  PII
	PIIReplaceMethod     ReplaceMethod
}

InputExtension represents options for pre-processing.

type Language 

type Language struct {
	// contains filtered or unexported fields
}

Language represents a language in the system.

func (Language) Equal 

func (l Language) Equal(l2 Language) bool

Equal provides support for the go-cmp package and testing.

func (Language) MarshalText 

func (l Language) MarshalText() ([]byte, error)

MarshalText implement the marshal interface for JSON conversions.

func (Language) String added in v0.7.0 

func (l Language) String() string

String returns the ISO-639 code of the language.

func (*Language) UnmarshalText 

func (l *Language) UnmarshalText(data []byte) error

UnmarshalText implement the unmarshal interface for JSON conversions.

type Logger 

type Logger func(context.Context, string, ...any)

Logger represents a function that will be called to add information to the user's application logs.

type ModelCapabilities added in v0.32.0 

type ModelCapabilities struct {
	ChatCompletion     bool `json:"chat_completion"`
	ChatWithImage      bool `json:"chat_with_image"`
	Completion         bool `json:"completion"`
	Embedding          bool `json:"embedding"`
	EmbeddingWithImage bool `json:"embedding_with_image"`
	Tokenize           bool `json:"tokenize"`
}

type ModelData added in v0.32.0 

type ModelData struct {
	ID               string            `json:"id"`
	Object           string            `json:"object"`
	Created          time.Time         `json:"created"`
	OwnedBy          string            `json:"owned_by"`
	Description      string            `json:"description"`
	MaxContextLength int               `json:"max_context_length"`
	PromptFormat     string            `json:"prompt_format"`
	Capabilities     ModelCapabilities `json:"capabilities"`
}

type ModelResponse added in v0.32.0 

type ModelResponse struct {
	Object string      `json:"object"`
	Data   []ModelData `json:"data"`
}

type OutputExtension added in v0.25.0 

type OutputExtension struct {
	Factuality bool
	Toxicity   bool
}

OutputExtension represents options for post-processing.

type PII added in v0.8.0 

type PII struct {
	// contains filtered or unexported fields
}

PII represents a PII in the system.

func (PII) Equal added in v0.8.0 

func (p PII) Equal(p2 PII) bool

Equal provides support for the go-cmp package and testing.

func (PII) MarshalText added in v0.8.0 

func (p PII) MarshalText() ([]byte, error)

MarshalText implement the marshal interface for JSON conversions.

func (PII) String added in v0.8.0 

func (p PII) String() string

String returns the name of the PII.

func (*PII) UnmarshalText added in v0.8.0 

func (p *PII) UnmarshalText(data []byte) error

UnmarshalText implement the unmarshal interface for JSON conversions.

type ReplaceMethod 

type ReplaceMethod struct {
	// contains filtered or unexported fields
}

ReplaceMethod represents a replace method in the system.

func (ReplaceMethod) Equal 

func (rm ReplaceMethod) Equal(rm2 ReplaceMethod) bool

Equal provides support for the go-cmp package and testing.

func (ReplaceMethod) MarshalText 

func (rm ReplaceMethod) MarshalText() ([]byte, error)

MarshalText implement the marshal interface for JSON conversions.

func (ReplaceMethod) String added in v0.7.0 

func (rm ReplaceMethod) String() string

String returns the name of the replace method.

func (*ReplaceMethod) UnmarshalText 

func (rm *ReplaceMethod) UnmarshalText(data []byte) error

UnmarshalText implement the unmarshal interface for JSON conversions.

type ReplacePII added in v0.11.0 

type ReplacePII struct {
	ID      string            `json:"id"`
	Object  string            `json:"object"`
	Created Time              `json:"created"`
	Checks  []ReplacePIICheck `json:"checks"`
}

ReplacePII represents the result for the pii call.

type ReplacePIICheck added in v0.11.0 

type ReplacePIICheck struct {
	NewPrompt string `json:"new_prompt"`
	Index     int    `json:"index"`
	Status    string `json:"status"`
}

ReplacePIICheck represents the result for the pii call.

type Role 

type Role struct {
	// contains filtered or unexported fields
}

Role represents a role in the system.

func (Role) Equal 

func (r Role) Equal(r2 Role) bool

Equal provides support for the go-cmp package and testing.

func (Role) MarshalText 

func (r Role) MarshalText() ([]byte, error)

MarshalText implement the marshal interface for JSON conversions.

func (Role) String added in v0.7.0 

func (r Role) String() string

String returns the name of the role.

func (*Role) UnmarshalText 

func (r *Role) UnmarshalText(data []byte) error

UnmarshalText implement the unmarshal interface for JSON conversions.

type Time 

type Time struct {
	time.Time
}

Time supports the ability to marshal and unmarshal unit epoch time.

func ToTime 

func ToTime(sec int64) Time

ToTime returns the local Time corresponding to the given Unix time, sec seconds and nsec nanoseconds since January 1, 1970 UTC. It is valid to pass nsec outside the range [0, 999999999]. Not all sec values have a corresponding time value. One such value is 1<<63-1 (the largest int64 value).

func (Time) MarshalJSON 

func (t Time) MarshalJSON() ([]byte, error)

MarshalJSON overrides the time.Time implementation so we can marshal to epoch time.

func (*Time) UnmarshalJSON 

func (t *Time) UnmarshalJSON(data []byte) error

UnmarshalJSON overrides the time.Time implementation so we can unmarshal from epoch time.

type TokenData added in v0.31.0 

type TokenData struct {
	ID    int    `json:"id"`
	Start int    `json:"start"`
	Stop  int    `json:"stop"`
	Text  string `json:"text"`
}

TokenData represents a single token of information.

type Tokenize added in v0.31.0 

type Tokenize struct {
	ID      string      `json:"id"`
	Object  string      `json:"object"`
	Created Time        `json:"created"`
	Data    []TokenData `json:"data"`
}

Tokenize represents the result for the toxicity call.

type TokenizeInput added in v0.31.0 

type TokenizeInput struct {
	Model string
	Input string
}

TokenizeInput represents input for tokenizing text.

type Toxicity 

type Toxicity struct {
	ID      string          `json:"id"`
	Object  string          `json:"object"`
	Created Time            `json:"created"`
	Checks  []ToxicityCheck `json:"checks"`
}

Toxicity represents the result for the toxicity call.

type ToxicityCheck 

type ToxicityCheck struct {
	Score float64 `json:"score"`
	Index int     `json:"index"`
}

ToxicityCheck represents the result for the toxicity call.

type Translate 

type Translate struct {
	ID                   string        `json:"id"`
	Object               string        `json:"object"`
	Created              Time          `json:"created"`
	BestTranslation      string        `json:"best_translation"`
	BestTranslationModel string        `json:"best_translation_model"`
	Score                float64       `json:"best_score"`
	Translations         []Translation `json:"translations"`
}

Translate represents the result for the translate call.

type Translation 

type Translation struct {
	Score       float64 `json:"score"`
	Translation string  `json:"translation"`
	Model       string  `json:"model"`
	Status      string  `json:"status"`
}

Translation represents the result for the translate call.

Source Files

View all Source files

Directories

Path Synopsis
examples
capability
chat/basic
chat/multi
chat/sse
chat/vision
completion
embedding/basic
embedding/ints
embedding/truncate
factuality
health_check
injection
replacepi
tokenize
toxicity
translate

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.