sd

README

stable-diffusion

pure go ( cgo free ) for stable-diffusion and support cross-platform.

sd.go is a wrapper around stable-diffusion.cpp, which is an adaption of ggml.cpp.

Installation

go get github.com/seasonjs/stable-diffusion

AutoModel Compatibility

See deps folder for dylib compatibility, push request is welcome.

Windows NVIDIA GPU User may need check cuda architecture to get more information.

Windows AMD/ROCM GPU User may need check system requirements to get more information.

platform	x32	x64	arm	AMD/ROCM	NVIDIA/CUDA
windows	not support	support avx/avx2/avx512	not support	rocm5.5 support	cuda12 support
linux	not support	support	not support	not support	not support
darwin	not support	support	support	not support	not support

AutoModel Dynamic Libraries Disclaimer

The Source of dynamic Libraries

These dynamic libraries come from stable-diffusion.cpp-build release, The dynamic library version can be obtained by viewing stable-diffusion.version file Anyone can check the consistency of the file by checksum ( MD5 ).

The Security Of Dynamic Libraries

All I can say is that the creation of the dynamic library is public and does not contain any subjective malicious logic. If you are worried about the security of the dynamic library during the use process, you can build it yourself.

I and any author related to dynamic libraries do not assume any problems, responsibilities or legal liability during use.

Usage

This stable-diffusion golang library provide two api Predict and ImagePredict.

Usually you can use NewAutoModel, so you don't need to load the dynamic library.

You can find a complete example in examples folder.

Here is a simple example:

package main

import (
	"github.com/seasonjs/hf-hub/api"
	sd "github.com/seasonjs/stable-diffusion"
	"io"
	"os"
)

func main() {
	options := sd.DefaultOptions

	model, err := sd.NewAutoModel(options)
	if err != nil {
		print(err.Error())
		return
	}
	defer model.Close()

	hapi, err := api.NewApi()
	if err != nil {
		print(err.Error())
		return
	}

	modelPath, err := hapi.Model("justinpinkney/miniSD").Get("miniSD.ckpt")
	if err != nil {
		print(err.Error())
		return
	}

	err = model.LoadFromFile(modelPath)
	if err != nil {
		print(err.Error())
		return
	}
	var writers []io.Writer
	filenames := []string{
		"../assets/love_cat0.png",
	}
	for _, filename := range filenames {
		file, err := os.Create(filename)
		if err != nil {
			print(err.Error())
			return
		}
		defer file.Close()
		writers = append(writers, file)
	}

	err = model.Predict("british short hair cat, high quality", sd.DefaultFullParams, writers)
	if err != nil {
		print(err.Error())
	}
}

Packaging

To ship a working program that includes this AI, you will need to include the following files:

• libstable-diffusion.dylib / libstable-diffusion.so / stable-diffusion.dll (buildin)
• the model file
• the tokenizer file (buildin)
• cuda runtime library, if you use cuda

Low level API

This package also provide low level Api which is same as stable-diffusion-cpp. See detail at stable-diffusion-doc.

Thanks

• stable-diffusion-cpp
• ggml.cpp
• purego

Successful Examples

License

Copyright (c) seasonjs. All rights reserved. Licensed under the MIT License. See License.txt in the project root for license information.

Documentation

Index

• Constants
• Variables
• type CLogCallback
• type CStableDiffusion
• type CStableDiffusionCtx
• type CStableDiffusionImpl
  • func NewCStableDiffusion(libraryPath string) (*CStableDiffusionImpl, error)
  • func (c *CStableDiffusionImpl) Close() error
  • func (c *CStableDiffusionImpl) FreeCtx(ctx *CStableDiffusionCtx)
  • func (c *CStableDiffusionImpl) FreeUpscalerCtx(ctx *CUpScalerCtx)
  • func (c *CStableDiffusionImpl) GetSystemInfo() string
  • func (c *CStableDiffusionImpl) ImagePredictImage(ctx *CStableDiffusionCtx, img Image, prompt string, negativePrompt string, ...) []Image
  • func (c *CStableDiffusionImpl) NewCtx(modelPath string, vaePath string, taesdPath string, loraModelDir string, ...) *CStableDiffusionCtx
  • func (c *CStableDiffusionImpl) NewUpscalerCtx(esrganPath string, nThreads int, wType WType) *CUpScalerCtx
  • func (c *CStableDiffusionImpl) PredictImage(ctx *CStableDiffusionCtx, prompt string, negativePrompt string, clipSkip int, ...) []Image
  • func (c *CStableDiffusionImpl) SetLogCallBack(cb CLogCallback)
  • func (c *CStableDiffusionImpl) UpscaleImage(ctx *CUpScalerCtx, img Image, upscaleFactor uint32) Image
• type CUpScalerCtx
• type FullParams
• type Image
• type LogLevel
• type Model
  • func NewAutoModel(options Options) (*Model, error)
  • func NewModel(dylibPath string, options Options) (*Model, error)
  • func (sd *Model) Close() error
  • func (sd *Model) ImagePredict(reader io.Reader, prompt string, params FullParams, writer []io.Writer) error
  • func (sd *Model) LoadFromFile(path string) error
  • func (sd *Model) Predict(prompt string, params FullParams, writer []io.Writer) error
  • func (sd *Model) SetLogCallback(cb CLogCallback)
  • func (sd *Model) SetOptions(options Options)
  • func (sd *Model) UpscaleImage(reader io.Reader, esrganPath string, upscaleFactor uint32, writer io.Writer) error
• type Options
• type OutputsImageType
• type RNGType
• type SampleMethod
• type Schedule
• type WType

Constants

const (
	F32   WType = 0
	F16         = 1
	Q4_0        = 2
	Q4_1        = 3
	Q5_0        = 6
	Q5_1        = 7
	Q8_0        = 8
	Q8_1        = 9
	Q2_K        = 10
	Q3_K        = 11
	Q4_K        = 12
	Q5_K        = 13
	Q6_K        = 14
	Q8_K        = 15
	I8          = 16
	I16         = 17
	I32         = 18
	COUNT       = 19 // don't use this when specifying a type
)

Variables

var DefaultFullParams = FullParams{
	NegativePrompt:   "out of frame, lowers, text, error, cropped, worst quality, low quality, jpeg artifacts, ugly, duplicate, morbid, mutilated, out of frame, extra fingers, mutated hands, poorly drawn hands, poorly drawn face, mutation, deformed, blurry, dehydrated, bad anatomy, bad proportions, extra limbs, cloned face, disfigured, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, fused fingers, too many fingers, long neck, username, watermark, signature",
	CfgScale:         7.0,
	Width:            512,
	Height:           512,
	SampleMethod:     EULER_A,
	SampleSteps:      20,
	Strength:         0.4,
	Seed:             42,
	BatchCount:       1,
	OutputsImageType: PNG,
}

var DefaultOptions = Options{
	Threads:               -1,
	VaeDecodeOnly:         true,
	FreeParamsImmediately: true,
	RngType:               CUDA_RNG,
	Wtype:                 F32,
	Schedule:              DEFAULT,
}

Types

type CLogCallback

type CLogCallback func(level LogLevel, text string)

type CStableDiffusion

type CStableDiffusion interface {
	NewCtx(modelPath string, vaePath string, taesdPath string, loraModelDir string, vaeDecodeOnly bool, vaeTiling bool, freeParamsImmediately bool, nThreads int, wType WType, rngType RNGType, schedule Schedule) *CStableDiffusionCtx
	PredictImage(ctx *CStableDiffusionCtx, prompt string, negativePrompt string, clipSkip int, cfgScale float32, width int, height int, sampleMethod SampleMethod, sampleSteps int, seed int64, batchCount int) []Image
	ImagePredictImage(ctx *CStableDiffusionCtx, img Image, prompt string, negativePrompt string, clipSkip int, cfgScale float32, width int, height int, sampleMethod SampleMethod, sampleSteps int, strength float32, seed int64, batchCount int) []Image
	SetLogCallBack(cb CLogCallback)
	GetSystemInfo() string
	FreeCtx(ctx *CStableDiffusionCtx)

	NewUpscalerCtx(esrganPath string, nThreads int, wType WType) *CUpScalerCtx
	FreeUpscalerCtx(ctx *CUpScalerCtx)
	UpscaleImage(ctx *CUpScalerCtx, img Image, upscaleFactor uint32) Image

	Close() error
}

type CStableDiffusionCtx

type CStableDiffusionCtx struct {
	// contains filtered or unexported fields
}

type CStableDiffusionImpl

type CStableDiffusionImpl struct {
	// contains filtered or unexported fields
}

func NewCStableDiffusion

func NewCStableDiffusion(libraryPath string) (*CStableDiffusionImpl, error)

func (*CStableDiffusionImpl) Close

func (c *CStableDiffusionImpl) Close() error

func (*CStableDiffusionImpl) FreeCtx

func (c *CStableDiffusionImpl) FreeCtx(ctx *CStableDiffusionCtx)

func (*CStableDiffusionImpl) FreeUpscalerCtx

func (c *CStableDiffusionImpl) FreeUpscalerCtx(ctx *CUpScalerCtx)

func (*CStableDiffusionImpl) GetSystemInfo

func (c *CStableDiffusionImpl) GetSystemInfo() string

func (*CStableDiffusionImpl) ImagePredictImage

func (c *CStableDiffusionImpl) ImagePredictImage(ctx *CStableDiffusionCtx, img Image, prompt string, negativePrompt string, clipSkip int, cfgScale float32, width int, height int, sampleMethod SampleMethod, sampleSteps int, strength float32, seed int64, batchCount int) []Image

func (*CStableDiffusionImpl) NewCtx

func (c *CStableDiffusionImpl) NewCtx(modelPath string, vaePath string, taesdPath string, loraModelDir string, vaeDecodeOnly bool, vaeTiling bool, freeParamsImmediately bool, nThreads int, wType WType, rngType RNGType, schedule Schedule) *CStableDiffusionCtx

func (*CStableDiffusionImpl) NewUpscalerCtx

func (c *CStableDiffusionImpl) NewUpscalerCtx(esrganPath string, nThreads int, wType WType) *CUpScalerCtx

func (*CStableDiffusionImpl) PredictImage

func (c *CStableDiffusionImpl) PredictImage(ctx *CStableDiffusionCtx, prompt string, negativePrompt string, clipSkip int, cfgScale float32, width int, height int, sampleMethod SampleMethod, sampleSteps int, seed int64, batchCount int) []Image

func (*CStableDiffusionImpl) SetLogCallBack

func (c *CStableDiffusionImpl) SetLogCallBack(cb CLogCallback)

func (*CStableDiffusionImpl) UpscaleImage

func (c *CStableDiffusionImpl) UpscaleImage(ctx *CUpScalerCtx, img Image, upscaleFactor uint32) Image

type CUpScalerCtx

type CUpScalerCtx struct {
	// contains filtered or unexported fields
}

type FullParams

type FullParams struct {
	NegativePrompt   string
	ClipSkip         int
	CfgScale         float32
	Width            int
	Height           int
	SampleMethod     SampleMethod
	SampleSteps      int
	Strength         float32
	Seed             int64
	BatchCount       int
	OutputsImageType OutputsImageType
}

type Image

type Image struct {
	Width   uint32
	Height  uint32
	Channel uint32
	Data    []byte
}

type LogLevel

type LogLevel int

const (
	DEBUG LogLevel = iota
	INFO
	WARN
	ERROR
)

type Model

type Model struct {
	// contains filtered or unexported fields
}

func NewAutoModel

func NewAutoModel(options Options) (*Model, error)

func NewModel

func NewModel(dylibPath string, options Options) (*Model, error)

func (*Model) Close

func (sd *Model) Close() error

func (*Model) ImagePredict

func (sd *Model) ImagePredict(reader io.Reader, prompt string, params FullParams, writer []io.Writer) error

func (*Model) LoadFromFile

func (sd *Model) LoadFromFile(path string) error

func (*Model) Predict

func (sd *Model) Predict(prompt string, params FullParams, writer []io.Writer) error

func (*Model) SetLogCallback

func (sd *Model) SetLogCallback(cb CLogCallback)

func (*Model) SetOptions

func (sd *Model) SetOptions(options Options)

func (*Model) UpscaleImage

func (sd *Model) UpscaleImage(reader io.Reader, esrganPath string, upscaleFactor uint32, writer io.Writer) error

type Options

type Options struct {
	VaePath               string
	TaesdPath             string
	LoraModelDir          string
	VaeDecodeOnly         bool
	VaeTiling             bool
	FreeParamsImmediately bool
	Threads               int
	Wtype                 WType
	RngType               RNGType
	Schedule              Schedule
	GpuEnable             bool
}

type OutputsImageType

type OutputsImageType string

const (
	PNG  OutputsImageType = "PNG"
	JPEG                  = "JPEG"
)

type RNGType

type RNGType int

const (
	STD_DEFAULT_RNG RNGType = iota
	CUDA_RNG
)

type SampleMethod

type SampleMethod int

const (
	EULER_A SampleMethod = iota
	EULER
	HEUN
	DPM2
	DPMPP2S_A
	DPMPP2M
	DPMPP2Mv2
	LCM
	N_SAMPLE_METHODS
)

type Schedule

type Schedule int

const (
	DEFAULT Schedule = iota
	DISCRETE
	KARRAS
	N_SCHEDULES
)

type WType

type WType int