pretrained

Documentation

Index

• Variables
• type Config
• type Model
• type Tokenizer

Variables

var BertConfigs map[string]string = map[string]string{
	"bert-base-uncased":       "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-config.json",
	"bert-ner":                "https://cdn.huggingface.co/dbmdz/bert-large-cased-finetuned-conll03-english/config.json",
	"bert-qa":                 "https://s3.amazonaws.com/models.huggingface.co/bert/bert-large-cased-whole-word-masking-finetuned-squad-config.json",
	"indo-sentence-bert-base": "https://huggingface.co/firqaaa/indo-sentence-bert-base/raw/main/config.json",

	"roberta-base":       "https://cdn.huggingface.co/roberta-base-config.json",
	"roberta-qa":         "https://s3.amazonaws.com/models.huggingface.co/bert/deepset/roberta-base-squad2/config.json",
	"xlm-roberta-ner-en": "https://s3.amazonaws.com/models.huggingface.co/bert/xlm-roberta-large-finetuned-conll03-english-config.json",
	"xlm-roberta-ner-de": "https://s3.amazonaws.com/models.huggingface.co/bert/xlm-roberta-large-finetuned-conll03-german-config.json",
	"xlm-roberta-ner-nl": "https://s3.amazonaws.com/models.huggingface.co/bert/xlm-roberta-large-finetuned-conll02-dutch-config.json",
	"xlm-roberta-ner-es": "https://s3.amazonaws.com/models.huggingface.co/bert/xlm-roberta-large-finetuned-conll02-spanish-config.json",
}

BertConfigs is a map of pretrained Bert configuration names to corresponding URLs.

var BertModels map[string]string = map[string]string{
	"bert-base-uncased":       "https://cdn.huggingface.co/bert-base-uncased-rust_model.ot",
	"bert-ner":                "https://cdn.huggingface.co/dbmdz/bert-large-cased-finetuned-conll03-english/rust_model.ot",
	"bert-qa":                 "https://cdn.huggingface.co/bert-large-cased-whole-word-masking-finetuned-squad-rust_model.ot",
	"indo-sentence-bert-base": "https://huggingface.co/firqaaa/indo-sentence-bert-base/resolve/main/pytorch_model.bin",
}

BertModels is a map of pretrained Bert model names to corresponding URLs.

var BertVocabs map[string]string = map[string]string{
	"bert-base-uncased":       "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-vocab.txt",
	"bert-ner":                "https://cdn.huggingface.co/dbmdz/bert-large-cased-finetuned-conll03-english/vocab.txt",
	"bert-qa":                 "https://cdn.huggingface.co/bert-large-cased-whole-word-masking-finetuned-squad-vocab.txt",
	"indo-sentence-bert-base": "https://huggingface.co/firqaaa/indo-sentence-bert-base/raw/main/vocab.txt",
}

BertVocabs is a map of BERT model vocab name to corresponding URLs.

var RobertaConfigs map[string]string = map[string]string{
	"roberta-base":       "https://cdn.huggingface.co/roberta-base-config.json",
	"roberta-qa":         "https://s3.amazonaws.com/models.huggingface.co/bert/deepset/roberta-base-squad2/config.json",
	"xlm-roberta-ner-en": "https://s3.amazonaws.com/models.huggingface.co/bert/xlm-roberta-large-finetuned-conll03-english-config.json",
	"xlm-roberta-ner-de": "https://s3.amazonaws.com/models.huggingface.co/bert/xlm-roberta-large-finetuned-conll03-german-config.json",
	"xlm-roberta-ner-nl": "https://s3.amazonaws.com/models.huggingface.co/bert/xlm-roberta-large-finetuned-conll02-dutch-config.json",
	"xlm-roberta-ner-es": "https://s3.amazonaws.com/models.huggingface.co/bert/xlm-roberta-large-finetuned-conll02-spanish-config.json",
}

RobertaConfigs is a map of pretrained Roberta configuration names to corresponding URLs.

var RobertaMerges map[string]string = map[string]string{
	"roberta-base": "https://cdn.huggingface.co/roberta-base-merges.txt",
	"roberta-qa":   "https://cdn.huggingface.co/deepset/roberta-base-squad2/merges.txt",
}

RobertaMerges is a map of pretrained Roberta vocab merges name to corresponding URLs.

var RobertaModels map[string]string = map[string]string{
	"roberta-base":       "https://cdn.huggingface.co/roberta-base-rust_model.ot",
	"roberta-qa":         "https://cdn.huggingface.co/deepset/roberta-base-squad2/rust_model.ot",
	"xlm-roberta-ner-en": "https://cdn.huggingface.co/xlm-roberta-large-finetuned-conll03-english-rust_model.ot",
	"xlm-roberta-ner-de": "https://cdn.huggingface.co/xlm-roberta-large-finetuned-conll03-german-rust_model.ot",
	"xlm-roberta-ner-nl": "https://cdn.huggingface.co/xlm-roberta-large-finetuned-conll02-dutch-rust_model.ot",
	"xlm-roberta-ner-es": "https://cdn.huggingface.co/xlm-roberta-large-finetuned-conll02-spanish-rust_model.ot",
}

RobertaModels is a map of pretrained Roberta model names to corresponding URLs.

var RobertaVocabs map[string]string = map[string]string{
	"roberta-base":       "https://cdn.huggingface.co/roberta-base-vocab.json",
	"roberta-qa":         "https://cdn.huggingface.co/deepset/roberta-base-squad2/vocab.json",
	"xlm-roberta-ner-en": "https://cdn.huggingface.co/xlm-roberta-large-finetuned-conll03-english-sentencepiece.bpe.model",
	"xlm-roberta-ner-de": "https://cdn.huggingface.co/xlm-roberta-large-finetuned-conll03-german-sentencepiece.bpe.model",
	"xlm-roberta-ner-nl": "https://cdn.huggingface.co/xlm-roberta-large-finetuned-conll02-dutch-sentencepiece.bpe.model",
	"xlm-roberta-ner-es": "https://cdn.huggingface.co/xlm-roberta-large-finetuned-conll02-spanish-sentencepiece.bpe.model",
}

RobertaVocabs is a map of pretrained Roberta vocab name to corresponding URLs.

Types

type Config

type Config interface {
	Load(modelNamOrPath string, params map[string]interface{}) error
}

Config is an interface for pretrained model configuration. It has only one method `Load(string) error` to load configuration from local or remote file.

type Model

type Model interface {
	Load(modelNamOrPath string, config interface{ Config }, params map[string]interface{}, device gotch.Device) error
}

Model is an interface for pretrained model. It has only one method `Load(string) error` to load model from local or remote file.

type Tokenizer

type Tokenizer interface {
	Load(modelNamOrPath string, params map[string]interface{}) error
}