hatchet

package module
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 21, 2024 License: Apache-2.0 Imports: 42 Imported by: 0

README

Hatchet - MongoDB JSON Log Analyzer and Viewer

Hatchet is a powerful and sophisticated logs analyzer and viewer specifically designed for MongoDB JSON logs. It provides advanced features for logs processing, aggregation and storage of the processed data. To make the data accessible and usable for its users, Hatchet utilizes an embedded SQLite3 database. This database allows for the storage of processed and aggregated data and makes it possible to offer RESTful APIs and a web interface to users.

The web interface of Hatchet is highly interactive and user-friendly, providing a seamless experience for searching logs and navigating through reports and charts. The intuitive design and easy-to-use interface makes it simple for users to find what they need, when they need it. Additionally, with the embedded database, Hatchet provides fast access to data and a high level of performance, making it the ideal solution for logs analysis and management. Further design details can be found at .

Change Log

Build

Clone and run the build.sh script; gcc is required to support CGO.

git clone --depth 1 https://github.com/simagix/hatchet.git
cd hatchet ; ./build.sh

An executable hatchet is output to the directory dist/. Note that the script also works and tested on Windows x64 using MingGW and Git Bash.

Quick Start

Use the command below to process a log file, mongod.log.gz and start a web server listening to port 3721. The default database is SQLite3.

./dist/hatchet -web logs/sample-mongod.log.gz

Load a file within a defined time range:

./dist/hatchet -web -from "2023-09-23T20:25:00" -to "2023-09-23T20:26:00" logs/sample-mongod.log.gz

Load multiple files and process them individually:

./dist/hatchet -web rs1/mongod.log rs2/mongod.log rs3/mongod.log

Load multiple files and process them collectively:

./dist/hatchet -web -merge rs1/mongod.log rs2/mongod.log rs3/mongod.log

Use the URL http://localhost:3721/ in a browser to view reports and charts. Alternatively, you can use the in-memory mode without persisting data, for example:

./dist/hatchet -url in-memory logs/sample-mongod.log.gz

if you choose to view in the legacy format without a browser, use the command below:

./dist/hatchet -legacy logs/sample-mongod.log.gz

For additional usages and integration details, see developer's guide.

A Smart Log Analyzer

How smart Hatchet is? A picture is worth a thousand words.

Sage Says

Other Usages

Other than its ability to read from files, Hatchet offers additional functionality that includes reading from S3 and web servers, as well as MongoDB Atlas. This means that users can use Hatchet to conveniently access and download data from these sources, providing a more versatile and efficient data analysis experience.

Web Servers

The tool supports reading from web servers using both the http:// and https:// protocols. The -user flag is optional when using basic authentication.

hatchet [-user {username}:{password}] https://{hostname}/{log name}
Atlas

To download logs directly from MongoDB Atlas, you will need to use the -user and -digest flags and provide the necessary information for both. These flags are used to authenticate and authorize your access to the database.

hatchet -user {pub key}:{private key} -digest https://cloud.mongodb.com/api/atlas/v1.0/groups/{group ID}/clusters/{hostname}/logs/mongodb.gz
AWS S3

Hatchet has the ability to download files from AWS S3. When downloading files, Hatchet will automatically retrieve the Region and Credentials information from the configuration files located at ${HOME}/.aws. This means that there's no need to provide this information manually each time you download files from AWS S3 using Hatchet.

hatchet -s3 [--endpoint-url {test endpoint}] {bucket}/{key name}

Logs Obfuscation

Use Hatchet to obfuscate logs. It automatically obfuscates the values of the matched patterns under the "attr" field, such as SSN, credit card numbers, phone numbers, email addresses, IP addresses, FQDNs, port numbers, namespaces, and other numbers. Note that, for example, replacing "host.example.com" with "rose.taipei.com" in the log file will consistently replace all other occurrences of "host.example.com" with "rose.taipei.com". To obfuscate logs and redirect them to a file, use the following syntax:

hatchet -obfuscate {log file} > {output file}

License

Apache-2.0 License

Documentation

Overview

Copyright 2022-present Kuei-chun Chen. All rights reserved.

Copyright 2022-present Kuei-chun Chen. All rights reserved. logs_template.go

Index

Constants

View Source 
const (
	BioDBName   = "hatchet"
	BioCollName = "bios"
)
View Source 
const (
	BAR_CHART    = "bar_chart"
	BUBBLE_CHART = "bubble_chart"
	PIE_CHART    = "pie_chart"

	T_OPS            = "ops"
	T_RESLEN_UP      = "reslen-ip"
	T_OPS_COUNTS     = "ops-counts"
	T_CONNS_ACCEPTED = "connections-accepted"
	T_CONNS_TIME     = "connections-time"
	T_CONNS_TOTAL    = "connections-total"
	T_RESLEN_NS      = "reslen-ns"
)
View Source 
const (
	SQLite3 = iota
	Mongo
)
View Source 
const (
	HTML_API_PREFIX = "/hatchets/"
	REST_API_PREFIX = "/api/hatchet/v1.0/hatchets/"
)
View Source 
const (
	CHEN_ICO    = `` /* 5716-byte string literal not displayed */
	SAGE_PNG    = `` /* 11196-byte string literal not displayed */
	SIMONE_PNG  = `` /* 18360-byte string literal not displayed */
	HATCHET_PNG = `` /* 2412-byte string literal not displayed */
)
View Source 
const (
	COLLSCAN   = "COLLSCAN"
	DOLLAR_CMD = "$cmd"
	LIMIT      = 100
	TOP_N      = 23
)
View Source 
const (
	MAX_DOC_SIZE = 16 * (1024 * 1024)
	BATCH_SIZE   = 1000
)
View Source 
const (
	MAX_SIZE  = 64
	TAIL_SIZE = 7
)
View Source 
const MIN_MONGO_VER = "5.0"
View Source 
const SQLITE3_FILE = "./data/hatchet.db"

Variables

View Source 
var (
	SEVERITIES = []string{"F", "E", "W", "I", "D", "D2"}
	SEVERITY_M = map[string]string{"F": "FATAL", "E": "ERROR", "W": "WARN", "I": "INFO",
		"D": "DEBUG", "D2": "DEBUG2"}
)

Functions

func APIHandler added in v0.3.1 

func APIHandler(w http.ResponseWriter, r *http.Request, params httprouter.Params)

APIHandler responds to API calls

func AddLegacyString 

func AddLegacyString(doc *Logv2Info) error

AddLegacyString converts log to legacy format

func BsonD2M added in v0.6.0 

func BsonD2M(d bson.D) bson.M

func ChartsHandler added in v0.3.1 

func ChartsHandler(w http.ResponseWriter, r *http.Request, params httprouter.Params)

ChartsHandler responds to charts API calls

func CheckDriverCompatibility added in v0.4.5 

func CheckDriverCompatibility(mongo string, driver string, version string) error

func CheckLuhn added in v0.4.5 

func CheckLuhn(card string) bool

func ContainsCreditCardNo added in v0.4.5 

func ContainsCreditCardNo(card string) bool

func ContainsEmailAddress added in v0.4.5 

func ContainsEmailAddress(email string) bool

func ContainsFQDN added in v0.4.5 

func ContainsFQDN(fqdn string) bool

func ContainsIP added in v0.4.5 

func ContainsIP(ip string) bool

func ContainsPhoneNo added in v0.4.5 

func ContainsPhoneNo(phoneNo string) bool

func CreateIndexes added in v0.5.0 

func CreateIndexes(db *sql.DB, hatchetName string) ([]string, error)

CreateIndexes returns init statement

func CreateTables added in v0.5.0 

func CreateTables(db *sql.DB, hatchetName string) ([]string, error)

CreateTables returns init statement

func DriverHandler added in v0.4.5 

func DriverHandler(w http.ResponseWriter, r *http.Request, params httprouter.Params)

DriverHandler responds to API calls

func EscapeString added in v0.2.0 

func EscapeString(value string) string

func FaviconHandler added in v0.3.1 

func FaviconHandler(w http.ResponseWriter, r *http.Request, params httprouter.Params)

func GetAuditTablesTemplate added in v0.3.1 

func GetAuditTablesTemplate() (*template.Template, error)

GetAuditTablesTemplate returns HTML

func GetBufioReader added in v0.3.3 

func GetBufioReader(data []byte) (*bufio.Reader, error)

func GetChartTemplate 

func GetChartTemplate(chartType string) (*template.Template, error)

GetChartTemplate returns HTML

func GetClientPreparedStmt added in v0.4.5 

func GetClientPreparedStmt(hatchetName string) string

GetClientPreparedStmt returns prepared statement of clients table

func GetDriverPreparedStmt added in v0.4.5 

func GetDriverPreparedStmt(hatchetName string) string

GetDriverPreparedStmt returns prepared statement of drivers table

func GetDriverVersions added in v0.4.5 

func GetDriverVersions(mongo string, driver string) ([]interface{}, error)

func GetDrivers added in v0.4.5 

func GetDrivers() *map[string]interface{}

GetDrivers returns *map[string]interface{} instance

func GetHTTPContent added in v0.3.3 

func GetHTTPContent(url, username, password string) (*bufio.Reader, error)

func GetHTTPDigestContent added in v0.3.3 

func GetHTTPDigestContent(url, user, secret string) (*bufio.Reader, error)

func GetHatchetPreparedStmt added in v0.4.5 

func GetHatchetPreparedStmt(hatchetName string) string

GetHatchetPreparedStmt returns prepared statement of the hatchet table

func GetHatchetSummary added in v0.2.0 

func GetHatchetSummary(info HatchetInfo) string

func GetLogTableTemplate 

func GetLogTableTemplate(attr string) (*template.Template, error)

GetLogTableTemplate returns HTML

func GetMarkerHTML added in v0.6.0 

func GetMarkerHTML(marker int) string

func GetMongoDateSubString added in v0.4.5 

func GetMongoDateSubString(start string, end string) bson.M

func GetOffsetLimit added in v0.2.0 

func GetOffsetLimit(str string) (int, int)

GetOffsetLimit returns offset, limit

func GetSQLDateSubString added in v0.4.5 

func GetSQLDateSubString(start string, end string) string

func GetStatsTableTemplate 

func GetStatsTableTemplate(collscan bool, orderBy string, download string) (*template.Template, error)

GetStatsTableTemplate returns HTML

func GetTablesTemplate 

func GetTablesTemplate() (*template.Template, error)

GetTablesTemplate returns HTML

func Handler added in v0.3.1 

func Handler(w http.ResponseWriter, r *http.Request, params httprouter.Params)

Handler responds to API calls

func Hello added in v0.3.1 

func Hello(w http.ResponseWriter, r *http.Request, ps httprouter.Params)

func Index added in v0.3.1 

func Index(w http.ResponseWriter, r *http.Request, _ httprouter.Params)

func InsertBiosIntoMongoDB added in v0.4.5 

func InsertBiosIntoMongoDB(uri string, numDocuments int) error

func IsNamespace added in v0.4.5 

func IsNamespace(ns string) bool

func IsSSN added in v0.4.5 

func IsSSN(s string) bool

func LogsHandler added in v0.3.1 

func LogsHandler(w http.ResponseWriter, r *http.Request, params httprouter.Params)

LogsHandler responds to charts API calls

func ObfuscateWord added in v0.4.5 

func ObfuscateWord(word string) string

func Run 

func Run(fullVersion string)

func SimulateReads added in v0.4.5 

func SimulateReads(url string) error

func SimulateTests added in v0.4.5 

func SimulateTests(test string, url string) error

func SimulateWrites added in v0.4.5 

func SimulateWrites(url string) error

func StatsHandler added in v0.3.1 

func StatsHandler(w http.ResponseWriter, r *http.Request, params httprouter.Params)

StatsHandler responds to API calls

func ToFloat64 added in v0.4.5 

func ToFloat64(num interface{}) float64

ToFloat64 converts to float64

func ToInt 

func ToInt(num interface{}) int

ToInt converts to int

Types

type Attributes 

type Attributes struct {
	Command            map[string]interface{} `json:"command" bson:"command"`
	ErrMsg             string                 `json:"errMsg" bson:"errMsg"`
	Milli              int                    `json:"durationMillis" bson:"durationMillis"`
	NS                 string                 `json:"ns" bson:"ns"`
	OriginatingCommand map[string]interface{} `json:"originatingCommand" bson:"originatingCommand"`
	PlanSummary        string                 `json:"planSummary" bson:"planSummary"`
	Reslen             int                    `json:"reslen" bson:"reslen"`
	Type               string                 `json:"type" bson:"type"`
}

type Bio added in v0.4.5 

type Bio struct {
	Age         int      `bson:"age" json:"age"`
	Company     string   `bson:"company" json:"company"`
	CreditCards []string `bson:"credit_cards" json:"credit_cards"`
	Emails      []string `bson:"emails" json:"emails"`
	FirstName   string   `bson:"first_name" json:"first_name"`
	Intro       string   `bson:"intro" json:"intro"`
	LastName    string   `bson:"last_name" json:"last_name"`
	Phones      []string `bson:"phones" json:"phones"`
	Title       string   `bson:"title" json:"title"`
	SSN         string   `bson:"ssn" json:"ssn"`
	State       string   `bson:"state" json:"state"`
	URL         string   `bson:"url" json:"url"`
}

type Chart 

type Chart struct {
	Index int
	Title string
	Descr string
	URL   string
}

type Database added in v0.2.0 

type Database interface {
	Begin() error
	Close() error
	Commit() error
	CreateMetaData() error
	Drop() error
	GetAcceptedConnsCounts(duration string) ([]NameValue, error)
	GetAuditData() (map[string][]NameValues, error)
	GetAverageOpTime(op string, duration string) ([]OpCount, error)
	GetConnectionStats(chartType string, duration string) ([]RemoteClient, error)
	GetHatchetInfo() HatchetInfo
	GetHatchetNames() ([]string, error)
	GetLogs(opts ...string) ([]LegacyLog, error)
	GetOpsCounts(duration string) ([]NameValue, error)
	GetReslenByNamespace(ip string, duration string) ([]NameValue, error)
	GetReslenByIP(ip string, duration string) ([]NameValue, error)
	GetSlowOps(orderBy string, order string, collscan bool) ([]OpStat, error)
	GetSlowestLogs(topN int) ([]LegacyLog, error)
	GetVerbose() bool
	InsertClientConn(index int, doc *Logv2Info) error
	InsertDriver(index int, doc *Logv2Info) error
	InsertFailedMessages(m *FailedMessages) error
	InsertLog(index int, end string, doc *Logv2Info, stat *OpStat) error
	SearchLogs(opts ...string) ([]LegacyLog, error)
	SetVerbose(v bool)
	UpdateHatchetInfo(info HatchetInfo) error
}

func GetDatabase added in v0.2.0 

func GetDatabase(hatchetName string) (Database, error)

type FailedMessages added in v0.5.0 

type FailedMessages struct {
	// contains filtered or unexported fields
}

type HatchetInfo added in v0.2.0 

type HatchetInfo struct {
	Arch    string `bson:"arch"`
	End     string `bson:"end"`
	Merge   bool   `bson:"merge"`
	Module  string `bson:"module"`
	Name    string `bson:"name"`
	OS      string `bson:"os"`
	Start   string `bson:"start"`
	Version string `bson:"version"`

	Drivers  []map[string]string
	Provider string `bson:"region"`
	Region   string `bson:"provider"`
}

type LegacyLog 

type LegacyLog struct {
	Timestamp string `json:"date" bson:"date"`
	Severity  string `json:"severity" bson:"severity"`
	Component string `json:"component" bson:"component"`
	Context   string `json:"context" bson:"context"`
	Marker    int
	Message   string `json:"message" bson:"message"` // remaining legacy message
}

type Logv2 

type Logv2 struct {
	// contains filtered or unexported fields
}

Logv2 keeps Logv2 object

func GetLogv2 

func GetLogv2() *Logv2

GetLogv2 returns Logv2 instance

func (*Logv2) Analyze 

func (ptr *Logv2) Analyze(logname string, marker int) error

Analyze analyzes logs from a file

func (*Logv2) GetDBType added in v0.4.5 

func (ptr *Logv2) GetDBType() int

func (*Logv2) PrintSummary 

func (ptr *Logv2) PrintSummary() error

type Logv2Info 

type Logv2Info struct {
	Attr      bson.D    `json:"attr" bson:"attr"`
	Component string    `json:"c" bson:"c"`
	Context   string    `json:"ctx" bson:"ctx"`
	ID        int       `json:"id" bson:"id"`
	Msg       string    `json:"msg" bson:"msg"`
	Severity  string    `json:"s" bson:"s"`
	Timestamp time.Time `json:"t" bson:"t"`

	Attributes Attributes
	Message    string // remaining legacy message
	Client     *RemoteClient
	Marker     int
}

Logv2Info stores logv2 struct

type MongoDB added in v0.4.5 

type MongoDB struct {
	// contains filtered or unexported fields
}

func NewMongoDB added in v0.4.5 

func NewMongoDB(connstr string, hatchetName string) (*MongoDB, error)

func (*MongoDB) Begin added in v0.4.5 

func (ptr *MongoDB) Begin() error

func (*MongoDB) Close added in v0.4.5 

func (ptr *MongoDB) Close() error

func (*MongoDB) Commit added in v0.4.5 

func (ptr *MongoDB) Commit() error

func (*MongoDB) CreateMetaData added in v0.4.5 

func (ptr *MongoDB) CreateMetaData() error

func (*MongoDB) Drop added in v0.4.5 

func (ptr *MongoDB) Drop() error

Drop drops all tables of a hatchet

func (*MongoDB) GetAcceptedConnsCounts added in v0.4.5 

func (ptr *MongoDB) GetAcceptedConnsCounts(duration string) ([]NameValue, error)

GetAcceptedConnsCounts returns opened connection counts

func (*MongoDB) GetAuditData added in v0.4.5 

func (ptr *MongoDB) GetAuditData() (map[string][]NameValues, error)

func (*MongoDB) GetAverageOpTime added in v0.4.5 

func (ptr *MongoDB) GetAverageOpTime(op string, duration string) ([]OpCount, error)

func (*MongoDB) GetConnectionStats added in v0.4.5 

func (ptr *MongoDB) GetConnectionStats(chartType string, duration string) ([]RemoteClient, error)

GetConnectionStats returns stats data of accepted and ended

func (*MongoDB) GetHatchetInfo added in v0.4.5 

func (ptr *MongoDB) GetHatchetInfo() HatchetInfo

func (*MongoDB) GetHatchetNames added in v0.4.5 

func (ptr *MongoDB) GetHatchetNames() ([]string, error)

func (*MongoDB) GetLogs added in v0.4.5 

func (ptr *MongoDB) GetLogs(opts ...string) ([]LegacyLog, error)

func (*MongoDB) GetOpsCounts added in v0.4.5 

func (ptr *MongoDB) GetOpsCounts(duration string) ([]NameValue, error)

GetOpsCounts returns opened connection counts

func (*MongoDB) GetReslenByIP added in v0.4.5 

func (ptr *MongoDB) GetReslenByIP(ip string, duration string) ([]NameValue, error)

GetReslenByIP returns total response length by ip

func (*MongoDB) GetReslenByNamespace added in v0.4.5 

func (ptr *MongoDB) GetReslenByNamespace(ns string, duration string) ([]NameValue, error)

GetReslenByNamespace returns total response length by ns

func (*MongoDB) GetSlowOps added in v0.4.5 

func (ptr *MongoDB) GetSlowOps(orderBy string, order string, collscan bool) ([]OpStat, error)

func (*MongoDB) GetSlowestLogs added in v0.4.5 

func (ptr *MongoDB) GetSlowestLogs(topN int) ([]LegacyLog, error)

func (*MongoDB) GetVerbose added in v0.4.5 

func (ptr *MongoDB) GetVerbose() bool

func (*MongoDB) InsertClientConn added in v0.4.5 

func (ptr *MongoDB) InsertClientConn(index int, doc *Logv2Info) error

func (*MongoDB) InsertDriver added in v0.4.5 

func (ptr *MongoDB) InsertDriver(index int, doc *Logv2Info) error

func (*MongoDB) InsertFailedMessages added in v0.5.0 

func (ptr *MongoDB) InsertFailedMessages(m *FailedMessages) error

func (*MongoDB) InsertLog added in v0.4.5 

func (ptr *MongoDB) InsertLog(index int, end string, doc *Logv2Info, stat *OpStat) error

func (*MongoDB) SearchLogs added in v0.4.5 

func (ptr *MongoDB) SearchLogs(opts ...string) ([]LegacyLog, error)

func (*MongoDB) SetVerbose added in v0.4.5 

func (ptr *MongoDB) SetVerbose(b bool)

func (*MongoDB) UpdateHatchetInfo added in v0.4.5 

func (ptr *MongoDB) UpdateHatchetInfo(info HatchetInfo) error

type NameValue 

type NameValue struct {
	Name  string `bson:"name"`
	Value int    `bson:"value"`
}

type NameValues added in v0.3.1 

type NameValues struct {
	Name   string
	Values []interface{}
}

type Obfuscation added in v0.4.5 

type Obfuscation struct {
	Coefficient float64           `json:"coefficient"`
	CardMap     map[string]string `json:"card_map"`

	IPMap   map[string]string `json:"ip_map"`
	NameMap map[string]string `json:"name_map"`

	PhoneMap map[string]string `json:"phone_map"`
	SSNMap   map[string]string `json:"ssn_map"`
	// contains filtered or unexported fields
}

func NewObfuscation added in v0.4.5 

func NewObfuscation() *Obfuscation

func (*Obfuscation) ObfuscateBsonA added in v0.4.5 

func (ptr *Obfuscation) ObfuscateBsonA(a bson.A) bson.A

func (*Obfuscation) ObfuscateBsonD added in v0.4.5 

func (ptr *Obfuscation) ObfuscateBsonD(d bson.D) bson.D

func (*Obfuscation) ObfuscateCreditCardNo added in v0.4.5 

func (ptr *Obfuscation) ObfuscateCreditCardNo(cardNo string) string

ObfuscateCreditCardNo obfuscate digits with '*' except for last 4 digits

func (*Obfuscation) ObfuscateEmail added in v0.4.5 

func (ptr *Obfuscation) ObfuscateEmail(email string) string

ObfuscateEmail replace domain name with a city name and user name with a flower name

func (*Obfuscation) ObfuscateFQDN added in v0.4.5 

func (ptr *Obfuscation) ObfuscateFQDN(fqdn string) string

ObfuscateFQDN returns a obfuscated FQDN

func (*Obfuscation) ObfuscateFile added in v0.4.5 

func (ptr *Obfuscation) ObfuscateFile(filename string) error

func (*Obfuscation) ObfuscateIP added in v0.4.5 

func (ptr *Obfuscation) ObfuscateIP(ip string) string

ObfuscateIP returns a new IP but keep the first and last octets the same

func (*Obfuscation) ObfuscateInt added in v0.4.5 

func (ptr *Obfuscation) ObfuscateInt(data interface{}) int

ObfuscateInt uses the original value times the coefficient

func (*Obfuscation) ObfuscateNS added in v0.4.5 

func (ptr *Obfuscation) ObfuscateNS(ns string) string

ObfuscateNS returns a obfuscated namespace

func (*Obfuscation) ObfuscateNumber added in v0.4.5 

func (ptr *Obfuscation) ObfuscateNumber(data interface{}) float64

ObfuscateNumber uses the original value times the coefficient

func (*Obfuscation) ObfuscatePhoneNo added in v0.4.5 

func (ptr *Obfuscation) ObfuscatePhoneNo(phoneNo string) string

ObfuscatePhoneNo returns a randome phone number with the same format

func (*Obfuscation) ObfuscateSSN added in v0.4.5 

func (ptr *Obfuscation) ObfuscateSSN(ssn string) string

ObfuscateSSN returns a obfuscated SSN

func (*Obfuscation) ObfuscateString added in v0.4.5 

func (ptr *Obfuscation) ObfuscateString(value string) string

type OpCount 

type OpCount struct {
	Date      string  `bson:"date"`
	Count     int     `bson:"count"`
	Milli     float64 `bson:"milli"`
	Op        string  `bson:"op"`
	Namespace string  `bson:"ns"`
	Filter    string  `bson:"filter"`
}

type OpStat 

type OpStat struct {
	AvgMilli     float64 `json:"avg_ms" bson:"avg_ms"`               // avg millisecond
	Count        int     `json:"count" bson:"count"`                 // number of ops
	Index        string  `json:"index" bson:"index"`                 // index used
	MaxMilli     int     `json:"max_ms" bson:"max_ms"`               // max millisecond
	Namespace    string  `json:"ns" bson:"ns"`                       // database.collectin
	Op           string  `json:"op" bson:"op"`                       // count, delete, find, remove, and update
	QueryPattern string  `json:"query_pattern" bson:"query_pattern"` // query pattern
	Reslen       int     `json:"total_reslen" bson:"total_reslen"`   // total reslen
	TotalMilli   int     `json:"total_ms" bson:"total_ms"`           // total milliseconds

	Marker int
}

OpStat stores performance data

func AnalyzeLog added in v0.2.0 

func AnalyzeLog(str string) (*OpStat, error)

AnalyzeLog analyzes slow op log

func AnalyzeSlowOp 

func AnalyzeSlowOp(doc *Logv2Info) (*OpStat, error)

AnalyzeSlowOp analyzes slow ops

type RemoteClient added in v0.3.1 

type RemoteClient struct {
	Accepted int    `json:"accepted" bson:"accepted"`
	Conns    int    `json:"conns" bson:"conns"`
	Ended    int    `json:"ended" bson:"ended"`
	IP       string `json:"value" bson:"ip"`
	Port     string `json:"port" bson:"port"`

	Driver  string `bsno:"driver"`  // driver name
	Version string `bsno:"version"` // driver version
}

type S3Client added in v0.3.3 

type S3Client struct {
	// contains filtered or unexported fields
}

S3Client provides methods to interact with an S3 service.

func NewS3Client added in v0.3.3 

func NewS3Client(profile string, params ...string) (*S3Client, error)

func (*S3Client) CreateBucket added in v0.3.3 

func (c *S3Client) CreateBucket(bucket string) error

CreateBucket creates a new S3 bucket.

func (*S3Client) DeleteBucket added in v0.3.3 

func (c *S3Client) DeleteBucket(bucket string) error

CreateBucket creates a new S3 bucket.

func (*S3Client) DeleteObject added in v0.3.3 

func (c *S3Client) DeleteObject(bucket, key string) error

DeleteObject deletes an object from S3.

func (*S3Client) GetObject added in v0.3.3 

func (c *S3Client) GetObject(logname string) ([]byte, error)

GetObject retrieves an object from S3 and returns its contents as a byte slice.

func (*S3Client) PutObject added in v0.3.3 

func (c *S3Client) PutObject(bucket, key, filePath string) error

PutObject uploads a file to S3.

type SQLite3DB added in v0.2.0 

type SQLite3DB struct {
	// contains filtered or unexported fields
}

func NewSQLite3DB added in v0.2.0 

func NewSQLite3DB(dbfile string, hatchetName string, cacheSize int) (*SQLite3DB, error)

func (*SQLite3DB) Begin added in v0.2.0 

func (ptr *SQLite3DB) Begin() error

func (*SQLite3DB) Close added in v0.2.0 

func (ptr *SQLite3DB) Close() error

func (*SQLite3DB) Commit added in v0.2.0 

func (ptr *SQLite3DB) Commit() error

func (*SQLite3DB) CreateMetaData added in v0.3.1 

func (ptr *SQLite3DB) CreateMetaData() error

func (*SQLite3DB) Drop added in v0.4.5 

func (ptr *SQLite3DB) Drop() error

Drop drops all tables of a hatchet

func (*SQLite3DB) GetAcceptedConnsCounts added in v0.2.0 

func (ptr *SQLite3DB) GetAcceptedConnsCounts(duration string) ([]NameValue, error)

GetAcceptedConnsCounts returns opened connection counts

func (*SQLite3DB) GetAuditData added in v0.3.1 

func (ptr *SQLite3DB) GetAuditData() (map[string][]NameValues, error)

func (*SQLite3DB) GetAverageOpTime added in v0.2.0 

func (ptr *SQLite3DB) GetAverageOpTime(op string, duration string) ([]OpCount, error)

func (*SQLite3DB) GetConnectionStats added in v0.2.0 

func (ptr *SQLite3DB) GetConnectionStats(chartType string, duration string) ([]RemoteClient, error)

GetConnectionStats returns stats data of accepted and ended

func (*SQLite3DB) GetHatchetInfo added in v0.2.0 

func (ptr *SQLite3DB) GetHatchetInfo() HatchetInfo

func (*SQLite3DB) GetHatchetNames added in v0.2.0 

func (ptr *SQLite3DB) GetHatchetNames() ([]string, error)

func (*SQLite3DB) GetLogs added in v0.2.0 

func (ptr *SQLite3DB) GetLogs(opts ...string) ([]LegacyLog, error)

func (*SQLite3DB) GetOpsCounts added in v0.2.0 

func (ptr *SQLite3DB) GetOpsCounts(duration string) ([]NameValue, error)

GetOpsCounts returns opened connection counts

func (*SQLite3DB) GetReslenByIP added in v0.3.1 

func (ptr *SQLite3DB) GetReslenByIP(ip string, duration string) ([]NameValue, error)

GetReslenByIP returns total response length by ip

func (*SQLite3DB) GetReslenByNamespace added in v0.3.1 

func (ptr *SQLite3DB) GetReslenByNamespace(ns string, duration string) ([]NameValue, error)

GetReslenByNamespace returns total response length by ns

func (*SQLite3DB) GetSlowOps added in v0.2.0 

func (ptr *SQLite3DB) GetSlowOps(orderBy string, order string, collscan bool) ([]OpStat, error)

func (*SQLite3DB) GetSlowestLogs added in v0.2.0 

func (ptr *SQLite3DB) GetSlowestLogs(topN int) ([]LegacyLog, error)

func (*SQLite3DB) GetVerbose added in v0.2.0 

func (ptr *SQLite3DB) GetVerbose() bool

func (*SQLite3DB) InsertClientConn added in v0.2.0 

func (ptr *SQLite3DB) InsertClientConn(index int, doc *Logv2Info) error

func (*SQLite3DB) InsertDriver added in v0.3.1 

func (ptr *SQLite3DB) InsertDriver(index int, doc *Logv2Info) error

func (*SQLite3DB) InsertFailedMessages added in v0.5.0 

func (ptr *SQLite3DB) InsertFailedMessages(m *FailedMessages) error

func (*SQLite3DB) InsertLog added in v0.2.0 

func (ptr *SQLite3DB) InsertLog(index int, end string, doc *Logv2Info, stat *OpStat) error

func (*SQLite3DB) SearchLogs added in v0.2.0 

func (ptr *SQLite3DB) SearchLogs(opts ...string) ([]LegacyLog, error)

func (*SQLite3DB) SetVerbose added in v0.2.0 

func (ptr *SQLite3DB) SetVerbose(b bool)

func (*SQLite3DB) UpdateHatchetInfo added in v0.2.0 

func (ptr *SQLite3DB) UpdateHatchetInfo(info HatchetInfo) error

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.