reftable

package module
v0.0.0-...-7bfe4a2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 2, 2020 License: BSD-3-Clause Imports: 18 Imported by: 0

README

This is a from scratch implementation of the reftable format for storing Git ref and reflog data.

The spec is at https://eclipse.googlesource.com/jgit/jgit/+/refs/heads/master/Documentation/technical/reftable.md

Based on the implementation, there is also a proposal for a v2 of the format.

Java

Reftable was originally implemented in JGit, and can be considered the reference implementation. The code has two parts:

GO

The Go implementation implements the spec completely. API doc.

C

An experimental implementation in C is under the directory c/ . It is functionally equivalent to the Go version. Build and test it using

bazel test c/...

API is documented in side the reftable.h header.

It includes a fragment of the zlib library to provide uncompress2(), which is a recent addition to the API. zlib is licensed as follows:

 (C) 1995-2017 Jean-loup Gailly and Mark Adler

  This software is provided 'as-is', without any express or implied
  warranty.  In no event will the authors be held liable for any damages
  arising from the use of this software.

  Permission is granted to anyone to use this software for any purpose,
  including commercial applications, and to alter it and redistribute it
  freely, subject to the following restrictions:

  1. The origin of this software must not be misrepresented; you must not
     claim that you wrote the original software. If you use this software
     in a product, an acknowledgment in the product documentation would be
     appreciated but is not required.
  2. Altered source versions must be plainly marked as such, and must not be
     misrepresented as being the original software.
  3. This notice may not be removed or altered from any source distribution.

  Jean-loup Gailly        Mark Adler
  jloup@gzip.org          madler@alumni.caltech.edu

Background reading

DISCLAIMER

This is not an official Google product

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrEmptyTable = errors.New("reftable: table is empty")

ErrEmptyTable indicates that a writer tried to create a table without blocks.

View Source 
var ErrLockFailure = errors.New("reftable: lock failure")

ErrLockFailure is returned for failed writes. On a failed write, the stack is reloaded, so the transaction may be retried.

View Source 
var NullHashID = HashID([4]byte{0, 0, 0, 0})
View Source 
var SHA1ID = HashID([4]byte{'s', 'h', 'a', '1'})
View Source 
var SHA256ID = HashID([4]byte{'s', '2', '5', '6'})

Functions

This section is empty.

Types

type Addition 

type Addition struct {
	// contains filtered or unexported fields
}

Addition is a transaction that adds new tables to the top of the stack.

func (*Addition) Add 

func (tr *Addition) Add(write func(w *Writer) error) error

Add calls the given function to write a new table at the top of the stack.

func (*Addition) Close 

func (tr *Addition) Close()

Close releases all non-committed data from the transaction.

func (*Addition) Commit 

func (tr *Addition) Commit() error

Commit commits the changes to the database, releasing the lock.

type BlockSource 

type BlockSource interface {
	Size() uint64
	ReadBlock(off uint64, size int) ([]byte, error)
	Close() error
}

BlockSource is an interface for reading reftable bytes.

func NewFileBlockSource 

func NewFileBlockSource(name string) (BlockSource, error)

NewFileBlockSource opens a file on local disk as a BlockSource

type BlockStats 

type BlockStats struct {
	Entries       int
	Restarts      int
	Blocks        int
	IndexBlocks   int
	MaxIndexLevel int

	Offset      uint64
	IndexOffset uint64
}

BlockStats provides write statistics data of a certain block type.

type ByteBlockSource 

type ByteBlockSource struct {
	Source []byte
}

ByteBlockSource is an in-memory block source.

func (*ByteBlockSource) Close 

func (s *ByteBlockSource) Close() error

func (*ByteBlockSource) ReadBlock 

func (s *ByteBlockSource) ReadBlock(off uint64, sz int) ([]byte, error)

func (*ByteBlockSource) Size 

func (s *ByteBlockSource) Size() uint64

type CompactionStats 

type CompactionStats struct {
	Bytes uint64

	// All entries written, including from failed compaction attempts.
	EntriesWritten uint64
	Attempts       int
	Failures       int
}

CompactionStats holds some statistics of compaction over the lifetime of the stack.

type Config 

type Config struct {
	// If set, do not pad blocks to blocksize.
	Unaligned bool

	// The block size, if not set 4096.
	BlockSize        uint32
	SkipIndexObjects bool
	RestartInterval  int

	// Hash identifier. Unset means sha1.
	HashID HashID

	// Allow dir/file conflicts and illegal refnames
	SkipNameCheck bool

	// If set, store reflog messages exactly. If unset, only allow
	// a single line, and a trailing '\n' is added if it is missing.
	ExactLogMessage bool
}

Options define write options for reftables.

type HashID 

type HashID [4]byte

func (HashID) Size 

func (i HashID) Size() int

type Iterator 

type Iterator struct {
	// contains filtered or unexported fields
}

iterator is an iterator over reftable

func (*Iterator) NextLog 

func (it *Iterator) NextLog(log *LogRecord) (bool, error)

func (*Iterator) NextRef 

func (it *Iterator) NextRef(ref *RefRecord) (bool, error)

type LogExpirationConfig 

type LogExpirationConfig struct {
	Time           uint64
	MaxUpdateIndex uint64
	MinUpdateIndex uint64
}

type LogRecord 

type LogRecord struct {
	RefName     string
	UpdateIndex uint64
	New         []byte
	Old         []byte
	Name        string
	Email       string
	Time        uint64
	TZOffset    int16
	Message     string
}

LogRecord is a Record from the reflog database.

func ReadLogAt 

func ReadLogAt(tab Table, name string, updateIndex uint64) (*LogRecord, error)

ReadRef reads the most recent log record of a certain ref.

func (*LogRecord) IsDeletion 

func (l *LogRecord) IsDeletion() bool

func (*LogRecord) String 

func (l *LogRecord) String() string

type Merged 

type Merged struct {
	// contains filtered or unexported fields
}

Merged is a stack of reftables.

func NewMerged 

func NewMerged(tabs []Table, hashID [4]byte) (*Merged, error)

NewMerged creates a reader for a merged reftable.

func (*Merged) HashID 

func (m *Merged) HashID() HashID

func (*Merged) MaxUpdateIndex 

func (m *Merged) MaxUpdateIndex() uint64

MaxUpdateIndex implements the Table interface.

func (*Merged) MinUpdateIndex 

func (m *Merged) MinUpdateIndex() uint64

MinUpdateIndex implements the Table interface.

func (*Merged) Name 

func (m *Merged) Name() string

func (*Merged) RefsFor 

func (m *Merged) RefsFor(oid []byte) (*Iterator, error)

RefsFor returns the refs that point to the given oid

func (*Merged) SeekLog 

func (m *Merged) SeekLog(refname string, updateIndex uint64) (*Iterator, error)

Seek returns an iterator positioned before the wanted record.

func (*Merged) SeekRef 

func (m *Merged) SeekRef(name string) (*Iterator, error)

type Reader 

type Reader struct {
	// contains filtered or unexported fields
}

Reader allows reading from a reftable.

func NewReader 

func NewReader(src BlockSource, name string) (*Reader, error)

NewReader creates a reader for a reftable file.

func (*Reader) Close 

func (r *Reader) Close()

func (*Reader) DebugData 

func (r *Reader) DebugData() string

func (*Reader) HashID 

func (r *Reader) HashID() HashID

func (*Reader) MaxUpdateIndex 

func (r *Reader) MaxUpdateIndex() uint64

func (*Reader) MinUpdateIndex 

func (r *Reader) MinUpdateIndex() uint64

func (*Reader) Name 

func (r *Reader) Name() string

func (*Reader) RefsFor 

func (r *Reader) RefsFor(oid []byte) (*Iterator, error)

RefsFor iterates over refs that point to `oid`.

func (*Reader) SeekLog 

func (r *Reader) SeekLog(name string, updateIndex uint64) (*Iterator, error)

func (*Reader) SeekRef 

func (r *Reader) SeekRef(name string) (*Iterator, error)

type RefRecord 

type RefRecord struct {
	RefName     string
	UpdateIndex uint64
	Value       []byte
	TargetValue []byte
	// is a 0-length target allowed?
	Target string
}

RefRecord is a Record from the ref database.

func ReadRef 

func ReadRef(tab Table, name string) (*RefRecord, error)

ReadRef reads a ref record by name.

func (*RefRecord) IsDeletion 

func (r *RefRecord) IsDeletion() bool

func (*RefRecord) String 

func (r *RefRecord) String() string

type Stack 

type Stack struct {
	Stats CompactionStats
	// contains filtered or unexported fields
}

Stack is an auto-compacting stack of reftables.

func NewStack 

func NewStack(dir string, cfg Config) (*Stack, error)

NewStack returns a new stack.

func (*Stack) Add 

func (st *Stack) Add(write func(w *Writer) error) error

Add a new reftable to stack, transactionally.

func (*Stack) AutoCompact 

func (st *Stack) AutoCompact() error

AutoCompact runs a compaction if the stack looks imbalanced.

func (*Stack) Close 

func (st *Stack) Close()

Close releases file descriptors associated with this stack.

func (*Stack) CompactAll 

func (st *Stack) CompactAll(expiration *LogExpirationConfig) error

CompactAll compacts the entire stack. If expiration is given, expire log entries.

func (*Stack) Merged 

func (st *Stack) Merged() *Merged

Returns the merged stack. The stack is only valid until the next write, as writes may trigger reloads

func (*Stack) NewAddition 

func (st *Stack) NewAddition() (*Addition, error)

NewAddition returns an Addition instance. As a side effect, this takes a global filesystem lock on the ref database.

func (*Stack) NextUpdateIndex 

func (st *Stack) NextUpdateIndex() uint64

NextUpdateIndex returns the update index at which to write the next table.

func (*Stack) String 

func (st *Stack) String() string

func (*Stack) UpToDate 

func (st *Stack) UpToDate() (bool, error)

type Stats 

type Stats struct {
	ObjStats BlockStats
	RefStats BlockStats
	LogStats BlockStats

	Blocks int

	ObjectIDLen int
	// contains filtered or unexported fields
}

Stats provides general write statistics

type Table 

type Table interface {
	MaxUpdateIndex() uint64
	MinUpdateIndex() uint64
	HashID() HashID
	SeekRef(refName string) (*Iterator, error)
	SeekLog(refName string, updateIndex uint64) (*Iterator, error)
	RefsFor(oid []byte) (*Iterator, error)
	Name() string
	// contains filtered or unexported methods
}

Table is a read interface for reftables, either file reftables or merged reftables.

type Writer 

type Writer struct {
	Stats Stats
	// contains filtered or unexported fields
}

Writer writes a single reftable.

func NewWriter 

func NewWriter(out io.Writer, cfg *Config) (*Writer, error)

NewWriter creates a writer.

func (*Writer) AddLog 

func (w *Writer) AddLog(l *LogRecord) error

AddLog adds a LogRecord to the table. AddLog must be called in ascending order.

func (*Writer) AddRef 

func (w *Writer) AddRef(r *RefRecord) error

AddRef adds a RefRecord to the table. AddRef must be called in ascending order. AddRef cannot be called after AddLog is called.

func (*Writer) Close 

func (w *Writer) Close() error

Close writes the footer and flushes the table to disk.

func (*Writer) SetLimits 

func (w *Writer) SetLimits(min, max uint64)

SetLimits sets the range of the records to be written. It should be called before calling AddRef or AddLog

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.