Documentation

Overview

    Package packfile implements encoding and decoding of packfile format.

    == pack-*.pack files have the following format:
    
      - A header appears at the beginning and consists of the following:
    
        4-byte signature:
            The signature is: {'P', 'A', 'C', 'K'}
    
        4-byte version number (network byte order):
            GIT currently accepts version number 2 or 3 but
            generates version 2 only.
    
        4-byte number of objects contained in the pack (network byte order)
    
        Observation: we cannot have more than 4G versions ;-) and
        more than 4G objects in a pack.
    
      - The header is followed by number of object entries, each of
        which looks like this:
    
        (undeltified representation)
        n-byte type and length (3-bit type, (n-1)*7+4-bit length)
        compressed data
    
        (deltified representation)
        n-byte type and length (3-bit type, (n-1)*7+4-bit length)
        20-byte base object name
        compressed delta data
    
        Observation: length of each object is encoded in a variable
        length format and is not constrained to 32-bit or anything.
    
     - The trailer records 20-byte SHA1 checksum of all of the above.
    

    Source: https://www.kernel.org/pub/software/scm/git/docs/v1.7.5/technical/pack-protocol.txt

    Index

    Constants

    View Source
    const (
    	// VersionSupported is the packfile version supported by this package
    	VersionSupported uint32 = 2
    )

    Variables

    View Source
    var (
    	// ErrInvalidObject is returned by Decode when an invalid object is
    	// found in the packfile.
    	ErrInvalidObject = NewError("invalid git object")
    	// ErrZLib is returned by Decode when there was an error unzipping
    	// the packfile contents.
    	ErrZLib = NewError("zlib reading error")
    )
    View Source
    var (
    	// ErrReferenceDeltaNotFound is returned when the reference delta is not
    	// found.
    	ErrReferenceDeltaNotFound = errors.New("reference delta not found")
    
    	// ErrNotSeekableSource is returned when the source for the parser is not
    	// seekable and a storage was not provided, so it can't be parsed.
    	ErrNotSeekableSource = errors.New("parser source is not seekable and storage was not provided")
    
    	// ErrDeltaNotCached is returned when the delta could not be found in cache.
    	ErrDeltaNotCached = errors.New("delta could not be found in cache")
    )
    View Source
    var (
    	ErrInvalidDelta = errors.New("invalid delta")
    	ErrDeltaCmd     = errors.New("wrong delta command")
    )
    View Source
    var (
    	// ErrEmptyPackfile is returned by ReadHeader when no data is found in the packfile
    	ErrEmptyPackfile = NewError("empty packfile")
    	// ErrBadSignature is returned by ReadHeader when the signature in the packfile is incorrect.
    	ErrBadSignature = NewError("malformed pack file signature")
    	// ErrUnsupportedVersion is returned by ReadHeader when the packfile version is
    	// different than VersionSupported.
    	ErrUnsupportedVersion = NewError("unsupported packfile version")
    	// ErrSeekNotSupported returned if seek is not support
    	ErrSeekNotSupported = NewError("not seek support")
    )
    View Source
    var T = []uint32{}/* 256 elements not displayed */
    

    Functions

    func ApplyDelta

    func ApplyDelta(target, base plumbing.EncodedObject, delta []byte) error

      ApplyDelta writes to target the result of applying the modification deltas in delta to base.

      func DiffDelta

      func DiffDelta(src, tgt []byte) []byte

        DiffDelta returns the delta that transforms src into tgt.

        func GetDelta

        func GetDelta(base, target plumbing.EncodedObject) (plumbing.EncodedObject, error)

          GetDelta returns an EncodedObject of type OFSDeltaObject. Base and Target object, will be loaded into memory to be able to create the delta object. To generate target again, you will need the obtained object and "base" one. Error will be returned if base or target object cannot be read.

          func PatchDelta

          func PatchDelta(src, delta []byte) ([]byte, error)

            PatchDelta returns the result of applying the modification deltas in delta to src. An error will be returned if delta is corrupted (ErrDeltaLen) or an action command is not copy from source or copy from delta (ErrDeltaCmd).

            func UpdateObjectStorage

            func UpdateObjectStorage(s storer.Storer, packfile io.Reader) error

              UpdateObjectStorage updates the storer with the objects in the given packfile.

              func WritePackfileToObjectStorage

              func WritePackfileToObjectStorage(
              	sw storer.PackfileWriter,
              	packfile io.Reader,
              ) (err error)

                WritePackfileToObjectStorage writes all the packfile objects into the given object storage.

                Types

                type Encoder

                type Encoder struct {
                	// contains filtered or unexported fields
                }

                  Encoder gets the data from the storage and write it into the writer in PACK format

                  func NewEncoder

                  func NewEncoder(w io.Writer, s storer.EncodedObjectStorer, useRefDeltas bool) *Encoder

                    NewEncoder creates a new packfile encoder using a specific Writer and EncodedObjectStorer. By default deltas used to generate the packfile will be OFSDeltaObject. To use Reference deltas, set useRefDeltas to true.

                    func (*Encoder) Encode

                    func (e *Encoder) Encode(
                    	hashes []plumbing.Hash,
                    	packWindow uint,
                    ) (plumbing.Hash, error)

                      Encode creates a packfile containing all the objects referenced in hashes and writes it to the writer in the Encoder. `packWindow` specifies the size of the sliding window used to compare objects for delta compression; 0 turns off delta compression entirely.

                      type Error

                      type Error struct {
                      	// contains filtered or unexported fields
                      }

                        Error specifies errors returned during packfile parsing.

                        func NewError

                        func NewError(reason string) *Error

                          NewError returns a new error.

                          func (*Error) AddDetails

                          func (e *Error) AddDetails(format string, args ...interface{}) *Error

                            AddDetails adds details to an error, with additional text.

                            func (*Error) Error

                            func (e *Error) Error() string

                              Error returns a text representation of the error.

                              type FSObject

                              type FSObject struct {
                              	// contains filtered or unexported fields
                              }

                                FSObject is an object from the packfile on the filesystem.

                                func NewFSObject

                                func NewFSObject(
                                	hash plumbing.Hash,
                                	finalType plumbing.ObjectType,
                                	offset int64,
                                	contentSize int64,
                                	index idxfile.Index,
                                	fs billy.Filesystem,
                                	path string,
                                	cache cache.Object,
                                ) *FSObject

                                  NewFSObject creates a new filesystem object.

                                  func (*FSObject) Hash

                                  func (o *FSObject) Hash() plumbing.Hash

                                    Hash implements the plumbing.EncodedObject interface.

                                    func (*FSObject) Reader

                                    func (o *FSObject) Reader() (io.ReadCloser, error)

                                      Reader implements the plumbing.EncodedObject interface.

                                      func (*FSObject) SetSize

                                      func (o *FSObject) SetSize(int64)

                                        SetSize implements the plumbing.EncodedObject interface. This method is a noop.

                                        func (*FSObject) SetType

                                        func (o *FSObject) SetType(plumbing.ObjectType)

                                          SetType implements the plumbing.EncodedObject interface. This method is a noop.

                                          func (*FSObject) Size

                                          func (o *FSObject) Size() int64

                                            Size implements the plumbing.EncodedObject interface.

                                            func (*FSObject) Type

                                            func (o *FSObject) Type() plumbing.ObjectType

                                              Type implements the plumbing.EncodedObject interface.

                                              func (*FSObject) Writer

                                              func (o *FSObject) Writer() (io.WriteCloser, error)

                                                Writer implements the plumbing.EncodedObject interface. This method always returns a nil writer.

                                                type ObjectHeader

                                                type ObjectHeader struct {
                                                	Type            plumbing.ObjectType
                                                	Offset          int64
                                                	Length          int64
                                                	Reference       plumbing.Hash
                                                	OffsetReference int64
                                                }

                                                  ObjectHeader contains the information related to the object, this information is collected from the previous bytes to the content of the object.

                                                  type ObjectToPack

                                                  type ObjectToPack struct {
                                                  	// The main object to pack, it could be any object, including deltas
                                                  	Object plumbing.EncodedObject
                                                  	// Base is the object that a delta is based on (it could be also another delta).
                                                  	// If the main object is not a delta, Base will be null
                                                  	Base *ObjectToPack
                                                  	// Original is the object that we can generate applying the delta to
                                                  	// Base, or the same object as Object in the case of a non-delta
                                                  	// object.
                                                  	Original plumbing.EncodedObject
                                                  	// Depth is the amount of deltas needed to resolve to obtain Original
                                                  	// (delta based on delta based on ...)
                                                  	Depth int
                                                  
                                                  	// offset in pack when object has been already written, or 0 if it
                                                  	// has not been written yet
                                                  	Offset int64
                                                  	// contains filtered or unexported fields
                                                  }

                                                    ObjectToPack is a representation of an object that is going to be into a pack file.

                                                    func (*ObjectToPack) BackToOriginal

                                                    func (o *ObjectToPack) BackToOriginal()

                                                      BackToOriginal converts that ObjectToPack to a non-deltified object if it was one

                                                      func (*ObjectToPack) CleanOriginal

                                                      func (o *ObjectToPack) CleanOriginal()

                                                        CleanOriginal sets Original to nil

                                                        func (*ObjectToPack) Hash

                                                        func (o *ObjectToPack) Hash() plumbing.Hash

                                                        func (*ObjectToPack) IsDelta

                                                        func (o *ObjectToPack) IsDelta() bool

                                                        func (*ObjectToPack) IsWritten

                                                        func (o *ObjectToPack) IsWritten() bool

                                                          IsWritten returns if that ObjectToPack was already written into the packfile or not

                                                          func (*ObjectToPack) MarkWantWrite

                                                          func (o *ObjectToPack) MarkWantWrite()

                                                            MarkWantWrite marks this ObjectToPack as WantWrite to avoid delta chain loops

                                                            func (*ObjectToPack) SaveOriginalMetadata

                                                            func (o *ObjectToPack) SaveOriginalMetadata()

                                                              SaveOriginalMetadata saves size, type and hash of Original object

                                                              func (*ObjectToPack) SetDelta

                                                              func (o *ObjectToPack) SetDelta(base *ObjectToPack, delta plumbing.EncodedObject)

                                                              func (*ObjectToPack) SetOriginal

                                                              func (o *ObjectToPack) SetOriginal(obj plumbing.EncodedObject)

                                                                SetOriginal sets both Original and saves size, type and hash. If object is nil Original is set but previous resolved values are kept

                                                                func (*ObjectToPack) Size

                                                                func (o *ObjectToPack) Size() int64

                                                                func (*ObjectToPack) Type

                                                                func (o *ObjectToPack) Type() plumbing.ObjectType

                                                                func (*ObjectToPack) WantWrite

                                                                func (o *ObjectToPack) WantWrite() bool

                                                                  WantWrite checks if this ObjectToPack was marked as WantWrite before

                                                                  type Observer

                                                                  type Observer interface {
                                                                  	// OnHeader is called when a new packfile is opened.
                                                                  	OnHeader(count uint32) error
                                                                  	// OnInflatedObjectHeader is called for each object header read.
                                                                  	OnInflatedObjectHeader(t plumbing.ObjectType, objSize int64, pos int64) error
                                                                  	// OnInflatedObjectContent is called for each decoded object.
                                                                  	OnInflatedObjectContent(h plumbing.Hash, pos int64, crc uint32, content []byte) error
                                                                  	// OnFooter is called when decoding is done.
                                                                  	OnFooter(h plumbing.Hash) error
                                                                  }

                                                                    Observer interface is implemented by index encoders.

                                                                    type Packfile

                                                                    type Packfile struct {
                                                                    	idxfile.Index
                                                                    	// contains filtered or unexported fields
                                                                    }

                                                                      Packfile allows retrieving information from inside a packfile.

                                                                      func NewPackfile

                                                                      func NewPackfile(index idxfile.Index, fs billy.Filesystem, file billy.File) *Packfile

                                                                        NewPackfile returns a packfile representation for the given packfile file and packfile idx. If the filesystem is provided, the packfile will return FSObjects, otherwise it will return MemoryObjects.

                                                                        func NewPackfileWithCache

                                                                        func NewPackfileWithCache(
                                                                        	index idxfile.Index,
                                                                        	fs billy.Filesystem,
                                                                        	file billy.File,
                                                                        	cache cache.Object,
                                                                        ) *Packfile

                                                                          NewPackfileWithCache creates a new Packfile with the given object cache. If the filesystem is provided, the packfile will return FSObjects, otherwise it will return MemoryObjects.

                                                                          func (*Packfile) Close

                                                                          func (p *Packfile) Close() error

                                                                            Close the packfile and its resources.

                                                                            func (*Packfile) Get

                                                                              Get retrieves the encoded object in the packfile with the given hash.

                                                                              func (*Packfile) GetAll

                                                                              func (p *Packfile) GetAll() (storer.EncodedObjectIter, error)

                                                                                GetAll returns an iterator with all encoded objects in the packfile. The iterator returned is not thread-safe, it should be used in the same thread as the Packfile instance.

                                                                                func (*Packfile) GetByOffset

                                                                                func (p *Packfile) GetByOffset(o int64) (plumbing.EncodedObject, error)

                                                                                  GetByOffset retrieves the encoded object from the packfile at the given offset.

                                                                                  func (*Packfile) GetByType

                                                                                    GetByType returns all the objects of the given type.

                                                                                    func (*Packfile) GetSizeByOffset

                                                                                    func (p *Packfile) GetSizeByOffset(o int64) (size int64, err error)

                                                                                      GetSizeByOffset retrieves the size of the encoded object from the packfile with the given offset.

                                                                                      func (*Packfile) ID

                                                                                      func (p *Packfile) ID() (plumbing.Hash, error)

                                                                                        ID returns the ID of the packfile, which is the checksum at the end of it.

                                                                                        func (*Packfile) Scanner

                                                                                        func (p *Packfile) Scanner() *Scanner

                                                                                          Scanner returns the packfile's Scanner

                                                                                          type Parser

                                                                                          type Parser struct {
                                                                                          	// contains filtered or unexported fields
                                                                                          }

                                                                                            Parser decodes a packfile and calls any observer associated to it. Is used to generate indexes.

                                                                                            func NewParser

                                                                                            func NewParser(scanner *Scanner, ob ...Observer) (*Parser, error)

                                                                                              NewParser creates a new Parser. The Scanner source must be seekable. If it's not, NewParserWithStorage should be used instead.

                                                                                              func NewParserWithStorage

                                                                                              func NewParserWithStorage(
                                                                                              	scanner *Scanner,
                                                                                              	storage storer.EncodedObjectStorer,
                                                                                              	ob ...Observer,
                                                                                              ) (*Parser, error)

                                                                                                NewParserWithStorage creates a new Parser. The scanner source must either be seekable or a storage must be provided.

                                                                                                func (*Parser) Parse

                                                                                                func (p *Parser) Parse() (plumbing.Hash, error)

                                                                                                  Parse start decoding phase of the packfile.

                                                                                                  type Scanner

                                                                                                  type Scanner struct {
                                                                                                  
                                                                                                  	// lsSeekable says if this scanner can do Seek or not, to have a Scanner
                                                                                                  	// seekable a r implementing io.Seeker is required
                                                                                                  	IsSeekable bool
                                                                                                  	// contains filtered or unexported fields
                                                                                                  }

                                                                                                  func NewScanner

                                                                                                  func NewScanner(r io.Reader) *Scanner

                                                                                                    NewScanner returns a new Scanner based on a reader, if the given reader implements io.ReadSeeker the Scanner will be also Seekable

                                                                                                    func (*Scanner) Checksum

                                                                                                    func (s *Scanner) Checksum() (plumbing.Hash, error)

                                                                                                      Checksum returns the checksum of the packfile

                                                                                                      func (*Scanner) Close

                                                                                                      func (s *Scanner) Close() error

                                                                                                        Close reads the reader until io.EOF

                                                                                                        func (*Scanner) Flush

                                                                                                        func (s *Scanner) Flush() error

                                                                                                          Flush is a no-op (deprecated)

                                                                                                          func (*Scanner) Header

                                                                                                          func (s *Scanner) Header() (version, objects uint32, err error)

                                                                                                            Header reads the whole packfile header (signature, version and object count). It returns the version and the object count and performs checks on the validity of the signature and the version fields.

                                                                                                            func (*Scanner) NextObject

                                                                                                            func (s *Scanner) NextObject(w io.Writer) (written int64, crc32 uint32, err error)

                                                                                                              NextObject writes the content of the next object into the reader, returns the number of bytes written, the CRC32 of the content and an error, if any

                                                                                                              func (*Scanner) NextObjectHeader

                                                                                                              func (s *Scanner) NextObjectHeader() (*ObjectHeader, error)

                                                                                                                NextObjectHeader returns the ObjectHeader for the next object in the reader

                                                                                                                func (*Scanner) Reset

                                                                                                                func (s *Scanner) Reset(r io.Reader)

                                                                                                                func (*Scanner) SeekFromStart

                                                                                                                func (s *Scanner) SeekFromStart(offset int64) (previous int64, err error)

                                                                                                                  SeekFromStart sets a new offset from start, returns the old position before the change.

                                                                                                                  func (*Scanner) SeekObjectHeader

                                                                                                                  func (s *Scanner) SeekObjectHeader(offset int64) (*ObjectHeader, error)

                                                                                                                    SeekObjectHeader seeks to specified offset and returns the ObjectHeader for the next object in the reader