v0.11.0 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Feb 2, 2023 License: BSD-3-Clause Imports: 15 Imported by: 296



Package cpio implements utilities for reading and writing cpio archives.

Currently, only newc-formatted cpio archives are supported through cpio.Newc.

Reading from or writing to a file:

f, err := os.Open(...)
if err ...
recReader := cpio.Newc.Reader(f)
err := ForEachRecord(recReader, func(r cpio.Record) error {


// Or...
recWriter := cpio.Newc.Writer(f)

Reading from or writing to an in-memory archive:

a := cpio.InMemArchive()
err := a.WriteRecord(...)

recReader := a.Reader() // Reads from the "beginning."

if a.Contains("bar/foo") {




View Source
const (
	S_IEXEC  = 0x40
	S_IFBLK  = 0x6000
	S_IFCHR  = 0x2000
	S_IFDIR  = 0x4000
	S_IFIFO  = 0x1000
	S_IFLNK  = 0xa000
	S_IFMT   = 0xf000
	S_IFREG  = 0x8000
	S_IFSOCK = 0xc000
	S_IFWHT  = 0xe000
	S_IREAD  = 0x100
	S_IRGRP  = 0x20
	S_IROTH  = 0x4
	S_IRUSR  = 0x100
	S_IRWXG  = 0x38
	S_IRWXO  = 0x7
	S_IRWXU  = 0x1c0
	S_ISGID  = 0x400
	S_ISTXT  = 0x200
	S_ISUID  = 0x800
	S_ISVTX  = 0x200

These Unix constants are needed everywhere cpio is used, Unix or not. But we are unable to import the unix package when plan 9 is enabled, so lucky us, the numbers have been the same for half a century. It is ok to just define them.

View Source
const Trailer = "TRAILER!!!"

Trailer is the name of the trailer record.


View Source
var (

	// Debug can be set e.g. to log.Printf to enable debug prints from
	// marshaling/unmarshaling cpio archives.
	Debug = func(string, ...interface{}) {}
View Source
var TrailerRecord = StaticRecord(nil, Info{Name: Trailer})

TrailerRecord is the last record in any CPIO archive.


func AllEqual

func AllEqual(r []Record, s []Record) bool

AllEqual compares all metadata and contents of r and s.

func Concat

func Concat(w RecordWriter, r RecordReader, transform func(Record) Record) error

Concat reads files from r one at a time, and writes them to w.

Concat does not write a trailer record and applies transform to every record before writing it. transform may be nil.

func CreateFile

func CreateFile(f Record) error

CreateFile creates a local file for f relative to the current working directory.

CreateFile will attempt to set all metadata for the file, including ownership, times, and permissions.

func CreateFileInRoot

func CreateFileInRoot(f Record, rootDir string, forcePriv bool) error

CreateFileInRoot creates a local file for f relative to rootDir.

It will attempt to set all metadata for the file, including ownership, times, and permissions. If these fail, it only returns an error if forcePriv is true.

Block and char device creation will only return error if forcePriv is true.

func Equal

func Equal(r Record, s Record) bool

Equal compares the metadata and contents of r and s.

func ForEachRecord

func ForEachRecord(rr RecordReader, fun func(Record) error) error

ForEachRecord reads every record from r and applies f.

func LSInfoFromRecord

func LSInfoFromRecord(rec Record) ls.FileInfo

LSInfoFromRecord converts a Record to be usable with the ls package for listing files.

func MakeAllReproducible

func MakeAllReproducible(files []Record)

MakeAllReproducible makes all given records reproducible as in MakeReproducible.

func Normalize

func Normalize(path string) string

Normalize normalizes path to be relative to /.

func Passthrough

func Passthrough(r RecordReader, w RecordWriter) error

Passthrough copies from a RecordReader to a RecordWriter.

Passthrough writes a trailer record.

It processes one record at a time to minimize the memory footprint.

func WriteRecords

func WriteRecords(w RecordWriter, files []Record) error

WriteRecords writes multiple records to w.

func WriteRecordsAndDirs added in v0.11.0

func WriteRecordsAndDirs(rw RecordWriter, files []Record) error

WriteRecordsAndDirs writes records to w, with a slight difference from WriteRecords: the record path is split and all the directories are written first, in order, mimic'ing what happens with find . -print

When is this function needed? Most cpio programs will create directories as needed for paths such as a/b/c/d The cpio creation process for Linux uses find, and will create a record for each directory in a/b/c/d

But when code programatically generates a cpio for the Linux kernel, the cpio is not generated via find, and Linux will not create intermediate directories. The result, seen in practice, is that a path, such as a/b/c/d, when unpacked by the linux kernel, will be ignored if a/b/c does not exist!

Again, this function is very rarely needed, save when we programatically generate an initramfs for Linux. This code only works with a deduplicating writer. Further, it will not accept a Record if the full pathname of that Record already exists. This is arguably overly restrictive but, at the same, avoids some very unpleasant programmer errors. There is overlap here with DedupWriter but given that this is a Special Snowflake function, it seems best to leave the DedupWriter code alone.

func WriteTrailer

func WriteTrailer(w RecordWriter) error

WriteTrailer writes the trailer record.


type Archive

type Archive struct {
	// Files is a map of relative archive path -> record.
	Files map[string]Record

	// Order is a list of relative archive paths and represents the order
	// in which Files were added.
	Order []string

Archive is an in-memory list of files.

Archive itself is a RecordWriter, and Archive.Reader() returns a new RecordReader for the archive starting from the first file.

func ArchiveFromReader

func ArchiveFromReader(r RecordReader) (*Archive, error)

ArchiveFromReader reads records from r into a new Archive in memory.

func ArchiveFromRecords

func ArchiveFromRecords(rs []Record) *Archive

ArchiveFromRecords creates a new Archive from the records.

func InMemArchive

func InMemArchive() *Archive

InMemArchive returns an in-memory file archive.

func (*Archive) Contains

func (a *Archive) Contains(r Record) bool

Contains returns true if a record matching r is in the archive.

func (*Archive) Empty

func (a *Archive) Empty() bool

Empty returns whether the archive has any files in it.

func (*Archive) Get

func (a *Archive) Get(path string) (Record, bool)

Get returns a record for the normalized path or false if there is none.

The path is normalized using Normalize, so Get("/bin/bar") is the same as Get("bin/bar") is the same as Get("bin//bar").

func (*Archive) Reader

func (a *Archive) Reader() RecordReader

Reader returns a RecordReader for the archive that starts at the first record.

func (*Archive) String

func (a *Archive) String() string

String implements fmt.Stringer.

String lists files like ls would.

func (*Archive) WriteRecord

func (a *Archive) WriteRecord(r Record) error

WriteRecord implements RecordWriter and adds a record to the archive.

WriteRecord uses Normalize to deduplicate paths.

type DedupWriter

type DedupWriter struct {
	// contains filtered or unexported fields

DedupWriter is a RecordWriter that does not write more than one record with the same path.

There seems to be no harm done in stripping duplicate names when the record is written, and lots of harm done if we don't do it.

func (*DedupWriter) WriteRecord

func (dw *DedupWriter) WriteRecord(rec Record) error

WriteRecord implements RecordWriter.

If rec.Name was already seen once before, it will not be written again and WriteRecord returns nil.

type EOFReader

type EOFReader struct {

EOFReader is a RecordReader that converts the Trailer record to io.EOF.

func (EOFReader) ReadRecord

func (r EOFReader) ReadRecord() (Record, error)

ReadRecord implements RecordReader.

ReadRecord returns io.EOF when the record name is TRAILER!!!.

type Info

type Info struct {
	Ino      uint64
	Mode     uint64
	UID      uint64
	GID      uint64
	NLink    uint64
	MTime    uint64
	FileSize uint64
	Dev      uint64
	Major    uint64
	Minor    uint64
	Rmajor   uint64
	Rminor   uint64
	Name     string

Info holds metadata about files.

func (Info) String

func (i Info) String() string

type Record

type Record struct {
	// ReaderAt contains the content of this CPIO record.

	// Info is metadata describing the CPIO record.

	// metadata about this item's place in the file
	RecPos  int64  // Where in the file this record is
	RecLen  uint64 // How big the record is.
	FilePos int64  // Where in the CPIO the file's contents are.

Record represents a CPIO record, which represents a Unix file.

func CharDev

func CharDev(name string, perm uint64, rmajor, rminor uint64) Record

CharDev returns a character device record at name.

func Directory

func Directory(name string, mode uint64) Record

Directory returns a directory record at name.

func MakeReproducible

func MakeReproducible(r Record) Record

MakeReproducible changes any fields in a Record such that if we run cpio again, with the same files presented to it in the same order, and those files have unchanged contents, the cpio file it produces will be bit-for-bit identical. This is an essential property for firmware-embedded payloads.

func ReadAllRecords

func ReadAllRecords(rr RecordReader) ([]Record, error)

ReadAllRecords returns all records in r in the order in which they were read.

func StaticFile

func StaticFile(name string, content string, perm uint64) Record

StaticFile returns a normal file record.

func StaticRecord

func StaticRecord(contents []byte, info Info) Record

StaticRecord returns a record with the given contents and metadata.

func Symlink(name string, target string) Record

Symlink returns a symlink record at name pointing to target.

func (Record) String

func (r Record) String() string

String implements a fmt.Stringer for Record.

String returns a string long-formatted like `ls` would format it.

type RecordFormat

type RecordFormat interface {
	Reader(r io.ReaderAt) RecordReader
	NewFileReader(*os.File) (RecordReader, error)
	Writer(w io.Writer) RecordWriter

A RecordFormat gives readers and writers for dealing with archives from io objects.

CPIO files have a number of records, of which newc is the most widely used today.

var Newc RecordFormat = newc{/* contains filtered or unexported fields */}

Newc is the newc CPIO record format.

func Format

func Format(name string) (RecordFormat, error)

Format returns the RecordFormat with that name, if it exists.

type RecordReader

type RecordReader interface {
	ReadRecord() (Record, error)

A RecordReader reads one record from an archive.

type RecordWriter

type RecordWriter interface {
	WriteRecord(Record) error

A RecordWriter writes one record to an archive.

func NewDedupWriter

func NewDedupWriter(rw RecordWriter) RecordWriter

NewDedupWriter returns a new deduplicating rw.

type Recorder

type Recorder struct {
	// contains filtered or unexported fields

A Recorder is a structure that contains variables used to calculate file parameters such as inode numbers for a CPIO file. The life-time of a Record structure is meant to be the same as the construction of a single CPIO archive. Do not reuse between CPIOs if you don't know what you're doing.

func NewRecorder

func NewRecorder() *Recorder

NewRecorder creates a new Recorder.

A recorder is a structure that contains variables used to calculate file parameters such as inode numbers for a CPIO file. The life-time of a Record structure is meant to be the same as the construction of a single CPIO archive. Do not reuse between CPIOs if you don't know what you're doing.

func (*Recorder) GetRecord

func (r *Recorder) GetRecord(path string) (Record, error)

GetRecord returns a cpio Record for the given path on the local file system.

GetRecord does not follow symlinks. If path is a symlink, the record returned will reflect that symlink.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL