stdlib

Documentation

Overview

Package stdlib provides wrappers of standard library packages to be imported natively in mvm.

Index

• Variables
• func EmbeddedStd() []byte
• func ForceShort(pkgPath string) bool
• func GorootSrcDir(importPath string) string
• func GorootTestFS() fs.FS
• func IsGenericOnly(pkgPath string) bool
• func IsStdlibImport(name string) bool
• func PackagePatchers() map[string][]PackagePatcher
• func RegisterPackagePatcher(importPath string, fn PackagePatcher)
• func SkipReason(pkgPath, testName string) string
• func TestOverlay() map[string]map[string]reflect.Value
• func UntestableReason(pkgPath string) string
• type PackagePatcher
• type SubtestSkip
  • func SubtestSkips(pkgPath string) []SubtestSkip

Variables

var ConstValues = map[string]map[string]string{}

ConstValues stores, per package, the exact (arbitrary-precision) textual form of bridged floating-point constants whose reflect.Value bridge would lose precision (e.g. math.Pi). It lets the compiler fold constant expressions such as 100000*math.Pi at full precision and round once, matching the Go compiler. The string is go/constant.Value.ExactString() (a decimal or "num/den" rational), reconstructed via goparser.ConstFromExact at import time.

var GenericOnly = map[string]bool{
	"crypto/hkdf":   true,
	"crypto/pbkdf2": true,
	"unique":        true,
	"weak":          true,
}

GenericOnly lists stdlib packages with an all-generic API: no reflect bridge (cmd/extract emits an empty stub) and no interpreted mirror, so mvm cannot load them. Keep in sync with the stub note in gen.go.

var Incompat = map[string]map[string]string{
	"crypto": {
		"TestSignMessage": "interpreted type can't satisfy native crypto.Signer via reflect.Call (synth attaches only fixed method shapes)",
	},
	"log/slog": {
		"Example_wrapping":              "runtime.Caller through the reflect.Call adapter masks the user source line (reports .:0)",
		"ExampleSetLogLoggerLevel_slog": "interpreted log and native-bridged slog hold separate default-logger state, so log.Print doesn't route through slog.SetDefault",
		"ExampleSetLogLoggerLevel_log":  "interpreted log and native-bridged slog hold separate default-logger state, so slog.Info/Debug don't interleave with log.Print",
	},
	"io": {
		"TestPipeAllocations": "testing.AllocsPerRun: interpreter call/marshal allocates more than native Pipe()'s 4",
	},
	"reflect": {
		"TestFields": "reflect.StructOf cannot build a struct embedding an unexported-named type (anonymous+PkgPath is rejected); VisibleFields misses its promoted fields",
	},
	"io/fs": {
		"TestReadDirPath":  "reflect.StructOf cannot build struct{FS} (promoted methods of embedded interfaces); mvm types the anon struct at runtime",
		"TestReadFilePath": "reflect.StructOf cannot build struct{FS} (promoted methods of embedded interfaces); mvm types the anon struct at runtime",
	},
	"flag": {
		"TestDefineAfterSet": "runtime.Caller through reflect.Call adapter masks the user call site",
	},
	"sync": {
		"TestIssue76126": "re-execs os.Args[0] with -test.run to observe a child crash; under mvm that is the mvm binary, not a test binary, so the child never panics",
	},
	"os": {
		"TestLargeCopyViaNetwork": "stress test: streams a 10MB random file through a localhost TCP pair; ~15s under the interpreter (no testing.Short path)",
		"TestCopyFileToFile":      "stress test: copies a 1MB random file across a srcStart x dstStart x limit subtest grid; ~17s under the interpreter (no testing.Short path)",
	},

	"bytes": {
		"TestNewBufferShallow": "testing.AllocsPerRun observes mvm interpreter allocations; native expects 0",
		"TestWriteAppend":      "testing.AllocsPerRun observes mvm interpreter allocations; native expects 0",
		"TestGrow":             "testing.AllocsPerRun observes mvm interpreter allocations; native expects 0",
	},
	"strings": {
		"TestBuilderGrow":            "testing.AllocsPerRun observes mvm interpreter allocations; native expects 0/1",
		"TestBuilderAllocs":          "testing.AllocsPerRun observes mvm interpreter allocations; native expects 1",
		"TestBuilderGrowSizeclasses": "testing.AllocsPerRun observes mvm interpreter allocations; native expects 1",
		"TestIndexRune":              "testing.AllocsPerRun observes mvm interpreter allocations; native expects 0",
		"TestReplace":                "testing.AllocsPerRun observes mvm interpreter allocations; native expects <=1",
	},
	"unicode/utf8": {
		"TestRuneCountNonASCIIAllocation": "testing.AllocsPerRun observes mvm interpreter allocations; native expects 0",
	},
	"strconv": {
		"TestAllocationsFromBytes": "testing.AllocsPerRun observes mvm interpreter allocations; native expects 0",
	},
	"fmt": {
		"TestCountMallocs": "testing.AllocsPerRun observes mvm interpreter allocations; native expects 0-4",
	},
	"testing": {
		"TestAllocsPerRun": "self-test of AllocsPerRun; mvm interpreter allocates more than the native expectation of 1",
	},
	"time": {
		"TestLinkname":       "uses //go:linkname to reach private time funcs; mvm does not parse linkname directives",
		"ExampleDate":        "expects local zone America/Los_Angeles; time's internal ForceUSPacificForTesting init cannot run against the bridge",
		"ExampleTime_Format": "expects local zone America/Los_Angeles; time's internal ForceUSPacificForTesting init cannot run against the bridge",
		"ExampleParse":       "expects local zone America/Los_Angeles; time's internal ForceUSPacificForTesting init cannot run against the bridge",
	},
	"runtime": {
		"TestHeapObjectsCanMove": "uses //go:linkname to reach private runtime.heapObjectsCanMove; mvm does not parse linkname directives",
		"TestPanicNil":           "depends on runtime/metrics + GODEBUG panicnil semantics not modeled by the bridge",
		"TestIssue48807":         "float32(uint64) double-rounds via float64; mvm lacks direct uint64->float32 rounding (Go issue 48807)",
	},
	"runtime/debug": {
		"TestPanicOnFault": "interpreted recover() cannot catch a SetPanicOnFault hardware fault: it surfaces as a raw Go panic from a reflect-driven store, caught by Run's recoverPanic, not routed through the interpreted defer/recover machinery",
		"TestSetGCPercent": "asserts host GC-pacer NextGC thresholds and forced-GC timing; interpreted allocation does not drive the pacer like native code (flaky even natively, SkipFlaky #20076)",
		"TestStack":        "debug.Stack reads the native goroutine stack; an interpreted method runs via reflect.MakeFunc, so frames show reflect/VM internals instead of the runtime/debug_test source the test greps for",
	},

	"github.com/google/btree": {
		"TestBTreeG":                     "stress test: builds a 10000-key B-tree x10 iterations; minutes under the interpreter (no testing.Short path)",
		"TestBTree":                      "stress test: builds a 10000-key B-tree x10 iterations; minutes under the interpreter (no testing.Short path)",
		"TestCloneConcurrentOperationsG": "stress test: 10000-key concurrent-clone workload; minutes under the interpreter (no testing.Short path)",
		"TestCloneConcurrentOperations":  "stress test: 10000-key concurrent-clone workload; minutes under the interpreter (no testing.Short path)",
	},

	"github.com/yuin/goldmark": {
		"TestDeepNestedLabelPerformance": "wall-clock bound: 50000-deep nested link labels under 5s is a native-speed assertion; ~40s under the interpreter",
	},

	"github.com/oklog/ulid/v2": {
		"TestLexicographicalOrder": "stress test: quick.Check MaxCount 1e6 (~286s); hardcoded count ignores -short/-quickchecks",
		"TestCompare":              "stress test: quick.CheckEqual MaxCount 1e5 (~42s); hardcoded count ignores -short/-quickchecks",
		"TestRoundTrips":           "stress test: quick.Check MaxCount 1e5 (~25s); hardcoded count ignores -short/-quickchecks",
		"TestEncoding":             "stress test: quick.Check MaxCount 1e5 (~23s); hardcoded count ignores -short/-quickchecks",
	},

	"github.com/stretchr/testify/assert": {
		"TestDirExists":   "upstream-red: asserts ../_codegen exists, but the dir is omitted from the published module (go test fails natively too)",
		"TestNoDirExists": "upstream-red: asserts ../_codegen exists, but the dir is omitted from the published module (go test fails natively too)",
	},

	"github.com/sirupsen/logrus": {
		"TestNestedLoggingReportsCorrectCaller": "asserts caller frame.File == cwd-relative on-disk path; virtualized runtime.Callers reports the modfs source path (func and line do match)",
		"TestCallerReportingOverhead":           "wall-clock bound: 5000 log calls under 1s is a native-speed assertion; interpreted execution exceeds it",
	},

	"github.com/shopspring/decimal": {
		"TestDecimal_QuoRem2":   "stress test: ~1e6 combinatorial QuoRem cases (createDivTestCases); ~30s under the interpreter (no testing.Short path)",
		"TestDecimal_DivRound2": "stress test: ~1e6 combinatorial DivRound cases (createDivTestCases); ~44s under the interpreter (no testing.Short path)",
	},

	"github.com/tidwall/gjson": {
		"TestRandomMany":         "stress test: 5e4 iterations of GetManyBytes over random bytes; ~57s under the interpreter (no testing.Short path)",
		"TestRandomData":         "stress test: 2e6 iterations of GetBytes+Parse over random bytes; ~34s under the interpreter (no testing.Short path)",
		"TestRandomValidStrings": "stress test: 1e5 random-string Get round-trips; ~14s under the interpreter (no testing.Short path)",
		"TestValidRandom":        "stress test: wall-clock bound, validpayload over random bytes for two fixed 3s loops; ~6s under the interpreter",
	},

	"github.com/tidwall/sjson": {
		"TestRandomData": "stress test: 2e6 iterations of SetRaw over random bytes; ~52s under the interpreter (no testing.Short path)",
	},

	"golang.org/x/text/unicode/norm": {
		"TestWriter": "stress test: streams the static normTests corpus through Form.Writer across all 16 bufSizes x4 forms; ~150s under the interpreter and ignores testing.Short (the same corpus runs via TestAppend/TestString)",
		"TestReader": "stress test: streams the static normTests corpus through Form.Reader across all 16 bufSizes x4 forms; ~14s under the interpreter and ignores testing.Short (the same corpus runs via TestAppend/TestString)",
	},
}

Incompat lists per-package tests that mvm cannot pass for reasons rooted in the bridge/interpreter design rather than a fixable bug in mvm's compiler. `mvm test` rewrites their entry to a t.Skip(reason) shim so they show as SKIP instead of FAIL, keeping the compat-matrix pass ratio honest.

Add an entry only when:

• the root cause is an architectural limit (bridge type erasure, reflect adapter frames, native-only protocols) -- not a bug worth chasing, AND
• the reason is short enough to land in the SKIP line without noise.

Drop the entry the moment the underlying limitation is fixed.

var ShortByDefault = map[string]bool{
	"sync/atomic":   true,
	"crypto/subtle": true,

	"github.com/gofrs/uuid": true,

	"github.com/russross/blackfriday/v2": true,
}

ShortByDefault lists import paths `mvm test` forces -short on (unless the caller set it): their stress tests loop 1e5-1e8 times -- minutes under the interpreter. Only list pkgs whose tests scale down under -short, not skip.

var TestValues = map[string]map[string]reflect.Value{
	"math": {
		"ExpGo":           reflect.ValueOf(math.Exp),
		"Exp2Go":          reflect.ValueOf(math.Exp2),
		"HypotGo":         reflect.ValueOf(math.Hypot),
		"SqrtGo":          reflect.ValueOf(math.Sqrt),
		"TrigReduce":      reflect.ValueOf(trigReduce),
		"ReduceThreshold": reflect.ValueOf(float64(reduceThreshold)),
	},
	"math/bits": {
		"DeBruijn64": reflect.ValueOf(uint64(0x03f79d71b4ca8b09)),
	},
	"reflect": {

		"MapGroupOf": reflect.ValueOf(func(x, y reflect.Type) reflect.Type {
			if x.Size() > 128 {
				x = reflect.PointerTo(x)
			}
			if y.Size() > 128 {
				y = reflect.PointerTo(y)
			}
			slot := reflect.StructOf([]reflect.StructField{
				{Name: "Key", Type: x},
				{Name: "Elem", Type: y},
			})
			return reflect.StructOf([]reflect.StructField{
				{Name: "Ctrl", Type: reflect.TypeFor[uint64]()},
				{Name: "Slots", Type: reflect.ArrayOf(8, slot)},
			})
		}),
	},
	"fmt": {
		"IsSpace":  reflect.ValueOf(fmtIsSpace),
		"Parsenum": reflect.ValueOf(fmtParsenum),
	},
	"strings": {
		"StringFind": reflect.ValueOf(func(pattern, text string) int {
			return strings.Index(text, pattern)
		}),
		"DumpTables": reflect.ValueOf(func(pattern string) ([]int, []int) {
			f := makeStringFinder(pattern)
			return f.badCharSkip[:], f.goodSuffixSkip
		}),
	},
	"go/types": {

		"CmpPos": reflect.ValueOf(func(p, q token.Pos) int { return int(p - q) }),
	},
}

TestValues holds faithful stand-ins for symbols that a stdlib package's own export_test.go injects into the package under test.

Merged over Values ONLY by `mvm test` (see TestOverlay); `mvm run` never sees these, so the real package surface stays clean.

Only faithfully-reproducible symbols belong here.

var Untestable = map[string]string{
	"runtime": "native-only: most external tests reference export_test.go symbols absent on the bridge, re-exec subprocesses, or use //go:linkname; the suite cannot complete under the interpreter",
}

Untestable lists packages whose whole test suite has no viable run under the interpreter, so `mvm test` skips them wholesale (exit 0, gray in the matrix). Coarser than Incompat, which skips individual tests.

var Values = map[string]map[string]reflect.Value{}

Values variable stores the map of stdlib values per package.

Functions

func EmbeddedStd

func EmbeddedStd() []byte

EmbeddedStd returns the Go-module-proxy-format zip bytes for the std module snapshot baked into this binary.

func ForceShort

func ForceShort(pkgPath string) bool

ForceShort reports whether pkgPath's tests should default to -short.

func GorootSrcDir

func GorootSrcDir(importPath string) string

GorootSrcDir returns the absolute path of $GOROOT/src/<importPath> on the host, or "" when GOROOT is unknown or the path is not a directory. Callers chdir there before driving bridged-stdlib tests so testdata- relative paths resolve against the host's stdlib source tree.

func GorootTestFS

func GorootTestFS() fs.FS

GorootTestFS returns an fs.FS rooted at $GOROOT/src, intended to serve stdlib *_test.go files for `mvm test <stdlib-path>` against the existing reflect bindings (e.g. `mvm test strings` loads strings_test.go from the host Go installation).

Returns nil if GOROOT cannot be determined or its src/ subtree is absent. Callers (the parser's test-source FS hook) must accept nil and treat it as "no test sources available" rather than failing.

Important: this FS is intentionally NOT chained into ordinary import resolution. Wiring it next to stdlibfs/remotefs would make `import "strings"` start loading interpreted source side-by-side with the reflect bridge, double-defining every exported symbol. It must only be consulted from the test-loading branch in LoadPackageSources.

func IsGenericOnly

func IsGenericOnly(pkgPath string) bool

IsGenericOnly reports whether pkgPath is a generic-only stub package.

func IsStdlibImport

func IsStdlibImport(name string) bool

IsStdlibImport reports whether path looks like a Go stdlib import path: non-empty and with no dot in its first segment (which would mark a module-qualified third-party path like example.com/x).

func PackagePatchers

func PackagePatchers() map[string][]PackagePatcher

PackagePatchers returns the registered patchers keyed by import path. Callers must not mutate the returned map.

func RegisterPackagePatcher

func RegisterPackagePatcher(importPath string, fn PackagePatcher)

RegisterPackagePatcher adds fn to the patcher list for importPath. Intended for init() in side-effect intercept packages such as stdlib/jsonx, so the interpreter itself needs no knowledge of them.

func SkipReason

func SkipReason(pkgPath, testName string) string

SkipReason returns the recorded reason for skipping testName when running `mvm test pkgPath`, or "" if the test should run normally. Subtest-path entries (names containing '/') are handled separately by SubtestSkips, so a top-level testName never matches one here.

func TestOverlay

func TestOverlay() map[string]map[string]reflect.Value

TestOverlay returns each TestValues package merged over its Values base, so a single ImportPackageValues installs the package with both its real bridge symbols and the test-only stand-ins.

func UntestableReason

func UntestableReason(pkgPath string) string

UntestableReason returns the wholesale-skip reason for pkgPath, or "" when its tests should run normally.

Types

type PackagePatcher

type PackagePatcher func(m *vm.Machine, values map[string]vm.Value)

PackagePatcher mutates a package's exported values at interpreter startup. m is the live VM; values is the package's vm.Value symbol map (the same map the interpreter resolves imports against).

type SubtestSkip

type SubtestSkip struct{ Name, Reason string }

SubtestSkip pairs a subtest path with its skip reason.

func SubtestSkips

func SubtestSkips(pkgPath string) []SubtestSkip

SubtestSkips returns pkgPath's Incompat entries whose name is a subtest path (contains '/', e.g. "TestX/Sub#03"), sorted by name. The driver skips these via testing's -skip so sibling subtests still run.