metamorphic

Documentation

Index

• Constants
• func ConstantWithTestBool(name string, defaultValue bool) bool
• func ConstantWithTestBoolWithoutLogging(name string, defaultValue bool) bool
• func ConstantWithTestChoice[T any](name string, defaultValue T, otherValues ...T) T
• func ConstantWithTestRange(name string, defaultValue, min, max int) int
• func ConstantWithTestValue(name string, defaultValue, metamorphicValue int) int
• func IsMetamorphicBuild() bool

Constants

const (
	// DisableMetamorphicEnvVar can be used to disable metamorphic tests for
	// sub-processes. If it exists and is set to something truthy as defined by
	// strconv.ParseBool then metamorphic testing will not be enabled.
	DisableMetamorphicEnvVar = "COCKROACH_INTERNAL_DISABLE_METAMORPHIC_TESTING"
	// MetamorphicOverridesEnvVar can be used to set metamorphic
	// variables to specific values.
	MetamorphicOverridesEnvVar = "COCKROACH_INTERNAL_METAMORPHIC_OVERRIDES"
)

const (
	IsMetamorphicBuildProbability = 0.8
)

Functions

func ConstantWithTestBool

func ConstantWithTestBool(name string, defaultValue bool) bool

ConstantWithTestBool is like ConstantWithTestValue except it returns the non-default value half of the time (if running a metamorphic build).

The given name is used for logging.

func ConstantWithTestBoolWithoutLogging

func ConstantWithTestBoolWithoutLogging(name string, defaultValue bool) bool

ConstantWithTestBoolWithoutLogging is like ConstantWithTestBool except it does not log the value. This is necessary to work around this issue: https://github.com/cockroachdb/cockroachdb-parser/issues/106667 TODO(test-eng): Remove this variant when the issue above is addressed.

func ConstantWithTestChoice

func ConstantWithTestChoice[T any](name string, defaultValue T, otherValues ...T) T

ConstantWithTestChoice is like ConstantWithTestValue except it returns a random choice (equally weighted) of the given values. The default value is included in the random choice.

The given name is used for logging.

func ConstantWithTestRange

func ConstantWithTestRange(name string, defaultValue, min, max int) int

ConstantWithTestRange is like ConstantWithTestValue except instead of returning a single metamorphic test value, it returns a random test value in the semi-open range [min, max).

The given name is used for logging.

func ConstantWithTestValue

func ConstantWithTestValue(name string, defaultValue, metamorphicValue int) int

ConstantWithTestValue should be used to initialize "magic constants" that should be varied during test scenarios to check for bugs at boundary conditions. When metamorphicutil.IsMetamorphicBuild is true, the test value will be used with metamorphicValueProbability probability. In all other cases, the production ("default") value will be used. The constant must be a "metamorphic variable": changing it cannot affect the output of any SQL DMLs. It can only affect the way in which the data is retrieved or processed, because otherwise the main test corpus would fail if this flag were enabled.

An example of a "magic constant" that behaves this way is a batch size. Batch sizes tend to present testing problems, because often the logic that deals with what to do when a batch is finished is less likely to be exercised by simple unit tests that don't use enough data to fill up a batch.

For example, instead of writing:

const batchSize = 64

you should write:

var batchSize = metamorphic.ConstantWithTestValue("batch-size", 64, 1)

This will often give your code a batch size of 1 in the crdb_test build configuration, increasing the amount of exercise the edge conditions get.

The given name is used for logging.

func IsMetamorphicBuild

func IsMetamorphicBuild() bool

IsMetamorphicBuild returns whether this build is metamorphic. By build being "metamorphic" we mean that some magic constants in the codebase might get initialized to non-default value. A build will become metamorphic with metamorphicutil.IsMetamorphicBuildProbability probability if 'crdb_test' build flag is specified (this is the case for all test targets).