test

package
v3.113.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 16, 2024 License: Apache-2.0 Imports: 28 Imported by: 1

README

SDK Codegen Tests

TestSDKCodegen runs the complete set of SDK code generation tests against a particular language's code generator. It also verifies that the generated code is structurally sound.

The test files live in pkg/codegen/testing/test/testdata and are registered in the following globals in pkg/codegen/testing/test.

  • sdk_driver.go: PulumiPulumiSDKTests
  • program_driver.go: PulumiPulumiProgramTests
  • program_driver.go: PulumiPulumiYAMLProgramTests

An SDK code generation test files consists of a schema and a set of expected outputs for each language. Each test is structured as a directory that contains that information:

 testdata/
     my-simple-schema/   # i.e. `simple-enum-schema`
         schema.(json|yaml)
         go/
         python/
         nodejs/
         dotnet/
         ...

The schema is the only piece that must be manually authored.

Once the schema has been written, the actual codegen outputs can be generated by running the following in pkg/codegen directory:

PULUMI_ACCEPT=true go test ./...

This will rebuild subfolders such as go/ from scratch and store the set of code-generated file names in go/codegen-manifest.json. To generate the code for a specific directory in testdata, run the following instead:

PULUMI_ACCEPT=true go test ./... -run TestGenerate/$dirName

If these outputs look correct, they need to be checked into git and will then serve as the expected values for the normal test runs:

$ go test ./...

That is, the normal test runs will fail if changes to codegen or schema lead to a diff in the generated file set. If the diff is intentional, it can be accepted again via PULUMI_ACCEPT=true.

Writing Program Tests on Generated Code

To support running unit tests over the generated code, the tests also support mixing in manually written $lang-extras files into the generated tree. For example, given the following input:

 testdata/
     my-simple-schema/
         schema.json
         go/
         go-extras/
             tests/
                 go_test.go

The system will copy go-extras/tests/go_test.go into go/tests/go_test.go before performing compilation and unit test checks over the project generated in go.

Documentation

Index

Constants

View Source 
const PulumiDotnetSDKVersion = "3.60.0"

PulumiDotnetSDKVersion is the version of the Pulumi .NET SDK to use in program-gen tests

Variables

View Source 
var PulumiPulumiProgramTests = []ProgramTest{
	{
		Directory:   "assets-archives",
		Description: "Assets and archives",
	},
	{
		Directory:   "synthetic-resource-properties",
		Description: "Synthetic resource properties",
		SkipCompile: codegen.NewStringSet("nodejs", "dotnet", "go"),
	},
	{
		Directory:      "aws-s3-folder",
		Description:    "AWS S3 Folder",
		ExpectNYIDiags: codegen.NewStringSet("dotnet", "python"),
		SkipCompile:    codegen.NewStringSet("go", "python"),
	},
	{
		Directory:   "aws-eks",
		Description: "AWS EKS",
	},
	{
		Directory:   "aws-fargate",
		Description: "AWS Fargate",
	},
	{
		Directory:   "aws-static-website",
		Description: "an example resource from AWS static website multi-language component",

		SkipCompile: codegen.NewStringSet("dotnet", "python"),
	},
	{
		Directory:   "aws-fargate-output-versioned",
		Description: "AWS Fargate Using Output-versioned invokes for python and typescript",
		Skip:        codegen.NewStringSet("go", "dotnet"),
		BindOptions: []pcl.BindOption{pcl.PreferOutputVersionedInvokes},
	},
	{
		Directory:   "aws-s3-logging",
		Description: "AWS S3 with logging",
		SkipCompile: codegen.NewStringSet("go"),
	},
	{
		Directory:   "aws-iam-policy",
		Description: "AWS IAM Policy",
	},
	{
		Directory:   "read-file-func",
		Description: "ReadFile function translation works",
	},
	{
		Directory:   "python-regress-10914",
		Description: "Python regression test for #10914",
		Skip:        allProgLanguages.Except("python"),
	},
	{
		Directory:   "simplified-invokes",
		Description: "Simplified invokes",
		Skip:        codegen.NewStringSet("python", "go"),
		SkipCompile: codegen.NewStringSet("dotnet", "nodejs"),
	},
	{
		Directory:   "aws-optionals",
		Description: "AWS get invoke with nested object constructor that takes an optional string",

		Skip: allProgLanguages.Except("go"),
	},
	{
		Directory:   "aws-webserver",
		Description: "AWS Webserver",
	},
	{
		Directory:   "simple-range",
		Description: "Simple range as int expression translation",
	},
	{
		Directory:   "azure-native",
		Description: "Azure Native",
		SkipCompile: codegen.NewStringSet("go", "nodejs", "dotnet"),
	},
	{
		Directory:   "azure-sa",
		Description: "Azure SA",
	},
	{
		Directory:   "string-enum-union-list",
		Description: "Contains resource which has a property of type List<Union<String, Enum>>",

		SkipCompile: codegen.NewStringSet("go"),
	},
	{
		Directory:   "using-object-as-input-for-any",
		Description: "Tests using object as input for a property of type 'any'",
	},
	{
		Directory:   "kubernetes-operator",
		Description: "K8s Operator",
	},
	{
		Directory:   "kubernetes-pod",
		Description: "K8s Pod",
		SkipCompile: codegen.NewStringSet("go", "nodejs"),
	},
	{
		Directory:   "kubernetes-template",
		Description: "K8s Template",
	},
	{
		Directory:   "kubernetes-template-quoted",
		Description: "K8s Template with quoted string property keys to ensure that resource binding works here",
	},
	{
		Directory:   "random-pet",
		Description: "Random Pet",
	},
	{
		Directory:   "aws-secret",
		Description: "Secret",
	},
	{
		Directory:   "functions",
		Description: "Functions",
	},
	{
		Directory:   "output-funcs-aws",
		Description: "Output Versioned Functions",
	},
	{
		Directory:   "third-party-package",
		Description: "Ensuring correct imports for third party packages",

		SkipCompile: codegen.NewStringSet("nodejs", "dotnet", "go"),
	},
	{
		Directory:   "invalid-go-sprintf",
		Description: "Regress invalid Go",
		Skip:        codegen.NewStringSet("python", "nodejs", "dotnet"),
	},
	{
		Directory:   "typed-enum",
		Description: "Supply strongly typed enums",
		Skip:        codegen.NewStringSet(golang),
	},
	{
		Directory:   "pulumi-stack-reference",
		Description: "StackReference as resource",
	},
	{
		Directory:   "python-resource-names",
		Description: "Repro for #9357",
		Skip:        codegen.NewStringSet("go", "nodejs", "dotnet"),
	},
	{
		Directory:   "logical-name",
		Description: "Logical names",
	},
	{
		Directory:   "aws-lambda",
		Description: "AWS Lambdas",

		Skip: codegen.NewStringSet("go", "nodejs", "dotnet"),
	},
	{
		Directory:   "basic-unions",
		Description: "Tests program generation of fields of type union",
		SkipCompile: allProgLanguages,
	},
	{
		Directory:   "traverse-union-repro",
		Description: `Handling the error "cannot traverse value of type union(...)"`,
		BindOptions: []pcl.BindOption{
			pcl.SkipResourceTypechecking,
			pcl.AllowMissingVariables,
			pcl.AllowMissingProperties,
		},

		SkipCompile: allProgLanguages,
	},
	{
		Directory:   "components",
		Description: "Components",
		SkipCompile: codegen.NewStringSet("go"),
	},
	{
		Directory:   "entries-function",
		Description: "Using the entries function",

		SkipCompile: allProgLanguages.Except("nodejs").Except("python"),
	},
	{
		Directory:   "retain-on-delete",
		Description: "Generate RetainOnDelete option",
	},
	{
		Directory:   "depends-on-array",
		Description: "Using DependsOn resource option with an array of resources",
	},
	{
		Directory:   "multiline-string",
		Description: "Multiline string literals",
	},
	{
		Directory:   "config-variables",
		Description: "Basic program with a bunch of config variables",

		SkipCompile: codegen.NewStringSet("go", "dotnet"),
	},
	{
		Directory:   "regress-11176",
		Description: "Regression test for https://github.com/pulumi/pulumi/issues/11176",
		Skip:        allProgLanguages.Except("go"),
	},
	{
		Directory:   "throw-not-implemented",
		Description: "Function notImplemented is compiled to a runtime error at call-site",
	},
	{
		Directory:   "python-reserved",
		Description: "Test python reserved words aren't used",
		Skip:        allProgLanguages.Except("python"),
	},
	{
		Directory:   "iterating-optional-range-expressions",
		Description: "Test that we can iterate over range expression that are option(iterator)",

		Skip: allProgLanguages.Except("nodejs").Except("python"),

		SkipCompile: allProgLanguages,
	},
	{
		Directory:   "output-literals",
		Description: "Tests that we can return various literal values via stack outputs",
		SkipCompile: codegen.NewStringSet("go"),
	},
	{
		Directory:   "dynamic-entries",
		Description: "Testing iteration of dynamic entries in TypeScript",
		Skip:        allProgLanguages.Except("nodejs"),
		SkipCompile: allProgLanguages,
	},
	{
		Directory:   "single-or-none",
		Description: "Tests using the singleOrNone function",

		SkipCompile: codegen.NewStringSet("go"),
	},
	{
		Directory:   "simple-splat",
		Description: "An example that shows we can compile splat expressions from array of objects",

		SkipCompile: allProgLanguages,
	},
	{
		Directory:   "invoke-inside-conditional-range",
		Description: "Using the result of an invoke inside a conditional range expression of a resource",
		Skip:        allProgLanguages.Except("nodejs").Except("dotnet"),
		SkipCompile: allProgLanguages,
	},
	{
		Directory:   "output-name-conflict",
		Description: "Tests whether we are able to generate programs where output variables have same id as config var",
		SkipCompile: codegen.NewStringSet("go"),
	},
	{
		Directory:   "snowflake-python-12998",
		Description: "Tests regression for issue https://github.com/pulumi/pulumi/issues/12998",
		Skip:        allProgLanguages.Except("python"),
		SkipCompile: allProgLanguages,
		BindOptions: []pcl.BindOption{pcl.AllowMissingVariables, pcl.AllowMissingProperties},
	},
	{
		Directory:   "unknown-resource",
		Description: "Tests generating code for unknown resources when skipping resource type-checking",
		SkipCompile: allProgLanguages,
		BindOptions: []pcl.BindOption{pcl.SkipResourceTypechecking},
	},
	{
		Directory:   "using-dashes",
		Description: "Test program generation on packages with a dash in the name",
		SkipCompile: allProgLanguages,
	},
	{
		Directory:   "unknown-invoke",
		Description: "Tests generating code for unknown invokes when skipping invoke type checking",
		SkipCompile: allProgLanguages,
		BindOptions: []pcl.BindOption{pcl.SkipInvokeTypechecking},
	},
	{
		Directory:   "optional-complex-config",
		Description: "Tests generating code for optional and complex config values",
		Skip:        allProgLanguages.Except("nodejs").Except("dotnet"),
		SkipCompile: allProgLanguages.Except("nodejs").Except("dotnet"),
	},
	{
		Directory:   "interpolated-string-keys",
		Description: "Tests that interpolated string keys are supported in maps. ",
		Skip:        allProgLanguages.Except("nodejs").Except("python"),
	},
	{
		Directory:   "regress-node-12507",
		Description: "Regression test for https://github.com/pulumi/pulumi/issues/12507",
		Skip:        allProgLanguages.Except("nodejs"),
		BindOptions: []pcl.BindOption{pcl.PreferOutputVersionedInvokes},
	},
	{
		Directory:   "csharp-plain-lists",
		Description: "Tests that plain lists are supported in C#",
		Skip:        allProgLanguages.Except("dotnet"),
	},
	{
		Directory:   "csharp-typed-for-expressions",
		Description: "Testing for expressions with typed target expressions in csharp",
		Skip:        allProgLanguages.Except("dotnet"),
	},
	{
		Directory:   "empty-list-property",
		Description: "Tests compiling empty list expressions of object properties",
	},
	{
		Directory:   "python-regress-14037",
		Description: "Regression test for rewriting qoutes in python",
		Skip:        allProgLanguages.Except("python"),
	},
	{
		Directory:   "inline-invokes",
		Description: "Tests whether using inline invoke expressions works",
		SkipCompile: codegen.NewStringSet("go"),
	},
}
View Source 
var PulumiPulumiSDKTests = []*SDKTest{
	{
		Directory:   "naming-collisions",
		Description: "Schema with types that could potentially produce collisions.",
	},
	{
		Directory:   "dash-named-schema",
		Description: "Simple schema with a two part name (foo-bar)",
	},
	{
		Directory:        "external-resource-schema",
		Description:      "External resource schema",
		SkipCompileCheck: codegen.NewStringSet(golang),
	},
	{
		Directory:   "nested-module",
		Description: "Nested module",
	},
	{
		Directory:   "simplified-invokes",
		Description: "Simplified invokes",
		Skip:        codegen.NewStringSet("python/any", "go/any"),
	},
	{
		Directory:   "nested-module-thirdparty",
		Description: "Third-party nested module",
	},
	{
		Directory:   "plain-schema-gh6957",
		Description: "Repro for #6957",
	},
	{
		Directory:   "resource-args-python-case-insensitive",
		Description: "Resource args with same named resource and type case insensitive",
	},
	{
		Directory:   "resource-args-python",
		Description: "Resource args with same named resource and type",
	},
	{
		Directory:   "simple-enum-schema",
		Description: "Simple schema with enum types",
	},
	{
		Directory:   "simple-plain-schema",
		Description: "Simple schema with plain properties",
	},
	{
		Directory:   "simple-plain-schema-with-root-package",
		Description: "Simple schema with root package set",
	},
	{
		Directory:   "simple-schema-pyproject",
		Description: "A simple schema that generates a pyproject.toml file",
		Skip:        codegen.NewStringSet("go/any", "nodejs/any", "dotnet/any"),
	},
	{
		Directory:   "simple-resource-schema",
		Description: "Simple schema with local resource properties",
	},
	{
		Directory:   "simple-resource-schema-custom-pypackage-name",
		Description: "Simple schema with local resource properties and custom Python package name",
	},
	{
		Directory:        "simple-methods-schema",
		Description:      "Simple schema with methods",
		SkipCompileCheck: codegen.NewStringSet(nodejs, golang),
	},
	{
		Directory:   "simple-methods-schema-single-value-returns",
		Description: "Simple schema with methods that return single values",
	},
	{
		Directory:   "methods-return-plain-resource",
		Description: "Test returning plain Resource objects from methods",
		Skip:        codegen.NewStringSet("dotnet/any"),
	},
	{
		Directory:   "simple-yaml-schema",
		Description: "Simple schema encoded using YAML",
	},
	{
		Directory:   "provider-config-schema",
		Description: "Simple provider config schema",

		SkipCompileCheck: codegen.NewStringSet(dotnet, golang),
	},
	{
		Directory:   "replace-on-change",
		Description: "Simple use of replaceOnChange in schema",
	},
	{
		Directory:   "simple-resource-with-aliases",
		Description: "Simple schema with a resource that has aliases",
	},
	{
		Directory:        "resource-property-overlap",
		Description:      "A resource with the same name as its property",
		SkipCompileCheck: codegen.NewStringSet(dotnet, nodejs),
	},
	{
		Directory:   "hyphen-url",
		Description: "A resource url with a hyphen in its path",
		Skip:        codegen.NewStringSet("go/any"),
	},
	{
		Directory:   "output-funcs",
		Description: "Tests targeting the $fn_output helper code generation feature",
	},
	{
		Directory:        "output-funcs-edgeorder",
		Description:      "Regresses Node compilation issues on a subset of azure-native",
		SkipCompileCheck: codegen.NewStringSet(golang, python),
		Skip:             codegen.NewStringSet("nodejs/test"),
	},
	{
		Directory:        "output-funcs-tfbridge20",
		Description:      "Similar to output-funcs, but with compatibility: tfbridge20, to simulate pulumi-aws use case",
		SkipCompileCheck: codegen.NewStringSet(python),
	},
	{
		Directory:   "cyclic-types",
		Description: "Cyclic object types",
	},
	{
		Directory:   "regress-node-8110",
		Description: "Test the fix for pulumi/pulumi#8110 nodejs compilation error",
		Skip:        codegen.NewStringSet("go/test", "dotnet/test"),
	},
	{
		Directory:   "dashed-import-schema",
		Description: "Ensure that we handle all valid go import paths",
		Skip:        codegen.NewStringSet("go/test", "dotnet/test"),
	},
	{
		Directory:   "plain-and-default",
		Description: "Ensure that a resource with a plain default property works correctly",
	},
	{
		Directory:   "plain-object-defaults",
		Description: "Ensure that object defaults are generated (repro #8132)",
	},
	{
		Directory:   "plain-object-disable-defaults",
		Description: "Ensure that we can still compile safely when defaults are disabled",
	},
	{
		Directory:        "regress-8403",
		Description:      "Regress pulumi/pulumi#8403",
		SkipCompileCheck: codegen.NewStringSet(python),
	},
	{
		Directory:   "different-package-name-conflict",
		Description: "different packages with the same resource",
		Skip:        allLanguages,
	},
	{
		Directory:   "different-enum",
		Description: "An enum in a different package namespace",
		Skip:        codegen.NewStringSet("dotnet/compile"),
	},
	{
		Directory:   "array-of-enum-map",
		Description: "A schema with an array of maps where the values are enums. Issue pulumi/pulumi#14734",
		Skip:        allLanguages.Except("go/any"),
	},
	{
		Directory:   "azure-native-nested-types",
		Description: "Condensed example of nested collection types from Azure Native",
		Skip:        codegen.NewStringSet("go/any"),
	},
	{
		Directory:   "regress-go-8664",
		Description: "Regress pulumi/pulumi#8664 affecting Go",
		Skip:        allLanguages.Except("go/any"),
	},
	{
		Directory:   "regress-go-10527",
		Description: "Regress pulumi/pulumi#10527 affecting Go",
		Skip:        allLanguages.Except("go/any"),
	},
	{
		Directory:   "regress-go-12971",
		Description: "Regress pulumi/pulumi#12971 affecting Go",
		Skip:        allLanguages.Except("go/any"),
	},
	{
		Directory:   "regress-go-15478",
		Description: "Regress pulumi/pulumi#15478 affecting Go ensuring SDK-gen doesn't panic",
		Skip:        allLanguages.Except("go/any"),

		SkipCompileCheck: codegen.NewStringSet(golang),
	},
	{
		Directory:   "regress-py-12546",
		Description: "Regress pulumi/pulumi#12546 affecting Python",
		Skip:        allLanguages.Except("python/any"),
	},
	{
		Directory:   "docs-collision",
		Description: "Tests that resources and functions with the same name do not clobber each other.",
		Skip:        allLanguages.Except("docs/any"),
	},
	{
		Directory:   "using-shared-types-in-config",
		Description: "Tests that shared types can be used in config.",
	},
	{
		Directory:   "other-owned",
		Description: "CSharp rootNamespaces",

		Skip: allLanguages.Except("dotnet/any").Except("docs/any"),
	},
	{
		Directory: "external-node-compatibility",

		Description: "Ensure external package compatibility modes are used when referencing external types",
		Skip:        allLanguages.Except("nodejs/any"),
	},
	{
		Directory: "external-go-import-aliases",

		Description: "Ensure external import aliases are honored, and any local import aliases override them",
		Skip:        allLanguages.Except("go/any"),
	},
	{
		Directory:   "external-python-same-module-name",
		Description: "Ensure referencing external types/resources with the same module name are referenced correctly",
		Skip:        allLanguages.Except("python/any"),
	},
	{
		Directory:   "enum-reference",
		Description: "Ensure referencing external types/resources with referenced enums import correctly",
	},
	{
		Directory:   "enum-reference-python",
		Description: "Ensure referencing external types/resources with referenced enums import correctly in Python",
		Skip:        allLanguages.Except("python/any"),
	},
	{
		Directory:   "external-enum",
		Description: "Ensure we generate valid tokens for external enums",
		Skip:        codegen.NewStringSet("dotnet/any"),
	},
	{
		Directory:   "internal-dependencies-go",
		Description: "Emit Go internal dependencies",
		Skip:        allLanguages.Except("go/any"),
	},
	{
		Directory:   "go-overridden-internal-module-name",
		Description: "Go SDK where the internal module name is overridden to be 'utilities'",
		Skip:        allLanguages.Except("go/any"),
	},
	{
		Directory:   "go-plain-ref-repro",
		Description: "Generate a resource that accepts a plain input type",
		Skip:        allLanguages.Except("go/any"),
	},
	{
		Directory:   "go-nested-collections",
		Description: "Generate a resource that outputs [][][]Foo",
		Skip:        allLanguages.Except("go/any"),
	},
	{
		Directory: "functions-secrets",

		Description: "functions that have properties that are secrets in the schema",
	},
	{
		Directory:        "secrets",
		Description:      "Generate a resource with secret properties",
		SkipCompileCheck: codegen.NewStringSet(dotnet),
	},
	{
		Directory:   "regress-py-tfbridge-611",
		Description: "Regresses pulumi/pulumi-terraform-bridge#611",
		Skip:        allLanguages.Except("python/any").Union(codegen.NewStringSet("python/test", "python/py_compile")),
	},
	{
		Directory:   "hyphenated-symbols",
		Description: "Test that types can have names with hyphens in them",
		Skip:        allLanguages.Except("go/any").Except("python/any"),
	},
	{
		Directory:   "provider-type-schema",
		Description: "A schema with a type called Provider schema",
	},
	{
		Directory:   "embedded-crd-types",
		Description: "A schema with CRD types with package names different from the main package",
		Skip:        codegen.NewStringSet("dotnet/any"),
	},
	{
		Directory:   "unions-inside-arrays",
		Description: "A schema with a union type inside an array",
	},
	{
		Directory:   "assets-and-archives",
		Description: "A schema with assets and archives",
	},
	{
		Directory:   "regress-py-14012",
		Description: "Regresses https://github.com/pulumi/pulumi/issues/14012",
		Skip:        allLanguages.Except("python/any"),
	},
	{
		Directory:   "regress-py-14539",
		Description: "Regresses https://github.com/pulumi/pulumi/issues/14539",
		Skip:        allLanguages.Except("python/any"),
	},
	{
		Directory:   "output-funcs-go-generics-only",
		Description: "Tests targeting the $fn_output code generation feature, only for Go generics == generics-only",
		Skip:        allLanguages.Except("go/any"),
	},
	{
		Directory:   "plain-and-default-go-generics-only",
		Description: "resource with a plain default property works, only for Go generics == generics-only",
		Skip:        allLanguages.Except("go/any"),
	},
	{
		Directory:   "secrets-go-generics-only",
		Description: "Generate a resource with secret properties, only for Go generics == generics-only",
		Skip:        allLanguages.Except("go/any"),
	},
	{
		Directory:   "simple-enum-schema-go-generics-only",
		Description: "Simple schema with enum types, only for Go generics == generics-only",
		Skip:        allLanguages.Except("go/any"),
	},
	{
		Directory:   "simple-plain-schema-go-generics-only",
		Description: "Simple schema with plain properties, only for Go generics == generics-only",
		Skip:        allLanguages.Except("go/any"),
	},
	{
		Directory:   "assets-and-archives-go-generics-only",
		Description: "Testing generating a schema with assets and archives for go using generics-only",
		Skip:        allLanguages.Except("go/any"),
	},
	{
		Directory:   "urn-id-properties",
		Description: "Testing urn and id properties in valid locations",
	},
	{
		Directory:   "regress-py-12980",
		Description: "Import resources across modules",
		Skip:        allLanguages.Except("python/any"),
	},
	{
		Directory:   "unions-inline",
		Description: "Testing the use of unions/oneOf in the schema inline with the property definition.",
	},
	{
		Directory:   "legacy-names",
		Description: "Testing the use of snake_case names and tokens.",
		Skip:        codegen.NewStringSet("go/test"),
	},
	{
		Directory:   "kubernetes20",
		Description: "Testing the kubernetes20 compatibility mode.",
		Skip:        codegen.NewStringSet("go/test"),
	},
}
View Source 
var PulumiPulumiYAMLProgramTests = []ProgramTest{

	{
		Directory:   transpiled("aws-eks"),
		Description: "AWS EKS",
		Skip:        codegen.NewStringSet("go", "nodejs", "dotnet"),
	},
	{
		Directory:   transpiled("aws-static-website"),
		Description: "AWS static website",
		Skip:        codegen.NewStringSet("go", "nodejs", "dotnet"),
		BindOptions: []pcl.BindOption{pcl.SkipResourceTypechecking},
	},
	{
		Directory:   transpiled("awsx-fargate"),
		Description: "AWSx Fargate",
		Skip:        codegen.NewStringSet("dotnet", "nodejs", "go"),
	},
	{
		Directory:   transpiled("azure-app-service"),
		Description: "Azure App Service",
		Skip:        codegen.NewStringSet("go", "dotnet"),
	},
	{
		Directory:   transpiled("azure-container-apps"),
		Description: "Azure Container Apps",
		Skip:        codegen.NewStringSet("go", "nodejs", "dotnet", "python"),
	},
	{
		Directory:   transpiled("azure-static-website"),
		Description: "Azure static website",
		Skip:        codegen.NewStringSet("go", "nodejs", "dotnet", "python"),
	},
	{
		Directory:   transpiled("cue-eks"),
		Description: "Cue EKS",
		Skip:        codegen.NewStringSet("go", "nodejs", "dotnet"),
	},
	{
		Directory:   transpiled("cue-random"),
		Description: "Cue random",
	},
	{
		Directory:   transpiled("getting-started"),
		Description: "Getting started",
	},
	{
		Directory:   transpiled("kubernetes"),
		Description: "Kubernetes",
		Skip:        codegen.NewStringSet("go"),
	},
	{
		Directory:   transpiled("pulumi-variable"),
		Description: "Pulumi variable",
		Skip:        codegen.NewStringSet("go", "nodejs", "dotnet"),
	},
	{
		Directory:   transpiled("random"),
		Description: "Random",
		Skip:        codegen.NewStringSet("nodejs"),
	},
	{
		Directory:   transpiled("readme"),
		Description: "README",
		Skip:        codegen.NewStringSet("go", "dotnet"),
	},
	{
		Directory:   transpiled("stackreference-consumer"),
		Description: "Stack reference consumer",
		Skip:        codegen.NewStringSet("go", "nodejs", "dotnet"),
	},
	{
		Directory:   transpiled("stackreference-producer"),
		Description: "Stack reference producer",
		Skip:        codegen.NewStringSet("go", "dotnet"),
	},
	{
		Directory:   transpiled("webserver-json"),
		Description: "Webserver JSON",
		Skip:        codegen.NewStringSet("go", "dotnet", "python"),
	},
	{
		Directory:   transpiled("webserver"),
		Description: "Webserver",
		Skip:        codegen.NewStringSet("go", "dotnet", "python"),
	},
}

Functions

func CheckAllFilesGenerated 

func CheckAllFilesGenerated(t *testing.T, actual, expected map[string][]byte)

CheckAllFilesGenerated ensures that the set of expected and actual files generated are exactly equivalent.

func CheckVersion added in v3.40.2 

func CheckVersion(t *testing.T, dir, depFilePath string, expectedVersionMap map[string]PkgVersionInfo)

CheckVersion checks for an expected package version Todo: support checking multiple package expected versions

func CopyExtraFiles 

func CopyExtraFiles(t *testing.T, dir, lang string)

Useful for populating code-generated destination `codeDir=$dir/$lang` with extra manually written files such as the unit test files. These files are copied from `$dir/$lang-extras` folder if present.

func GenProjectCleanUp added in v3.40.2 

func GenProjectCleanUp(t *testing.T, dir, depFilePath, outfilePath string)

func GeneratePackageFilesFromSchema 

func GeneratePackageFilesFromSchema(schemaPath string, genPackageFunc GenPkgSignature) (map[string][]byte, error)

GeneratePackageFilesFromSchema loads a schema and generates files using the provided GeneratePackage function.

func LoadBaseline 

func LoadBaseline(dir, lang string) (map[string][]byte, error)

`LoadBaseline` loads the contents of the given baseline directory, by inspecting its `codegen-manifest.json`.

func LoadFiles 

func LoadFiles(dir, lang string, files []string) (map[string][]byte, error)

LoadFiles loads the provided list of files from a directory.

func NoSDKCodegenChecks 

func NoSDKCodegenChecks() bool

func PathExists 

func PathExists(path string) (bool, error)

func RewriteFilesWhenPulumiAccept 

func RewriteFilesWhenPulumiAccept(t *testing.T, dir, lang string, actual map[string][]byte) bool

If PULUMI_ACCEPT is set, writes out actual output to the expected file set, so we can continue enjoying golden tests without manually modifying the expected output.

func RunCommand 

func RunCommand(t *testing.T, name string, cwd string, exec string, args ...string)

func RunCommandWithOptions 

func RunCommandWithOptions(
	t *testing.T,
	opts *integration.ProgramTestOptions,
	name string, cwd string, exec string, args ...string,
)

func TestProgramCodegen 

func TestProgramCodegen(
	t *testing.T,
	testcase ProgramCodegenOptions,
)

TestProgramCodegen runs the complete set of program code generation tests against a particular language's code generator.

A program code generation test consists of a PCL file (.pp extension) and a set of expected outputs for each language.

The PCL file is the only piece that must be manually authored. Once the schema has been written, the expected outputs can be generated by running `PULUMI_ACCEPT=true go test ./..." from the `pkg/codegen` directory.

func TestSDKCodegen 

func TestSDKCodegen(t *testing.T, opts *SDKCodegenOptions)

TestSDKCodegen runs the complete set of SDK code generation tests against a particular language's code generator. It also verifies that the generated code is structurally sound.

The test files live in `pkg/codegen/testing/test/testdata` and are registered in `var sdkTests` in `sdk_driver.go`.

An SDK code generation test files consists of a schema and a set of expected outputs for each language. Each test is structured as a directory that contains that information: 

testdata/
    my-simple-schema/   # i.e. `simple-enum-schema`
        schema.(json|yaml)
        go/
        python/
        nodejs/
        dotnet/
        ...

The schema is the only piece that *must* be manually authored.

Once the schema has been written, the actual codegen outputs can be generated by running the following in `pkg/codegen` directory: 

PULUMI_ACCEPT=true go test ./...

This will rebuild subfolders such as `go/` from scratch and store the set of code-generated file names in `go/codegen-manifest.json`. If these outputs look correct, they need to be checked into git and will then serve as the expected values for the normal test runs: 

go test ./...

That is, the normal test runs will fail if changes to codegen or schema lead to a diff in the generated file set. If the diff is intentional, it can be accepted again via `PULUMI_ACCEPT=true`.

To support running unit tests over the generated code, the tests also support mixing in manually written `$lang-extras` files into the generated tree. For example, given the following input: 

testdata/
    my-simple-schema/
        schema.json
        go/
        go-extras/
            tests/
                go_test.go

The system will copy `go-extras/tests/go_test.go` into `go/tests/go_test.go` before performing compilation and unit test checks over the project generated in `go`.

func TestTypeNameCodegen 

func TestTypeNameCodegen(t *testing.T, language string, newTypeNameGenerator NewTypeNameGeneratorFunc)

func ValidateFileEquality 

func ValidateFileEquality(t *testing.T, actual, expected map[string][]byte) bool

ValidateFileEquality compares maps of files for equality.

func ValidateFileTransformer 

func ValidateFileTransformer(
	t *testing.T,
	inputFile string,
	expectedOutputFile string,
	transformer func(reader io.Reader, writer io.Writer) error,
)

Validates a transformer on a single file.

Types

type CheckProgramOutput 

type CheckProgramOutput = func(*testing.T, string, codegen.StringSet)

Checks that a generated program is correct

The arguments are to be read: (Testing environment, path to generated code, set of dependencies)

type CodegenCheck 

type CodegenCheck func(t *testing.T, codedir string)

Defines an extra check logic that accepts the directory with the generated code, typically `$TestDir/$test.Directory/$language`.

type GenPkgSignature 

type GenPkgSignature func(string, *schema.Package, map[string][]byte) (map[string][]byte, error)

GenPkgSignature corresponds to the shape of the codegen GeneratePackage functions.

type GenProgram 

type GenProgram = func(program *pcl.Program) (map[string][]byte, hcl.Diagnostics, error)

Generates a program from a pcl.Program

type GenProject added in v3.40.2 

type GenProject = func(
	directory string, project workspace.Project,
	program *pcl.Program, localDependencies map[string]string,
) error

Generates a project from a pcl.Program

type NewTypeNameGeneratorFunc 

type NewTypeNameGeneratorFunc func(pkg *schema.Package) TypeNameGeneratorFunc

type PkgVersionInfo added in v3.40.2 

type PkgVersionInfo struct {
	Pkg          string
	OpAndVersion string
}

type ProgramCodegenOptions 

type ProgramCodegenOptions struct {
	Language   string
	Extension  string
	OutputFile string
	Check      CheckProgramOutput
	GenProgram GenProgram
	TestCases  []ProgramTest

	// For generating a full project
	IsGenProject bool
	GenProject   GenProject
	// Maps a test file (i.e. "aws-resource-options") to a struct containing a package
	// (i.e. "github.com/pulumi/pulumi-aws/sdk/v5", "pulumi-aws) and its
	// version prefixed by an operator (i.e. " v5.11.0", ==5.11.0")
	ExpectedVersion map[string]PkgVersionInfo
	DependencyFile  string
}

type ProgramTest 

type ProgramTest struct {
	Directory          string
	Description        string
	Skip               codegen.StringSet
	ExpectNYIDiags     codegen.StringSet
	SkipCompile        codegen.StringSet
	BindOptions        []pcl.BindOption
	MockPluginVersions map[string]string
}

func ProgramTestBatch added in v3.44.0 

func ProgramTestBatch(k, n int) []ProgramTest

Get batch number k (base-1 indexed) of tests out of n batches total.

func SingleTestCase added in v3.78.0 

func SingleTestCase(directoryName string) []ProgramTest

Useful when debugging a single test case.

type SDKCodegenOptions 

type SDKCodegenOptions struct {
	// Name of the programming language.
	Language string

	// Language-aware code generator; such as `GeneratePackage`.
	// from `codegen/dotnet`.
	GenPackage GenPkgSignature

	// Extra checks for all the tests. They keys of this map are
	// of the form "$language/$check" such as "go/compile".
	Checks map[string]CodegenCheck

	// The tests to run. A testcase `tt` are assumed to be located at
	// ../testing/test/testdata/${tt.Directory}
	TestCases []*SDKTest
}

SDKCodegenOptions describes the set of codegen tests for a language.

type SDKTest 

type SDKTest struct {
	Directory   string
	Description string

	// Extra checks for this test. They keys of this map
	// are of the form "$language/$check" such as "go/compile".
	Checks map[string]CodegenCheck

	// Skip checks, identified by "$language/$check".
	// "$language/any" is special, skipping generating the
	// code as well as all tests.
	Skip codegen.StringSet

	// Do not compile the generated code for the languages in this set.
	// This is a helper form of `Skip`.
	SkipCompileCheck codegen.StringSet

	// Mutex to ensure only a single test operates on directory at a time
	Mutex sync.Mutex
}

func (*SDKTest) ShouldSkipCodegen 

func (tt *SDKTest) ShouldSkipCodegen(language string) bool

ShouldSkipCodegen determines if codegen should be run. ShouldSkipCodegen=true further implies no other tests will be run.

func (*SDKTest) ShouldSkipTest 

func (tt *SDKTest) ShouldSkipTest(language, test string) bool

ShouldSkipTest indicates if a given test for a given language should be run.

type SchemaVersion 

type SchemaVersion = string
const (
	AwsSchema              SchemaVersion = "4.26.0"
	AzureNativeSchema      SchemaVersion = "1.29.0"
	AzureSchema            SchemaVersion = "4.18.0"
	KubernetesSchema       SchemaVersion = "3.7.2"
	RandomSchema           SchemaVersion = "4.11.2"
	EksSchema              SchemaVersion = "0.37.1"
	AwsStaticWebsiteSchema SchemaVersion = "0.4.0"
	AwsNativeSchema        SchemaVersion = "0.99.0"
)

Schemas are downloaded in the makefile, and the versions specified here should be in sync with the makefile.

type TypeNameGeneratorFunc 

type TypeNameGeneratorFunc func(t schema.Type) string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.