ingest

Documentation

Overview

Package ingest handles parsing of Pulumi infrastructure definitions.

The primary function is converting "pulumi preview --json" output into resource descriptors that can be processed by the cost calculation engine.

Pulumi Plan Parsing

The package extracts from Pulumi preview JSON:

• Resource types (e.g., "aws:ec2:Instance")
• Provider information
• Resource properties and configurations
• Resource dependencies and relationships

Resource Descriptors

Output is a normalized set of ResourceDescriptor objects that provide a provider-agnostic view of infrastructure resources for cost analysis.

Index

• Constants
• func MapResource(pulumiResource PulumiResource) (engine.ResourceDescriptor, error)
• func MapResources(resources []PulumiResource) ([]engine.ResourceDescriptor, error)
• func MapStateResource(resource StackExportResource) (engine.ResourceDescriptor, error)
• func MapStateResources(resources []StackExportResource) ([]engine.ResourceDescriptor, error)
• func MergeProperties(outputs, inputs map[string]interface{}) map[string]interface{}
• type PulumiPlan
  • func LoadPulumiPlan(path string) (*PulumiPlan, error)
  • func LoadPulumiPlanWithContext(ctx context.Context, path string) (*PulumiPlan, error)
  • func ParsePulumiPlan(data []byte) (*PulumiPlan, error)
  • func ParsePulumiPlanWithContext(ctx context.Context, data []byte) (*PulumiPlan, error)
  • func (p *PulumiPlan) GetResources() []PulumiResource
  • func (p *PulumiPlan) GetResourcesWithContext(ctx context.Context) []PulumiResource
• type PulumiResource
• type PulumiState
• type PulumiStep
• type StackExport
  • func LoadStackExport(path string) (*StackExport, error)
  • func LoadStackExportWithContext(ctx context.Context, path string) (*StackExport, error)
  • func ParseStackExport(data []byte) (*StackExport, error)
  • func ParseStackExportWithContext(ctx context.Context, data []byte) (*StackExport, error)
  • func (s *StackExport) GetCustomResources() []StackExportResource
  • func (s *StackExport) GetCustomResourcesWithContext(ctx context.Context) []StackExportResource
  • func (s *StackExport) GetResourceByURN(urn string) *StackExportResource
  • func (s *StackExport) HasTimestamps() bool
• type StackExportDeployment
• type StackExportManifest
• type StackExportResource

Constants

const (
	// PropertyPulumiCreated is the property key for resource creation timestamp.
	PropertyPulumiCreated = "pulumi:created"
	// PropertyPulumiModified is the property key for resource modification timestamp.
	PropertyPulumiModified = "pulumi:modified"
	// PropertyPulumiExternal indicates the resource was imported (not created by Pulumi).
	PropertyPulumiExternal = "pulumi:external"
	// PropertyPulumiCloudID is the cloud-provider resource ID (e.g., "i-0abc123", "db-instance-primary").
	PropertyPulumiCloudID = "pulumi:cloudId"
	// PropertyPulumiARN is the canonical cloud identifier from outputs (e.g., AWS ARN).
	PropertyPulumiARN = "pulumi:arn"
	// PropertyPulumiURN is the Pulumi URN preserved for correlation.
	PropertyPulumiURN = "pulumi:urn"
)

Property keys for Pulumi metadata injected into ResourceDescriptor.Properties.

Functions

func MapResource

func MapResource(pulumiResource PulumiResource) (engine.ResourceDescriptor, error)

MapResource converts a PulumiResource into an engine.ResourceDescriptor. The returned descriptor contains the resource Type, URN as ID, the provider derived from the resource type, and Properties produced by merging the resource's outputs with its inputs (inputs take precedence). The function does not currently produce an error; the returned error is nil.

func MapResources

func MapResources(resources []PulumiResource) ([]engine.ResourceDescriptor, error)

MapResources converts multiple Pulumi resources to ResourceDescriptors.

func MapStateResource

func MapStateResource(resource StackExportResource) (engine.ResourceDescriptor, error)

MapStateResource converts a StackExportResource to a ResourceDescriptor. Timestamps, cloud identifiers (ID, ARN), and outputs are injected into Properties. Properties are built by merging Outputs (base) with Inputs (overlay), so provider- computed values like size, iops, and tagsAll are included while user-declared inputs MapStateResource converts a StackExportResource into an engine.ResourceDescriptor. It merges the resource's outputs with its inputs (inputs take precedence on conflict), injects Pulumi-specific metadata (created/modified timestamps as RFC3339 strings, external flag, cloud resource ID, URN), and, if present, copies the merged "arn" property into the Pulumi ARN key. The given resource's Type becomes the descriptor Type and the resource URN is used as the descriptor ID.

The resource parameter is the StackExportResource to convert.

It returns the mapped engine.ResourceDescriptor and an error. The function currently does not return non-nil errors.

func MapStateResources

func MapStateResources(resources []StackExportResource) ([]engine.ResourceDescriptor, error)

MapStateResources converts a slice of StackExportResource into a slice of engine.ResourceDescriptor. It maps each resource using MapStateResource and preserves the input order. If mapping any resource fails, it returns an error that wraps the underlying error and includes the resource URN.

func MergeProperties

func MergeProperties(outputs, inputs map[string]interface{}) map[string]interface{}

MergeProperties merges two property maps into a single map. Keys from the inputs map override keys from the outputs map when they conflict. If both inputs and outputs are nil, MergeProperties returns nil.

Types

type PulumiPlan

type PulumiPlan struct {
	Steps []PulumiStep `json:"steps"`
}

PulumiPlan represents the top-level structure of a Pulumi preview JSON output.

func LoadPulumiPlan

func LoadPulumiPlan(path string) (*PulumiPlan, error)

LoadPulumiPlan loads and parses a Pulumi plan JSON file from the specified path.

func LoadPulumiPlanWithContext

func LoadPulumiPlanWithContext(ctx context.Context, path string) (*PulumiPlan, error)

LoadPulumiPlanWithContext loads and parses the Pulumi plan JSON file at the given path using the logger carried in ctx. It logs progress and any errors encountered to the context's logger. path is the filesystem path to the Pulumi plan JSON file to read and parse. It returns the parsed *PulumiPlan on success. It returns a non-nil error if the file cannot be read or if parsing the JSON fails.

func ParsePulumiPlan

func ParsePulumiPlan(data []byte) (*PulumiPlan, error)

ParsePulumiPlan parses a Pulumi plan from JSON bytes. The data parameter should contain the raw Pulumi plan JSON output. It returns the parsed *PulumiPlan, or an error if the data cannot be unmarshaled into a PulumiPlan.

func ParsePulumiPlanWithContext

func ParsePulumiPlanWithContext(ctx context.Context, data []byte) (*PulumiPlan, error)

ParsePulumiPlanWithContext parses Pulumi plan JSON from the provided byte slice using the given context to obtain a logger. The ctx is used only for logging; data should contain the raw Pulumi plan JSON. ParsePulumiPlanWithContext parses a Pulumi preview JSON document from the provided byte slice and returns the resulting PulumiPlan. ctx provides cancellation and carries logging context used during parsing. data is the raw JSON bytes of a Pulumi preview plan. On success, the parsed *PulumiPlan is returned; if the JSON cannot be unmarshaled, an error wrapping the unmarshal failure is returned.

func (*PulumiPlan) GetResources

func (p *PulumiPlan) GetResources() []PulumiResource

GetResources extracts all resources from the Pulumi plan steps.

func (*PulumiPlan) GetResourcesWithContext

func (p *PulumiPlan) GetResourcesWithContext(ctx context.Context) []PulumiResource

GetResourcesWithContext extracts all resources from the Pulumi plan steps with logging context.

type PulumiResource

type PulumiResource struct {
	Type     string
	URN      string
	Provider string
	Inputs   map[string]interface{}
	Outputs  map[string]interface{}
}

PulumiResource contains the detailed information about a resource in a Pulumi step.

type PulumiState

type PulumiState struct {
	Type     string                 `json:"type"`
	URN      string                 `json:"urn"`
	Inputs   map[string]interface{} `json:"inputs"`
	Outputs  map[string]interface{} `json:"outputs"`
	Provider string                 `json:"provider"`
}

PulumiState represents the state of a resource in a Pulumi step.

type PulumiStep

type PulumiStep struct {
	Op       string                 `json:"op"`
	URN      string                 `json:"urn"`
	Type     string                 `json:"type"`
	Provider string                 `json:"provider"`
	Inputs   map[string]interface{} `json:"inputs"`
	Outputs  map[string]interface{} `json:"outputs"`
	NewState *PulumiState           `json:"newState,omitempty"`
	OldState *PulumiState           `json:"oldState,omitempty"`
}

PulumiStep represents a single resource operation step in a Pulumi plan.

type StackExport

type StackExport struct {
	Version    int                   `json:"version"`
	Deployment StackExportDeployment `json:"deployment"`
}

StackExport represents the structure of `pulumi stack export` output.

func LoadStackExport

func LoadStackExport(path string) (*StackExport, error)

LoadStackExport loads and parses a Pulumi state JSON file from the specified path. The state file is typically generated via `pulumi stack export > state.json`.

func LoadStackExportWithContext

func LoadStackExportWithContext(ctx context.Context, path string) (*StackExport, error)

LoadStackExportWithContext loads and parses a Pulumi state JSON file located at the given path using the provided context.

The path parameter specifies the filesystem location of the Pulumi state JSON to read. It returns the parsed *StackExport on success, or an error if the file cannot be read or the contents cannot be parsed as a StackExport.

func ParseStackExport

func ParseStackExport(data []byte) (*StackExport, error)

ParseStackExport parses Pulumi state JSON from bytes.

func ParseStackExportWithContext

func ParseStackExportWithContext(ctx context.Context, data []byte) (*StackExport, error)

ParseStackExportWithContext parses Pulumi state JSON from data using ctx for logging.

The ctx is used to obtain a logger for diagnostic messages. The data parameter is the raw JSON bytes representing a Pulumi stack export; the function unmarshals those bytes into a StackExport value. On success it returns a pointer to the parsed StackExport. If JSON unmarshalling fails the returned error wraps the underlying unmarshal error.

func (*StackExport) GetCustomResources

func (s *StackExport) GetCustomResources() []StackExportResource

GetCustomResources returns only custom resources (cloud resources) from state. Component resources and providers are filtered out.

func (*StackExport) GetCustomResourcesWithContext

func (s *StackExport) GetCustomResourcesWithContext(ctx context.Context) []StackExportResource

GetCustomResourcesWithContext returns custom resources with logging context.

func (*StackExport) GetResourceByURN

func (s *StackExport) GetResourceByURN(urn string) *StackExportResource

GetResourceByURN finds a resource by its URN.

func (*StackExport) HasTimestamps

func (s *StackExport) HasTimestamps() bool

HasTimestamps checks if the state contains resources with timestamp data. Returns true if at least one resource has Created or Modified timestamps.

type StackExportDeployment

type StackExportDeployment struct {
	Manifest  StackExportManifest   `json:"manifest"`
	Resources []StackExportResource `json:"resources"`
}

StackExportDeployment contains the deployment manifest and resources.

type StackExportManifest

type StackExportManifest struct {
	Time    string `json:"time"`
	Magic   string `json:"magic"`
	Version string `json:"version"`
}

StackExportManifest contains deployment metadata.

type StackExportResource

type StackExportResource struct {
	URN      string                 `json:"urn"`
	Type     string                 `json:"type"`
	ID       string                 `json:"id,omitempty"`
	Custom   bool                   `json:"custom,omitempty"`
	External bool                   `json:"external,omitempty"`
	Provider string                 `json:"provider,omitempty"`
	Inputs   map[string]interface{} `json:"inputs,omitempty"`
	Outputs  map[string]interface{} `json:"outputs,omitempty"`
	// Created tracks when the remote resource was first added to state.
	// Available since Pulumi v3.60.0 (March 2023).
	Created *time.Time `json:"created,omitempty"`
	// Modified tracks when the resource state was last altered.
	// Available since Pulumi v3.60.0 (March 2023).
	Modified *time.Time `json:"modified,omitempty"`
}

StackExportResource represents a resource in Pulumi state (ResourceV3). Timestamps are available since Pulumi v3.60.0 (March 2023).