pulumi-gcp-ai-endpoint

README

pulumi-gcp-ai-endpoint

Deploy a model and an internet-facing Vertex AI endpoint for real-time predictions with Pulumi and Go.

Features

• Vertex AI Endpoint: internet-accessible endpoint for real-time predictions
• Service Account: dedicated service account with necessary IAM permissions
• Bring your own docker image: set ModelImageURL to serve the model with a custom image in a private google registry.
• Model Upload and Deployment: models automatically uploaded to GCS for Vertex to consume.

Example model deployments

See working end-to-end model deployments:

Example	Model Source	README
Prompt-ready gemma-3-1b-it as an API and with a dockerized custom prediction routine	HuggingFace Model	examples/gemma-prediction-on-pytorch

Upload and deploy a model

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        args := &gcp.AIEndpointArgs{
            Project:  "my-gcp-project",
            Region:   "us-central1",

            ModelDir:      "./my-tuned-model",
            ModelImageURL: pulumi.String("gcr.io/my-project/my-model-server:latest"),
        }

        endpoint, err := gcp.NewAIEndpoint(ctx, "my-predictions-endpoint", args)
        if err != nil {
            return err
        }

        ctx.Export("endpointName", endpoint.GetEndpoint().Name)
        ctx.Export("modelServiceAccountEmail", endpoint.GetModelServiceAccount().Email)

        return nil
    })
}

See:

• https://cloud.google.com/vertex-ai/docs/general/deployment
• https://cloud.google.com/go/docs/reference/cloud.google.com/go/aiplatform/latest/apiv1
• https://github.com/davidmontoyago/pulumi-gcp-vertex-model-deployment
• https://cloud.google.com/vertex-ai/docs/predictions/pre-built-containers
• https://cloud.google.com/vertex-ai/docs/model-registry/import-model

Install

go get github.com/davidmontoyago/pulumi-gcp-ai-endpoint

Full Config

args := &gcp.AIEndpointArgs{
    // Required fields
    Project: "my-gcp-project",
    Region:  "us-central1",

    // Model configuration
    ModelDir:                          "./model",
    ModelPredictionInputSchemaPath:    "input_schema.yaml",
    ModelPredictionOutputSchemaPath:   "output_schema.yaml",
    ModelPredictionBehaviorSchemaPath: "behavior_schema.yaml",
    ModelBucketBasePath:               "model/v1/",
    ModelImageURL:                     pulumi.String("gcr.io/my-project/my-model:latest"),
    ModelDisplayName:                  pulumi.String("production-model"),
    ModelCommandArgs:                  []string{"--allow_precompilation=false", "--disable_optimizer=true"},
    EnvVars: map[string]string{
        "HF_TOKEN":        "your-huggingface-token",
        "MAX_INPUT_TOKENS": "2048",
    },

    // Container configuration
    ContainerPort: pulumi.Int(8080),
    HealthRoute:   pulumi.String("/health"),
    PredictRoute:  pulumi.String("/predict"),

    // Compute configuration
    MachineType:       pulumi.String("g2-standard-4"),
    AcceleratorType:   pulumi.String("NVIDIA_TESLA_T4"),
    AcceleratorCount:  pulumi.Int(1),
    EnableSpotVMs:     pulumi.Bool(true),

    // Endpoint configuration
    EndpointDisplayName: pulumi.String("production-endpoint"),
    MinReplicaCount:     pulumi.Int(1),
    MaxReplicaCount:     pulumi.Int(10),

    // Logging
    EnableAccessLogging: pulumi.Bool(true),
    DisableContainerLogging: pulumi.Bool(true),

    // Security and access
    EnablePrivateRegistryAccess: true,
    DeletionProtection:          pulumi.Bool(true),

    // Resource labels
    Labels: map[string]string{
        "environment": "production",
        "team":        "ai-platform",
    },
}

Architecture

Infrastructure components:

1. Model Service Account with IAM roles:

   • To write model artifacts to GCS
   • Write logs and custom metrics during prediction
   • To access Vertex platform

2. Vertex AI Endpoint ready for:

   • Model upload and deployment. See: https://github.com/davidmontoyago/pulumi-gcp-vertex-model-deployment
   • Configurable resource allocation

3. Model storage & registry integration:

   • Upload model artifacts to GCS
   • Upload models to the model registry and deploy to endpoints
   • Configurable machine specs for model server

Model deployment lifecycle

1. Upload model artifacts to GCS
2. Upload model to model registry
3. Deploy model to endpoint
4. Replace model on changes to the model server image URL
5. Update model artifacts, mode and endpoint on property changes

Development

• Build: make build
• Test: make test
• Lint: make lint
• Clean: make clean

Requirements

• Go 1.24+
• GCP project with Vertex AI API enabled
• Pulumi CLI