README

Knative + Google Cloud Platform = 🚀

GoDoc Go Report Card LICENSE

Knative-GCP builds on Kubernetes to enable easy configuration and consumption of Google Cloud Platform events and services. It can be useful independently, but is best mixed with Knative.

If you are interested in contributing, see DEVELOPMENT.md.

Installing Knative-GCP

Follow this guide to install Knative-GCP components on a platform of your choice.

  1. Installing Knative-GCP
  2. Installing a Service Account for the Data Plane
  3. Installing GCP Broker
  4. Installing Broker with PubSub Channel
  5. Managing Multiple Projects

Operating Knative-GCP

The following guides pertain to operating an existing Knative-GCP installation.

  1. Accessing Event Traces in Cloud Trace

Knative-GCP Sources

In order to consume events from different GCP services, Knative-GCP provides different Sources. A Source is a Kubernetes object that generate or import events into the cluster and sends them downstream in CloudEvents format. Use the examples below to learn how to configure and consume events from different GCP services.

  1. CloudPubSubSource
  2. CloudStorageSource
  3. CloudSchedulerSource
  4. CloudAuditLogsSource
  5. CloudBuildSource

All of the above Sources are Pull-based, i.e., they poll messages from Pub/Sub subscriptions. Different mechanisms can be used to scale them out. Roughly speaking, all such mechanisms need metrics to understand how "congested" the Pub/Sub subscription is and inform their scaling decision subsystem. We currently support the following scaling mechanisms:

  1. Keda-based Scaling

Pub/Sub Channel

A Channel is a Knative Eventing logical construct that provides an event delivery mechanism which can fan-out received events to multiple destinations via Subscriptions. A Channel has a single inbound HTTP-addressable interface, which may accept events delivered directly or forwarded from multiple Subscriptions. Use the example below if you want to use our Channel backed by Cloud Pub/Sub, which offers at-least-once message delivery and best-effort ordering to existing subscribers.

  1. Channel

Pub/Sub Core Resources

In Cloud Pub/Sub, a publisher application creates and sends messages to a topic, while subscriber applications create a subscription to a topic in order to receive messages from it. If you want to interact directly with Cloud Pub/Sub topics and subscriptions within your Kubernetes cluster, use our custom Kubernetes resources below.

  1. Topic
  2. PullSubscription

Note: This repo is still in development, APIs and resource names are subject to change in the future.

Expand ▾ Collapse ▴

Directories

Path Synopsis
cmd
Package core is a placeholder that allows us to pull in config files via go mod vendor.
Package core is a placeholder that allows us to pull in config files via go mod vendor.
core
Package core is a placeholder that allows us to pull in config files via go mod vendor.
Package core is a placeholder that allows us to pull in config files via go mod vendor.
core/configmaps
Package core is a placeholder that allows us to pull in config files via go mod vendor.
Package core is a placeholder that allows us to pull in config files via go mod vendor.
core/deployments
Package core is a placeholder that allows us to pull in config files via go mod vendor.
Package core is a placeholder that allows us to pull in config files via go mod vendor.
core/resources
Package core is a placeholder that allows us to pull in config files via go mod vendor.
Package core is a placeholder that allows us to pull in config files via go mod vendor.
core/roles
Package core is a placeholder that allows us to pull in config files via go mod vendor.
Package core is a placeholder that allows us to pull in config files via go mod vendor.
core/services
Package core is a placeholder that allows us to pull in config files via go mod vendor.
Package core is a placeholder that allows us to pull in config files via go mod vendor.
core/webhooks
Package core is a placeholder that allows us to pull in config files via go mod vendor.
Package core is a placeholder that allows us to pull in config files via go mod vendor.
istio
Package core is a placeholder that allows us to pull in config files via go mod vendor.
Package core is a placeholder that allows us to pull in config files via go mod vendor.
monitoring/metrics/grafana
Package core is a placeholder that allows us to pull in config files via go mod vendor.
Package core is a placeholder that allows us to pull in config files via go mod vendor.
monitoring/metrics/prometheus
Package core is a placeholder that allows us to pull in config files via go mod vendor.
Package core is a placeholder that allows us to pull in config files via go mod vendor.
pre-install/v0.18.0
Package preinstall is a placeholder that allows us to pull in config files via go mod vendor.
Package preinstall is a placeholder that allows us to pull in config files via go mod vendor.
hack
pkg
apis
+k8s:deepcopy-gen=package
+k8s:deepcopy-gen=package
apis/broker
Package broker contains API versions for the Cloud Run Events broker.
Package broker contains API versions for the Cloud Run Events broker.
apis/broker/v1beta1
Package v1beta1 defines the custom Broker and Trigger types in eventing.knative.dev/v1beta1 used for the Google Cloud Broker.
Package v1beta1 defines the custom Broker and Trigger types in eventing.knative.dev/v1beta1 used for the Google Cloud Broker.
apis/configs/broker
broker holds the typed objects that define the schemas for default Broker delivery settings.
broker holds the typed objects that define the schemas for default Broker delivery settings.
apis/configs/dataresidency
dataresidency holds the typed objects that define the schemas for default DataResidency for all components.
dataresidency holds the typed objects that define the schemas for default DataResidency for all components.
apis/configs/gcpauth
gcpauth holds the typed objects that define the schemas for default GCP auth of sources and channels.
gcpauth holds the typed objects that define the schemas for default GCP auth of sources and channels.
apis/duck
Package duck contains Cloud Run Events API versions for duck components Package duck contains Cloud Run Events API versions for duck components Package duck contains Cloud Run Events API versions for duck components
Package duck contains Cloud Run Events API versions for duck components Package duck contains Cloud Run Events API versions for duck components Package duck contains Cloud Run Events API versions for duck components
apis/duck/v1
+k8s:deepcopy-gen=package +groupName=duck.cloud.google.com
+k8s:deepcopy-gen=package +groupName=duck.cloud.google.com
apis/duck/v1alpha1
+k8s:deepcopy-gen=package +groupName=duck.cloud.google.com
+k8s:deepcopy-gen=package +groupName=duck.cloud.google.com
apis/duck/v1beta1
+k8s:deepcopy-gen=package +groupName=duck.cloud.google.com
+k8s:deepcopy-gen=package +groupName=duck.cloud.google.com
apis/events
Package events contains Cloud Run Events API versions for eventing components
Package events contains Cloud Run Events API versions for eventing components
apis/events/v1
Package v1 contains API Schema definitions for the events v1 API group +k8s:openapi-gen=true +k8s:deepcopy-gen=package,register +k8s:conversion-gen=github.com/google/knative-gcp/pkg/apis/events +k8s:defaulter-gen=TypeMeta +groupName=events.cloud.google.com
Package v1 contains API Schema definitions for the events v1 API group +k8s:openapi-gen=true +k8s:deepcopy-gen=package,register +k8s:conversion-gen=github.com/google/knative-gcp/pkg/apis/events +k8s:defaulter-gen=TypeMeta +groupName=events.cloud.google.com
apis/events/v1alpha1
Package v1alpha1 contains API Schema definitions for the events v1alpha1 API group +k8s:openapi-gen=true +k8s:deepcopy-gen=package,register +k8s:conversion-gen=github.com/google/knative-gcp/pkg/apis/events +k8s:defaulter-gen=TypeMeta +groupName=events.cloud.google.com
Package v1alpha1 contains API Schema definitions for the events v1alpha1 API group +k8s:openapi-gen=true +k8s:deepcopy-gen=package,register +k8s:conversion-gen=github.com/google/knative-gcp/pkg/apis/events +k8s:defaulter-gen=TypeMeta +groupName=events.cloud.google.com
apis/events/v1beta1
Package v1beta1 contains API Schema definitions for the events v1beta1 API group +k8s:openapi-gen=true +k8s:deepcopy-gen=package,register +k8s:conversion-gen=github.com/google/knative-gcp/pkg/apis/events +k8s:defaulter-gen=TypeMeta +groupName=events.cloud.google.com
Package v1beta1 contains API Schema definitions for the events v1beta1 API group +k8s:openapi-gen=true +k8s:deepcopy-gen=package,register +k8s:conversion-gen=github.com/google/knative-gcp/pkg/apis/events +k8s:defaulter-gen=TypeMeta +groupName=events.cloud.google.com
apis/intevents
Package intevents contains API versions for internal use by other resources.
Package intevents contains API versions for internal use by other resources.
apis/intevents/v1
Package v1 defines internal types in internal.events.cloud.google.com/v1 for use by other resources.
Package v1 defines internal types in internal.events.cloud.google.com/v1 for use by other resources.
apis/intevents/v1alpha1
Package v1alpha1 contains API Schema definitions for the sources v1alpha1 API group +k8s:deepcopy-gen=package +groupName=sources.knative.dev
Package v1alpha1 contains API Schema definitions for the sources v1alpha1 API group +k8s:deepcopy-gen=package +groupName=sources.knative.dev
apis/intevents/v1beta1
Package v1beta1 defines internal types in internal.events.cloud.google.com/v1beta1 for use by other resources.
Package v1beta1 defines internal types in internal.events.cloud.google.com/v1beta1 for use by other resources.
apis/messaging
Package messaging contains Cloud Run Events API versions for messaging components
Package messaging contains Cloud Run Events API versions for messaging components
apis/messaging/v1alpha1
Package v1alpha1 contains API Schema definitions for the eventing v1alpha1 API group +k8s:openapi-gen=true +k8s:deepcopy-gen=package,register +k8s:conversion-gen=github.com/google/knative-gcp/pkg/apis/messaging +k8s:defaulter-gen=TypeMeta +groupName=messaging.cloud.google.com
Package v1alpha1 contains API Schema definitions for the eventing v1alpha1 API group +k8s:openapi-gen=true +k8s:deepcopy-gen=package,register +k8s:conversion-gen=github.com/google/knative-gcp/pkg/apis/messaging +k8s:defaulter-gen=TypeMeta +groupName=messaging.cloud.google.com
apis/messaging/v1beta1
Package v1beta1 contains API Schema definitions for the eventing v1beta1 API group +k8s:openapi-gen=true +k8s:deepcopy-gen=package,register +k8s:conversion-gen=github.com/google/knative-gcp/pkg/apis/messaging +k8s:defaulter-gen=TypeMeta +groupName=messaging.cloud.google.com
Package v1beta1 contains API Schema definitions for the eventing v1beta1 API group +k8s:openapi-gen=true +k8s:deepcopy-gen=package,register +k8s:conversion-gen=github.com/google/knative-gcp/pkg/apis/messaging +k8s:defaulter-gen=TypeMeta +groupName=messaging.cloud.google.com
client/clientset/versioned
This package has the automatically generated clientset.
This package has the automatically generated clientset.
client/clientset/versioned/fake
This package has the automatically generated fake clientset.
This package has the automatically generated fake clientset.
client/clientset/versioned/scheme
This package contains the scheme of the automatically generated clientset.
This package contains the scheme of the automatically generated clientset.
client/clientset/versioned/typed/broker/v1beta1
This package has the automatically generated typed clients.
This package has the automatically generated typed clients.
client/clientset/versioned/typed/broker/v1beta1/fake
Package fake has the automatically generated clients.
Package fake has the automatically generated clients.
client/clientset/versioned/typed/events/v1
This package has the automatically generated typed clients.
This package has the automatically generated typed clients.
client/clientset/versioned/typed/events/v1/fake
Package fake has the automatically generated clients.
Package fake has the automatically generated clients.
client/clientset/versioned/typed/events/v1alpha1
This package has the automatically generated typed clients.
This package has the automatically generated typed clients.
client/clientset/versioned/typed/events/v1alpha1/fake
Package fake has the automatically generated clients.
Package fake has the automatically generated clients.
client/clientset/versioned/typed/events/v1beta1
This package has the automatically generated typed clients.
This package has the automatically generated typed clients.
client/clientset/versioned/typed/events/v1beta1/fake
Package fake has the automatically generated clients.
Package fake has the automatically generated clients.
client/clientset/versioned/typed/intevents/v1
This package has the automatically generated typed clients.
This package has the automatically generated typed clients.
client/clientset/versioned/typed/intevents/v1/fake
Package fake has the automatically generated clients.
Package fake has the automatically generated clients.
client/clientset/versioned/typed/intevents/v1alpha1
This package has the automatically generated typed clients.
This package has the automatically generated typed clients.
client/clientset/versioned/typed/intevents/v1alpha1/fake
Package fake has the automatically generated clients.
Package fake has the automatically generated clients.
client/clientset/versioned/typed/intevents/v1beta1
This package has the automatically generated typed clients.
This package has the automatically generated typed clients.
client/clientset/versioned/typed/intevents/v1beta1/fake
Package fake has the automatically generated clients.
Package fake has the automatically generated clients.
client/clientset/versioned/typed/messaging/v1alpha1
This package has the automatically generated typed clients.
This package has the automatically generated typed clients.
client/clientset/versioned/typed/messaging/v1alpha1/fake
Package fake has the automatically generated clients.
Package fake has the automatically generated clients.
client/clientset/versioned/typed/messaging/v1beta1
This package has the automatically generated typed clients.
This package has the automatically generated typed clients.
client/clientset/versioned/typed/messaging/v1beta1/fake
Package fake has the automatically generated clients.
Package fake has the automatically generated clients.
gclient/iam
Package iam provides interfaces and wrappers around the google iam client.
Package iam provides interfaces and wrappers around the google iam client.
gclient/logging/logadmin
Package logadmin contains mockable logadmin client wrappers.
Package logadmin contains mockable logadmin client wrappers.
gclient/logging/logadmin/testing
Package testing provides a fake logadmin client for test purposes.
Package testing provides a fake logadmin client for test purposes.
gclient/metadata
Package iam provides interfaces and wrappers around the google iam client.
Package iam provides interfaces and wrappers around the google iam client.
gclient/pubsub
Package pubsub contains Pub/Sub client wrappers to be able to UT things.
Package pubsub contains Pub/Sub client wrappers to be able to UT things.
gclient/scheduler
Package scheduler contains Scheduler client wrappers to be able to UT things.
Package scheduler contains Scheduler client wrappers to be able to UT things.
gclient/storage
Package storage contains Storage client wrappers to be able to UT things.
Package storage contains Storage client wrappers to be able to UT things.
logging
Package logging is a copy of knative/pkg's logging package, except it uses desugared loggers.
Package logging is a copy of knative/pkg's logging package, except it uses desugared loggers.
observability
Package observability contains common setup and utilities for metrics, logging, and tracing.
Package observability contains common setup and utilities for metrics, logging, and tracing.
pubsub/adapter/converters
Package converters contains pubsub message to cloudevent converters used by pubsub-based source.
Package converters contains pubsub message to cloudevent converters used by pubsub-based source.
reconciler
Package reconciler defines implementations of the Reconciler interface defined at knative.dev/pkg/controller.Reconciler.
Package reconciler defines implementations of the Reconciler interface defined at knative.dev/pkg/controller.Reconciler.
reconciler/broker
Package broker implements the Broker controller and reconciler reconciling Brokers and Triggers.
Package broker implements the Broker controller and reconciler reconciling Brokers and Triggers.
reconciler/brokercell/testingdata
Package testingdata creates data plane resources used by tests by reading YAML files.
Package testingdata creates data plane resources used by tests by reading YAML files.
reconciler/events/auditlogs
Package auditlogs implements the CloudAuditLogsSource controller.
Package auditlogs implements the CloudAuditLogsSource controller.
reconciler/events/auditlogs/resources
Package resources contains helpers for audit log source resources.
Package resources contains helpers for audit log source resources.
reconciler/events/build
Package channel implements the CloudBuildSource controller.
Package channel implements the CloudBuildSource controller.
reconciler/events/pubsub
Package channel implements the CloudPubSubSource controller.
Package channel implements the CloudPubSubSource controller.
reconciler/events/scheduler
Package channel implements the Scheduler Source controller.
Package channel implements the Scheduler Source controller.
reconciler/events/storage
Package channel implements the CloudStorageSource controller.
Package channel implements the CloudStorageSource controller.
reconciler/identity
Package identity contains the identity reconciler
Package identity contains the identity reconciler
reconciler/intevents/pullsubscription
Package pullsubscription implements the Pub/Sub PullSubscription controllers.
Package pullsubscription implements the Pub/Sub PullSubscription controllers.
reconciler/intevents/pullsubscription/keda
Package keda implements the Pub/Sub PullSubscription controller for Keda-based PullSubscriptions.
Package keda implements the Pub/Sub PullSubscription controller for Keda-based PullSubscriptions.
reconciler/intevents/pullsubscription/static
Package static implements the Pub/Sub PullSubscription controller for non-scalable PullSubscriptions.
Package static implements the Pub/Sub PullSubscription controller for non-scalable PullSubscriptions.
reconciler/intevents/topic
Package topic implements the Pub/Sub Topic controller.
Package topic implements the Pub/Sub Topic controller.
reconciler/messaging/channel
Package channel implements the Pub/Sub Channel controller.
Package channel implements the Pub/Sub Channel controller.
schemas/v1
Package v1 defines constants and util functions for event v1 schemas.
Package v1 defines constants and util functions for event v1 schemas.
tracing
TODO Move this to knative/pkg's tracing package.
TODO Move this to knative/pkg's tracing package.
utils/appcredentials
Package appcredentials provides utilities for the application credentials used to access GCP services.
Package appcredentials provides utilities for the application credentials used to access GCP services.
utils/mainhelper
Package mainhelper provides helper functions for common boilerplate code in writing a main function such as setting up kube informers.
Package mainhelper provides helper functions for common boilerplate code in writing a main function such as setting up kube informers.
e2e
e2e/lib
Utility functions used for testing
Utility functions used for testing