adapt

Documentation

Overview

Package adapt adds functionality related to converting bigquery representations like schema and data type representations.

It is EXPERIMENTAL and subject to change or removal without notice.

Index

• func BQSchemaToStorageTableSchema(in bigquery.Schema) (*storagepb.TableSchema, error)
• func NormalizeDescriptor(in protoreflect.MessageDescriptor) (*descriptorpb.DescriptorProto, error)
• func StorageSchemaToProto2Descriptor(inSchema *storagepb.TableSchema, scope string) (protoreflect.Descriptor, error)
• func StorageSchemaToProto3Descriptor(inSchema *storagepb.TableSchema, scope string) (protoreflect.Descriptor, error)
• func StorageTableSchemaToBQSchema(in *storagepb.TableSchema) (bigquery.Schema, error)

Functions

func BQSchemaToStorageTableSchema

func BQSchemaToStorageTableSchema(in bigquery.Schema) (*storagepb.TableSchema, error)

BQSchemaToStorageTableSchema converts a bigquery Schema into the protobuf-based TableSchema used by the BigQuery Storage WriteClient.

func NormalizeDescriptor

func NormalizeDescriptor(in protoreflect.MessageDescriptor) (*descriptorpb.DescriptorProto, error)

NormalizeDescriptor builds a self-contained DescriptorProto suitable for communicating schema information with the BigQuery Storage write API. It's primarily used for cases where users are interested in sending data using a predefined protocol buffer message.

The storage API accepts a single DescriptorProto for decoding message data. In many cases, a message is comprised of multiple independent messages, from the same .proto file or from multiple sources. Rather than being forced to communicate all these messages independently, what this method does is rewrite the DescriptorProto to inline all messages as nested submessages. As the backend only cares about the types and not the namespaces when decoding, this is sufficient for the needs of the API's representation.

In addition to nesting messages, this method also handles some encapsulation of enum types to avoid possible conflicts due to ambiguities, and clears oneof indices as oneof isn't a concept that maps into BigQuery schemas.

To enable proto3 usage, this function will also rewrite proto3 descriptors into equivalent proto2 form. Such rewrites include setting the appropriate default values for proto3 fields.

func StorageSchemaToProto2Descriptor

func StorageSchemaToProto2Descriptor(inSchema *storagepb.TableSchema, scope string) (protoreflect.Descriptor, error)

StorageSchemaToProto2Descriptor builds a protoreflect.Descriptor for a given table schema using proto2 syntax.

func StorageSchemaToProto3Descriptor

func StorageSchemaToProto3Descriptor(inSchema *storagepb.TableSchema, scope string) (protoreflect.Descriptor, error)

StorageSchemaToProto3Descriptor builds a protoreflect.Descriptor for a given table schema using proto3 syntax.

NOTE: Currently the write API doesn't yet support proto3 behaviors (default value, wrapper types, etc), but this is provided for completeness.

func StorageTableSchemaToBQSchema

func StorageTableSchemaToBQSchema(in *storagepb.TableSchema) (bigquery.Schema, error)

StorageTableSchemaToBQSchema converts a TableSchema from the BigQuery Storage WriteClient into the equivalent BigQuery Schema.