bqutil

Documentation

Overview

Package bqutil contains utility functions for BigQuery.

Index

• Constants
• func Client(ctx context.Context, gcpProject string) (*bigquery.Client, error)
• func GenerateSchema(fdset *desc.FileDescriptorSet, message string) (schema bigquery.Schema, err error)
• func NewWriterClient(ctx context.Context, gcpProject string) (*managedwriter.Client, error)
• func ProjectForDataset(dataset string) (string, error)
• func TestFailureAnalysisToBqRow(ctx context.Context, tfa *model.TestFailureAnalysis) (*bqpb.TestAnalysisRow, error)
• type Writer
  • func NewWriter(client *managedwriter.Client, tableName string, ...) *Writer
  • func (s *Writer) AppendRowsWithPendingStream(ctx context.Context, rows []proto.Message) error

Constants

const InternalDatasetID = "internal"

InternalDatasetID is the name of the BigQuery dataset which is intended for internal service use only.

Functions

func Client

func Client(ctx context.Context, gcpProject string) (*bigquery.Client, error)

Client returns a new BigQuery client for use with the given GCP project, that authenticates as LUCI Bisection.

func GenerateSchema

func GenerateSchema(fdset *desc.FileDescriptorSet, message string) (schema bigquery.Schema, err error)

GenerateSchema generates BigQuery schema for the given proto message using the given set of message definitions.

func NewWriterClient

func NewWriterClient(ctx context.Context, gcpProject string) (*managedwriter.Client, error)

NewWriterClient returns a new BigQuery managedwriter client for use with the given GCP project, that authenticates as LUCI Bisection itself.

func ProjectForDataset

func ProjectForDataset(dataset string) (string, error)

ProjectForDataset returns the name of the LUCI Project that corresponds to the given BigQuery dataset.

func TestFailureAnalysisToBqRow

func TestFailureAnalysisToBqRow(ctx context.Context, tfa *model.TestFailureAnalysis) (*bqpb.TestAnalysisRow, error)

TestFailureAnalysisToBqRow returns a TestAnalysisRow for a TestFailureAnalysis. This is very similar to TestFailureAnalysisToPb in protoutil, but we intentionally keep them separate because they are for 2 different purposes. It gives us the flexibility to change one without affecting the others.

Types

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer is used to export rows to BigQuery table.

func NewWriter

func NewWriter(
	client *managedwriter.Client,
	tableName string,
	tableSchemaDescriptor *descriptorpb.DescriptorProto,
) *Writer

NewWriter creates a writer for exporting rows to the provided BigQuery table via the provided managedWriter client.

func (*Writer) AppendRowsWithPendingStream

func (s *Writer) AppendRowsWithPendingStream(ctx context.Context, rows []proto.Message) error

AppendRowsWithPendingStream append rows to BigQuery table via the pending stream. This provides all-or-nothing semantics for insertion.