bqclient

Documentation

Overview

package bigquery contains functions to

1. run bigquery jobs such as query to table and export table to google storage.

Index

• type BQClient
  • func NewBQClient() (*BQClient, error)
  • func (c *BQClient) JobExtract(table, gspath string) *bigquery.Job
  • func (c *BQClient) JobFromFile(file string) (*bigquery.Job, error)
  • func (c *BQClient) JobQuery(query, dstTable string) *bigquery.Job
  • func (c *BQClient) RunJob(job *bigquery.Job) error
  • func (c *BQClient) RunJobsInFolder(pattern string) error
  • func (c *BQClient) RunSequentialJobs(jobs []*bigquery.Job) error

Types

type BQClient

type BQClient struct {
	ProjectId string
	DatasetId string

	JobStatusPollingInterval time.Duration
	JobStatusPollingMaxTries int
	// contains filtered or unexported fields
}

func NewBQClient

func NewBQClient() (*BQClient, error)

func (*BQClient) JobExtract

func (c *BQClient) JobExtract(table, gspath string) *bigquery.Job

func (*BQClient) JobFromFile

func (c *BQClient) JobFromFile(file string) (*bigquery.Job, error)

parseJobFile generate a bigQuery job by parsing a yaml file.

func (*BQClient) JobQuery

func (c *BQClient) JobQuery(query, dstTable string) *bigquery.Job

func (*BQClient) RunJob

func (c *BQClient) RunJob(job *bigquery.Job) error

RunJob runs a bq Job synchronously.

func (*BQClient) RunJobsInFolder

func (c *BQClient) RunJobsInFolder(pattern string) error

RunJobsInFolder scan jobs file in a folder and run them sequentially. Files are sorted by filename. Subdirectories are not scanned.

func (*BQClient) RunSequentialJobs

func (c *BQClient) RunSequentialJobs(jobs []*bigquery.Job) error

run a series of jobs sequentially and synchronously