integrations

Documentation

Index

• type FileFormat
• type GCSSink
  • func NewGCSSink(ctx context.Context, bucketName, credsFile string) (*GCSSink, error)
  • func (s *GCSSink) Close() error
  • func (s *GCSSink) WriteToGCS(ctx context.Context, reader arrio.Reader, filePath string, format FileFormat, ...) error

Types

type FileFormat

type FileFormat string

FileFormat represents the supported file formats for output.

const (
	ParquetFormat FileFormat = "parquet"
	CSVFormat     FileFormat = "csv"
)

type GCSSink

type GCSSink struct {
	// contains filtered or unexported fields
}

GCSSink represents a Google Cloud Storage sink for writing files.

func NewGCSSink

func NewGCSSink(ctx context.Context, bucketName, credsFile string) (*GCSSink, error)

NewGCSSink creates a new GCSSink with the specified bucket name and credentials file.

func (*GCSSink) Close

func (s *GCSSink) Close() error

Close releases resources associated with the GCSSink.

func (*GCSSink) WriteToGCS

func (s *GCSSink) WriteToGCS(ctx context.Context, reader arrio.Reader, filePath string, format FileFormat, delimiter rune, includeHeader bool, nullValue string, stringsReplacer *strings.Replacer, boolFormatter func(bool) string) error

WriteToGCS writes data from an Arrow reader to a GCS object in the specified format.