lds3

package module
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 2, 2026 License: Apache-2.0 Imports: 25 Imported by: 0

README

LaunchDarkly Server-side SDK for Go - S3 data initializer

Actions Status Documentation

[!CAUTION] This library is in pre-release and not subject to backward compatibility guarantees. The API may change based on feedback.

Pin to a specific minor version and review the changelog before upgrading.

This library provides an Amazon S3 data initializer for the LaunchDarkly Go SDK's data system. It reads one object from an S3 bucket at SDK startup and uses its contents as the SDK's initial flag and segment data.

Initializers are one-shot: the SDK runs them once during startup to get flag values available as quickly as possible. Ongoing updates, if any, come from separately configured synchronizers. A typical use for this initializer is as a fallback tier so that an application can always start with a known-good flag snapshot, even when LaunchDarkly is unreachable.

Requirements

This library requires version 7.17.0 or higher of the LaunchDarkly Go SDK, and Go 1.25 or higher.

Usage

Import this package (with an alias for readability):

import (
    lds3 "github.com/launchdarkly/go-server-sdk-s3"
)

Configure it as an initializer in the SDK's data system. For example, to try LaunchDarkly first and fall back to an S3 snapshot, with streaming for ongoing updates:

config := ld.Config{
    DataSystem: ldcomponents.DataSystem().Custom().
        Initializers(
            ldcomponents.PollingDataSourceV2().AsInitializer(),
            lds3.DataInitializerForObject("my-bucket", "flags/snapshot.json"),
        ).
        Synchronizers(ldcomponents.StreamingDataSourceV2()),
}
client, err := ld.MakeCustomClient("sdk-key", config, 5*time.Second)

By default, the S3 client is created from the default AWS configuration sources (environment variables, shared config files, instance metadata). To customize it:

// With a specific AWS configuration:
lds3.DataInitializerForObject("my-bucket", "snapshot.json").AWSConfig(myAWSConfig)

// Or with a preconfigured client, e.g. for an S3-compatible service:
lds3.DataInitializerForObject("my-bucket", "snapshot.json").S3Client(myS3Client)

S3Client is the sanctioned way to use custom endpoints, path-style addressing, or local S3 stand-ins such as LocalStack or MinIO; the builder deliberately does not mirror the AWS SDK's endpoint options.

Two safety limits apply and can be adjusted:

// The whole fetch must finish within this time; default 30s. The limit keeps a
// slow S3 request from consuming the SDK's entire initialization wait budget. Zero disables it.
lds3.DataInitializerForObject("my-bucket", "snapshot.json").Timeout(10 * time.Second)

// The object's content (after decompression, if compressed) may be at most this many bytes;
// default 100 MiB.
lds3.DataInitializerForObject("my-bucket", "snapshot.json").MaxObjectSizeBytes(500 * 1024 * 1024)

The object may be gzip-compressed if its Content-Encoding metadata is set to gzip. An object that contains gzip data without that metadata is rejected with an explanatory error.

Data format

The object must contain flag data in the same JSON or YAML document format used by the SDK's file data sources, with any combination of flags and segments properties:

{
  "flags": {
    "my-boolean-flag-key": {
      "key": "my-boolean-flag-key",
      "version": 1,
      "on": true,
      "fallthrough": { "variation": 0 },
      "variations": [true, false]
    }
  }
}

See "Reading flags from a file" for a full description of the format. The file data sources' simplified flagValues entries are not supported: this initializer consumes snapshots of real flag data, which express full definitions, and an object that contains a flagValues section is rejected with an explanatory error.

Because this initializer's purpose is to provide a known-good fallback, it validates the data rather than applying whatever it finds:

  • A snapshot that contains no flags and no segments is rejected. A corrupted, truncated, or misproduced object therefore causes a loud fallthrough to the next data source instead of silently turning off every flag. This also means a misspelled top-level property (for example "flagz") is caught.
  • Documents must be valid UTF-8.
  • A flag or segment whose embedded key property does not match the key it is defined under is rejected, as is a negative version.

The initializer reads a single object per snapshot. Put the whole snapshot in one object, so a publisher cannot produce a mixed snapshot with concurrent updates.

Initialization semantics

Data loaded from S3 is a snapshot: it has no FDv2 payload state, so the SDK treats it as "cached" data rather than "refreshed" (current) data. Applied snapshot data completes initialization: MakeCustomClient succeeds, client.Initialized() is true, and flag evaluations serve the snapshot values, while any configured synchronizers keep trying for current data in the background.

Because of these semantics, place this initializer last in the initializer chain, after any initializers that can deliver current data from LaunchDarkly.

Out of scope

This library initializes once at startup. It does not watch S3 for changes or act as a synchronizer, it does not ask the SDK to persist the snapshot into a persistent store (Persist is always false), and it does not support the older FDv1 data system.

Contributing

We encourage pull requests and other contributions from the community. Check out our contributing guidelines for instructions on how to contribute to this library.

About LaunchDarkly

  • LaunchDarkly is a continuous delivery platform that provides feature flags as a service and allows developers to iterate quickly and safely. We allow you to easily flag your features and manage them from the LaunchDarkly dashboard. With LaunchDarkly, you can:
    • Roll out a new feature to a subset of your users (like a group of users who opt-in to a beta tester group), gathering feedback and bug reports from real-world use cases.
    • Gradually roll out a feature to an increasing percentage of users, and track the effect that the feature has on key metrics (for instance, how likely is a user to complete a purchase if they have feature A versus feature B?).
    • Turn off a feature that you realize is causing performance problems in production, without needing to re-deploy, or even restart the application with a changed configuration file.
    • Grant access to certain features based on user attributes, like payment plan (eg: users on the 'gold' plan get access to more features than users in the 'silver' plan). Disable parts of your application to facilitate maintenance, without taking everything offline.
  • LaunchDarkly provides feature flag SDKs for a wide variety of languages and technologies. Check out our documentation for a complete list.
  • Explore LaunchDarkly

Documentation

Overview

Package lds3 provides an Amazon S3 data initializer for the LaunchDarkly Go SDK's data system.

The initializer reads one or more objects from an S3 bucket at startup and uses their contents as the SDK's initial set of flag and segment data. Initializers are one-shot: the SDK calls the initializer once during startup, and any ongoing updates must come from a separately configured synchronizer (or from nothing at all, if the S3 data is meant to be the only data source).

A typical use is a two-tier startup strategy: try LaunchDarkly first, and fall back to a known-good snapshot stored in S3 so that the application can always start, even during a network or service outage: 

import (
    ld "github.com/launchdarkly/go-server-sdk/v7"
    "github.com/launchdarkly/go-server-sdk/v7/ldcomponents"
    lds3 "github.com/launchdarkly/go-server-sdk-s3"
)

config := ld.Config{
    DataSystem: ldcomponents.DataSystem().Custom().
        Initializers(
            ldcomponents.PollingDataSourceV2().AsInitializer(),
            lds3.DataInitializerForObject("my-bucket", "flags/snapshot.json"),
        ).
        Synchronizers(ldcomponents.StreamingDataSourceV2()),
}
client, err := ld.MakeCustomClient("sdk-key", config, 5*time.Second)

The objects must be in the same JSON or YAML document format that the SDK's file data sources use, containing any combination of "flags" and "segments" properties. The file data sources' simplified "flagValues" entries are not supported; an object that contains a "flagValues" section is rejected. See the SDK's ldfiledata package documentation for a description of the format.

By default, the initializer creates an S3 client from the default AWS configuration sources (environment variables, shared config files, and instance metadata). Use the builder's methods to supply a custom aws.Config or a preconfigured client.

The initializer validates what it fetches: documents must be valid UTF-8, may be gzip-compressed when the object's Content-Encoding says so, are limited to a configurable maximum size, and a snapshot containing no flag or segment data is rejected rather than applied. The whole fetch is bounded by a configurable timeout so that a slow S3 request cannot consume the SDK's entire initialization wait budget.

Index

Constants

View Source 
const (
	// DefaultMaxObjectSizeBytes is the default value for DataInitializerBuilder.MaxObjectSizeBytes:
	// 100 MiB.
	DefaultMaxObjectSizeBytes int64 = 100 * 1024 * 1024

	// DefaultMaxObjectSize is the default value for DataInitializerBuilder.MaxObjectSize: 100 MiB.
	//
	// Deprecated: Use [DefaultMaxObjectSizeBytes] instead.
	DefaultMaxObjectSize = DefaultMaxObjectSizeBytes

	// DefaultTimeout is the default value for DataInitializerBuilder.Timeout.
	DefaultTimeout = 30 * time.Second
)

Variables

This section is empty.

Functions

This section is empty.

Types

type DataInitializerBuilder 

type DataInitializerBuilder struct {
	// contains filtered or unexported fields
}

DataInitializerBuilder is a builder for configuring the S3 data initializer.

Obtain an instance of this type by calling DataInitializer. After calling its methods to specify any desired custom settings, pass it to the Initializers method of github.com/launchdarkly/go-server-sdk/v7/ldcomponents.DataSystemConfigurationBuilder.

Builder calls can be chained, for example: 

lds3.DataInitializer("my-bucket", "snapshot.json").AWSConfig(myConfig)

You do not need to call the builder's Build method yourself; that will be done by the SDK.

func DataInitializer deprecated 

func DataInitializer(bucket string, objectKeys ...string) *DataInitializerBuilder

DataInitializer returns a configurable builder for an S3 data initializer.

Deprecated: Use DataInitializerForObject instead. The initializer reads one object; when more than one object key is given, only the first is used.

func DataInitializerForObject added in v0.2.0 

func DataInitializerForObject(bucket string, objectKey string) *DataInitializerBuilder

DataInitializerForObject returns a configurable builder for an S3 data initializer.

The bucket is the name of the S3 bucket to read from, and objectKey is the key of an object within it, containing flag data in the SDK's file data source format.

func (*DataInitializerBuilder) AWSConfig 

func (b *DataInitializerBuilder) AWSConfig(config aws.Config) *DataInitializerBuilder

AWSConfig specifies an AWS configuration to create the S3 client from, instead of the default configuration sources (environment variables, shared config files, and instance metadata).

This setting is ignored if a client is provided with DataInitializerBuilder.S3Client.

func (*DataInitializerBuilder) Build 

func (b *DataInitializerBuilder) Build(
	clientContext subsystems.ClientContext,
) (subsystems.DataInitializer, error)

Build creates a DataInitializer instance that can be used by the SDK client.

func (*DataInitializerBuilder) DuplicateKeysHandling deprecated 

func (b *DataInitializerBuilder) DuplicateKeysHandling(
	DuplicateKeysHandling,
) *DataInitializerBuilder

DuplicateKeysHandling specifies how to handle flag or segment keys that are duplicated across objects.

Deprecated: The initializer reads one object, so this option has no effect.

func (*DataInitializerBuilder) MaxObjectSize deprecated 

func (b *DataInitializerBuilder) MaxObjectSize(limit int64) *DataInitializerBuilder

MaxObjectSize specifies the maximum size in bytes of the object's flag data.

Deprecated: Use DataInitializerBuilder.MaxObjectSizeBytes instead. The two methods have the same behavior.

func (*DataInitializerBuilder) MaxObjectSizeBytes added in v0.2.0 

func (b *DataInitializerBuilder) MaxObjectSizeBytes(limit int64) *DataInitializerBuilder

MaxObjectSizeBytes specifies the maximum size in bytes of the object's flag data. The limit applies to the decompressed content if the object is compressed. Reading stops with an error if the object exceeds it.

The limit exists so that a misconfigured object key cannot make the SDK buffer an arbitrarily large download during startup. If this is not specified, the default is DefaultMaxObjectSizeBytes. The value must be greater than zero.

func (*DataInitializerBuilder) ObjectKeys deprecated 

func (b *DataInitializerBuilder) ObjectKeys(objectKeys ...string) *DataInitializerBuilder

ObjectKeys adds keys of objects to read from the bucket, after any keys that were previously specified.

Deprecated: The initializer reads one object; keys after the first are ignored.

func (*DataInitializerBuilder) S3Client 

func (b *DataInitializerBuilder) S3Client(client S3Client) *DataInitializerBuilder

S3Client specifies an existing S3 client instance for the initializer to use, instead of creating one. Use this if you need full control of the client's configuration, for example to point it at an S3-compatible service.

func (*DataInitializerBuilder) Timeout 

func (b *DataInitializerBuilder) Timeout(timeout time.Duration) *DataInitializerBuilder

Timeout specifies a time limit for the entire data-fetching operation, including reading every configured object. If the limit is reached, the initializer fails and the SDK moves on to the next initializer or synchronizer, instead of letting a slow S3 request consume the SDK's whole initialization wait budget.

If this is not specified, the default is DefaultTimeout. A value of zero disables the limit, leaving the operation bounded only by the SDK's initialization context. The value must not be negative.

type DuplicateKeysHandling deprecated 

type DuplicateKeysHandling string

DuplicateKeysHandling is a parameter type used with DataInitializerBuilder.DuplicateKeysHandling.

Deprecated: The initializer reads one object, so keys cannot be duplicated across objects. 

const (
	// DuplicateKeysFail is an option for DataInitializerBuilder.DuplicateKeysHandling.
	//
	// Deprecated: The initializer reads one object, so this option has no effect.
	DuplicateKeysFail DuplicateKeysHandling = "fail"

	// DuplicateKeysKeepFirst is an option for DataInitializerBuilder.DuplicateKeysHandling.
	//
	// Deprecated: The initializer reads one object, so this option has no effect.
	DuplicateKeysKeepFirst DuplicateKeysHandling = "ignore"
)

type S3Client 

type S3Client interface {
	GetObject(ctx context.Context, params *s3.GetObjectInput, optFns ...func(*s3.Options)) (*s3.GetObjectOutput, error)
}

S3Client is the subset of the AWS S3 client's operations that the data initializer uses. It is implemented by *s3.Client, and can be implemented by test fakes.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.