Package tracer contains Datadog's core tracing client. It is used to trace requests as they flow across web servers, databases and microservices, giving developers visibility into bottlenecks and troublesome requests. To start the tracer, simply call the start method along with an optional set of options. By default, the trace agent is considered to be found at "localhost:8126". In a setup where this would be different (let's say, we could do:

    defer tracer.Stop()

    The tracing client can perform trace sampling. While the trace agent already samples traces to reduce bandwidth usage, client sampling reduces performance overhead. To make use of it, the package comes with a ready-to-use rate sampler that can be passed to the tracer. To use it and keep only 30% of the requests, one would do:

    s := tracer.NewRateSampler(0.3)

    More precise control of sampling rates can be configured using sampling rules. This can be applied based on span name, service or both, and is used to determine the sampling rate to apply.

    rules := []tracer.SamplingRule{
          // sample 10% of traces with the span name "web.request"
          tracer.NameRule("web.request", 0.1),
          // sample 20% of traces for the service "test-service"
          tracer.ServiceRule("test-service", 0.2),
          // sample 30% of traces when the span name is "db.query" and the service
          // is "postgres.db"
          tracer.NameServiceRule("db.query", "postgres.db", 0.3),
          // sample 100% of traces when service and name match these regular expressions
          {Service: regexp.MustCompile("^test-"), Name: regexp.MustCompile("http\\..*"), Rate: 1.0},
    defer tracer.Stop()

    Sampling rules can also be configured at runtime using the DD_TRACE_SAMPLING_RULES environment variable. When set, it overrides rules set by tracer.WithSamplingRules. The value is a JSON array of objects. Each object must have a "sample_rate", and the "name" and "service" fields are optional.

    export DD_TRACE_SAMPLING_RULES='[{"name": "web.request", "sample_rate": 1.0}]'

    All spans created by the tracer contain a context hereby referred to as the span context. Note that this is different from Go's context. The span context is used to package essential information from a span, which is needed when creating child spans that inherit from it. Thus, a child span is created from a span's span context. The span context can originate from within the same process, but also a different process or even a different machine in the case of distributed tracing.

    To make use of distributed tracing, a span's context may be injected via a carrier into a transport (HTTP, RPC, etc.) to be extracted on the other end and used to create spans that are direct descendants of it. A couple of carrier interfaces which should cover most of the use-case scenarios are readily provided, such as HTTPCarrier and TextMapCarrier. Users are free to create their own, which will work with our propagation algorithm as long as they implement the TextMapReader and TextMapWriter interfaces. An example alternate implementation is the MDCarrier in our gRPC integration.

    As an example, injecting a span's context into an HTTP request would look like this:

    req, err := http.NewRequest("GET", "", nil)
    // ...
    err := tracer.Inject(span.Context(), tracer.HTTPHeadersCarrier(req.Header))
    // ...

    Then, on the server side, to continue the trace one would do:

    sctx, err := tracer.Extract(tracer.HTTPHeadersCarrier(req.Header))
    // ...
    span := tracer.StartSpan("child.span", tracer.ChildOf(sctx))

    In the same manner, any means can be used as a carrier to inject a context into a transport. Go's context can also be used as a means to transport spans within the same process. The methods StartSpanFromContext, ContextWithSpan and SpanFromContext exist for this reason.

    Some libraries and frameworks are supported out-of-the-box by using one of our integrations. You can see a list of supported integrations here:


      A basic example demonstrating how to start the tracer, as well as how to create a root span and a child span that is a descendant of it.





      View Source
      const (
      	// DefaultBaggageHeaderPrefix specifies the prefix that will be used in
      	// HTTP headers or text maps to prefix baggage keys.
      	DefaultBaggageHeaderPrefix = "ot-baggage-"
      	// DefaultTraceIDHeader specifies the key that will be used in HTTP headers
      	// or text maps to store the trace ID.
      	DefaultTraceIDHeader = "x-datadog-trace-id"
      	// DefaultParentIDHeader specifies the key that will be used in HTTP headers
      	// or text maps to store the parent ID.
      	DefaultParentIDHeader = "x-datadog-parent-id"
      	// DefaultPriorityHeader specifies the key that will be used in HTTP headers
      	// or text maps to store the sampling priority value.
      	DefaultPriorityHeader = "x-datadog-sampling-priority"


      View Source
      var (
      	// ErrInvalidCarrier is returned when the carrier provided to the propagator
      	// does not implemented the correct interfaces.
      	ErrInvalidCarrier = errors.New("invalid carrier")
      	// ErrInvalidSpanContext is returned when the span context found in the
      	// carrier is not of the expected type.
      	ErrInvalidSpanContext = errors.New("invalid span context")
      	// ErrSpanContextCorrupted is returned when there was a problem parsing
      	// the information found in the carrier.
      	ErrSpanContextCorrupted = errors.New("span context corrupted")
      	// ErrSpanContextNotFound represents missing information in the given carrier.
      	ErrSpanContextNotFound = errors.New("span context not found")


      func ContextWithSpan

      func ContextWithSpan(ctx context.Context, s Span) context.Context

        ContextWithSpan returns a copy of the given context which includes the span s.

        func Extract

        func Extract(carrier interface{}) (ddtrace.SpanContext, error)

          Extract extracts a SpanContext from the carrier. The carrier is expected to implement TextMapReader, otherwise an error is returned. If the tracer is not started, calling this function is a no-op.

          func Inject

          func Inject(ctx ddtrace.SpanContext, carrier interface{}) error

            Inject injects the given SpanContext into the carrier. The carrier is expected to implement TextMapWriter, otherwise an error is returned. If the tracer is not started, calling this function is a no-op.

            func Start

            func Start(opts ...StartOption)

              Start starts the tracer with the given set of options. It will stop and replace any running tracer, meaning that calling it several times will result in a restart of the tracer by replacing the current instance with a new one.

              func Stop

              func Stop()

                Stop stops the started tracer. Subsequent calls are valid but become no-op.


                type FinishOption

                type FinishOption = ddtrace.FinishOption

                  FinishOption is a configuration option for FinishSpan. It is aliased in order to help godoc group all the functions returning it together. It is considered more correct to refer to it as the type as the origin, ddtrace.FinishOption.

                  func FinishTime

                  func FinishTime(t time.Time) FinishOption

                    FinishTime sets the given time as the finishing time for the span. By default, the current time is used.

                    func NoDebugStack

                    func NoDebugStack() FinishOption

                      NoDebugStack prevents any error presented using the WithError finishing option from generating a stack trace. This is useful in situations where errors are frequent and performance is critical.

                      func StackFrames

                      func StackFrames(n, skip uint) FinishOption

                        StackFrames limits the number of stack frames included into erroneous spans to n, starting from skip.

                        func WithError

                        func WithError(err error) FinishOption

                          WithError marks the span as having had an error. It uses the information from err to set tags such as the error message, error type and stack trace. It has no effect if the error is nil.

                          type HTTPHeadersCarrier

                          type HTTPHeadersCarrier http.Header

                            HTTPHeadersCarrier wraps an http.Header as a TextMapWriter and TextMapReader, allowing it to be used using the provided Propagator implementation.

                            func (HTTPHeadersCarrier) ForeachKey

                            func (c HTTPHeadersCarrier) ForeachKey(handler func(key, val string) error) error

                              ForeachKey implements TextMapReader.

                              func (HTTPHeadersCarrier) Set

                              func (c HTTPHeadersCarrier) Set(key, val string)

                                Set implements TextMapWriter.

                                type Propagator

                                type Propagator interface {
                                	// Inject takes the SpanContext and injects it into the carrier.
                                	Inject(context ddtrace.SpanContext, carrier interface{}) error
                                	// Extract returns the SpanContext from the given carrier.
                                	Extract(carrier interface{}) (ddtrace.SpanContext, error)

                                  Propagator implementations should be able to inject and extract SpanContexts into an implementation specific carrier.

                                  func NewPropagator

                                  func NewPropagator(cfg *PropagatorConfig) Propagator

                                    NewPropagator returns a new propagator which uses TextMap to inject and extract values. It propagates trace and span IDs and baggage. To use the defaults, nil may be provided in place of the config.

                                    type PropagatorConfig

                                    type PropagatorConfig struct {
                                    	// BaggagePrefix specifies the prefix that will be used to store baggage
                                    	// items in a map. It defaults to DefaultBaggageHeaderPrefix.
                                    	BaggagePrefix string
                                    	// TraceHeader specifies the map key that will be used to store the trace ID.
                                    	// It defaults to DefaultTraceIDHeader.
                                    	TraceHeader string
                                    	// ParentHeader specifies the map key that will be used to store the parent ID.
                                    	// It defaults to DefaultParentIDHeader.
                                    	ParentHeader string
                                    	// PriorityHeader specifies the map key that will be used to store the sampling priority.
                                    	// It deafults to DefaultPriorityHeader.
                                    	PriorityHeader string

                                      PropagatorConfig defines the configuration for initializing a propagator.

                                      type RateSampler

                                      type RateSampler interface {
                                      	// Rate returns the current sample rate.
                                      	Rate() float64
                                      	// SetRate sets a new sample rate.
                                      	SetRate(rate float64)

                                        RateSampler is a sampler implementation which randomly selects spans using a provided rate. For example, a rate of 0.75 will permit 75% of the spans. RateSampler implementations should be safe for concurrent use.

                                        func NewAllSampler

                                        func NewAllSampler() RateSampler

                                          NewAllSampler is a short-hand for NewRateSampler(1). It is all-permissive.

                                          func NewRateSampler

                                          func NewRateSampler(rate float64) RateSampler

                                            NewRateSampler returns an initialized RateSampler with a given sample rate.

                                            type Sampler

                                            type Sampler interface {
                                            	// Sample returns true if the given span should be sampled.
                                            	Sample(span Span) bool

                                              Sampler is the generic interface of any sampler. It must be safe for concurrent use.

                                              type SamplingRule

                                              type SamplingRule struct {
                                              	Service *regexp.Regexp
                                              	Name    *regexp.Regexp
                                              	Rate    float64
                                              	// contains filtered or unexported fields

                                                SamplingRule is used for applying sampling rates to spans that match the service name, operation name or both. For basic usage, consider using the helper functions ServiceRule, NameRule, etc.

                                                func NameRule

                                                func NameRule(name string, rate float64) SamplingRule

                                                  NameRule returns a SamplingRule that applies the provided sampling rate to spans that match the operation name provided.

                                                  func NameServiceRule

                                                  func NameServiceRule(name string, service string, rate float64) SamplingRule

                                                    NameServiceRule returns a SamplingRule that applies the provided sampling rate to spans matching both the operation and service names provided.

                                                    func RateRule

                                                    func RateRule(rate float64) SamplingRule

                                                      RateRule returns a SamplingRule that applies the provided sampling rate to all spans.

                                                      func ServiceRule

                                                      func ServiceRule(service string, rate float64) SamplingRule

                                                        ServiceRule returns a SamplingRule that applies the provided sampling rate to spans that match the service name provided.

                                                        func (*SamplingRule) MarshalJSON

                                                        func (sr *SamplingRule) MarshalJSON() ([]byte, error)

                                                          MarshalJSON implements the json.Marshaler interface.

                                                          type Span

                                                          type Span = ddtrace.Span

                                                            Span is an alias for ddtrace.Span. It is here to allow godoc to group methods returning ddtrace.Span. It is recommended and is considered more correct to refer to this type as ddtrace.Span instead.

                                                            func SpanFromContext

                                                            func SpanFromContext(ctx context.Context) (Span, bool)

                                                              SpanFromContext returns the span contained in the given context. A second return value indicates if a span was found in the context. If no span is found, a no-op span is returned.

                                                              func StartSpan

                                                              func StartSpan(operationName string, opts ...StartSpanOption) Span

                                                                StartSpan starts a new span with the given operation name and set of options. If the tracer is not started, calling this function is a no-op.

                                                                func StartSpanFromContext

                                                                func StartSpanFromContext(ctx context.Context, operationName string, opts ...StartSpanOption) (Span, context.Context)

                                                                  StartSpanFromContext returns a new span with the given operation name and options. If a span is found in the context, it will be used as the parent of the resulting span. If the ChildOf option is passed, the span from context will take precedence over it as the parent span.

                                                                  type StartOption

                                                                  type StartOption func(*config)

                                                                    StartOption represents a function that can be provided as a parameter to Start.

                                                                    func WithAgentAddr

                                                                    func WithAgentAddr(addr string) StartOption

                                                                      WithAgentAddr sets the address where the agent is located. The default is localhost:8126. It should contain both host and port.

                                                                      func WithAnalytics

                                                                      func WithAnalytics(on bool) StartOption

                                                                        WithAnalytics allows specifying whether Trace Search & Analytics should be enabled for integrations.

                                                                        func WithAnalyticsRate

                                                                        func WithAnalyticsRate(rate float64) StartOption

                                                                          WithAnalyticsRate sets the global sampling rate for sampling APM events.

                                                                          func WithDebugMode

                                                                          func WithDebugMode(enabled bool) StartOption

                                                                            WithDebugMode enables debug mode on the tracer, resulting in more verbose logging.

                                                                            func WithDebugStack

                                                                            func WithDebugStack(enabled bool) StartOption

                                                                              WithDebugStack can be used to globally enable or disable the collection of stack traces when spans finish with errors. It is enabled by default. This is a global version of the NoDebugStack FinishOption.

                                                                              func WithDogstatsdAddress

                                                                              func WithDogstatsdAddress(addr string) StartOption

                                                                                WithDogstatsdAddress specifies the address to connect to for sending metrics to the Datadog Agent. If not set, it defaults to "localhost:8125" or to the combination of the environment variables DD_AGENT_HOST and DD_DOGSTATSD_PORT. This option is in effect when WithRuntimeMetrics is enabled.

                                                                                func WithEnv

                                                                                func WithEnv(env string) StartOption

                                                                                  WithEnv sets the environment to which all traces started by the tracer will be submitted. The default value is the environment variable DD_ENV, if it is set.

                                                                                  func WithGlobalTag

                                                                                  func WithGlobalTag(k string, v interface{}) StartOption

                                                                                    WithGlobalTag sets a key/value pair which will be set as a tag on all spans created by tracer. This option may be used multiple times.

                                                                                    func WithHTTPClient

                                                                                    func WithHTTPClient(client *http.Client) StartOption

                                                                                      WithHTTPClient specifies the HTTP client to use when emitting spans to the agent.

                                                                                      func WithHTTPRoundTripper

                                                                                      func WithHTTPRoundTripper(r http.RoundTripper) StartOption

                                                                                        WithHTTPRoundTripper is deprecated. Please consider using WithHTTPClient instead. The function allows customizing the underlying HTTP transport for emitting spans.

                                                                                        func WithLambdaMode

                                                                                        func WithLambdaMode(enabled bool) StartOption

                                                                                          WithLambdaMode enables lambda mode on the tracer, for use with AWS Lambda.

                                                                                          func WithLogger

                                                                                          func WithLogger(logger ddtrace.Logger) StartOption

                                                                                            WithLogger sets logger as the tracer's error printer.

                                                                                            func WithPrioritySampling

                                                                                            func WithPrioritySampling() StartOption

                                                                                              WithPrioritySampling is deprecated, and priority sampling is enabled by default. When using distributed tracing, the priority sampling value is propagated in order to get all the parts of a distributed trace sampled. To learn more about priority sampling, please visit:

                                                                                              func WithPropagator

                                                                                              func WithPropagator(p Propagator) StartOption

                                                                                                WithPropagator sets an alternative propagator to be used by the tracer.

                                                                                                func WithRuntimeMetrics

                                                                                                func WithRuntimeMetrics() StartOption

                                                                                                  WithRuntimeMetrics enables automatic collection of runtime metrics every 10 seconds.

                                                                                                  func WithSampler

                                                                                                  func WithSampler(s Sampler) StartOption

                                                                                                    WithSampler sets the given sampler to be used with the tracer. By default an all-permissive sampler is used.

                                                                                                    func WithSamplingRules

                                                                                                    func WithSamplingRules(rules []SamplingRule) StartOption

                                                                                                      WithSamplingRules specifies the sampling rates to apply to spans based on the provided rules.

                                                                                                      func WithService

                                                                                                      func WithService(name string) StartOption

                                                                                                        WithService sets the default service name for the program.

                                                                                                        func WithServiceName

                                                                                                        func WithServiceName(name string) StartOption

                                                                                                          WithServiceName is deprecated. Please use WithService. If you are using an older version and you are upgrading from WithServiceName to WithService, please note that WithService will determine the service name of server and framework integrations.

                                                                                                          func WithServiceVersion

                                                                                                          func WithServiceVersion(version string) StartOption

                                                                                                            WithServiceVersion specifies the version of the service that is running. This will be included in spans from this service in the "version" tag.

                                                                                                            type StartSpanOption

                                                                                                            type StartSpanOption = ddtrace.StartSpanOption

                                                                                                              StartSpanOption is a configuration option for StartSpan. It is aliased in order to help godoc group all the functions returning it together. It is considered more correct to refer to it as the type as the origin, ddtrace.StartSpanOption.

                                                                                                              func AnalyticsRate

                                                                                                              func AnalyticsRate(rate float64) StartSpanOption

                                                                                                                AnalyticsRate sets a custom analytics rate for a span. It decides the percentage of events that will be picked up by the App Analytics product. It's represents a float64 between 0 and 1 where 0.5 would represent 50% of events.

                                                                                                                func ChildOf

                                                                                                                func ChildOf(ctx ddtrace.SpanContext) StartSpanOption

                                                                                                                  ChildOf tells StartSpan to use the given span context as a parent for the created span.

                                                                                                                  func Measured

                                                                                                                  func Measured() StartSpanOption

                                                                                                                    Measured marks this span to be measured for metrics and stats calculations.

                                                                                                                    func ResourceName

                                                                                                                    func ResourceName(name string) StartSpanOption

                                                                                                                      ResourceName sets the given resource name on the started span. A resource could be an SQL query, a URL, an RPC method or something else.

                                                                                                                      func ServiceName

                                                                                                                      func ServiceName(name string) StartSpanOption

                                                                                                                        ServiceName sets the given service name on the started span. For example "http.server".

                                                                                                                        func SpanType

                                                                                                                        func SpanType(name string) StartSpanOption

                                                                                                                          SpanType sets the given span type on the started span. Some examples in the case of the Datadog APM product could be "web", "db" or "cache".

                                                                                                                          func StartTime

                                                                                                                          func StartTime(t time.Time) StartSpanOption

                                                                                                                            StartTime sets a custom time as the start time for the created span. By default a span is started using the creation time.

                                                                                                                            func Tag

                                                                                                                            func Tag(k string, v interface{}) StartSpanOption

                                                                                                                              Tag sets the given key/value pair as a tag on the started Span.

                                                                                                                              func WithSpanID

                                                                                                                              func WithSpanID(id uint64) StartSpanOption

                                                                                                                                WithSpanID sets the SpanID on the started span, instead of using a random number. If there is no parent Span (eg from ChildOf), then the TraceID will also be set to the value given here.

                                                                                                                                type TextMapCarrier

                                                                                                                                type TextMapCarrier map[string]string

                                                                                                                                  TextMapCarrier allows the use of a regular map[string]string as both TextMapWriter and TextMapReader, making it compatible with the provided Propagator.

                                                                                                                                  func (TextMapCarrier) ForeachKey

                                                                                                                                  func (c TextMapCarrier) ForeachKey(handler func(key, val string) error) error

                                                                                                                                    ForeachKey conforms to the TextMapReader interface.

                                                                                                                                    func (TextMapCarrier) Set

                                                                                                                                    func (c TextMapCarrier) Set(key, val string)

                                                                                                                                      Set implements TextMapWriter.

                                                                                                                                      type TextMapReader

                                                                                                                                      type TextMapReader interface {
                                                                                                                                      	// ForeachKey iterates over all keys that exist in the underlying
                                                                                                                                      	// carrier. It takes a callback function which will be called
                                                                                                                                      	// using all key/value pairs as arguments. ForeachKey will return
                                                                                                                                      	// the first error returned by the handler.
                                                                                                                                      	ForeachKey(handler func(key, val string) error) error

                                                                                                                                        TextMapReader allows iterating over sets of key/value pairs. Carriers implementing TextMapReader are compatible to be used with Datadog's TextMapPropagator.

                                                                                                                                        type TextMapWriter

                                                                                                                                        type TextMapWriter interface {
                                                                                                                                        	// Set sets the given key/value pair.
                                                                                                                                        	Set(key, val string)

                                                                                                                                          TextMapWriter allows setting key/value pairs of strings on the underlying data structure. Carriers implementing TextMapWriter are compatible to be used with Datadog's TextMapPropagator.

                                                                                                                                          GOOS=linux, GOARCH=amd64