README
¶
Introduction
This repository implements a whole-system, cross-language profiler for Linux via eBPF.
Core features and strengths
- Implements the experimental OTel profiling signal
- Very low CPU and memory overhead (1% CPU and 250MB memory are our upper limits in testing and the agent typically manages to stay way below that)
- Support for native C/C++ executables without the need for DWARF debug
information (by leveraging
.eh_framedata as described in US11604718B1) - Support profiling of system libraries without frame pointers and without debug symbols on the host.
- Support for mixed stacktraces between runtimes - stacktraces go from Kernel space through unmodified system libraries all the way into high-level languages.
- Support for native code (C/C++, Rust, Zig, Go, etc. without debug symbols on host)
- Support for a broad set of HLLs (Hotspot JVM, Python, Ruby, PHP, Node.JS, V8, Perl), .NET is in preparation.
- 100% non-intrusive: there's no need to load agents or libraries into the processes that are being profiled.
- No need for any reconfiguration, instrumentation or restarts of HLL interpreters and VMs: the agent supports unwinding each of the supported languages in the default configuration.
- ARM64 support for all unwinders except NodeJS.
- Support for native
inline frames, which provide insights into compiler optimizations and offer a higher precision of function call chains.
Building
We are working towards integrating the profiling functionality into the OTel Collector as a receiver,
which will be the supported configuration going forward. In the meantime, we also offer a standalone profiling agent binary named ebpf-profiler,
to aid with development and debugging. The expectation is that this will go away once the integration with the OTel Collector is complete.
Platform Requirements
The agent can be built with the provided make targets. Docker is required for containerized builds, and both amd64 and arm64 architectures are supported.
For Linux, the following steps apply:
- Build the agent for your current machine's architecture:
Ormake agentmake debug-agentfor debug build. - To cross-compile for a different architecture (e.g. arm64):
make agent TARGET_ARCH=arm64
The resulting binary will be named ebpf-profiler in the current directory.
Other OSes
Since the profiler is Linux-only, macOS and Windows users need to set up a Linux VM to build and run the agent. Ensure the appropriate architecture is specified if using cross-compilation. Use the same make targets as above after the Linux environment is configured in the VM.
Supported Linux kernel version
7ddc23ea is the last commit with support for 4.19. Changes after this commit may require a minimal Linux kernel version of 5.4.
Alternative Build (Without Docker)
You can build the agent without Docker by directly installing the dependencies listed in the Dockerfile. Once dependencies are set up, simply run:
make
or
make debug
This will build the profiler natively on your machine.
Running
You can start the agent with the following command:
sudo ./ebpf-profiler -collection-agent=127.0.0.1:11000 -disable-tls
The agent comes with a functional but work-in-progress / evolving implementation of the recently released OTel profiling signal.
The agent loads the eBPF program and its maps, starts unwinding and reports captured traces to the backend.
Development
To understand how this project works and learn more about profiling, check out Profiling internals
Legal
Licensing Information
This project is licensed under the Apache License 2.0 (Apache-2.0). Apache License 2.0
The eBPF source code is licensed under the GPL 2.0 license. GPL 2.0
Licenses of dependencies
To display a summary of the dependencies' licenses:
make legal
Documentation
¶
There is no documentation for this package.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
This package contains a series of helper functions that are useful for ARM disassembly.
|
This package contains a series of helper functions that are useful for ARM disassembly. |
|
asm
|
|
|
Package host implements types and methods specific to interacting with eBPF maps.
|
Package host implements types and methods specific to interacting with eBPF maps. |
|
internal
|
|
|
apmint
Package apmint implements a pseudo interpreter handler that detects APM agent libraries, establishes socket connections with them and notifies them about the stack traces that we collected for their process.
|
Package apmint implements a pseudo interpreter handler that detects APM agent libraries, establishes socket connections with them and notifies them about the stack traces that we collected for their process. |
|
Package kallsyms provides functionality for reading /proc/kallsyms and using it to symbolize kernel addresses.
|
Package kallsyms provides functionality for reading /proc/kallsyms and using it to symbolize kernel addresses. |
|
basehash
Package basehash provides basic types to implement hash identifiers.
|
Package basehash provides basic types to implement hash identifiers. |
|
hash
Package hash provides the same hash primitives as used by the eBPF.
|
Package hash provides the same hash primitives as used by the eBPF. |
|
pfelf
package pfelf implements functions for processing of ELF files and extracting data from them.
|
package pfelf implements functions for processing of ELF files and extracting data from them. |
|
pfelf/internal/mmap
Package mmap is inspired by golang.org/x/exp/mmap with additional functionality.
|
Package mmap is inspired by golang.org/x/exp/mmap with additional functionality. |
|
xsync
Package xsync provides thin wrappers around locking primitives in an effort towards better documenting the relationship between locks and the data they protect.
|
Package xsync provides thin wrappers around locking primitives in an effort towards better documenting the relationship between locks and the data they protect. |
|
lpm package provides helpers for calculating prefix lists from ranges
|
lpm package provides helpers for calculating prefix lists from ranges |
|
Package maccess provides functionality to check if a certain bug in copy_from_user_nofault is patched.
|
Package maccess provides functionality to check if a certain bug in copy_from_user_nofault is patched. |
|
Package metrics contains the code for reporting metrics.
|
Package metrics contains the code for reporting metrics. |
|
genids
command
|
|
|
stackdeltatypes
Package stackdeltatypes provides types used to represent stack delta information as constructed by `nativeunwind.GetIntervalStructures` This information is a post-processed form of the stack delta information that is used in all relevant packages.
|
Package stackdeltatypes provides types used to represent stack delta information as constructed by `nativeunwind.GetIntervalStructures` This information is a post-processed form of the stack delta information that is used in all relevant packages. |
|
nopanicslicereader provides little convenience utilities to read "native" endian values from a slice at given offset.
|
nopanicslicereader provides little convenience utilities to read "native" endian values from a slice at given offset. |
|
Package periodiccaller allows periodic calls of functions.
|
Package periodiccaller allows periodic calls of functions. |
|
Package processmanager manages the loading and unloading of information related to processes.
|
Package processmanager manages the loading and unloading of information related to processes. |
|
remotememory provides access to memory space of a process.
|
remotememory provides access to memory space of a process. |
|
rust-crates
|
|
|
symblib-capi/go
command
|
|
|
successfailurecounter provides a wrapper to atomically increment success or failure counters.
|
successfailurecounter provides a wrapper to atomically increment success or failure counters. |
|
support maps the definitions from headers in the C world into a nice go way
|
support maps the definitions from headers in the C world into a nice go way |
|
tools
|
|
|
coredump
command
|
|
|
coredump/cloudstore
cloudstore provides access to the cloud based storage used in the tests.
|
cloudstore provides access to the cloud based storage used in the tests. |
|
coredump/testsources/go
command
|
|
|
errors-codegen
command
errors-codegen generates the code containing the host agent error code enums.
|
errors-codegen generates the code containing the host agent error code enums. |
|
gooffsets
command
|
|
|
stackdeltas
command
A command-line tool to parse stack deltas from given ELF files.
|
A command-line tool to parse stack deltas from given ELF files. |
|
zstpak
command
|
|
|
Package tracer contains functionality for populating tracers.
|
Package tracer contains functionality for populating tracers. |
|
Package vc provides buildtime information.
|
Package vc provides buildtime information. |