README
¶
wandb-core: A new backend for the W&B SDK
Introduction
wandb-core is a new and improved backend for the W&B SDK that is more performant, versatile, and robust. wandb-core is enabled by default with wandb>=0.18.0.
wandb-core enables:
- 🚀 Logging performance: Experience up to 88% performance improvements when logging with multiple processes.
- 🤏 Reduced resource consumption: Lower memory footprint allows you to run more experiments on your machines.
- 📊 Improved Table logging: Experience up to 40% faster table logging performance!
- ⚡ Faster startups & shutdowns: Enjoy up to 36% faster startup and shutdown times.
- ⏩ Enhanced artifact handling: Experience up to 33% faster Artifact uploads speed and up to 27% faster Artifact retrieval!
- 🌐 Faster offline sync: Keep your long-running experiments synced effortlessly with improved offline speeds.
See our benchmark analysis for more information on performance improvements.
Contributing
Your contributions are welcome! Please follow our contributing guide for more details.
Feedback and bug reporting
We're eager to hear your thoughts on wandb-core. Your feedback and bug reports are invaluable. If you encounter any issues, please raise a GitHub issue and mention your use of wandb-core.
Compatibility
Platform
wandb-core is pre-built for the following platforms:
- Linux:
x86_64,aarch64 - macOS:
x86_64,arm64 - Windows:
amd64
If it is not supported for your platform, you will see an error if you try to start a run. If you're interested in support for additional platforms, please inform us by opening a GitHub issue. Your feedback helps us prioritize new platform support.
W&B Server compatibility
wandb-core is compatible with our production and dedicated cloud deployments as well as
W&B Server versions >=0.40.0.
Disabling wandb-core
If you need to revert to the previous SDK backend while using wandb >= 0.18.0, you can do so by following one of these steps:
Option 1: Add the following line to your script:
wandb.require("legacy-service")
Option 2: Alternatively, set the environment variable WANDB__REQUIRE_LEGACY_SERVICE to TRUE:
export WANDB__REQUIRE_LEGACY_SERVICE=TRUE
Note:
- Starting from version 0.18.0, the
wandb-coreservice is the default runtime service. As a result, callingwandb.require("core")is unnecessary and has no effect inwandb>=0.18.0.
Directories
¶
| Path | Synopsis |
|---|---|
|
api
|
|
|
cmd
|
|
|
generate_gql
genqlient is a GraphQL client generator for Go.
|
genqlient is a GraphQL client generator for Go. |
|
internal
|
|
|
api
W&B backend API.
|
W&B backend API. |
|
clients
Deprecated package providing utilities for defining HTTP clients.
|
Deprecated package providing utilities for defining HTTP clients. |
|
filestream
Package filestream communicates with the W&B backend filestream service.
|
Package filestream communicates with the W&B backend filestream service. |
|
filetransfertest
Testability for the filetransfer package.
|
Testability for the filetransfer package. |
|
gqlmock
Package gqlmock provides utilities to mock a GraphQL API for tests.
|
Package gqlmock provides utilities to mock a GraphQL API for tests. |
|
paths
Package paths enables a "parse-don't-validate" approach to file paths.
|
Package paths enables a "parse-don't-validate" approach to file paths. |
|
runconsolelogs
Package runconsolelogs uploads a run's captured console output.
|
Package runconsolelogs uploads a run's captured console output. |
|
runfiles
Package runfiles deals with the saved files of a W&B run.
|
Package runfiles deals with the saved files of a W&B run. |
|
runwork
Package runwork manages all work that's part of a run.
|
Package runwork manages all work that's part of a run. |
|
runworktest
Package runworktest provides testing utilities for runwork.
|
Package runworktest provides testing utilities for runwork. |
|
settings
Defines W&B settings.
|
Defines W&B settings. |
|
tensorboard
Package tensorboard integrates wandb with TensorBoard.
|
Package tensorboard integrates wandb with TensorBoard. |
|
terminalemulator
Package terminalemulator implements a virtual terminal that supports common ANSI / xterm escape sequences.
|
Package terminalemulator implements a virtual terminal that supports common ANSI / xterm escape sequences. |
|
waiting
Package waiting helps write testable code that sleeps.
|
Package waiting helps write testable code that sleeps. |
|
waitingtest
Package waitingtest defines fakes for package `waiting`.
|
Package waitingtest defines fakes for package `waiting`. |
|
watcher
Package watcher notifies on changes to files or directories.
|
Package watcher notifies on changes to files or directories. |
|
watchertest
Package watchertest defines a fake Watcher implementation for testing.
|
Package watchertest defines a fake Watcher implementation for testing. |
|
wbvalue
Package wbvalue allows saving complex run objects.
|
Package wbvalue allows saving complex run objects. |
|
pkg
|
|
|
launch
TODO: this code desperately needs love and refactoring
|
TODO: this code desperately needs love and refactoring |
|
leveldb
Package crc implements the checksum algorithm used throughout leveldb.
|
Package crc implements the checksum algorithm used throughout leveldb. |