README
¶
Fleetspeak
Fleetspeak is a framework for communicating with a fleet of machines, with a focus on security monitoring and basic administrative use cases. It is a subproject of GRR, and can be seen as an effort to modularizing and modernizing its communication mechanism.
Status
We have this code working internally as part of our GRR installation.
Getting Started
On linux, assuming a recent version of the go development environment (see the
go.mod file for the exact requirement) and virtualenv, the following sequence
of commands will build and test this pre-release:
go get -v -t github.com/google/fleetspeak/...
# Assuming default $GOPATH:
cd ~/go/src/github.com/google/fleetspeak
# Setup virtualenv - fleetspeak provides some python integration libraries,
# and this ensures they are set up in a known way.
virtualenv $HOME/FSENV
source $HOME/FSENV/bin/activate
pip install -e fleetspeak_python/
# Set mysql parameters. The mysql datastore test will run if the following environment
# variables are set. Otherwise it will be skipped.
export MYSQL_TEST_USER=<username>
export MYSQL_TEST_PASS=<password> # will assume null password if unset.
export MYSQL_TEST_ADDR=<host:port>
# Build and test the release:
fleetspeak/build.sh
fleetspeak/test.sh
# After modifying proto files, the resulting go and python files need to be regenerated:
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.1
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.4
fleetspeak/generate_go_py_protos.sh
Once built, you can take a look at the files and instructions in our demo directory.
DISCLAIMER
While the code presented here is in some sense feature complete, much of it is barely tested or documented, and breaking changes are still possible. Therefore, please consider this a preview release while the dust settles. Suggestions and pull requests are very much appreciated.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
fleetspeak_admin
Package main implements a general command line interface which performs administrative actions on a fleetspeak installation.
|
Package main implements a general command line interface which performs administrative actions on a fleetspeak installation. |
|
fleetspeak_config
Package main implements a configuration tool which (partially) automates the configuration of a Fleetspeak installation.
|
Package main implements a configuration tool which (partially) automates the configuration of a Fleetspeak installation. |
|
fleetspeak_server
Package main defines an entry point for a general purpose fleetspeak server.
|
Package main defines an entry point for a general purpose fleetspeak server. |
|
fleetspeak
|
|
|
src/admin/cli
Package cli contains methods useful for implementing administrative command line utilities.
|
Package cli contains methods useful for implementing administrative command line utilities. |
|
src/admin/history
Package history analyzes client contact history to compute statistics and find anomalies.
|
Package history analyzes client contact history to compute statistics and find anomalies. |
|
src/client
Package client contains the components and utilities that every Fleetspeak client should include.
|
Package client contains the components and utilities that every Fleetspeak client should include. |
|
src/client/channel
Package channel provides fleetspeak.Message passing over interprocess pipes.
|
Package channel provides fleetspeak.Message passing over interprocess pipes. |
|
src/client/clienttestutils
Package clienttestutils contains utility functions for the client test, in part platform-specific.
|
Package clienttestutils contains utility functions for the client test, in part platform-specific. |
|
src/client/clitesting
Package clitesting contains utilities useful for testing clients and client components.
|
Package clitesting contains utilities useful for testing clients and client components. |
|
src/client/comms
Package comms defines the interface between the Fleetspeak client base library and the Communicator component used to talk to the server.
|
Package comms defines the interface between the Fleetspeak client base library and the Communicator component used to talk to the server. |
|
src/client/config
Package config defines structures and definitions relating to the client's configuration.
|
Package config defines structures and definitions relating to the client's configuration. |
|
src/client/daemonservice
Package daemonservice implements a service which runs and communicates with a separate daemon subprocess.
|
Package daemonservice implements a service which runs and communicates with a separate daemon subprocess. |
|
src/client/daemonservice/client
Package client is a go client library for daemonservice.
|
Package client is a go client library for daemonservice. |
|
src/client/daemonservice/command
Package command provides a relatively thin wrapper around exec.Cmd, adding support for communicating with the dependent process using a channel.Channel.
|
Package command provides a relatively thin wrapper around exec.Cmd, adding support for communicating with the dependent process using a channel.Channel. |
|
src/client/daemonservice/execution
Package execution provides an abstraction for a single execution of a command with the context of daemonservice.
|
Package execution provides an abstraction for a single execution of a command with the context of daemonservice. |
|
src/client/daemonservice/testclient
Package testclient implements a daemonservice client meant for testing.
|
Package testclient implements a daemonservice client meant for testing. |
|
src/client/flow
Package flow contains structures and utility method relating to client-server flow control configuration.
|
Package flow contains structures and utility method relating to client-server flow control configuration. |
|
src/client/generic
Package generic provides support methods to build a generic client, not bound to a particular installation.
|
Package generic provides support methods to build a generic client, not bound to a particular installation. |
|
src/client/https
Package https provides comms.Communicator implementations which connect to the Fleetspeak server using HTTPS.
|
Package https provides comms.Communicator implementations which connect to the Fleetspeak server using HTTPS. |
|
src/client/internal/config
Package config contains internal structures and methods relating to managing a client's configuration.
|
Package config contains internal structures and methods relating to managing a client's configuration. |
|
src/client/internal/message
Package message implements utility structures and methods used by the FS client to manage messages.
|
Package message implements utility structures and methods used by the FS client to manage messages. |
|
src/client/internal/monitoring
Package monitoring contains utilities for gathering data about resource usage in order to monitor client-side resource usage.
|
Package monitoring contains utilities for gathering data about resource usage in order to monitor client-side resource usage. |
|
src/client/internal/process
Package process encapsulates OS-independent process management helpers.
|
Package process encapsulates OS-independent process management helpers. |
|
src/client/service
Package service defines the interface that fleetspeak client side services must implement, along with some related types.
|
Package service defines the interface that fleetspeak client side services must implement, along with some related types. |
|
src/client/signer
Package signer defines an interface to add additional signatures to communications with the Fleetspeak server.
|
Package signer defines an interface to add additional signatures to communications with the Fleetspeak server. |
|
src/client/socketservice
Package socketservice implements a service which runs and communicates with a separate independent process over a local unix socket (or similar).
|
Package socketservice implements a service which runs and communicates with a separate independent process over a local unix socket (or similar). |
|
src/client/socketservice/checks
Package checks implements code which checks permissions of socket files to mitigate the possibility of a non-root attacker messing with socketservice communications channel.
|
Package checks implements code which checks permissions of socket files to mitigate the possibility of a non-root attacker messing with socketservice communications channel. |
|
src/client/socketservice/client
Package client is a go client library for socketservice.Service.
|
Package client is a go client library for socketservice.Service. |
|
src/client/socketservice/testclient
Package testclient implements a socketservice client meant for testing.
|
Package testclient implements a socketservice client meant for testing. |
|
src/client/stats
Package stats contains interfaces and utilities relating to the collection of statistics from a fleetspeak client.
|
Package stats contains interfaces and utilities relating to the collection of statistics from a fleetspeak client. |
|
src/client/stdinservice
Package stdinservice implements a service which, on request, executes a command on the client, passes data to its stdin and returns the result.
|
Package stdinservice implements a service which, on request, executes a command on the client, passes data to its stdin and returns the result. |
|
src/client/watchdog
Package watchdog implements a simple watchdog timer which dumps a trace of all goroutines to a file, and then ends the program, if the timer reaches its limit.
|
Package watchdog implements a simple watchdog timer which dumps a trace of all goroutines to a file, and then ends the program, if the timer reaches its limit. |
|
src/common/anypbtest
Package anypbtest offers test helpers for working with AnyPb protos.
|
Package anypbtest offers test helpers for working with AnyPb protos. |
|
src/common/fscontext
Package fscontext contains helpers for migrating Fleetspeak to context.Context.
|
Package fscontext contains helpers for migrating Fleetspeak to context.Context. |
|
src/common/should
Package should lets callers indicate impossible conditions from the code.
|
Package should lets callers indicate impossible conditions from the code. |
|
src/comtesting
Package comtesting implements utility methods useful for testing both client and server components.
|
Package comtesting implements utility methods useful for testing both client and server components. |
|
src/config/certs
Package certs contains utility methods for reading and verifying the certificates needed to configure a Fleetspeak installation.
|
Package certs contains utility methods for reading and verifying the certificates needed to configure a Fleetspeak installation. |
|
src/config/server
Package serer
|
Package serer |
|
src/inttesting/frr
Package frr implements the "Fake Rapid Response" service.
|
Package frr implements the "Fake Rapid Response" service. |
|
src/inttesting/integrationtest
Package integrationtest defines methods to implement integration tests in which a server and one or more clients are brought up and exercised.
|
Package integrationtest defines methods to implement integration tests in which a server and one or more clients are brought up and exercised. |
|
src/server
Package server contains the components and utilities that every Fleetspeak server should include.
|
Package server contains the components and utilities that every Fleetspeak server should include. |
|
src/server/admin
Package admin defines an administrative interface into the fleetspeak system.
|
Package admin defines an administrative interface into the fleetspeak system. |
|
src/server/authorizer
Package authorizer defines interfaces and utility methods to validate and limit client communications.
|
Package authorizer defines interfaces and utility methods to validate and limit client communications. |
|
src/server/comms
Package comms defines the interface used by the Fleetspeak modules which communicate with clients.
|
Package comms defines the interface used by the Fleetspeak modules which communicate with clients. |
|
src/server/components
Package components defines and instantiates the components needed by a generic Fleetspeak server.
|
Package components defines and instantiates the components needed by a generic Fleetspeak server. |
|
src/server/components/authorizer
Package authorizer provide generic implementations and utility methods for Fleetspeak's authorizer component type.
|
Package authorizer provide generic implementations and utility methods for Fleetspeak's authorizer component type. |
|
src/server/components/https
Package https provides generic implementations and utility methods for Fleetspeak https communication component type.
|
Package https provides generic implementations and utility methods for Fleetspeak https communication component type. |
|
src/server/components/notifications
Package notifications defines basic Listener/Notification support for generic Fleetspeak servers.
|
Package notifications defines basic Listener/Notification support for generic Fleetspeak servers. |
|
src/server/cpsservice
Package cpsservice defines a service.Service which publishes all received messages to a Google Cloud Pub/Sub topic.
|
Package cpsservice defines a service.Service which publishes all received messages to a Google Cloud Pub/Sub topic. |
|
src/server/db
Package db defines the interface that fleetspeak expects from its persistence layer.
|
Package db defines the interface that fleetspeak expects from its persistence layer. |
|
src/server/grpcservice
Package grpcservice defines a service.Service which passes all received messages to a destination host using grpc.
|
Package grpcservice defines a service.Service which passes all received messages to a destination host using grpc. |
|
src/server/grpcservice/client/testing
Package main tests a fleetspeak server talking through a grpcserver to a python loopback process.
|
Package main tests a fleetspeak server talking through a grpcserver to a python loopback process. |
|
src/server/https
Package https implements an https-based server Communicator.
|
Package https implements an https-based server Communicator. |
|
src/server/ids
Package ids defines identifier types and utility methods specific to the fleetspeak server and server components.
|
Package ids defines identifier types and utility methods specific to the fleetspeak server and server components. |
|
src/server/internal/broadcasts
Package broadcasts contains code for a Fleetspeak server to manage broadcasts.
|
Package broadcasts contains code for a Fleetspeak server to manage broadcasts. |
|
src/server/internal/cache
Package cache contains caching structures using internally by the Fleetspeak server.
|
Package cache contains caching structures using internally by the Fleetspeak server. |
|
src/server/internal/ftime
Package ftime defines "fleetspeak time" as a global variable.
|
Package ftime defines "fleetspeak time" as a global variable. |
|
src/server/internal/notifications
Package internal contains miscellaneous small components used internally by the Fleetspeak server.
|
Package internal contains miscellaneous small components used internally by the Fleetspeak server. |
|
src/server/internal/services
Package services defines internal fleetspeak components relating to services.
|
Package services defines internal fleetspeak components relating to services. |
|
src/server/internal/signatures
Package signatures implements utility methods for checking cryptographic signatures.
|
Package signatures implements utility methods for checking cryptographic signatures. |
|
src/server/mysql
Package mysql implements the fleetspeak datastore interface using a mysql database.
|
Package mysql implements the fleetspeak datastore interface using a mysql database. |
|
src/server/notifications
Package notifications defines the plugin interface for components which allow the fleetspeak servers within an installation to notify each other about interesting events.
|
Package notifications defines the plugin interface for components which allow the fleetspeak servers within an installation to notify each other about interesting events. |
|
src/server/sertesting
Package sertesting contains utilities useful for testing the fleetspeak server and server components.
|
Package sertesting contains utilities useful for testing the fleetspeak server and server components. |
|
src/server/service
Package service defines the interface that fleetspeak expects from its service implementations.
|
Package service defines the interface that fleetspeak expects from its service implementations. |
|
src/server/sqlite
Package sqlite implements the fleetspeak datastore interface using an sqlite database.
|
Package sqlite implements the fleetspeak datastore interface using an sqlite database. |
|
src/server/stats
Package stats contains interfaces and utilities relating to the collection of statistics from a fleetspeak server.
|
Package stats contains interfaces and utilities relating to the collection of statistics from a fleetspeak server. |
|
src/server/testserver
Package testserver configures and creates a Fleetspeak server instance suitable for unit tests.
|
Package testserver configures and creates a Fleetspeak server instance suitable for unit tests. |
|
terraform
|
|
Click to show internal directories.
Click to hide internal directories.