Affected by 
README
¶
| Continuous integration | License |
|---|---|
 |
 |
Minder
Minder is a platform to manage the security of your software supply chain. It helps teams automate and enforce secure supply chain best practices across the software development lifecycle.
It is currently in early and active development and is considered experimental.
Get Hacking (Development)
Prerequisites
- Go
- Docker or..
- Podman
- Docker Compose or..
Once you have these and have cloned the repository, you'll also need to install the other tools and make sure that $HOME/go/bin is in your PATH.
Clone the repository
git clone git@github.com:stacklok/minder.git
Build the application
make build
Initialize the configuration
Before running the makefile targets, initialize the application's configuration file. You may do so by doing.
cp config/config.yaml.example config.yaml
Alternatively, you may bootstrap the whole development environment, which includes initializing this file with:
make bootstrap
Initialize the database
Both the minder application and the tests need a Postgres database to be running. For development use, the standard defaults should suffice:
docker-compose up -d postgres
make migrateup
Start the identity provider (Keycloak)
To login, we rely on an identity provider that stores the usernames and passwords.
docker-compose up -d keycloak
Run the application
You must initialize the database before starting the application. Then run the application:
bin/minder-server serve
Or direct from source
make run-server
The application will be available on http://localhost:8080 and gRPC on localhost:8090.
Running the server under Compose:
NOTE: the command will be docker-compose. You'll need to install the ko tool do the build and run.
# The repo to push to; "ko.local" is a special string meaning your local Docker repo
KO_DOCKER_REPO=ko.local
# ko adds YAML document separators at the end of each document, which docker-compose doesn't like
docker-compose -f <(ko resolve -f docker-compose.yaml | sed 's/^---$//') up
Run the tests
Note that you need to have started the database and loaded the schema before running the tests:
make test
You can alse use make cover to check coverage.
Install tools
make bootstrap
CLI
The CLI is available in the cmd/cli directory.
go run cmd/cli/main.go --help
To start with, you will need to run cli auth login using root:root as the credentials.
This will open a browser window with the identity provider login page.
APIs
API Doc here
The APIs are defined in protobuf here.
An OpenAPI / swagger spec is generated to JSON
It can be accessed over gRPC or HTTP using gprc-gateway.
How to generate protobuf stubs
We use buf to generate the gRPC / HTTP stubs (both protobuf and openAPI).
To build the stubs, run:
make gen
Database migrations and tooling
Minder uses sqlc to generate Go code from SQL.
The main configuration file is sqlc.yaml.
To make changes to the database schema, create a new migration file in the
database/migrations directory.
Add any queries to the database/queries/sqlc.sql file.
To generate the Go code, run:
make sqlc
Users will then need to peform a migration
make migrateup
make migratedown
Mock
Minder uses mockgen to generate mocks.
To generate the mocks, run:
mockgen -package mockdb -destination database/mock/store.go github.com/stacklok/minder/internal/db Store
and
mockgen -package auth -destination internal/auth/mock/jwtauth.go github.com/stacklok/minder/internal/auth JwtValidator,KeySetFetcher
Configuration
Minder uses viper for configuration.
An example configuration file is config/config.yaml.example.
Most values should be quite self-explanatory.
Before running the app, please copy the content of config/config.yaml.example into $PWD/config.yaml file, and modify to use your own settings.
Social login configuration
First, create an OAuth2 application for GitHub here. Select
New OAuth App and fill in the details. The callback URL should be http://localhost:8081/realms/stacklok/broker/github/endpoint.
Create a new client secret for your OAuth2 client.
Using the client ID and client secret you created above, enable GitHub login on Keycloak by running the following command:
make KC_GITHUB_CLIENT_ID=<client_id> KC_GITHUB_CLIENT_SECRET=<client_secret> github-login
Initial setup / Getting started
Login
First, login with the default credentials:
go run ./cmd/cli/main.go auth login
This will open a browser window with the identity provider login page.
Enter the credentials root:root.
You will immediately be prompted to change your password.
Upon successful authentication you can close your browser.
You will see the following prompt in your terminal:
You have been successfully logged in. Your access credentials saved to /var/home/jaosorior/.config/minder/credentials.json
Enroll provider
First, you'll need to enroll your first provider. Before doing this, make sure to set up a GitHub OAuth2 Application,
and fill in the appropriate settings in your config.yaml file.
You may create an OAuth2 application here. Select
New OAuth App and fill in the details. The callback URL should be http://localhost:8080/api/v1/auth/callback/github.
Create a new client secret and fill in the client_id and client_secret in your config.yaml file.
Once the Application is registered and the configuration is set, you can enroll the provider:
go run ./cmd/cli/main.go provider enroll -n github
This will take you through the OAuth2 flow and will result in the provider filling up the repositories table with the repositories you have access to.
Register repositories
Now that you've granted the GitHub app permissions to access your repositories, you can register them:
go run ./cmd/cli/main.go repo register -n github
Once you've registered the repositories, the Minder server will listen for events from GitHub and will automatically create the necessary webhooks for you.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
cli
command
Package main provides the entrypoint for the minder cli
|
Package main provides the entrypoint for the minder cli |
|
cli/app
Package app provides the root command for the minder CLI
|
Package app provides the root command for the minder CLI |
|
cli/app/apply
Package apply provides the apply command for the minder CLI
|
Package apply provides the apply command for the minder CLI |
|
cli/app/artifact
Package artifact provides the artifact subcommands
|
Package artifact provides the artifact subcommands |
|
cli/app/auth
Package auth provides the auth command project for the minder CLI.
|
Package auth provides the auth command project for the minder CLI. |
|
cli/app/docs
Package apply provides the apply command for the minder CLI
|
Package apply provides the apply command for the minder CLI |
|
cli/app/profile
Package profile provides the CLI subcommand for managing profiles
|
Package profile provides the CLI subcommand for managing profiles |
|
cli/app/profile_status
Package profile_status provides the CLI subcommand for managing profile statuses
|
Package profile_status provides the CLI subcommand for managing profile statuses |
|
cli/app/provider
Package provider is the root command for the provider subcommands
|
Package provider is the root command for the provider subcommands |
|
cli/app/repo
Package repo contains the repo logic for the control plane
|
Package repo contains the repo logic for the control plane |
|
cli/app/rule_type
Package rule_type provides the CLI subcommand for managing rules
|
Package rule_type provides the CLI subcommand for managing rules |
|
dev
command
Package main provides the entrypoint for the rule development cli
|
Package main provides the entrypoint for the rule development cli |
|
dev/app
Package app provides the root command for the medev CLI
|
Package app provides the root command for the medev CLI |
|
dev/app/rule_type
Package rule_type provides the CLI subcommand for developing rules e.g.
|
Package rule_type provides the CLI subcommand for developing rules e.g. |
|
server
command
Package main provides the entrypoint for the minder server
|
Package main provides the entrypoint for the minder server |
|
server/app
Package app provides the entrypoint for the minder migrations
|
Package app provides the entrypoint for the minder migrations |
|
database
|
|
|
mock
Package mockdb is a generated GoMock package.
|
Package mockdb is a generated GoMock package. |
|
internal
|
|
|
assets
Package assets contains the static assets.
|
Package assets contains the static assets. |
|
auth/mock
Package auth is a generated GoMock package.
|
Package auth is a generated GoMock package. |
|
config
Package config contains a centralized structure for all configuration options.
|
Package config contains a centralized structure for all configuration options. |
|
container
Package container provides a client for interacting with container images
|
Package container provides a client for interacting with container images |
|
controlplane
Package controlplane contains the gRPC server implementation for the control plane
|
Package controlplane contains the gRPC server implementation for the control plane |
|
crypto
Package crypto provides cryptographic functions
|
Package crypto provides cryptographic functions |
|
engine
Package engine provides an implementation of the profile-engine-related objects, interfaces and functionality.
|
Package engine provides an implementation of the profile-engine-related objects, interfaces and functionality. |
|
engine/actions
Package actions provide necessary interfaces and implementations for processing actions, such as remediation and alerts.
|
Package actions provide necessary interfaces and implementations for processing actions, such as remediation and alerts. |
|
engine/actions/alert
Package alert provides necessary interfaces and implementations for processing alerts.
|
Package alert provides necessary interfaces and implementations for processing alerts. |
|
engine/actions/alert/noop
Package noop provides a fallback alert engine for cases where no alert is set.
|
Package noop provides a fallback alert engine for cases where no alert is set. |
|
engine/actions/alert/security_advisory
Package security_advisory provides necessary interfaces and implementations for creating alerts of type security advisory.
|
Package security_advisory provides necessary interfaces and implementations for creating alerts of type security advisory. |
|
engine/actions/remediate
Package remediate provides necessary interfaces and implementations for remediating rules.
|
Package remediate provides necessary interfaces and implementations for remediating rules. |
|
engine/actions/remediate/gh_branch_protect
Package gh_branch_protect provides the github branch protection remediation engine
|
Package gh_branch_protect provides the github branch protection remediation engine |
|
engine/actions/remediate/noop
Package noop provides a fallback remediation engine for cases where no remediation is set.
|
Package noop provides a fallback remediation engine for cases where no remediation is set. |
|
engine/actions/remediate/pull_request
Package pull_request provides the pull request remediation engine
|
Package pull_request provides the pull request remediation engine |
|
engine/actions/remediate/rest
Package rest provides the REST remediation engine
|
Package rest provides the REST remediation engine |
|
engine/errors
Package errors provides errors for the evaluator engine
|
Package errors provides errors for the evaluator engine |
|
engine/eval
Package eval provides necessary interfaces and implementations for evaluating rules.
|
Package eval provides necessary interfaces and implementations for evaluating rules. |
|
engine/eval/jq
Package jq provides the jq profile evaluator
|
Package jq provides the jq profile evaluator |
|
engine/eval/pr_actions
Package pr_actions contains shared code to take on PRs
|
Package pr_actions contains shared code to take on PRs |
|
engine/eval/rego
Package rego provides the rego rule evaluator
|
Package rego provides the rego rule evaluator |
|
engine/eval/trusty
Package trusty provides an evaluator that uses the trusty API
|
Package trusty provides an evaluator that uses the trusty API |
|
engine/eval/vulncheck
Package vulncheck provides the vulnerability check evaluator
|
Package vulncheck provides the vulnerability check evaluator |
|
engine/ingestcache
Package ingestcache a cache that is used to cache the results of ingesting data.
|
Package ingestcache a cache that is used to cache the results of ingesting data. |
|
engine/ingester
Package ingester provides necessary interfaces and implementations for ingesting data for rules.
|
Package ingester provides necessary interfaces and implementations for ingesting data for rules. |
|
engine/ingester/artifact
Package artifact provides the artifact ingestion engine
|
Package artifact provides the artifact ingestion engine |
|
engine/ingester/builtin
Package builtin provides the builtin ingestion engine
|
Package builtin provides the builtin ingestion engine |
|
engine/ingester/diff
Package diff provides the diff rule data ingest engine
|
Package diff provides the diff rule data ingest engine |
|
engine/ingester/git
Package git provides the git rule data ingest engine
|
Package git provides the git rule data ingest engine |
|
engine/ingester/rest
Package rest provides the REST rule data ingest engine
|
Package rest provides the REST rule data ingest engine |
|
engine/interfaces
Package interfaces provides necessary interfaces and implementations for implementing engine plugins
|
Package interfaces provides necessary interfaces and implementations for implementing engine plugins |
|
entities
Package entities contains internal helper functions to deal with, validate and print the Entity protobuf enum.
|
Package entities contains internal helper functions to deal with, validate and print the Entity protobuf enum. |
|
events
Package events provides the eventer object which is responsible for setting up the watermill router and handling the incoming events
|
Package events provides the eventer object which is responsible for setting up the watermill router and handling the incoming events |
|
logger
Package logger provides a general logging tools
|
Package logger provides a general logging tools |
|
providers
Package providers contains general utilities for interacting with providers.
|
Package providers contains general utilities for interacting with providers. |
|
providers/git
Package git provides a client for interacting with Git providers
|
Package git provides a client for interacting with Git providers |
|
providers/github
Package github provides a client for interacting with the GitHub API
|
Package github provides a client for interacting with the GitHub API |
|
providers/github/mock
Package mockgh is a generated GoMock package.
|
Package mockgh is a generated GoMock package. |
|
providers/http
Package http implements an HTTP client for interacting with an HTTP API.
|
Package http implements an HTTP client for interacting with an HTTP API. |
|
providers/telemetry
Package telemetry provides the telemetry interfaces and implementations for providers
|
Package telemetry provides the telemetry interfaces and implementations for providers |
|
reconcilers
Package reconcilers contains the reconcilers for the various types of entities in minder.
|
Package reconcilers contains the reconcilers for the various types of entities in minder. |
|
util
Package util provides helper functions for the minder CLI.
|
Package util provides helper functions for the minder CLI. |
|
util/cli
Package cli contains utility for the cli
|
Package cli contains utility for the cli |
|
util/jsonyaml
Package jsonyaml contains utility functions for converting to/from json and yaml
|
Package jsonyaml contains utility functions for converting to/from json and yaml |
|
util/rand
Package rand contains utility functions largely for unit testing.
|
Package rand contains utility functions largely for unit testing. |
|
util/schemaupdate
Package schemaupdate contains utility functions to compare two schemas for updates
|
Package schemaupdate contains utility functions to compare two schemas for updates |
|
pkg
|
|
|
api/protobuf/go/minder/v1
Package v1 is a reverse proxy.
|
Package v1 is a reverse proxy. |
|
providers/v1
Package v1 for providers provides the public interfaces for the providers implemented by minder.
|
Package v1 for providers provides the public interfaces for the providers implemented by minder. |
|
rule_methods
Package rule_methods provides the methods that are used by the rules
|
Package rule_methods provides the methods that are used by the rules |