query/

directory
Version: v1.4.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 18, 2022 License: Apache-2.0

README

WARNING: This is Alpha software and not intended for use until a stable release.

M3Coordinator

M3Coordinator is a service which provides APIs for reading/writing to M3DB at a global and placement specific level. It also acts as a bridge between Prometheus and M3DB. Using this bridge, M3DB acts as a long term storage for Prometheus using the remote read/write endpoints. A detailed explanation of setting up long term storage for Prometheus can be found here.

Running in Docker

Note that all commands are run within the root of the m3coordinator directory except where specified.

Running both m3coordinator and Prometheus in containers:

You can launch a Prometheus and m3coordinator container using docker-compose. However, you must first build the m3coordinator Docker image.

To do so, you will need the m3coordinator binary:

$ GOOS=linux GOARCH=amd64 CGO_ENABLED=0 make services

Once you have the binary, you can run the following in the root of the project to make the Docker image:

$ docker build -t m3coordinator -f docker/m3coordinator/Dockerfile .

Finally, you can spin up the two containers using docker-compose within the docker/ directory:

$ docker-compose up

Note: The default local ports for Prometheus and m3coordinator are 9090 and 7201, respectively, and the default prometheus.yml file is docker/prometheus.yml

If you want to override these, you can pass in the following environment variables to the docker-compose command:

  • LOCAL_PROM_PORT
  • LOCAL_M3COORD_PORT
  • LOCAL_PROM_YML

(e.g. $ LOCAL_PROM_PORT=XXXX LOCAL_M3COORD_PORT=XXXX LOCAL_PROM_YML=/path/to/yml docker-compose up)

Running m3coordinator locally (on mac only) and Prometheus in Docker container (for development):

Build m3coordinator binary:

$ make services

Run m3coordinator binary:

$ ./bin/m3coordinator -f src/query/config/m3query-dev-etcd.yml

Run Prometheus Docker image:

$ docker run -p 9090:9090 -v $GOPATH/src/github.com/m3db/m3/src/query/docker/prometheus-mac.yml:/etc/prometheus/prometheus.yml quay.io/prometheus/prometheus

Running on GCP

Setup GCP for single m3db node

For a multi-node cluster, see here

Setup GCP for m3coordinator:

1. Make sure you select a base image with Docker pre-installed
2. Follow steps 1-5 from the above section (clone `m3coordinator` instead of `m3db`)
3. The config file, which is located at `m3coordinator/benchmark/configs/benchmark.yml` will need the same config topology as the m3db config
4. Run m3coordinator - you should see this message with the number of hosts you specified: `[I] successfully updated topology to 3 hosts` with no other warning or error messsages
    $ ./bin/m3coordinator --config.file benchmark/configs/benchmark.yml

Setup and run Prometheus:

1. You can run Prometheus from the same box as m3coordinator
2. Update the `prometheus.yml` in `m3coordinator/docker` so that the `remote_read` and `remote_write` endpoints are bound to the IP address of the host that m3coordinator is running on:
    - e.g.
        ```
        remote_read:
            - url: http://10.142.0.8:7201/api/v1/prom/remote/read

        remote_write:
            - url: http://10.142.0.8:7201/api/v1/prom/remote/write
        ```
3. Run Prometheus
    $ sudo docker run -p 9090:9090 -v $GOPATH/src/github.com/m3db/m3/src/query/docker/prometheus.yml:/etc/prometheus/prometheus.yml quay.io/prometheus/prometheus

Directories

Path Synopsis
api
v1/handler/database
Package database contains API endpoints for managing the database.
Package database contains API endpoints for managing the database.
v1/handler/prom
Package prom provides custom handlers that support the prometheus query endpoints.
Package prom provides custom handlers that support the prometheus query endpoints.
v1/handler/prometheus/native
Package native handles the prometheus api endpoints using m3.
Package native handles the prometheus api endpoints using m3.
v1/httpd
Package httpd contains http routers.
Package httpd contains http routers.
v1/middleware
Package middleware contains HTTP middleware functions.
Package middleware contains HTTP middleware functions.
v1/options
Package options configures query http handlers.
Package options configures query http handlers.
v1/route
Package route stores common HTTP route handling code
Package route stores common HTTP route handling code
v1/validators
package validators contains validation logics for the api.
package validators contains validation logics for the api.
Package block is a generated GoMock package.
Package block is a generated GoMock package.
cluster
Package executor is a generated GoMock package.
Package executor is a generated GoMock package.
transform
Package transform is a generated GoMock package.
Package transform is a generated GoMock package.
tag
temporal
Package temporal is a generated GoMock package.
Package temporal is a generated GoMock package.
generated
proto/admin
Package admin is a generated protocol buffer package.
Package admin is a generated protocol buffer package.
proto/prompb
Package prompb is a generated protocol buffer package.
Package prompb is a generated protocol buffer package.
proto/rpcpb
Package rpcpb is a generated protocol buffer package.
Package rpcpb is a generated protocol buffer package.
storage
Package storage is a generated GoMock package.
Package storage is a generated GoMock package.
ts
m3ql
Package m3ql is a generated GoMock package.
Package m3ql is a generated GoMock package.
policy
Package promqlengine contains utilities for PromQL engine.
Package promqlengine contains utilities for PromQL engine.
Package source identifies the source of query requests.
Package source identifies the source of query requests.
Package storage is a generated GoMock package.
Package storage is a generated GoMock package.
m3
Package m3 is a generated GoMock package.
Package m3 is a generated GoMock package.
promremote
Package promremote implements storage interface backed by Prometheus remote write capable endpoints.
Package promremote implements storage interface backed by Prometheus remote write capable endpoints.
promremote/promremotetest
Package promremotetest provides test utilities.
Package promremotetest provides test utilities.
stores
compare
Package compare provides comparison functions for testing.
Package compare provides comparison functions for testing.
m3
Package ts is a generated GoMock package.
Package ts is a generated GoMock package.
json
Package json contains logic for writing JSON.
Package json contains logic for writing JSON.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL