atlas

package module
Version: v0.0.0-...-13f0d38 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 6, 2021 License: MIT Imports: 15 Imported by: 4

README

ripe-atlas

  • RIPE Atlas v2 API access in Go. *

GitHub release GitHub issues Go Version Build Status GoDoc SemVer License Go Report Card

ripe-atlas is a Go library to access the RIPE Atlas REST API.

It features a simple CLI-based tool called atlas which serve both as a collection of use-cases for the library and an easy way to use it.

Work in progress, still incomplete

Table of content

Features

I am trying to implement the full REST API in Go. The API itself is not particularly complex but the settings and parameters are.

The following topic are available:

  • probes

    you can query one probe or ask for a list of probes with a few criterias

  • measurements

    you can create and list measurements.

  • results

    every measurement has a URI in the result json that points to the actual results. This fetch and display them.

In addition to these major commands, there are a few shortcut commands (see below):

  • dns
  • http
  • ip
  • keys
  • ntp
  • ping
  • sslcert/tls
  • traceroute

Installation

NOTE: you MUST have Go 1.8 or later. Previous versions did not have the ProxyHeader fields and thus no support for HTTP proxy.

Like many Go-based tools, installation is very easy

go get github.com/keltia/ripe-atlas/cmd/...

or

git clone https://github.com/keltia/ripe-atlas
make install

The library is fetched, compiled and installed in whichever directory is specified by $GOPATH. The atlas binary will also be installed (on windows, this will be called atlas.exe).

You can install the dependencies with go get

  • github.com/urfave/cli
  • github.com/naoina/toml

To run the tests, you will also need:

  • github.com/stretchr/assert

NOTE: please use and test the Windows version (use make windowsto generate it). It should work but I lack resources to play much with it.

API usage

You must foremost instanciate a new API client with

client, err := atlas.NewClient(config)

where config is an atlas.Config{} struct with various options.

All API calls after that will use client:

probe, err := client.GetProbe(12345)
Basics
  • Authentication
  • Probes
  • Measurements
  • Applications

CLI utility

The atlas command is a command-line client for the Go API.

Configuration

The atlas utility uses a configuration file in the TOML file format.

On UNIX, it is located in $HOME/.config/ripe-atlas/config.toml and in %LOCALAPPDATA%\RIPE-ATLAS on Windows.

There are only a few parameters for now, the most important one being your API Key for authenticate against the RIPE API endpoint. You can now specify the default probe set (and override it from the CLI):

# Default configuration file

API_key = "<INSERT-API-KEY>"
default_probe = <PROBE-ID>

[probe_set]

pool_size = <POOL-SIZE>
type = "area"
value = "WW"

Everything is a string except for pool_size and default_probe which are integers.

Be aware that if you ask for an IPv6 object (like a domain or machine name), the API will refuse your request if the IPv6 version of that object does not exist.

Important note

Not all parameters specified for the different commands are implemented, as you can see in the API Reference, there are a lot of different parameters like all the id__{gt,gte,lt,lte,in} stuff.

Proxy authentication

If you want to use atlas or the API behind an authenticating HTTP/HTTPS proxy, you need to create another configuration file holding the proxy credentials. The file is called .netrc under UNIX and is located in your $HOME directory. On Windows, it is in the same directory as config.toml inside %LOCALAPPDATA%\RIPE-ATLAS.

The format is described in the ftp(1) manpage:

machine <service> username <username> password <password>

in our case, the service is proxy (or default), you just need to fill in username and password.

machine proxy username john.doe password secret

The API now look by itself for the .netrc file, using my own github.com/keltia/proxy package.

atlas.go:

import "github.com/keltia/proxy"

// Check whether we have proxy authentication (from a separate config file)
authstr, err := proxy.SetupProxyAuth()

client, err = atlas.NewClient(atlas.Config{
    APIKey:       mycnf.APIKey,
    DefaultProbe: mycnf.DefaultProbe,
    PoolSize:     mycnf.PoolSize,
    ProxyAuth:    auth,
    Verbose:      fVerbose,
})

As an alternative, you can do the encoding yourself and put that in config.toml as proxy_auth.

Last, you can also use the full form for the https_proxy environment variable with user:password@proxy but it is not recommended to put your password in the clear like this.

Usage
NAME:
   atlas - RIPE Atlas CLI interface

USAGE:
   atlas [global options] command [command options] [arguments...]

VERSION:
   0.41

AUTHOR:
   Ollivier Robert <roberto@keltia.net>

COMMANDS:
     credits, c                 credits-related keywords
     dns, dig, drill            send dns queries
     http, https                connect to host/IP through HTTP
     ip                         returns current ip
     keys, k, key               key-related keywords
     measurements, measures, m  measurements-related keywords
     ntp                        get time from ntp server
     ping                       ping selected address
     probes, p, pb              probe-related keywords
     results, r, res            results for one measurement
     sslcert, tlscert, tls      get TLS certificate from host/IP
     traceroute, trace          traceroute to given host/IP
     help, h                    Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --format value, -f value      specify output format (NOT IMPLEMENTED)
   --debug, -D                   debug mode
   --verbose, -v                 verbose mode
   --fields value, -F value      specify which fields are wanted
   --include value, -I value     specify whether objects should be expanded
   --logfile value, -L value     specify a log file
   --mine, -M                    limit output to my objects
   --opt-fields value, -O value  specify which optional fields are wanted
   --page-size value, -P value   page size for results
   --sort value, -S value        sort results
   -1, --is-oneoff               one-time measurement
   -6, --ipv6                    Only IPv6
   -4, --ipv4                    Only IPv4
   --pool-size value, -N value   Number of probes to request (default: 0)
   --area-type value             Set type for probes (area, country, etc.)
   --area-value value            Value for the probe set (WW, West, etc.)
   --country value, -C value     Short cut to specify a country
   --tags value, -T value        Include/exclude tags for probesets
   --help, -h                    show help
   --version, -V

In addition to the main probes and measurements commands, it features fast-access to common tasks like pingand traceroute.

Now, every command has a -T or -tags parameter to add user-defined tags when the measurement is created. This is different from -T at the higher level. The latter is for selecting probes.

$ atlas ping -h
NAME:
   atlas ping - ping selected address

USAGE:
   atlas ping [command options] [arguments...]

DESCRIPTION:
   send echo/reply to an IP

OPTIONS:
   -T value, --tags value  add tags to measurement

You can use it like that:

$ atlas -N 10 -4 ping -T test-tag,foobar www.example.com

When looking at measurement results, it is very easy to use something like jq to properly display JSON data:

atlas results <ID> | jq .

You can also analyze the results, as explained here.

Here,to find the maximum RTT:

% ./atlas measurements results 10185594 | jq 'map(.result[0].rtt) | max'
24.10811

And with this jq file, to get more information from a measurement:

% cat ping-report.jq
map(.result) | flatten(1) | map(.rtt) | length as $total | 
 "Median: " + (sort |
      if length % 2 == 0 then .[length/2] else .[(length-1)/2] end | tostring),
 "Average: " + (map(select(. != null)) | add/length | tostring) + " ms",
 "Min: " + (map(select(. != null)) | min | tostring) + " ms",
 "Max: " + (max | tostring) + " ms",
 "Failures: " + (map(select(. == null)) | (length*100/$total) | tostring) + " %"
%./atlas measurements results 10185594 |  jq --raw-output --from-file ping-report.jq
Median: 15.068505
Average: 15.480822916666666 ms
Min: 3.786365 ms
Max: 24.164375 ms
Failures: 14.285714285714286 %

TODO

  • implement "anchors" & "participation requests"
  • more tests (and better ones!)
  • better display of results
  • refactoring to reduce code duplication: always in progress
  • even more tests

External Documentation

Contributing

Please see CONTRIBUTING.md for some simple rules.

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// ProbeTypes should be obvious
	ProbeTypes = []string{"area", "country", "prefix", "asn", "probes", "msm"}
	// AreaTypes should also be obvious
	AreaTypes = []string{"WW", "West", "North-Central", "South-Central", "North-East", "South-East"}
)
View Source
var ErrAPIKeyIsMandatory = errors.New("API call requires an API key")

ErrAPIKeyIsMandatory is returned when a call need one

View Source
var ErrInvalidAPIKey = errors.New("invalid API key")

ErrInvalidAPIKey is returned when the key is invalid

View Source
var ErrInvalidError = errors.New("Invalid request")

ErrInvalidAPIAnswer

View Source
var ErrInvalidMeasurementType = errors.New("invalid measurement type")

ErrInvalidMeasurementType is a new error

Functions

func AddQueryParameters

func AddQueryParameters(baseURL string, queryParams map[string]string) string

AddQueryParameters adds query parameters to the URL.

func FillDefinition

func FillDefinition(d *Definition, fields map[string]string) error

FillDefinition set a few parameters in a definition list

The goal here is to give a dictionary of string and let it figure out each field's type depending on the recipient's type in the struct.

func GetVersion

func GetVersion() string

GetVersion returns the API wrapper version

Types

type APIError

type APIError struct {
	Err struct {
		Status int    `json:"status"`
		Code   int    `json:"code"`
		Detail string `json:"detail"`
		Title  string `json:"title"`
		Errors []struct {
			Source struct {
				Pointer string
			} `json:"source"`
			Detail string
		} `json:"errors"`
	} `json:"error"`
}

APIError is for errors returned by the RIPE API.

func (APIError) Error

func (e APIError) Error() string

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is the main struct holding state in an API client

func NewClient

func NewClient(cfgs ...Config) (*Client, error)

NewClient is the first function to call. Yes, it does take multiple config and the last one wins.

func (*Client) DNS

func (c *Client) DNS(d *MeasurementRequest) (m *MeasurementResp, err error)

DNS creates a measurement

func (*Client) DeleteMeasurement

func (c *Client) DeleteMeasurement(id int) (err error)

DeleteMeasurement stops (not really deletes) a given measurement

func (*Client) FetchResult

func (c *Client) FetchResult(url string) (string, error)

FetchResult downloads result for a given measurement

func (*Client) GetCredits

func (c *Client) GetCredits() (credits *Credits, err error)

GetCredits returns high-level data for credits

func (*Client) GetKey

func (c *Client) GetKey(uuid string) (k Key, err error)

GetKey returns a given API key

func (*Client) GetKeys

func (c *Client) GetKeys(opts map[string]string) (kl []Key, err error)

GetKeys returns all your API keys

func (*Client) GetMeasurement

func (c *Client) GetMeasurement(id int) (m *Measurement, err error)

GetMeasurement gets info for a single one

func (*Client) GetMeasurements

func (c *Client) GetMeasurements(opts map[string]string) (m []Measurement, err error)

GetMeasurements gets info for a set

func (*Client) GetProbe

func (c *Client) GetProbe(id int) (p *Probe, err error)

GetProbe returns data for a single probe

func (*Client) GetProbes

func (c *Client) GetProbes(opts map[string]string) (p []Probe, err error)

GetProbes returns data for a collection of probes

func (*Client) HTTP

func (c *Client) HTTP(d *MeasurementRequest) (m *MeasurementResp, err error)

HTTP creates a measurement

func (*Client) HasAPIKey

func (c *Client) HasAPIKey() (string, bool)

HasAPIKey returns whether an API key is stored

func (*Client) NTP

func (c *Client) NTP(d *MeasurementRequest) (m *MeasurementResp, err error)

NTP creates a measurement

func (*Client) NewMeasurement

func (c *Client) NewMeasurement() (req *MeasurementRequest)

NewMeasurement create a new MeasurementRequest and fills some fields

func (*Client) Ping

func (c *Client) Ping(d *MeasurementRequest) (m *MeasurementResp, err error)

Ping creates a measurement

func (*Client) SSLCert

func (c *Client) SSLCert(d *MeasurementRequest) (m *MeasurementResp, err error)

SSLCert creates a measurement

func (*Client) SetOption

func (c *Client) SetOption(name, value string) *Client

SetOption sets a global option

func (*Client) Traceroute

func (c *Client) Traceroute(d *MeasurementRequest) (m *MeasurementResp, err error)

Traceroute creates a measurement

type Config

type Config struct {
	APIKey       string
	DefaultProbe int
	AreaType     string
	AreaValue    string
	IsOneOff     bool
	PoolSize     int
	WantAF       string
	ProxyAuth    string
	Verbose      bool
	Tags         string
	Log          *log.Logger
	Level        int
	// contains filtered or unexported fields
}

Config is the main object when creating an API Client

type Credits

type Credits struct {
	CurrentBalance            int    `json:"current_balance"`
	EstimatedDailyIncome      int    `json:"estimated_daily_income"`
	EstimatedDailyExpenditure int    `json:"estimated_daily_expenditure"`
	EstimatedDailyBalance     int    `json:"estimated_daily_balance"`
	CalculationTime           string `json:"calculation_time"`
	EstimatedRunoutSeconds    int    `json:"estimated_runout_seconds"`
	PastDayMeasurementResults int    `json:"past_day_measurement_results"`
	PastDayCreditsSpent       int    `json:"past_day_credits_spent"`
	IncomeItems               string `json:"income_items"`
	ExpenseItems              string `json:"expense_items"`
	Transactions              string `json:"transactions"`
}

Credits is holding credits data

type Definition

type Definition struct {
	// Required fields
	Description string `json:"description"`
	Type        string `json:"type"`
	AF          int    `json:"af"`

	// Required for all but "dns"
	Target string `json:"target,omitempty"`

	GroupID        int      `json:"group_id,omitempty"`
	Group          string   `json:"group,omitempty"`
	InWifiGroup    bool     `json:"in_wifi_group,omitempty"`
	Spread         int      `json:"spread,omitempty"`
	Packets        int      `json:"packets,omitempty"`
	PacketInterval int      `json:"packet_interval,omitempty"`
	Tags           []string `json:"tags,omitempty"`

	// Common parameters
	ExtraWait      int  `json:"extra_wait,omitempty"`
	IsOneoff       bool `json:"is_oneoff,omitempty"`
	IsPublic       bool `json:"is_public,omitempty"`
	ResolveOnProbe bool `json:"resolve_on_probe,omitempty"`

	// Default depends on type
	Interval int `json:"interval,omitempty"`

	// dns & traceroute parameters
	Protocol string `json:"protocol,omitempty"`

	// dns parameters
	QueryClass       string `json:"query_class,omitempty"`
	QueryType        string `json:"query_type,omitempty"`
	QueryArgument    string `json:"query_argument,omitempty"`
	Retry            int    `json:"retry,omitempty"`
	SetCDBit         bool   `json:"set_cd_bit,omitempty"`
	SetDOBit         bool   `json:"set_do_bit,omitempty"`
	SetNSIDBit       bool   `json:"set_nsid_bit,omitempty"`
	SetRDBit         bool   `json:"set_rd_bit,omitempty"`
	UDPPayloadSize   int    `json:"udp_payload_size,omitempty"`
	UseProbeResolver bool   `json:"use_probe_resolver"`

	// traceroute parameters
	DestinationOptionSize int  `json:"destination_option_size,omitempty"`
	DontFragment          bool `json:"dont_fragment,omitempty"`
	DuplicateTimeout      int  `json:"duplicate_timeout,omitempty"`
	FirstHop              int  `json:"first_hop,omitempty"`
	HopByHopOptionSize    int  `json:"hop_by_hop_option_size,omitempty"`
	MaxHops               int  `json:"max_hops,omitempty"`
	Paris                 int  `json:"paris,omitempty"`

	// http parameters
	ExtendedTiming     bool   `json:"extended_timing,omitempty"`
	HeaderBytes        int    `json:"header_bytes,omitempty"`
	Method             string `json:"method,omitempty"`
	MoreExtendedTiming bool   `json:"more_extended_timing,omitempty"`
	Path               string `json:"path,omitempty"`
	QueryOptions       string `json:"query_options,omitempty"`
	UserAgent          string `json:"user_agent,omitempty"`
	Version            string `json:"version,omitempty"`

	// sslcert & traceroute & http parameters
	Port int `json:"port,omitempty"`

	// ping & traceroute parameters
	Size int `json:"size,omitempty"`

	// wifi parameters
	AnonymousIdentity string `json:"anonymous_identity,omitempty"`
	Cert              string `json:"cert,omitempty"`
	EAP               string `json:"eap,omitempty"`
}

Definition is used to create measurements

type Grant

type Grant struct {
	Permission string `json:"permission"`
	Target     struct {
		Type string `json:"type"`
		ID   string `json:"id"`
	} `json:"target"`
}

Grant is the permission(s) associated with a key

type Key

type Key struct {
	UUID      string `json:"uuid"`
	ValidFrom string `json:"valid_from"`
	ValidTo   string `json:"valid_to"`
	Enabled   bool
	IsActive  bool    `json:"is_active"`
	CreatedAt string  `json:"created_at"`
	Label     string  `json:"label"`
	Grants    []Grant `json:"grants"`
	Type      string  `json:"type"`
}

Key is holding the API key parameters

type Measurement

type Measurement struct {
	Af                    int                    `json:"af"`
	CreationTime          int                    `json:"creation_time"`
	Description           string                 `json:"description"`
	DestinationOptionSize interface{}            `json:"destination_option_size"`
	DontFragment          interface{}            `json:"dont_fragment"`
	DuplicateTimeout      interface{}            `json:"duplicate_timeout"`
	FirstHop              int                    `json:"first_hop"`
	Group                 string                 `json:"group"`
	GroupID               int                    `json:"group_id"`
	HopByHopOptionSize    interface{}            `json:"hop_by_hop_option_size"`
	ID                    int                    `json:"id"`
	InWifiGroup           bool                   `json:"in_wifi_group"`
	Interval              int                    `json:"interval"`
	IsAllScheduled        bool                   `json:"is_all_scheduled"`
	IsOneoff              bool                   `json:"is_oneoff"`
	IsPublic              bool                   `json:"is_public"`
	MaxHops               int                    `json:"max_hops"`
	PacketInterval        interface{}            `json:"packet_interval"`
	Packets               int                    `json:"packets"`
	Paris                 int                    `json:"paris"`
	ParticipantCount      int                    `json:"participant_count"`
	ParticipationRequests []ParticipationRequest `json:"participation_requests"`
	Port                  interface{}            `json:"port"`
	ProbesRequested       int                    `json:"probes_requested"`
	ProbesScheduled       int                    `json:"probes_scheduled"`
	Protocol              string                 `json:"protocol"`
	ResolveOnProbe        bool                   `json:"resolve_on_probe"`
	ResolvedIPs           []string               `json:"resolved_ips"`
	ResponseTimeout       int                    `json:"response_timeout"`
	Result                string                 `json:"result"`
	Size                  int                    `json:"size"`
	Spread                interface{}            `json:"spread"`
	StartTime             int                    `json:"start_time"`
	Status                struct {
		ID   int    `json:"id"`
		Name string `json:"name"`
	} `json:"status"`
	StopTime  int    `json:"stop_time"`
	Target    string `json:"target"`
	TargetASN int    `json:"target_asn"`
	TargetIP  string `json:"target_ip"`
	Type      string `json:"type"`
}

Measurement is what we are working with

type MeasurementRequest

type MeasurementRequest struct {
	// see below for definition
	Definitions []Definition `json:"definitions"`

	// requested set of probes
	Probes []ProbeSet `json:"probes"`
	//
	BillTo       int  `json:"bill_to,omitempty"`
	IsOneoff     bool `json:"is_oneoff,omitempty"`
	SkipDNSCheck bool `json:"skip_dns_check,omitempty"`
	Times        int  `json:"times,omitempty"`
	StartTime    int  `json:"start_time,omitempty"`
	StopTime     int  `json:"stop_time,omitempty"`
}

MeasurementRequest contains the different measurement to create/view

func (*MeasurementRequest) AddDefinition

func (m *MeasurementRequest) AddDefinition(fields map[string]string) *MeasurementRequest

AddDefinition create a new MeasurementRequest and fills some fields

type MeasurementResp

type MeasurementResp struct {
	Measurements []int
}

MeasurementResp contains all the results of the measurements

type ParticipationRequest

type ParticipationRequest struct {
	Action        string `json:"action"`
	CreatedAt     int    `json:"created_at,omitempty"`
	ID            int    `json:"id,omitempty"`
	Self          string `json:"self,omitempty"`
	Measurement   string `json:"measurement,omitempty"`
	MeasurementID int    `json:"measurement_id,omitempty"`
	Requested     int    `json:"requested,omitempty"`
	Type          string `json:"type,omitempty"`
	Value         string `json:"value,omitempty"`
	Logs          string `json:"logs,omitempty"`
}

ParticipationRequest allow you to add or remove probes from a measurement that was already created

type Probe

type Probe struct {
	AddressV4      string `json:"address_v4"`
	AddressV6      string `json:"address_v6"`
	AsnV4          int    `json:"asn_v4"`
	AsnV6          int    `json:"asn_v6"`
	CountryCode    string `json:"country_code"`
	Description    string `json:"description"`
	FirstConnected int    `json:"first_connected"`
	Geometry       struct {
		Type        string    `json:"type"`
		Coordinates []float64 `json:"coordinates"`
	} `json:"geometry"`
	ID            int    `json:"id"`
	IsAnchor      bool   `json:"is_anchor"`
	IsPublic      bool   `json:"is_public"`
	LastConnected int    `json:"last_connected"`
	PrefixV4      string `json:"prefix_v4"`
	PrefixV6      string `json:"prefix_v6"`
	Status        struct {
		Since string `json:"since"`
		ID    int    `json:"id"`
		Name  string `json:"name"`
	} `json:"status"`
	StatusSince int `json:"status_since"`
	Tags        []struct {
		Name string `json:"name"`
		Slug string `json:"slug"`
	} `json:"tags"`
	Type string `json:"type"`
}

Probe is holding probe's data

type ProbeSet

type ProbeSet struct {
	Requested   int    `json:"requested"` // number of probes
	Type        string `json:"type"`      // area, country, prefix, asn, probes, msm
	Value       string `json:"value"`     // can be numeric or string
	TagsInclude string `json:"tags_include,omitempty"`
	TagsExclude string `json:"tags_exclude,omitempty"`
}

ProbeSet is a set of probes obviously

func NewProbeSet

func NewProbeSet(howmany int, settype, value string, tags string) (ps ProbeSet)

NewProbeSet create a set of probes for later requests

Directories

Path Synopsis
cmd
atlas
This package is just a collection of use-case for the various aspects of the RIPE API.
This package is just a collection of use-case for the various aspects of the RIPE API.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL