runn

package module
v0.120.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 4, 2024 License: MIT Imports: 129 Imported by: 6

README

runn

build Coverage Code to Test Ratio Test Execution Time

runn ( means "Run N". is pronounced /rʌ́n én/. ) is a package/tool for running operations following a scenario.

Key features of runn are:

  • As a tool for scenario based testing.
  • As a test helper package for the Go language.
  • As a tool for workflow automation.
  • Support HTTP request, gRPC request, DB query, Chrome DevTools Protocol, and SSH/Local command execution
  • OpenAPI Document-like syntax for HTTP request testing.
  • Single binary = CI-Friendly.

Online book

Quickstart

You can use the runn new command to quickly start creating scenarios (runbooks).

🚀 Create and run scenario using curl or grpcurl commands:

docs/runn.svg

Command details 
$ curl https://httpbin.org/json -H "accept: application/json"
{
  "slideshow": {
    "author": "Yours Truly",
    "date": "date of publication",
    "slides": [
      {
        "title": "Wake up to WonderWidgets!",
        "type": "all"
      },
      {
        "items": [
          "Why <em>WonderWidgets</em> are great",
          "Who <em>buys</em> WonderWidgets"
        ],
        "title": "Overview",
        "type": "all"
      }
    ],
    "title": "Sample Slide Show"
  }
}
$ runn new --and-run --desc 'httpbin.org GET' --out http.yml -- curl https://httpbin.org/json -H "accept: application/json"
$ grpcurl -d '{"greeting": "alice"}' grpcb.in:9001 hello.HelloService/SayHello
{
  "reply": "hello alice"
}
$ runn new --and-run --desc 'grpcb.in Call' --out grpc.yml -- grpcurl -d '{"greeting": "alice"}' grpcb.in:9001 hello.HelloService/SayHello
$ runn list *.yml
  Desc             Path      If
---------------------------------
  grpcb.in Call    grpc.yml
  httpbin.org GET  http.yml
$ runn run *.yml
..

2 scenarios, 0 skipped, 0 failures

🚀 Create scenario using access log:

docs/runn_axslog.svg

Command details 
$ cat access_log
183.87.255.54 - - [18/May/2019:05:37:09 +0200] "GET /?post=%3script%3ealert(1); HTTP/1.0" 200 42433
62.109.16.162 - - [18/May/2019:05:37:12 +0200] "GET /core/files/js/editor.js/?form=\xeb\x2a\x5e\x89\x76\x08\xc6\x46\x07\x00\xc7\x46\x0c\x00\x00\x00\x80\xe8\xdc\xff\xff\xff/bin/sh HTTP/1.0" 200 81956
87.251.81.179 - - [18/May/2019:05:37:13 +0200] "GET /login.php/?user=admin&amount=100000 HTTP/1.0" 400 4797
103.36.79.144 - - [18/May/2019:05:37:14 +0200] "GET /authorize.php/.well-known/assetlinks.json HTTP/1.0" 200 9436
$ cat access_log| runn new --out axslog.yml
$ cat axslog.yml| yq
desc: Generated by `runn new`
runners:
  req: https://dummy.example.com
steps:
  - req:
      /?post=%3script%3ealert(1);:
        get:
          body: null
  - req:
      /core/files/js/editor.js/?form=xebx2ax5ex89x76x08xc6x46x07x00xc7x46x0cx00x00x00x80xe8xdcxffxffxff/bin/sh:
        get:
          body: null
  - req:
      /login.php/?user=admin&amount=100000:
        get:
          body: null
  - req:
      /authorize.php/.well-known/assetlinks.json:
        get:
          body: null
$

Usage

runn can run a multi-step scenario following a runbook written in YAML format.

As a tool for scenario based testing / As a tool for automation.

runn can run one or more runbooks as a CLI tool.

$ runn list path/to/**/*.yml
  id:      desc:             if:       steps:  path
-------------------------------------------------------------------------
  a1b7b02  Only if included  included       2  p/t/only_if_included.yml
  85ccd5f  List projects.                   4  p/t/p/list.yml
  47d7ef7  List users.                      3  p/t/u/list.yml
  97f9884  Login                            2  p/t/u/login.yml
  2249d1b  Logout                           3  p/t/u/logout.yml
$ runn run path/to/**/*.yml
S....

5 scenarios, 1 skipped, 0 failures

As a test helper package for the Go language.

runn can also behave as a test helper for the Go language.

Run N runbooks using httptest.Server and sql.DB
func TestRouter(t *testing.T) {
	ctx := context.Background()
	dsn := "username:password@tcp(localhost:3306)/testdb"
	db, err := sql.Open("mysql", dsn)
	if err != nil {
		log.Fatal(err)
	}
	dbr, err := sql.Open("mysql", dsn)
	if err != nil {
		log.Fatal(err)
	}
	ts := httptest.NewServer(NewRouter(db))
	t.Cleanup(func() {
		ts.Close()
		db.Close()
		dbr.Close()
	})
	opts := []runn.Option{
		runn.T(t),
		runn.Runner("req", ts.URL),
		runn.DBRunner("db", dbr),
	}
	o, err := runn.Load("testdata/books/**/*.yml", opts...)
	if err != nil {
		t.Fatal(err)
	}
	if err := o.RunN(ctx); err != nil {
		t.Fatal(err)
	}
}
Run single runbook using httptest.Server and sql.DB
func TestRouter(t *testing.T) {
	ctx := context.Background()
	dsn := "username:password@tcp(localhost:3306)/testdb"
	db, err := sql.Open("mysql", dsn)
	if err != nil {
		log.Fatal(err)
	}
	dbr, err := sql.Open("mysql", dsn)
	if err != nil {
		log.Fatal(err)
	}
	ts := httptest.NewServer(NewRouter(db))
	t.Cleanup(func() {
		ts.Close()
		db.Close()
		dbr.Close()
	})
	opts := []runn.Option{
		runn.T(t),
		runn.Book("testdata/books/login.yml"),
		runn.Runner("req", ts.URL),
		runn.DBRunner("db", dbr),
	}
	o, err := runn.New(opts...)
	if err != nil {
		t.Fatal(err)
	}
	if err := o.Run(ctx); err != nil {
		t.Fatal(err)
	}
}
Run N runbooks using grpc.Server
func TestServer(t *testing.T) {
	addr := "127.0.0.1:8080"
	l, err := net.Listen("tcp", addr)
	if err != nil {
		t.Fatal(err)
	}
	ts := grpc.NewServer()
	myapppb.RegisterMyappServiceServer(s, NewMyappServer())
	reflection.Register(s)
	go func() {
		ts.Serve(l)
	}()
	t.Cleanup(func() {
		ts.GracefulStop()
	})
	opts := []runn.Option{
		runn.T(t),
		runn.Runner("greq", fmt.Sprintf("grpc://%s", addr),
	}
	o, err := runn.Load("testdata/books/**/*.yml", opts...)
	if err != nil {
		t.Fatal(err)
	}
	if err := o.RunN(ctx); err != nil {
		t.Fatal(err)
	}
}
Run N runbooks with http.Handler and sql.DB
func TestRouter(t *testing.T) {
	ctx := context.Background()
	dsn := "username:password@tcp(localhost:3306)/testdb"
	db, err := sql.Open("mysql", dsn)
	if err != nil {
		log.Fatal(err)
	}
	dbr, err := sql.Open("mysql", dsn)
	if err != nil {
		log.Fatal(err)
	}
	t.Cleanup(func() {
		db.Close()
		dbr.Close()
	})
	opts := []runn.Option{
		runn.T(t),
		runn.HTTPRunnerWithHandler("req", NewRouter(db)),
		runn.DBRunner("db", dbr),
	}
	o, err := runn.Load("testdata/books/**/*.yml", opts...)
	if err != nil {
		t.Fatal(err)
	}
	if err := o.RunN(ctx); err != nil {
		t.Fatal(err)
	}
}

Examples

See the details

Runbook ( runn scenario file )

The runbook file has the following format.

step: section accepts list or ordered map.

List:

desc: Login and get projects.
runners:
  req: https://example.com/api/v1
  db: mysql://root:mypass@localhost:3306/testdb
vars:
  username: alice
  password: ${TEST_PASS}
steps:
  -
    db:
      query: SELECT * FROM users WHERE name = '{{ vars.username }}'
  -
    req:
      /login:
        post:
          body:
            application/json:
              email: "{{ steps[0].rows[0].email }}"
              password: "{{ vars.password }}"
    test: steps[1].res.status == 200
  -
    req:
      /projects:
        get:
          headers:
            Authorization: "token {{ steps[1].res.body.session_token }}"
          body: null
    test: steps[2].res.status == 200
  -
    test: len(steps[2].res.body.projects) > 0

Map:

desc: Login and get projects.
runners:
  req: https://example.com/api/v1
  db: mysql://root:mypass@localhost:3306/testdb
vars:
  username: alice
  password: ${TEST_PASS}
steps:
  find_user:
    db:
      query: SELECT * FROM users WHERE name = '{{ vars.username }}'
  login:
    req:
      /login:
        post:
          body:
            application/json:
              email: "{{ steps.find_user.rows[0].email }}"
              password: "{{ vars.password }}"
    test: steps.login.res.status == 200
  list_projects:
    req:
      /projects:
        get:
          headers:
            Authorization: "token {{ steps.login.res.body.session_token }}"
          body: null
    test: steps.list_projects.res.status == 200
  count_projects:
    test: len(steps.list_projects.res.body.projects) > 0

List:

color

Map:

color

desc:

Description of runbook.

desc: Login and get projects.
runners:
  req: https://example.com/api/v1
vars:
  username: alice
steps:
[...]

labels:

Labels of runbook.

desc: Login
runners:
  req: https://example.com/api/v1
labels:
  - users
  - auth
steps:
[...]

Runbooks to be run can be filtered by labels.

$ runn run path/to/**/*.yml --label users --label projects

$ runn run path/to/**/*.yml --label 'users and auth'

runners:

Mapping of runners that run steps: of runbook.

In the steps: section, call the runner with the key specified in the runners: section.

Built-in runners such as test runner do not need to be specified in this section.

runners:
  ghapi: ${GITHUB_API_ENDPOINT}
  idp: https://auth.example.com
  db: my:dbuser:${DB_PASS}@hostname:3306/dbname

In the example, each runner can be called by ghapi:, idp: or db: in steps:.

hostRules:

Allows remapping any request hostname to another hostname, IP address in HTTP/gRPC/DB/CDP/SSH runners.

hostRules:
  example.com: 127.0.0.1:8080
  '*.example.test': 192.168.0.16

vars:

Mapping of variables available in the steps: of runbook.

vars:
  username: alice@example.com
  token: ${SECRET_TOKEN}

In the example, each variable can be used in {{ vars.username }} or {{ vars.token }} in steps:.

debug:

Enable debug output for runn.

debug: true

interval:

Interval between steps.

interval: 1

if:

Conditions for skip all steps.

if: included # Run steps only if included

skipTest:

Skip all test: sections

skipTest: true

force:

Force all steps to run.

force: true

trace:

Add tokens for tracing to headers and queries by default.

Currently, HTTP runner, gRPC runner and DB runner are supported.

trace: true

loop:

Loop setting for runbook.

Simple loop runbook
loop: 10
steps:
  [...]

or

loop:
  count: 10
steps:
  [...]
Retry runbook

It can be used as a retry mechanism by setting a condition in the until: section.

If the condition of until: is met, the loop is broken without waiting for the number of count: to be run.

Also, if the run of the number of count: completes but does not satisfy the condition of until:, then the step is considered to be failed.

loop:
  count: 10
  until: 'outcome == "success"' # until the runbook outcome is successful.
  minInterval: 0.5 # sec
  maxInterval: 10  # sec
  # jitter: 0.0
  # interval: 5
  # multiplier: 1.5
steps:
  waitingroom:
    req:
      /cart/in:
        post:
          body:
[...]
  • outcome ... the result of a completed (success, failure, skipped).

concurrency:

Runbooks with the same key are assured of a single run at the same time.

concurrency: use-shared-db

or

concurrency:
  - use-shared-db
  - use-shared-api

needs:

It is possible to identify runbooks that must be pre-run.

needs:
  prebook: path/to/prebook.yml
  prebook2: path/to/prebook2.yml

Values bound by the bind runner can be referenced by needs.<key>. *.

steps:

Steps to run in runbook.

The steps are invoked in order from top to bottom.

Any return values are recorded for each step.

When steps: is array, recorded values can be retrieved with {{ steps[*].* }}.

steps:
  -
    db:
      query: SELECT * FROM users WHERE name = '{{ vars.username }}'
  -
    req:
      /users/{{ steps[0].rows[0].id }}:
        get:
          body: null

When steps: is map, recorded values can be retrieved with {{ steps.<key>.* }}.

steps:
  find_user:
    db:
      query: SELECT * FROM users WHERE name = '{{ vars.username }}'
  user_info:
    req:
      /users/{{ steps.find_user.rows[0].id }}:
        get:
          body: null

steps[*].desc: steps.<key>.desc:

Description of step.

steps:
  -
    desc: Login
    req:
      /login:
        post:
          body:
[...]

steps[*].if: steps.<key>.if:

Conditions for skip step.

steps:
  login:
    if: 'len(vars.token) == 0' # Run step only if var.token is not set
    req:
      /login:
        post:
          body:
[...]

steps[*].loop: steps.<key>.loop:

Loop settings for steps.

Simple loop step
steps:
  multicartin:
    loop: 10
    req:
      /cart/in:
        post:
          body:
            application/json:
              product_id: "{{ i }}" # The loop count (0..9) is assigned to `i`.
[...]

or

steps:
  multicartin:
    loop:
      count: 10
    req:
      /cart/in:
        post:
          body:
            application/json:
              product_id: "{{ i }}" # The loop count (0..9) is assigned to `i`.
[...]
Retry step

It can be used as a retry mechanism by setting a condition in the until: section.

If the condition of until: is met, the loop is broken without waiting for the number of count: to be run.

Also, if the run of the number of count: completes but does not satisfy the condition of until:, then the step is considered to be failed.

steps:
  waitingroom:
    loop:
      count: 10
      until: 'steps.waitingroom.res.status == "201"' # Store values of latest loop
      minInterval: 500ms
      maxInterval: 10 # sec
      # jitter: 0.0
      # interval: 5
      # multiplier: 1.5
    req:
      /cart/in:
        post:
          body:
[...]

Variables to be stored

runn can use variables and functions when running step.

Also, after step runs, HTTP responses, DB query results, etc. are automatically stored in variables.

The values are stored in predefined variables.

Variable name Description
vars Values set in the vars: section
steps Return values for each step
i Loop index (only in loop: section)
env Environment variables
current Return values of current step
previous Return values of previous step
parent Variables of parent runbook (only included)

Runner

HTTP Runner: Do HTTP request

Use https:// or http:// scheme to specify HTTP Runner.

When the step is invoked, it sends the specified HTTP Request and records the response.

runners:
  req: https://example.com
steps:
  -
    desc: Post /users                     # description of step
    req:                                  # key to identify the runner. In this case, it is HTTP Runner.
      /users:                             # path of http request
        post:                             # method of http request
          headers:                        # headers of http request
            Authorization: 'Bearer xxxxx'
          body:                           # body of http request
            application/json:             # Content-Type specification. In this case, it is "Content-Type: application/json"
              username: alice
              password: passw0rd
          trace: false                    # add `X-Runn-Trace` header to HTTP request for tracing
    test: |                               # test for current step
      current.res.status == 201

See testdata/book/http.yml and testdata/book/http_multipart.yml.

Structure of recorded responses

The following response

HTTP/1.1 200 OK
Content-Length: 29
Content-Type: application/json
Date: Wed, 07 Sep 2022 06:28:20 GMT
Set-Cookie: cookie-name=cookie-value

{"data":{"username":"alice"}}

is recorded with the following structure.

[`step key` or `current` or `previous`]:
  res:
    status: 200                              # current.res.status
    headers:
      Content-Length:
        - '29'                               # current.res.headers["Content-Length"][0]
      Content-Type:
        - 'application/json'                 # current.res.headers["Content-Type"][0]
      Date:
        - 'Wed, 07 Sep 2022 06:28:20 GMT'    # current.res.headers["Date"][0]
      Set-Cookie:
        - 'cookie-name=cookie-value'         # current.res.headers["Set-Cookie"][0]
    cookies:
      cookie-name: *http.Cookie              # current.res.cookies["cookie-name"].Value
    body:
      data:
        username: 'alice'                    # current.res.body.data.username
    rawBody: '{"data":{"username":"alice"}}' # current.res.rawBody
Do not follow redirect

The HTTP Runner interprets HTTP responses and automatically redirects. To disable this, set notFollowRedirect to true.

runners:
  req:
    endpoint: https://example.com
    notFollowRedirect: true

The HTTP Runner automatically saves cookies by interpreting HTTP responses. To enable cookie sending during requests, set useCookie to true.

runners:
  req:
    endpoint: https://example.com
    useCookie: true

See testdata/book/cookie.yml and testdata/book/cookie_in_requests_automatically.yml.

Validation of HTTP request and HTTP response

HTTP requests sent by runn and their HTTP responses can be validated.

OpenAPI v3:

runners:
  myapi:
    endpoint: https://api.example.com
    openapi3: path/to/openapi.yaml
    # skipValidateRequest: false
    # skipValidateResponse: false
    # skipCircularReferenceCheck: false # skip checking circular references in OpenAPIv3 document.
Custom CA and Certificates
runners:
  myapi:
    endpoint: https://api.github.com
    cacert: path/to/cacert.pem
    cert: path/to/cert.pem
    key: path/to/key.pem
    # skipVerify: false
Add X-Runn-Trace header to HTTP request for tracing
runners:
  myapi:
    endpoint: https://api.github.com
    trace: true

gRPC Runner: Do gRPC request

Use grpc:// scheme to specify gRPC Runner.

When the step is invoked, it sends the specified gRPC Request and records the response.

runners:
  greq: grpc://grpc.example.com:80
steps:
  -
    desc: Request using Unary RPC                     # description of step
    greq:                                             # key to identify the runner. In this case, it is gRPC Runner.
      grpctest.GrpcTestService/Hello:                 # package.Service/Method of rpc
        headers:                                      # headers of rpc
          authentication: tokenhello
        message:                                      # message of rpc
          name: alice
          num: 3
          request_time: 2022-06-25T05:24:43.861872Z
        trace: false                                  # add `x-runn-trace` header to gRPC request for tracing
  -
    desc: Request using Server streaming RPC
    greq:
      grpctest.GrpcTestService/ListHello:
        headers:
          authentication: tokenlisthello
        message:
          name: bob
          num: 4
          request_time: 2022-06-25T05:24:43.861872Z
        timeout: 3sec                                 # timeout for rpc
    test: |
      steps.server_streaming.res.status == 0 && len(steps.server_streaming.res.messages) > 0
  -
    desc: Request using Client streaming RPC
    greq:
      grpctest.GrpcTestService/MultiHello:
        headers:
          authentication: tokenmultihello
        messages:                                     # messages of rpc
          -
            name: alice
            num: 5
            request_time: 2022-06-25T05:24:43.861872Z
          -
            name: bob
            num: 6
            request_time: 2022-06-25T05:24:43.861872Z

runners:
  greq:
    addr: grpc.example.com:8080
    tls: true
    cacert: path/to/cacert.pem
    cert: path/to/cert.pem
    key: path/to/key.pem
    # skipVerify: false
    # importPaths:
    #   - protobuf/proto
    # protos:
    #   - general/health.proto
    #   - myapp/**/*.proto

See testdata/book/grpc.yml.

Structure of recorded responses

The following response

message HelloResponse {
  string message = 1;

  int32 num = 2;

  google.protobuf.Timestamp create_time = 3;
}

{"create_time":"2022-06-25T05:24:43.861872Z","message":"hello","num":32}

and headers

content-type: ["application/grpc"]
hello: ["this is header"]

and trailers

hello: ["this is trailer"]

are recorded with the following structure.

[`step key` or `current` or `previous`]:
  res:
    status: 0                                      # current.res.status
    headers:
      content-type:
        - 'application/grpc'                       # current.res.headers[0].content-type
      hello:
        - 'this is header'                         # current.res.headers[0].hello
    trailers:
      hello:
        - 'this is trailer'                        # current.res.trailers[0].hello
    message:
      create_time: '2022-06-25T05:24:43.861872Z'   # current.res.message.create_time
      message: 'hello'                             # current.res.message.message
      num: 32                                      # current.res.message.num
    messages:
      -
        create_time: '2022-06-25T05:24:43.861872Z' # current.res.messages[0].create_time
        message: 'hello'                           # current.res.messages[0].message
        num: 32                                    # current.res.messages[0].num
Add x-runn-trace header to gRPC request for tracing
runners:
  greq:
    addr: grpc.example.com:8080
    trace: true
Buf

gRPC Runner supports Buf ecosystem includes Buf Schema Registry.

It can use the buf modules ( and protos ) it depends on.

runners:
  greq:
    addr: grpc.example.com:8080
    bufDirs:
      - path/to # Set buf directories for registering buf modules and protos

runners:
  greq:
    addr: grpc.example.com:8080
    bufLocks:
      - path/to/buf.lock # Register buf modules using buf.lock

runners:
  greq:
    addr: grpc.example.com:8080
    bufConfigs:
      - path/to/buf.yaml # Register buf modules using buf.yaml

runners:
  greq:
    addr: grpc.example.com:8080
    bufModules:
        - buf.build/owner/repository
        - buf.build/owner2/repository2

DB Runner: Query a database

Use dsn (Data Source Name) to specify DB Runner.

When step is invoked, it executes the specified query the database.

runners:
  db: postgres://dbuser:dbpass@hostname:5432/dbname
steps:
  -
    desc: Select users            # description of step
    db:                           # key to identify the runner. In this case, it is DB Runner.
      query: SELECT * FROM users; # query to execute
      trace: false                # add comment with trace token to query for tracing

See testdata/book/db.yml.

Structure of recorded responses

If the query is a SELECT clause, it records the selected rows,

[`step key` or `current` or `previous`]:
  rows:
    -
      id: 1                           # current.rows[0].id
      username: 'alice'               # current.rows[0].username
      password: 'passw0rd'            # current.rows[0].password
      email: 'alice@example.com'      # current.rows[0].email
      created: '2017-12-05T00:00:00Z' # current.rows[0].created
    -
      id: 2                           # current.rows[1].id
      username: 'bob'                 # current.rows[1].username
      password: 'passw0rd'            # current.rows[1].password
      email: 'bob@example.com'        # current.rows[1].email
      created: '2022-02-22T00:00:00Z' # current.rows[1].created

otherwise it records last_insert_id and rows_affected .

[`step key` or `current` or `previous`]:
  last_insert_id: 3 # current.last_insert_id
  rows_affected: 1  # current.rows_affected
Add comment with trace token to query for tracing
runners:
  db:
    dsn: mysql://dbuser:dbpass@hostname:3306/dbname
    trace: true
Support Databases

PostgreSQL:

runners:
  mydb: postgres://dbuser:dbpass@hostname:5432/dbname

runners:
  db: pg://dbuser:dbpass@hostname:5432/dbname

MySQL:

runners:
  testdb: mysql://dbuser:dbpass@hostname:3306/dbname

runners:
  db: my://dbuser:dbpass@hostname:3306/dbname

SQLite3:

runners:
  db: sqlite:///path/to/dbname.db

runners:
  local: sq://dbname.db

Cloud Spanner:

runners:
  testdb: spanner://test-project/test-instance/test-database

runners:
  db: sp://test-project/test-instance/test-database

CDP Runner: Control browser using Chrome DevTools Protocol (CDP)

Use cdp:// or chrome:// scheme to specify CDP Runner.

When the step is invoked, it controls browser via Chrome DevTools Protocol.

runners:
  cc: chrome://new
steps:
  -
    desc: Navigate, click and get h1 using CDP  # description of step
    cc:                                         # key to identify the runner. In this case, it is CDP Runner.
      actions:                                  # actions to control browser
        - navigate: https://pkg.go.dev/time
        - click: 'body > header > div.go-Header-inner > nav > div > ul > li:nth-child(2) > a'
        - waitVisible: 'body > footer'
        - text: 'h1'
  -
    test: |
      previous.text == 'Install the latest version of Go'

See testdata/book/cdp.yml.

Functions for action to control browser

attributes (aliases: getAttributes, attrs, getAttrs)

Get the element attributes for the first element node matching the selector (sel).

actions:
  - attributes:
      sel: 'h1'
# record to current.attrs:

or

actions:
  - attributes: 'h1'

click

Send a mouse click event to the first element node matching the selector (sel).

actions:
  - click:
      sel: 'nav > div > a'

or

actions:
  - click: 'nav > div > a'

doubleClick

Send a mouse double click event to the first element node matching the selector (sel).

actions:
  - doubleClick:
      sel: 'nav > div > li'

or

actions:
  - doubleClick: 'nav > div > li'

evaluate (aliases: eval)

Evaluate the Javascript expression (expr).

actions:
  - evaluate:
      expr: 'document.querySelector("h1").textContent = "hello"'

or

actions:
  - evaluate: 'document.querySelector("h1").textContent = "hello"'

fullHTML (aliases: getFullHTML, getHTML, html)

Get the full html of page.

actions:
  - fullHTML
# record to current.html:

innerHTML (aliases: getInnerHTML)

Get the inner html of the first element node matching the selector (sel).

actions:
  - innerHTML:
      sel: 'h1'
# record to current.html:

or

actions:
  - innerHTML: 'h1'

latestTab (aliases: latestTarget)

Change current frame to latest tab.

actions:
  - latestTab

localStorage (aliases: getLocalStorage)

Get localStorage items.

actions:
  - localStorage:
      origin: 'https://github.com'
# record to current.items:

or

actions:
  - localStorage: 'https://github.com'

location (aliases: getLocation)

Get the document location.

actions:
  - location
# record to current.url:

navigate

Navigate the current frame to url page.

actions:
  - navigate:
      url: 'https://pkg.go.dev/time'

or

actions:
  - navigate: 'https://pkg.go.dev/time'

outerHTML (aliases: getOuterHTML)

Get the outer html of the first element node matching the selector (sel).

actions:
  - outerHTML:
      sel: 'h1'
# record to current.html:

or

actions:
  - outerHTML: 'h1'

screenshot (aliases: getScreenshot)

Take a full screenshot of the entire browser viewport.

actions:
  - screenshot
# record to current.png:

scroll (aliases: scrollIntoView)

Scroll the window to the first element node matching the selector (sel).

actions:
  - scroll:
      sel: 'body > footer'

or

actions:
  - scroll: 'body > footer'

sendKeys

Send keys (value) to the first element node matching the selector (sel).

actions:
  - sendKeys:
      sel: 'input[name=username]'
      value: 'k1lowxb@gmail.com'

sessionStorage (aliases: getSessionStorage)

Get sessionStorage items.

actions:
  - sessionStorage:
      origin: 'https://github.com'
# record to current.items:

or

actions:
  - sessionStorage: 'https://github.com'

setUploadFile (aliases: setUpload)

Set upload file (path) to the first element node matching the selector (sel).

actions:
  - setUploadFile:
      sel: 'input[name=avator]'
      path: '/path/to/image.png'

setUserAgent (aliases: setUA, ua, userAgent)

Set the default User-Agent

actions:
  - setUserAgent:
      userAgent: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36'

or

actions:
  - setUserAgent: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36'

submit

Submit the parent form of the first element node matching the selector (sel).

actions:
  - submit:
      sel: 'form.login'

or

actions:
  - submit: 'form.login'

text (aliases: getText)

Get the visible text of the first element node matching the selector (sel).

actions:
  - text:
      sel: 'h1'
# record to current.text:

or

actions:
  - text: 'h1'

textContent (aliases: getTextContent)

Get the text content of the first element node matching the selector (sel).

actions:
  - textContent:
      sel: 'h1'
# record to current.text:

or

actions:
  - textContent: 'h1'

title (aliases: getTitle)

Get the document title.

actions:
  - title
# record to current.title:

value (aliases: getValue)

Get the Javascript value field of the first element node matching the selector (sel).

actions:
  - value:
      sel: 'input[name=address]'
# record to current.value:

or

actions:
  - value: 'input[name=address]'

wait (aliases: sleep)

Wait for the specified time.

actions:
  - wait:
      time: '10sec'

or

actions:
  - wait: '10sec'

waitReady

Wait until the element matching the selector (sel) is ready.

actions:
  - waitReady:
      sel: 'body > footer'

or

actions:
  - waitReady: 'body > footer'

waitVisible

Wait until the element matching the selector (sel) is visible.

actions:
  - waitVisible:
      sel: 'body > footer'

or

actions:
  - waitVisible: 'body > footer'

SSH Runner: execute commands on a remote server connected via SSH

Use ssh:// scheme to specify SSH Runner.

When step is invoked, it executes commands on a remote server connected via SSH.

runners:
  sc: ssh://username@hostname:port
steps:
  -
    desc: 'execute `hostname`' # description of step
    sc:
      command: hostname

runners:
  sc:
    hostname: hostname
    user: username
    port: 22
    # host: myserver
    # sshConfig: path/to/ssh_config
    # keepSession: false
    # localForward: '33306:127.0.0.1:3306'
    # keyboardInteractive:
    #   - match: Username
    #     answer: k1low
    #   - match: OTP
    #     answer: ${MY_OTP}

See testdata/book/sshd.yml.

Structure of recorded responses

The response to the run command is always stdout and stderr.

[`step key` or `current` or `previous`]:
  stdout: 'hello world' # current.stdout
  stderr: ''            # current.stderr

Exec Runner: execute command

Note Exec runner requires run:exec scope to run.

The exec runner is a built-in runner, so there is no need to specify it in the runners: section.

It execute command using command:, stdin:, shell: and background:.

-
  exec:
    command: grep hello
    stdin: '{{ steps[3].res.rawBody }}'

-
  exec:
    command: echo $0
    shell: bash

background: set to true to run the command in the background.

See testdata/book/exec.yml.

Structure of recorded responses

The response to the run command is always stdout, stderr and exit_code.

[`step key` or `current` or `previous`]:
  stdout: 'hello world' # current.stdout
  stderr: ''            # current.stderr
  exit_code: 0          # current.exit_code

Test Runner: test using recorded values

The test runner is a built-in runner, so there is no need to specify it in the runners: section.

It evaluates the conditional expression using the recorded values.

-
  test: steps[3].res.status == 200

The test runner can run in the same steps as the other runners.

Dump Runner: dump recorded values

The dump runner is a built-in runner, so there is no need to specify it in the runners: section.

It dumps the specified recorded values.

-
  dump: steps[4].rows

or

-
  dump:
    expr: steps[4].rows
    out: path/to/dump.out
    disableTrailingNewline: true

The dump runner can run in the same steps as the other runners.

Include Runner: include other runbook

The include runner is a built-in runner, so there is no need to specify it in the runners: section.

Include runner reads and runs the runbook in the specified path.

Recorded values are nested.

-
  include: path/to/get_token.yml

It is also possible to override vars: of included runbook.

-
  include:
    path: path/to/login.yml
    vars:
      username: alice
      password: alicepass
-
  include:
    path: path/to/login.yml
    vars:
      username: bob
      password: bobpass

It is also possible to skip all test: sections in the included runbook.

-
  include:
    path: path/to/signup.yml
    skipTest: true

It is also possible to force all steps in the included runbook to run.

-
  include:
    path: path/to/signup.yml
    force: true

Bind Runner: bind variables

The bind runner is a built-in runner, so there is no need to specify it in the runners: section.

It bind runner binds any values with another key.

  -
    req:
      /users/k1low:
        get:
          body: null
  -
    bind:
      user_id: steps[0].res.body.data.id
  -
    dump: user_id

The bind runner can run in the same steps as the other runners.

Runner Runner: Define runner in the middle of steps.

The runner runner is a built-in runner, so there is no need to specify it in the runners: section.

It defines a runner in the middle of steps.

  -
    runner:
      sc: ssh://username@hostname:port
  -
    sc:
      command: hostname

The runner runner can not run in the same steps as the other runners.

Expression evaluation engine

runn has embedded expr-lang/expr as the evaluation engine for the expression.

See Language Definition.

Additional built-in functions

  • urlencode ... url.QueryEscape
  • bool ... cast.ToBool
  • compare ... Compare two values ( func(x, y any, ignorePaths ...string) bool ). The optional ignorePaths argument is a list of jq syntax path expressions to ignore when comparing two values.
  • diff ... Difference between two values ( func(x, y any, ignorePaths ...string) string ). The optional ignorePaths argument is a list of jq syntax path expressions to ignore when comparing two values.
  • pick ... Returns same map type filtered by given keys left lo.PickByKeys.
  • omit ... Returns same map type filtered by given keys excluded lo.OmitByKeys.
  • merge ... Merges multiple maps from left to right lo.Assign.
  • input ... prompter.Prompt
  • intersect ... Find the intersection of two iterable values ( func(x, y any) any ).
  • secret ... prompter.Password
  • select ... Select from candidates. func(message string, candidates []string, default string) string
  • basename ... filepath.Base
  • time ... Converts the given string or number to time.Time{}.
  • faker.* ... Generate fake data using Faker ).

Option

See https://pkg.go.dev/github.com/k1LoW/runn#Option

Example: Run as a test helper ( func T )

https://pkg.go.dev/github.com/k1LoW/runn#T

o, err := runn.Load("testdata/**/*.yml", runn.T(t))
if err != nil {
	t.Fatal(err)
}
if err := o.RunN(ctx); err != nil {
	t.Fatal(err)
}

Example: Add custom function ( func Func )

https://pkg.go.dev/github.com/k1LoW/runn#Func

desc: Test using GitHub
runners:
  req:
    endpoint: https://github.com
steps:
  -
    req:
      /search?l={{ urlencode('C++') }}&q=runn&type=Repositories:
        get:
          body:
            application/json:
              null
    test: 'steps[0].res.status == 200'

o, err := runn.Load("testdata/**/*.yml", runn.Func("urlencode", url.QueryEscape))
if err != nil {
	t.Fatal(err)
}
if err := o.RunN(ctx); err != nil {
	t.Fatal(err)
}

Scope

runn requires explicit specification of scope for some features.

runn has the following scopes.

Scope Description Default
read:parent Required for reading files above the working directory. false
read:remote Required for reading remote files. false
run:exec Required for running Exec runner. false

To specify scopes, using the --scopes option or the environment variable RUNN_SCOPES.

$ runn run path/to/**/*.yml --scopes read:parent,read:remote

$ env RUNN_SCOPES=read:parent,read:remote runn run path/to/**/*.yml

Also, runn.Scopes can be used in the code

o, err := runn.Load("path/to/**/*.yml", runn.Scopes(runn.ScopeAllowReadParent, runn.ScopeAllowReadRemote))
if err != nil {
	t.Fatal(err)
}
if err := o.RunN(ctx); err != nil {
	t.Fatal(err)
}

To disable scope, can use !read:* instead of read:*

$ runn run path/to/**/*.yml --scopes '!read:parent'

Filter runbooks to be executed by the environment variable RUNN_RUN

Run only runbooks matching the filename "login".

$ env RUNN_RUN=login go test ./... -run TestRouter

Measure elapsed time as profile

opts := []runn.Option{
	runn.T(t),
	runn.Book("testdata/books/login.yml"),
	runn.Profile(true),
}
o, err := runn.New(opts...)
if err != nil {
	t.Fatal(err)
}
if err := o.Run(ctx); err != nil {
	t.Fatal(err)
}
f, err := os.Open("profile.json")
if err != nil {
	t.Fatal(err)
}
if err := o.DumpProfile(f); err != nil {
	t.Fatal(err)
}

or

$ runn run testdata/books/login.yml --profile

The runbook run profile can be read with runn rprof command.

$ runn rprof runn.prof
  runbook[login site](t/b/login.yml)           2995.72ms
    steps[0].req                                747.67ms
    steps[1].req                                185.69ms
    steps[2].req                                192.65ms
    steps[3].req                                188.23ms
    steps[4].req                                569.53ms
    steps[5].req                                299.88ms
    steps[6].test                                 0.14ms
    steps[7].include                            620.88ms
      runbook[include](t/b/login_include.yml)   605.56ms
        steps[0].req                            605.54ms
    steps[8].req                                190.92ms
  [total]                                      2995.84ms

Capture runbook runs

opts := []runn.Option{
	runn.T(t),
	runn.Capture(capture.Runbook("path/to/dir")),
}
o, err := runn.Load("testdata/books/**/*.yml", opts...)
if err != nil {
	t.Fatal(err)
}
if err := o.RunN(ctx); err != nil {
	t.Fatal(err)
}

or

$ runn run path/to/**/*.yml --capture path/to/dir

Load test using runbooks

You can use the runn loadt command for load testing using runbooks.

$ runn loadt --load-concurrent 2 --max-rps 0 path/to/*.yml

Number of runbooks per RunN....: 15
Warm up time (--warm-up).......: 5s
Duration (--duration)..........: 10s
Concurrent (--load-concurrent).: 2
Max RunN per second (--max-rps): 0

Total..........................: 12
Succeeded......................: 12
Failed.........................: 0
Error rate.....................: 0%
RunN per seconds...............: 1.2
Latency .......................: max=1,835.1ms min=1,451.3ms avg=1,627.8ms med=1,619.8ms p(90)=1,741.5ms p(99)=1,788.4ms

It also checks the results of the load test with the --threshold option. If the condition is not met, it returns exit status 1.

$ runn loadt --load-concurrent 2 --max-rps 0 --threshold 'error_rate < 10' path/to/*.yml

Number of runbooks per RunN...: 15
Warm up time (--warm-up)......: 5s
Duration (--duration).........: 10s
Concurrent (--load-concurrent): 2

Total.........................: 13
Succeeded.....................: 12
Failed........................: 1
Error rate....................: 7.6%
RunN per seconds..............: 1.3
Latency ......................: max=1,790.2ms min=95.0ms avg=1,541.4ms med=1,640.4ms p(90)=1,749.7ms p(99)=1,786.5ms

Error: (error_rate < 10) is not true
error_rate < 10
├── error_rate => 14.285714285714285
└── 10 => 10

Variables for threshold

Variable name Type Description
total int Total
succeeded int Succeeded
failed int Failed
error_rate float Error rate
rps float RunN per seconds
max float Latency max (ms)
mid float Latency mid (ms)
min float Latency min (ms)
p90 float Latency p(90) (ms)
p99 float Latency p(99) (ms)
avg float Latency avg (ms)

Install

As a CLI tool

deb:

$ export RUNN_VERSION=X.X.X
$ curl -o runn.deb -L https://github.com/k1LoW/runn/releases/download/v$RUNN_VERSION/runn_$RUNN_VERSION-1_amd64.deb
$ dpkg -i runn.deb

RPM:

$ export RUNN_VERSION=X.X.X
$ yum install https://github.com/k1LoW/runn/releases/download/v$RUNN_VERSION/runn_$RUNN_VERSION-1_amd64.rpm

apk:

$ export RUNN_VERSION=X.X.X
$ curl -o runn.apk -L https://github.com/k1LoW/runn/releases/download/v$RUNN_VERSION/runn_$RUNN_VERSION-1_amd64.apk
$ apk add runn.apk

homebrew tap:

$ brew install k1LoW/tap/runn

aqua:

$ aqua g -i k1LoW/runn

manually:

Download binary from releases page

docker:

$ docker container run -it --rm --name runn -v $PWD:/books ghcr.io/k1low/runn:latest list /books/*.yml

go install:

$ go install github.com/k1LoW/runn/cmd/runn@latest

As a test helper

$ go get github.com/k1LoW/runn

Alternatives

References

License

  • MIT License
    • Include logo as well as source code.
    • Only logo license can be selected CC BY 4.0.
    • Also, if there is no alteration to the logo and it is used for technical information about runn, I would not say anything if the copyright notice is omitted.

Documentation

Index

Constants

View Source 
const (
	MediaTypeApplicationJSON           = "application/json"
	MediaTypeTextPlain                 = "text/plain"
	MediaTypeApplicationFormUrlencoded = "application/x-www-form-urlencoded"
	MediaTypeMultipartFormData         = "multipart/form-data"
	MediaTypeApplicationOctetStream    = "application/octet-stream"
)
View Source 
const (
	ScopeAllowReadParent = "read:parent"
	ScopeAllowReadRemote = "read:remote"
	ScopeAllowRunExec    = "run:exec" //nostyle:repetition
	ScopeDenyReadParent  = "!read:parent"
	ScopeDenyReadRemote  = "!read:remote"
	ScopeDenyRunExec     = "!run:exec" //nostyle:repetition
)
View Source 
const DefaultTraceHeaderName = "X-Runn-Trace"

Variables

View Source 
var (
	AsTestHelper = T
	Runbook      = Book
	RunPart      = RunShard //nostyle:repetition
)
View Source 
var CDPFnMap = map[string]CDPFn{
	"navigate": {
		Desc: "Navigate the current frame to `url` page.",
		Fn:   chromedp.Navigate,
		Args: CDPFnArgs{
			{CDPArgTypeArg, "url", "https://pkg.go.dev/time"},
		},
	},
	"latestTab": {
		Desc: "Change current frame to latest tab.",
		Fn: func() chromedp.Action {

			return nil
		},
		Args:    CDPFnArgs{},
		Aliases: []string{"latestTarget"},
	},
	"click": {
		Desc: "Send a mouse click event to the first element node matching the selector (`sel`).",
		Fn:   chromedp.Click,
		Args: CDPFnArgs{
			{CDPArgTypeArg, "sel", "nav > div > a"},
		},
	},
	"doubleClick": {
		Desc: "Send a mouse double click event to the first element node matching the selector (`sel`).",
		Fn:   chromedp.DoubleClick,
		Args: CDPFnArgs{
			{CDPArgTypeArg, "sel", "nav > div > li"},
		},
	},
	"sendKeys": {
		Desc: "Send keys (`value`) to the first element node matching the selector (`sel`).",
		Fn:   chromedp.SendKeys,
		Args: CDPFnArgs{
			{CDPArgTypeArg, "sel", "input[name=username]"},
			{CDPArgTypeArg, "value", "k1lowxb@gmail.com"},
		},
	},
	"submit": {
		Desc: "Submit the parent form of the first element node matching the selector (`sel`).",
		Fn:   chromedp.Submit,
		Args: CDPFnArgs{
			{CDPArgTypeArg, "sel", "form.login"},
		},
	},
	"scroll": {
		Desc: "Scroll the window to the first element node matching the selector (`sel`).",
		Fn:   chromedp.ScrollIntoView,
		Args: CDPFnArgs{
			{CDPArgTypeArg, "sel", "body > footer"},
		},
		Aliases: []string{"scrollIntoView"},
	},
	"wait": {
		Desc: "Wait for the specified `time`.",
		Fn: func(d string) chromedp.Action {
			return &waitAction{d: d}
		},
		Args: CDPFnArgs{
			{CDPArgTypeArg, "time", "10sec"},
		},
		Aliases: []string{"sleep"},
	},
	"waitReady": {
		Desc: "Wait until the element matching the selector (`sel`) is ready.",
		Fn:   chromedp.WaitReady,
		Args: CDPFnArgs{
			{CDPArgTypeArg, "sel", "body > footer"},
		},
	},
	"waitVisible": {
		Desc: "Wait until the element matching the selector (`sel`) is visible.",
		Fn:   chromedp.WaitVisible,
		Args: CDPFnArgs{
			{CDPArgTypeArg, "sel", "body > footer"},
		},
	},
	"setUserAgent": {
		Desc: "Set the default User-Agent",
		Fn: func(ua string) []chromedp.Action {
			headers := map[string]any{"User-Agent": ua}
			return []chromedp.Action{
				network.Enable(),
				network.SetExtraHTTPHeaders(network.Headers(headers)),
			}
		},
		Args: CDPFnArgs{
			{CDPArgTypeArg, "userAgent", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36"},
		},
		Aliases: []string{"setUA", "ua", "userAgent"},
	},
	"text": {
		Desc: "Get the visible text of the first element node matching the selector (`sel`).",
		Fn:   chromedp.Text,
		Args: CDPFnArgs{
			{CDPArgTypeArg, "sel", "h1"},
			{CDPArgTypeRes, "text", "Install the latest version of Go"},
		},
		Aliases: []string{"getText"},
	},
	"textContent": {
		Desc: "Get the text content of the first element node matching the selector (`sel`).",
		Fn:   chromedp.TextContent,
		Args: CDPFnArgs{
			{CDPArgTypeArg, "sel", "h1"},
			{CDPArgTypeRes, "text", "Install the latest version of Go"},
		},
		Aliases: []string{"getTextContent"},
	},
	"innerHTML": {
		Desc: "Get the inner html of the first element node matching the selector (`sel`).",
		Fn:   chromedp.InnerHTML,
		Args: CDPFnArgs{
			{CDPArgTypeArg, "sel", "h1"},
			{CDPArgTypeRes, "html", "Install the latest version of Go"},
		},
		Aliases: []string{"getInnerHTML"},
	},
	"outerHTML": {
		Desc: "Get the outer html of the first element node matching the selector (`sel`).",
		Fn:   chromedp.OuterHTML,
		Args: CDPFnArgs{
			{CDPArgTypeArg, "sel", "h1"},
			{CDPArgTypeRes, "html", "<h1>Install the latest version of Go</h1>"},
		},
		Aliases: []string{"getOuterHTML"},
	},
	"fullHTML": {
		Desc: "Get the full html of page.",
		Fn: func(html *string) chromedp.Action {
			expr := "new XMLSerializer().serializeToString(document);"
			return chromedp.Evaluate(expr, html)
		},
		Args: CDPFnArgs{
			{CDPArgTypeRes, "html", "<!DOCTYPE html><html><body><h1>hello</h1></body></html>"},
		},
		Aliases: []string{"getFullHTML", "getHTML", "html"},
	},
	"value": {
		Desc: "Get the Javascript value field of the first element node matching the selector (`sel`).",
		Fn:   chromedp.Value,
		Args: CDPFnArgs{
			{CDPArgTypeArg, "sel", "input[name=address]"},
			{CDPArgTypeRes, "value", "Fukuoka"},
		},
		Aliases: []string{"getValue"},
	},
	"setUploadFile": {
		Desc: "Set upload file (`path`) to the first element node matching the selector (`sel`).",
		Fn: func(sel, path string) chromedp.Action {
			abs, err := filepath.Abs(path)
			if err != nil {
				return &errAction{err: err}
			}
			if _, err := os.Stat(abs); err != nil {
				return &errAction{err: err}
			}
			return chromedp.SetUploadFiles(sel, []string{abs})
		},
		Args: CDPFnArgs{
			{CDPArgTypeArg, "sel", "input[name=avator]"},
			{CDPArgTypeArg, "path", "/path/to/image.png"},
		},
		Aliases: []string{"setUpload"},
	},
	"title": {
		Desc: "Get the document `title`.",
		Fn:   chromedp.Title,
		Args: CDPFnArgs{
			{CDPArgTypeRes, "title", "GitHub"},
		},
		Aliases: []string{"getTitle"},
	},
	"location": {
		Desc: "Get the document location.",
		Fn:   chromedp.Location,
		Args: CDPFnArgs{
			{CDPArgTypeRes, "url", "https://github.com"},
		},
		Aliases: []string{"getLocation"},
	},
	"attributes": {
		Desc: "Get the element attributes for the first element node matching the selector (`sel`).",
		Fn:   chromedp.Attributes,
		Args: CDPFnArgs{
			{CDPArgTypeArg, "sel", "h1"},
			{CDPArgTypeRes, "attrs", `{"class": "sr-only"}`},
		},
		Aliases: []string{"getAttributes", "attrs", "getAttrs"},
	},
	"screenshot": {
		Desc: "Take a full screenshot of the entire browser viewport.",
		Fn: func(b *[]byte) chromedp.Action {
			return chromedp.FullScreenshot(b, 100)
		},
		Args: CDPFnArgs{
			{CDPArgTypeRes, "png", "[]byte"},
		},
		Aliases: []string{"getScreenshot"},
	},
	"evaluate": {
		Desc: "Evaluate the Javascript expression (`expr`).",
		Fn: func(expr string) chromedp.Action {

			return chromedp.Evaluate(expr, nil)
		},
		Args: CDPFnArgs{
			{CDPArgTypeArg, "expr", `document.querySelector("h1").textContent = "hello"`},
		},
		Aliases: []string{"eval"},
	},
	"localStorage": {
		Desc: "Get localStorage items.",
		Fn: func(origin string, items *map[string]string) chromedp.Action {
			return chromedp.ActionFunc(func(ctx context.Context) error {
				frameTree, err := page.GetFrameTree().Do(ctx)
				if err != nil {
					return err
				}
				strageKey := domstorage.SerializedStorageKey(frameTree.Frame.SecurityOrigin + "/")
				storageID := &domstorage.StorageID{
					StorageKey:     strageKey,
					IsLocalStorage: true,
				}
				resp, err := domstorage.GetDOMStorageItems(storageID).Do(ctx)
				if err != nil {
					return err
				}
				m := make(map[string]string)
				for _, v := range resp {
					if len(v) != 2 {
						continue
					}
					m[v[0]] = v[1]
				}
				*items = m
				return nil
			})
		},
		Args: CDPFnArgs{
			{CDPArgTypeArg, "origin", "https://github.com"},
			{CDPArgTypeRes, "items", `{"key": "value"}`},
		},
		Aliases: []string{"getLocalStorage"},
	},
	"sessionStorage": {
		Desc: "Get sessionStorage items.",
		Fn: func(origin string, items *map[string]string) chromedp.Action {
			return chromedp.ActionFunc(func(ctx context.Context) error {
				frameTree, err := page.GetFrameTree().Do(ctx)
				if err != nil {
					return err
				}
				strageKey := domstorage.SerializedStorageKey(frameTree.Frame.SecurityOrigin + "/")
				storageID := &domstorage.StorageID{
					StorageKey:     strageKey,
					IsLocalStorage: false,
				}
				resp, err := domstorage.GetDOMStorageItems(storageID).Do(ctx)
				if err != nil {
					return err
				}
				m := make(map[string]string)
				for _, v := range resp {
					if len(v) != 2 {
						continue
					}
					m[v[0]] = v[1]
				}
				*items = m
				return nil
			})
		},
		Args: CDPFnArgs{
			{CDPArgTypeArg, "origin", "https://github.com"},
			{CDPArgTypeRes, "items", `{"key": "value"}`},
		},
		Aliases: []string{"getSessionStorage"},
	},
}
View Source 
var ErrFailFast = errors.New("fail fast")
View Source 
var ErrInvalidScope = errors.New("invalid scope")
View Source 
var ErrNilBook = errors.New("runbook is nil")

Functions

func BufConfig added in v0.105.0 

func BufConfig(configs ...string) grpcRunnerOption

func BufDir added in v0.106.0 

func BufDir(dirs ...string) grpcRunnerOption

func BufLock added in v0.105.0 

func BufLock(locks ...string) grpcRunnerOption

func BufModule added in v0.106.0 

func BufModule(modules ...string) grpcRunnerOption

func CACert added in v0.26.0 

func CACert(path string) grpcRunnerOption

func CACertFromData added in v0.26.0 

func CACertFromData(b []byte) grpcRunnerOption

func CDPFlag added in v0.102.0 

func CDPFlag(flag string, tf any) cdpRunnerOption

CDPFlag set chromedp flag.

func Cert added in v0.26.0 

func Cert(path string) grpcRunnerOption

func CertFromData added in v0.26.0 

func CertFromData(b []byte) grpcRunnerOption

func CreateHTTPStepMapSlice added in v0.40.0 

func CreateHTTPStepMapSlice(key string, req *http.Request) (yaml.MapSlice, error)

CreateHTTPStepMapSlice creates yaml.MapSlice from *http.Request.

func DBTrace added in v0.88.0 

func DBTrace(trace bool) dbRunnerOption

func Eval added in v0.50.0 

func Eval(e string, store exprtrace.EvalEnv) (any, error)

func EvalAny added in v0.80.0 

func EvalAny(e any, store exprtrace.EvalEnv) (any, error)

EvalAny evaluate any type. but, EvalAny do not evaluate map key.

func EvalCond added in v0.50.0 

func EvalCond(cond string, store exprtrace.EvalEnv) (bool, error)

func EvalCount added in v0.50.0 

func EvalCount(count string, store exprtrace.EvalEnv) (int, error)

func EvalExpand added in v0.50.0 

func EvalExpand(in any, store exprtrace.EvalEnv) (any, error)

EvalExpand evaluates `in` and expand `{{ }}` in `in` using `store`.

func EvalWithTrace added in v0.108.0 

func EvalWithTrace(e string, store exprtrace.EvalEnv) (*exprtrace.EvalResult, error)

func GRPCTrace added in v0.88.0 

func GRPCTrace(trace bool) grpcRunnerOption

func HTTPCACert added in v0.57.2 

func HTTPCACert(path string) httpRunnerOption

func HTTPCert added in v0.57.2 

func HTTPCert(path string) httpRunnerOption

func HTTPKey added in v0.57.2 

func HTTPKey(path string) httpRunnerOption

func HTTPSkipVerify added in v0.74.1 

func HTTPSkipVerify(skip bool) httpRunnerOption

func HTTPTimeout added in v0.76.0 

func HTTPTimeout(timeout string) httpRunnerOption

func HTTPTrace added in v0.88.0 

func HTTPTrace(trace bool) httpRunnerOption

func Host added in v0.52.0 

func Host(h string) sshRunnerOption

func Hostname added in v0.52.0 

func Hostname(h string) sshRunnerOption

func IdentityFile added in v0.52.0 

func IdentityFile(p string) sshRunnerOption

func IdentityKey added in v0.59.3 

func IdentityKey(k []byte) sshRunnerOption

func ImportPaths added in v0.75.0 

func ImportPaths(paths []string) grpcRunnerOption

ImportPaths set import paths.

func KeepSession added in v0.52.0 

func KeepSession(enable bool) sshRunnerOption

func Key added in v0.26.0 

func Key(path string) grpcRunnerOption

func KeyFromData added in v0.26.0 

func KeyFromData(b []byte) grpcRunnerOption

func Load added in v0.2.0 

func Load(pathp string, opts ...Option) (*operatorN, error)

func LoadBook 

func LoadBook(path string) (*book, error)

func LoadEnvFile added in v0.111.0 

func LoadEnvFile(path string) error

LoadEnvFile loads the environment variables from the given file immediately.

func LocalForward added in v0.59.0 

func LocalForward(l string) sshRunnerOption

func MultipartBoundary added in v0.49.0 

func MultipartBoundary(b string) httpRunnerOption

func New 

func New(opts ...Option) (*operator, error)

New returns *operator.

func NewCmdOut added in v0.39.0 

func NewCmdOut(out io.Writer, verbose bool) *cmdOut

func NewDebugger added in v0.33.0 

func NewDebugger(out io.Writer) *debugger

func NewLoadtResult added in v0.57.0 

func NewLoadtResult(rc int, w, d time.Duration, c, m int, r *or.Result) (*loadtResult, error)

func NewRunbook added in v0.40.0 

func NewRunbook(desc string) *runbook

func NotFollowRedirect added in v0.38.0 

func NotFollowRedirect(nf bool) httpRunnerOption

func OpenAPI3 added in v0.100.0 

func OpenAPI3(l string) httpRunnerOption

OpenAPI3 sets OpenAPI Document using file path.

func OpenAPI3FromData added in v0.100.0 

func OpenAPI3FromData(d []byte) httpRunnerOption

OpenAPI3FromData sets OpenAPI Document from data.

func OpenApi3 added in v0.13.0 

func OpenApi3(l string) httpRunnerOption

OpenApi3 sets OpenAPI Document using file path. Deprecated: Use OpenAPI3 instead.

func OpenApi3FromData added in v0.13.0 

func OpenApi3FromData(d []byte) httpRunnerOption

OpenApi3FromData sets OpenAPI Document from data. Deprecated: Use OpenAPI3FromData instead.

func ParseRunbook added in v0.49.0 

func ParseRunbook(in io.Reader) (*runbook, error)

func Port added in v0.52.0 

func Port(p int) sshRunnerOption

func Protos added in v0.75.0 

func Protos(protos []string) grpcRunnerOption

Protos append protos.

func RemoveCacheDir added in v0.54.0 

func RemoveCacheDir() error

RemoveCacheDir remove cache directory for remote runbooks.

func SSHConfig added in v0.52.0 

func SSHConfig(p string) sshRunnerOption

func SetCacheDir added in v0.54.0 

func SetCacheDir(dir string) error

SetCacheDir set cache directory for remote runbooks.

func ShortenPath added in v0.45.0 

func ShortenPath(p string) string

ShortenPath shorten path.

func SkipCircularReferenceCheck added in v0.118.0 

func SkipCircularReferenceCheck(skip bool) httpRunnerOption

SkipCircularReferenceCheck sets whether to skip circular reference check in OpenAPI Document.

func SkipValidateRequest added in v0.13.0 

func SkipValidateRequest(skip bool) httpRunnerOption

SkipValidateRequest sets whether to skip validation of HTTP request with OpenAPI Document.

func SkipValidateResponse added in v0.13.0 

func SkipValidateResponse(skip bool) httpRunnerOption

SkipValidateResponse sets whether to skip validation of HTTP response with OpenAPI Document.

func SprintMultilinef added in v0.68.0 

func SprintMultilinef(lineformat, format string, a ...any) string

func TLS added in v0.26.0 

func TLS(useTLS bool) grpcRunnerOption

func UseCookie added in v0.77.0 

func UseCookie(use bool) httpRunnerOption

func User added in v0.52.0 

func User(u string) sshRunnerOption

Types

type AfterFuncError added in v0.48.0 

type AfterFuncError struct {
	// contains filtered or unexported fields
}

func (*AfterFuncError) Error added in v0.48.0 

func (e *AfterFuncError) Error() string

func (*AfterFuncError) Unwrap added in v0.48.0 

func (e *AfterFuncError) Unwrap() error

type BeforeFuncError added in v0.48.0 

type BeforeFuncError struct {
	// contains filtered or unexported fields
}

func (*BeforeFuncError) Error added in v0.48.0 

func (e *BeforeFuncError) Error() string

func (*BeforeFuncError) Unwrap added in v0.48.0 

func (e *BeforeFuncError) Unwrap() error

type CDPAction added in v0.47.0 

type CDPAction struct {
	Fn   string
	Args map[string]any
}

type CDPActions added in v0.47.0 

type CDPActions []CDPAction

type CDPArgType added in v0.47.0 

type CDPArgType string
const (
	CDPArgTypeArg CDPArgType = "arg"
	CDPArgTypeRes CDPArgType = "res"
)

type CDPFn added in v0.47.0 

type CDPFn struct {
	Desc    string
	Fn      any
	Args    CDPFnArgs
	Aliases []string
}

type CDPFnArg added in v0.47.0 

type CDPFnArg struct {
	Typ     CDPArgType
	Key     string
	Example string
}

type CDPFnArgs added in v0.47.0 

type CDPFnArgs []CDPFnArg

func (CDPFnArgs) ArgArgs added in v0.47.0 

func (a CDPFnArgs) ArgArgs() CDPFnArgs

func (CDPFnArgs) ResArgs added in v0.47.0 

func (a CDPFnArgs) ResArgs() CDPFnArgs

type Capturer added in v0.33.0 

type Capturer interface {
	CaptureStart(trs Trails, bookPath, desc string)
	CaptureResult(trs Trails, result *RunResult)
	CaptureEnd(trs Trails, bookPath, desc string)

	CaptureResultByStep(trs Trails, result *RunResult)

	CaptureHTTPRequest(name string, req *http.Request)
	CaptureHTTPResponse(name string, res *http.Response)

	CaptureGRPCStart(name string, typ GRPCType, service, method string)
	CaptureGRPCRequestHeaders(h map[string][]string)
	CaptureGRPCRequestMessage(m map[string]any)
	CaptureGRPCResponseStatus(*status.Status)
	CaptureGRPCResponseHeaders(h map[string][]string)
	CaptureGRPCResponseMessage(m map[string]any)
	CaptureGRPCResponseTrailers(t map[string][]string)
	CaptureGRPCClientClose()
	CaptureGRPCEnd(name string, typ GRPCType, service, method string)

	CaptureCDPStart(name string)
	CaptureCDPAction(a CDPAction)
	CaptureCDPResponse(a CDPAction, res map[string]any)
	CaptureCDPEnd(name string)

	CaptureSSHCommand(command string)
	CaptureSSHStdout(stdout string)
	CaptureSSHStderr(stderr string)

	CaptureDBStatement(name string, stmt string)
	CaptureDBResponse(name string, res *DBResponse)

	CaptureExecCommand(command, shell string, background bool)
	CaptureExecStdin(stdin string)
	CaptureExecStdout(stdout string)
	CaptureExecStderr(stderr string)

	SetCurrentTrails(trs Trails)
	Errs() error
}

type Coverage added in v0.84.1 

type Coverage struct {
	Specs []*SpecCoverage `json:"specs"`
}

Coverage is a coverage of runbooks.

type DBResponse added in v0.33.0 

type DBResponse struct {
	LastInsertID int64
	RowsAffected int64
	Columns      []string
	Rows         []map[string]any
}

type GRPCOp added in v0.35.0 

type GRPCOp string
const (
	GRPCOpMessage GRPCOp = "message"
	GRPCOpReceive GRPCOp = "receive"
	GRPCOpClose   GRPCOp = "close"
)

type GRPCType added in v0.35.0 

type GRPCType string
const (
	GRPCUnary           GRPCType = "unary"
	GRPCServerStreaming GRPCType = "server"
	GRPCClientStreaming GRPCType = "client"
	GRPCBidiStreaming   GRPCType = "bidi"
)

type Loop added in v0.29.0 

type Loop struct {
	Count       string   `yaml:"count,omitempty"`
	Interval    string   `yaml:"interval,omitempty"`
	MinInterval string   `yaml:"minInterval,omitempty"`
	MaxInterval string   `yaml:"maxInterval,omitempty"`
	Jitter      *float64 `yaml:"jitter,omitempty"`
	Multiplier  *float64 `yaml:"multiplier,omitempty"`
	Until       string   `yaml:"until"`
	// contains filtered or unexported fields
}

func (*Loop) Clear added in v0.110.0 

func (l *Loop) Clear()

func (*Loop) Loop added in v0.29.0 

func (l *Loop) Loop(ctx context.Context) bool

type Option 

type Option func(*book) error

func AfterFunc added in v0.20.0 

func AfterFunc(fn func(*RunResult) error) Option

AfterFunc - Register the function to be run after the runbook is run.

func AfterFuncIf added in v0.50.0 

func AfterFuncIf(fn func(*RunResult) error, ifCond string) Option

AfterFuncIf - Register the function to be run after the runbook is run if condition is true.

func Attach added in v0.99.4 

func Attach(enable bool) Option

Attach - Enable or disable debbuging attachment.

func BeforeFunc added in v0.20.0 

func BeforeFunc(fn func(*RunResult) error) Option

BeforeFunc - Register the function to be run before the runbook is run.

func Book 

func Book(path string) Option

Book - Load runbook.

func Books added in v0.4.0 

func Books(pathp string) ([]Option, error)

Books - Load multiple runbooks.

func CDPRunner added in v0.102.0 

func CDPRunner(name string, opts ...cdpRunnerOption) Option

CDPRunner - Set CDP runner to runbook.

func Capture added in v0.33.0 

func Capture(c Capturer) Option

Capture - Register the capturer to capture steps.

func DBRunner 

func DBRunner(name string, client Querier) Option

DBRunner - Set DB runner to runbook.

func DBRunnerWithOptions added in v0.88.0 

func DBRunnerWithOptions(name, dsn string, opts ...dbRunnerOption) Option

DBRunnerWithOptions - Set DB runner to runbook using options.

func Debug added in v0.3.0 

func Debug(debug bool) Option

Debug - Enable debug output.

func Desc 

func Desc(desc string) Option

Desc - Set description to runbook.

func FailFast added in v0.10.1 

func FailFast(enable bool) Option

FailFast - Enable fail-fast.

func Force added in v0.66.0 

func Force(enable bool) Option

Force - Force all steps to run.

func Func added in v0.18.0 

func Func(k string, v any) Option

Func - Set function to runner.

func GRPCBufConfig added in v0.105.1 

func GRPCBufConfig(configs ...string) Option

GRPCBufConfig - Set the path to the buf.yaml file for gRPC runners.

func GRPCBufDir added in v0.106.0 

func GRPCBufDir(dirs ...string) Option

GRPCBufDir - Set the buf directory for gRPC runners.

func GRPCBufLock added in v0.105.1 

func GRPCBufLock(locks ...string) Option

GRPCBufLock - Set the path to the buf.lock file for gRPC runners.

func GRPCBufModule added in v0.105.1 

func GRPCBufModule(modules ...string) Option

GRPCBufModule - Set the buf modules for gRPC runners.

func GRPCImportPaths added in v0.75.0 

func GRPCImportPaths(paths []string) Option

GRPCImportPaths - Set the path to the directory where proto sources can be imported for gRPC runners.

func GRPCNoTLS added in v0.40.0 

func GRPCNoTLS(noTLS bool) Option

GRPCNoTLS - Disable TLS use in all gRPC runners.

func GRPCProtos added in v0.75.0 

func GRPCProtos(protos []string) Option

GRPCProtos - Set the name of proto source for gRPC runners.

func GrpcRunner added in v0.24.0 

func GrpcRunner(name string, cc *grpc.ClientConn) Option

GrpcRunner - Set gRPC runner to runbook.

func GrpcRunnerWithOptions added in v0.66.0 

func GrpcRunnerWithOptions(name, target string, opts ...grpcRunnerOption) Option

GrpcRunnerWithOptions - Set gRPC runner to runbook using options.

func HTTPOpenApi3s added in v0.86.0 

func HTTPOpenApi3s(locations []string) Option

HTTPOpenApi3s - Set the path of OpenAPI Document for HTTP runners.

func HTTPRunner 

func HTTPRunner(name, endpoint string, client *http.Client, opts ...httpRunnerOption) Option

HTTPRunner - Set HTTP runner to runbook.

func HTTPRunnerWithHandler added in v0.6.0 

func HTTPRunnerWithHandler(name string, h http.Handler, opts ...httpRunnerOption) Option

HTTPRunnerWithHandler - Set HTTP runner to runbook with http.Handler.

func HostRules added in v0.95.0 

func HostRules(rules ...string) Option

HostRules - Set host rules for runn.

func Interval added in v0.9.0 

func Interval(d time.Duration) Option

Interval - Set interval between steps.

func LoadOnly added in v0.62.0 

func LoadOnly() Option

LoadOnly - Load only.

func Overlay added in v0.36.0 

func Overlay(path string) Option

Overlay - Overlay values on a runbook.

func Profile added in v0.28.0 

func Profile(enable bool) Option

Profile - Enable profile.

func RunConcurrent added in v0.64.0 

func RunConcurrent(enable bool, max int) Option

RunConcurrent - Run runbooks concurrently.

func RunID added in v0.78.0 

func RunID(ids ...string) Option

RunID - Run the matching runbooks in order if there is only one runbook with a forward matching ID.

func RunLabel added in v0.90.0 

func RunLabel(labels ...string) Option

RunLabel - Run all runbooks matching the label specification.

func RunMatch added in v0.22.0 

func RunMatch(m string) Option

RunMatch - Run only runbooks with matching paths.

func RunRandom added in v0.46.0 

func RunRandom(n int) Option

RunRandom - Run the specified number of runbooks at random. Sometimes the same runbook is run multiple times.

func RunSample added in v0.22.0 

func RunSample(n int) Option

RunSample - Sample the specified number of runbooks.

func RunShard added in v0.23.0 

func RunShard(n, i int) Option

RunShard - Distribute runbooks into a specified number of shards and run the specified shard of them.

func RunShuffle added in v0.39.0 

func RunShuffle(enable bool, seed int64) Option

RunShuffle - Randomize the order of running runbooks.

func Runner 

func Runner(name, dsn string, opts ...httpRunnerOption) Option

Runner - Set runner to runbook.

func SSHRunner added in v0.51.0 

func SSHRunner(name string, client *ssh.Client) Option

SSHRunner - Set SSH runner to runbook.

func SSHRunnerWithOptions added in v0.52.0 

func SSHRunnerWithOptions(name string, opts ...sshRunnerOption) Option

SSHRunnerWithOptions - Set SSH runner to runbook using options.

func Scopes added in v0.89.0 

func Scopes(scopes ...string) Option

Scopes - Set scopes for runn.

func SkipIncluded added in v0.19.0 

func SkipIncluded(enable bool) Option

SkipIncluded - Skip running the included step by itself.

func SkipTest added in v0.20.0 

func SkipTest(enable bool) Option

SkipTest - Skip test section.

func Stderr added in v0.52.2 

func Stderr(w io.Writer) Option

Stderr - Set STDERR.

func Stdout added in v0.52.2 

func Stdout(w io.Writer) Option

Stdout - Set STDOUT.

func T 

func T(t *testing.T) Option

T - Acts as test helper.

func Trace added in v0.87.0 

func Trace(enable bool) Option

Trace - Add tokens for tracing to headers and queries by default.

func Underlay added in v0.36.0 

func Underlay(path string) Option

Underlay - Lay values under the runbook.

func Var added in v0.4.0 

func Var(k any, v any) Option

Var - Set variable to runner.

func WaitTimeout added in v0.99.0 

func WaitTimeout(d time.Duration) Option

WaitTimeout - Set the timeout for waiting for sub-processes to complete after the Run or RunN context is canceled.

type Querier added in v0.43.0 

type Querier interface {
	sqlexp.Querier
}

type RunResult added in v0.48.0 

type RunResult struct {
	// runbook ID
	ID          string
	Desc        string
	Labels      []string
	Path        string
	Skipped     bool
	Err         error
	StepResults []*StepResult
	Elapsed     time.Duration
	// contains filtered or unexported fields
}

RunResult is the result of a runbook run.

func (*RunResult) OutFailure added in v0.81.1 

func (rr *RunResult) OutFailure(out io.Writer) error

func (*RunResult) Store added in v0.50.0 

func (rr *RunResult) Store() map[string]any

type RunnerType added in v0.45.0 

type RunnerType string
const (
	RunnerTypeHTTP    RunnerType = "http"
	RunnerTypeDB      RunnerType = "db"
	RunnerTypeGRPC    RunnerType = "grpc"
	RunnerTypeCDP     RunnerType = "cdp"
	RunnerTypeSSH     RunnerType = "ssh"
	RunnerTypeExec    RunnerType = "exec"
	RunnerTypeTest    RunnerType = "test"
	RunnerTypeDump    RunnerType = "dump"
	RunnerTypeInclude RunnerType = "include"
	RunnerTypeBind    RunnerType = "bind"
)

type SpecCoverage added in v0.84.1 

type SpecCoverage struct {
	Key       string         `json:"key"`
	Coverages map[string]int `json:"coverages"`
}

SpecCoverage is a coverage of spec (e.g. OpenAPI Document, servive of protocol buffers).

type StepResult added in v0.68.0 

type StepResult struct {
	// runbook ID
	ID      string
	Key     string
	Desc    string
	Skipped bool
	Err     error
	// Run results of runbook loaded by include runner
	IncludedRunResults []*RunResult
	Elapsed            time.Duration
}

StepResult is the result of a step run.

type Trail added in v0.78.0 

type Trail struct {
	Type           TrailType  `json:"type"`
	Desc           string     `json:"desc,omitempty"`
	RunbookID      string     `json:"id,omitempty"`
	RunbookPath    string     `json:"path,omitempty"`
	StepIndex      *int       `json:"step_index,omitempty"`
	StepKey        string     `json:"step_key,omitempty"`
	StepRunnerType RunnerType `json:"step_runner_type,omitempty"`
	StepRunnerKey  string     `json:"step_runner_key,omitempty"`
	FuncIndex      *int       `json:"func_index,omitempty"`
	LoopIndex      *int       `json:"loop_index,omitempty"`
}

Trail - The trail of elements in the runbook at runtime. Trail does not use slices to copy values.

func (Trail) String added in v0.78.0 

func (tr Trail) String() string

type TrailType added in v0.78.0 

type TrailType string
const (
	TrailTypeRunbook    TrailType = "runbook"
	TrailTypeStep       TrailType = "step"
	TrailTypeBeforeFunc TrailType = "beforeFunc"
	TrailTypeAfterFunc  TrailType = "afterFunc"
	TrailTypeLoop       TrailType = "loop"
)

type Trails added in v0.78.0 

type Trails []Trail

type TxQuerier added in v0.43.0 

type TxQuerier interface {
	nest.Querier
	BeginTx(ctx context.Context, opts *nest.TxOptions) (*nest.Tx, error)
}

type UnsupportedError added in v0.17.1 

type UnsupportedError struct {
	Cause error
}

func (*UnsupportedError) Error added in v0.17.1 

func (e *UnsupportedError) Error() string

func (*UnsupportedError) Unwrap added in v0.17.1 

func (e *UnsupportedError) Unwrap() error

Source Files

View all Source files

Directories

Path Synopsis
cmd
runn
examples
go-test
internal
deprecation

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.