README

Hargo

Hargo Build Status GoDoc Go Report Card GitHub license GitHub issues Twitter

Hargo parses HAR files, can convert to curl format, and serve as a load test driver.

NAME:
   hargo - work with HTTP Archive (.har) files

USAGE:
   hargo <command> [arguments] <.har file>

VERSION:
   0.1.2-dev.57 (da53069)

AUTHOR:
   Mark A. Richman <mark@markrichman.com>

COMMANDS:
     fetch, f     Fetch URLs in .har
     curl, c      Convert .har to curl
     run, r       Run .har file
     validate, v  Validate .har file
     dump, d      Dump .har file
     load, l      Load test .har file
     help, h      Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --debug        Show debug output
   --help, -h     show help
   --version, -v  print the version

COPYRIGHT:
   (c) 2019 Mark A. Richman

Running Hargo

go get github.com/mrichman/hargo
cd $GOPATH/src/github.com/mrichman/hargo
go run cmd/hargo/hargo.go validate test/golang.org.har

Building from source

Make sure that you have Go version 1.9 or greater (I haven't tested with lower) and that your GOPATH env variable is set (I recommand setting it to ~/go if you don't have one). If GOBIN is not set, also try setting that to ~/go/bin, as make install may fail. You can check all Go environment variables with go env.

go get -d github.com/mrichman/hargo
cd $GOPATH/src/github.com/mrichman/hargo
make install

About HAR Files

If you use Google Chrome, you can record these files by following the steps below:

  1. Right-click anywhere on that page and click on Inspect Element to open Chrome's Developer Tools
  2. The Developer Tools will open as a panel at the bottom of the page. Click on the Network tab.
  3. Click the Record button, which is the solid black circle at the bottom of the Network tab, and you'll start recording activity in your browser.
  4. Refresh the page and start working normally
  5. Right-click within the Network tab and click Save as HAR with Content to save a copy of the activity that you recorded.
  6. Within the file window, save the HAR file.

Commands

Fetch

The fetch command downloads all resources references in .har file:

hargo fetch foo.har

This will produce a directory named hargo-fetch-yyyymmddhhmmss containing all assets references by the .har file. This is similar to what you'd see when invoking wget on a particular URL.

Curl

The curl command will output a curl command line for each entry in the .har file.

hargo curl foo.har

Run

The run command executes each HTTP request in .har file:

hargo run foo.har

This is similar to fetch but will not save any output.

Validate

The validate command will report any errors in the format of a .har file.

hargo validate foo.har

HAR file format is defined here: https://w3c.github.io/web-performance/specs/HAR/Overview.html

Dump

Dump prints information about all HTTP requests in .har file

hargo dump foo.har

Load

Hargo can act as a load test agent. Given a .har file, hargo can spawn a number of concurrent workers to repeat each HTTP request in order. By default, hargo will spawn 10 workers and run for a duration of 60 seconds.

Hargo will also save its results to InfluxDB, if available. Each HTTP response is stored as a point of time-series data, which can be graphed by Chronograf, Grafana, or similar visualization tool for analysis.

Docker

Build container
docker build -t hargo .
Run container
docker run --rm -v `pwd`/test:/test hargo hargo run /test/golang.org.har

Docker-compose

The example docker-compose file will start three containers:

  • hargo
  • influxdb
  • grafana

The hargo container will first needs to be built. See build. When the compose file is run it will start a hargo load process that will write the results to InfluxDB. This InfluxDB instance can be viewed using the grafana container. This contains an example dashboard showing the latency of the executed request. Username/password for all the containers is hargo/hargo.

commands
cd example/docker-compose
docker-compose up
docker-compose down -v
Expand ▾ Collapse ▴

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Dump

func Dump(r *bufio.Reader)

    Dump prints all HTTP requests in .har file

    func EntryToRequest

    func EntryToRequest(entry *Entry, ignoreHarCookies bool) (*http.Request, error)

      EntryToRequest converts a HAR entry type to an http.Request

      func Fetch

      func Fetch(r *bufio.Reader) error

        Fetch downloads all resources references in .har file

        func LoadTest

        func LoadTest(harfile string, file *os.File, workers int, timeout time.Duration, u url.URL, ignoreHarCookies bool, insecureSkipVerify bool) error

          LoadTest executes all HTTP requests in order concurrently for a given number of workers.

          func NewReader

          func NewReader(r io.Reader) *bufio.Reader

            NewReader returns a bufio.Reader that will skip over initial UTF-8 byte order marks. https://tools.ietf.org/html/rfc7159#section-8.1

            func ReadStream

            func ReadStream(file *os.File, entries chan Entry, stop chan bool)

              ReadStream reads the har file as a stream and puts the entries on a chan for consumption. When the end of a file is reached it will start over until the stop signal is given. https://golang.org/pkg/encoding/json/#example_Decoder_Decode_stream

              func Run

              func Run(r *bufio.Reader, ignoreHarCookies bool, insecureSkipVerify bool) error

                Run executes all entries in .har file

                func ToCurl

                func ToCurl(r *bufio.Reader) (string, error)

                  ToCurl converts a HAR Entry to a curl command line curl -X <method> -b "<name=value&name=value...>" -H <name: value> ... -d "<postData>" <url>

                  func Validate

                  func Validate(r *bufio.Reader) (bool, error)

                    Validate validates the format of a .har file

                    func WritePoint

                    func WritePoint(u url.URL, results chan TestResult)

                      WritePoint inserts data to InfluxDB

                      Types

                      type Browser

                      type Browser struct {
                      	// Required. The name of the browser that created the log.
                      	Name string `json:"name"`
                      	// Required. The version number of the browser that created the log.
                      	Version string `json:"version"`
                      	// Optional. A comment provided by the user or the browser.
                      	Comment string `json:"comment"`
                      }

                        Browser that created the log

                        type Cache

                        type Cache struct {
                        	// optional State of a cache entry before the request. Leave out this field
                        	// if the information is not available.
                        	BeforeRequest CacheObject `json:"beforeRequest,omitempty"`
                        	// optional State of a cache entry after the request. Leave out this field if
                        	// the information is not available.
                        	AfterRequest CacheObject `json:"afterRequest,omitempty"`
                        	// optional (new in 1.2) A comment provided by the user or the application.
                        	Comment string `json:"comment,omitempty"`
                        }

                          Cache contains info about a request coming from browser cache.

                          type CacheObject

                          type CacheObject struct {
                          	// optional - Expiration time of the cache entry.
                          	Expires string `json:"expires,omitempty"`
                          	// The last time the cache entry was opened.
                          	LastAccess string `json:"lastAccess"`
                          	// Etag
                          	ETag string `json:"eTag"`
                          	// The number of times the cache entry has been opened.
                          	HitCount int `json:"hitCount"`
                          	// optional (new in 1.2) A comment provided by the user or the application.
                          	Comment string `json:"comment,omitempty"`
                          }

                            CacheObject is used by both beforeRequest and afterRequest

                            type Content

                            type Content struct {
                            	// Length of the returned content in bytes. Should be equal to
                            	// response.bodySize if there is no compression and bigger when the content
                            	// has been compressed.
                            	Size int `json:"size"`
                            	// optional Number of bytes saved. Leave out this field if the information
                            	// is not available.
                            	Compression int `json:"compression,omitempty"`
                            	// MIME type of the response text (value of the Content-Type response
                            	// header). The charset attribute of the MIME type is included (if
                            	// available).
                            	MimeType string `json:"mimeType"`
                            	// optional Response body sent from the server or loaded from the browser
                            	// cache. This field is populated with textual content only. The text field
                            	// is either HTTP decoded text or a encoded (e.g. "base64") representation of
                            	// the response body. Leave out this field if the information is not
                            	// available.
                            	Text string `json:"text,omitempty"`
                            	// optional (new in 1.2) Encoding used for response text field e.g
                            	// "base64". Leave out this field if the text field is HTTP decoded
                            	// (decompressed & unchunked), than trans-coded from its original character
                            	// set into UTF-8.
                            	Encoding string `json:"encoding,omitempty"`
                            	// optional (new in 1.2) A comment provided by the user or the application.
                            	Comment string `json:"comment,omitempty"`
                            }

                              Content describes details about response content (embedded in <response> object).

                              type Cookie struct {
                              	// The name of the cookie.
                              	Name string `json:"name"`
                              	// The cookie value.
                              	Value string `json:"value"`
                              	// optional The path pertaining to the cookie.
                              	Path string `json:"path,omitempty"`
                              	// optional The host of the cookie.
                              	Domain string `json:"domain,omitempty"`
                              	// optional Cookie expiration time.
                              	// (ISO 8601 YYYY-MM-DDThh:mm:ss.sTZD, e.g. 2009-07-24T19:20:30.123+02:00).
                              	Expires string `json:"expires,omitempty"`
                              	// optional Set to true if the cookie is HTTP only, false otherwise.
                              	HTTPOnly bool `json:"httpOnly,omitempty"`
                              	// optional (new in 1.2) True if the cookie was transmitted over ssl, false
                              	// otherwise.
                              	Secure bool `json:"secure,omitempty"`
                              	// optional (new in 1.2) A comment provided by the user or the application.
                              	Comment bool `json:"comment,omitempty"`
                              }

                                Cookie contains list of all cookies (used in <request> and <response> objects).

                                type Creator

                                type Creator struct {
                                	// Required. The name of the application that created the log.
                                	Name string `json:"name"`
                                	// Required. The version number of the application that created the log.
                                	Version string `json:"version"`
                                	// Optional. A comment provided by the user or the application.
                                	Comment string `json:"comment,omitempty"`
                                }

                                  Creator contains information about the log creator application

                                  type Entry

                                  type Entry struct {
                                  	Pageref string `json:"pageref,omitempty"`
                                  	// Date and time stamp of the request start
                                  	// (ISO 8601 YYYY-MM-DDThh:mm:ss.sTZD).
                                  	StartedDateTime string `json:"startedDateTime"`
                                  	// Total elapsed time of the request in milliseconds. This is the sum of all
                                  	// timings available in the timings object (i.e. not including -1 values) .
                                  	Time float32 `json:"time"`
                                  	// Detailed info about the request.
                                  	Request Request `json:"request"`
                                  	// Detailed info about the response.
                                  	Response Response `json:"response"`
                                  	// Info about cache usage.
                                  	Cache Cache `json:"cache"`
                                  	// Detailed timing info about request/response round trip.
                                  	PageTimings PageTimings `json:"pageTimings"`
                                  	// optional (new in 1.2) IP address of the server that was connected
                                  	// (result of DNS resolution).
                                  	ServerIPAddress string `json:"serverIPAddress,omitempty"`
                                  	// optional (new in 1.2) Unique ID of the parent TCP/IP connection, can be
                                  	// the client port number. Note that a port number doesn't have to be unique
                                  	// identifier in cases where the port is shared for more connections. If the
                                  	// port isn't available for the application, any other unique connection ID
                                  	// can be used instead (e.g. connection index). Leave out this field if the
                                  	// application doesn't support this info.
                                  	Connection string `json:"connection,omitempty"`
                                  	// (new in 1.2) A comment provided by the user or the application.
                                  	Comment string `json:"comment,omitempty"`
                                  }

                                    Entry is a unique, optional Reference to the parent page. Leave out this field if the application does not support grouping by pages.

                                    type Har

                                    type Har struct {
                                    	Log Log
                                    }

                                      Har is a container type for deserialization

                                      func Decode

                                      func Decode(r *bufio.Reader) (Har, error)

                                        Decode reads from a reader and returns Har object

                                        type Log

                                        type Log struct {
                                        
                                        	// Required. Version number of the format.
                                        	Version string `json:"version"`
                                        	// Required. An object of type creator that contains the name and version
                                        	// information of the log creator application.
                                        	Creator Creator `json:"creator"`
                                        	// Optional. An object of type browser that contains the name and version
                                        	// information of the user agent.
                                        	Browser Browser `json:"browser"`
                                        	// Optional. An array of objects of type page, each representing one exported
                                        	// (tracked) page. Leave out this field if the application does not support
                                        	// grouping by pages.
                                        	Pages []Page `json:"pages,omitempty"`
                                        	// Required. An array of objects of type entry, each representing one
                                        	// exported (tracked) HTTP request.
                                        	Entries []Entry `json:"entries"`
                                        	// Optional. A comment provided by the user or the application. Sorting
                                        	// entries by startedDateTime (starting from the oldest) is preferred way how
                                        	// to export data since it can make importing faster. However the reader
                                        	// application should always make sure the array is sorted (if required for
                                        	// the import).
                                        	Comment string `json:"comment"`
                                        }

                                          Log represents the root of the exported data. This object MUST be present and its name MUST be "log".

                                          type NVP

                                          type NVP struct {
                                          	Name    string `json:"name"`
                                          	Value   string `json:"value"`
                                          	Comment string `json:"comment,omitempty"`
                                          }

                                            NVP is simply a name/value pair with a comment

                                            type Page

                                            type Page struct {
                                            
                                            	// Date and time stamp for the beginning of the page load
                                            	// (ISO 8601 YYYY-MM-DDThh:mm:ss.sTZD, e.g. 2009-07-24T19:20:30.45+01:00).
                                            	StartedDateTime string `json:"startedDateTime"`
                                            	// Unique identifier of a page within the . Entries use it to refer the parent page.
                                            	ID string `json:"id"`
                                            	// Page title.
                                            	Title string `json:"title"`
                                            	// Detailed timing info about page load.
                                            	PageTiming PageTiming `json:"pageTiming"`
                                            	// (new in 1.2) A comment provided by the user or the application.
                                            	Comment string `json:"comment,omitempty"`
                                            }

                                              Page object for every exported web page and one <entry> object for every HTTP request. In case when an HTTP trace tool isn't able to group requests by a page, the <pages> object is empty and individual requests doesn't have a parent page.

                                              type PageTiming

                                              type PageTiming struct {
                                              	// Content of the page loaded. Number of milliseconds since page load started
                                              	// (page.startedDateTime). Use -1 if the timing does not apply to the current
                                              	// request.
                                              	// Depeding on the browser, onContentLoad property represents DOMContentLoad
                                              	// event or document.readyState == interactive.
                                              	OnContentLoad int `json:"onContentLoad"`
                                              	// Page is loaded (onLoad event fired). Number of milliseconds since page
                                              	// load started (page.startedDateTime). Use -1 if the timing does not apply
                                              	// to the current request.
                                              	OnLoad int `json:"onLoad"`
                                              	// (new in 1.2) A comment provided by the user or the application.
                                              	Comment string `json:"comment"`
                                              }

                                                PageTiming describes timings for various events (states) fired during the page load. All times are specified in milliseconds. If a time info is not available appropriate field is set to -1.

                                                type PageTimings

                                                type PageTimings struct {
                                                	Blocked int `json:"blocked,omitempty"`
                                                	// optional - Time spent in a queue waiting for a network connection. Use -1
                                                	// if the timing does not apply to the current request.
                                                	DNS int `json:"dns,omitempty"`
                                                	// optional - DNS resolution time. The time required to resolve a host name.
                                                	// Use -1 if the timing does not apply to the current request.
                                                	Connect int `json:"connect,omitempty"`
                                                	// optional - Time required to create TCP connection. Use -1 if the timing
                                                	// does not apply to the current request.
                                                	Send int `json:"send"`
                                                	// Time required to send HTTP request to the server.
                                                	Wait int `json:"wait"`
                                                	// Waiting for a response from the server.
                                                	Receive int `json:"receive"`
                                                	// Time required to read entire response from the server (or cache).
                                                	Ssl int `json:"ssl,omitempty"`
                                                	// optional (new in 1.2) - Time required for SSL/TLS negotiation. If this
                                                	// field is defined then the time is also included in the connect field (to
                                                	// ensure backward compatibility with HAR 1.1). Use -1 if the timing does not
                                                	// apply to the current request.
                                                	Comment string `json:"comment,omitempty"`
                                                }

                                                  PageTimings describes various phases within request-response round trip. All times are specified in milliseconds.

                                                  type PostData

                                                  type PostData struct {
                                                  	//  Mime type of posted data.
                                                  	MimeType string `json:"mimeType"`
                                                  	//  List of posted parameters (in case of URL encoded parameters).
                                                  	Params []PostParam `json:"params"`
                                                  	//  Plain text posted data
                                                  	Text string `json:"text"`
                                                  	// optional (new in 1.2) A comment provided by the user or the
                                                  	// application.
                                                  	Comment string `json:"comment,omitempty"`
                                                  }

                                                    PostData describes posted data, if any (embedded in <request> object).

                                                    type PostParam

                                                    type PostParam struct {
                                                    	// name of a posted parameter.
                                                    	Name string `json:"name"`
                                                    	// optional value of a posted parameter or content of a posted file.
                                                    	Value string `json:"value,omitempty"`
                                                    	// optional name of a posted file.
                                                    	FileName string `json:"fileName,omitempty"`
                                                    	// optional content type of a posted file.
                                                    	ContentType string `json:"contentType,omitempty"`
                                                    	// optional (new in 1.2) A comment provided by the user or the application.
                                                    	Comment string `json:"comment,omitempty"`
                                                    }

                                                      PostParam is a list of posted parameters, if any (embedded in <postData> object).

                                                      type Request

                                                      type Request struct {
                                                      	// Request method (GET, POST, ...).
                                                      	Method string `json:"method"`
                                                      	// Absolute URL of the request (fragments are not included).
                                                      	URL string `json:"url"`
                                                      	// Request HTTP Version.
                                                      	HTTPVersion string `json:"httpVersion"`
                                                      	// List of cookie objects.
                                                      	Cookies []Cookie `json:"cookies"`
                                                      	// List of header objects.
                                                      	Headers []NVP `json:"headers"`
                                                      	// List of query parameter objects.
                                                      	QueryString []NVP `json:"queryString"`
                                                      	// Posted data.
                                                      	PostData PostData `json:"postData"`
                                                      	// Total number of bytes from the start of the HTTP request message until
                                                      	// (and including) the double CRLF before the body. Set to -1 if the info
                                                      	// is not available.
                                                      	HeaderSize int `json:"headerSize"`
                                                      	// Size of the request body (POST data payload) in bytes. Set to -1 if the
                                                      	// info is not available.
                                                      	BodySize int `json:"bodySize"`
                                                      	// (new in 1.2) A comment provided by the user or the application.
                                                      	Comment string `json:"comment"`
                                                      }

                                                        Request contains detailed info about performed request.

                                                        type Response

                                                        type Response struct {
                                                        	// Response status.
                                                        	Status int `json:"status"`
                                                        	// Response status description.
                                                        	StatusText string `json:"statusText"`
                                                        	// Response HTTP Version.
                                                        	HTTPVersion string `json:"httpVersion"`
                                                        	// List of cookie objects.
                                                        	Cookies []Cookie `json:"cookies"`
                                                        	// List of header objects.
                                                        	Headers []NVP `json:"headers"`
                                                        	// Details about the response body.
                                                        	Content Content `json:"content"`
                                                        	// Redirection target URL from the Location response header.
                                                        	RedirectURL string `json:"redirectURL"`
                                                        	// Total number of bytes from the start of the HTTP response message until
                                                        	// (and including) the double CRLF before the body. Set to -1 if the info is
                                                        	// not available.
                                                        	// The size of received response-headers is computed only from headers that
                                                        	// are really received from the server. Additional headers appended by the
                                                        	// browser are not included in this number, but they appear in the list of
                                                        	// header objects.
                                                        	HeadersSize int `json:"headersSize"`
                                                        	// Size of the received response body in bytes. Set to zero in case of
                                                        	// responses coming from the cache (304). Set to -1 if the info is not
                                                        	// available.
                                                        	BodySize int `json:"bodySize"`
                                                        	// optional (new in 1.2) A comment provided by the user or the application.
                                                        	Comment string `json:"comment,omitempty"`
                                                        }

                                                          Response contains detailed info about the response.

                                                          type TestResult

                                                          type TestResult struct {
                                                          	URL       string    `json:"url"`
                                                          	Status    int       `json:"status"` // 200, 500, etc.
                                                          	StartTime time.Time `json:"startTime"`
                                                          	EndTime   time.Time `json:"endTime"`
                                                          	Latency   int       `json:"latency"` // milliseconds
                                                          	Method    string    `json:"method"`
                                                          	HarFile   string    `json:"harfile"`
                                                          }

                                                            TestResult contains results for an individual HTTP request

                                                            Directories

                                                            Path Synopsis
                                                            cmd