webpagetest

package module
v0.0.0-...-a1026e1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 15, 2019 License: MIT Imports: 10 Imported by: 0

README

go-webpagetest

Simple, opionated wrapper for WebPagetest API and CLI (mostly as a test for wrapper itself)

Usage

Get it with:

go get -d -v github.com/olegfedoseev/go-webpagetest

And than in you code:

import  "github.com/olegfedoseev/go-webpagetest"

wpt, err := webpagetest.NewClient("https://webpagetest.org")
if err != nil {
  log.Fatalf("Failed to create client: %v", err)
}

result, err := wpt.RunTest(webpagetest.TestSettings{
  URL:      "https://google.com",
  Location: "Frankfurt_Ruxit:Chrome",
  Runs:     3,
  ScreenWidth:  1280,
  ScreenHeight: 720,
})
if err != nil {
  log.Fatalf("Error: %v", err)
}
fmt.Printf("Result: %#v", result)

Or you can look at source code of CLI at cmd/main.go

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Breakdown

type Breakdown struct {
	Color []int `json:"color"`
	Bytes int   `json:"bytes"`

	Requests int `json:"requests"`
}

Breakdown is struct for data for pie charts of resource distribution

type Client

type Client struct {
	Host string
}

Client is client of WebPageTest

func NewClient

func NewClient(host string) (*Client, error)

NewClient returns new ready to use Client

func (*Client) CancelTest

func (c *Client) CancelTest(testID string) error

CancelTest will try to cancel test by it's ID With a test ID (and if required, API key) you can cancel a test if it has not started running.

func (*Client) GetLocations

func (c *Client) GetLocations() (*Locations, error)

GetLocations will retrieve all available locations from server You can request a list of locations as well as the number of pending tests for each

func (*Client) GetTestResult

func (c *Client) GetTestResult(testID string) (*ResultData, error)

GetTestResult returns result of test with testID

func (*Client) GetTestStatus

func (c *Client) GetTestStatus(testID string) (*TestStatus, error)

GetTestStatus will return status of test run by given testID StatusCode 200 indicates test is completed. 1XX means the test is still in progress. And 4XX indicates some error.

func (*Client) GetTesters

func (c *Client) GetTesters() (*Testers, error)

GetTesters will retrieve all available agents and their status

func (*Client) RunTest

func (c *Client) RunTest(settings TestSettings) (string, error)

RunTest will submit given test to WPT server

func (*Client) RunTestAndWait

func (c *Client) RunTestAndWait(settings TestSettings, callback StatusCallback) (*ResultData, error)

RunTestAndWait will start new WebPageTest test run with given TestSettings and will wait for it to complete. While it wait, it will poll status updates from server and will call StatusCallback with it

type Connectivity

type Connectivity struct {
	// Profile name
	Name string `json:"connectivity"`
	// Download bandwidth in Kbps
	BandwidthDown int `json:"bwDown"`
	// Upload bandwidth in Kbps
	BandwidthUp int `json:"bwUp"`
	// First-hop Round Trip Time in ms
	Latency int `json:"latency"`
	// Packet loss rate - percent of packets to drop
	RawPacketLossRate *json.RawMessage `json:"plr"`
	PacketLossRate    int
}

Connectivity settings for test

func (Connectivity) String

func (c Connectivity) String() string

String gives human readable string for connectivity profile

type Domain

type Domain struct {
	Bytes       int    `json:"bytes"`
	Requests    int    `json:"requests"`
	CDNProvider string `json:"cdn_provider"`
	Connections int    `json:"connections"`
}

Domain is struct for stats about requests form particular domain

type Headers

type Headers struct {
	Request  []string `json:"request"`
	Response []string `json:"response"`
}

Headers is struct for http headers of request and response

type Images

type Images struct {
	Waterfall      string `json:"waterfall"`
	ConnectionView string `json:"connectionView"`
	Checklist      string `json:"checklist"`
	ScreenShot     string `json:"screenShot"`
	ScreenShotPng  string `json:"screenShotPng"`
}

Images is struct for links to originals of various images for test tun

type Location

type Location struct {
	Label      string
	LabelShort string
	Location   string
	Browsers   []string
	Status     string

	Default bool

	RelayServer   string
	RelayLocation string

	PendingTests map[string]int
}

Location is where your agents are

type Locations

type Locations map[string][]Location

Locations grouped by Group

type Pages

type Pages struct {
	Details    string `json:"details"`
	Checklist  string `json:"checklist"`
	Breakdown  string `json:"breakdown"`
	Domains    string `json:"domains"`
	ScreenShot string `json:"screenShot"`
}

Pages is struct for links to various pages about test run

type RawData

type RawData struct {
	Headers      string `json:"headers"`
	PageData     string `json:"pageData"`
	RequestsData string `json:"requestsData"`
	Utilization  string `json:"utilization"`
}

RawData is struct for links to raw data about test tun

type ResultData

type ResultData struct {
	Connectivity

	ID       string `json:"id"`
	URL      string `json:"url"`
	Summary  string `json:"summary"`
	TestURL  string `json:"testUrl"`
	Location string `json:"location"`
	Label    string `json:"label"`
	From     string `json:"from"`

	Mobile           int    `json:"mobile"`
	Completed        int    `json:"completed"`
	Tester           string `json:"tester"`
	TesterDNS        string `json:"testerDNS"`
	FirstViewOnly    bool   `json:"fvonly"`
	SuccessfulFVRuns int    `json:"successfulFVRuns"`
	SuccessfulRVRuns int    `json:"successfulRVRuns"`

	Runs   map[string]TestRun `json:"runs"`
	Median TestRun            `json:"median"`
}

ResultData holds all info about test

func (*ResultData) GetMedianRun

func (rd *ResultData) GetMedianRun(step int, metric string) (*TestRun, error)

GetMedianRun will calculate and return median run by given metric and step Step is 0-based

type StatusCallback

type StatusCallback func(testID, status string, duration int)

StatusCallback is helper type for function to be called while waiting for test to complete

type TestInfo

type TestInfo struct {
	URL           string `json:"url"`
	Runs          int    `json:"runs"`
	FirstViewOnly int    `json:"fvonly"`
	Web10         int    `json:"web10"`     // Stop Test at Document Complete
	IgnoreSSL     int    `json:"ignoreSSL"` // Ignore SSL Certificate Errors
	Video         string `json:"video"`
	Label         string `json:"label"`
	Priority      int    `json:"priority"`
	Location      string `json:"location"`
	Browser       string `json:"browser"`

	Connectivity string `json:"connectivity"`
	BandwidthIn  int    `json:"bwIn"`
	BandwidthOut int    `json:"bwOut"`
	Latency      int    `json:"latency"`

	// It can be string or int
	RawPacketLossRate *json.RawMessage `json:"plr"`
	PacketLossRate    int

	Tcpdump      int `json:"tcpdump"`  // Capture network packet trace (tcpdump)
	Timeline     int `json:"timeline"` // Capture Dev Tools Timeline
	Trace        int `json:"trace"`    // Capture Chrome Trace (about://tracing)
	Bodies       int `json:"bodies"`
	NetLog       int `json:"netlog"`    // Capture Network Log
	Standards    int `json:"standards"` // Disable Compatibility View (IE Only)
	NoScript     int `json:"noscript"`  // Disable Javascript
	Pngss        int `json:"pngss"`
	ImageQuality int `json:"iq"`
	KeepUA       int `json:"keepua"` // Preserve original User Agent string
	Mobile       int `json:"mobile"`
	Scripted     int `json:"scripted"`
}

TestInfo is info about test

type TestRun

type TestRun struct {
	FirstView  TestView `json:"firstView"`
	RepeatView TestView `json:"repeatView"`
}

TestRun is a test run info

type TestSettings

type TestSettings struct {
	// URL to be tested
	URL string `json:",omitempty"`
	// Label for the test
	Label string `json:",omitempty"`

	Where    string `json:",omitempty"`
	Browser  string `json:",omitempty"`
	Location string `json:",omitempty"`

	// Viewport Width in css pixels
	ScreenWidth int `json:",omitempty"`
	// Viewport Height in css pixels
	ScreenHeight int `json:",omitempty"`
	// Default metric to use when calculating the median run (loadTime)
	MedianMetric string `json:",omitempty"`
	// Number of test runs (1-10 on the public instance) (1)
	Runs int `json:",omitempty"`
	// Scripted test to execute ("")
	Script string `json:",omitempty"`
	// Custom Headers
	CustomHeaders string `json:",omitempty"`
	// Set to 1 to have Chrome capture the Dev Tools timeline (0)
	Timeline bool `json:",omitempty"`
	// Set to 1 to skip the Repeat View test (0)
	FirstViewOnly bool `json:",omitempty"`
	// Set to 1 to keep the test hidden from the test log (0)
	Private bool `json:",omitempty"`
	// Set to 1 to capture video (video is required for calculating Speed Index) (0)
	CaptureVideo bool `json:",omitempty"`
	// Set to 1 to save a full-resolution version of the fully loaded screen shot as a png (0)
	PNGScreenShot bool `json:",omitempty"`
	// Specify a jpeg compression level (30-100) for the screen shots and video capture (75)
	ImageQuality int `json:",omitempty"`
	// (optional) URL to ping when the test is complete (the test ID will be passed as an "id" parameter)
	Pingback string `json:",omitempty"`
	// (DOM) Element to record for sub-measurement
	DOMElement string `json:",omitempty"`
	// (Override) the number of concurrent connections IE uses (0 to not override)	0
	Connections int `json:",omitempty"`
	// (optional) Set to between 1 - 5 to have Chrome include the Javascript call stack. Must be used in conjunction with "timeline". 	 0
	TimelineStack int `json:",omitempty"`
	// (optional) Set to 1 to force the test to stop at Document Complete (onLoad)	0
	Web10 bool `json:",omitempty"`
	// (optional) space-delimited list of urls to block
	Block string `json:",omitempty"`
	// (optional) User name to use for authenticated tests (http authentication)
	Login string `json:",omitempty"`
	// (optional) Password to use for authenticated tests (http authentication)
	Password string `json:",omitempty"`
	// (optional) Type of authentication to use: 0 = Basic Auth, 1 = SNS	0
	AuthType string `json:",omitempty"`
	// (optional) e-mail address to notify with the test results
	Notify string `json:",omitempty"`
	// (optional) Download bandwidth in Kbps (used when specifying a custom connectivity profile)
	BWDown int `json:",omitempty"`
	// (optional) Upload bandwidth in Kbps (used when specifying a custom connectivity profile)
	BWUp int `json:",omitempty"`
	// (optional) First-hop Round Trip Time in ms (used when specifying a custom connectivity profile)
	Latency int `json:",omitempty"`
	// (optional) Packet loss rate - percent of packets to drop (used when specifying a custom connectivity profile)
	PacketLossRate int `json:",omitempty"`
	// (optional) (required for public instance)	API Key (if assigned) - applies only to runtest.php calls. Contact the site owner for a key if required (http://www.webpagetest.org/getkey.php for the public instance)
	APIKey string `json:",omitempty"`
	// (optional) Set to 1 to enable tcpdump capture	 0
	TCPDump bool `json:",omitempty"`
	// (optional) Set to 1 to disable optimization checks (for faster testing)	0
	NoOpt bool `json:",omitempty"`
	// (optional) Set to 1 to disable screen shot capturing	0
	NoImages bool `json:",omitempty"`
	// (optional) Set to 1 to disable saving of the http headers (as well as browser status messages and CPU utilization)	0
	NoHeaders bool `json:",omitempty"`
	// (optional) Set to 1 to disable javascript (IE, Chrome, Firefox)
	NoScript bool `json:",omitempty"`
	// (optional) Set to 1 to clear the OS certificate caches (causes IE to do OCSP/CRL checks during SSL negotiation if the certificates are not already cached). Added in 2.11	 0
	ClearCerts bool `json:",omitempty"`
	// (optional) Set to 1 to have Chrome emulate a mobile browser (screen resolution, UA string, fixed viewport).  Added in 2.11	 0
	Mobile bool `json:",omitempty"`
	// (optional) Set to 1 to preserve the original browser User Agent string (don't append PTST to it)
	KeepUA bool `json:",omitempty"`
	// (optional) Custom User Agent String to use
	UAString string `json:",omitempty"`
	// (optional) Device Pixel Ratio to use when emulating mobile
	DPR int `json:",omitempty"`
	// (optional) Set to 1 when capturing video to only store the video from the median run.	 0
	MedianRunVideo bool `json:",omitempty"`
	// (optional)  Custom command-line options (Chrome only)
	CmdLine string `json:",omitempty"`
	// (optional) Set to 1 to save the content of the first response (base page) instead of all of the text responses (bodies=1)
	HTMLBody bool `json:",omitempty"`
	// (optional)  Custom metrics to collect at the end of a test
	CustomMetrics string `json:",omitempty"`
	// (optional) Specify a specific tester that the test should run on (must match the PC name in /getTesters.php).  If the tester is not available the job will never run.
	Tester string `json:",omitempty"`
	// (optional) Specify a string that will be used to hash the test to a specific test agent.  The tester will be picked by index among the available testers.  If the number of testers changes then the tests will be distributed to different machines but if the counts remain consistent then the same string will always run the tests on the same test machine.  This can be useful for controlling variability when comparing a given URL over time or different parameters against each other (using the URL as the hash string).
	Affinity string `json:",omitempty"`
	// (optional) Set to 1 to Ignore SSL Certificate Errors e.g. Name mismatch, Self-signed certificates, etc.	 0
	IgnoreSSL bool `json:",omitempty"`
	// (optional)  Device name from mobile_devices.ini to use for mobile emulation (only when mobile=1 is specified to enable emulation and only for Chrome)
	MobileDevice string `json:",omitempty"`
	// (optional)  String to append to the user agent string. This is in addition to the default PTST/ver string. If "keepua" is also specified it will still append. Allows for substitution with some test parameters:
	// %TESTID% - Replaces with the test ID for the current test
	// %RUN% - Replaces with the current run number
	// %CACHED% - Replaces with 1 for repeat view tests and 0 for initial view
	// %VERSION% - Replaces with the current wptdriver version number
	AppendUA   string `json:",omitempty"`
	Lighthouse int    `json:"lighthouse,omitempty"`
}

TestSettings is structure for describing what should be done in test run

func (TestSettings) GetFormParams

func (s TestSettings) GetFormParams() url.Values

GetFormParams returns settings that was set ready to be passed to POST

type TestStatus

type TestStatus struct {
	StatusCode int    `json:"statusCode"`
	StatusText string `json:"statusText"`

	ID           string `json:"id"`
	TestID       string `json:"testId"`
	Location     string `json:"location"`
	StartTime    string `json:"startTime"`
	CompleteTime string `json:"completeTime"`

	Runs        int `json:"runs"`
	BehindCount int `json:"behindCount"`

	Remote         bool `json:"remote"` // Relay Test
	FirstViewOnly  int  `json:"fvonly"`
	Elapsed        int  `json:"elapsed"`
	ElapsedUpdate  int  `json:"elapsedUpdate"`
	TestsExpected  int  `json:"testsExpected"`
	TestsCompleted int  `json:"testsCompleted"`

	FirstViewRunsCompleted  int `json:"fvRunsCompleted"`
	RepeatViewRunsCompleted int `json:"rvRunsCompleted"`

	TestInfo TestInfo `json:"testInfo"`
}

TestStatus is status of a test

type TestStep

type TestStep struct {
	URL    string  `json:"URL"`
	Run    int     `json:"run"`
	Date   float64 `json:"date"`   // 1479973600
	Error  string  `json:"error"`  // Timed out waiting for the browser to start.
	Result int     `json:"result"` // 99999

	Tester         string `json:"tester"`
	BrowserName    string `json:"browser_name"`    // "Google Chrome"
	BrowserVersion string `json:"browser_version"` // "54.0.2840.99",
	NumSteps       int    `json:"numSteps"`
	Step           int    `json:"step"`
	EventName      string `json:"eventName"` // "Step 1"

	PageTitle string `json:"title"`
	// Estimated RTT to Server (ms)
	ServerRTT int `json:"server_rtt"`
	// Time to First Byte (ms)
	// The First Byte time (often abbreviated as TTFB) is measured as the time from the start of
	// the initial navigation until the first byte of the base page is received by the browser (after following redirects).
	TTFB int `json:"TTFB"`
	// Time to DOM Loading - From Navigation Timing
	DOMLoading int `json:"domLoading"`
	// Browser-reported first paint time (IE-specific right now - window.performance.timing.msFirstPaint)
	FirstPaint           float64 `json:"firstPaint"`
	FirstMeaningfulPaint int     `json:"chromeUserTiming.firstMeaningfulPaint"`
	// Time from the start of the operation until the title first changed (in ms)
	TitleTime int `json:"titleTime"`
	// Time to DOM Interactive - From Navigation Timing
	DOMInteractive int `json:"domInteractive"`
	// DOM Content Loaded - From Navigation Timing
	DOMContentLoadedEventStart int `json:"domContentLoadedEventStart"`
	DOMContentLoadedEventEnd   int `json:"domContentLoadedEventEnd"` // 455,
	// Browser-reported Load Time (Navigation Timing onload)
	LoadEventStart int `json:"loadEventStart"`
	LoadEventEnd   int `json:"loadEventEnd"`
	// Load Time (onload, ms)
	// The Load Time is measured as the time from the start of the initial navigation until the beginning of the window load event (onload).
	LoadTime int `json:"loadTime"`
	DocTime  int `json:"docTime"`
	DOMTime  int `json:"domTime"`
	// Time to Start Render (ms)
	// The Start Render time is measured as the time from the start of the initial
	// navigation until the first non-white content is painted to the browser display.
	StartRender int `json:"render"`
	// Time to Visually Complete (ms)
	VisualComplete int `json:"visualComplete"`
	// Fully Loaded (ms)
	// The Fully Loaded time is measured as the time from the start of the initial navigation until
	// there was 2 seconds of no network activity after Document Complete.  This will usually
	// include any activity that is triggered by javascript after the main page loads.
	FullyLoaded int `json:"fullyLoaded"`
	// Time of the last visual change to the page (in ms, only available when video capture is enabled)
	LastVisualChange int `json:"lastVisualChange"`
	// Time until the above-the-fold stabilized (if explicitly requested)
	AboveTheFoldTime int `json:"aft"`
	SpeedIndex       int `json:"SpeedIndex"`

	// Number of DOM Elements
	// The DOM Elements metric is the count of the DOM elements on the tested page as measured at the end of the test.
	DOMElements int `json:"domElements"`

	// CPU Busy Time (ms)
	DocCPUms         float32 `json:"docCPUms"`         // 951.606
	FullyLoadedCPUms float32 `json:"fullyLoadedCPUms"` // 1294.808,

	CPUTimes    map[string]int `json:"cpuTimes"`
	CPUTimesDoc map[string]int `json:"cpuTimesDoc"`

	DocCPUpct         int     `json:"docCPUpct"`         // 39
	FullyLoadedCPUpct float64 `json:"fullyLoadedCPUpct"` // 19,

	// The number of bytes downloaded before the Document Complete time
	BytesIn         int `json:"bytesIn"`
	BytesOut        int `json:"bytesOut"`
	BytesInDoc      int `json:"bytesInDoc"`
	BytesOutDoc     int `json:"bytesOutDoc"`
	EffectiveBps    int `json:"effectiveBps"`    // 433693
	EffectiveBpsDoc int `json:"effectiveBpsDoc"` // 466135
	// Total bytes in server-supplied TLS certificates
	CertificateBytes int `json:"certificate_bytes"` // 17499,

	Connections int `json:"connections"`

	RequestsFull int `json:"requestsFull"`
	// The number of http(s) requests before the Document Complete time
	RequestsDoc int `json:"requestsDoc"`

	Responses200   int `json:"responses_200"`
	Responses404   int `json:"responses_404"`
	ResponsesOther int `json:"responses_other"`

	OptimizationChecked  int `json:"optimization_checked"`   // 1
	ScoreCache           int `json:"score_cache"`            // 0
	ScoreCDN             int `json:"score_cdn"`              // -1
	ScoreGZip            int `json:"score_gzip"`             // -1
	ScoreCookies         int `json:"score_cookies"`          // -1
	ScoreKeepAlive       int `json:"score_keep-alive"`       // -1
	ScoreMinify          int `json:"score_minify"`           // -1
	ScoreCombine         int `json:"score_combine"`          // 100
	ScoreCompress        int `json:"score_compress"`         // -1
	ScoreETags           int `json:"score_etags"`            // -1
	ScoreProgressiveJpeg int `json:"score_progressive_jpeg"` // -1,

	GZipTotal   int `json:"gzip_total"`   // 0
	GZipSavings int `json:"gzip_savings"` // 0,

	MinifyTotal   int `json:"minify_total"`   // 0
	MinifySavings int `json:"minify_savings"` // 0,

	ImageTotal   int `json:"image_total"`   // 0
	ImageSavings int `json:"image_savings"` // 0,

	PageSpeedVersion string `json:"pageSpeedVersion"` // "1.9",

	ServerCount int `json:"server_count"` // 16,

	Cached        int `json:"cached"`         // 0,
	AdultSite     int `json:"adult_site"`     // 0,
	FixedViewport int `json:"fixed_viewport"` // 0

	BasePageCDN       string `json:"base_page_cdn"`       // "Google"
	BasePageRedirects int    `json:"base_page_redirects"` // 2
	BasePageTTFB      int    `json:"base_page_ttfb"`      // 524,

	BrowserProcessCount         int `json:"browser_process_count"`           // 8
	BrowserMainMemoryKB         int `json:"browser_main_memory_kb"`          // 69752
	BrowserWorkingSetKB         int `json:"browser_working_set_kb"`          // 136568
	BrowserOtherPrivateMemoryKB int `json:"browser_other_private_memory_kb"` // 66816,

	TimeToInteractive int `json:"TTIMeasurementEnd"` // 11846
	LastInteractive   int `json:"LastInteractive"`   // 9571

	Pages        Pages      `json:"pages"`
	Thumbnails   Thumbnails `json:"thumbnails"`
	ImagesString string     `json:"Images"`
	Images       struct {
		Waterfall      string `json:"waterfall"`
		ConnectionView string `json:"connectionView"`
		Checklist      string `json:"checklist"`
		ScreenShot     string `json:"screenShot"`
	} `json:"images"`
	RawData     RawData              `json:"rawData"`
	VideoFrames []VideoFrame         `json:"videoFrames"`
	Breakdown   map[string]Breakdown `json:"breakdown"`

	RawDomains json.RawMessage   `json:"domains"`
	Domains    map[string]Domain `json:"-"` // may be empty array

	TestTiming map[string]int `json:"testTiming"`
}

TestStep is struct with information of one particular test "run"

type TestView

type TestView struct {
	Run           int        `json:"run"`
	Tester        string     `json:"tester"`
	NumberOfSteps int        `json:"numSteps"`
	Steps         []TestStep `json:"steps"`
}

TestView struct tries to combine to kinds of testViews than WebPagetest returns With Steps in case of scripted run and without steps, when we test single url Because Go is strictly typed, we have to "merge" them in one data type

func (*TestView) UnmarshalJSON

func (tv *TestView) UnmarshalJSON(b []byte) error

UnmarshalJSON implements custom unmarshaling logic than mitigates "dynamic" nature of test result's json

type Tester

type Tester struct {
	ID           string
	Name         string
	AgentVersion string

	// Status
	ErrorRate  int
	Elapsed    int
	LastWork   int
	IsRebooted bool
	IsOffline  bool
	IsBusy     bool

	// Network
	EC2 string
	IP  string
	DNS string

	// Screen
	ScreenWidth  int64
	ScreenHeight int64

	// Windows
	IEVersion      string
	WindowsVersion string
	IsWinServer    bool
	IsWin64        bool

	// Hardware
	FreeDisk float64
	GPU      bool
	CPU      int
}

Tester is an agent that will run tests

type Testers

type Testers map[string][]Tester

Testers is an array of Tester

type Thumbnails

type Thumbnails struct {
	Waterfall  string `json:"waterfall"`
	Checklist  string `json:"checklist"`
	ScreenShot string `json:"screenShot"`
}

Thumbnails is struct for links to thumbnails of various images for test tun

type VideoFrame

type VideoFrame struct {
	Time  int    `json:"time"`
	Image string `json:"image"`

	VisuallyComplete int `json:"VisuallyComplete"`
}

VideoFrame is struct for one video frame

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL