Documentation
¶
Overview ¶
Package wsw is the Go client for Website Screenshot World.
It covers the two HTTP APIs, Instant Snap (on-demand captures) and Auto Schedule (start a scheduled web page's capture now), and verifies the signature of the webhooks both send.
A Client is safe for concurrent use; create one and reuse it:
client := wsw.NewClient() // keys from WSW_INSTANT_SNAP_API_KEY and WSW_AUTO_SCHEDULE_API_KEY
capture, err := client.InstantSnap.Capture(ctx, &wsw.CaptureOptions{
URL: "https://example.com",
Format: wsw.FormatWebP,
})
Options ¶
An option that is not set is not sent, so the API applies its own default. Scalar options are pointers, set with String, Bool, Int and Float64; false, 0 and "" are values and are sent. Token options such as CaptureOptions.Format use typed strings, where the empty string means "not set". Nested blocks are struct pointers.
Errors ¶
Every failure the API or the network reports is an *Error. Its Kind groups the codes; test it with errors.Is and the sentinels (ErrRateLimit, ErrNotFound and the others), or read Code with errors.As for the exact reason. Network failures still match context.DeadlineExceeded and context.Canceled.
Retries ¶
Rate limits (but not queue_full), busy capture slots, unavailable services and connection failures are retried twice by default, waiting for the server's hint when it gives one. Captures always carry an Idempotency-Key, so a retry never runs a capture twice. A context deadline or cancellation stops retrying at once.
Index ¶
- Constants
- Variables
- func Bool(v bool) *bool
- func Float64(v float64) *float64
- func Int(v int) *int
- func String(v string) *string
- type AutoScheduleCaptureFailedEvent
- type AutoScheduleCaptureSucceededEvent
- type AutoScheduleService
- type Capture
- type CaptureChange
- type CaptureChangedEvent
- type CaptureFile
- type CaptureOptions
- type CaptureStatus
- type Catalog
- type Client
- type ClientOption
- func WithAutoScheduleKey(key string) ClientOption
- func WithBaseURLs(instantSnap, autoSchedule string) ClientOption
- func WithCaptureTimeout(d time.Duration) ClientOption
- func WithHTTPClient(hc *http.Client) ClientOption
- func WithInstantSnapKey(key string) ClientOption
- func WithMaxRetries(n int) ClientOption
- func WithTimeout(d time.Duration) ClientOption
- type Defaults
- type Delivery
- type DeliveryCompletedEvent
- type Destination
- type DeviceCategory
- type Error
- type ErrorKind
- type Event
- type EventEnvelope
- type Format
- type Geolocation
- type InstantSnapCaptureFailedEvent
- type InstantSnapCaptureSucceededEvent
- type InstantSnapService
- func (s *InstantSnapService) Capture(ctx context.Context, opts *CaptureOptions, reqOpts ...RequestOption) (*Capture, error)
- func (s *InstantSnapService) CaptureFile(ctx context.Context, opts *CaptureOptions, reqOpts ...RequestOption) (*CaptureFile, error)
- func (s *InstantSnapService) CaptureToBucket(ctx context.Context, opts *CaptureOptions, reqOpts ...RequestOption) (*Capture, error)
- func (s *InstantSnapService) Catalog(ctx context.Context) (*Catalog, error)
- func (s *InstantSnapService) Defaults(ctx context.Context) (*Defaults, error)
- func (s *InstantSnapService) Download(ctx context.Context, capture *Capture, w io.Writer) (int64, error)
- func (s *InstantSnapService) DownloadToFile(ctx context.Context, capture *Capture, path string) error
- func (s *InstantSnapService) Get(ctx context.Context, id string) (*Capture, error)
- func (s *InstantSnapService) Submit(ctx context.Context, opts *CaptureOptions, reqOpts ...RequestOption) (*Capture, error)
- func (s *InstantSnapService) SubmitToBucket(ctx context.Context, opts *CaptureOptions, reqOpts ...RequestOption) (*Capture, error)
- func (s *InstantSnapService) Wait(ctx context.Context, id string, opts ...WaitOption) (*Capture, error)
- type MediaCategory
- type PageSource
- type PageSourceFormat
- type PageSourceOptions
- type Ping
- type PingEvent
- type Product
- type Proxy
- type RequestOption
- type ResponseType
- type S3Delivery
- type S3DeliveryOptions
- type S3DeliveryStatus
- type SchedulePause
- type SchedulePausedEvent
- type ScheduledCapture
- type ScrollMode
- type Selectors
- type SitemapRun
- type SitemapRunCompletedEvent
- type TimestampWatermark
- type TriggerResult
- type TriggerStatus
- type TriggeredBy
- type UnknownEvent
- type VerifyOption
- type Viewport
- type ViewportDefaults
- type WaitOption
- type WatermarkLocation
Examples ¶
Constants ¶
const ( DefaultInstantSnapBaseURL = "https://api.websitescreenshotworld.com" DefaultAutoScheduleBaseURL = "https://autoapi.websitescreenshotworld.com" // DefaultCaptureTimeout bounds each capture attempt. The API can take up to 120 seconds. DefaultCaptureTimeout = 180 * time.Second // DefaultTimeout bounds each attempt of every other call. DefaultTimeout = 30 * time.Second // DefaultMaxRetries is how many times a retryable failure is retried. DefaultMaxRetries = 2 )
Defaults used by NewClient.
const ( EnvInstantSnapKey = "WSW_INSTANT_SNAP_API_KEY" EnvAutoScheduleKey = "WSW_AUTO_SCHEDULE_API_KEY" )
Environment variables NewClient reads the keys from.
const ( EventTypeCaptureSucceeded = "capture.succeeded" EventTypeCaptureFailed = "capture.failed" EventTypeCaptureChanged = "capture.changed" EventTypeDeliveryCompleted = "delivery.completed" EventTypeSchedulePaused = "schedule.paused" EventTypeSitemapRunCompleted = "sitemap_run.completed" EventTypePing = "ping" )
Event types, as sent in EventEnvelope.Type.
const ( // DefaultWaitTimeout is how long Wait reads a capture before returning it unfinished. DefaultWaitTimeout = 360 * time.Second // DefaultPollInterval is how long Wait pauses between reads. DefaultPollInterval = 3 * time.Second )
Defaults used by InstantSnapService.Wait.
const DefaultWebhookTolerance = 300 * time.Second
DefaultWebhookTolerance is how far a webhook's timestamp may be from now.
const SignatureHeader = "X-WSW-Signature"
SignatureHeader is the header that carries a webhook's signature.
const Version = "0.2.0"
Version is the version of this library, sent in the User-Agent header.
Variables ¶
var ( ErrAuthentication = errors.New("wsw: authentication failed") ErrInvalidRequest = errors.New("wsw: invalid request") ErrPlanLimit = errors.New("wsw: plan limit reached") ErrNotFound = errors.New("wsw: not found") ErrRateLimit = errors.New("wsw: rate limited") ErrCapture = errors.New("wsw: capture failed") ErrService = errors.New("wsw: service error") ErrConnection = errors.New("wsw: connection failed") ErrSignatureVerification = errors.New("wsw: webhook signature verification failed") )
Sentinels for use with errors.Is. An *Error matches the sentinel of its Kind. KindUnknown has none.
var ErrMissingKey = errors.New("wsw: missing API key")
ErrMissingKey is wrapped by the error a call returns, before sending anything, when the client has no API key for that product.
Functions ¶
Types ¶
type AutoScheduleCaptureFailedEvent ¶
type AutoScheduleCaptureFailedEvent struct {
EventEnvelope
Data ScheduledCapture `json:"data"`
}
AutoScheduleCaptureFailedEvent is sent when a scheduled capture fails.
type AutoScheduleCaptureSucceededEvent ¶
type AutoScheduleCaptureSucceededEvent struct {
EventEnvelope
Data ScheduledCapture `json:"data"`
}
AutoScheduleCaptureSucceededEvent is sent when a scheduled capture succeeds.
type AutoScheduleService ¶
type AutoScheduleService struct {
// contains filtered or unexported fields
}
AutoScheduleService starts the capture of a web page scheduled in the dashboard. Use it through Client.AutoSchedule. Results arrive by webhook.
func (*AutoScheduleService) TriggerNow ¶
func (s *AutoScheduleService) TriggerNow(ctx context.Context, webPageID string) (*TriggerResult, error)
TriggerNow queues a capture of the web page now. webPageID is the id shown in the dashboard, which webhooks call schedule_id. It is idempotent: while a capture of the page is pending, it answers TriggerStatusAlreadyPending and starts nothing.
Example ¶
client := wsw.NewClient()
result, err := client.AutoSchedule.TriggerNow(context.Background(), "7d4e2b1a-6c5f-4a3e-9d8c-2b1a0f9e8d7c")
if err != nil {
log.Fatal(err)
}
if result.Status == wsw.TriggerStatusAlreadyPending {
fmt.Println("already queued at", result.TriggeredAt)
}
type Capture ¶
type Capture struct {
ID string `json:"id"`
// URL is the page the capture was asked for; FinalURL says where the browser ended up.
// Sent from API 1.3.0; empty only from an older server.
URL string `json:"url"`
Status CaptureStatus `json:"status"`
ResponseType ResponseType `json:"response_type"`
MediaCategory MediaCategory `json:"media_category"`
// Format is the format written.
Format Format `json:"format"`
ContentType string `json:"content_type"`
SizeBytes int64 `json:"size_bytes"`
Width int `json:"width"`
Height int `json:"height"`
// DurationMS is how long the capture took.
DurationMS int `json:"duration_ms"`
// PageHTTPStatus is the status the website answered with.
PageHTTPStatus int `json:"page_http_status"`
CapturedAt time.Time `json:"captured_at"`
// FinalURL is where the browser ended up, when the page redirected.
FinalURL string `json:"final_url"`
// MediaSHA256 is the lowercase hex SHA-256 digest of the file.
MediaSHA256 string `json:"media_sha256"`
// DownloadURL is valid for 60 minutes and reissued by every [InstantSnapService.Get].
// It is empty for captures uploaded to a bucket and in webhooks.
DownloadURL string `json:"download_url"`
// ExpiresAt is when the file is deleted, 24 hours after the capture.
ExpiresAt time.Time `json:"expires_at"`
PageSource *PageSource `json:"page_source"`
// S3Delivery is set for captures uploaded to a bucket.
S3Delivery *S3Delivery `json:"s3_delivery"`
CallbackURL string `json:"callback_url"`
// ErrorCode says why a failed or timed-out capture ended; it is one of the error codes.
ErrorCode string `json:"error_code"`
// SpecVersion names the default set the capture was taken under.
SpecVersion string `json:"spec_version"`
// Replayed is true when the API answered with the result of an earlier request that had
// the same Idempotency-Key.
Replayed bool `json:"-"`
}
Capture is an Instant Snap capture as the API describes it. Fields the API sends as null hold their zero value.
type CaptureChange ¶
type CaptureChange struct {
ScheduledCapture
ChangePercent float64 `json:"change_percent"`
ComparedToCaptureID string `json:"compared_to_capture_id"`
ChangeImageAvailable bool `json:"change_image_available"`
}
CaptureChange is a scheduled capture that differs from an earlier one.
type CaptureChangedEvent ¶
type CaptureChangedEvent struct {
EventEnvelope
Data CaptureChange `json:"data"`
}
CaptureChangedEvent is sent when a scheduled capture differs from the one before it.
type CaptureFile ¶
type CaptureFile struct {
// Body streams the file. The capture timeout keeps running until it is closed.
Body io.ReadCloser
ID string
Status CaptureStatus
// ContentType is the file's media type, such as "image/png".
ContentType string
// Filename is the name the API suggests, from Content-Disposition.
Filename string
Width int
Height int
SizeBytes int64
DurationMS int
// PageHTTPStatus is the status the website answered with.
PageHTTPStatus int
SpecVersion string
CapturedAt time.Time
// FinalURL is where the browser ended up. It is empty when the URL is not plain ASCII.
FinalURL string
// SHA256 is the lowercase hex digest of the file.
SHA256 string
// PartCount is the number of parts in the answer, 1 unless the API says otherwise.
PartCount int
}
CaptureFile is the answer of InstantSnapService.CaptureFile: the file as a stream, and its metadata. Read Body once and close it, or call SaveTo.
func (*CaptureFile) SaveTo ¶
func (f *CaptureFile) SaveTo(path string) error
SaveTo writes the file to path, replacing any file there, and closes Body. Nothing is left at path if writing fails.
type CaptureOptions ¶
type CaptureOptions struct {
// URL is the page to capture: absolute http or https, without credentials.
URL string `json:"url"`
// MediaCategory defaults to image.
MediaCategory MediaCategory `json:"media_category,omitempty"`
// Format must belong to MediaCategory. image: png, jpeg, webp, tiff; video: mp4, webm,
// mov, gif; pdf: image_pdf, document_pdf.
Format Format `json:"format,omitempty"`
Viewport *Viewport `json:"viewport,omitempty"`
// DeviceCategory defaults to desktop.
DeviceCategory DeviceCategory `json:"device_category,omitempty"`
// EmulatedDevice is a device from [InstantSnapService.Catalog], matched without regard
// to case. It defaults to "iPhone 13" when DeviceCategory is mobile.
EmulatedDevice *string `json:"emulated_device,omitempty"`
// ScrollMode defaults to viewport for images and to scroll for video.
ScrollMode ScrollMode `json:"scroll_mode,omitempty"`
AutoHeight *bool `json:"auto_height,omitempty"`
// MaxHeight is refused, not lowered, above the format's ceiling (WebP 16383 divided by the
// scale factor, JPEG 65500).
MaxHeight *int `json:"max_height,omitempty"`
// Zoom is a percentage from 50 to 100.
Zoom *int `json:"zoom,omitempty"`
BlockAds *bool `json:"block_ads,omitempty"`
BlockPopUps *bool `json:"block_pop_ups,omitempty"`
// PressEscape presses Escape after the pop-up pass and again after scrolling (once in a
// video), for a modal that closes on Escape. The default is false.
PressEscape *bool `json:"press_escape,omitempty"`
PageTimeoutMS *int `json:"page_timeout_ms,omitempty"`
// ScrollingDelayMS is the pause after each scroll step of a full-page capture, from 0 to
// 10000; the default is 500.
ScrollingDelayMS *int `json:"scrolling_delay_ms,omitempty"`
UserAgent *string `json:"user_agent,omitempty"`
// Locale is a BCP 47 language tag.
Locale *string `json:"locale,omitempty"`
// TimeZone is an IANA time zone name.
TimeZone *string `json:"time_zone,omitempty"`
// Headers are sent with every request the page makes, with their names exactly as given.
// They need the RequestHeadersCookies plan feature.
Headers map[string]string `json:"headers,omitempty"`
// Cookies are set for the URL's origin, with their names exactly as given. They need the
// RequestHeadersCookies plan feature.
Cookies map[string]string `json:"cookies,omitempty"`
Geolocation *Geolocation `json:"geolocation,omitempty"`
Selectors *Selectors `json:"selectors,omitempty"`
Proxy *Proxy `json:"proxy,omitempty"`
TimestampWatermark *TimestampWatermark `json:"timestamp_watermark,omitempty"`
PageSource *PageSourceOptions `json:"page_source,omitempty"`
// S3Delivery is where [InstantSnapService.CaptureToBucket] and
// [InstantSnapService.SubmitToBucket] upload the file. The other capture methods refuse it.
S3Delivery *S3DeliveryOptions `json:"s3_delivery,omitempty"`
// Timeout is the browser time, in seconds, an async capture may use: async only
// ([InstantSnapService.Submit] and [InstantSnapService.SubmitToBucket]), from 30 to 300;
// the server default is 120. The server refuses it on the other capture methods.
Timeout *int `json:"timeout,omitempty"`
// CallbackURL also receives the capture.succeeded or capture.failed webhook. It needs
// the Webhooks plan feature.
CallbackURL *string `json:"callback_url,omitempty"`
}
CaptureOptions describes an Instant Snap capture. Only URL is required; every field left unset is not sent and takes the API's default (see InstantSnapService.Defaults).
type CaptureStatus ¶
type CaptureStatus string
CaptureStatus is where an Instant Snap capture is in its life. A capture from InstantSnapService.Submit or InstantSnapService.SubmitToBucket goes from queued to running to succeeded or failed.
const ( CaptureStatusQueued CaptureStatus = "queued" CaptureStatusRunning CaptureStatus = "running" CaptureStatusPending CaptureStatus = "pending" CaptureStatusSucceeded CaptureStatus = "succeeded" CaptureStatusDelivered CaptureStatus = "delivered" CaptureStatusFailed CaptureStatus = "failed" CaptureStatusTimedOut CaptureStatus = "timed_out" CaptureStatusExpired CaptureStatus = "expired" )
Capture statuses. CaptureStatusQueued (accepted, waiting for a browser) and CaptureStatusRunning (a browser has it) are the states of an async capture before it ends. CaptureStatusDelivered is the end state of a file returned by InstantSnapService.CaptureFile: the file went back in the response and is not kept.
type Catalog ¶
type Catalog struct {
EmulatedDevices []string `json:"emulated_devices"`
ProxyCountries []string `json:"proxy_countries"`
}
Catalog lists what CaptureOptions.EmulatedDevice and Proxy.Country accept.
type Client ¶
type Client struct {
// InstantSnap takes captures on demand.
InstantSnap *InstantSnapService
// AutoSchedule starts the capture of a scheduled web page.
AutoSchedule *AutoScheduleService
// contains filtered or unexported fields
}
Client calls the Website Screenshot World APIs. It is safe for concurrent use.
func NewClient ¶
func NewClient(opts ...ClientOption) *Client
NewClient returns a client. Either key may be left out; a call to a product whose key is missing fails before anything is sent.
Example ¶
// Keys default to WSW_INSTANT_SNAP_API_KEY and WSW_AUTO_SCHEDULE_API_KEY.
client := wsw.NewClient(
wsw.WithInstantSnapKey(os.Getenv("MY_INSTANT_SNAP_KEY")),
wsw.WithMaxRetries(3),
wsw.WithCaptureTimeout(4*time.Minute),
)
_ = client
type ClientOption ¶
type ClientOption func(*Client)
ClientOption configures a Client.
func WithAutoScheduleKey ¶
func WithAutoScheduleKey(key string) ClientOption
WithAutoScheduleKey sets the Auto Schedule API key. By default it is read from the WSW_AUTO_SCHEDULE_API_KEY environment variable.
func WithBaseURLs ¶
func WithBaseURLs(instantSnap, autoSchedule string) ClientOption
WithBaseURLs points the client at other servers, such as a test double. An empty string keeps that product's default.
func WithCaptureTimeout ¶
func WithCaptureTimeout(d time.Duration) ClientOption
WithCaptureTimeout bounds a whole capture or download call: every attempt, the waits between retries and reading the file. Zero means no bound beyond the context's. The default is DefaultCaptureTimeout.
func WithHTTPClient ¶
func WithHTTPClient(hc *http.Client) ClientOption
WithHTTPClient sets the HTTP client used for every request, for a custom transport or proxy. Leave its Timeout at zero: the library bounds each call itself (see WithTimeout). The client is copied, and the copy never follows redirects, so the API key is never sent anywhere else.
func WithInstantSnapKey ¶
func WithInstantSnapKey(key string) ClientOption
WithInstantSnapKey sets the Instant Snap API key. By default it is read from the WSW_INSTANT_SNAP_API_KEY environment variable.
func WithMaxRetries ¶
func WithMaxRetries(n int) ClientOption
WithMaxRetries sets how many times a retryable failure is retried; zero turns retries off. The default is DefaultMaxRetries.
func WithTimeout ¶
func WithTimeout(d time.Duration) ClientOption
WithTimeout bounds every other call as a whole, retries included; it also bounds InstantSnapService.Submit and InstantSnapService.SubmitToBucket, which are answered at once. Zero means no bound beyond the context's. The default is DefaultTimeout.
type Defaults ¶
type Defaults struct {
SpecVersion string `json:"spec_version"`
ResponseType ResponseType `json:"response_type"`
MediaCategory MediaCategory `json:"media_category"`
// Format is the default format of each media category.
Format map[MediaCategory]Format `json:"format"`
// Formats lists the formats each media category accepts.
Formats map[MediaCategory][]Format `json:"formats"`
DeviceCategory DeviceCategory `json:"device_category"`
EmulatedDevice string `json:"emulated_device"`
Viewport ViewportDefaults `json:"viewport"`
Zoom int `json:"zoom"`
ScrollMode ScrollMode `json:"scroll_mode"`
AutoHeight bool `json:"auto_height"`
// AutoHeightMaxHeight is zero when there is no general ceiling.
AutoHeightMaxHeight int `json:"auto_height_max_height"`
AutoHeightMaxHeightWebP int `json:"auto_height_max_height_webp"`
BlockAds bool `json:"block_ads"`
BlockPopUps bool `json:"block_pop_ups"`
PressEscape bool `json:"press_escape"`
PageTimeoutMS int `json:"page_timeout_ms"`
ScrollingDelayMS int `json:"scrolling_delay_ms"`
Geolocation bool `json:"geolocation"`
GeolocationAccuracy float64 `json:"geolocation_accuracy"`
WaitForCloudSync bool `json:"wait_for_cloud_sync"`
// TimestampWatermark, PageSource and Limits are passed through as the API sends them.
TimestampWatermark map[string]any `json:"timestamp_watermark"`
PageSource map[string]any `json:"page_source"`
Limits map[string]any `json:"limits"`
// LeftToTheBrowser names the settings the API does not set, leaving the browser's own.
LeftToTheBrowser []string `json:"left_to_the_browser"`
}
Defaults are the values a capture takes for the options it does not send, and the limits it is checked against. They are versioned by SpecVersion.
type Delivery ¶
type Delivery struct {
ScheduledCapture
Destinations []Destination `json:"destinations"`
}
Delivery is a scheduled capture and where it was delivered.
type DeliveryCompletedEvent ¶
type DeliveryCompletedEvent struct {
EventEnvelope
Data Delivery `json:"data"`
}
DeliveryCompletedEvent is sent when a scheduled capture has been delivered to its destinations.
type Destination ¶
type Destination struct {
Slot int `json:"slot"`
Name string `json:"name"`
Provider string `json:"provider"`
Status string `json:"status"`
CompletedAt time.Time `json:"completed_at"`
}
Destination is one place a scheduled capture was delivered to.
type DeviceCategory ¶
type DeviceCategory string
DeviceCategory is the kind of browser a capture emulates.
const ( DeviceCategoryDesktop DeviceCategory = "desktop" DeviceCategoryMobile DeviceCategory = "mobile" )
Device categories.
type Error ¶
type Error struct {
Kind ErrorKind
// Code is the API's error_code, a stable lowercase token such as "rate_limited". When a
// refusal had no JSON body it is derived from the HTTP status. Empty for connection and
// signature errors.
Code string
// Message is the API's error_message, one sentence for a person. Its wording can change.
Message string
// HTTPStatus is zero when no answer came back.
HTTPStatus int
// RetryAfter is how long the API asked to wait, from retry_after_seconds or the
// Retry-After header. Zero when it gave no hint.
RetryAfter time.Duration
// Body is the raw body of the refusal.
Body []byte
// contains filtered or unexported fields
}
Error is a refusal from the API, a connection failure, or a webhook that did not verify.
var apiErr *wsw.Error
if errors.As(err, &apiErr) && apiErr.Code == "quota_exhausted" {
// ...
}
Example ¶
client := wsw.NewClient()
ctx, cancel := context.WithTimeout(context.Background(), 4*time.Minute)
defer cancel()
_, err := client.InstantSnap.Capture(ctx, &wsw.CaptureOptions{URL: "https://example.com"})
var apiErr *wsw.Error
switch {
case err == nil:
case errors.Is(err, wsw.ErrPlanLimit):
log.Print("upgrade the plan: ", err)
case errors.Is(err, wsw.ErrCapture):
log.Print("the website could not be captured: ", err)
case errors.Is(err, context.DeadlineExceeded):
log.Print("gave up waiting")
case errors.As(err, &apiErr):
log.Printf("%s (%d): %s", apiErr.Code, apiErr.HTTPStatus, apiErr.Message)
default:
log.Print(err)
}
type ErrorKind ¶
type ErrorKind string
ErrorKind groups the error codes the API can answer with. Branch on the kind, or on Error.Code for the exact reason; never on the HTTP status or the message.
const ( // KindAuthentication means the API key is malformed, revoked or for the other product. KindAuthentication ErrorKind = "authentication" // KindInvalidRequest means the request was refused as it stands; retrying will not help. KindInvalidRequest ErrorKind = "invalid_request" // KindPlanLimit means the plan has no captures left or does not include the feature. KindPlanLimit ErrorKind = "plan_limit" // KindNotFound means there is no such capture or web page for this account. KindNotFound ErrorKind = "not_found" // KindRateLimit means too many requests were sent. rate_limited is retried automatically; // queue_full (too many async captures waiting or running) is not. KindRateLimit ErrorKind = "rate_limit" // KindCapture means the website could not be captured, or not in time. It is not retried. KindCapture ErrorKind = "capture" // KindService means the service was busy or failed. Some of its codes are retried // automatically; see [Error.Retryable]. KindService ErrorKind = "service" // KindConnection means the request could not be sent, or no answer came back. It is // retried automatically. KindConnection ErrorKind = "connection" // KindSignatureVerification means a webhook's signature did not verify. KindSignatureVerification ErrorKind = "signature_verification" // KindUnknown is a refusal whose error code this version of the library does not know. KindUnknown ErrorKind = "unknown" )
Error kinds.
type Event ¶
type Event interface {
// Envelope returns the fields every delivery carries.
Envelope() EventEnvelope
}
Event is a webhook delivery, returned by VerifyWebhook. Switch on its concrete type:
switch e := event.(type) {
case *wsw.InstantSnapCaptureSucceededEvent:
// e.Data is the capture
case *wsw.UnknownEvent:
// a type this version does not know; e.Type and e.Data are set
}
func VerifyWebhook ¶
func VerifyWebhook(body []byte, header, secret string, opts ...VerifyOption) (Event, error)
VerifyWebhook checks that body was signed with secret and returns the event it carries.
body must be the request body exactly as received: parsing and re-encoding it changes the bytes and breaks the signature. header is the value of the X-WSW-Signature header. A signature that does not verify returns an *Error matching ErrSignatureVerification.
Example ¶
secret := os.Getenv("WSW_WEBHOOK_SECRET")
http.HandleFunc("/webhooks/wsw", func(w http.ResponseWriter, r *http.Request) {
body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 1<<20))
if err != nil {
http.Error(w, "body too large", http.StatusRequestEntityTooLarge)
return
}
event, err := wsw.VerifyWebhook(body, r.Header.Get(wsw.SignatureHeader), secret)
if err != nil {
http.Error(w, "invalid signature", http.StatusBadRequest)
return
}
switch e := event.(type) {
case *wsw.InstantSnapCaptureSucceededEvent:
log.Printf("capture %s is ready", e.Data.ID)
case *wsw.AutoScheduleCaptureFailedEvent:
log.Printf("%s failed with HTTP %d", e.Data.ScheduleName, e.Data.PageHTTPStatus)
case *wsw.UnknownEvent:
log.Printf("ignoring %s", e.Type)
}
w.WriteHeader(http.StatusNoContent)
})
type EventEnvelope ¶
type EventEnvelope struct {
// ID identifies the delivery; retries of the same delivery share it.
ID string `json:"id"`
Type string `json:"type"`
CreatedAt time.Time `json:"created_at"`
Product Product `json:"product"`
}
EventEnvelope is the part of a delivery common to every event.
type Format ¶
type Format string
Format is the file format of a capture. It must belong to the capture's MediaCategory.
const ( FormatPNG Format = "png" FormatJPEG Format = "jpeg" FormatWebP Format = "webp" FormatTIFF Format = "tiff" FormatMP4 Format = "mp4" FormatWebM Format = "webm" FormatMOV Format = "mov" FormatGIF Format = "gif" FormatImagePDF Format = "image_pdf" FormatDocumentPDF Format = "document_pdf" )
Formats, grouped by media category.
type Geolocation ¶
type Geolocation struct {
Latitude *float64 `json:"latitude,omitempty"`
Longitude *float64 `json:"longitude,omitempty"`
// Accuracy is in metres, from 1 to 10000; the default is 100.
Accuracy *float64 `json:"accuracy,omitempty"`
}
Geolocation is the position the page sees. Latitude and longitude are sent together or not at all.
type InstantSnapCaptureFailedEvent ¶
type InstantSnapCaptureFailedEvent struct {
EventEnvelope
Data Capture `json:"data"`
}
InstantSnapCaptureFailedEvent is sent when an Instant Snap capture with a callback URL fails. Data.ErrorCode says why.
type InstantSnapCaptureSucceededEvent ¶
type InstantSnapCaptureSucceededEvent struct {
EventEnvelope
Data Capture `json:"data"`
}
InstantSnapCaptureSucceededEvent is sent when an Instant Snap capture with a callback URL succeeds. Data has no download URLs; call InstantSnapService.Get for them.
type InstantSnapService ¶
type InstantSnapService struct {
// contains filtered or unexported fields
}
InstantSnapService takes captures on demand. Use it through Client.InstantSnap.
func (*InstantSnapService) Capture ¶
func (s *InstantSnapService) Capture(ctx context.Context, opts *CaptureOptions, reqOpts ...RequestOption) (*Capture, error)
Capture captures a page and answers with the capture, whose DownloadURL is valid for an hour. It can take up to 120 seconds.
Example ¶
client := wsw.NewClient()
ctx := context.Background()
capture, err := client.InstantSnap.Capture(ctx, &wsw.CaptureOptions{
URL: "https://example.com",
Format: wsw.FormatWebP,
ScrollMode: wsw.ScrollModeScroll,
Viewport: &wsw.Viewport{Width: wsw.Int(1440), Height: wsw.Int(900)},
BlockAds: wsw.Bool(false),
Headers: map[string]string{"X-Preview": "1"},
})
if err != nil {
log.Fatal(err)
}
fmt.Println(capture.ID, capture.Width, capture.Height)
if err := client.InstantSnap.DownloadToFile(ctx, capture, "example.webp"); err != nil {
log.Fatal(err)
}
func (*InstantSnapService) CaptureFile ¶
func (s *InstantSnapService) CaptureFile(ctx context.Context, opts *CaptureOptions, reqOpts ...RequestOption) (*CaptureFile, error)
CaptureFile captures a page and answers with the file itself, up to 64 MB. The caller must read and close the returned file's Body, or call CaptureFile.SaveTo. The file is not kept by the API.
Example ¶
client := wsw.NewClient()
file, err := client.InstantSnap.CaptureFile(context.Background(), &wsw.CaptureOptions{
URL: "https://example.com",
MediaCategory: wsw.MediaCategoryPDF,
Format: wsw.FormatDocumentPDF,
})
if err != nil {
log.Fatal(err)
}
if err := file.SaveTo(file.Filename); err != nil {
log.Fatal(err)
}
func (*InstantSnapService) CaptureToBucket ¶
func (s *InstantSnapService) CaptureToBucket(ctx context.Context, opts *CaptureOptions, reqOpts ...RequestOption) (*Capture, error)
CaptureToBucket captures a page and uploads the file to the bucket in opts.S3Delivery. The answer's S3Delivery reports the upload.
Example ¶
client := wsw.NewClient()
capture, err := client.InstantSnap.CaptureToBucket(context.Background(), &wsw.CaptureOptions{
URL: "https://example.com",
S3Delivery: &wsw.S3DeliveryOptions{
AccessKeyID: os.Getenv("AWS_ACCESS_KEY_ID"),
SecretAccessKey: os.Getenv("AWS_SECRET_ACCESS_KEY"),
Bucket: "captures",
Region: wsw.String("eu-west-2"),
KeyPrefix: wsw.String("homepage/"),
},
}, wsw.WithIdempotencyKey("homepage-2026-09-26"))
if err != nil {
log.Fatal(err)
}
fmt.Println(capture.S3Delivery.Status)
func (*InstantSnapService) Catalog ¶
func (s *InstantSnapService) Catalog(ctx context.Context) (*Catalog, error)
Catalog lists the devices CaptureOptions.EmulatedDevice accepts and the countries Proxy.Country accepts.
func (*InstantSnapService) Defaults ¶
func (s *InstantSnapService) Defaults(ctx context.Context) (*Defaults, error)
Defaults returns the values a capture takes for the options it does not send, and the limits it is checked against. It needs no key.
func (*InstantSnapService) Download ¶
func (s *InstantSnapService) Download(ctx context.Context, capture *Capture, w io.Writer) (int64, error)
Download writes the file of a capture to w and returns the number of bytes written. The capture needs a DownloadURL, which only InstantSnapService.Capture, InstantSnapService.Get and InstantSnapService.Wait return; it is valid for an hour.
func (*InstantSnapService) DownloadToFile ¶
func (s *InstantSnapService) DownloadToFile(ctx context.Context, capture *Capture, path string) error
DownloadToFile writes the file of a capture to path, replacing any file there. Nothing is left at path if the download fails.
func (*InstantSnapService) Submit ¶
func (s *InstantSnapService) Submit(ctx context.Context, opts *CaptureOptions, reqOpts ...RequestOption) (*Capture, error)
Submit accepts a capture to run when a browser is free and answers at once with the capture, whose Status is CaptureStatusQueued. Read it back with InstantSnapService.Get, wait for it with InstantSnapService.Wait, or receive the webhook. opts.Timeout bounds the browser time the capture may use. A capture still queued 15 minutes after it was accepted ends failed with the error code capture_expired and is not charged.
The request is bounded by the ordinary timeout (see WithTimeout), not the capture timeout, since no capture runs while the caller waits. A queue_full refusal means the organization already has as many async captures waiting or running as it may; it is a KindRateLimit error carrying RetryAfter and is not retried.
Example ¶
client := wsw.NewClient()
ctx := context.Background()
queued, err := client.InstantSnap.Submit(ctx, &wsw.CaptureOptions{
URL: "https://example.com",
Timeout: wsw.Int(240),
})
if err != nil {
log.Fatal(err)
}
capture, err := client.InstantSnap.Wait(ctx, queued.ID)
if err != nil {
log.Fatal(err)
}
switch capture.Status {
case wsw.CaptureStatusSucceeded:
fmt.Println(capture.DownloadURL)
case wsw.CaptureStatusQueued, wsw.CaptureStatusRunning:
fmt.Println("still running; wait again later")
default:
fmt.Println("ended", capture.Status, capture.ErrorCode)
}
func (*InstantSnapService) SubmitToBucket ¶
func (s *InstantSnapService) SubmitToBucket(ctx context.Context, opts *CaptureOptions, reqOpts ...RequestOption) (*Capture, error)
SubmitToBucket is InstantSnapService.Submit for a capture uploaded to the bucket in opts.S3Delivery, as InstantSnapService.CaptureToBucket does.
func (*InstantSnapService) Wait ¶
func (s *InstantSnapService) Wait(ctx context.Context, id string, opts ...WaitOption) (*Capture, error)
Wait reads a capture with InstantSnapService.Get until its status is no longer queued, running or pending, and returns it. It is meant for a capture from InstantSnapService.Submit or InstantSnapService.SubmitToBucket.
When the wait timeout (see WithWaitTimeout) runs out first, Wait returns the capture as last read, still unfinished, and no error, so the caller can decide whether to wait again. An error from Get is returned as it is. A context deadline or cancellation stops the wait at once with an error that matches ctx.Err().
type MediaCategory ¶
type MediaCategory string
MediaCategory is what kind of file a capture produces.
const ( MediaCategoryImage MediaCategory = "image" MediaCategoryVideo MediaCategory = "video" MediaCategoryPDF MediaCategory = "pdf" )
Media categories.
type PageSource ¶
type PageSource struct {
Format PageSourceFormat `json:"format"`
ContentType string `json:"content_type"`
SizeBytes int64 `json:"size_bytes"`
// SHA256 is the lowercase hex digest of the source.
SHA256 string `json:"sha256"`
// DownloadURL is short-lived and reissued on every read. It is empty in webhooks.
DownloadURL string `json:"download_url"`
}
PageSource is the page source kept alongside a capture.
type PageSourceFormat ¶
type PageSourceFormat string
PageSourceFormat is the form in which a capture keeps the page source.
const ( PageSourceFormatHTML PageSourceFormat = "html" PageSourceFormatJSON PageSourceFormat = "json" PageSourceFormatText PageSourceFormat = "text" )
Page source formats.
type PageSourceOptions ¶
type PageSourceOptions struct {
// Format defaults to html.
Format PageSourceFormat `json:"format,omitempty"`
}
PageSourceOptions also keeps the page source, capped at 10 MB and charged as a second capture. It is not available with InstantSnapService.CaptureFile.
type Ping ¶
type Ping struct {
EndpointID string `json:"endpoint_id"`
EndpointName string `json:"endpoint_name"`
Message string `json:"message"`
SentAt time.Time `json:"sent_at"`
}
Ping is the data of a test delivery.
type PingEvent ¶
type PingEvent struct {
EventEnvelope
Data Ping `json:"data"`
}
PingEvent is a test delivery sent from the dashboard, by either product.
type Proxy ¶
type Proxy struct {
// Country is one of the proxy countries in [InstantSnapService.Catalog].
Country string `json:"country"`
}
Proxy sends the capture through a proxy in another country. It needs the ProxyLocation plan feature.
type RequestOption ¶
type RequestOption func(*requestConfig)
RequestOption configures a single call.
func WithIdempotencyKey ¶
func WithIdempotencyKey(key string) RequestOption
WithIdempotencyKey sends key as the capture's Idempotency-Key instead of a new random one. A second capture with the same key within 24 hours returns the first capture rather than capturing again. The key is up to 255 printable ASCII characters.
type ResponseType ¶
type ResponseType string
ResponseType is how the API answered a capture. The capture methods choose it; it is not an option.
const ( ResponseTypeJSON ResponseType = "json" ResponseTypeByFormat ResponseType = "by_format" ResponseTypeCloudStorage ResponseType = "cloud_storage" )
Response types.
type S3Delivery ¶
type S3Delivery struct {
Status S3DeliveryStatus `json:"status"`
Bucket string `json:"bucket"`
Key string `json:"key"`
// AccessKeyID is masked to its last four characters. It is empty in webhooks.
AccessKeyID string `json:"access_key_id"`
Error string `json:"error"`
}
S3Delivery is the state of the upload to the caller's bucket.
type S3DeliveryOptions ¶
type S3DeliveryOptions struct {
AccessKeyID string `json:"access_key_id"`
SecretAccessKey string `json:"secret_access_key"`
Bucket string `json:"bucket"`
Region *string `json:"region,omitempty"`
// EndpointURL is for providers other than AWS: https only, and not a private address.
EndpointURL *string `json:"endpoint_url,omitempty"`
ForcePathStyle *bool `json:"force_path_style,omitempty"`
KeyPrefix *string `json:"key_prefix,omitempty"`
// WaitForSync waits up to 30 seconds for the upload to finish before answering.
WaitForSync *bool `json:"wait_for_sync,omitempty"`
}
S3DeliveryOptions is an S3-compatible bucket to upload the capture to. It needs Region or EndpointURL.
type S3DeliveryStatus ¶
type S3DeliveryStatus string
S3DeliveryStatus is how far the upload to the caller's bucket has got.
const ( S3DeliveryStatusNotStarted S3DeliveryStatus = "not_started" S3DeliveryStatusInProgress S3DeliveryStatus = "in_progress" S3DeliveryStatusCompleted S3DeliveryStatus = "completed" S3DeliveryStatusFailed S3DeliveryStatus = "failed" )
S3 delivery statuses.
type SchedulePause ¶
type SchedulePause struct {
ScheduleID string `json:"schedule_id"`
ScheduleName string `json:"schedule_name"`
URL string `json:"url"`
Reason string `json:"reason"`
PausedAt time.Time `json:"paused_at"`
}
SchedulePause says which schedule stopped and why.
type SchedulePausedEvent ¶
type SchedulePausedEvent struct {
EventEnvelope
Data SchedulePause `json:"data"`
}
SchedulePausedEvent is sent when a schedule stops running, for example because the plan's quota is exhausted.
type ScheduledCapture ¶
type ScheduledCapture struct {
ID string `json:"id"`
// ScheduleID is the web page's id, the one [AutoScheduleService.TriggerNow] takes.
ScheduleID string `json:"schedule_id"`
ScheduleName string `json:"schedule_name"`
URL string `json:"url"`
Status string `json:"status"`
MediaCategory MediaCategory `json:"media_category"`
Format Format `json:"format"`
PageHTTPStatus int `json:"page_http_status"`
CapturedAt time.Time `json:"captured_at"`
FinalURL string `json:"final_url"`
MediaSHA256 string `json:"media_sha256"`
// PageSource carries only Format and SHA256, and is nil when no source was kept.
PageSource *PageSource `json:"page_source"`
// SitemapRunID is set when the capture was part of a sitemap run.
SitemapRunID string `json:"sitemap_run_id"`
}
ScheduledCapture is an Auto Schedule capture as webhooks describe it.
type ScrollMode ¶
type ScrollMode string
ScrollMode chooses between the visible viewport and the full page.
const ( ScrollModeViewport ScrollMode = "viewport" ScrollModeScroll ScrollMode = "scroll" )
Scroll modes.
type Selectors ¶
type Selectors struct {
// XPath limits the capture to the element it selects. At most 512 characters.
XPath *string `json:"xpath,omitempty"`
// Hide is up to 25 comma-separated CSS selectors (an XPath starting with // works too) of
// elements to hide, at most 2000 characters. Each is applied on its own; one the browser
// rejects is skipped.
Hide *string `json:"hide,omitempty"`
// Click is a CSS selector of an element to click. At most 512 characters.
Click *string `json:"click,omitempty"`
// Hover is a CSS selector of an element to hover over. At most 512 characters.
Hover *string `json:"hover,omitempty"`
}
Selectors act on parts of the page before it is captured.
type SitemapRun ¶
type SitemapRun struct {
SitemapRunID string `json:"sitemap_run_id"`
ScheduleID string `json:"schedule_id"`
ScheduleName string `json:"schedule_name"`
SitemapURL string `json:"sitemap_url"`
Status string `json:"status"`
RunAt time.Time `json:"run_at"`
CompletedAt time.Time `json:"completed_at"`
PagesListed int `json:"pages_listed"`
PagesMatched int `json:"pages_matched"`
PagesQueued int `json:"pages_queued"`
PagesSucceeded int `json:"pages_succeeded"`
PagesFailed int `json:"pages_failed"`
PagesOverLimit int `json:"pages_over_limit"`
PagesOverQuota int `json:"pages_over_quota"`
Message string `json:"message"`
}
SitemapRun is the outcome of a run over a sitemap's pages.
type SitemapRunCompletedEvent ¶
type SitemapRunCompletedEvent struct {
EventEnvelope
Data SitemapRun `json:"data"`
}
SitemapRunCompletedEvent is sent when a run over a sitemap's pages ends.
type TimestampWatermark ¶
type TimestampWatermark struct {
Enabled *bool `json:"enabled,omitempty"`
Location WatermarkLocation `json:"location,omitempty"`
// FontColor is a hex colour; the default is #000000.
FontColor *string `json:"font_color,omitempty"`
// BackgroundColor is a hex colour; the default is #FFFFFF.
BackgroundColor *string `json:"background_color,omitempty"`
}
TimestampWatermark adds a strip with the capture time and URL. Sending the block, even empty, turns it on. It is not available for video.
type TriggerResult ¶
type TriggerResult struct {
Status TriggerStatus `json:"status"`
WebPageID string `json:"web_page_id"`
TriggeredBy TriggeredBy `json:"triggered_by"`
// TriggeredAt is when the capture was queued; for an already pending capture, when it was
// first triggered.
TriggeredAt time.Time `json:"triggered_at"`
// Message is one sentence for a person.
Message string `json:"message"`
}
TriggerResult is the answer of AutoScheduleService.TriggerNow.
type TriggerStatus ¶
type TriggerStatus string
TriggerStatus is the outcome of AutoScheduleService.TriggerNow.
const ( TriggerStatusTriggered TriggerStatus = "triggered" TriggerStatusAlreadyPending TriggerStatus = "already_pending" )
Trigger statuses. TriggerStatusAlreadyPending means a capture for the web page was already queued or running: nothing new was started and nothing was charged.
type TriggeredBy ¶
type TriggeredBy string
TriggeredBy says who started a pending Auto Schedule capture.
const ( TriggeredByAPI TriggeredBy = "api" TriggeredByUI TriggeredBy = "ui" )
Trigger sources.
type UnknownEvent ¶
type UnknownEvent struct {
EventEnvelope
// Data is the event's data as sent.
Data json.RawMessage `json:"data"`
}
UnknownEvent is an event whose type this version of the library does not know.
type VerifyOption ¶
type VerifyOption func(*verifyConfig)
VerifyOption configures VerifyWebhook.
func WithNow ¶
func WithNow(now func() time.Time) VerifyOption
WithNow sets the clock the timestamp is checked against, for tests.
func WithTolerance ¶
func WithTolerance(d time.Duration) VerifyOption
WithTolerance sets how far the signature's timestamp may be before or after now. Values of zero or less keep the default, DefaultWebhookTolerance.
type Viewport ¶
type Viewport struct {
// Width is from 250 to 7680; the default is 1920.
Width *int `json:"width,omitempty"`
// Height is from 250 to 7680; the default is 1080.
Height *int `json:"height,omitempty"`
// DeviceScaleFactor is the device pixel ratio, from 1 to 3; the default is 1.
DeviceScaleFactor *float64 `json:"device_scale_factor,omitempty"`
}
Viewport sets the browser window, in CSS pixels.
type ViewportDefaults ¶
type ViewportDefaults struct {
Width int `json:"width"`
Height int `json:"height"`
DeviceScaleFactor float64 `json:"device_scale_factor"`
}
ViewportDefaults is the default viewport.
type WaitOption ¶
type WaitOption func(*waitConfig)
WaitOption configures InstantSnapService.Wait.
func WithPollInterval ¶
func WithPollInterval(d time.Duration) WaitOption
WithPollInterval sets the pause between reads. Zero or less does not pause. The default is DefaultPollInterval.
func WithWaitTimeout ¶
func WithWaitTimeout(d time.Duration) WaitOption
WithWaitTimeout sets how long Wait keeps reading before it returns the capture unfinished. Zero or less reads it once. The default is DefaultWaitTimeout.
type WatermarkLocation ¶
type WatermarkLocation string
WatermarkLocation is where the timestamp watermark strip goes.
const ( WatermarkLocationTop WatermarkLocation = "top" WatermarkLocationBottom WatermarkLocation = "bottom" )
Watermark locations.