azuretls

README

AzureTLS Client

📖 Introduction

Welcome to AzureTLS Client, a robust and flexible HTTP client library for Golang designed with security and customization in mind. Whether you're building a web scraper, an API client, or any application that requires HTTP requests, AzureTLS Client provides a suite of features to meet your needs.

Why AzureTLS Client?

• Security: With built-in SSL pinning and proxy support, AzureTLS Client takes security seriously, ensuring your connections are secure and reliable.

• Customization: Modify TLS ClientHello fingerprints, configure HTTP/2 settings, and even set ordered headers. AzureTLS Client is built to be as flexible as your project requires.

• Performance: Built on top of Golang's native packages, AzureTLS Client is designed for speed and efficiency, making it suitable for both small and large-scale applications.

Whether you're a seasoned developer looking for a feature-rich HTTP client or you're just getting started with Golang, AzureTLS Client offers a blend of performance, customization, and security to help you build better applications.

🌟 Features

• Latest Chrome ClientHello Support
• HTTP/1.1 and HTTP/2 Compatibility
• Customizable ClientHello TLS (JA3 strings and extensions)
• Configurable HTTP/2 Frames (SETTINGS, PRIORITY, WINDOW_UPDATE)
• Built-in Proxy Support
• SSL Pinning
• PreHook and Callback Functions
• Integrated Cookie Jar
• Websocket with JA3

📑 Table of Contents

• Table of Contents
• Dependencies
• Installation
• Usage
  • Create a Session
  • Make Requests
    • GET
    • POST
    • PUT
    • PATCH
    • DELETE
    • OPTIONS
    • HEAD
    • CONNECT
  • Modify TLS Client Hello (JA3)
  • Modify HTTP2
  • Headers
  • Proxy
  • SSL Pinning
  • Timeout
  • PreHook and CallBack
  • Cookies
  • Websocket
  • Utils
    • Response to JSON
    • Url encode
  • Dump
  • Log

Dependencies

golang ^1.22

Installation

$ go get github.com/Noooste/azuretls-client

Usage

import (
    "github.com/Noooste/azuretls-client"
)

Create a Session

// without context
session := azuretls.NewSession()
// don't forget to close the session when you no longer need it, to free up resources
defer session.Close() 

// or with context
session := azuretls.NewSessionWithContext(context.Background())
defer session.Close()

Predefined browsers/devices

Some browsers/devices are already defined in the module:

• Chrome
• Firefox
• Opera
• Safari
• Edge
• Ios

In order to use them, you can simply do:

// without context
session := azuretls.NewSession()
// don't forget to close the session when you no longer need it, to free up resources
defer session.Close() 

session.Browser = azuretls.Firefox // JA3 and HTTP2 specifications will be automatically set

Make Requests

REQUEST ARGUMENTS

You can pass arguments to the request methods. Valid arguments are:

• azuretls.OrderedHeaders: for ordered headers (of type [][]string)
• http.Header: for headers (of type map[string]string)
• azuretls.HeaderOrder: for http.Header order (of type []string)
• time.Duration: for the request timeout

REQUEST REDIRECTS

By default, the azuretls client follows redirects. If you want to disable it, you can do request.DisableRedirects = true. Otherwise, it will follow redirects until the MaxRedirects limit is reached (default: 10). You can modify the maximum number of redirects with session.MaxRedirects or request.MaxRedirects.

GET

session := azuretls.NewSession()
defer session.Close()

response, err := session.Get("https://tls.peet.ws/api/all")

if err != nil {
    panic(err)
}

fmt.Println(response.StatusCode, string(response.Body))

To do a POST, PUT or PATCH requests, you can use as body:

• string
• []byte
• io.Reader
• anything that can be marshalled to JSON

POST

session := azuretls.NewSession()
defer session.Close()

response, err := session.Post("https://tls.peet.ws/api/all", `{"test": "test"}`)
// or
response, err := session.Post("https://tls.peet.ws/api/all", map[string]string{"test": "test"})
// or
response, err := session.Post("https://tls.peet.ws/api/all", []byte(`{"test": "test"}`))

PUT

session := azuretls.NewSession()
defer session.Close()

// the body follows the same semantics as the POST request.
response, err := session.Put("https://tls.peet.ws/api/all", `{"test": "test"}`)

PATCH

session := azuretls.NewSession()
defer session.Close()

// the body follows the same semantics as the POST request.
response, err := session.Patch("https://tls.peet.ws/api/all", `{"test": "test"}`)

DELETE

session := azuretls.NewSession()
defer session.Close()

response, err := session.Delete("https://tls.peet.ws/api/all")

OPTIONS

session := azuretls.NewSession()
defer session.Close()

response, err := session.Options("https://tls.peet.ws/api/all")

HEAD

session := azuretls.NewSession()
defer session.Close()

response, err := session.Head("https://tls.peet.ws/api/all")

CONNECT

session.Connect is a method that allows you to connect to a website without sending any HTTP request. It initiates the TLS handshake and the HTTP connection. This ensures that the server connection is made first, to avoid having to make these connections during the next requests.

session := azuretls.NewSession()
defer session.Close()

response, err := session.Connect("https://tls.peet.ws/api/all")

Modify TLS Client Hello (JA3)

To modify your ClientHello, you have two options:

• The first one is to use the session.ApplyJA3 method, which takes a JA3 fingerprint and the target browser (chrome, firefox, safari, ...).
• The second one is to assign a method to session.GetClientHelloSpec that returns TLS configuration.

You can retrieve your JA3 fingerprint there : tls.peet.ws

session := azuretls.NewSession()
defer session.Close()

// First method
if err := session.ApplyJa3("771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,45-13-43-0-16-65281-51-18-11-27-35-23-10-5-17513-21,29-23-24-25-26,0", azuretls.Chrome); err != nil {
    panic(err)
}

// Second method
session.GetClientHelloSpec = azuretls.GetLastChromeVersion //func() *tls.ClientHelloSpec

response, err := session.Get("https://tls.peet.ws/api/all")

if err != nil {
    panic(err)
}

// response *azuretls.Response
fmt.Println(response.StatusCode, string(response.Body))

Modify HTTP2

To modify HTTP2, you have to apply the HTTP2 fingerprint to the session. You can retrieve your HTTP/2 fingerprint there : tls.peet.ws

session := azuretls.NewSession()
defer session.Close()

if err := session.ApplyHTTP2("1:65536,2:0,3:1000,4:6291456,6:262144|15663105|0|m,s,a,p"); err != nil {
    panic(err)
}

response, err := session.Get("https://tls.peet.ws/api/all")

if err != nil {
    panic(err)
}

fmt.Println(response.StatusCode, string(response.Body))

Headers

You can define headers for the session with the session.Header attribute, or use the session.OrderedHeaders attribute to maintain header order.

session := azuretls.NewSession()
defer session.Close()

// it will keep the order
session.OrderedHeaders = azuretls.OrderedHeaders{
    {"user-agent", "test"},
    {"content-type", "application/json"},
    {"accept", "application/json"},
}

response, err = session.Get("https://tls.peet.ws/api/all")

if err != nil {
    panic(err)
}

fmt.Println(response.StatusCode, string(response.Body))

This also works if you pass azuretls.Header (or azuretls.OrderedHeaders) as arguments to the request methods.

session := azuretls.NewSession()
defer session.Close()

headers := azuretls.OrderedHeaders{
    {"user-agent", "test"},
    {"content-type", "application/json"},
    {"accept", "application/json"},
}

response, err = session.Get("https://tls.peet.ws/api/all", headers)

if err != nil {
    panic(err)
}

NOTE: For azuretls.OrderedHeaders, if you specify only the key, the order will be applied.

session := azuretls.NewSession()
defer session.Close()

headers := azuretls.OrderedHeaders{
    {"Host"}, // it will only apply the order of the Host header
    {"user-agent", "test"},
    {"content-type", "application/json"},
    {"accept", "application/json"},
}

response, err = session.Get("https://tls.peet.ws/api/all", headers)

if err != nil {
    panic(err)
}

Proxy

You can set a proxy to the session with the session.SetProxy method.

If the proxy needs to be cleared, you can do session.ClearProxy.

Supported proxy formats include:

• http(s)://ip
• http(s)://ip:port
• http(s)://username:password@ip:port
• socks5(h)://ip
• socks5(h)://ip:port
• socks5(h)://username:password@ip:port
• ip
• ip:port
• ip:port:username:password
• username:password:ip:port
• username:password@ip:port

If a scheme is not provided, http will be used by default.

If you need to use IPV6 proxies, please format it correctly with the appropriate scheme, for example: http://user:pass@[2001:0db8:85a3:0000:0000:8a2e:0370:7334]:8080

session := azuretls.NewSession()
defer session.Close()

if err := session.SetProxy("http://username:password@ip:port"); err != nil {
    panic(err)
}

response, err := session.Get("https://api.ipify.org")

if err != nil {
    panic(err)
}

fmt.Println(response.StatusCode, string(response.Body))

SSL Pinning

SSL pinning is enabled by default.

SSL pinning ensures that you are connecting to the intended server and mitigates the risk of man-in-the-middle attacks, such as those potentially executed using tools like Charles Proxy.

session := azuretls.NewSession()
defer session.Close()

// secured request
response, err := session.Get("https://tls.peet.ws/api/all")

if err != nil {
    panic(err)
}

fmt.Println(response.StatusCode, string(response.Body))

If you're concerned about the reliability of a server, you can improve security by manually setting pins prior to initiating any requests within the session, using the session.AddPins method. The pins are generated through the following series of steps:

1. SubjectPublicKeyInfo is first DER-encoded.
2. The DER-encoded data is then hashed using the SHA-256 algorithm.
3. Finally, the hashed output is base64 encoded to generate the pin.

It's not necessary for every certificate in the chain to match a pin. If even a single certificate in the chain matches one of the pre-defined pins, the entire chain is considered valid : this approach allows for flexibility in the certificate chain while still providing an additional layer of security.

session := azuretls.NewSession()
defer session.Close()

session.AddPins(&url.URL{
        Scheme: "https",
        Host:   "httpbin.org",
    }, []string{
        "j5bzD/UjYVE+0feXsngcrVs3i1vSaoOOtPgpLBb9Db8=",
        "18tkPyr2nckv4fgo0dhAkaUtJ2hu2831xlO2SKhq8dg=",
        "++MBgDH5WGvL9Bcn5Be30cRcL0f5O+NyoXuWtQdX1aI=",
        "KwccWaCgrnaw6tsrrSO61FgLacNgG2MMLq8GE6+oP5I=",
})

_, err := session.Get("https://httpbin.org/get")

if err != nil {
    panic(err)
}

You can also call session.ClearPins beforehand to remove any saved pins in the session for the given URL

To disable SSL Pinning, you can do session.InsecureSkipVerify = true

session := azuretls.NewSession()
defer session.Close()

session.InsecureSkipVerify = true

// do it at your own risk
response, err := session.Get("https://tls.peet.ws/api/all")

if err != nil {
    panic(err)
}

fmt.Println(response.StatusCode, string(response.Body))

Timeout

You can set a timeout to the session with the session.SetTimeout method.

session := azuretls.NewSession()
defer session.Close()

session.SetTimeout(5 * time.Second)

response, err := session.Get("https://tls.peet.ws/api/all")

if err != nil {
    panic(err)
}

fmt.Println(response.StatusCode, string(response.Body))

PreHook and CallBack

You can use the session.PreHook method to modify all outgoing requests in the session before they are executed. You can also set a callback for the session using the session.CallBack method which will be called just after the response received.

session := azuretls.NewSession()
defer session.Close()

session.PreHook = func(request *azuretls.Request) error {
    request.Header.Set("user-agent", "test")
    return nil
}

session.CallBack = func(request *azuretls.Request, response *azuretls.Response, err error) error {
    fmt.Println(response.StatusCode, string(response.Body))
    return nil
}

response, err := session.Get("https://tls.peet.ws/api/all")

if err != nil {
    panic(err)
}

Cookies

You can manage cookies with the jar of the session. Note that azuretls automatically manage cookies when doing requests.

session := azuretls.NewSession()
defer session.Close()

parsed, err := url.Parse("https://tls.peet.ws/api/all")

session.CookieJar.SetCookies(parsed, []*http.Cookie{
    {
        Name:  "test",
        Value: "test",
    },
})

response, err := session.Get("https://tls.peet.ws/api/all")

if err != nil {
    panic(err)
}

Websocket

You can use websocket with session.NewWebsocket method.

session := azuretls.NewSession()
defer session.Close()

ws, err := session.NewWebsocket("wss://demo.piesocket.com/v3/channel_123?api_key=VCXCEuvhGcBDP7XhiJJUDvR1e1D3eiVjgZ9VRiaV&notify_self", 1024, 1024,
		azuretls.OrderedHeaders{
			{"User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36"},
		},
})
if err = ws.WriteJSON(map[string]string{
  "event": "new_message",
}); err != nil {
  panic(err)
}

Utils

Response to JSON

You can unmarshal the response body (JSON format) into a struct with the response.JSON or response.MustJSON methods.

session := azuretls.NewSession()
defer session.Close()

response, err := session.Get("https://tls.peet.ws/api/all")

if err != nil {
    panic(err)
}

var data map[string]any

if err := response.JSON(&data); err != nil {
    panic(err)
}

fmt.Println(data)

Url encode

You can convert a struct into an url encoded string (used for urls or application/x-www-form-urlencoded) with the azuretls.UrlEncode method.

session := azuretls.NewSession()
defer session.Close()

type Foo struct {
	Bar string `url:"bar"`
	Baz string `url:"baz"`
}

body := azuretls.UrlEncode(Foo{
	Bar: "bar",
	Baz: "baz baz baz",
})

response, err := session.Post("https://tls.peet.ws/api/all", body)

if err != nil {
    panic(err)
}

fmt.Println(response.StatusCode, string(response.Body))

Dump

You can dump the request and response with the session.Dump method.

session := azuretls.NewSession()

session.Dump("./my_dump_dir", 
    "/any/path/to/ignore", 
    "can.ignore.this", 
    "*.all.subdomains",
)

session.Get("https://www.google.com")
// the request and response dump will be in the "my_dump_dir" directory.

Log

You can log the request and response with the session.Log method. This will display the request and response in the console.

session := azuretls.NewSession()

session.Log( 
    "/any/path/to/ignore", 
    "can.ignore.this", 
    "*.all.subdomains",
)

session.Get("https://www.google.com")

Documentation

Index

• Constants
• Variables
• func CookiesToString(cookies []*http.Cookie) string
• func Fingerprint(c *x509.Certificate) string
• func GetBrowserClientHelloFunc(browser string) func() *tls.ClientHelloSpec
• func GetCookiesMap(cookies []*http.Cookie) map[string]string
• func GetLastChromeVersion() *tls.ClientHelloSpec
• func GetLastFirefoxVersion() *tls.ClientHelloSpec
• func GetLastIosVersion() *tls.ClientHelloSpec
• func RedirectBehavior(reqMethod string, resp *Response, ireq *Request) (redirectMethod string, shouldRedirect, includeBody bool)
• func RefererForURL(ireq, newReq *url.URL) string
• func ToBytes(b any) []byte
• func ToReader(b any) (io.Reader, error)
• func UrlEncode(obj any) string
• type Context
  • func (c *Context) Context() context.Context
• type ContextKeyHeader
• type HeaderOrder
• type OrderedHeaders
  • func (oh *OrderedHeaders) Add(field string, value ...string)
  • func (oh *OrderedHeaders) Clone() OrderedHeaders
  • func (oh *OrderedHeaders) Del(field string) OrderedHeaders
  • func (oh *OrderedHeaders) Get(field string) string
  • func (oh *OrderedHeaders) Remove(field string) OrderedHeaders
  • func (oh *OrderedHeaders) Set(field string, value ...string)
  • func (oh *OrderedHeaders) ToHeader() http.Header
• type PHeader
  • func GetDefaultPseudoHeaders() PHeader
• type PinManager
  • func NewPinManager() *PinManager
  • func (p *PinManager) AddPin(pin string)
  • func (p *PinManager) AddPins(pin []string)
  • func (p *PinManager) GetPins() []string
  • func (p *PinManager) New(addr string) (err error)
  • func (p *PinManager) Verify(c *x509.Certificate) bool
• type Request
  • func (r *Request) CloseBody()
  • func (r *Request) Context() context.Context
  • func (r *Request) SetContext(ctx context.Context)
• type Response
  • func (r *Response) CloseBody() error
  • func (r *Response) JSON(v any) error
  • func (r *Response) MustJSON(v any)
  • func (r *Response) ReadBody() (body []byte, err error)
• type Session
  • func NewSession() *Session
  • func NewSessionWithContext(ctx context.Context) *Session
  • func (s *Session) AddPins(u *url.URL, pins []string) error
  • func (s *Session) ApplyHTTP2(fp string) error
  • func (s *Session) ApplyJa3(ja3, navigator string) error
  • func (s *Session) ApplyJa3WithSpecifications(ja3 string, specifications *TlsSpecifications, navigator string) error
  • func (s *Session) ClearPins(u *url.URL) error
  • func (s *Session) ClearProxy()
  • func (s *Session) Close()
  • func (s *Session) Connect(u string) error
  • func (s *Session) Context() context.Context
  • func (s *Session) Delete(url string, args ...any) (*Response, error)
  • func (s *Session) DisableDump()
  • func (s *Session) DisableLog()
  • func (s *Session) Do(request *Request, args ...any) (*Response, error)
  • func (s *Session) Dump(dir string, uris ...string) error
  • func (s *Session) DumpAndLog(dir string, uris ...string) error
  • func (s *Session) DumpIgnore(uri string) bool
  • func (s *Session) EnableDump()
  • func (s *Session) EnableLog()
  • func (s *Session) EnableVerbose(path string, ignoreHost []string) error
  • func (s *Session) Get(url string, args ...any) (*Response, error)
  • func (s *Session) Head(url string, args ...any) (*Response, error)
  • func (s *Session) InitTransport(browser string) (err error)
  • func (s *Session) Ip() (ip string, err error)
  • func (s *Session) Log(uris ...string)
  • func (s *Session) LogIgnore(uri string) bool
  • func (s *Session) NewWebsocket(url string, readBufferSize, writeBufferSize int, args ...any) (*Websocket, error)
  • func (s *Session) NewWebsocketWithContext(ctx context.Context, url string, readBufferSize, writeBufferSize int, ...) (*Websocket, error)
  • func (s *Session) Options(url string, args ...any) (*Response, error)
  • func (s *Session) Patch(url string, data any, args ...any) (*Response, error)
  • func (s *Session) Pin(addr string) error
  • func (s *Session) Post(url string, data any, args ...any) (*Response, error)
  • func (s *Session) Put(url string, data any, args ...any) (*Response, error)
  • func (s *Session) SetContext(ctx context.Context)
  • func (s *Session) SetProxy(proxy string) error
  • func (s *Session) SetTimeout(timeout time.Duration)
• type TlsSpecifications
  • func DefaultTlsSpecifications(navigator string) *TlsSpecifications
• type Websocket

Examples

• NewSession
• Session.ApplyHTTP2
• Session.ApplyJa3
• Session.Dump
• Session.DumpIgnore
• Session.Get
• Session.Log
• Session.Post
• Session.SetProxy
• ToBytes
• UrlEncode

Constants

const (
	Chrome  = "chrome"
	Firefox = "firefox"
	Opera   = "opera"
	Safari  = "safari"
	Edge    = "edge"
	Ios     = "ios"
	Android = "android" //deprecated
)

const (
	Path      = ":path"
	Method    = ":method"
	Authority = ":authority"
	Scheme    = ":scheme"
)

const (
	SchemeHttp  = "http"
	SchemeHttps = "https"
	SchemeWs    = "ws"
	SchemeWss   = "wss"
	Socks5      = "socks5"
	Socks5H     = "socks5h"
)

Variables

var ErrTooManyRedirects = errors.New("too many redirects")

var ErrUseLastResponse = errors.New("azuretls: use last response")

Functions

func CookiesToString

func CookiesToString(cookies []*http.Cookie) string

func Fingerprint

func Fingerprint(c *x509.Certificate) string

Fingerprint computes the SHA256 Fingerprint of a given certificate's RawSubjectPublicKeyInfo. This is useful for obtaining a consistent identifier for a certificate's public key. The result is then base64-encoded to give a string representation which can be conveniently stored or compared.

func GetBrowserClientHelloFunc

func GetBrowserClientHelloFunc(browser string) func() *tls.ClientHelloSpec

GetBrowserClientHelloFunc returns a function that returns a ClientHelloSpec for a specific browser

func GetCookiesMap

func GetCookiesMap(cookies []*http.Cookie) map[string]string

func GetLastChromeVersion

func GetLastChromeVersion() *tls.ClientHelloSpec

GetLastChromeVersion apply the latest Chrome version Current Chrome version : 127

func GetLastFirefoxVersion

func GetLastFirefoxVersion() *tls.ClientHelloSpec

GetLastFirefoxVersion apply the latest Firefox, version 120+

func GetLastIosVersion

func GetLastIosVersion() *tls.ClientHelloSpec

func RedirectBehavior

func RedirectBehavior(reqMethod string, resp *Response, ireq *Request) (redirectMethod string, shouldRedirect, includeBody bool)

RedirectBehavior describes what should happen when the client encounters a 3xx status code from the server

func RefererForURL

func RefererForURL(ireq, newReq *url.URL) string

RefererForURL returns a referer without any authentication info or an empty string if lastReq scheme is https and newReq scheme is http.

func ToBytes

func ToBytes(b any) []byte

ToBytes converts any type to []byte, it supports string, []byte, io.Reader, strings.Builder and any other type that can be marshaled to json

Example

fmt.Println(string(azuretls.ToBytes("test1")))

fmt.Println(string(azuretls.ToBytes([]byte("test2"))))

buf := bytes.NewBufferString("test3")
fmt.Println(string(azuretls.ToBytes(buf)))

type s struct {
	A string
	B int
}
fmt.Println(string(azuretls.ToBytes(s{"test4", 4})))

Output:

test1
test2
test3
{"A":"test4","B":4}

func ToReader

func ToReader(b any) (io.Reader, error)

func UrlEncode

func UrlEncode(obj any) string

UrlEncode encodes a map[string]string to url encoded string If you want to encode a struct, you will use the `url` tag

Example

type Foo struct {
	Bar       string `url:"bar"`
	Baz       string `url:"baz"`
	Omit      string `url:"-"`
	OmitEmpty string `url:"no,omitempty"`
}

fmt.Println(azuretls.UrlEncode(Foo{
	Bar:       "bar",
	Baz:       "baz baz baz",
	Omit:      "omit",
	OmitEmpty: "",
}))

Output:

bar=bar&baz=baz+baz+baz

Types

type Context

type Context struct {
	// Session is the session associated with the request.
	Session *Session

	// Request is the request being made.
	Request *Request

	// Response is the response received.
	// It can be modified to change the response returned by the request.
	Response *Response

	// Err is the error, if any, that occurred during the request.
	// It can be modified to change the error returned by the request.
	Err error

	// RequestStartTime is the time when the request was started.
	RequestStartTime time.Time
	// contains filtered or unexported fields
}

Context represents the context of a request. It holds the session, request, response, error, and other details associated with the request.

func (*Context) Context

func (c *Context) Context() context.Context

type ContextKeyHeader

type ContextKeyHeader struct{}

type HeaderOrder

type HeaderOrder []string

HeaderOrder is a slice of header names.

type OrderedHeaders

type OrderedHeaders [][]string

OrderedHeaders is a slice of headers.

func (*OrderedHeaders) Add

func (oh *OrderedHeaders) Add(field string, value ...string)

Add adds the value to the field. It appends to any existing values associated with the field.

func (*OrderedHeaders) Clone

func (oh *OrderedHeaders) Clone() OrderedHeaders

Clone returns a copy of the header.

func (*OrderedHeaders) Del

func (oh *OrderedHeaders) Del(field string) OrderedHeaders

Del removes the first instance of the field from the header. If the field is not present, it does nothing.

func (*OrderedHeaders) Get

func (oh *OrderedHeaders) Get(field string) string

Get returns the first value associated with the given field. If the field is not present, it returns an empty string.

func (*OrderedHeaders) Remove

func (oh *OrderedHeaders) Remove(field string) OrderedHeaders

Remove removes the first instance of the field from the header. If the field is not present, it does nothing. Deprecated: Use Del instead.

func (*OrderedHeaders) Set

func (oh *OrderedHeaders) Set(field string, value ...string)

Set sets the field to the given value. It replaces any existing values associated with the field.

func (*OrderedHeaders) ToHeader

func (oh *OrderedHeaders) ToHeader() http.Header

type PHeader

type PHeader []string

PHeader is a slice of pseudo headers.

func GetDefaultPseudoHeaders

func GetDefaultPseudoHeaders() PHeader

GetDefaultPseudoHeaders returns the default pseudo headers.

type PinManager

type PinManager struct {
	// contains filtered or unexported fields
}

PinManager is a concurrency-safe struct designed to manage and verify public key pinning for SSL/TLS certificates. Public key pinning is a security feature which can be used to specify a set of valid public keys for a particular web service, thus preventing man-in-the-middle attacks due to rogue certificates.

func NewPinManager

func NewPinManager() *PinManager

NewPinManager initializes a new instance of PinManager with an empty set of pins. This is the entry point to begin using the pinning functionality.

func (*PinManager) AddPin

func (p *PinManager) AddPin(pin string)

AddPin safely adds a new pin (Fingerprint) to the PinManager. If a service's certificate changes (e.g., due to renewal), new pins should be added to continue trusting the service.

func (*PinManager) AddPins

func (p *PinManager) AddPins(pin []string)

func (*PinManager) GetPins

func (p *PinManager) GetPins() []string

func (*PinManager) New

func (p *PinManager) New(addr string) (err error)

New establishes a connection to the provided address, retrieves its SSL/TLS certificates, and pins their public keys in the PinManager. This can be used initially to populate the PinManager with pins from a trusted service.

func (*PinManager) Verify

func (p *PinManager) Verify(c *x509.Certificate) bool

Verify checks whether a given certificate's public key is currently pinned in the PinManager. This method should be used during the SSL/TLS handshake to ensure the remote service's certificate matches a previously pinned public key.

type Request

type Request struct {
	HttpRequest *http.Request
	Response    *Response

	// HTTP method, e.g., GET, POST.
	Method string

	Url string

	Body any

	PHeader        PHeader
	OrderedHeaders OrderedHeaders

	// Headers for the request. Deprecated: Use OrderedHeaders instead.
	Header http.Header
	// Order of headers for the request.
	HeaderOrder HeaderOrder

	// If true, redirects won't be followed.
	DisableRedirects bool
	// Maximum number of redirects to follow.
	MaxRedirects uint
	// If true, cookies won't be included in the request.
	NoCookie bool
	// Maximum time to wait for request to complete.
	TimeOut time.Duration
	// Indicates if the current request is a result of a redirection.
	IsRedirected bool
	// If true, server's certificate is not verified.
	InsecureSkipVerify bool

	// If true, the body of the response is not read.
	// The response body can be read from Response.RawBody and
	// you will have to close using Response.CloseBody manually.
	IgnoreBody bool
	Proto      string
	ForceHTTP1 bool

	// Length of content in the request.
	ContentLength int64
	// contains filtered or unexported fields
}

Request represents the details and configuration for an individual HTTP(S) request. It encompasses URL, headers, method, body, proxy settings, timeouts, and other configurations necessary for customizing the request and its execution.

func (*Request) CloseBody

func (r *Request) CloseBody()

func (*Request) Context

func (r *Request) Context() context.Context

Context returns the context for the request.

func (*Request) SetContext

func (r *Request) SetContext(ctx context.Context)

SetContext sets the context for the request.

type Response

type Response struct {
	// HTTP status code, e.g., 200, 404.
	StatusCode int

	// HTTP status message, e.g., "OK", "Not Found".
	Status string

	// Byte representation of the response body.
	Body []byte
	// Raw body stream.
	RawBody io.ReadCloser
	// Response headers.
	Header http.Header
	// Parsed cookies from the response.
	Cookies map[string]string
	// URL from which the response was received.
	Url string
	// Indicates if the body of the response was ignored.
	IgnoreBody bool

	// The underlying HTTP response.
	HttpResponse *http.Response
	// Reference to the associated request.
	Request *Request
	// Length of content in the response.
	ContentLength int64

	Session *Session
}

Response encapsulates the received data and metadata from an HTTP(S) request. This includes status code, body, headers, cookies, associated request details, TLS connection state, etc.

func (*Response) CloseBody

func (r *Response) CloseBody() error

func (*Response) JSON

func (r *Response) JSON(v any) error

func (*Response) MustJSON

func (r *Response) MustJSON(v any)

func (*Response) ReadBody

func (r *Response) ReadBody() (body []byte, err error)

type Session

type Session struct {
	PHeader        PHeader
	OrderedHeaders OrderedHeaders

	// Default headers for all requests. Deprecated: Use OrderedHeaders instead.
	Header http.Header
	// Order of headers for all requests.
	HeaderOrder HeaderOrder

	// Stores cookies across session requests.
	CookieJar http.CookieJar

	// Name or identifier of the browser used in the session.
	Browser string

	HTTP2Transport *http2.Transport
	Transport      *http.Transport

	// Function to provide custom TLS handshake details.
	GetClientHelloSpec func() *tls.ClientHelloSpec

	// Proxy address.
	Proxy string
	// If true, use HTTP2 for proxy connections.
	H2Proxy     bool
	ProxyDialer *proxyDialer

	// If true, print detailed logs or debugging information. Deprecated: Use Dump instead.
	Verbose bool
	// Path for logging verbose information. Deprecated: Use Log instead.
	VerbosePath string
	// List of hosts to ignore when logging verbose info. Deprecated: Use Log instead.
	VerboseIgnoreHost []string
	// Custom function to handle verbose logging. Deprecated: Use Log instead.
	VerboseFunc func(request *Request, response *Response, err error)

	// Maximum number of redirects to follow.
	MaxRedirects uint

	// Maximum time to wait for request to complete.
	TimeOut time.Duration

	// Deprecated, use PreHookWithContext instead.
	PreHook func(request *Request) error
	// Function called before sending a request.
	PreHookWithContext func(ctx *Context) error

	// Deprecated, use CallbackWithContext instead.
	Callback func(request *Request, response *Response, err error)

	// Function called after receiving a response.
	CallbackWithContext func(ctx *Context)

	// Function to modify the dialer used for establishing connections.
	ModifyDialer func(dialer *net.Dialer) error

	// CheckRedirect specifies the policy for handling redirects.
	// If CheckRedirect is not nil, the client calls it before
	// following an HTTP redirect. The arguments req and via are
	// the upcoming request and the requests made already, oldest
	// first. If CheckRedirect returns an error, the Session's Get
	// method returns both the previous Response (with its Body
	// closed) and CheckRedirect's error (wrapped in an url.Error)
	// instead of issuing the Request req.
	// As a special case, if CheckRedirect returns ErrUseLastResponse,
	// then the most recent response is returned, along with a nil error.
	//
	// If CheckRedirect is nil, the Session uses its default policy,
	// which is to stop after 10 consecutive requests.
	CheckRedirect func(req *Request, reqs []*Request) error

	// Deprecated: This field is ignored as pin verification is always true.
	// To disable pin verification, use InsecureSkipVerify.
	VerifyPins bool

	PinManager map[string]*PinManager

	// If true, server's certificate is not verified (insecure: this may facilitate attack from middleman).
	InsecureSkipVerify bool

	// Headers for User-Agent and Sec-Ch-Ua, respectively.
	UserAgent string

	// HeaderPriority specifies the priority of the request's headers.
	// As this information is not included in the Akamai fingerprint, you may have to specify it manually.
	// Note that you can also specify the browser in the session so that this is done automatically.
	HeaderPriority *http2.PriorityParam

	// ProxyHeader defines the headers used for the CONNECT method to the proxy,
	// you may define the order with the http.HeaderOrderKey
	ProxyHeader http.Header
	// contains filtered or unexported fields
}

Session represents the core structure for managing and conducting HTTP(S) sessions. It holds configuration settings, headers, cookie storage, connection pool, and other attributes necessary to perform and customize requests.

func NewSession

func NewSession() *Session

NewSession creates a new session It is a shortcut for NewSessionWithContext(context.Background())

Example

session := azuretls.NewSession()

resp, err := session.Get("https://www.google.com")

if err != nil {
	panic(err)
}

fmt.Println(resp.StatusCode)

Output:

200

func NewSessionWithContext

func NewSessionWithContext(ctx context.Context) *Session

NewSessionWithContext creates a new session with context It is recommended to use this function to create a new session instead of creating a new Session struct

func (*Session) AddPins

func (s *Session) AddPins(u *url.URL, pins []string) error

AddPins associates a set of certificate pins with a given URL within a session. This allows for URL-specific pinning, useful in scenarios where different services (URLs) are trusted with different certificates.

func (*Session) ApplyHTTP2

func (s *Session) ApplyHTTP2(fp string) error

ApplyHTTP2 applies HTTP2 settings to the session from a fingerprint. The fingerprint is in the format:

<SETTINGS>|<WINDOW_UPDATE>|<PRIORITY>|<PSEUDO_HEADER>

egs :

1:65536,2:0,3:1000,4:6291456,6:262144|15663105|0|m,s,a,p

Any 0 value will be ignored.

Example

session := azuretls.NewSession()

preset := "1:65536;2:0;3:1000;4:6291456;6:262144|15663105|0|m,s,a,p"

if err := session.ApplyHTTP2(preset); err != nil {
	panic(err)
}

resp, err := session.Get("https://tls.peet.ws/api/all")

if err != nil {
	panic(err)
}

fmt.Println(strings.Contains(string(resp.Body), preset))

Output:

true

func (*Session) ApplyJa3

func (s *Session) ApplyJa3(ja3, navigator string) error

ApplyJa3 applies JA3 settings to the session from a fingerprint. JA3 is a method for creating fingerprints from SSL/TLS client hellos, which can be used for client identification or detection. The fingerprint is constructed from an MD5 hash of string representations of various handshake parameters, specifically:

<SSL Version>|<Accepted Ciphers>|<List of Extensions>|<Elliptic Curves>|<Elliptic Curve Formats>

e.g.,

769,4865-4866-4867-49196-49195-52393-49200-49199-49172...|0-5-10-11-...|23-24-25|0

This string is then MD5-hashed to produce a 32-character representation, which is the JA3 fingerprint.

Any absent field in the client hello will raise an error.

Example

session := azuretls.NewSession()

ja3 := "771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,45-13-43-0-16-65281-51-18-11-27-35-23-10-5-17513-21,29-23-24,0"

if err := session.ApplyJa3(ja3, azuretls.Chrome); err != nil {
	panic(err)
}

resp, err := session.Get("https://tls.peet.ws/api/all")

if err != nil {
	panic(err)
}

fmt.Println(strings.Contains(string(resp.Body), ja3))

Output:

true

func (*Session) ApplyJa3WithSpecifications

func (s *Session) ApplyJa3WithSpecifications(ja3 string, specifications *TlsSpecifications, navigator string) error

ApplyJa3WithSpecifications applies JA3 settings to the session from a fingerprint. JA3 is a method for creating fingerprints from SSL/TLS client hellos, which can be used for client identification or detection. The fingerprint is constructed from an MD5 hash of string representations of various handshake parameters, specifically:

<SSL Version>|<Accepted Ciphers>|<List of Extensions>|<Elliptic Curves>|<Elliptic Curve Formats>

e.g.,

769,4865-4866-4867-49196-49195-52393-49200-49199-49172...|0-5-10-11-...|23-24-25|0

This string is then MD5-hashed to produce a 32-character representation, which is the JA3 fingerprint.

Any absent field in the client hello will raise an error.

func (*Session) ClearPins

func (s *Session) ClearPins(u *url.URL) error

ClearPins removes all pinned certificates associated with a specific URL in the session. This can be used to reset trust settings or in scenarios where a service's certificate is no longer deemed trustworthy.

func (*Session) ClearProxy

func (s *Session) ClearProxy()

ClearProxy removes the proxy from the session

func (*Session) Close

func (s *Session) Close()

Close closes the session and all its connections. It is recommended to call this function when the session is no longer needed.

After calling this function, the session is no longer usable.

func (*Session) Connect

func (s *Session) Connect(u string) error

Connect initiates a connection to the specified URL

func (*Session) Context

func (s *Session) Context() context.Context

func (*Session) Delete

func (s *Session) Delete(url string, args ...any) (*Response, error)

Delete provides shortcut for sending DELETE request

func (*Session) DisableDump

func (s *Session) DisableDump()

DisableDump will disable requests and responses dumping

func (*Session) DisableLog

func (s *Session) DisableLog()

DisableLog will disable request and response logging

func (*Session) Do

func (s *Session) Do(request *Request, args ...any) (*Response, error)

Do sends a request and returns a response

func (*Session) Dump

func (s *Session) Dump(dir string, uris ...string) error

Dump will activate requests and responses dumping to the specified directory

dir is the directory to save the logs

ignore (optional) is a list of uri to ignore, if ignore is empty, all uri will be logged

Example

session := azuretls.NewSession()

session.Dump("./logs", "*.httpbin.org")

session.Get("https://www.google.com", azuretls.OrderedHeaders{
	{"User-Agent", "Mozilla/5.0"},
	{"Accept", "text/html"},
})

session.Get("https://httpbin.org/get")

time.Sleep(1 * time.Second)

f, _ := os.ReadDir("./logs")

fmt.Println(len(f))

Output:

1

func (*Session) DumpAndLog

func (s *Session) DumpAndLog(dir string, uris ...string) error

DumpAndLog will activate requests and responses dumping to the specified directory and log the requests and responses

func (*Session) DumpIgnore

func (s *Session) DumpIgnore(uri string) bool

DumpIgnore will check if the given uri is ignored from dumping

Example

session := azuretls.NewSession()

if err := session.Dump("./logs", "*.google.com"); err != nil {
	panic(err)
}

fmt.Println(session.DumpIgnore("https://www.google.com"))
fmt.Println(session.DumpIgnore("https://google.com/search"))
fmt.Println(session.DumpIgnore("https://www.google.com/search"))

if err := session.Dump("./logs", "/get"); err != nil {
	panic(err)
}

fmt.Println(session.DumpIgnore("https://www.google.com"))
fmt.Println(session.DumpIgnore("https://google.com/search"))
fmt.Println(session.DumpIgnore("https://www.google.com/get/the/thing"))

Output:

true
true
true
false
false
true

func (*Session) EnableDump

func (s *Session) EnableDump()

EnableDump will enable requests and responses dumping

func (*Session) EnableLog

func (s *Session) EnableLog()

EnableLog will enable request and response logging

func (*Session) EnableVerbose

func (s *Session) EnableVerbose(path string, ignoreHost []string) error

EnableVerbose enables verbose logging

Deprecated: use Dump instead

func (*Session) Get

func (s *Session) Get(url string, args ...any) (*Response, error)

Get provides shortcut for sending GET request

Example

session := azuretls.NewSession()

resp, err := session.Get("https://www.google.com")

if err != nil {
	panic(err)
}

fmt.Println(resp.StatusCode)

Output:

200

func (*Session) Head

func (s *Session) Head(url string, args ...any) (*Response, error)

Head provides shortcut for sending HEAD request

func (*Session) InitTransport

func (s *Session) InitTransport(browser string) (err error)

func (*Session) Ip

func (s *Session) Ip() (ip string, err error)

Ip returns the public IP address of the session

func (*Session) Log

func (s *Session) Log(uris ...string)

Log will print the request and response to the console

uris (optional) is a list of uris to ignore, if ignore is empty, all uris will be logged

Example

session := azuretls.NewSession()

session.Log("/any/path/to/ignore", "can.ignore.this", "*.all.subdomains")

session.Get("https://www.google.com")
session.Get("https://www.google.com/any/path/to/ignore")

func (*Session) LogIgnore

func (s *Session) LogIgnore(uri string) bool

LogIgnore will check if the given uri is ignored from dumping

func (*Session) NewWebsocket

func (s *Session) NewWebsocket(url string, readBufferSize, writeBufferSize int, args ...any) (*Websocket, error)

NewWebsocket returns a new websocket connection.

func (*Session) NewWebsocketWithContext

func (s *Session) NewWebsocketWithContext(ctx context.Context, url string, readBufferSize, writeBufferSize int, args ...any) (*Websocket, error)

NewWebsocketWithContext returns a new websocket connection with a context.

func (*Session) Options

func (s *Session) Options(url string, args ...any) (*Response, error)

Options provides shortcut for sending OPTIONS request

func (*Session) Patch

func (s *Session) Patch(url string, data any, args ...any) (*Response, error)

Patch provides shortcut for sending PATCH request

func (*Session) Pin

func (s *Session) Pin(addr string) error

func (*Session) Post

func (s *Session) Post(url string, data any, args ...any) (*Response, error)

Post provides shortcut for sending POST request

Example

session := azuretls.NewSession()

resp, err := session.Post("https://httpbin.org/post", `post me`)

if err != nil {
	panic(err)
}

fmt.Println(resp.StatusCode)
fmt.Println(bytes.Contains(resp.Body, []byte("post me")))

Output:

200
true

func (*Session) Put

func (s *Session) Put(url string, data any, args ...any) (*Response, error)

Put provides shortcut for sending PUT request

func (*Session) SetContext

func (s *Session) SetContext(ctx context.Context)

SetContext sets the given context for the session

func (*Session) SetProxy

func (s *Session) SetProxy(proxy string) error

SetProxy sets the proxy for the session

Example

session := azuretls.NewSession()

err := session.SetProxy("http://username:password@proxy:8080")

if err != nil {
	panic(err)
}

fmt.Println(session.Proxy)

Output:

http://username:password@proxy:8080

func (*Session) SetTimeout

func (s *Session) SetTimeout(timeout time.Duration)

SetTimeout sets timeout for the session

type TlsSpecifications

type TlsSpecifications struct {
	AlpnProtocols                           []string
	SignatureAlgorithms                     []tls.SignatureScheme
	SupportedVersions                       []uint16
	CertCompressionAlgos                    []tls.CertCompressionAlgo
	DelegatedCredentialsAlgorithmSignatures []tls.SignatureScheme
	PSKKeyExchangeModes                     []uint8
	SignatureAlgorithmsCert                 []tls.SignatureScheme
	ApplicationSettingsProtocols            []string
	RenegotiationSupport                    tls.RenegotiationSupport
	RecordSizeLimit                         uint16
}

TlsSpecifications struct contains various fields representing TLS handshake settings.

func DefaultTlsSpecifications

func DefaultTlsSpecifications(navigator string) *TlsSpecifications

type Websocket

type Websocket struct {
	Url     string
	Headers http.Header

	Request  *Request
	Response *http.Response

	*websocket.Conn
	// contains filtered or unexported fields
}