README
¶
goada: Fast WHATWG URL library in Go
The goada library provides support for the WHATWG URL standard in Go.
Requirements
- Go 1.24 or better.
Examples
url, nil := New("https:// www.GOoglé.com/./path/../path2/")
fmt.Println(url.Href()) // "https://www.xn--googl-fsa.com/path2/"
A common use of a URL parser is to take a URL string and normalize it. The WHATWG URL specification has been adopted by most browsers. Other tools, such as the Go runtime, follow the RFC 3986. The following table illustrates possible differences in practice (encoding of the host, encoding of the path):
| string source | string value |
|---|---|
| input string | https://www.7-Eleven.com/Home/../Privacy/Montréal |
| ada's normalized string | https://www.xn--7eleven-506c.com/Home/Privacy/Montr%C3%A9al |
| curl 7.87 | https://www.7-Eleven.com/Privacy/Montr%C3%A9al |
Go runtime (net/url) |
https://www.7-Eleven.com/Home/../Privacy/Montr%C3%A9al |
The Go runtime (net/url) does not normalize hostnames, and it does not process pathnames properly.
Usage
import (
"github.com/ada-url/goada"
"fmt"
)
url, err := goada.New("https://www.GOogle.com")
if err != nil {
t.Error("Expected no error")
}
fmt.Println(url.Href()) // prints https://www.google.com/
url.SetProtocol("http:")
url.SetHash("goada")
fmt.Println(url.Hash()) // prints #goada
fmt.Println(url.Href()) // prints http://www.google.com/#goada
Performance
Benchmarks comparing URL parsing performance using the top 100k URLs dataset:
| Library | ns/op (full dataset) | ns/op per URL | WHATWG URL compliant ? |
|---|---|---|---|
| github.com/ada-url/goada | 2,206,395 | 22.0 | YES |
Go net/url |
1,937,154 | 19.4 | NO (sad face) |
| github.com/nlnwa/whatwg-url | 13,916,373 | 139.0 | Yes |
Run go test -bench BenchmarkTop100 -run - to reproduce these results.
Running Benchmarks
To run the URL parsing benchmarks:
-
Clone the repository and navigate to the directory:
git clone https://github.com/ada-url/goada.git cd goada -
Install dependencies:
go mod download -
Run the benchmarks:
# Run all benchmarks go test -bench=. -run=^$ # Run specific benchmark for top 100k URLs go test -bench=BenchmarkTop100 -run=^ # Run with more detailed output go test -bench=BenchmarkTop100 -benchmem -run=^ -
Compare with other libraries: The benchmarks compare goada (Ada), Go's standard
net/url, andwhatwg-urllibrary performance.
Dataset
The benchmarks use the top 100k URLs dataset from ada-url/url-various-datasets, which contains the most popular URLs collected from real web traffic.
Compliance Top 100
Testing with the top 100k URLs dataset shows that goada and whatwg-url produce identical normalized output for all URLs they both successfully parse. Go's standard net/url library produces different results in 1398 cases. The main differences are:
- Query parameter encoding: Go net/url does not encode spaces and special characters in query parameters, while Ada/whatwg-url do
- Path normalization: Different handling of path components
Run go test -run TestParserComparison -v to see some differences.
Citation
Yagiz Nizipli, Daniel Lemire, Parsing Millions of URLs per Second, Software: Practice and Experience 54(5) May 2024.
Documentation
¶
Index ¶
- Variables
- type Url
- func (u *Url) HasCredentials() bool
- func (u *Url) HasEmptyHostname() bool
- func (u *Url) HasHash() bool
- func (u *Url) HasHostname() bool
- func (u *Url) HasNonEmptyPassword() bool
- func (u *Url) HasNonEmptyUsername() bool
- func (u *Url) HasPassword() bool
- func (u *Url) HasPort() bool
- func (u *Url) HasSearch() bool
- func (u *Url) Hash() string
- func (u *Url) Host() string
- func (u *Url) Hostname() string
- func (u *Url) Href() string
- func (u *Url) Password() string
- func (u *Url) Pathname() string
- func (u *Url) Port() string
- func (u *Url) Protocol() string
- func (u *Url) Search() string
- func (u *Url) SetHash(s string)
- func (u *Url) SetHost(s string) bool
- func (u *Url) SetHostname(s string) bool
- func (u *Url) SetHref(s string) bool
- func (u *Url) SetPassword(s string) bool
- func (u *Url) SetPathname(s string) bool
- func (u *Url) SetPort(s string) bool
- func (u *Url) SetProtocol(s string) bool
- func (u *Url) SetSearch(s string)
- func (u *Url) SetUsername(s string) bool
- func (u *Url) Username() string
- func (u *Url) Valid() bool
Constants ¶
This section is empty.
Variables ¶
var ErrEmptyString = errors.New("empty url string")
var ErrInvalidUrl = errors.New("invalid url")
Functions ¶
This section is empty.
Types ¶
type Url ¶
type Url struct {
// contains filtered or unexported fields
}
func New ¶
parse the given string into a URL, a finalizer will be set to free the URL when it is no longer needed.
func NewWithBase ¶
parse the given strings into a URL, a finalizer will be set to free the URL when it is no longer needed.
Source Files