urlverify

README

URLVerify

A Go library for extracting and validating URLs and domains from text.

Features

• Extracts URLs (with http/https) and plain domains from text
• Validates domains using the Public Suffix List
• Supports IPv4 and IPv6 addresses
• Handles dynamic DNS services (e.g., dyndns.org, no-ip.org)
• Returns domains exactly as they appear in the original text
• Provides detailed validation results for testing and debugging
• Supports internationalized domain names
• Preserves the original case/format from the original text in the result

Usage

Basic Usage

package main

import (
    "fmt"
    "github.com/potakhov/urlverify"
)

func main() {
    text := `
    Check out these sites:
    - https://example.com
    - foo.dyndns.org  
    - 192.168.1.1:8080
    - invalid.fake
    `
    
    validDomains := urlverify.ExtractAll(text)
    fmt.Printf("Found %d valid URLs/domains:\n", len(validDomains))
    for _, domain := range validDomains {
        fmt.Printf("- %s\n", domain)
    }
}

Detailed Validation

result := urlverify.ValidateDomain("foo.dyndns.org")
if result.Valid {
    fmt.Printf("✅ Valid %s: %s (TLD: %s)\n", result.Type, "foo.dyndns.org", result.TLD)
} else {
    fmt.Printf("❌ Invalid: %s (%s)\n", "foo.dyndns.org", result.Reason)
}

API

ParseURL(raw string) (*url.URL, error)

Parses a raw URL string and returns a *url.URL object or an error if parsing fails.

NormalizeURI(uri string) (string, error)

NormalizeURI normalizes a URI by converting it to ASCII and lowercasing it. Returns the normalized URI or an error if the conversion fails.

ExtractAll(text string) []string

Extracts all valid URLs and domains from the given text, returning them exactly as they appeared in the original text.

ValidateDomain(domain string) ValidationResult

Validates a single URL or domain string and returns detailed validation information.

ValidationResult

type ValidationResult struct {
    Valid  bool    // Whether the domain is valid
    Reason string  // Explanation of the validation result
    Type   URLType // type of the URL
    TLD    string  // The effective TLD or IP address
}

Testing

Run tests with:

go test ./pkg/urlverify

Documentation

Overview

Package urlverify provides functionality to extract and validate URLs and domains from text.

The package uses the Public Suffix List to validate domains and supports: - HTTP/HTTPS URLs - Plain domain names - IPv4 and IPv6 addresses - Dynamic DNS services (e.g., dyndns.org, no-ip.org)

Example usage:

text := "Visit https://example.com and check foo.dyndns.org"
validDomains := urlverify.ExtractAll(text)

Returns: ["https://example.com", "foo.dyndns.org"]

result := urlverify.ValidateDomain("foo.dyndns.org")
if result.Valid {
    fmt.Printf("Valid domain: %s (TLD: %s)\n", "foo.dyndns.org", result.TLD)
}

Index

• func ExtractAll(text string) []string
• func NormalizeURI(uri string) (string, error)
• func ParseURL(raw string) (*url.URL, error)
• type URLType
  • func (t URLType) String() string
• type ValidationResult
  • func ValidateDomain(raw string) ValidationResult

Functions

func ExtractAll

func ExtractAll(text string) []string

ExtractAll extracts and validates all URLs and domains from the given text, returning them exactly as they appeared in the original text (without adding schema).

func NormalizeURI

func NormalizeURI(uri string) (string, error)

NormalizeURI normalizes a URI by converting it to ASCII and lowercasing it. Returns the normalized URI or an error if the conversion fails.

func ParseURL

func ParseURL(raw string) (*url.URL, error)

ParseURL tries to parse a single URL or domain string and returns a pointer to url.URL structure and/or error.

Types

type URLType

type URLType int

const (
	URLTypeInvalid URLType = iota
	URLTypeIP
	URLTypeICANN
	URLTypeNonICANN
)

func (URLType) String

func (t URLType) String() string

type ValidationResult

type ValidationResult struct {
	Valid  bool     // Whether the URL or domain is valid
	Reason string   // Explanation of the validation result
	Type   URLType  // Type of URL or domain
	TLD    string   // The effective TLD, if applicable or an IP address
	URL    *url.URL // Only set if the URL was successfully parsed
}

ValidationResult represents the result of domain validation.

func ValidateDomain

func ValidateDomain(raw string) ValidationResult

ValidateDomain validates a single URL or domain string and returns detailed validation result.