GoHtml

package module
v0.2.3-beta.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 17, 2025 License: BSD-3-Clause Imports: 8 Imported by: 0

README

GoHTML

A HTML parse and a serializer for Go. GoHTML tries to keep semantic similar to JS-DOM API while trying to keep the API simple by not forcing JS-DOM model into GoHTML. Because of this GoHTML has node tree model. GoHTML tokenizer uses std net/html module for tokenizing in underlining layer. There for it's users responsibility to make sure inputs to GoHTML is UTF-8 encoded. GoHTML allows direct access to the node tree.

Installation

Run the following command in project directory in order to install.

go get github.com/udan-jayanith/GoHTML

Then GoHTML can import like this.

import (
	GoHtml "github.com/udan-jayanith/GoHTML"
)

Features

  • Parsing
  • Serialization
  • Node tree traversing
  • Querying

Example

Heres an example of fetching a website and parsing and then using querying methods.

res, err := http.Get("https://www.metalsucks.net/")
if err != nil {
	t.Fatal(err)
}
defer res.Body.Close()

//Parses the given html reader and then returns the root node and an error.
node, err := GoHtml.Decode(res.Body)
if err != nil {
	t.Fatal(err)
}

nodeList := node.GetElementsByClassName("post-title")
iter := nodeList.IterNodeList()
for node := range iter{
	print(node.GetInnerText())
}

Changelog

Changes, bug fixes and new features in this version.

  • add: Closest
  • add: QuerySearch
  • add: QuerySelector
  • add: QuerySelectorAll
  • renamed: Tokenizer CurrentNode method to GetCurrentNode.

Documentation

Fully fledged documentation is available at go.pkg

Contributions

Contributions are welcome and pull requests and issues will be viewed by an official.

Documentation

Overview

A HTML parse and a serializer for Go. GoHTML tries to keep semantic similar to JS-DOM API while trying to keep the API simple by not forcing JS-DOM model into GoHTML. Because of this GoHTML has node tree model. GoHTML tokenizer uses std net/html module for tokenizing in underlining layer. There for it's users responsibility to make sure inputs to GoHTML is UTF-8 encoded. GoHTML allows direct access to the node tree.

Index

Constants

View Source 
const (
	Id int = iota
	Tag
	Class
)

QueryToken types

View Source 
const (
	Area   string = "area"
	Base   string = "base"
	Br     string = "br"
	Col    string = "col"
	Embed  string = "embed"
	Hr     string = "hr"
	Img    string = "img"
	Input  string = "input"
	Link   string = "link"
	Meta   string = "meta"
	Param  string = "param"
	Source string = "source"
	Track  string = "track"
	Wbr    string = "wbr"
)

Void tags

View Source 
const (
	//This is not a void el. but added it anyway.
	DOCTYPEDTD string = "!DOCTYPE"
)

A DTD defines the structure and the legal elements and attributes of an XML document.

Variables

View Source 
var (
	SyntaxError error = fmt.Errorf("Syntax error")
)

Functions

func Encode 

func Encode(w io.Writer, rootNode *Node)

Encode writes to w encoding of rootNode

func IsVoidTag 

func IsVoidTag(tagName string) bool

IsVoidTag returns whether the tagName is a void tag or DTD

func NodeTreeToHTML 

func NodeTreeToHTML(rootNode *Node) string

NodeTreeToHTML returns encoding of node-tree as a string.

func QuerySearch added in v0.2.3 

func QuerySearch(node *Node, query string) iter.Seq[*Node]

QuerySearch tokenizes the query string and search for nodes that matches with the right most query token. After matching right most query it proceeds to match nodes parents nodes for left over tokens and then passed that node to (yield/range). QuerySearch search the whole node tree for matches unless yield get canceled or range iterator get cancel.

Types

type ClassList 

type ClassList struct {
	// contains filtered or unexported fields
}

func NewClassList 

func NewClassList() ClassList

NewClassList returns a new empty ClassList.

func (ClassList) AppendClass 

func (classList ClassList) AppendClass(className string)

AppendClass append className to classList. className that contains multiple classes is also a valid className.

func (ClassList) Contains 

func (classList ClassList) Contains(className string) bool

Contains returns whether the className exists or not.

func (ClassList) DecodeFrom added in v0.0.1 

func (classList ClassList) DecodeFrom(node *Node)

DecodeFrom append classes in the node to classList. If node is nil SetClass does nothing.

func (ClassList) DeleteClass 

func (classList ClassList) DeleteClass(className string)

DeleteClass deletes the specified classes in className.

func (ClassList) Encode 

func (classList ClassList) Encode() string

Encode returns the full className.

func (ClassList) EncodeTo 

func (classList ClassList) EncodeTo(node *Node)

EncodeTo encode className for the node. If node is nil EncodeTo does nothing.

type Node 

type Node struct {
	// contains filtered or unexported fields
}

Node is a struct that represents a html elements. Nodes can have sibling nodes(NextNode and Previous Node) and child node that represent the child elements. Text is also stored as a node which can be checked by using IsTextNode method.

func CloneNode 

func CloneNode(node *Node) *Node

CloneNode copy the node. But have one way connections to it's parent, next and previous nodes. If node is nil CloneNode returns nil.

func CreateNode 

func CreateNode(tagName string) *Node

CreateNode returns a initialized new node.

func CreateTextNode 

func CreateTextNode(text string) *Node

CreateTextNode returns a new node that represents the given text. HTML tags in text get escaped.

func Decode 

func Decode(r io.Reader) (*Node, error)

Decode reads from rd and create a node-tree. Then returns the root node and nil.

func DeepCloneNode 

func DeepCloneNode(node *Node) *Node

DeepCloneNode clones the node without having references to it's original parent node, previous node and next node. If node is nil DeepCloneNode returns nil.

func HTMLToNodeTree 

func HTMLToNodeTree(html string) (*Node, error)

HTMLToNodeTree return html code as a node-tree. If error were to occur it would be SyntaxError.

func (*Node) Append 

func (node *Node) Append(newNode *Node)

Append inserts the newNode to end of the node chain.

func (*Node) AppendChild 

func (node *Node) AppendChild(childNode *Node)

The AppendChild() method of the Node adds a node to the end of the list of children of a specified parent node.

func (*Node) AppendText 

func (node *Node) AppendText(text string)

AppendText append text to the node.

func (*Node) Closest added in v0.2.3 

func (node *Node) Closest(query string) *Node

Closest traverses the node tree and its parents (heading toward the root node) until it finds a node that matches the full query.

Ex: query = "div .video-container #div"

<div class="video-container id="div"></div>

As shown elements with every described specifiers will only match. But this is not the case for QuerySearch, QuerySelector and QuerySelectorAll.

Adapted from [https://developer.mozilla.org/en-US/docs/Web/API/Element/closest](MDN Element: closest() method)

func (*Node) GetAttribute 

func (node *Node) GetAttribute(attributeName string) (string, bool)

GetAttribute returns the specified attribute value form the node. If the specified attribute doesn't exists GetAttribute returns a empty string and false.

func (*Node) GetChildNode 

func (node *Node) GetChildNode() *Node

GetChildNode returns the first child elements of this node.

func (*Node) GetElementByClassName added in v0.0.1 

func (node *Node) GetElementByClassName(className string) *Node

GetElementByClassName returns the first node that match with the given className by advancing from the node.

func (*Node) GetElementByID added in v0.0.1 

func (node *Node) GetElementByID(idName string) *Node

GetElementByID returns the first node that match with the given idName by advancing from the node.

func (*Node) GetElementByTagName added in v0.0.1 

func (node *Node) GetElementByTagName(tagName string) *Node

GetElementByTagName returns the first node that match with the given tagName by advancing from the node.

func (*Node) GetElementsByClassName added in v0.0.1 

func (node *Node) GetElementsByClassName(className string) NodeList

GetElementsByClassName returns a NodeList containing nodes that have the given className from the node.

func (*Node) GetElementsById added in v0.0.1 

func (node *Node) GetElementsById(idName string) NodeList

GetElementsByClassName returns a NodeList containing nodes that have the given idName from the node.

func (*Node) GetElementsByTagName added in v0.0.1 

func (node *Node) GetElementsByTagName(tagName string) NodeList

GetElementsByTagName returns a NodeList containing nodes that have the given tagName from the node.

func (*Node) GetFirstNode 

func (node *Node) GetFirstNode() *Node

GetFirstNode returns the first node of the node branch.

func (*Node) GetInnerText 

func (node *Node) GetInnerText() string

GetInnerText returns all of the text inside the node.

func (*Node) GetLastNode 

func (node *Node) GetLastNode() *Node

GetLastNode returns the last node in the node branch.

func (*Node) GetNextNode 

func (node *Node) GetNextNode() *Node

GetNextNode returns node next to the node.

func (*Node) GetParent 

func (node *Node) GetParent() *Node

GetParent returns a pointer to the parent node.

func (*Node) GetPreviousNode 

func (node *Node) GetPreviousNode() *Node

GetPreviousNode returns the previous node.

func (*Node) GetTagName 

func (node *Node) GetTagName() string

Returns a string with the name of the tag for the given node.

func (*Node) GetText 

func (node *Node) GetText() string

GetText returns text on the node. This does not returns text on it's child nodes. If you also wants child nodes text use GetInnerText method on the node. HTML tags in returns value get escaped.

func (*Node) IsTextNode 

func (node *Node) IsTextNode() bool

IsTextNode returns a boolean value indicating node is a text node or not.

func (*Node) IterateAttributes 

func (node *Node) IterateAttributes(callback func(attribute, value string))

IterateAttributes calls callback at every attribute in the node by passing attribute and value of the node.

func (*Node) Query 

func (node *Node) Query(query string) *Node

Query returns the first node that matches with the give query.

func (*Node) QueryAll 

func (node *Node) QueryAll(query string) NodeList

QueryAll returns a NodeList containing nodes that matched with the given query.

func (*Node) QuerySelector added in v0.0.2 

func (node *Node) QuerySelector(query string) *Node

QuerySelector only returns the first node that matches with the QuerySearch.

func (*Node) QuerySelectorAll added in v0.0.2 

func (node *Node) QuerySelectorAll(query string) NodeList

QuerySelectorAll stores nodes passed down by QuerySearch in a nodeList and returns the nodeList.

func (*Node) RemoveAttribute 

func (node *Node) RemoveAttribute(attributeName string)

RemoveAttribute remove or delete the specified attribute.

func (*Node) RemoveNode 

func (node *Node) RemoveNode()

RemoveNode removes the node from the branch safely by connecting sibling nodes.

func (*Node) SetAttribute 

func (node *Node) SetAttribute(attribute, value string)

SetAttribute add a attribute to the node.

func (*Node) SetNextNode 

func (node *Node) SetNextNode(nextNode *Node)

SetNextNode make nodes next node as nextNode.

func (*Node) SetPreviousNode 

func (node *Node) SetPreviousNode(previousNode *Node)

SetPreviousNode sets nodes previous node to previousNode.

func (*Node) SetTagName 

func (node *Node) SetTagName(tagName string)

SetTagName changes the html tag name to the tagName.

func (*Node) SetText 

func (node *Node) SetText(text string)

SetText add text to the node. SetText unescapes entities like "&lt;" to become "<".

type NodeList added in v0.0.1 

type NodeList struct {
	// contains filtered or unexported fields
}

NodeList can store nodes by appended order.

func NewNodeList added in v0.0.1 

func NewNodeList() NodeList

New returns an initialized node list.

func (*NodeList) Append added in v0.0.1 

func (nl *NodeList) Append(node *Node)

Append append a node to the back of the list.

func (*NodeList) Back added in v0.0.1 

func (nl *NodeList) Back() *Node

Back returns the last node of list or nil if the list is empty.

func (*NodeList) Front added in v0.0.1 

func (nl *NodeList) Front() *Node

Front returns the first node of list or nil if the list is empty.

func (*NodeList) IterNodeList added in v0.0.1 

func (nl *NodeList) IterNodeList() iter.Seq[*Node]

IterNodeList returns a iterator over the node list.

func (*NodeList) Len added in v0.0.1 

func (nl *NodeList) Len() int

Len returns the number of node in the list. The complexity is O(1).

func (*NodeList) Next added in v0.0.1 

func (nl *NodeList) Next() *Node

Next advanced to the next node and returns that node.

func (*NodeList) Previous added in v0.0.1 

func (nl *NodeList) Previous() *Node

Previous advanced to the previous node and return that node.

type NodeTreeBuilder added in v0.2.3 

type NodeTreeBuilder struct {
	// contains filtered or unexported fields
}

NodeTreeBuilder is used to build a node tree given a node and it's type.

func NewNodeTreeBuilder added in v0.2.3 

func NewNodeTreeBuilder() NodeTreeBuilder

NewNodeTreeBuilder returns a new NodeTreeBuilder.

func (*NodeTreeBuilder) GetRootNode added in v0.2.3 

func (ntb *NodeTreeBuilder) GetRootNode() *Node

GetRootNode returns the root node of the accumulated node tree and resets the NodeTreeBuilder.

func (*NodeTreeBuilder) WriteNodeTree added in v0.2.3 

func (ntb *NodeTreeBuilder) WriteNodeTree(node *Node, tt html.TokenType)

WriteNodeTree append the node given html.TokenType

type QueryToken added in v0.0.2 

type QueryToken struct {
	Type         int
	SelectorName string
	Selector     string
}

QueryToken store data about basic css selectors(ids, classes, tags).

func TokenizeQuery added in v0.0.2 

func TokenizeQuery(query string) []QueryToken

TokenizeQuery tokenizes the query and returns a list of QueryToken.

query should be of only consists of class, tag and/or id. This applies to every function that accepts a parameter name query. query should not consists of css selectors, Combinators and separators.

type Tokenizer added in v0.2.3 

type Tokenizer struct {
	// contains filtered or unexported fields
}

Tokenizer contains a *html.Tokenizer.

func NewTokenizer added in v0.2.3 

func NewTokenizer(r io.Reader) Tokenizer

NewTokenizer returns a new Tokenizer.

func (*Tokenizer) Advanced added in v0.2.3 

func (t *Tokenizer) Advanced() html.TokenType

Advanced scans the next token and returns its type.

func (*Tokenizer) GetCurrentNode added in v0.2.3 

func (t *Tokenizer) GetCurrentNode() *Node

CurrentNode returns the current node. Returned value can be nil regardless of tt.

type TraverseCondition 

type TraverseCondition = bool
const (
	StopWalkthrough     TraverseCondition = false
	ContinueWalkthrough TraverseCondition = true
)

type Traverser 

type Traverser struct {
	// contains filtered or unexported fields
}

func NewTraverser added in v0.0.1 

func NewTraverser(startingNode *Node) Traverser

NewTraverser returns a new traverser that can be used to navigate the node tree.

func (*Traverser) GetCurrentNode 

func (t *Traverser) GetCurrentNode() *Node

GetCurrentNode returns the current node.

func (*Traverser) Next 

func (t *Traverser) Next() *Node

Next returns the node next to current node and change CurrentNode to the new node. Make sure t.currentNode is not nil otherwise program will panic.

func (*Traverser) Previous 

func (t *Traverser) Previous() *Node

Previous returns the previous node and change CurrentNode to the new node. Make sure t.currentNode is not nil otherwise program will panic.

func (*Traverser) SetCurrentNodeTo 

func (t *Traverser) SetCurrentNodeTo(newNode *Node)

SetCurrentNodeTo changes the current node to the newNode.

func (*Traverser) Walkthrough 

func (t *Traverser) Walkthrough(callback func(node *Node) TraverseCondition)

Walkthrough traverse the node tree from the current node to the end of the node tree by visiting every node. Walkthrough traverse the node tree similar to DFS without visiting visited nodes iteratively. Walkthrough can be used as a range over iterator or a function that takes a callback and pass every node one by one.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.