README
¶
SPA - Single Page App Processor
A Single Page App processor using go-rod/rod.
Assumption and Concept
Implementation
Property Structure
IInfo and IInfoList Interfaces
Processor Structure
The V* field func
The build-in field functions.
Responsibilities of the Element Func
| Field Function | Responsibility |
|---|---|
V030_ElementInfo |
Extract information from [element] and put into an [IInfo] structure and return it. |
V040_ElementMatch |
Determine [element] is a match or not base on [info] |
V050_ElementProcessMatched |
Do some processing (eg, print, write to file, db, etc) if [element] is a match |
V060_ElementProcessUnmatch |
Do some processing (eg, print, write to file, db, etc) if [element] is not a match |
V070_ElementProcess |
Do some processing (eg, print, write to file, db, etc) regardless [element] is a match or not |
Use What Is Needed
Change Log
- v0.9.0
- Initial commit
- v0.9.1
- add:
json:"-"for field func - add: Run() exit if LoadPage() error
- fix: LoadPage() CheckErrInit() logical error
- fix: LoadPage() err handling
- add:
License
The MIT License (MIT)
Copyright © 2025 John, Sing Dao, Siu john.sd.siu@gmail.com
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Documentation
¶
Overview ¶
Package spa A Single Page App processor using go-rod/rod(https://github.com/go-rod/rod).
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type IInfo ¶
type IInfo interface {
Matched() bool // Get matched bool value
MatchedStr() string // Get matched string value
SetMatched(matched bool) // Set matched bool value
SetMatchedStr(matchedStr string) // Set matched string value
String() string // Info struct to string
}
Interface for info struct
type IInfoList ¶
type IInfoList []IInfo
func (*IInfoList) Print ¶
func (list *IInfoList) Print(mode IInfoListPrintMode)
type IInfoListPrintMode ¶
type IInfoListPrintMode int8
const ( PrintAll IInfoListPrintMode = iota PrintMatched PrintUnmatched )
type InfoBase ¶
type InfoBase struct {
// contains filtered or unexported fields
}
IInfo base struct to be embedded
- Only String() should be overloaded
func (*InfoBase) SetMatchedStr ¶
Set matched string value
type Processor ¶
type Processor struct {
*basestruct.Base
*Property
// Load [UrlStr] into [Page]
//
// No overloading needed.
LoadPage func() `json:"-"`
// Determine whether the scroll loop should continue running
//
// No overloading needed.
ScrollLoopBreak func(state *State) bool `json:"-"`
// Detect end of page, scroll no longer possible.
//
// No overloading needed.
//
// If elements are removed during [Run()], overload [V100_ExitScroll()] to do custom override.
// As both of following checks can be flawed if elements are removed from page DOM.
ScrollCalculation func(state *State) (scroll bool) `json:"-"`
// Use [MustScrollIntoView] on [element]
//
// No overloading needed.
ScrollElement func(element *rod.Element) `json:"-"`
// Return the container element.
//
// build-in behavior is to return [property.Container]
//
// Override if needed
V010_Container func() *rod.Element `json:"-"`
// Return collection of repeating elements within [property.Page] or [property.Container]
//
// build-in behavior is to return `nil`
//
// **Must override**
V020_Elements func(element *rod.Element) *rod.Elements `json:"-"`
// Extract information from [element] and put into an [IInfo] structure and return it.
//
// build-in behavior is to return `nil`
//
// **Must override**
V030_ElementInfo func(element *rod.Element, index int) (info IInfo) `json:"-"`
// Determine [element] is a match or not base on [info]
//
// build-in behavior is to return (`true`, `""`)
//
// Override if needed
V040_ElementMatch func(element *rod.Element, index int, info IInfo) (matched bool, matchedStr string) `json:"-"`
// Do some processing (eg, print, write to file, db, etc) if [element] is a match
//
// build-in behavior is to do nothing
//
// Override if needed
V050_ElementProcessMatched func(element *rod.Element, index int, info IInfo) `json:"-"`
// Do some processing (eg, print, write to file, db, etc) if [element] is not a match
//
// build-in behavior is to do nothing
//
// Override if needed
V060_ElementProcessUnmatch func(element *rod.Element, index int, info IInfo) `json:"-"`
// Do some processing (eg, print, write to file, db, etc) regardless [element] is a match or not
//
// build-in behavior is to do nothing
//
// Override if needed
V070_ElementProcess func(element *rod.Element, index int, info IInfo) `json:"-"`
// Determine if an element is scrollable
//
// build-in behavior is to return `true“
//
// Override if needed
V080_ElementScrollable func(element *rod.Element, index int, info IInfo) bool `json:"-"`
// Do some processing if required
//
// build-in behavior is to do nothing
//
// Override if needed
V090_ElementLoopEnd func(element *rod.Element, index int, info IInfo) `json:"-"`
// Do some processing if required
//
// build-in behavior is to do nothing
//
// Override if needed
V100_ScrollLoopEnd func(state *State) `json:"-"`
}
type Property ¶
type Property struct {
Page *rod.Page `json:"Page,omitempty"` // Page element of [rod]
Container *rod.Element `json:"Container,omitempty"` // The outer most rod.Element containing all repeating items
UrlCheck bool `json:"UrlCheck,omitempty"` // Check [UrlStr] before loading
UrlLoad bool `json:"UrlLoad,omitempty"` // Control if [UrlStr] should be load at the beginning of [Run]
UrlStr string `json:"UrlStr,omitempty"` // URL string used in [LoadPage]
ScrollMax int `json:"ScrollMax,omitempty"` // Maximum time the page should be scrolled
IInfoList *IInfoList `json:"IInfoList,omitempty"` // Pointer of array of IInfo. If not nil, IInfo item will be added to the array
}
type State ¶
type State struct {
*basestruct.Base
Elements *rod.Elements `json:"elements,omitempty"` // Result of [Processor.V020_Elements()]
ElementLast *rod.Element `json:"element_last,omitempty"` // Last element of the previous scroll loop iteration
ElementLastScroll *rod.Element `json:"element_last_scroll,omitempty"` // Element used for previous scroll (not necessary last loop iteration)
ElementCountLast int `json:"element_count_last,omitempty"` // Number of elements of previous loop iteration
InfoLast IInfo `json:"info_last,omitempty"` // [Info] of [ElementLast]. Return from [Processor.V030_ElementInfo()]
Scroll bool `json:"scroll,omitempty"` // Used by [breakLoop]. True = to scroll. False = don't scroll.
ScrollCount int `json:"scroll_count,omitempty"` // Total number of times [Processor.ElementScroll()] called
}
[State] ¶
Used at the beginning of [Processor.Run()] scroll loop for [breakLoop] calculation, and [Processor.ElementScroll()] for scrolling.
At the bottom of 'Run()' scroll loop, it is passed into [Processor.V100_ScrollLoopEnd()] for customized scroll calculation.
Source Files