queryutil

package module
v0.3.12 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 11, 2025 License: MIT Imports: 14 Imported by: 1

README

queryutil

A Go library that implements query parsing utilities compatible with Refine's Simple REST Data Provider (source) for filtering, sorting, and pagination.

Features

  • Filter parsing compatible with Refine's data provider spec
  • Sort handling with multiple fields and directions
  • Pagination with server/client modes
  • GORM integration helpers

Installation

go get go.lumeweb.com/queryutil

Usage

Request Parsing
JSON Input Support
// Parse filters from JSON string
jsonFilters := `[
  {
    "field": "name",
    "operator": "contains",
    "value": "john"
  },
  {
    "operator": "or",
    "value": [
      {
        "field": "age",
        "operator": "gte",
        "value": 30
      }
    ]
  }
]`

filters, _, _, err := queryutil.ParseRequestFromJSON(jsonFilters)
if err != nil {
    // Handle error
}
Mixed Input Handling
// If using both parameter and JSON parsing:
paramsFilters, paramSorts, pagination, err := queryutil.ParseRequest(r)
jsonFilters, _, _, err := queryutil.ParseRequestFromJSON(body)

// Merge filter sets
allFilters := append(paramsFilters, jsonFilters...)
Request Parsing
// Parse all query parameters from request
filters, sorts, pagination, err := queryutil.ParseRequest(r)
if err != nil {
    // Handle error
}

// Apply to GORM query
db = queryutil.ApplyFilters(db, filters)
db = queryutil.ApplySort(db, sorts)
db = db.Offset(pagination.GetOffset()).Limit(pagination.GetLimit())

With global search support:

// Configure searchable columns
searchConfig := &queryutil.GlobalSearchConfig{
    SearchableColumns: []string{"name", "email", "description"},
}

// Parse request with search config
filters, sorts, pagination, err := queryutil.ParseRequestWithSearch(r, searchConfig)
if err != nil {
    // Handle error
}

// Apply to GORM query - pass searchConfig for global search support
db = queryutil.ApplyFilters(db, filters, searchConfig)
db = queryutil.ApplySort(db, sorts)
db = db.Offset(pagination.GetOffset()).Limit(pagination.GetLimit())
Individual Parameter Parsing

You can also parse parameters individually if needed:

// Parse filters
filters, err := query.ParseQueryFilters(r.URL.Query())
if err != nil {
    // Handle error
}

// Parse sort
sorts := query.ParseQuerySort(r.URL.Query())

// Parse pagination
pagination := query.ParseQueryPagination(r.URL.Query())

Supported operators:

  • Default (no suffix): Exact match
  • _ne: Not equals
  • _gte: Greater than or equal
  • _lte: Less than or equal
  • _like: Contains search

Special parameter q is reserved for global search.

Sorting
// Parse sort parameters
sorts := query.ParseQuerySort(
    r.URL.Query().Get("_sort"),
    r.URL.Query().Get("_order"),
)

// Apply to GORM query
db = query.ApplySort(db, sorts)
Pagination
// Parse pagination parameters
pagination := query.ParseQueryPagination(r.URL.Query())

// Get SQL LIMIT/OFFSET
limit := pagination.GetLimit()
offset := pagination.GetOffset()

// Apply to GORM query
db = db.Offset(offset).Limit(limit)

Compatibility

This library implements the query parameter specification used by Refine's Simple REST Data Provider.

License

MIT License - see LICENSE file

Documentation

Overview

Package queryutil provides a comprehensive toolkit for building data-driven web services with advanced filtering, sorting and pagination capabilities. The module follows a clean architecture pattern with separate packages for: - HTTP adapter: Request/response handling - Filter: Core filtering logic and query building - Parser: Input parsing from different formats - Builder: Database-specific query building

Key Features: - Full CRUD filter support with complex logical operators - Automatic validation of filter/sort/pagination parameters - Database-agnostic query building (currently supports GORM) - Server-side pagination with Content-Range headers - Support for both JSON and query parameter input formats - Type-safe value conversions and validation - Global search across multiple fields

Usage Example: 

// Parse HTTP request params
filters, sorts, pagination, err := queryutil.ParseRequestHTTP(r)

// Build database query
query := db.Model(&User{})
gormBuilder := builder.NewGORMBuilder(db, nil)
filteredQuery, err := gormBuilder.Apply(query, filters)

// Apply sorting and pagination
filteredQuery = gormBuilder.ApplySorts(filteredQuery, sorts)
filteredQuery = filteredQuery.Limit(pagination.GetLimit()).Offset(pagination.GetOffset())

// Execute and format response
var results []User
filteredQuery.Find(&results)
total := queryutil.GetResultCount(results)
w.Header().Set("Content-Range", queryutil.FormatContentRange("users", pagination, total, total))
http.EncodeJSON(w, results)

Index

Constants

View Source 
const (
	OpEq           = filter.OpEq
	OpNe           = filter.OpNe
	OpLt           = filter.OpLt
	OpGt           = filter.OpGt
	OpLte          = filter.OpLte
	OpGte          = filter.OpGte
	OpContains     = filter.OpContains
	OpContainss    = filter.OpContainss
	OpNcontains    = filter.OpNcontains
	OpNcontainss   = filter.OpNcontainss
	OpIn           = filter.OpIn
	OpNin          = filter.OpNin
	OpIna          = filter.OpIna
	OpNina         = filter.OpNina
	OpBetween      = filter.OpBetween
	OpNbetween     = filter.OpNbetween
	OpNull         = filter.OpNull
	OpNnull        = filter.OpNnull
	OpStartswith   = filter.OpStartswith
	OpStartswiths  = filter.OpStartswiths
	OpNstartswith  = filter.OpNstartswith
	OpNstartswiths = filter.OpNstartswiths
	OpEndswith     = filter.OpEndswith
	OpEndswiths    = filter.OpEndswiths
	OpNendswith    = filter.OpNendswith
	OpNendswiths   = filter.OpNendswiths
)

Filter operators

View Source 
const (
	OrderAsc  = filter.OrderAsc
	OrderDesc = filter.OrderDesc
)

Sort directions

Variables

View Source 
var (
	NewPagination     = filter.NewPagination
	CreatePage        = filter.CreatePage
	DefaultPagination = filter.DefaultPagination
	LargePagination   = filter.LargePagination
	XLargePagination  = filter.XLargePagination
	XXLargePagination = filter.XXLargePagination
)
View Source 
var (
	ParseQuerySort       = filter.ParseQuerySort
	GetResultCount       = filter.GetResultCount
	FormatContentRange   = filter.FormatContentRange
	NewPaginationError   = filter.NewPaginationError
	NewSortError         = filter.NewSortError
	NewFilterError       = filter.NewFilterError
	ParseQueryPagination = filter.ParseQueryPagination
)

Core functions

View Source 
var (
	NewLogicalFilter     = filter.NewLogicalFilter
	NewConditionalFilter = filter.NewConditionalFilter
)

Filter constructors

View Source 
var (
	Filters = filter.Filters
	AndF    = filter.AndF
	OrF     = filter.OrF
)

Filter composition helpers

View Source 
var (
	Equal          = filter.Equal
	NotEqual       = filter.NotEqual
	GreaterThan    = filter.GreaterThan
	GreaterOrEqual = filter.GreaterOrEqual
	LessThan       = filter.LessThan
	LessOrEqual    = filter.LessOrEqual
	Search         = filter.Search
	Contains       = filter.Contains
	In             = filter.In
	Between        = filter.Between
)

Core predicates

View Source 
var (
	FieldNotBetween = filter.FieldNotBetween
	FieldNotIn      = filter.FieldNotIn
	FieldIsNull     = filter.FieldIsNull
	FieldIsNotNull  = filter.FieldIsNotNull
	FieldIn         = filter.FieldIn
	FieldEqual      = filter.FieldEqual
	FieldNotEqual   = filter.FieldNotEqual
	FieldGt         = filter.FieldGt
	FieldLt         = filter.FieldLt
	FieldGte        = filter.FieldGte
	FieldLte        = filter.FieldLte
	FieldContains   = filter.FieldContains
	FieldBetween    = filter.FieldBetween
)

Field operations

View Source 
var (
	And = filter.And
	Or  = filter.Or
	Not = filter.Not
)

Logical combinators

View Source 
var (
	MergeStrategyOverride = filter.MergeStrategyOverride
	MergeStrategyDeep     = filter.MergeStrategyDeep
	MergeStrategyAppend   = filter.MergeStrategyAppend
	MergeFilters          = filter.MergeFilters
)

Merge strategies

View Source 
var (
	StringField = filter.StringField
	BoolField   = filter.BoolField
	TimeField   = filter.TimeField
)

Type-safe field helpers

View Source 
var (
	OperatorMap = filter.OperatorMap
)

Re-exported variables

Functions

func ApplyFilters 

func ApplyFilters(tx *gorm.DB, filters []CrudFilter, searchConfig *GlobalSearchConfig) *gorm.DB

ApplyFilters applies filters to a GORM query using the global search configuration

func ApplyPagination added in v0.3.3 

func ApplyPagination(tx *gorm.DB, pagination Pagination) *gorm.DB

ApplyPagination applies pagination parameters to a GORM query Only applies offset/limit if either value is non-zero

func ApplySort 

func ApplySort(tx *gorm.DB, sorts []Sort) *gorm.DB

ApplySort applies sort parameters to a GORM query

func DeepFindFilter added in v0.3.6 

func DeepFindFilter(filters []filter.CrudFilter, field string) filter.CrudFilter

DeepFindFilter recursively searches for the first CrudFilter with a matching field. It traverses ConditionalFilters (AND, OR, NOT) to find nested filters. Returns nil if no matching filter is found.

func DeepFindFilterWithOperator added in v0.3.7 

func DeepFindFilterWithOperator(filters []filter.CrudFilter, field string, operator filter.Operator) filter.CrudFilter

DeepFindFilterWithOperator recursively searches for the first CrudFilter with matching field and operator. It traverses ConditionalFilters (AND, OR, NOT) to find nested filters. Returns nil if no matching filter is found.

func DeepFindFilters added in v0.3.6 

func DeepFindFilters(filters []filter.CrudFilter, field string) []filter.CrudFilter

DeepFindFilters recursively searches for all CrudFilters with a matching field. It traverses ConditionalFilters (AND, OR, NOT) to find nested filters. Returns an empty slice if no matching filters are found.

func DeepFindFiltersWithOperator added in v0.3.7 

func DeepFindFiltersWithOperator(filters []filter.CrudFilter, field string, operator filter.Operator) []filter.CrudFilter

DeepFindFiltersWithOperator recursively searches for all CrudFilters with matching field and operator. It traverses ConditionalFilters (AND, OR, NOT) to find nested filters. Returns an empty slice if no matching filters are found.

func FindFilter added in v0.3.6 

func FindFilter(filters []filter.CrudFilter, field string) filter.CrudFilter

FindFilter extracts the first CrudFilter with a matching field from a slice of filters. It performs a shallow search, only examining the top-level filters. Returns nil if no matching filter is found.

func FindFilterWithOperator added in v0.3.7 

func FindFilterWithOperator(filters []filter.CrudFilter, field string, operator filter.Operator) filter.CrudFilter

FindFilterWithOperator extracts the first CrudFilter with matching field and operator from a slice of filters. It performs a shallow search, only examining the top-level filters. Returns nil if no matching filter is found.

func FindFilters added in v0.3.6 

func FindFilters(filters []filter.CrudFilter, field string) []filter.CrudFilter

FindFilters extracts all CrudFilters with a matching field from a slice of filters. It performs a shallow search, only examining the top-level filters. Returns an empty slice if no matching filters are found.

func FindFiltersWithOperator added in v0.3.7 

func FindFiltersWithOperator(filters []filter.CrudFilter, field string, operator filter.Operator) []filter.CrudFilter

FindFiltersWithOperator extracts all CrudFilters with matching field and operator from a slice of filters. It performs a shallow search, only examining the top-level filters. Returns an empty slice if no matching filters are found.

func NumberField added in v0.3.0 

func NumberField[T constraints.Integer | constraints.Float](field string) filter.NumberFieldHelper[T]

NumberField creates a type-safe query builder helper for numeric fields. Example: queryutil.NumberField[int]("age").Gt(18)

func ParseFromCustomSource added in v0.2.0 

func ParseFromCustomSource(parser RequestParser) ([]filter.CrudFilter, []Sort, Pagination, error)

ParseFromCustomSource allows using queryutil with custom request sources by accepting any implementation of the RequestParser interface.

This function provides a way to extend queryutil to work with different request types beyond HTTP requests.

Example with a custom parser: 

type MyCustomParser struct {
    // Custom fields
}

func (p *MyCustomParser) ParseFilters() ([]Filter, error) {
    // Custom implementation
}

func (p *MyCustomParser) ParseQuerySort() ([]Sort, error) {
    // Custom implementation
}

func (p *MyCustomParser) ParsePagination() (Pagination, error) {
    // Custom implementation
}

// Usage
parser := &MyCustomParser{}
filters, sorts, pagination, err := ParseFromCustomSource(parser)

func ParseFromSource added in v0.3.0 

func ParseFromSource(parser RequestParser) ([]CrudFilter, []Sort, Pagination, error)

ParseFromSource is the unified parsing entry point using the RequestParser interface

func ParseQuery added in v0.2.0 

func ParseQuery(query map[string][]string) ([]CrudFilter, []Sort, Pagination, error)

ParseQuery parses all query parameters from a map. This is the main entry point for processing query parameters in an HTTP-agnostic way. It combines parsing of filters, sorts, and pagination parameters.

Returns parsed Filter, Sort, and Pagination structs or an error if any validation fails.

Example: 

// With a map of query parameters
query := map[string][]string{
    "name": {"john"},
    "age_gte": {"18"},
    "_sort": {"name"},
    "_order": {"desc"},
    "_start": {"0"},
    "_end": {"10"},
}
filters, sorts, pagination, err := ParseQuery(query)

func ParseQueryWithSearch added in v0.2.0 

func ParseQueryWithSearch(query map[string][]string, searchConfig *filter.GlobalSearchConfig) ([]CrudFilter, []Sort, Pagination, error)

ParseQueryWithSearch parses query parameters with global search support. Similar to ParseQuery but includes configuration for global search functionality across multiple columns.

The searchConfig parameter specifies which columns should be included in global search operations when the 'q' parameter is present.

func ParseRequest 

func ParseRequest(r any) ([]CrudFilter, []Sort, Pagination, error)

ParseRequest parses all query parameters from an HTTP request. This is maintained for backward compatibility with older code.

func ParseRequestWithSearch 

func ParseRequestWithSearch(r any, searchConfig *GlobalSearchConfig) ([]CrudFilter, []Sort, Pagination, error)

ParseRequestWithSearch parses query parameters with global search support. This is maintained for backward compatibility.

func ToQueryString added in v0.3.9 

func ToQueryString(params map[string][]string) string

ToQueryString converts a map of query parameters to a URL-encoded query string. The keys are sorted alphabetically to ensure consistent output. Example: 

params := map[string][]string{
    "name": {"john"},
    "age":  {"30"},
}

query := queryutil.ToQueryString(params) // "age=30&name=john"

Types

type CrudFilter added in v0.3.0 

type CrudFilter = filter.CrudFilter

Re-exported types

type EntityFunc added in v0.2.0 

type EntityFunc[T any] func([]CrudFilter, []Sort, Pagination) ([]T, int64, error)

EntityFunc is a generic function type for entity list operations. It takes filters, sorts, and pagination parameters and returns a slice of items, total count, and an error.

This type is commonly used for service layer functions that retrieve entities from a data source with filtering, sorting, and pagination.

Example implementation: 

func ListUsers(filters []Filter, sorts []Sort, pagination Pagination) ([]User, int64, error) {
    db := database.GetDB()
    query := db.Model(&User{})

    // Apply filters and sorts
    query = ApplyFilters(query, filters, nil)
    query = ApplySort(query, sorts)

    // Count total before pagination
    var total int64
    query.Count(&total)

    // Apply pagination
    query = query.Offset(pagination.GetOffset()).Limit(pagination.GetLimit())

    // Execute query
    var users []User
    err := query.Find(&users).Error
    return users, total, err
}

type Filter 

type Filter = filter.LogicalFilter

Re-exported types

type GlobalSearchConfig 

type GlobalSearchConfig = filter.GlobalSearchConfig

Re-exported types

type HTTPRequestParser added in v0.2.0 

type HTTPRequestParser struct {
	Query        url.Values
	SearchConfig *filter.GlobalSearchConfig
	// contains filtered or unexported fields
}

HTTPRequestParser implements the RequestParser interface for HTTP requests. It provides a way to parse query parameters from an HTTP request's URL query.

func NewHTTPRequestParser added in v0.2.0 

func NewHTTPRequestParser(r *http.Request, searchConfig *filter.GlobalSearchConfig, sortConfig *filter.SortConfig) *HTTPRequestParser

NewHTTPRequestParser creates a new HTTPRequestParser from an http.Request

func (*HTTPRequestParser) GetSortConfig added in v0.3.0 

func (p *HTTPRequestParser) GetSortConfig() *filter.SortConfig

GetSortConfig returns the parser's sort configuration

func (*HTTPRequestParser) ParseFilters added in v0.2.0 

func (p *HTTPRequestParser) ParseFilters() ([]filter.CrudFilter, error)

ParseFilters implements RequestParser.ParseFilters

func (*HTTPRequestParser) ParsePagination added in v0.2.0 

func (p *HTTPRequestParser) ParsePagination() (Pagination, error)

ParsePagination implements RequestParser.ParsePagination

func (*HTTPRequestParser) ParseSorts added in v0.2.0 

func (p *HTTPRequestParser) ParseSorts(config *filter.SortConfig) ([]filter.Sort, error)

ParseSorts implements RequestParser.ParseSorts

type JsonSchamaProvider added in v0.3.0 

type JsonSchamaProvider struct {
	// contains filtered or unexported fields
}

JsonSchamaProvider implements httputil.SchemaProvider using jsonschema reflection

func NewSchemaProvider added in v0.3.0 

func NewSchemaProvider() *JsonSchamaProvider

NewSchemaProvider creates a new schema adapter with default configuration

func (*JsonSchamaProvider) ForType added in v0.3.0 

func (a *JsonSchamaProvider) ForType(dto any) router.FieldSchema

ForType returns a FieldSchema implementation for a given DTO type

type LogicalOperator added in v0.3.0 

type LogicalOperator = filter.LogicalOperator

Re-exported types

type Operator added in v0.3.0 

type Operator = filter.Operator

Re-exported types

type Pagination 

type Pagination = filter.Pagination

Re-exported types

type PaginationError 

type PaginationError = filter.PaginationError

Re-exported types

type QueryBuilder added in v0.3.0 

type QueryBuilder interface {
	Apply(tx any, filters []filter.CrudFilter) (any, error)
}

QueryBuilder defines the interface for building query clauses

type RequestParser added in v0.2.0 

type RequestParser = parser.Parser

RequestParser re-exports the parser.Parser interface

type Response 

type Response[T any] struct {
	Data  T     `json:"data"`
	Total int64 `json:"total"`
}

Response represents a standard response structure for list operations. It includes both the data payload and a total count for pagination. This structure is compatible with Refine's Simple REST Data Provider.

func BuildResponse 

func BuildResponse[T any](data T, total int64) *Response[T]

BuildResponse creates a standard response object for list operations. Takes the data payload and total count as parameters. The total count is used for client-side pagination calculations.

Example: 

users := []User{...}
totalCount := int64(100)
response := BuildResponse(users, totalCount)
// response can be encoded to JSON and sent to the client

type SchemaWrapper added in v0.3.0 

type SchemaWrapper struct {
	// contains filtered or unexported fields
}

SchemaWrapper implements httputil.FieldSchema

func (*SchemaWrapper) FilterOperators added in v0.3.0 

func (sw *SchemaWrapper) FilterOperators() map[string][]string

FilterOperators returns allowed operators per field

func (*SchemaWrapper) SortableFields added in v0.3.0 

func (sw *SchemaWrapper) SortableFields() []string

SortableFields returns valid sort fields from the schema

type Sort 

type Sort = filter.Sort

Re-exported types

type SortConfig 

type SortConfig = filter.SortConfig

Re-exported types

type SortOrder 

type SortOrder = filter.SortOrder

Re-exported types

Source Files

View all Source files

Directories

Path Synopsis
Package filter provides utilities for parsing, validating, and applying filters, sorts, and pagination parameters from HTTP requests to database queries.
 Package filter provides utilities for parsing, validating, and applying filters, sorts, and pagination parameters from HTTP requests to database queries.
builder
Package builder provides database-agnostic query building capabilities for filters.
 Package builder provides database-agnostic query building capabilities for filters.
generator
parser
Package parser provides components for parsing filters, sorts, and pagination parameters from different input formats.
 Package parser provides components for parsing filters, sorts, and pagination parameters from different input formats.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.