rulego

README

RuleGo

English| 简体中文

Official Website | Docs | Contribution Guide

RuleGo is a lightweight, high-performance, embedded, orchestrable component-based rule engine built on the Go language.

It can help you quickly build loosely coupled and flexible systems that can respond and adjust to changes in business requirements in real time.

RuleGo also provides a large number of reusable components that support the aggregation, filtering, distribution, transformation, enrichment, and execution of various actions on data, and can also interact and integrate with various protocols and systems. It has a wide range of application potential in low-code, business code orchestration, data integration, workflows, large model intelligent agents, edge computing, automation, IoT, and other scenarios.

Features

• Lightweight: No external middleware dependencies, efficient data processing and linkage on low-cost devices, suitable for IoT edge computing.
• High Performance: Thanks to Go's high-performance characteristics, RuleGo also employs technologies such as coroutine pools and object pools.
• Dual Mode: Embedded and Standalone Deployment modes. Supports embedding RuleGo into existing applications. It can also be deployed independently as middleware, providing rule engine and orchestration services.
• Componentized: All business logic is component-based, allowing flexible configuration and reuse.
• Rule Chains: Flexibly combine and reuse different components to achieve highly customized and scalable business processes.
• Workflow Orchestration: Supports dynamic orchestration of rule chain components, replacing or adding business logic without restarting the application.
• Easy Extension: Provides rich and flexible extension interfaces, making it easy to implement custom components or introduce third-party components.
• Dynamic Loading: Supports dynamic loading of components and extensions through Go plugins.
• Nested Rule Chains: Supports nesting of sub-rule chains to reuse processes.
• Built-in Components: Includes a large number of components such as Message Type Switch, JavaScript Switch, JavaScript Filter, JavaScript Transformer, HTTP Push, MQTT Push, Send Email, Log Recording, etc. Other components can be extended as needed.
• Context Isolation Mechanism: Reliable context isolation mechanism, no need to worry about data streaming in high concurrency situations.
• AOP Mechanism: Allows adding extra behavior to the execution of rule chains or directly replacing the original logic of rule chains or nodes without modifying their original logic.
• Data Integration: Allows dynamic configuration of Endpoints, such as HTTP Endpoint, MQTT Endpoint, TCP/UDP Endpoint, UDP Endpoint, Kafka Endpoint, Schedule Endpoint, etc.

Use Cases

RuleGo is an orchestrable rule engine that excels at decoupling your systems.

• If your system's business is complex and the code is bloated
• If your business scenarios are highly customized or frequently changing
• If your system needs to interface with a large number of third-party applications or protocols
• Or if you need an end-to-end IoT solution
• Or if you need centralized processing of heterogeneous system data
• Or if you want to try hot deployment in the Go language... Then the RuleGo framework will be a very good solution.

Typical Use Cases

• Edge Computing: Deploy RuleGo on edge servers to preprocess data, filter, aggregate, or compute before reporting to the cloud. Data processing rules and distribution rules can be dynamically configured and modified through rule chains without restarting the system.
• IoT: Collect device data reports, make rule judgments through rule chains, and trigger one or more actions, such as sending emails, alarms, and linking with other devices or systems.
• Data Distribution: Distribute data to different systems using HTTP, MQTT, or gRPC based on different message types.
• Application Integration: Use RuleGo as glue to connect various systems or protocols, such as SSH, webhook, Kafka, message queues, databases, ChatGPT, third-party application systems.
• Centralized Processing of Heterogeneous System Data: Receive data from different sources (such as MQTT, HTTP, WS, TCP/UDP, etc.), then filter, format convert, and distribute to databases, business systems, or dashboards.
• Highly Customized Business: Decouple highly customized or frequently changing business and manage it with RuleGo rule chains. Business requirements change without needing to restart the main program.
• Complex Business Orchestration: Encapsulate business into custom components, orchestrate and drive these custom components through RuleGo, and support dynamic adjustment and replacement of business logic.
• Microservice Orchestration: Orchestrate and drive microservices through RuleGo, or dynamically call third-party services to process business and return results.
• Decoupling of Business Code and Logic: For example, user points calculation systems, risk control systems.
• Automation: For example, CI/CD systems, process automation systems, marketing automation systems.
• Low Code: For example, low-code platforms, iPaaS systems, ETL, LangFlow-like systems (interfacing with large models to extract user intent, then triggering rule chains to interact with other systems or process business).

Architecture Diagram

RuleGo Architecture Diagram

Rule Chain Running Example Diagram

More Running Modes

Installation

Install RuleGo using the go get command:

go get github.com/rulego/rulego
# or
go get gitee.com/rulego/rulego

Usage

RuleGo is extremely simple to use. Just follow these 3 steps:

1. Define rule chains using JSON:

   • Rule Chain DSL Doc
   • example_chain.json

2. Import the RuleGo package and use the rule chain definition to create a rule engine instance:

import "github.com/rulego/rulego"
//Load the rule chain definition file.
ruleFile := fs.LoadFile("chain_call_rest_api.json")
// Create a rule engine instance using the rule chain definition
ruleEngine, err := rulego.New("rule01", ruleFile)

1. Hand over the message payload, message type, and message metadata to the rule engine instance for processing, and then the rule engine will process the message according to the rule chain's definition:

// Define message metadata
metaData := types.NewMetadata()
metaData.PutValue("productType", "test01")
// Define message payload and message type
msg := types.NewMsg(0, "TELEMETRY_MSG", types.JSON, metaData, "{\"temperature\":35}")

// Hand over the message to the rule engine for processing
ruleEngine.OnMsg(msg)

Real time update of rule chain logic without restarting the application

Rule Engine Management API

• Dynamically update rule chains

// Dynamically update rule chain logic
err := ruleEngine.ReloadSelf(ruleFile)
// Update a node under the rule chain
ruleEngine.ReloadChild("node01", nodeFile)
// Get the rule chain definition
ruleEngine.DSL()

• Rule Engine Instance Management:

// Load all rule chain definitions in a folder into the rule engine pool
rulego.Load("/rules", rulego.WithConfig(config))
// Get an already created rule engine instance by ID
ruleEngine, ok := rulego.Get("rule01")
// Delete an already created rule engine instance
rulego.Del("rule01")

• Config:Documentation

// Create a default configuration
config := rulego.NewConfig()
// Debug node callback, the node configuration must be set to debugMode:true to trigger the call
// Both node entry and exit information will call this callback function
config.OnDebug = func (chainId,flowType string, nodeId string, msg types.RuleMsg, relationType string, err error) {
}
// Use the configuration
ruleEngine, err := rulego.New("rule01", []byte(ruleFile), rulego.WithConfig(config))

Rule Chain Definition DSL

Rule Chain Definition DSL

Rule Chain Node Components

The core feature of RuleGo is its component-based architecture, where all business logic is encapsulated in components that can be flexibly configured and reused. Currently, RuleGo has built-in a vast array of commonly used components.

• Standard Components
• rulego-components :Documentation
• rulego-components-ai
• rulego-components-ci
• rulego-components-iot
• rulego-components-etl
• rulego-marketplace :Dynamic component and rule chain marketplace
• Custom Node Component Example :Documentation

Data Integration

RuleGo provides the Endpoint module for unified data integration and processing of heterogeneous systems. For details, refer to: Endpoint

Input Endpoint Components

• Endpoint Components
• Endpoint DSL

RuleGo-Server

We have provided a ready-to-use automation workflow platform based on RuleGo: RuleGo-Server , and a front-end visual editor: RuleGo-Editor .

Performance

RuleGo completes most of its work during initialization, so running the rule chain almost doesn't add extra overhead to the system, and the resource consumption is extremely low, making it particularly suitable for running on edge servers. Additionally, RuleGo uses Directed Acyclic Graph (DAG) to represent the rule chain, where each input message only needs to be processed along the path in the graph without matching all the rules, which greatly improves the efficiency of message processing and routing, and also saves system resources.

Performance test cases:

Machine: Raspberry Pi 2 (900MHz Cortex-A7*4,1GB LPDDR2)  
Data size: 260B   
Rule chain: JS script filtering->JS complex transformation->HTTP push   
Test results: 100 concurrent and 500 concurrent, memory consumption does not change much around 19M

More performance test cases

Ecosystem

• RuleGo-Editor : Rule chain visual editor
• RuleGo-Server : Automation workflow platform
• rulego-components : Extension component library
• rulego-components-ai : AI scenario component library
• rulego-components-ci : CI/CD scenario component library
• rulego-components-iot : IoT scenario component library
• rulego-components-etl : ETL scenario component library
• rulego-marketplace : Dynamic component and rule chain marketplace

Contribution

Any form of contribution is welcome, including submitting issues, suggestions, documentation, tests or code. Contribution Guide

License

RuleGo uses Apache 2.0 license, please refer to LICENSE file for details.

Contact Us

Email: rulego@outlook.com

Documentation

Overview

Package rulego provides a lightweight, high-performance, embedded, orchestrable component-based rule engine.

Usage

Implement your business requirements by configuring components in the rule chain, and support dynamic modification. Rule chain definition format:

{
	  "ruleChain": {
		"id":"rule01"
	  },
	  "metadata": {
	    "nodes": [
	    ],
		"connections": [
		]
	 }
}

nodes:configure components. You can use built-in components and third-party extension components without writing any code.

connections:configure the relation type between components. Determine the data flow.

Example:

var ruleFile = `
{
	"ruleChain": {
	"id":"rule02",
	"name": "test",
	"root": true
	},
	"metadata": {
	"nodes": [
		{
		"id": "s1",
		"type": "jsTransform",
		"name": "transform",
		"debugMode": true,
		"configuration": {
			"jsScript": "metadata['state']='modify by js';\n msg['addField']='addValueFromJs'; return {'msg':msg,'metadata':metadata,'msgType':msgType};"
			}
		},
		{
			"id": "s2",
			"type": "restApiCall",
			"name": "push data",
			"debugMode": true,
			"configuration": {
				"restEndpointUrlPattern": "http://127.0.0.1:9090/api/msg",
				"requestMethod": "POST",
			}
		}
	],
	"connections": [
		{
			"fromId": "s1",
			"toId": "s2",
			"type": "Success"
		}
	]
	}
}
`

Create Rule Engine Instance

ruleEngine, err := rulego.New("rule01", []byte(ruleFile))

Define Message Metadata

metaData := types.NewMetadata()
metaData.PutValue("productType", "test01")

Define Message Payload And Type

msg := types.NewMsg(0, "TELEMETRY_MSG", types.JSON, metaData, "{\"temperature\":35}")

Processing Message

ruleEngine.OnMsg(msg)

Update Rule Chain

err := ruleEngine.ReloadSelf([]byte(ruleFile))

Load All Rule Chain

err := ruleEngine.Load("./rulechains")

Get Engine Instance

ruleEngine, ok := rulego.Get("rule01")

Index

• Variables
• func Del(id string)
• func Get(id string) (types.RuleEngine, bool)
• func Load(folderPath string, opts ...types.RuleEngineOption) error
• func New(id string, rootRuleChainSrc []byte, opts ...types.RuleEngineOption) (types.RuleEngine, error)
• func NewConfig(opts ...types.Option) types.Config
• func OnMsg(msg types.RuleMsg)
• func Range(f func(key, value any) bool)
• func Reload(opts ...types.RuleEngineOption)
• func Stop()
• func WithConfig(config types.Config) types.RuleEngineOption
• type RuleGo
  • func NewRuleGo() *RuleGo
  • func (g *RuleGo) Del(id string)
  • func (g *RuleGo) Get(id string) (types.RuleEngine, bool)
  • func (g *RuleGo) Load(folderPath string, opts ...types.RuleEngineOption) error
  • func (g *RuleGo) New(id string, rootRuleChainSrc []byte, opts ...types.RuleEngineOption) (types.RuleEngine, error)
  • func (g *RuleGo) OnMsg(msg types.RuleMsg)
  • func (g *RuleGo) Pool() *engine.Pool
  • func (g *RuleGo) Range(f func(key, value any) bool)
  • func (g *RuleGo) Reload(opts ...types.RuleEngineOption)
  • func (g *RuleGo) SetCallbacks(callbacks types.Callbacks)
  • func (g *RuleGo) Stop()

Variables

var Endpoints = endpoint.DefaultPool

var Registry = engine.Registry

Registry is the default registrar for rule engine components.

var Rules = &RuleGo{pool: engine.DefaultPool}

Rules is the default instance of RuleGo with the rule engine pool set to the default pool.

Functions

func Del

func Del(id string)

Del removes a rule engine instance by its ID.

func Get

func Get(id string) (types.RuleEngine, bool)

Get retrieves a rule engine instance by its ID.

func Load

func Load(folderPath string, opts ...types.RuleEngineOption) error

Load loads all rule chain configurations from the specified folder and its subFolders into the rule engine instance pool. The rule chain ID is taken from the ruleChain.id specified in the rule chain file.

func New

func New(id string, rootRuleChainSrc []byte, opts ...types.RuleEngineOption) (types.RuleEngine, error)

New creates a new RuleEngine and stores it in the RuleGo rule chain pool.

func NewConfig

func NewConfig(opts ...types.Option) types.Config

NewConfig creates a new Config and applies the options.

func OnMsg

func OnMsg(msg types.RuleMsg)

OnMsg calls all rule engine instances to process a message. All rule chains in the rule engine instance pool will attempt to process the message.

func Range

func Range(f func(key, value any) bool)

Range iterates over all rule engine instances.

func Reload

func Reload(opts ...types.RuleEngineOption)

Reload reloads all rule engine instances.

func Stop

func Stop()

Stop releases all rule engine instances.

func WithConfig

func WithConfig(config types.Config) types.RuleEngineOption

WithConfig is an option that sets the Config of the RuleEngine.

Types

type RuleGo

type RuleGo struct {
	// contains filtered or unexported fields
}

RuleGo is a pool of rule engine instances.

func NewRuleGo

func NewRuleGo() *RuleGo

NewRuleGo creates a new RuleGo instance.

func (*RuleGo) Del

func (g *RuleGo) Del(id string)

Del removes a rule engine instance by its ID.

func (*RuleGo) Get

func (g *RuleGo) Get(id string) (types.RuleEngine, bool)

Get retrieves a rule engine instance by its ID.

func (*RuleGo) Load

func (g *RuleGo) Load(folderPath string, opts ...types.RuleEngineOption) error

Load loads all rule chain configurations from the specified folder and its subFolders into the rule engine instance pool. The rule chain ID is taken from the ruleChain.id specified in the rule chain file.

func (*RuleGo) New

func (g *RuleGo) New(id string, rootRuleChainSrc []byte, opts ...types.RuleEngineOption) (types.RuleEngine, error)

New creates a new RuleEngine and stores it in the RuleGo rule chain pool. If the specified id is empty (""), the ruleChain.id from the rule chain file is used.

func (*RuleGo) OnMsg

func (g *RuleGo) OnMsg(msg types.RuleMsg)

OnMsg calls all rule engine instances to process a message. All rule chains in the rule engine instance pool will attempt to process the message.

func (*RuleGo) Pool

func (g *RuleGo) Pool() *engine.Pool

Pool returns the rule engine pool.

func (*RuleGo) Range

func (g *RuleGo) Range(f func(key, value any) bool)

Range iterates over all rule engine instances.

func (*RuleGo) Reload

func (g *RuleGo) Reload(opts ...types.RuleEngineOption)

Reload reloads all rule engine instances.

func (*RuleGo) SetCallbacks

func (g *RuleGo) SetCallbacks(callbacks types.Callbacks)

SetCallbacks sets the callbacks for the rule engine pool.

func (*RuleGo) Stop

func (g *RuleGo) Stop()

Stop releases all rule engine instances.