gosense

package module
v0.2.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 15, 2026 License: MIT Imports: 5 Imported by: 0

README

🚀 GoSense

A highly configurable, generic sensor data generation engine written in Go that can simulate any type of sensor data (medical, weather, industrial, etc.) with configurable production rates, batching, and multiple publishing options.

🚀 Latest Release

v0.2.3 - Console Publisher Added ✅

📦 Install: go get github.com/Utsav-pixel/gosense@v0.2.3

✨ What's New:

  • ✅ Added console publisher to public API
  • ✅ Complete publisher ecosystem (HTTP, Kafka, gRPC, Console)
  • ✅ Enhanced development and testing experience
  • ✅ Visual formatted output for debugging

🔄 Quick Start:

import "github.com/Utsav-pixel/gosense"

// Create publisher (console for development)
publisher := gosense.NewConsolePublisher[YourData]()

// Or use HTTP publisher
httpPublisher := gosense.NewGenericHTTPPublisher[YourData]("https://api.example.com/data")

// Create engine
engine := gosense.NewEngine(config, seeder, sensorFunc, publisher)

📖 Documentation

Features

  • Generic Type Support: Works with any data type using Go generics
  • Flexible Seeders: Multiple input generation strategies (time-based, random, linear, custom)
  • Configurable Functions: Function-based sensor data generation
  • Multiple Publishers: HTTP, Kafka, and gRPC support
  • Batch Processing: Configurable batch sizes and timeouts
  • Production Rate Control: Adjustable data generation frequency
  • Quality Simulation: Realistic data quality variations
  • Concurrent Processing: Multi-worker architecture for high throughput

Architecture

The engine consists of several key components available through the public API:

1. Generic Types

Available when you import github.com/Utsav-pixel/gosense:

  • SensorData[T]: Generic container for sensor readings
  • Seeder: Interface for input value generation
  • SensorFunction[T]: Interface for data transformation
  • Publisher[T]: Interface for data publishing
  • Engine[T]: Main engine orchestrator
2. Seeders

Available via gosense.New*Seeder() functions:

  • TimeSeeder: Time-based oscillating values
  • RandomSeeder: Random values within range
  • LinearSeeder: Linearly increasing values
  • NormalSeeder: Normal distribution values
  • CustomSeeder: Custom generation functions
3. Sensor Functions

Available via gosense.New*Function() constructors:

  • BasicSensorFunction[T]: Basic sensor data transformation
  • Function[T]: User-defined sensor data generation
  • LambdaSensorFunction[T]: Inline anonymous functions
  • WeatherSensorFunction: Weather data generation
  • CustomSensorFunction[T]: Custom transformation functions
4. Publishers

Available via gosense.New*Publisher() functions:

  • GenericHTTPPublisher[T]: HTTP/REST API publishing
  • GenericKafkaPublisher[T]: Apache Kafka publishing
  • GenericGRPCPublisher[T]: gRPC streaming
  • ConsolePublisher[T]: Console output for development and testing

📊 Performance Benchmarks

Real benchmark results (macOS ARM64, 8 cores, Go 1.24.1):

  • 🚀 Throughput: 980 data points/second
  • 💾 Memory Usage: 69MB per 100K concurrent data points
  • ⚡ Latency: 1.0ms per data point (1000 microseconds)
Performance by Configuration
Configuration Use Case Throughput Latency Memory
HighThroughputConfig Data pipelines 980/sec 1ms 69MB/100K
LowLatencyConfig Real-time systems 500/sec 0.5ms 35MB/100K
DefaultConfig General purpose 300/sec 1ms 25MB/100K

💡 Performance Tips:

  • Use HighThroughputConfig for bulk data processing
  • Use LowLatencyConfig for real-time applications
  • Adjust MaxWorkers based on your CPU cores
  • Tune BatchSize for your publisher's optimal throughput

Quick Start

Installation
As a Library
go get github.com/Utsav-pixel/gosense
From Source
git clone https://github.com/Utsav-pixel/gosense.git
cd gosense
go mod tidy
go build ./cmd/sensor-engine
Running Examples
# Weather sensor with HTTP publisher (default)
./sensor-engine -type=weather -publisher=http -duration=30s

# Medical sensor with Kafka publisher
./sensor-engine -type=medical -publisher=kafka -brokers=localhost:9092 -topic=medical.data

# Industrial machinery sensor with gRPC publisher
./sensor-engine -type=machinery -publisher=grpc -grpc=localhost:50051

# Legacy pasture simulation
./sensor-engine -type=legacy
Command Line Options
  • -type: Sensor type (medical, weather, machinery, legacy)
  • -publisher: Publisher type (http, kafka, grpc)
  • -duration: How long to run the engine
  • -endpoint: HTTP endpoint URL
  • -brokers: Kafka broker addresses
  • -topic: Kafka topic name
  • -grpc: gRPC server address

Usage Examples

Medical Sensor Example
import "github.com/Utsav-pixel/gosense"

// Configuration for medical sensors
config := gosense.DefaultConfig()
config.ProductionRate = 1 * time.Second // Generate data every second
config.BatchSize = 10
config.BatchTimeout = 5 * time.Second

// Create a stress level seeder (0.0 to 1.0)
stressSeeder := gosense.NewNormalSeeder(0.5, 0.2) // Patient variability

// Create medical sensor function with your own logic
medicalFunc := gosense.NewFunction(func(input float64, timestamp time.Time) MedicalData {
    // Your business logic here
    heartRate := 70 + int(input*40) // Stress increases heart rate
    bloodPressure := BloodPressure{120 + int(input*20), 80 + int(input*15)}
    oxygenLevel := 95 + input*5 // Stress affects oxygen level
    temperature := 36.5 + input*2 // Stress affects temperature
    
    return MedicalData{
        HeartRate:     heartRate,
        BloodPressure: bloodPressure,
        OxygenLevel:   oxygenLevel,
        Temperature:   temperature,
    }
})

// Create publisher
httpPublisher := gosense.NewGenericHTTPPublisher[MedicalData]("https://api.medical.example.com/vitals")

// Or use Kafka publisher
kafkaPublisher := gosense.NewGenericKafkaPublisher[MedicalData](
    []string{"localhost:9092"},
    "medical.data",
)

// Or use gRPC publisher
grpcPublisher, err := gosense.NewGenericGRPCPublisher[MedicalData]("localhost:50051")

// Create and start engine
medicalEngine := gosense.NewEngine(config, stressSeeder, medicalFunc, httpPublisher)
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

if err := medicalEngine.Start(ctx); err != nil {
    log.Printf("Engine error: %v", err)
}
Weather Sensor Example
// High-throughput configuration
config := gosense.HighThroughputConfig()
config.ProductionRate = 500 * time.Millisecond

// Weather pattern seeder
weatherSeeder := gosense.NewTimeSeeder(1.0, 0.05, 0.0)

// Weather sensor function with your own logic
weatherFunc := gosense.NewFunction(func(input float64, timestamp time.Time) WeatherData {
    // Your business logic here
    hour := float64(timestamp.Hour())
    dayOfYear := float64(timestamp.YearDay())
    
    // Temperature follows normal pattern with seasonal variation
    seasonalTemp := 15.0 + 10.0*math.Sin((dayOfYear/365.0)*2*math.Pi-math.Pi/2)
    dailyTemp := 5.0*math.Sin((hour/24.0)*2*math.Pi-math.Pi/2)
    temperature := seasonalTemp + dailyTemp + (input-0.5)*10.0
    
    // Humidity inversely related to temperature
    humidity := 70.0 - temperature + (rand.Float64()-0.5)*20.0
    if humidity < 20.0 { humidity = 20.0 } else if humidity > 95.0 { humidity = 95.0 }
    
    // Pressure varies with weather systems
    pressure := 1013.25 + (input-0.5)*50.0 + (rand.Float64()-0.5)*10.0
    
    // Wind speed
    windSpeed := math.Max(0, 10.0+input*30.0+(rand.Float64()-0.5)*5.0)
    
    return WeatherData{
        Temperature:  temperature,
        Humidity:    humidity,
        Pressure:    pressure,
        WindSpeed:   windSpeed,
    }
})

// Kafka publisher for high throughput
kafkaPublisher := gosense.NewGenericKafkaPublisher[WeatherData](
    []string{"localhost:9092"},
    "weather.data.v1",
)

// Create and start engine
weatherEngine := gosense.NewEngine(config, weatherSeeder, weatherFunc, kafkaPublisher)
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Minute)
defer cancel()

if err := weatherEngine.Start(ctx); err != nil {
    log.Printf("Engine error: %v", err)
}
Custom Sensor Example
// Custom seeder that simulates market behavior
type MarketSeeder struct {
    cycle float64
}

func (m *MarketSeeder) Generate() float64 {
    m.cycle += 0.1
    baseValue := 0.5
    
    // Add market cycles
    cycle := math.Sin(m.cycle * 0.1) * 0.3
    
    // Add random market noise
    noise := (rand.Float64() - 0.5) * 0.2
    
    // Add trend component
    trend := math.Sin(m.cycle * 0.01) * 0.2
    
    result := baseValue + cycle + noise + trend
    
    // Keep within bounds
    if result < 0 { result = 0 } else if result > 1 { result = 1 }
    
    return result
}

// Custom sensor function with your own logic
customFunc := gosense.NewFunction(func(input float64, timestamp time.Time) YourData {
    // Your business logic here
    value := input * 100.0
    status := "normal"
    if value > 80 { status = "high" } else if value > 60 { status = "medium" }
    
    return YourData{
        Value:    value,
        Status:   status,
        Location: "sensor-001",
    }
})

// Create publisher
consolePublisher := NewConsolePublisher[YourData]()

// Create and start engine
customEngine := gosense.NewEngine(config, &MarketSeeder{cycle: 0}, customFunc, consolePublisher)
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

if err := customEngine.Start(ctx); err != nil {
    log.Printf("Engine error: %v", err)
}

Configuration Presets

DefaultConfig
  • Production Rate: 100ms
  • Batch Size: 100
  • Batch Timeout: 500ms
  • Max Workers: 3
HighThroughputConfig
  • Production Rate: 10ms
  • Batch Size: 1000
  • Batch Timeout: 100ms
  • Max Workers: 10
LowLatencyConfig
  • Production Rate: 50ms
  • Batch Size: 10
  • Batch Timeout: 25ms
  • Max Workers: 5

Data Quality

The engine simulates realistic data quality variations:

  • OK (89%): Normal quality data
  • NOISY (5%): Data with some noise
  • PARTIAL (2%): Partially complete data
  • CORRUPT (1%): Corrupted data

Extending the Engine

Adding Custom Seeders
type MyCustomSeeder struct {
    // your fields
}

func (m *MyCustomSeeder) Generate() float64 {
    // your implementation
}
Adding Custom Sensor Functions
type MyCustomSensorFunction struct {
    // your fields
}

func (m *MyCustomSensorFunction) Generate(input float64, timestamp time.Time) MyDataType {
    // your implementation
}
Adding Custom Publishers
type MyCustomPublisher[T any] struct {
    // your fields
}

func (m *MyCustomPublisher[T]) Publish(ctx context.Context, data gosense.SensorData[T]) error {
    // your implementation
}

func (m *MyCustomPublisher[T]) PublishBatch(ctx context.Context, data []gosense.SensorData[T]) error {
    // your implementation
}

func (m *MyCustomPublisher[T]) Close() error {
    // your implementation
}

Performance Considerations

  • Use HighThroughputConfig for high-volume data generation
  • Use LowLatencyConfig for real-time applications
  • Adjust batch sizes based on your downstream system capacity
  • Consider using Kafka for high-throughput scenarios
  • Use gRPC for low-latency real-time monitoring

Dependencies

  • Go 1.24+
  • google.golang.org/grpc v1.65.0
  • google.golang.org/protobuf v1.34.2
  • github.com/segmentio/kafka-go v0.4.50

License

This project is licensed under the MIT License - see the LICENSE file for details.

Documentation

Overview

Package gosense provides a generic sensor data generation engine. It supports various seeders, sensor functions, and publishers for generating realistic sensor data.

Index

Constants

View Source 
const (
	QualityOK      = engine.QualityOK
	QualityNoisy   = engine.QualityNoisy
	QualityPartial = engine.QualityPartial
	QualityCorrupt = engine.QualityCorrupt
)

Re-export quality constants

Variables

This section is empty.

Functions

This section is empty.

Types

type BasicSensorFunction 

type BasicSensorFunction[T any] = engine.BasicSensorFunction[T]

BasicSensorFunction provides a simple implementation for basic sensor data generation

func NewBasicSensorFunction 

func NewBasicSensorFunction[T any](transformFunc func(float64, time.Time) T) *BasicSensorFunction[T]

NewBasicSensorFunction creates a new basic sensor function with a custom transform function

type Config 

type Config = engine.Config

Config holds the engine configuration

func DefaultConfig 

func DefaultConfig() Config

Re-export configuration functions

func HighThroughputConfig 

func HighThroughputConfig() Config

func LowLatencyConfig 

func LowLatencyConfig() Config

type ConsolePublisher added in v0.2.3 

type ConsolePublisher[T any] struct{}

ConsolePublisher is a simple publisher that outputs sensor data to the console Useful for development, testing, and demonstration purposes

func NewConsolePublisher added in v0.2.3 

func NewConsolePublisher[T any]() *ConsolePublisher[T]

NewConsolePublisher creates a new console publisher

func (*ConsolePublisher[T]) Close added in v0.2.3 

func (p *ConsolePublisher[T]) Close() error

Close closes the console publisher

func (*ConsolePublisher[T]) Publish added in v0.2.3 

func (p *ConsolePublisher[T]) Publish(ctx context.Context, data SensorData[T]) error

Publish publishes a single sensor data point to the console

func (*ConsolePublisher[T]) PublishBatch added in v0.2.3 

func (p *ConsolePublisher[T]) PublishBatch(ctx context.Context, data []SensorData[T]) error

PublishBatch publishes a batch of sensor data points to the console

type CustomSeeder 

type CustomSeeder = engine.CustomSeeder

CustomSeeder allows for custom generation functions

func NewCustomSeeder 

func NewCustomSeeder(generateFunc func() float64) *CustomSeeder

NewCustomSeeder creates a new custom seeder

type Engine 

type Engine[T any] = engine.Engine[T]

Engine is the generic sensor engine

func NewEngine 

func NewEngine[T any](
	config Config,
	seeder Seeder,
	function SensorFunction[T],
	publisher Publisher[T],
) *Engine[T]

NewEngine creates a new generic sensor engine

type Function 

type Function[T any] = engine.Function[T]

Function allows users to define their own sensor data generation logic

func NewFunction 

func NewFunction[T any](generateFunc func(float64, time.Time) T) *Function[T]

NewFunction creates a new user-defined sensor function

type GenericGRPCPublisher added in v0.2.1 

type GenericGRPCPublisher[T any] = publisher.GenericGRPCPublisher[T]

GenericGRPCPublisher is a generic gRPC publisher

func NewGenericGRPCPublisher added in v0.2.1 

func NewGenericGRPCPublisher[T any](address string) (*GenericGRPCPublisher[T], error)

NewGenericGRPCPublisher creates a new generic gRPC publisher

type GenericHTTPPublisher added in v0.2.1 

type GenericHTTPPublisher[T any] = publisher.GenericHTTPPublisher[T]

GenericHTTPPublisher is a generic HTTP publisher

func NewGenericHTTPPublisher added in v0.2.1 

func NewGenericHTTPPublisher[T any](endpoint string) *GenericHTTPPublisher[T]

NewGenericHTTPPublisher creates a new generic HTTP publisher

type GenericKafkaPublisher added in v0.2.1 

type GenericKafkaPublisher[T any] = publisher.GenericKafkaPublisher[T]

GenericKafkaPublisher is a generic Kafka publisher

func NewGenericKafkaPublisher added in v0.2.1 

func NewGenericKafkaPublisher[T any](brokers []string, topic string) *GenericKafkaPublisher[T]

NewGenericKafkaPublisher creates a new generic Kafka publisher

type LambdaSensorFunction 

type LambdaSensorFunction[T any] = engine.LambdaSensorFunction[T]

LambdaSensorFunction provides a simple function wrapper for inline usage

func NewLambdaSensorFunction 

func NewLambdaSensorFunction[T any](lambda func(float64, time.Time) T) *LambdaSensorFunction[T]

NewLambdaSensorFunction creates a sensor function from a lambda/anonymous function

type LinearSeeder 

type LinearSeeder = engine.LinearSeeder

LinearSeeder generates values that increase linearly over time

func NewLinearSeeder 

func NewLinearSeeder(slope, offset float64) *LinearSeeder

NewLinearSeeder creates a new linear seeder

type NormalSeeder 

type NormalSeeder = engine.NormalSeeder

NormalSeeder generates values from a normal distribution

func NewNormalSeeder 

func NewNormalSeeder(mean, stdDev float64) *NormalSeeder

NewNormalSeeder creates a new normal distribution seeder

type Publisher 

type Publisher[T any] = engine.Publisher[T]

Publisher defines the interface for publishing sensor data

type Quality 

type Quality = engine.Quality

Quality represents the quality of sensor data

type RandomSeeder 

type RandomSeeder = engine.RandomSeeder

RandomSeeder generates random values within a range

func NewRandomSeeder 

func NewRandomSeeder(min, max float64) *RandomSeeder

NewRandomSeeder creates a new random seeder

type Seeder 

type Seeder = engine.Seeder

Seeder generates input values for sensor functions

type SensorData 

type SensorData[T any] = engine.SensorData[T]

SensorData represents any sensor reading with generic data

type SensorFunction 

type SensorFunction[T any] = engine.SensorFunction[T]

SensorFunction defines the interface for sensor data generation functions

type TimeSeeder 

type TimeSeeder = engine.TimeSeeder

TimeSeeder generates values based on time

func NewTimeSeeder 

func NewTimeSeeder(amplitude, frequency, offset float64) *TimeSeeder

NewTimeSeeder creates a new time-based seeder

Source Files

View all Source files

Directories

Path Synopsis
cmd
sensor-engine command
test-engine command
internal
engine
publisher

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.