marketstream

README

MarketStream

🚀 Free Self-Hosted Real-Time Stock Data Platform

A robust HTTP API service that provides real-time market data and historical candle data using the TradingView WebSocket Client library. MarketStream gives you unlimited access to TradingView's data through your own self-hosted platform.

🚀 Features

Core Capabilities

• REST API for real-time market data access
• Real-time quote data with comprehensive market information
• Historical candle data with multiple timeframes
• Dynamic symbol management (add/remove subscriptions during runtime)
• PostgreSQL integration for persistent data storage
• High-performance caching with Ristretto for real-time quotes
• Automatic database migrations with Ent ORM

Enhanced Architecture

• Clean interfaces for better testability and dependency injection
• Message router pattern for extensible WebSocket message processing
• Structured error handling with context and retry logic
• Comprehensive input validation for security and data integrity
• Advanced configuration system with YAML, environment variables, and CLI flags
• Development tooling with Makefile automation

Technical Indicators Framework

• Comprehensive technical analysis supporting 10+ indicators
• Study session management for grouping related indicators
• Real-time calculation with candle data integration
• REST API endpoints for indicator CRUD operations

📦 Installation

go get github.com/iiiyu/marketstream

⚡ Quick Start

1. Setup Development Environment

# Clone the repository
git clone https://github.com/iiiyu/marketstream
cd marketstream

# Install development tools and dependencies
make setup

# Copy example configuration
cp config.example.yaml config.yaml

2. Configure the Application

Edit config.yaml with your settings:

server:
  port: "3333"
  environment: "development"

database:
  host: "localhost"
  port: 5432
  user: "postgres"
  password: "your_password"
  db_name: "tradingview"
  ssl_mode: "disable"

tradingview:
  device_token: "your_device_token_here"
  session_id: "your_session_id_here" 
  session_sign: "your_session_signature_here"

logging:
  level: "info"
  format: "json"

Or use environment variables:

# Database Configuration
DB_HOST=localhost
DB_PORT=5432
DB_USER=postgres
DB_PASSWORD=your_password
DB_NAME=tradingview
DB_SSLMODE=disable

# TradingView Configuration (Required)
TRADINGVIEW_DEVICE_TOKEN=your_device_token
TRADINGVIEW_SESSION_ID=your_session_id
TRADINGVIEW_SESSION_SIGN=your_session_signature

# API Authentication (Required for production)
API_KEY=your_secure_api_key_here
AUTH_ENABLED=true

# Optional Server Configuration
PORT=3333
LOG_LEVEL=info

3. Run the Application

# Development mode with live reload
make dev

# Or run directly
make run

# Or build and run
make build
./bin/tradingview-http-service

🔐 API Authentication

MarketStream uses API key authentication to secure all endpoints except /health.

Configuration

Set your API key in one of the following ways:

Environment Variables:

API_KEY=your_secure_api_key_here
AUTH_ENABLED=true

Configuration File (config.yaml):

auth:
  enabled: true
  api_key: "your_secure_api_key_here"

Command Line:

./marketstream --auth.enabled=true --auth.api_key=your_key

Making Authenticated Requests

Include your API key in requests using one of these methods:

Option 1: X-API-Key Header

curl -H "X-API-Key: your_api_key_here" http://localhost:3333/symbols

Option 2: Authorization Bearer Token

curl -H "Authorization: Bearer your_api_key_here" http://localhost:3333/symbols

Response Codes

• 200 - Success
• 401 - Unauthorized (missing or invalid API key)
• 400 - Bad Request
• 500 - Internal Server Error

📡 API Documentation

Health Check (Public - No API Key Required)

GET /health

Symbol Management

# Add a symbol subscription
POST /symbols
{
  "exchange": "NASDAQ",
  "symbol": "AAPL", 
  "type": "quote"  # or "candle"
  "timeframe": "1D" # required for candle type
}

# Remove a symbol subscription  
DELETE /symbols
{
  "exchange": "NASDAQ",
  "symbol": "AAPL",
  "type": "quote"
}

# List active subscriptions
GET /symbols

# Get subscriptions by exchange/symbol
GET /symbols/NASDAQ/AAPL

Data Access

# Get real-time quote data
GET /quotes/NASDAQ/AAPL

# Get historical candlestick data
GET /candles/NASDAQ/AAPL/1D/100  # exchange/symbol/timeframe/limit

# Force reconnection
GET /reconnect

Technical Indicators

# Create a study session
POST /study-sessions
{
  "name": "My Analysis",
  "symbol": "NASDAQ:AAPL",
  "interval": "1D",
  "max_bars": 1000
}

# Add indicators to a session
POST /indicators
{
  "study_session_id": "session_uuid",
  "type": "RSI",
  "parameters": {"period": 14}
}

# Get indicator data
GET /indicators/{indicator_id}/data

🏗 Architecture

Data Flow

1. HTTP API → Manages symbol subscriptions and data requests
2. TradingView WebSocket Client → Connects to TradingView real-time data
3. Message Processing → Routes and processes incoming market data
4. Data Storage → Quote data cached in Ristretto, candle data persisted to PostgreSQL
5. API Response → Serves cached quotes and historical candles via REST API

Database Schema

The service uses PostgreSQL with Ent ORM and includes these entities:

• Symbol: Comprehensive symbol metadata with exchange, type, sector information
• Candle: OHLCV data with quality indicators and market microstructure fields
• ActiveSession: Real-time subscription tracking with status and error handling
• StudySession: Technical analysis sessions for indicator calculations
• Indicator: Individual technical indicators with parameters
• IndicatorData: Time-series calculated indicator values

🔧 Configuration

Configuration Methods (Priority Order)

1. Command line flags: --server.port=3333
2. Environment variables: PORT=3333
3. Configuration file: config.yaml
4. Default values

Complete Configuration Reference

server:
  port: "3333"
  host: "0.0.0.0"
  environment: "development"  # development, staging, production
  read_timeout: "30s"
  write_timeout: "30s"

database:
  host: "localhost"
  port: 5432
  user: "postgres"
  password: "secret"
  db_name: "tradingview"
  ssl_mode: "disable"
  max_open_conns: 25
  max_idle_conns: 5

tradingview:
  device_token: "required"
  session_id: "required"
  session_sign: "required"
  base_url: "https://www.tradingview.com"

websocket:
  url: "wss://prodata.tradingview.com/socket.io/websocket?from=screener%2F"
  max_retries: 5
  ping_interval: "30s"
  read_timeout: "60s"

cache:
  max_cost: 1073741824  # 1GB
  num_counters: 10000000
  default_ttl: "1h"

logging:
  level: "info"          # debug, info, warn, error
  format: "json"         # json, text

🛠 Development

Available Commands

make help           # Show all available commands
make dev            # Start development server with live reload
make build          # Build the application
make test           # Run tests
make test-cover     # Run tests with coverage
make lint           # Run linter
make fmt            # Format code
make clean          # Clean build artifacts
make docker-build   # Build Docker image

Project Structure

├── cmd/
│   ├── app/          # Main HTTP service application
│   ├── example/      # Example usage
│   └── test-config/  # Configuration testing tool
├── internal/
│   ├── config/       # Configuration management
│   ├── dto/          # Data Transfer Objects
│   ├── handler/      # HTTP API handlers
│   └── service/      # Business logic services
├── ent/              # Database schema and ORM (Ent)
├── docs/             # Documentation
└── deploy/           # Deployment configurations

🐳 Docker Deployment

Using Docker Compose (Recommended)

The service includes a complete Docker Compose setup with PostgreSQL database:

# Navigate to docker directory
cd deploy/docker

# Copy environment template
cp .env.example .env

# Edit .env with your credentials
nano .env

# Start services (includes PostgreSQL + TradingView service)
docker-compose up -d

# View logs
docker-compose logs -f

# Stop services
docker-compose down

Using Docker Build

# Build Docker image
make docker-build

# Run with external database
docker run -p 3333:3333 \
  -e DB_HOST=your_db_host \
  -e DB_USER=postgres \
  -e DB_PASSWORD=your_password \
  -e DB_NAME=marketstream \
  -e TRADINGVIEW_DEVICE_TOKEN=your_token \
  -e TRADINGVIEW_SESSION_ID=your_session_id \
  -e TRADINGVIEW_SESSION_SIGN=your_signature \
  -e API_KEY=your_secure_api_key \
  -e AUTH_ENABLED=true \
  marketstream:latest

☁️ Coolify Deployment

Coolify is a self-hosted alternative to Heroku/Netlify. Deploy with these steps:

1. Fork/Clone Repository

git clone https://github.com/your-username/marketstream
cd marketstream

2. Create New Project in Coolify

1. Login to your Coolify dashboard
2. Create new project: "MarketStream"
3. Choose "Docker Compose" as deployment type
4. Set repository URL to your fork
5. Set docker-compose file path: deploy/docker/docker-compose.yml

3. Configure Environment Variables

Add these secrets in Coolify dashboard:

TRADINGVIEW_DEVICE_TOKEN=your_device_token_here
TRADINGVIEW_SESSION_ID=your_session_id_here
TRADINGVIEW_SESSION_SIGN=your_session_signature_here
DB_PASSWORD=your_secure_postgres_password
API_KEY=your_secure_api_key_here

4. Deploy

1. Click "Deploy" in Coolify dashboard
2. Monitor deployment logs
3. Service will be available at your configured domain
4. Health check endpoint: https://your-domain.com/health

5. Post-Deployment

• PostgreSQL data persisted in Docker volume
• Automatic health checks configured
• Auto-restart on failure enabled
• Logs available in Coolify dashboard

🧪 Testing

# Run all tests
make test

# Run tests with coverage
make test-cover

# Run benchmarks
make benchmark

# Test configuration
go run cmd/test-config/main.go

📊 API Examples

Subscribe to Apple Stock Quotes

curl -X POST http://localhost:3333/symbols \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your_api_key_here" \
  -d '{"exchange":"NASDAQ","symbol":"AAPL","type":"quote"}'

Get Real-time Quote Data

curl -H "X-API-Key: your_api_key_here" http://localhost:3333/quotes/NASDAQ/AAPL

Subscribe to Bitcoin Candlestick Data

curl -X POST http://localhost:3333/symbols \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your_api_key_here" \
  -d '{"exchange":"BINANCE","symbol":"BTCUSDT","type":"candle","timeframe":"1D"}'

Get Historical Candle Data

curl -H "X-API-Key: your_api_key_here" http://localhost:3333/candles/BINANCE/BTCUSDT/1D/50

📋 Requirements

• Go 1.24+
• PostgreSQL 12+
• TradingView credentials (device token, session ID, session signature)

🔗 Dependencies

• TradingView WebSocket Client - Core WebSocket functionality
• Ent - Database ORM
• Fiber - HTTP web framework
• Ristretto - High-performance cache
• Viper - Configuration management

🤝 Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Make your changes
4. Run tests (make test)
5. Commit your changes (git commit -m 'Add amazing feature')
6. Push to the branch (git push origin feature/amazing-feature)
7. Open a Pull Request

Development Guidelines

• Follow Go best practices and idioms
• Add tests for new functionality
• Update documentation for API changes
• Use the provided Makefile commands
• Ensure all checks pass (make check)

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🔗 Related Projects

• TradingView WebSocket Client - Go library for TradingView WebSocket connections

🆘 Support

If you encounter any issues:

1. Check the configuration examples
2. Run configuration test: go run cmd/test-config/main.go
3. Enable debug logging (LOG_LEVEL=debug)
4. Check the TradingView WebSocket Client documentation
5. Open an issue with detailed information