yfin

README

yfin — Yahoo Finance Client for Go

⚠️ IMPORTANT DISCLAIMER ⚠️

This project is NOT affiliated with, endorsed by, or sponsored by Yahoo Finance or Yahoo Inc.

This is an independent, open-source Go client that accesses publicly available Yahoo Finance HTTP endpoints and web pages. Yahoo Finance does not provide an official supported API contract for this data, so endpoint or page changes can break the client.

Use at your own risk. Yahoo Finance may change their website structure at any time, which could break this client. We make no guarantees about data accuracy, availability, or compliance with Yahoo Finance's terms of service.

Legal Notice: Users are responsible for ensuring their use of this software complies with Yahoo Finance's terms of service and applicable laws in their jurisdiction.

---

🎯 Problem We're Solving

The Challenge: Most financial data clients suffer from inconsistent data formats, unreliable APIs, and poor error handling. When building financial applications, developers often face:

• Inconsistent Data Formats: Different APIs return data in various shapes and formats
• Floating Point Precision Issues: Financial calculations require exact decimal precision
• Rate Limiting Problems: Unbounded requests lead to API bans and throttling
• Poor Error Handling: Limited retry logic and circuit breaking
• Currency Conversion Complexity: Multi-currency support is often missing or buggy
• No Standardization: Each client has its own data structures and conventions

Our Solution: A production-grade Go client that provides:

✅ Standardized Data Formats - One canonical model.* shape per concept, whatever the source (API or scrape)
✅ High Precision Decimals - Scaled decimal arithmetic for financial accuracy
✅ Robust Rate Limiting - Built-in backoff, circuit breakers, and QPS rate limiting
✅ Multi-Currency Support - Automatic currency conversion with FX providers
✅ Production Ready - Comprehensive error handling, observability, and monitoring
✅ Easy Integration - Simple API with both library and CLI interfaces

---

🚀 Installation

As a Go Module

go get github.com/bizshuk/yfin

From Source

package main lives at the repo root — there is no cmd/yfin directory.

git clone https://github.com/bizshuk/yfin.git
cd yfin
make build          # or: go build -o yfin .

---

🧱 Architecture

Dependencies point strictly downward — the graph is a DAG, and model/ sits at the bottom importing nothing internal.

flowchart TD
    C["cmd/* — CLI"] --> F["facade — public contract"]
    C --> FMT["cmd/format — shared CLI formatters"]
    F --> S["svc/{yahoo,scrape,twse} — fetch + decode"]
    S --> M["model — types + normalization"]
    F --> M
    FMT --> M
    S --> H["utils/httpx"]

The contract is cmd → facade → svc → model. The CLI never reaches into svc/*; every fetch goes through facade.Client, which is the same handle external consumers use. That means anything the CLI can do, your code can do — there is no privileged internal path.

Downstream packages should import facade/ (convenience, float64 prices) or model/ (raw types, ScaledDecimal precision). Never svc/*.

Canonical domain terms are defined in docs/terminology.md.

---

📖 Quick Start

Basic Usage

package main

import (
    "context"
    "fmt"
    "log"
    "time"

    "github.com/bizshuk/yfin/facade"
)

func main() {
    // Create a new client (uses default HTTP config).
    client := facade.NewClient()
    ctx := context.Background()

    // Fetch daily bars for Apple
    start := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)
    end := time.Date(2024, 1, 31, 0, 0, 0, 0, time.UTC)

    // facade returns plain structs with float64 prices — no ScaledDecimal math.
    batch, err := client.FetchDailyBars(ctx, "AAPL", start, end, true, "my-run-id")
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("Fetched %d bars for %s\n", len(batch.Bars), batch.Symbol)
    for _, bar := range batch.Bars {
        fmt.Printf("Date: %s, Close: %.4f %s\n",
            bar.Date, bar.Close, bar.CurrencyCode)
    }
}

---

🔧 API Reference

Client Creation

// Default client with standard configuration
client := facade.NewClient()

// Custom HTTP config (QPS, retries, timeout, circuit breaker)
client := facade.NewClientWithConfig(&httpx.Config{ /* ... */ })

Available Functions

📊 Historical Data

FetchDailyBars - Get daily OHLCV data

bars, err := client.FetchDailyBars(ctx, "AAPL", start, end, adjusted, runID)

FetchIntradayBars - Get intraday data (1m, 5m, 15m, 30m, 60m)

bars, err := client.FetchIntradayBars(ctx, "AAPL", start, end, "1m", runID)

Note: Intraday data may not be available for all symbols and may return HTTP 422 errors for some requests.

FetchWeeklyBars - Get weekly OHLCV data

bars, err := client.FetchWeeklyBars(ctx, "AAPL", start, end, adjusted, runID)

FetchMonthlyBars - Get monthly OHLCV data

bars, err := client.FetchMonthlyBars(ctx, "AAPL", start, end, adjusted, runID)

💰 Real-time Data

FetchQuote - Get current market quote

quote, err := client.FetchQuote(ctx, "AAPL", runID)

FetchMarketData - Get comprehensive market data

marketData, err := client.FetchMarketData(ctx, "AAPL", runID)

🏢 Company Information

FetchCompanyInfo - Get basic company information

companyInfo, err := client.FetchCompanyInfo(ctx, "AAPL", runID)

FetchFundamentalsQuarterly - Get quarterly financials (requires paid subscription)

fundamentals, err := client.FetchFundamentalsQuarterly(ctx, "AAPL", runID)

Annual statements use the free fundamentals-timeseries endpoint:

income, err := client.FetchIncomeStatement(ctx, "AAPL")
balance, err := client.FetchBalanceSheet(ctx, "AAPL")
cashflow, err := client.FetchCashFlowStatement(ctx, "AAPL")
news, err := client.FetchNews(ctx, "AAPL")

Read-through raw cache

Downstream applications choose the app-owned raw root; yfin owns provider access, cache identity, refresh policy, path validation, and atomic writes.

barsClient, err := facade.NewCachedClient(rawRoot)
twseClient, err := facade.NewCachedTwseClient(rawRoot)
foundationClient, err := facade.NewCachedFoundationClient(rawRoot)

bars, err := barsClient.FetchDailyBars(ctx, "AAPL", start, end, true, runID)
artifact, err := foundationClient.Fetch(
    ctx, "info", "AAPL", runID, false,
)

The cache layouts are:

yahoo/bars/<symbol>/<start>_<end>_<adjusted>.json
twse/<endpoint>/<date>_<query-sha256>.json
<foundation-command>/<ticker>.<YYYY-MM-DD>.json

facade.FoundationCommands() returns the ordered 30-command surface. FetchFoundation and CachedFoundationClient.Fetch keep generic foundation dispatch inside the SDK instead of downstream applications.

---

🗂️ Facade Layer (Reflection-free Plain Go Structs)

The facade package exposes normalized Yahoo Finance models (bars, quotes, company info, market data, fundamentals, news) as plain Go structs that use float64 for prices and standard time.Time for timestamps, instead of the internal ScaledDecimal representations. External consumers should use facade.Client directly — no manual conversion is needed.

facade.Client is the public contract for downstream packages (stock, data, ...). It is the same handle that the CLI uses internally via cmd.CreateClient().

Usage Example

package main

import (
    "context"
    "fmt"
    "log"
    "time"

    "github.com/bizshuk/yfin/facade"
)

func main() {
    client := facade.NewClient()
    ctx := context.Background()

    // facade.Client returns plain structs directly — no ScaledDecimal math.
    batch, err := client.FetchDailyBars(ctx, "AAPL",
        time.Now().AddDate(0, 0, -5), time.Now(), true, "run-id")
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("Symbol: %s (MIC=%s)\n", batch.Symbol, batch.MIC)
    for _, bar := range batch.Bars {
        // Close is a float64 directly
        fmt.Printf("Date: %s, Close: %.2f %s\n",
            bar.Date, bar.Close, bar.CurrencyCode)
    }
}

Available Public Types

• *facade.BarBatch — {Symbol, MIC, []Bar}; each Bar has Date (UTC YYYY-MM-DD), Open/High/Low/Close as float64, Volume int64, Adjusted bool, CurrencyCode string.
• *facade.Quote — {Symbol, Price float64, Currency, EventTime time.Time}.
• *facade.CompanyInfo — string pass-through of Security metadata (Symbol, LongName, Exchange, Currency, Timezone, ...).
• *facade.MarketData — nullable *float64 price fields + *int64 volume; nil = missing (not zero).
• *facade.FundamentalsSnapshot — annual statements via fundamentals-timeseries or quarterly fundamentals via the paid quoteSummary surface.
• []facade.NewsItem — {Title, URL, Source, Summary, PublishedAt, Symbols}.

The raw FromBarBatch / FromQuote / FromCompanyInfo converters are also exported for callers that already hold a *model.Normalized* value, but new code should prefer the facade.Client methods, which return the plain structs directly.

Need ScaledDecimal precision instead of float64? Use the Norm variants — FetchDailyBarsNorm / FetchQuoteNorm / FetchFundamentalsNorm / FetchMarketDataNorm — which stop at the normalization step and hand back the *model.Normalized* types.

---

🕸️ Scrape Fallback System

For fields that still lack a stdlib-compatible JSON endpoint, yfin exposes explicit web-scraping methods with the same model types. Callers choose these Scrape* methods directly; there is no implicit API-to-scrape switch.

Key Features

• Automatic Fallback: Seamlessly switches between API and scraping
• Data Consistency: Identical output formats regardless of source
• Production Safety: Respects robots.txt, implements proper rate limiting
• Comprehensive Coverage: Access data not available through APIs

Supported Scrape Endpoints

• Key Statistics: P/E ratios, market cap, financial metrics
• Financials: Income statements, balance sheets, cash flow
• Analysis: Comprehensive analyst data including:
  • Earnings estimates (current/next quarter, current/next year)
  • EPS trends (current estimate, 7/30/60/90 days ago)
  • EPS revisions (up/down revisions in last 7/30 days)
  • Revenue estimates (quarterly and annual)
  • Growth estimates
• Analyst Insights: Target prices, recommendations, analyst counts
• Profile: Company information, executives, business summary
• News: Recent news articles and press releases

Quick Scrape Examples

// Scrape key statistics (not available through free API)
keyStats, err := client.ScrapeKeyStatistics(ctx, "AAPL", runID)

// Scrape financial statements
financials, err := client.ScrapeFinancials(ctx, "AAPL", runID)

// Scrape comprehensive analysis data (earnings trends, EPS revisions, revenue estimates)
analysis, err := client.ScrapeAnalysis(ctx, "AAPL", runID)

// Scrape analyst insights (target prices, recommendations)
analystInsights, err := client.ScrapeAnalystInsights(ctx, "AAPL", runID)

// Scrape news articles
news, err := client.ScrapeNews(ctx, "AAPL", runID)

CLI Scraping

# Scrape key statistics with preview
yfin scrape --ticker AAPL --endpoint key-statistics --preview

# Multiple endpoints with JSON output
yfin scrape --ticker AAPL --endpoints key-statistics,financials,news --preview-json

# Soak testing is a separate binary, not a yfin subcommand
go run ./cmd/soak --universe-file universe.txt --duration 2h --qps 5

📖 Complete Scrape Documentation →

---

📝 Usage Examples

Example 1: Fetch Daily Bars for Multiple Symbols

package main

import (
    "context"
    "fmt"
    "log"
    "sync"
    "time"

    "github.com/bizshuk/yfin/facade"
)

func main() {
    client := facade.NewClient()
    ctx := context.Background()

    symbols := []string{"AAPL", "GOOGL", "MSFT", "TSLA"}
    start := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)
    end := time.Date(2024, 1, 31, 0, 0, 0, 0, time.UTC)

    var wg sync.WaitGroup
    results := make(chan string, len(symbols))

    for _, symbol := range symbols {
        wg.Add(1)
        go func(sym string) {
            defer wg.Done()

            batch, err := client.FetchDailyBars(ctx, sym, start, end, true, "batch-run")
            if err != nil {
                results <- fmt.Sprintf("Error fetching %s: %v", sym, err)
                return
            }

            results <- fmt.Sprintf("%s: %d bars fetched", sym, len(batch.Bars))
        }(symbol)
    }

    wg.Wait()
    close(results)

    for result := range results {
        fmt.Println(result)
    }
}

Example 2: Get Current Market Quote

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/bizshuk/yfin/facade"
)

func main() {
    client := facade.NewClient()
    ctx := context.Background()

    quote, err := client.FetchQuote(ctx, "AAPL", "quote-run")
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("Symbol: %s\n", quote.Symbol)
    fmt.Printf("Price: %.4f %s\n", quote.Price, quote.Currency)
    fmt.Printf("Event Time: %s\n", quote.EventTime.UTC().Format("2006-01-02 15:04:05"))
}

Example 3: Fetch Company Information

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/bizshuk/yfin/facade"
)

func main() {
    client := facade.NewClient()
    ctx := context.Background()

    info, err := client.FetchCompanyInfo(ctx, "AAPL", "company-run")
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("Company: %s\n", info.LongName)
    fmt.Printf("Exchange: %s\n", info.Exchange)
    fmt.Printf("Full Exchange: %s\n", info.FullExchangeName)
    fmt.Printf("Currency: %s\n", info.Currency)
    fmt.Printf("Instrument Type: %s\n", info.InstrumentType)
    fmt.Printf("Timezone: %s\n", info.Timezone)
}

Example 4: Error Handling

package main

import (
    "context"
    "fmt"
    "log"
    "time"

    "github.com/bizshuk/yfin/facade"
)

func main() {
    client := facade.NewClient()
    ctx := context.Background()

    // Fetch data with proper error handling
    batch, err := client.FetchDailyBars(ctx, "AAPL",
        time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC),
        time.Date(2024, 1, 31, 0, 0, 0, 0, time.UTC),
        true, "error-handling-run")

    if err != nil {
        log.Printf("Error fetching bars: %v", err)
        return
    }

    fmt.Printf("Successfully fetched %d bars\n", len(batch.Bars))

    // Handle empty results
    if len(batch.Bars) == 0 {
        fmt.Println("No data available for the specified date range")
        return
    }

    // Process the data — Bar.Close is a plain float64
    for _, bar := range batch.Bars {
        fmt.Printf("Date: %s, Close: %.4f %s\n",
            bar.Date, bar.Close, bar.CurrencyCode)
    }
}

FetchDailyBars 可成功回傳 metadata 完整但 Bars 為空的 batch，表示 Yahoo 已處理該 symbol/window、期間內沒有 observation；呼叫端不得將它改寫成虛構 價格。Yahoo 偶爾也會在有效 chart response 省略 currency，此時 bar 的 CurrencyCode 保持空字串，應由擁有官方 market context 的下游補值。

🐍→🦫 Batch Mode (Python yf parity)

yfin batch 以 Go 實作 skills/scripts/all_ticker_yf.py 的 30-command 批次、 分級快取與 artifact lifecycle；Python 版本只保留為明確執行的 live oracle， 不屬於 Stock runtime：

• commandOrder 取自 facade.FoundationCommands()；CLI registry 與 SDK consumer 都呼叫 FetchFoundation。
• 預設 universe 由 binary embedded cmd/dispatch/ticker_list.csv 提供，不依賴 current working directory。
• CLI artifacts 寫入 ~/.config/yfin/data/raw/<command>/<ticker>.<YYYY-MM-DD>.json。Downstream 可把自己的 app raw root 傳給 NewCachedFoundationClient；artifact 仍由 yfin 寫入。
• JSON 與 error artifacts 皆以同目錄 temporary file + atomic rename 發布；cache 只以最新有效 artifact 判 freshness。
• HTTP 404/422 記為 not_found；任何 failed command 會保留已成功 artifacts，並令 batch exit non-zero。

# 預設：抓 embedded universe 中所有 ticker × 30 指令
go run . batch

# 單股 / 強制重抓 / 調整並行
go run . batch --ticker 2330.TW
go run . batch --ticker 2330.TW --force
go run . batch --max-workers 5

# Live semantic gate：Python oracle → Go batch → 30-command comparator
./scripts/verify-yf-parity.sh AAPL
./scripts/verify-yf-parity.sh 2330.TW

對應 Python 端：見 skills/SKILL.md 的 ## Go client parity gate。

30-command implementation matrix

下表只表示 Go command 已接線到對應資料來源，不等同 live parity 已通過。release 前必須以 scripts/verify-yf-parity.sh 對 AAPL 與 2330.TW 驗證 artifact 存在、JSON validity、empty semantics 與 top-level type；Yahoo rate limit 或頁面變更會讓 gate 以 non-zero 明確失敗。

指令	來源 (Python)	Go 端實作	接線
info	5 quoteSummary 模組	(*yahoo.Client).FetchInfo	✅
history	chart 30d/daily	(*facade.Client).FetchDailyBars 30d	✅
actions	chart events	(*yahoo.Client).FetchActions	✅
income / balance / cashflow	fundamentals-timeseries annual API	FetchIncomeStatement/BalanceSheet/CashFlowStatement	✅
major-holders / institutional-holders / mutualfund-holders	quoteSummary 7 模組(單次 HTTP)	FetchHolders(同 4 模組)	✅
insider-transactions / insider-roster	quoteSummary	FetchInsider	✅
insider-purchases	netSharePurchaseActivity 整形為 label/value table	InsiderPurchaseSummaryTable	✅
recommendations / recommendations-summary	同源(quoteSummary)	FetchRecommendationTrend	✅
upgrades	quoteSummary	FetchUpgrades	✅
earnings-dates	HTML scrape /calendar/earnings?symbol=	(*yahoo.Client).FetchEarningsDates	✅
earnings-history / eps-trend / eps-revisions / earnings-estimates / revenue-estimates / growth-estimates	quoteSummary	ScrapeAnalysisDimension	✅
price-targets	quoteSummary	ScrapeAnalystInsights	✅
news	POST /xhr/ncp tickerStream	FetchNews	✅
calendar	quoteSummary calendarEvents	FetchCalendar	✅
sec-filings	quoteSummary	FetchSecFilings	✅
sustainability	quoteSummary esgScores	FetchESG	✅
isin	business-insider	FetchISIN	✅
options	/v7/finance/options/	FetchOptions	✅
metadata	1d chart(不重抓)	FetchMetadata(1d range)	✅

---

📚 Documentation

Note: For the latest release notes, see Release Notes. For the complete changelog, see CHANGELOG.md.

Core Documentation

• Installation Guide - Setup and installation instructions
• Usage Guide - Comprehensive usage examples and patterns
• API Reference - Complete API documentation with method capabilities and limitations
• Data Structures - Detailed data structure guide with field naming conventions
• Complete Examples - Working code examples with data processing and error handling

Method Comparison & Migration

• Method Comparison - Method comparison table and use case guidance
• Migration Guide - Migration from Python yfinance with feature comparison

Error Handling & Quality

• Error Handling Guide - Comprehensive error handling and troubleshooting
• Data Quality Guide - Data quality expectations and validation guidelines
• Performance Guide - Performance optimization and best practices

Scrape Fallback System

• Scrape Overview - Architecture and data flow
• Configuration Guide - All configuration options and best practices
• CLI Usage - Command-line interface examples
• Troubleshooting - Common issues and solutions

Operations & Monitoring

• Observability Guide - Metrics, logging, and monitoring
• Soak Testing Guide - Load testing and validation
• Production Readiness Report - Production readiness assessment and checklist

Development & Testing

• Testing Implementation - Testing strategy and implementation details
• Release Guide - Release process and procedures
• Release Notes - Version release notes and changelog

Audit & Quality Assurance

• Final Audit Summary - Production readiness audit & applied fixes (2025-01-30)

Operator Runbooks

• Scrape Fallback Runbook - Operational procedures
• Incident Response Playbook - Emergency response procedures

Examples & Code Samples

• Library Examples - Go code examples and patterns
• CLI Examples - Ready-to-run shell scripts
• API Usage Example - Basic API usage
• Historical Data Example - Time series data

---

🖥️ CLI Usage

The yfin CLI tool provides command-line access to all functionality:

Note: All CLI commands require a configuration file. Use --config config/effective.yaml or set up your own config file.

Installation

# Build from source (package main is at the repo root)
go build -o yfin .

# Or install globally
go install github.com/bizshuk/yfin@latest

Basic Commands

# Fetch daily bars for a single symbol
yfin pull --ticker AAPL --start 2024-01-01 --end 2024-12-31 --adjusted split_dividend --preview --config config/effective.yaml

# Fetch data for multiple symbols from a file
yfin pull --universe-file symbols.txt --start 2024-01-01 --end 2024-12-31 --out json --out-dir ./out --config config/effective.yaml

# Get current quote
yfin quote --tickers AAPL --config config/effective.yaml

# Get fundamentals (requires paid subscription)
yfin fundamentals --ticker AAPL --preview --config config/effective.yaml

Scraping Commands

# Scrape key statistics (not available through free API)
yfin scrape --ticker AAPL --endpoint key-statistics --preview --config config/effective.yaml

# Multiple endpoints with JSON preview
yfin scrape --ticker AAPL --endpoints key-statistics,financials,analysis --preview-json --config config/effective.yaml

# News articles preview
yfin scrape --ticker AAPL --endpoint news --preview-news --config config/effective.yaml

# Health check for endpoints
yfin scrape --ticker AAPL --endpoint key-statistics --check --config config/effective.yaml

Soak Testing Commands

soak is a standalone binary, not a yfin subcommand — run it with go run ./cmd/soak.

# Quick smoke test (10 minutes)
go run ./cmd/soak --universe-file tests/testdata/universe/soak.txt --endpoints key-statistics,news --duration 10m --concurrency 8 --qps 5 --config config/effective.yaml

# Full production soak test (2 hours)
go run ./cmd/soak --universe-file tests/testdata/universe/soak.txt --endpoints key-statistics,financials,analysis,profile,news --duration 2h --concurrency 12 --qps 5 --config config/effective.yaml

CLI Options

Core Options

• --ticker - Single symbol to fetch
• --universe-file - File containing list of symbols
• --start, --end - Date range (UTC)
• --adjusted - Adjustment policy (raw | split_only | split_dividend); defaults to markets.default_adjustment_policy from the YAML config (CLI flag overrides the YAML default)
• --out, --out-dir - Local export format (json) and output directory
• --concurrency - Number of concurrent requests
• --qps - Requests per second limit
• --retry-max, --timeout - HTTP retry attempts and timeout tuning

Scraping Options (yfin scrape)

• --endpoint - Single endpoint to scrape (profile, key-statistics, financials, balance-sheet, cash-flow, analysis, analyst-insights, news)
• --endpoints - Comma-separated list of endpoints, for --preview-json
• --check - Check connectivity only (no parsing)
• --preview - Show preview without parsing
• --preview-json - JSON preview across endpoints
• --preview-news - Preview news articles
• --force - Scrape even when the API is available

Soak Testing Options (go run ./cmd/soak)

• --duration - Test duration (e.g., 2h, 30m)
• --endpoints - Endpoints to exercise
• --fallback - Fallback strategy
• --memory-check - Enable memory leak detection
• --probe-interval - Correctness probe interval
• --failure-rate - Simulated failure rate for testing

📖 Complete CLI Documentation →

---

🎯 Mission & Success Criteria

Mission
Provide a reliable, consistent, and fast Yahoo Finance client in Go with one canonical shape per concept (model.*), so ingestion pipelines and research tools see identical data whether it came from the API or from a scraped page.

Success looks like

• Library returns validated model.* structs with correct UTC times, currency semantics, and adjustment flags.
• CLI supports on-demand pulls and batch backfills; ops can dry‑run and preview with a single command, then export locally as JSON.
• Concurrency and backoff keep error rates and 429/503 responses within policy; throughput is tunable and predictable.
• Observability shows latency/throughput, decode failures, and backoff behavior; alerts catch regressions.

---

📊 Data Coverage

✅ Supported Data Types

• Historical Bars - Daily, weekly, monthly, and intraday OHLCV data
• Real-time Quotes - Current market prices, bid/ask, volume
• Company Information - Basic company details, exchange info, industry/sector
• Market Data - 52-week ranges, market state, trading hours
• Multi-Currency Support - Automatic currency conversion with FX providers

⚠️ Mixed API and Explicit Scrape Coverage

Annual financial statements and news have first-class JSON endpoint methods. Other detailed surfaces remain available through explicit client.Scrape* methods:

• Financial Statements - FetchIncomeStatement, FetchBalanceSheet, FetchCashFlowStatement
• Analyst Recommendations - Price targets, ratings
• Key Statistics - P/E ratios, market cap, financial metrics
• Company Profiles - Business summary, executives, sector info
• News Articles - FetchNews

⚠️ Generic foundation surfaces

These surfaces do not each have a first-class Fetch* method. Use yfin batch --ticker X, client.FetchFoundation(...), or the cache-backed CachedFoundationClient.Fetch(...):

• Options Data - Options chains and pricing
• Insider Trading - Transactions, roster, purchase summary
• Institutional Holdings - Major / institutional / mutual-fund holders
• Corporate Events - Calendar, SEC filings, sustainability (ESG), upgrades, ISIN

❌ Not Supported

• Level 2 Market Data - Order book, bid/ask depth

🌍 Supported Markets

• US Markets - NYSE, NASDAQ, AMEX
• International - Major exchanges worldwide
• Currencies - Forex pairs and cryptocurrency
• Commodities - Gold, oil, agricultural products
• Indices - S&P 500, Dow Jones, NASDAQ Composite

---

⚡ Key Features

🛡️ Production Ready

• Rate Limiting - Built-in QPS limits and burst control
• Circuit Breakers - Rolling failure detection with explicit Yahoo endpoint-family isolation
• Retry Logic - Exponential backoff with jitter
• Scrape Surface - Explicit robots.txt-compliant scrape methods for remaining API gaps
• Observability - Comprehensive metrics, logs, and tracing
• Soak Testing - Built-in load testing and robustness validation

💰 Financial Accuracy

• High Precision Decimals - Scaled decimal arithmetic for exact calculations
• Currency Support - Multi-currency with automatic conversion
• Corporate Actions - Split and dividend adjustments
• Market Hours - Proper handling of trading sessions and holidays

🚀 Performance

• Concurrent Requests - Configurable goroutine pools
• Connection Pooling - Efficient HTTP connection reuse
• Caching - Built-in response caching for FX rates
• Batching - Efficient data batching and chunking

🔧 Developer Experience

• Simple API - Clean, intuitive Go interface
• Type Safety - Strongly typed data structures
• Error Handling - Comprehensive error types and messages
• CLI Tool - Command-line interface for operations
• Documentation - Extensive examples and API docs

---

📋 Data Formats & Conventions

1. Time: All timestamps UTC ISO‑8601. Bars use start inclusive, end exclusive; event_time at bar close.
2. Precision: Prices/amounts are scaled decimals (scaled, scale). Volumes are integers.
3. Currency: Attach ISO‑4217 code to monetary fields and fundamentals lines.
4. Identity: Use SecurityId = { symbol, mic?, figi?, isin? }. If MIC is unknown, prefer primary listing inference; document fallback rules.
5. Adjustments: Bars declare adjusted: true|false and adjustment_policy_id: "raw" | "split_only" | "split_dividend".
6. Lineage: Every message has meta.run_id, meta.source="yfin", meta.producer="<host|pod>", schema_version.
7. Batching: Prefer BarBatch for efficiency. Maintain in‑batch order by event_time ascending.
8. Compatibility: Additive evolution only; breaking changes require new major (bars.v2, fundamentals.v2).

💰 Price Formatting

There are two price representations, and which one you get depends on which method you called.

The plain Fetch* / Scrape* methods return model.Bar / model.Quote etc., whose price fields are already float64. No conversion needed — this is what most callers want:

batch, _ := client.FetchDailyBars(ctx, "AAPL", start, end, true, runID)
fmt.Printf("Price: %.4f %s\n", batch.Bars[0].Close, batch.Bars[0].CurrencyCode)

The Fetch*Norm methods return model.Normalized*, which keep prices as ScaledDecimal (an integer plus an explicit scale) so no precision is lost. Convert with model.FromScaledDecimal — never hardcode the divisor:

// ✅ CORRECT — read the scale off the value itself
price := model.FromScaledDecimal(bar.Close)

// ❌ WRONG — the scale is not always 4
// price := float64(bar.Close.Scaled) / 10000

Example: Yahoo reports $221.74 → stored as {Scaled: 22174, Scale: 2} → 22174 / 10^2 = 221.74.

---

⚙️ Configuration

Environment Variables

# Rate limiting
export YFIN_QPS=2.0
export YFIN_BURST=5
export YFIN_CONCURRENCY=32

# Timeouts
export YFIN_TIMEOUT=30s
export YFIN_BACKOFF_BASE=1s
export YFIN_BACKOFF_MAX=10s

# Circuit breaker
export YFIN_CIRCUIT_THRESHOLD=5
export YFIN_CIRCUIT_RESET=30s

# Observability
export YFIN_LOG_LEVEL=info
export YFIN_METRICS_ENABLED=true

Configuration File

# config.yaml
yahoo:
    timeout_ms: 30000
    base_url: "https://query1.finance.yahoo.com"

concurrency:
    global_workers: 32
    max_inflight: 64

rate_limit:
    per_host_qps: 2.0
    burst: 5

retry:
    attempts: 3
    backoff_base_ms: 1000
    backoff_max_ms: 10000

circuit_breaker:
    window: 50
    failure_threshold: 0.30
    minimum_requests: 10
    reset_timeout_ms: 30000

observability:
    log_level: "info"
    metrics_enabled: true
    tracing_enabled: false

---

🚀 Quick Start Examples

Run CLI Examples

# Make scripts executable
chmod +x cmd/samples/*.sh

# Run AAPL preview examples
./cmd/samples/preview_aapl.sh

# Run soak testing examples
./cmd/samples/soak_smoke.sh

# Run batch processing examples
./cmd/samples/batch_processing.sh

Build Library Examples

# Build and run library examples
go build -o /tmp/scrape_fallback facade/samples/scrape_fallback/scrape_fallback.go
/tmp/scrape_fallback

📖 Library Samples → · CLI Samples →

---

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Setup

# Clone the repository
git clone https://github.com/bizshuk/yfin.git
cd yfin

# Install dependencies
go mod download

# Run tests
go test ./...

# Build CLI
go build -o yfin .

# Run integration tests
go test -tags=integration ./...

Code Style

• Follow Go standard formatting (gofmt)
• Use meaningful variable and function names
• Add tests for new functionality
• Update documentation for API changes

---

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

---

🙏 Acknowledgments

• Yahoo Finance for providing publicly accessible financial data
• Go Community for excellent libraries and tools
• Contributors who help improve this project

---

📞 Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Documentation: Complete Documentation
• API Reference: GoDoc

Getting Help

1. Check Documentation: Start with docs/ for comprehensive guides
2. Review Examples: See facade/samples/ and cmd/samples/ for code samples
3. Search Issues: Check existing GitHub Issues
4. Troubleshooting: See docs/scrape/troubleshooting.md
5. Runbooks: For operational issues, see dashboards/runbooks/

---

⭐ If you find this project useful, please give it a star on GitHub!

Documentation

Overview

main.go — yfin CLI composition root; wires every sub-package's `Register` onto `cmd.RootCmd` then forwards `cmd.Execute()` to the process. Exit non-zero on error. Capacity: 1 entrypoint + 1 os.Exit fallback + 6 Register calls.