tgup

README

tgup

High-throughput Telegram Saved Messages media uploader CLI.
Built on gotd (MTProto) for batch photo/video uploads with resume, queue coordination, and parallel isolation.

English · 简体中文 · Changelog · Issues

Table of contents

TOC

• ✨ Features
• 📋 Telegram Constraints
• 📦 Installation
• 🚀 Quick Start
• ⌨️ Commands
• 🔀 Run Modes
• ⚙️ Configuration
• 🌐 Environment Variables
• 🔍 Key Behaviors
• 📁 Project Structure
• ❓ FAQ
• 🛠 Development

✨ Features

[!IMPORTANT]

Star Us — you will receive all release notifications from GitHub without any delay ~ ⭐️

tgup solves one problem well: reliably upload local media to Telegram Saved Messages. Core goals:

• 🎯 Predictable — scan, group, sort, upload with clear plans
• 🔄 Resumable — SQLite state DB with checkpoint resume
• 🤝 Coordinated — default FIFO queue prevents multi-terminal conflicts
• ⚡ Parallel — force-parallel mode with isolated state/session when needed

Feature	Default	Flags	Notes
🔐 Login (Code / QR)	Reuse session	login --code / login --qr	2FA supported
🧪 Quick verify	Generate test images → upload → cleanup	demo	One-command E2E verification
📂 Media scan	Recursive, no symlinks	--src --recursive --include-ext	Multiple --src, dedup by real path
📑 Group & slice	Group by parent dir, slice by 10	--order --album-max	Telegram album limit = 10
🚀 Album concurrency	5 albums in parallel	--concurrency-album	Concurrency unit = album
🧩 Chunked upload	8 threads per file	--threads	512 KB chunks + multi-threaded
🔌 DC connection pool	8 MTProto connections	--pool-size	Break single-connection bandwidth limit
💾 Resume	Enabled	--resume / --no-resume	Identify by path + size + mtime_ns
📋 Duplicate policy	ask	--duplicate {skip,ask,upload}	Only effective with resume on
🔒 Queue coordination	Same-state FIFO queue	Default	Cross-process SQLite run_queue
⚡ Force parallel	Off	--force-multi-command	Auto-isolate state/session
🧹 State maintenance	Enabled	--maintenance --cleanup-now	Time / size / row-count triggers
📊 Progress bar	On	--no-progress	Total bytes + album/files status
📝 Upload plan preview	Off	--plan --plan-files	Preview grouping before upload
🌐 MCP HTTP server	Off	mcp serve	Streamable HTTP + SSE

📋 Telegram Constraints

[!NOTE]

These are Telegram platform limitations, not tgup limitations.

• Max 10 media per album (hard limit)
• Only 1 caption per album (assigned to first media)
• MTProto client login only — not Bot API

📦 Installation

• Go >= 1.25
• Single binary, no external runtime dependencies

Download pre-built binaries from GitHub Releases (Linux / macOS / Windows, amd64 / arm64), or build from source:

go build -o tgup ./cmd/tgup

[!TIP]

• Video metadata requires system ffprobe (part of FFmpeg). Without it, Telegram may show duration=0 / 1x1 for videos.
• tgup pre-checks ffprobe availability and warns if missing.
• SQLite uses pure Go (modernc.org/sqlite) — no CGo required.

🚀 Quick Start

1. Get api_id and api_hash from my.telegram.org

2. Login and create a session:

tgup login --code
# or
tgup login --qr

3. (Optional) Quick E2E verification:

tgup demo

4. Preview the upload plan:

tgup dry-run --src /path/to/media --order mtime

5. Start uploading:

tgup run --src /path/to/media --caption "daily"

⌨️ Commands

Top-level

tgup [-h] {login,dry-run,run,demo,mcp,version}

demo

tgup demo [--config CONFIG] [--api-id API_ID] [--api-hash API_HASH] [--session SESSION]

Quick E2E verification: generate 2 test images → upload to Saved Messages → auto cleanup.

login

tgup login [--config CONFIG] [--api-id API_ID] [--api-hash API_HASH] [--session SESSION] (--qr | --code) [--phone PHONE]

Flag	Description
--qr	QR code login in terminal
--code	SMS code login, prompts for 2FA if needed
--session	Session file path (default ./secrets/session.session)

dry-run

tgup dry-run [--src SRC] [--recursive] [--follow-symlinks] [--include-ext CSV] [--exclude-ext CSV] [--order {name,mtime,size,random}] [--reverse] [--album-max N]

Scan + build plan only, no upload. Prints file counts, image/video breakdown, album list.

run

tgup run [--src SRC] [--caption CAPTION] [--concurrency-album N] [--threads N] [--pool-size N] [--duplicate {skip,ask,upload}] [--force-multi-command] [--plan] ...

Key flags:

Flag	Default	Description
--target	me	Upload target
--parse-mode	plain	plain or md
--concurrency-album	5	Album-level concurrency
--threads	8	Per-file parallel chunks (512 KB each)
--pool-size	8	DC connection pool size (0 to disable)
--strict-metadata	off	Reject album on bad video metadata
--image-mode	auto	Image send strategy
--video-thumbnail	auto	Video thumbnail strategy
--state	./data/state.sqlite	SQLite state DB path
--artifacts-dir	./data/runs	Per-run artifacts root
--duplicate	ask	ask / skip / upload for sent files
--plan	off	Print plan before upload

mcp

tgup mcp serve [--host HOST] [--port PORT] [--token TOKEN] [--allow-root PATH ...] [--enable-sse]
tgup mcp schema --out /path/to/schema.json

• mcp serve — local MCP Streamable HTTP server, default 127.0.0.1:8765
• mcp schema — export all MCP tool JSON Schemas
• All /mcp requests require Bearer token
• GET /mcp + Accept: text/event-stream — subscribe to SSE events (supports Last-Event-ID)

🔀 Run Modes

Default Queue Mode (Recommended)

Multiple terminals running tgup run with the same --state:

• Auto-enter unified FIFO queue
• Only the head-of-queue task uploads
• Others wait with waiting ahead=N

Best for: same media library + shared checkpoint state.

Force Parallel Mode

tgup run --src /path/to/media --force-multi-command

• Bypass global queue
• Auto-derive isolated state/session (.force.<pid>.<ts> suffix)
• No shared checkpoint state
• Auto-disable maintenance (skip temp state cleanup)

Best for: independent batch jobs running in parallel.

⚙️ Configuration

Priority: CLI > ENV > config file > defaults

Config File Sources

Source	Path
Global	~/.config/tgup/config.toml
Project	./tgup.toml (overrides global)
Explicit	--config /path/to/config.toml (exclusive)

[!NOTE]

Relative paths in config files resolve relative to the config file's directory, not the current shell directory. Affected fields: telegram.session, paths.state, scan.src, mcp.allow_roots, mcp.control_db.

Full config template: tgup.example.toml

🌐 Environment Variables

Core

Variable	Description
TGUP_API_ID	Telegram API ID
TGUP_API_HASH	Telegram API Hash
TGUP_SESSION / TGUP_SESSION_PATH	Session path (mutually exclusive)
TGUP_STATE / TGUP_STATE_PATH	State DB path (mutually exclusive)
TGUP_ARTIFACTS_DIR	Artifacts root directory
TGUP_THREADS	Per-file upload threads
TGUP_POOL_SIZE	DC connection pool size

Maintenance

Variable	Description
TGUP_MAINTENANCE_ENABLED	Enable maintenance
TGUP_MAINTENANCE_INTERVAL_HOURS	Cleanup interval
TGUP_MAINTENANCE_RETENTION_SENT_DAYS	Sent record retention
TGUP_MAINTENANCE_RETENTION_FAILED_DAYS	Failed record retention
TGUP_MAINTENANCE_RETENTION_QUEUE_DAYS	Queue record retention
TGUP_MAINTENANCE_MAX_DB_MB	Max DB size trigger
TGUP_MAINTENANCE_MAX_UPLOAD_ROWS	Max row count trigger
TGUP_MAINTENANCE_FIRST_RUN_PREVIEW	Preview before first cleanup
TGUP_MAINTENANCE_VACUUM_COOLDOWN_HOURS	VACUUM cooldown
TGUP_MAINTENANCE_VACUUM_MIN_RECLAIM_MB	Min reclaimable for VACUUM

MCP

Variable	Description
TGUP_MCP_ENABLED	Enable MCP server
TGUP_MCP_HOST	Listen host
TGUP_MCP_PORT	Listen port
TGUP_MCP_TOKEN	Bearer token
TGUP_MCP_ALLOW_ROOTS	Allowed root paths
TGUP_MCP_CONTROL_DB	Control DB path
TGUP_MCP_EVENT_RETENTION_HOURS	Event retention
TGUP_MCP_MAX_CONCURRENT_JOBS	Max concurrent jobs
TGUP_MCP_ENABLE_SSE	Enable SSE
TGUP_MCP_ALLOWED_ORIGINS	CORS origins

Boolean values: 1/0, true/false, yes/no, on/off

🔍 Key Behaviors

Scan & Grouping

• Group by src_root + parent_dir — different --src roots never mix
• Default media extensions:
  • Image: .jpg .jpeg .png .webp .heic
  • Video: .mp4 .mov .mkv .webm

Upload & Retry

• Album and single-file uploads supported
• FloodWait — wait per Telegram's cooldown, then retry
• Other errors — exponential backoff with jitter
• ImageProcessFailedError — auto-fallback to document
• Video metadata pre-check: duration > 0 && width > 1 && height > 1
• Post-upload verification: DocumentAttributeVideo with supports_streaming = true
• Per-run artifacts: data/runs/<run_id>/upload.log, report.json, report.md

Resume

• uploads table key: (path, size, mtime_ns)
• status='sent' = completed, skipped on next run
• --duplicate only controls re-upload of already-sent items

Maintenance

Triggers (any condition met):

• Time since last cleanup > interval_hours
• DB size > max_db_mb
• Upload rows > max_upload_rows

Cleanup flow: preview on first run → delete expired rows → evaluate reclaimable space → VACUUM if threshold met.

Use --cleanup-now to skip preview and execute immediately.

📁 Project Structure

cmd/tgup/           Entry point
internal/
  app/               Command orchestration & business logic
  artifacts/          Run artifacts (logs, reports)
  cli/                Argument parsing & command entry
  config/             TOML config loading, merging, validation
  files/              Filesystem abstraction, path safety
  logging/            Structured logging & sanitization
  mcp/                MCP HTTP control plane & SSE
  media/              Video metadata & thumbnail extraction
  plan/               Album grouping & sorting
  progress/           Terminal upload progress rendering
  queue/              Cross-process FIFO queue coordination
  scan/               File discovery & extension filtering
  state/              SQLite state persistence & maintenance
  tg/                 Telegram transport layer (gotd adapter)
  upload/             Upload orchestration, retry & failure handling
  xerrors/             Application-level error classification

❓ FAQ

Q: Error missing api_id / missing api_hash

Credentials not found in the config chain. Check:

1. Did you pass --api-id / --api-hash?
2. Are TGUP_API_ID / TGUP_API_HASH set?
3. Does tgup.toml contain a [telegram] section?

Q: Why is only one terminal uploading?

This is the default FIFO coordination — prevents concurrent state conflicts. Use --force-multi-command for true parallel execution.

Q: Why does the first cleanup only show a preview?

Default first_run_preview=true — shows "what would be deleted" first. Use --cleanup-now or set first_run_preview=false to execute immediately.

Q: Why are some images sent as documents?

Telegram's ImageProcessFailedError triggers auto-fallback to document upload. This is intentional fault tolerance.

Q: Why do videos show as 0s / 1×1 in Telegram?

Video metadata extraction failed. Check:

1. Is ffprobe installed? (ffprobe -version)
2. Any precheck / ffprobe_missing warnings in upload logs?
3. Enable --strict-metadata to block bad-metadata uploads

🛠 Development

# Build
go build ./cmd/tgup

# Test (with race detection)
go test -count=1 -race ./...

# Format check
gofmt -l .
go vet ./...

[!TIP]

See also: AI.md · MCP.md

---

📝 License

This project is licensed under the Apache License 2.0.

Copyright © 2025 babywbx.
This project is Apache-2.0 licensed.