rtve

package module

v0.2.2 Latest Latest Go to latest Published: Oct 4, 2025 License: MIT Imports: 10 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/rubiojr/rtve-go

Links

Open Source Insights

README ¶

RTVE Scraper

A Go library and command-line tool for downloading subtitles and video metadata from RTVE (Radio Televisión Española).

Features

Go API: Programmatic access to RTVE video metadata and subtitles
CLI Tool: Command-line interface for downloading content
Scrape videos from RTVE show pages
Download video metadata in JSON format
Download subtitles in VTT format (multiple languages)
Organize videos by publication date
Support for pagination and date range filtering
Fetch latest videos from one or all shows

Installation

Prerequisites

Go 1.23 or higher

Building from source

go install github.com/rubiojr/rtve-go/cmd/rtve-subs@latest

Or clone and build:

git clone https://github.com/rubiojr/rtve-go
cd rtve-go
go build -o rtve-subs cmd/rtve-subs/main.go

Usage

Command Line Interface

Fetch videos from a specific show

rtve-subs fetch --show telediario-1

# Specify output directory
rtve-subs fetch --output="/path/to/videos" --show telediario-1

# Fetch multiple pages
rtve-subs fetch --show telediario-1 --max-pages 5

# Enable verbose output
rtve-subs fetch --show telediario-1 --verbose

Fetch latest videos

# Fetch the latest video from a specific show
rtve-subs fetch-latest --show telediario-1 --count 1

# Fetch latest 5 videos from a show
rtve-subs fetch-latest --show telediario-1 --count 5

# Fetch latest video from ALL available shows
rtve-subs fetch-latest --count 1

# Fetch latest 3 videos from all shows
rtve-subs fetch-latest --count 3

List available shows

rtve-subs list-shows

Go API

The package also provides a programmatic API for Go applications:

package main

import (
    "fmt"
    "log"
    "time"
    
    "github.com/rubiojr/rtve-go/api"
)

func main() {
    // Define date range
    start := time.Date(2025, 10, 1, 0, 0, 0, 0, time.UTC)
    end := time.Date(2025, 10, 5, 23, 59, 59, 0, time.UTC)
    
    // Create visitor function to process each video
    visitor := func(result *api.VideoResult) error {
        fmt.Printf("Title: %s\n", result.Metadata.LongTitle)
        fmt.Printf("Published: %s\n", result.Metadata.PublicationDate)
        
        if result.Subtitles != nil {
            fmt.Printf("Subtitles: %d tracks\n", len(result.Subtitles.Subtitles))
        }
        return nil
    }
    
    // Fetch videos
    stats, err := api.FetchShow("telediario-1", start, end, visitor)
    if err != nil {
        log.Fatal(err)
    }
    
    fmt.Printf("Processed %d videos\n", stats.VideosProcessed)
}

See the API documentation for more details.

Supported Shows

Currently supported shows include:

telediario-1 - Telediario 15 horas
telediario-2 - Telediario 21 horas
telediario-matinal - Telediario Matinal
informe-semanal - Informe Semanal

Use the list-shows command to see the complete list of available shows.

Command-line Options

`fetch` command

Option	Alias	Default	Description
`--output`	`-o`	`rtve-videos`	Output directory for downloaded content
`--show`	`-s`	(required)	Show to scrape
`--max-pages`	`-m`	`1`	Maximum number of pages to scrape
`--verbose`	`-v`	`false`	Enable verbose output

`fetch-latest` command

Option	Alias	Default	Description
`--output`	`-o`	`rtve-videos`	Output directory for downloaded content
`--show`	`-s`	(optional)	Show to fetch (if not specified, fetches from all shows)
`--count`	`-n`	`1`	Number of latest videos to fetch per show
`--verbose`	`-v`	`false`	Enable verbose output

Output Structure

The scraper organizes videos by year and date:

rtve-videos/
  ├── 2023/
  │   ├── 2023-01-01/
  │   │   ├── video_12345.json
  │   │   └── subs/
  │   │       ├── 12345_es.vtt
  │   │       └── 12345_en.vtt
  │   └── 2023-01-02/
  │       └── ...
  └── 2022/
      └── ...

How It Works

Scraper (fetch command)

Navigates through pages of the specified RTVE show URL
Extracts video links using regex patterns
For each video:
- Downloads metadata from RTVE's JSON API
- Downloads available subtitles (Spanish, English, Catalan, Basque, Galician)
- Organizes content by publication date
- Sets folder timestamps to match publication date

API (fetch-latest, FetchShow)

Uses the high-level API package for cleaner integration
Supports date range filtering
Processes videos through a visitor function pattern
Provides detailed statistics about the fetch operation

Development

Running Tests

# Run unit tests (fast)
go test ./...

# Run integration tests (requires network access)
go test ./api -v -run TestIntegration

CI/CD

The project uses GitHub Actions for continuous integration:

On Push/PR: Runs unit tests and builds the binary
Weekly (Monday 9 AM UTC): Runs integration tests against RTVE's live API
On Integration Test Failure: Automatically creates a GitHub issue

The weekly integration tests help detect when RTVE changes their API or website structure, allowing for proactive maintenance.

API Documentation

Full API documentation is available at pkg.go.dev/github.com/rubiojr/rtve-go/api.

Key types and functions:

FetchShow(showID, startDate, endDate, visitor) - Fetch videos within a date range
FetchShowLatest(showID, maxVideos, visitor) - Fetch the most recent videos
FetchShowAll(showID, visitor) - Fetch all available videos
AvailableShows() - Get list of supported shows
VideoResult - Contains metadata and subtitles for a video
VisitorFunc - Function type for processing each video

License

This project is licensed under the MIT License - see the LICENSE file for details.

Disclaimer

This tool is provided for personal use only. Be mindful of RTVE's terms of service and copyright restrictions when using this tool. The developers are not responsible for any misuse of this software.

Documentation ¶

Index ¶

Constants
Variables
func GetLanguageName(langCode string) string
func ListShows() []string
type Option
- func WithOutputPath(path string) Option
type Scrapper
- func NewScrapper(program string, options ...Option) *Scrapper
- func (s *Scrapper) DownloadSubtitles(meta *VideoMetadata, outputDir string) error
- func (s *Scrapper) DownloadVideoMeta(videoID string) (*VideoMetadata, error)
- func (s *Scrapper) FetchSubtitles(meta *VideoMetadata) (*Subtitles, error)
- func (s *Scrapper) SaveVideoToFile(meta *VideoMetadata, directory string) error
- func (s *Scrapper) Scrape(maxPages int) (int, []error)
- func (s *Scrapper) ScrapePage(page int) ([]*VideoInfo, error)
type Show
- func ShowMap(name string) *Show
type SubtitleItem
type SubtitlePage
type SubtitleResponse
type Subtitles
type VideoInfo
type VideoMetadata
- func (m *VideoMetadata) Parse(body string) error
type VideoPage
type VideoResponse

Constants ¶

View Source

const ApiURL = "https://api2.rtve.es/api/videos/%s.json"

View Source

const SubsURL = "https://api2.rtve.es/api/videos/%s/subtitulos.json"

Variables ¶

View Source

var ErrForbidden = errors.New("access not allowed")

View Source

var ErrPageNotFound = errors.New("page not found")

Functions ¶

func GetLanguageName ¶

func GetLanguageName(langCode string) string

Helper function to get language name from language code

func ListShows ¶

func ListShows() []string

Types ¶

type Option ¶

type Option func(*Scrapper)

func WithOutputPath ¶

func WithOutputPath(path string) Option

type Scrapper ¶

type Scrapper struct {
	Program string
	// contains filtered or unexported fields
}

func NewScrapper ¶

func NewScrapper(program string, options ...Option) *Scrapper

func (*Scrapper) DownloadSubtitles ¶

func (s *Scrapper) DownloadSubtitles(meta *VideoMetadata, outputDir string) error

DownloadSubtitles downloads all available subtitles for a given video ID and saves them to the specified directory

func (*Scrapper) DownloadVideoMeta ¶

func (s *Scrapper) DownloadVideoMeta(videoID string) (*VideoMetadata, error)

DownloadVideoMeta fetches and parses video metadata for a given video ID

func (*Scrapper) FetchSubtitles ¶

func (s *Scrapper) FetchSubtitles(meta *VideoMetadata) (*Subtitles, error)

FetchSubtitles fetches subtitle metadata for a video and returns a Subtitles object

func (*Scrapper) SaveVideoToFile ¶

func (s *Scrapper) SaveVideoToFile(meta *VideoMetadata, directory string) error

func (*Scrapper) Scrape ¶

func (s *Scrapper) Scrape(maxPages int) (int, []error)

func (*Scrapper) ScrapePage ¶

func (s *Scrapper) ScrapePage(page int) ([]*VideoInfo, error)

type Show ¶

type Show struct {
	ID    string
	URL   string
	Regex string
}

func ShowMap ¶

func ShowMap(name string) *Show

type SubtitleItem ¶

type SubtitleItem struct {
	Src  string `json:"src"`  // URL of the subtitle file
	Lang string `json:"lang"` // Language code for the subtitle
}

type SubtitlePage ¶

type SubtitlePage struct {
	Items       []SubtitleItem `json:"items"`       // List of subtitle items
	Number      int            `json:"number"`      // Current page number
	Size        int            `json:"size"`        // Page size
	Offset      int            `json:"offset"`      // Offset in the results
	Total       int            `json:"total"`       // Total number of items
	TotalPages  int            `json:"totalPages"`  // Total number of pages
	NumElements int            `json:"numElements"` // Number of elements in the current page
}

SubtitlePage represents the page of subtitle items along with pagination info

type SubtitleResponse ¶

type SubtitleResponse struct {
	Page SubtitlePage `json:"page"` // Subtitle page information
}

SubtitleResponse represents the top-level JSON response

type Subtitles ¶ added in v0.2.0

type Subtitles struct {
	// VideoID is the ID of the video these subtitles belong to
	VideoID string
	// Subtitles is a list of available subtitle tracks
	Subtitles []SubtitleItem
}

Subtitles represents parsed subtitle data for a video

type VideoInfo ¶

type VideoInfo struct {
	URL string
	ID  string
}

type VideoMetadata ¶

type VideoMetadata struct {
	URI             string `json:"uri"`
	HTMLUrl         string `json:"htmlUrl"`
	ID              string `json:"id"`
	LongTitle       string `json:"longTitle"`
	PublicationDate string `json:"publicationDate"`
}

VideoMetadata represents essential metadata from a video

func (*VideoMetadata) Parse ¶

func (m *VideoMetadata) Parse(body string) error

type VideoPage ¶

type VideoPage struct {
	Items       []VideoMetadata `json:"items"`
	Number      int             `json:"number"`
	Size        int             `json:"size"`
	Offset      int             `json:"offset"`
	Total       int             `json:"total"`
	TotalPages  int             `json:"totalPages"`
	NumElements int             `json:"numElements"`
}

VideoPage represents the page of video items

type VideoResponse ¶

type VideoResponse struct {
	Page VideoPage `json:"page"`
}

VideoResponse represents the top-level JSON response

Source Files ¶

View all Source files

Directories ¶

Path	Synopsis
api Package api provides a high-level interface for fetching video metadata and subtitles from RTVE (Radio Televisión Española) on-demand streaming service.	Package api provides a high-level interface for fetching video metadata and subtitles from RTVE (Radio Televisión Española) on-demand streaming service.
cmd
rtve-subs command

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

README ¶

RTVE Scraper

Features

Installation

Prerequisites

Building from source

Usage

Command Line Interface

Fetch videos from a specific show

Fetch latest videos

List available shows

Go API

Supported Shows

Command-line Options

fetch command

fetch-latest command

Output Structure

How It Works

Scraper (fetch command)

API (fetch-latest, FetchShow)

Development

Running Tests

CI/CD

API Documentation

License

Disclaimer

Documentation ¶

Index ¶

Constants ¶

Variables ¶

Functions ¶

func GetLanguageName ¶

func ListShows ¶

Types ¶

type Option ¶

func WithOutputPath ¶

type Scrapper ¶

func NewScrapper ¶

func (*Scrapper) DownloadSubtitles ¶

func (*Scrapper) DownloadVideoMeta ¶

func (*Scrapper) FetchSubtitles ¶

func (*Scrapper) SaveVideoToFile ¶

func (*Scrapper) Scrape ¶

func (*Scrapper) ScrapePage ¶

type Show ¶

func ShowMap ¶

type SubtitleItem ¶

type SubtitlePage ¶

type SubtitleResponse ¶

type Subtitles ¶ added in v0.2.0

type VideoInfo ¶

type VideoMetadata ¶

func (*VideoMetadata) Parse ¶

type VideoPage ¶

type VideoResponse ¶

Source Files ¶

Directories ¶

`fetch` command

`fetch-latest` command