nesdc

package
v0.4.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 8, 2026 License: MIT Imports: 17 Imported by: 0

Documentation

Overview

Package nesdc implements an unofficial scraping client for the NESDC (중앙선거여론조사심의위원회 / National Election Survey Deliberation Commission) public portal at https://www.nesdc.go.kr.

All data exposed here is legally mandated public disclosure under the 선거여론조사기준. The client is deliberately polite: requests are rate-limited and identify themselves via a descriptive User-Agent.

Index

Constants

View Source 
const DefaultBaseURL = "https://www.nesdc.go.kr/portal"

DefaultBaseURL is the portal root used for every request.

View Source 
const DefaultDelay = 700 * time.Millisecond

DefaultDelay is the minimum spacing between requests.

View Source 
const DefaultUserAgent = "nesdc-cli (+https://github.com/JungHoonGhae/nesdc-cli)"

DefaultUserAgent identifies the client honestly to the server operator.

Variables

View Source 
var DateField = map[string]string{
	"registered": "1",
	"published":  "2",
	"surveyed":   "3",
}

DateField maps a friendly CLI name to the results board's searchTime code, naming which date the --from/--to range filters on.

View Source 
var SearchField = map[string]string{
	"regno":  "5",
	"agency": "1",
	"client": "2",
	"method": "6",
	"frame":  "11",
	"name":   "3",
	"sido":   "4",
}

SearchField maps a friendly CLI name to the results board's searchCnd code. These select which column a keyword search matches against.

Functions

func BoardNames 

func BoardNames() []string

BoardNames returns the sorted list of registered board names.

func ResolveDateField 

func ResolveDateField(v string) string

ResolveDateField maps a --date-field value (friendly name or raw code) to the searchTime code.

func ResolveSearchField 

func ResolveSearchField(v string) string

ResolveSearchField maps a --field value (friendly name or raw code) to the searchCnd code.

func SortedKeys 

func SortedKeys(m map[string]string) []string

SortedKeys returns a filter map's friendly names in a stable order, for help text and the `nesdc fields` command.

Types

type Attachment 

type Attachment struct {
	Name       string `json:"name"`
	AtchFileID string `json:"atchFileId"`
	FileSn     string `json:"fileSn"`
	BbsID      string `json:"bbsId"`
	BbsKey     string `json:"bbsKey"`
}

Attachment is a downloadable file referenced by a detail page.

func PickTabulationAttachment added in v0.4.0 

func PickTabulationAttachment(atts []Attachment) (Attachment, bool)

PickTabulationAttachment picks the 집계표/통계표 (tabulation) PDF from a survey's attachments — the file that carries the per-question result tables (긍정/부정 등). Preference: name contains 집계 or 통계; else the sole non-questionnaire PDF (설문지/질문지 excluded); else it fails rather than guess.

type Board 

type Board struct {
	// Name is the short CLI identifier (e.g. "results").
	Name string
	// Title is the human-readable Korean label.
	Title string
	// BbsID is the eGovFrame board id (e.g. "B0000005").
	BbsID string
	// MenuNo is the portal menu number used as a query parameter.
	MenuNo string
	// Rich is true for boards whose detail page carries the full survey
	// metadata table (currently only the results board).
	Rich bool
}

Board describes an eGovFrame standard board on the NESDC portal. Every board shares the same list.do / view.do / FileDown.do plumbing; only the menu id, board id, and detail layout differ.

func BoardByName 

func BoardByName(name string) (Board, error)

BoardByName returns the board registered under name.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is a rate-limited HTTP client for the NESDC portal. It is safe for concurrent use; requests are serialized through the rate limiter.

func New 

func New(opts ...Option) *Client

New creates a Client with sane defaults.

func (*Client) Detail 

func (c *Client) Detail(ctx context.Context, b Board, nttID string) (*Detail, error)

Detail fetches and parses a view.do page for the given board and post id.

func (*Client) Download 

func (c *Client) Download(ctx context.Context, a Attachment, destDir string) (string, error)

Download fetches an attachment into destDir and returns the written path. The server-supplied filename (from Content-Disposition) is used when available, otherwise the attachment's display name is used.

Some attachments are embargoed until the survey's scheduled publication time; for those the server returns an HTML notice instead of a file, which surfaces here as an error.

func (*Client) DownloadURL 

func (c *Client) DownloadURL(a Attachment) string

DownloadURL returns the absolute FileDown.do URL for an attachment. The atchFileId/fileSn/bbsKey values are already percent-encoded as they appear in the page markup, so they are concatenated verbatim (no re-encoding).

func (*Client) LatestBulkXlsx 

func (c *Client) LatestBulkXlsx(ctx context.Context, b Board) (Attachment, error)

LatestBulkXlsx finds the cumulative master workbook attached to the most recent post of the data board and returns its attachment. The data board re-attaches the same growing .xlsx ("전국단위 …") to every weekly post, so the newest post always carries the freshest copy.

func (*Client) List 

func (c *Client) List(ctx context.Context, b Board, opts ListOptions) (*ListResult, error)

List fetches and parses one page of a board listing.

func (*Client) Tabulation added in v0.4.0 

func (c *Client) Tabulation(ctx context.Context, b Board, nttID, destDir string) (path string, att Attachment, d *Detail, err error)

Tabulation locates a survey's 집계표 PDF and downloads it into destDir. It does NOT parse the PDF — number reading is left to the consumer (person or AI agent). Returns the saved file path, the chosen attachment, and the survey Detail (for metadata like 조사기관명/조사일시).

type CompositionCell 

type CompositionCell struct {
	Category  string `json:"category"`
	Completed int    `json:"completed"`
	Weighted  int    `json:"weighted"`
}

CompositionCell is one demographic category's sample counts.

type Crosstab 

type Crosstab struct {
	Dimension string            `json:"dimension"`
	Cells     []CompositionCell `json:"cells"`
}

Crosstab is one demographic dimension's breakdown of the sample.

type Detail 

type Detail struct {
	NttID       string            `json:"nttId"`
	Board       string            `json:"board"`
	Title       string            `json:"title,omitempty"`
	Summary     map[string]string `json:"summary,omitempty"`
	Fields      []Field           `json:"fields"`
	Attachments []Attachment      `json:"attachments"`
}

Detail is a parsed view.do page. Fields is the complete, ordered metadata table (lossless); Summary surfaces a curated set of headline scalars for quick consumption.

type Field 

type Field struct {
	Labels []string `json:"labels"`
	Values []string `json:"values"`
}

Field is one labelled row of a detail page's metadata table. A row may carry several stacked labels (e.g. ["성별", "남"]) and several values, so both are preserved as slices to stay lossless.

type ListItem 

type ListItem struct {
	NttID  string            `json:"nttId"`
	Board  string            `json:"board"`
	Values map[string]string `json:"values"`
}

ListItem is one row of a board listing. NttID is the stable post identifier; Values maps the board's column headers to this row's cell text.

type ListOptions 

type ListOptions struct {
	Page       int    // pageIndex (1-based); 0 → 1
	Keyword    string // searchWrd
	SearchCnd  string // searchCnd (search field selector); default "0"
	SearchTime string // searchTime: date field that From/To apply to (1 등록일 / 2 최초공표일 / 3 조사일시)
	From       string // sdate, format YYYY-MM-DD
	To         string // edate, format YYYY-MM-DD
	PollGubun  string // pollGubuncd (results board only)
}

ListOptions filters a board listing. Zero values mean "no filter".

type ListResult 

type ListResult struct {
	Board   string     `json:"board"`
	Page    int        `json:"page"`
	Columns []string   `json:"columns"`
	Items   []ListItem `json:"items"`
}

ListResult is a parsed page of a board listing.

type Option 

type Option func(*Client)

Option configures a Client.

func WithBaseURL 

func WithBaseURL(u string) Option

WithBaseURL overrides the portal root (useful for tests).

func WithDelay 

func WithDelay(d time.Duration) Option

WithDelay sets the minimum spacing between requests.

func WithHTTPClient 

func WithHTTPClient(h *http.Client) Option

WithHTTPClient injects a custom *http.Client.

func WithUserAgent 

func WithUserAgent(ua string) Option

WithUserAgent overrides the User-Agent header.

type PollRecord 

type PollRecord struct {
	Period       string            `json:"period"` // source sheet (date range)
	RegNo        string            `json:"regNo,omitempty"`
	Agency       string            `json:"agency,omitempty"`
	Client       string            `json:"client,omitempty"`
	SurveyDate   string            `json:"surveyDate,omitempty"`
	Method       string            `json:"method,omitempty"`
	Frame        string            `json:"frame,omitempty"`
	SampleSize   string            `json:"sampleSize,omitempty"`
	ContactRate  string            `json:"contactRate,omitempty"`
	ResponseRate string            `json:"responseRate,omitempty"`
	MarginError  string            `json:"marginError,omitempty"`
	PartySupport map[string]string `json:"partySupport,omitempty"`
}

PollRecord is one normalized row of the cumulative "주요 데이터" workbook: a single registered poll with its metadata and per-party support figures. The fixed metadata columns are mapped to named fields; the variable party columns (which change over time as parties appear and merge) are kept as an ordered map so no figure is lost.

func ParseBulkXlsx 

func ParseBulkXlsx(path string) ([]PollRecord, error)

ParseBulkXlsx reads the cumulative workbook at path into normalized records, one per poll, across every sheet. Each sheet uses a two-row header: fixed metadata labels on the first row, and — under the merged 정당지지율 cell — individual party names on the second row.

type SampleComposition 

type SampleComposition struct {
	Total       *CompositionCell `json:"total,omitempty"`
	Crosstabs   []Crosstab       `json:"crosstabs"`
	Weighting   string           `json:"weighting,omitempty"`
	MarginError string           `json:"marginError,omitempty"`
}

SampleComposition is the structured 표본 구성: who was sampled (completed vs weighted counts) by 성별/연령대별/지역별, plus the weighting method and margin of error. All raw counts are preserved; nothing is interpreted.

func SampleCompositionOf 

func SampleCompositionOf(d *Detail) *SampleComposition

SampleCompositionOf derives the sample-composition crosstab from a detail page's already-parsed Fields. Returns nil when the block is absent.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.