github

package
v0.0.0-...-6c2e7eb Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 6, 2025 License: BSD-3-Clause Imports: 32 Imported by: 0

Documentation

Overview

Package github implements sync mechanism to mirror GitHub issue state into a storage.DB as well as code to inspect that state and to make issue changes on GitHub. All the functionality is provided by the Client, created by New.

Index

Constants

View Source 
const DocWatcherID = "githubdocs"

Variables

This section is empty.

Functions

func CleanBody 

func CleanBody(body string) string

CleanBody should clean the body for indexing. For now we assume the LLM is good enough at Markdown not to bother. In the future we may want to make various changes like inlining the programs associated with playground URLs, and we may also want to remove any HTML tags from the Markdown.

func CleanTitle 

func CleanTitle(title string) string

CleanTitle should clean the title for indexing. For now we assume the LLM is good enough at Markdown not to bother.

func Events 

func Events(db storage.DB, project string, issueMin, issueMax int64) iter.Seq[*Event]

Events returns an iterator over issue events for the given project, limited to issues in the range issueMin ≤ issue ≤ issueMax. If issueMax < 0, there is no upper limit. The events are iterated over in (Project, Issue, API, ID) order, so "/issues" events come first, then "/issues/comments", then "/issues/events". Within a specific API, the events are ordered by increasing ID, which corresponds to increasing event time on GitHub.

func LookupIssues 

func LookupIssues(db storage.DB, project string, issueMin, issueMax int64) iter.Seq[*Issue]

LookupIssues returns an iterator over issues between issueMin and issueMax, only consulting the database (not actual GitHub).

func ParseIssueCommentURL 

func ParseIssueCommentURL(u string) (project string, issue, comment int64, err error)

ParseIssueCommentURL returns the project, issue number and comment number for a URL (for example https://github.com/golang/go/issues/12345#issuecomment-135132324). Note: this is the URL format stored in the [IssueComment.HTMLURL] field.

func ParseIssueURL 

func ParseIssueURL(u string) (project string, number int64, err error)

ParseIssueURL expects a GitHUB API URL for an issue (for example "https://api.github.com/repos/org/r/issues/123") and returns the project (for example "org/r") and issue number.

func ParseMarkdown 

func ParseMarkdown(text string) *markdown.Document

ParseMarkdown parses text that is in GitHub-flavored markdown format.

func Scrub 

func Scrub(req *http.Request) error

Scrub is a scrubber for use with rsc.io/httprr when writing tests that access GitHub through an httprr.RecordReplay. It removes auth credentials from the request.

func Token 

func Token(sdb secret.DB) string

Token returns the secret for "api.github.com".

func ValidWebhookTestdata 

func ValidWebhookTestdata(t *testing.T, event WebhookEventType, payload any) (*http.Request, secret.DB)

ValidWebhookTestdata returns an HTTP request and a secret DB (inputs to ValidateWebhookRequest) that will pass validation. payload is marshaled into JSON as the body of the returned request.

For testing.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

A Client is a connection to GitHub state in a database and on GitHub itself.

func New 

func New(lg *slog.Logger, db storage.DB, sdb secret.DB, hc *http.Client) *Client

New returns a new client that uses the given logger, databases, and HTTP client.

The secret database is expected to have a secret named "api.github.com" of the form "user:pass" where user is a user-name (ignored by GitHub) and pass is an API token ("ghp_...").

func (*Client) Add 

func (c *Client) Add(project string) error

Add adds a GitHub project of the form "owner/repo" (for example "golang/go") to the database. It only adds the project sync metadata. The initial data fetch does not happen until [Sync] or [SyncProject] is called. If the project is already present, Add does nothing and returns nil.

func (*Client) Comments 

func (c *Client) Comments(iss *Issue) iter.Seq[*IssueComment]

Comments returns an iterator over the comments for the issue in the db.

func (*Client) CreateLabel 

func (c *Client) CreateLabel(ctx context.Context, project string, lab Label) error

CreateLabel creates a new label.

func (*Client) DocWatcher 

func (c *Client) DocWatcher() *timed.Watcher[*Event]

DocWatcher returns the event watcher with name "githubdocs". Implements docs.Source.DocWatcher.

func (*Client) DownloadIssue 

func (c *Client) DownloadIssue(ctx context.Context, url string) (*Issue, error)

DownloadIssue downloads the current issue JSON from the given URL and decodes it into an issue. Given an issue, c.DownloadIssue(issue.URL) fetches the very latest state for the issue.

func (*Client) DownloadIssueComment 

func (c *Client) DownloadIssueComment(ctx context.Context, url string) (*IssueComment, error)

DownloadIssueComment downloads the current comment JSON from the given URL and decodes it into an IssueComment. Given a comment, c.DownloadIssueComment(comment.URL) fetches the very latest state for the comment.

func (*Client) DownloadLabel 

func (c *Client) DownloadLabel(ctx context.Context, project, name string) (Label, error)

DownloadLabel downloads information about a label from GitHub.

func (*Client) EditIssue 

func (c *Client) EditIssue(ctx context.Context, issue *Issue, changes *IssueChanges) error

EditIssue applies the changes to issue on GitHub.

func (*Client) EditIssueComment 

func (c *Client) EditIssueComment(ctx context.Context, comment *IssueComment, changes *IssueCommentChanges) error

EditIssueComment changes the comment on GitHub to have the new body. It is typically a good idea to use c.DownloadIssueComment first and check that the live comment body matches the one obtained from the database, to minimize race windows.

func (*Client) EditLabel 

func (c *Client) EditLabel(ctx context.Context, project, name string, changes LabelChanges) error

EditLabel changes a label.

func (*Client) EnableTesting 

func (c *Client) EnableTesting()

EnableTesting enables testing mode, in which edits are diverted and a TestingClient is available. If the program is itself a test binary (built or run using “go test”), testing mode is enabled automatically. EnableTesting can be useful in experimental programs to make sure that no edits are applied to GitHub.

func (*Client) EventWatcher 

func (c *Client) EventWatcher(name string) *timed.Watcher[*Event]

EventWatcher returns a new timed.Watcher with the given name. It picks up where any previous Watcher of the same name left off.

func (*Client) Events 

func (c *Client) Events(project string, issueMin, issueMax int64) iter.Seq[*Event]

Events calls Events with the client's db.

func (*Client) EventsAfter 

func (c *Client) EventsAfter(t timed.DBTime, project string) iter.Seq[*Event]

EventsAfter returns an iterator over events in the given project after DBTime t, which should be e.DBTime from the most recent processed event. The events are iterated over in DBTime order, so the DBTime of the last successfully processed event can be used in a future call to EventsAfter. If project is the empty string, then events from all projects are returned.

func (*Client) EventsByAPI 

func (c *Client) EventsByAPI(project string, issue int64, api string) iter.Seq[*Event]

EventsByAPI returns an iterator over the events for the issue in the Client's db with the given API.

func (*Client) ListLabels 

func (c *Client) ListLabels(ctx context.Context, project string) ([]Label, error)

ListLabels lists all the labels in a project.

func (*Client) LookupIssueCommentEvent 

func (c *Client) LookupIssueCommentEvent(project string, issue, comment int64) (*Event, error)

LookupIssueCommentEvent looks up an issue comment event by project, issue, and comment number, (for example "golang/go", 12345, 135132324) only consulting the database (not actual GitHub).

func (*Client) LookupIssueCommentURL 

func (c *Client) LookupIssueCommentURL(url string) (*IssueComment, error)

LookupIssueCommentURL looks up an issue comment by HTML URL (for example https://github.com/golang/go/issues/12345#issuecomment-135132324), only consulting the database (not actual GitHub).

func (*Client) LookupIssueURL 

func (c *Client) LookupIssueURL(url string) (*Issue, error)

LookupIssueURL looks up an issue by URL (for example "https://github.com/golang/go/issues/12345"), only consulting the database (not actual GitHub).

func (*Client) PostIssueComment 

func (c *Client) PostIssueComment(ctx context.Context, issue *Issue, changes *IssueCommentChanges) (id, url string, err error)

PostIssueComment posts a new comment with the given body (written in Markdown) on issue. It returns an API URL for the new comment, and a URL suitable for display.

func (*Client) Sync 

func (c *Client) Sync(ctx context.Context) error

Sync syncs all projects.

func (*Client) SyncProject 

func (c *Client) SyncProject(ctx context.Context, project string) (err error)

SyncProject syncs a single project.

func (*Client) Testing 

func (c *Client) Testing() *TestingClient

Testing returns a TestingClient, which provides access to Client functionality intended for testing. Testing only returns a non-nil TestingClient in testing mode, which is active if the current program is a test binary (that is, testing.Testing returns true) or if Client.EnableTesting has been called. Otherwise, Testing returns nil.

Each Client has only one TestingClient associated with it. Every call to Testing returns the same TestingClient.

func (*Client) ToDocs 

func (*Client) ToDocs(e *Event) (iter.Seq[*docs.Doc], bool)

ToDocs converts an event containing an issue to an embeddable document. It returns (nil, false) if the event is not an issue. Implements docs.Source.ToDocs.

type Comment 

type Comment struct {
	URL  string `json:"url"`
	Body string `json:"body"`
	User User   `json:"user"`
}

Comment is the comment that triggered an event. https://docs.github.com/en/rest/issues/comments#get-an-issue-comment

type Event 

type Event struct {
	DBTime  timed.DBTime // when event was last written
	Project string       // project ("golang/go")
	Issue   int64        // issue number
	API     string       // API endpoint for event: "/issues", "/issues/comments", or "/issues/events"
	ID      int64        // ID of event; each API has a different ID space. (Project, Issue, API, ID) is assumed unique
	JSON    []byte       // JSON for the event data
	Typed   any          // Typed unmarshaling of the event data, of type *Issue, *IssueComment, or *IssueEvent
}

An Event is a single GitHub issue event stored in the database.

func (*Event) LastWritten 

func (e *Event) LastWritten() timed.DBTime

LastWritten implements docs.Entry.LastWritten.

type Issue 

type Issue struct {
	URL              string    `json:"url"`
	HTMLURL          string    `json:"html_url"`
	Number           int64     `json:"number"`
	User             User      `json:"user"`
	Title            string    `json:"title"`
	CreatedAt        string    `json:"created_at"`
	UpdatedAt        string    `json:"updated_at"`
	ClosedAt         string    `json:"closed_at"`
	Body             string    `json:"body"`
	Assignees        []User    `json:"assignees"`
	Milestone        Milestone `json:"milestone"`
	State            string    `json:"state"`
	PullRequest      *struct{} `json:"pull_request"`
	Locked           bool      `json:"locked"`
	ActiveLockReason string    `json:"active_lock_reason"`
	Labels           []Label   `json:"labels"`
}

Issue is the GitHub JSON structure for an issue creation event.

func LookupIssue 

func LookupIssue(db storage.DB, project string, issue int64) (*Issue, error)

LookupIssue looks up an issue by project and issue number (for example "golang/go", 12345), only consulting the database (not actual GitHub).

func (*Issue) Author 

func (x *Issue) Author() *model.Identity

func (*Issue) Body_ 

func (x *Issue) Body_() string

func (*Issue) CanEdit 

func (x *Issue) CanEdit() bool

func (*Issue) CanHaveChildren 

func (x *Issue) CanHaveChildren() bool

func (*Issue) CreatedAt_ 

func (x *Issue) CreatedAt_() time.Time

func (*Issue) DocID 

func (i *Issue) DocID() string

DocID returns the ID of this issue for storage in a docs.Corpus or a storage.VectorDB.

func (*Issue) ID 

func (x *Issue) ID() string

Methods implementing model.Post.

func (*Issue) ParentID 

func (x *Issue) ParentID() string

func (*Issue) Project 

func (x *Issue) Project() string

Project returns the issue's GitHub project (for example, "golang/go").

func (*Issue) Properties 

func (x *Issue) Properties() map[string]any

func (*Issue) Title_ 

func (x *Issue) Title_() string

func (*Issue) ToLLMDoc 

func (i *Issue) ToLLMDoc() *llmapp.Doc

ToLLMDoc converts an Issue to a format that can be used as an input to an LLM.

func (*Issue) UpdatedAt_ 

func (x *Issue) UpdatedAt_() time.Time

func (*Issue) Updates 

func (x *Issue) Updates() model.PostUpdates

type IssueChanges 

type IssueChanges struct {
	Title  string    `json:"title,omitempty"`
	Body   string    `json:"body,omitempty"`
	State  string    `json:"state,omitempty"`
	Labels *[]string `json:"labels,omitempty"`
}

An IssueChanges specifies changes to make to an issue. Fields that are the empty string or a nil pointer are ignored.

Note that Labels is the new set of all labels for the issue, not labels to add. If you are adding a single label, you need to include all the existing labels as well. Labels is a *[]string so that it can be set to new([]string) to clear the labels.

func (*IssueChanges) SetBody 

func (ch *IssueChanges) SetBody(s string) error

func (*IssueChanges) SetTitle 

func (ch *IssueChanges) SetTitle(s string) error

type IssueComment 

type IssueComment struct {
	URL       string `json:"url"`
	IssueURL  string `json:"issue_url"`
	HTMLURL   string `json:"html_url"`
	User      User   `json:"user"`
	CreatedAt string `json:"created_at"`
	UpdatedAt string `json:"updated_at"`
	Body      string `json:"body"`
}

IssueComment is the GitHub JSON structure for an issue comment event.

func (*IssueComment) Author 

func (x *IssueComment) Author() *model.Identity

func (*IssueComment) Body_ 

func (x *IssueComment) Body_() string

func (*IssueComment) CanEdit 

func (x *IssueComment) CanEdit() bool

func (*IssueComment) CanHaveChildren 

func (x *IssueComment) CanHaveChildren() bool

func (*IssueComment) CommentID 

func (x *IssueComment) CommentID() int64

CommentID returns the issue comment's numeric ID. The ID appears to be unique across all comments on GitHub, but we only assume it is unique within a single issue.

func (*IssueComment) CreatedAt_ 

func (x *IssueComment) CreatedAt_() time.Time

func (*IssueComment) ID 

func (x *IssueComment) ID() string

Methods implementing model.Post.

func (*IssueComment) Issue 

func (x *IssueComment) Issue() int64

Issue returns the issue comment's issue number.

func (*IssueComment) ParentID 

func (x *IssueComment) ParentID() string

func (*IssueComment) Project 

func (x *IssueComment) Project() string

Project returns the issue comment's GitHub project (for example, "golang/go").

func (*IssueComment) Properties 

func (x *IssueComment) Properties() map[string]any

func (*IssueComment) Title_ 

func (x *IssueComment) Title_() string

func (*IssueComment) ToLLMDoc 

func (ic *IssueComment) ToLLMDoc() *llmapp.Doc

ToLLMDoc converts an IssueComment to a format that can be used as an input to an LLM.

func (*IssueComment) UpdatedAt_ 

func (x *IssueComment) UpdatedAt_() time.Time

func (*IssueComment) Updates 

func (x *IssueComment) Updates() model.PostUpdates

type IssueCommentChanges 

type IssueCommentChanges struct {
	Body string `json:"body,omitempty"`
}

func (*IssueCommentChanges) SetBody 

func (ch *IssueCommentChanges) SetBody(s string) error

func (*IssueCommentChanges) SetTitle 

func (ch *IssueCommentChanges) SetTitle(string) error

type IssueEvent 

type IssueEvent struct {
	// NOTE: Issue field is not present when downloading for a specific issue,
	// only in the master feed for the whole repo. So do not add it here.
	ID         int64     `json:"id"`
	URL        string    `json:"url"`
	Actor      User      `json:"actor"`
	Event      string    `json:"event"`
	Label      Label     `json:"label"` // for "labeled" and "unlabeled" events
	Labels     []Label   `json:"labels"`
	LockReason string    `json:"lock_reason"`
	CreatedAt  string    `json:"created_at"`
	CommitID   string    `json:"commit_id"`
	Assigner   User      `json:"assigner"`
	Assignees  []User    `json:"assignees"`
	Milestone  Milestone `json:"milestone"`
	Rename     Rename    `json:"rename"`
}

IssueEvent is the GitHub JSON structure for an issue metadata event.

type Label 

type Label struct {
	Name        string `json:"name"`
	Description string `json:"description"`
	Color       string `json:"color"` // hex code without '#'
}

A Label represents a project issue tracker label in GitHub JSON.

type LabelChanges 

type LabelChanges struct {
	NewName     string `json:"new_name,omitempty"`
	Description string `json:"description,omitempty"`
	Color       string `json:"color,omitempty"`
}

LabelChanges specifies changes to make to a label. Only non-empty fields will be changed.

type Milestone 

type Milestone struct {
	Title string `json:"title"`
}

A Milestone represents a project issue milestone in GitHub JSON.

type Rename 

type Rename struct {
	From string `json:"from"`
	To   string `json:"to"`
}

A Rename describes an issue title renaming in GitHub JSON.

type Repository 

type Repository struct {
	Project string `json:"full_name"`
}

Repository is the repository in which an event occurred. https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#get-a-repository

type TestingClient 

type TestingClient struct {
	// contains filtered or unexported fields
}

A TestingClient provides access to Client functionality intended for testing.

See Client.Testing for a description of testing mode.

func (*TestingClient) AddIssue 

func (tc *TestingClient) AddIssue(project string, issue *Issue)

AddIssue adds the given issue to the identified project, assigning it a new issue number starting at 10⁹. AddIssue creates a new entry in the associated Client's underlying database, so other Client's using the same database will see the issue too.

NOTE: Only one TestingClient should be adding issues, since they do not coordinate in the database about ID assignment. Perhaps they should, but normally there is just one Client.

func (*TestingClient) AddIssueComment 

func (tc *TestingClient) AddIssueComment(project string, issue int64, comment *IssueComment) int64

AddIssueComment adds the given issue comment to the identified project issue, assigning it a new comment ID starting at 10¹⁰. AddIssueComment creates a new entry in the associated Client's underlying database, so other Client's using the same database will see the issue comment too.

NOTE: Only one TestingClient should be adding issues, since they do not coordinate in the database about ID assignment, Perhaps they should, but normally there is just one Client.

func (*TestingClient) AddIssueEvent 

func (tc *TestingClient) AddIssueEvent(project string, issue int64, event *IssueEvent)

AddIssueEvent adds the given issue event to the identified project issue, assigning it a new comment ID starting at 10¹¹. AddIssueEvent creates a new entry in the associated Client's underlying database, so other Client's using the same database will see the issue event too.

NOTE: Only one TestingClient should be adding issues, since they do not coordinate in the database about ID assignment. Perhaps they should, but normally there is just one Client.

func (*TestingClient) AddLabel 

func (tc *TestingClient) AddLabel(project string, lab Label)

AddLabel adds the given label to the client, so that calls to DownloadLabel and ListLabels will return it. It does not affect the database, since labels aren't stored there.

func (*TestingClient) ClearEdits 

func (tc *TestingClient) ClearEdits()

ClearEdits clears the list of edits that are meant to be applied

func (*TestingClient) Edits 

func (tc *TestingClient) Edits() []*TestingEdit

Edits returns a list of all the edits that have been applied using Client methods (for example Client.EditIssue, Client.EditIssueComment, Client.PostIssueComment). These edits have not been applied on GitHub, only diverted into the TestingClient.

See Client.Testing for a description of testing mode.

NOTE: These edits are not applied to the underlying database, since they are also not applied to the underlying database when using a real connection to GitHub; instead we wait for the next sync to download GitHub's view of the edits. See Client.EditIssue.

func (*TestingClient) LoadTxtar 

func (tc *TestingClient) LoadTxtar(file string) error

LoadTxtar loads issue histories from the named txtar file, writing them to the database using TestingClient.AddIssue, TestingClient.AddIssueComment, and TestingClient.AddIssueEvent.

The file should contain a txtar archive (see golang.org/x/tools/txtar). Each file in the archive should be named “project#n” (for example “golang/go#123”) and contain an issue history in the format printed by the rsc.io/github/issue command. See the file ../testdata/rsctmp.txt for an example.

To download a specific set of issues into a new file, you can use a script like: 

go install rsc.io/github/issue@latest
project=golang/go
(for i in 1 2 3 4 5
do
	echo "-- $project#$i --"
	issue -p $project $i
done) > testdata/proj.txt

func (*TestingClient) LoadTxtarData 

func (tc *TestingClient) LoadTxtarData(data []byte) error

LoadTxtarData loads issue histories from the txtar file content data. See [LoadTxtar] for a description of the format.

type TestingEdit 

type TestingEdit struct {
	Project             string
	Issue               int64
	Comment             int64
	IssueChanges        *IssueChanges
	IssueCommentChanges *IssueCommentChanges
	Label               Label
	LabelChanges        *LabelChanges
}

A TestingEdit is a diverted edit, which was logged instead of actually applied on GitHub.

func (*TestingEdit) String 

func (e *TestingEdit) String() string

String returns a basic string representation of the edit.

type User 

type User struct {
	Login string `json:"login"`
}

A User represents a user or organization account in GitHub JSON.

func (*User) ForDisplay 

func (u *User) ForDisplay() string

ForDisplay returns the user's login username.

type WebhookEvent 

type WebhookEvent struct {
	// Type specifies the type of event's Payload.
	Type WebhookEventType
	// Payload is the unmarshaled JSON payload of the webhook event,
	// of the Go type specified by Type.
	//
	// Event types that are not implemented use Go type [json.RawMessage].
	Payload any
}

WebhookEvent contains the data sent in a GitHub webhook request that is relevant for responding to the event.

func ValidateWebhookRequest 

func ValidateWebhookRequest(r *http.Request, db secret.DB) (*WebhookEvent, error)

ValidateWebhookRequest verifies that the request's payload matches the HMAC tag in the header and returns a WebhookEvent containing the unmarshaled payload.

The function is intended to validate authenticated POST requests received from GitHub webhooks.

It expects:

  • a POST request
  • an "X-GitHub-Event" header entry with a non-empty event type (see WebhookEventType)
  • a non-empty request body containing valid JSON representing an event of the specified event type
  • an "X-Hub-Signature-256" header entry of the form "sha256=HMAC", where HMAC is a valid hex-encoded HMAC tag of the request body computed with the key in db named "github-webhook"

The function returns an error if any of these conditions is not met.

func (*WebhookEvent) Project 

func (e *WebhookEvent) Project() string

Project returns the GitHub project (e.g., "golang/go") for the event, or an empty string if the project cannot be determined.

type WebhookEventType 

type WebhookEventType string

WebhookEventType is the name GitHub uses to refer to a type of GitHub webhook event.

Event types that are not implemented use Go type json.RawMessage.

See https://docs.github.com/en/webhooks/webhook-events-and-payloads for all possible event types. 

const (
	// An issue event.
	// Corresponds to Go type [*WebhookIssueEvent].
	WebhookEventTypeIssue WebhookEventType = "issues"
	// An issue comment event.
	// Corresponds to Go type [*WebhookIssueCommentEvent].
	WebhookEventTypeIssueComment WebhookEventType = "issue_comment"
)

type WebhookIssueAction 

type WebhookIssueAction string
const (
	WebhookIssueActionOpened WebhookIssueAction = "opened"
)

type WebhookIssueCommentAction 

type WebhookIssueCommentAction string
const (
	WebhookIssueCommentActionCreated WebhookIssueCommentAction = "created"
)

type WebhookIssueCommentEvent 

type WebhookIssueCommentEvent struct {
	Action     WebhookIssueCommentAction `json:"action"`
	Issue      Issue                     `json:"issue"`
	Repository Repository                `json:"repository"`
	Comment    Comment                   `json:"comment"`
}

WebhookIssueEvent is the structure of the JSON payload for a GitHub "issue_comment" event (for example, a new comment posted). https://docs.github.com/en/webhooks/webhook-events-and-payloads#issue_comment

type WebhookIssueEvent 

type WebhookIssueEvent struct {
	Action     WebhookIssueAction `json:"action"`
	Issue      Issue              `json:"issue"`
	Repository Repository         `json:"repository"`
}

WebhookIssueEvent is the structure of the JSON payload for a GitHub "issues" event (for example, a new issue created). https://docs.github.com/en/webhooks/webhook-events-and-payloads#issues

Source Files

View all Source files

Directories

Path Synopsis
Package wrap is used to wrap comments/edits made to GitHub so that they can later be identified without referencing a database.
 Package wrap is used to wrap comments/edits made to GitHub so that they can later be identified without referencing a database.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.