cocogh

README

coco-gh

A Go library to fetch contents from GitHub

Features

• Fetch all file paths based on the configuration.
• Fetch a list of file paths that were changed in the last X hours.

Getting Started

Download the library using go get:

go get github.com/shaharia-lab/coco-gh

Import the library in your code:

import "github.com/shaharia-lab/coco-gh"

Usage

package cocogh

import (
   "context"
   "log"
   "time"

   "github.com/shurcooL/githubv4"
   "golang.org/x/oauth2"
)

func main() {
   src := oauth2.StaticTokenSource(
      &oauth2.Token{AccessToken: os.Getenv("GITHUB_TOKEN")},
   )
   httpClient := oauth2.NewClient(context.Background(), src)

   ghCommitsOpsClient := NewGitHubCommitsOpsClient(httpClient)
   graphQLClient := githubv4.NewClient(httpClient)
   ghConfig := GitHubConfig{
      Owner:         "kubernetes",
      Repositories:  []string{"website"},
      DefaultBranch: "main",
      Filter: GitHubFilter{
         FilePath: "content/en/blog/_posts",
         FileTypes: []string{
            ".md",
         },
      },
   }

   ch := NewGitHubClient(ghCommitsOpsClient, graphQLClient, ghConfig)

   // Get all the file paths from the repositories
   allFilePaths, err := ch.GetFilePathsFromRepositories()
   if err != nil {
      // handle errors
   }

   for _, path := range allFilePaths {
      log.Println(path)
   }

   // Get the list of files that were changed in the last X hours
   contentChanged, err := ch.GetChangedFilePathsSince(time.Now())
   if err != nil {
      // handle errors
   }

   log.Println(contentChanged.Added)
   log.Println(contentChanged.Modified)
   log.Println(contentChanged.Removed)
}

Contributing

Contributions to coco-gh are more than welcome! If you're looking to contribute to our project, you're in the right place. Here are some ways you can help:

1. Report Bugs: If you find a bug, please open an issue to report it. Describe the bug, how to reproduce it, and the environment (e.g., OS, Go version).

2. Suggest Enhancements: Have an idea to make this project better? Open an issue to suggest your idea. Whether it's a new feature, code improvement, or documentation updates, we'd love to hear from you.

3. Submit Pull Requests: Feel free to fork the repository and submit pull requests. Before submitting your pull request, please ensure the following:

   • Your code follows the project's coding standards.
   • All tests are passing.
   • Add or update tests as necessary for your code.
   • Update the documentation to reflect your changes, if applicable.
   • Include a clear description in your PR about the changes you have made.

4. Review Pull Requests: If you're interested in contributing by reviewing pull requests, please feel free to do so. Any feedback or suggestions are highly valuable.

We appreciate your contributions and look forward to your active participation in the development of coco-gh!

License

coco-gh is licensed under the MIT License.

Documentation

Overview

Package cocogh to collect GitHub contents

Index

• type CommitOpsClient
• type GHQueryForListFiles
• type GitHub
  • func NewGitHubClient(commitOpsClient CommitOpsClient, graphQLClient GraphQLClient, ...) *GitHub
  • func (c *GitHub) GetChangedFilePathsSince(since time.Time) (Paths, error)
  • func (c *GitHub) GetFilePathsFromRepositories() ([]string, error)
• type GitHubCommitsOpsClient
  • func NewGitHubCommitsOpsClient(httpClient *http.Client) *GitHubCommitsOpsClient
  • func (gClient *GitHubCommitsOpsClient) GetCommit(ctx context.Context, owner, repo, sha string, opts *github.ListOptions) (*github.RepositoryCommit, *github.Response, error)
  • func (gClient *GitHubCommitsOpsClient) ListCommits(ctx context.Context, owner, repo string, opts *github.CommitsListOptions) ([]*github.RepositoryCommit, *github.Response, error)
• type GitHubConfig
• type GitHubFilter
• type GraphQLClient
• type Paths

Types

type CommitOpsClient

type CommitOpsClient interface {
	ListCommits(ctx context.Context, owner, repo string, opts *github.CommitsListOptions) ([]*github.RepositoryCommit, *github.Response, error)
	GetCommit(ctx context.Context, owner, repo, sha string, opts *github.ListOptions) (*github.RepositoryCommit, *github.Response, error)
}

CommitOpsClient is an interface to help test the GitHub GitHubCommitsOpsClient.

type GHQueryForListFiles

type GHQueryForListFiles struct {
	Repository struct {
		Object struct {
			Tree struct {
				Entries []struct {
					Name string
					Path string
					Type string
				}
			} `graphql:"... on Tree"`
		} `graphql:"object(expression: $expression)"`
	} `graphql:"repository(owner: $owner, name: $name)"`
}

GHQueryForListFiles is a struct representing the GraphQL query for listing files in a GitHub repository. It contains the information necessary to make the query, including the owner, name, expression, and path of the repository.

type GitHub

type GitHub struct {
	Configuration GitHubConfig
	// contains filtered or unexported fields
}

GitHub stores CommitOpsClient, GraphQLClient and configuration.

func NewGitHubClient

func NewGitHubClient(commitOpsClient CommitOpsClient, graphQLClient GraphQLClient, configuration GitHubConfig) *GitHub

NewGitHubClient creates a new instance of the GitHub client. It takes a CommitOpsClient, GraphQLClient, and GitHubConfig as parameters and returns a pointer to a GitHub struct. The CommitOpsClient is responsible for making REST API calls to the GitHub API. The GraphQLClient is responsible for making GraphQL API calls to the GitHub API. The GitHubConfig contains the configuration parameters for the GitHub client, such as owner, repositories, default branch, and filter options. The new GitHub client is initialized with the provided CommitOpsClient, GraphQLClient, and GitHubConfig. The GitHub client can be used to interact with the GitHub API and perform various operations, such as retrieving file paths for repositories and getting changed file paths since a specified time. Usage:

 commitOpsClient := NewGitHubCommitsOpsClient()
 graphQLClient := NewGraphQLClient()

	config := GitHubConfig{
	    Owner:         "testowner",
	    Repositories:  []string{"repo1", "repo2"},
	    DefaultBranch: "main",
	    Filter: GitHubFilter{
	        FilePath:  "path/to/files",
	        FileTypes: []string{".txt"},
	    },
	}

 githubClient := NewGitHubClient(commitOpsClient, graphQLClient, config)
 filepaths, err := githubClient.GetFilePathsFromRepositories()

	if err != nil {
	    log.Fatal(err)
	}

	for _, path := range filepaths {
	    fmt.Println(path)
	}

func (*GitHub) GetChangedFilePathsSince

func (c *GitHub) GetChangedFilePathsSince(since time.Time) (Paths, error)

GetChangedFilePathsSince retrieves the list of file paths that have changed in the specified repositories within the specified time frame. The function iterates over repositories defined in the GitHub configuration and uses the GitHub commit operations client to fetch the commits and commit details for each repository. It aggregates the file paths from all repositories into a single Paths object. The function filters these file paths based on the directory filter and the specified time frame and file path filter defined in the GitHub configuration. The Paths object is populated with lists of added, removed, and modified file paths accordingly.

Parameters:

• hoursSince: An integer representing the number of hours since the specified time. This parameter is used to determine the time frame for fetching changed file paths.

Returns:

• Paths: A struct containing lists of added, removed, and modified file paths. This struct provides an organized way to access the changed files.
• error: An error instance if an error occurs during the execution of the function. It will be nil if the function executes successfully.

Usage:

const sinceHours = 24
changedFiles, err := c.GetChangedFilePathsSince(sinceHours)
if err != nil {
    log.Fatal(err)
}

fmt.Println("Added files:", changedFiles.Added)
fmt.Println("Modified files:", changedFiles.Modified)
fmt.Println("Removed files:", changedFiles.Removed)

func (*GitHub) GetFilePathsFromRepositories

func (c *GitHub) GetFilePathsFromRepositories() ([]string, error)

GetFilePathsFromRepositories retrieves the file paths for the repositories specified in the GitHub configuration. It iterates over each repository, calls the getFilePathsForRepo method to get the file paths, and appends them to the files slice. If there are no file types specified in the configuration, it returns the files directly. Otherwise, it filters the files based on the file types specified in the configuration and returns the filtered files. If there's an error during the process, it returns nil and the error.

Usage:

repos := []string{"repo1", "repo2", "repo3"}
filePaths, err := GetFilePathsFromRepositories(repos)
if err != nil {
    log.Fatal(err)
}

for _, paths := range filePaths {
    for _, path := range paths {
        fmt.Println(path)
    }
}

type GitHubCommitsOpsClient

type GitHubCommitsOpsClient struct {
	GitHubClient *github.Client
}

GitHubCommitsOpsClient is a type that represents an operations client for GitHub commits. It contains a pointer to a GitHub client from the go-github library.

func NewGitHubCommitsOpsClient

func NewGitHubCommitsOpsClient(httpClient *http.Client) *GitHubCommitsOpsClient

NewGitHubCommitsOpsClient creates a new GitHubCommitsOpsClient with the given http.Client. It initializes the GitHubClient inside GitHubCommitsOpsClient using the provided http.Client. The GitHubClient is responsible for interacting with the GitHub API.

func (*GitHubCommitsOpsClient) GetCommit

func (gClient *GitHubCommitsOpsClient) GetCommit(ctx context.Context, owner, repo, sha string, opts *github.ListOptions) (*github.RepositoryCommit, *github.Response, error)

GetCommit retrieves a specific commit from a repository.

Parameters:

• ctx: The context.Context used for the API call. It allows you to cancel the request, set deadlines, etc.
• owner: The username or organization name of the repository owner. This string identifies the owner of the repository.
• repo: The name of the repository. It specifies which repository's commit is being retrieved.
• sha: The SHA hash of the commit. This string uniquely identifies the commit within the repository.
• opts: Optional parameters for the API call, provided as a pointer to github.ListOptions. This includes pagination options.

Returns:

• *github.RepositoryCommit: The retrieved commit information, including details like the commit message, author, etc.
• *github.Response: The HTTP response from the API call. This includes information like the status code and headers.
• error: An error instance if an error occurs during the API call. It will be nil if the call is successful.

Example:

commit, resp, err := gClient.GetCommit(ctx, "octocat", "hello-world", "6dcb09b5b57875f334f61aebed695e2e4193db5e", nil)

func (*GitHubCommitsOpsClient) ListCommits

func (gClient *GitHubCommitsOpsClient) ListCommits(ctx context.Context, owner, repo string, opts *github.CommitsListOptions) ([]*github.RepositoryCommit, *github.Response, error)

ListCommits fetches the list of commits for a specific repository.

type GitHubConfig

type GitHubConfig struct {
	Owner         string
	Repositories  []string
	DefaultBranch string
	Filter        GitHubFilter
}

GitHubConfig represents the configuration for GitHub repositories.

Owner represents the owner of the repositories. Repositories represents a list of repository names. DefaultBranch represents the default branch for the repositories. Filter represents the filter to apply when fetching file paths from the repositories.

type GitHubFilter

type GitHubFilter struct {
	FilePath  string
	FileTypes []string
}

GitHubFilter represents a filter used to narrow down the file paths in a GitHub repository based on the file path and file types.

type GraphQLClient

type GraphQLClient interface {
	Query(ctx context.Context, q interface{}, variables map[string]interface{}) error
}

GraphQLClient is an interface to help test the GitHub GraphQLClient.

type Paths

type Paths struct {
	Added    []string
	Removed  []string
	Modified []string
}

Paths represents a collection of file paths that have been added, removed, or modified.