fetcher

Documentation

Overview

Package fetcher 는 데이터를 네트워크에서 받아오는 함수를 정의합니다.

Index

• type Fetcher
  • func New(client paperswithcode_go.Client, youtubeService *youtube.Service) *Fetcher
  • func (f *Fetcher) FetchMultiYouTubeVideoByIDs(videoIDs []string) ([]*pr12er.YouTubeVideo, error)
  • func (f *Fetcher) FetchOnlyPapers(prRow *pr12er.MappingTableRow) (*pr12er.PrVideo, error)
  • func (f *Fetcher) FetchYouTubeVideos(videoIDToPr map[string]int32, batchSize int) (map[int32]*pr12er.YouTubeVideo, error)

Types

type Fetcher

type Fetcher struct {
	// contains filtered or unexported fields
}

func New

func New(client paperswithcode_go.Client, youtubeService *youtube.Service) *Fetcher

func (*Fetcher) FetchMultiYouTubeVideoByIDs

func (f *Fetcher) FetchMultiYouTubeVideoByIDs(videoIDs []string) ([]*pr12er.YouTubeVideo, error)

FetchMultiYouTubeVideoByIDs is a low level function that returns videos by its IDs. If there is a next page token, it will iterate each page.

func (*Fetcher) FetchOnlyPapers

func (f *Fetcher) FetchOnlyPapers(prRow *pr12er.MappingTableRow) (*pr12er.PrVideo, error)

FetchOnlyPapers fetches papers without video information.

func (*Fetcher) FetchYouTubeVideos

func (f *Fetcher) FetchYouTubeVideos(videoIDToPr map[string]int32, batchSize int) (map[int32]*pr12er.YouTubeVideo, error)

FetchYouTubeVideos fetches YouTubeVideo and returns a map[PR-ID]Video. Because we can't send 200+ IDs requests at once, we use a wrapper function to split by batchSize. We need to return the map so that it can plug back to the correct PR video.