speakerdeck

package module
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 19, 2020 License: MIT Imports: 12 Imported by: 1

README

speakerdeck-api

A scraper implementation for SpeakerDeck, as a workaround due to the lack of an official API.

You can either use this as a library (import speakerdeck "github.com/luxas/speakerdeck-api") or the ready-made (very lightweight) API implementation under cmd/speakerdeck-api.

API Usage

Install and start the API:

$ go get github.com/luxas/speakerdeck-api/cmd/speakerdeck-api
$ $GOPATH/bin/speakerdeck-api
INFO[0000] Starting Speakerdeck API...

Get information about a user:

curl http://localhost:8080/api/users/luxas

{
  "author": {
    "name": "Lucas Käldström",
    "handle": "luxas",
    "link": "https://speakerdeck.com/luxas",
    "avatarLink": "https://secure.gravatar.com/avatar/111ac0b31c0dc219c84ddadedc8e5f67?s=128"
  },
  "abstract": "Lucas is a cloud native enthusiast who has been serving the Kubernetes \u0026 CNCF communities in lead positions for more than 4 years. Lucas is a CNCF Ambassador, running 3 meetup groups in Finland and coordinating the Cloud Native Nordics meetups. He got the \"Top Cloud Native Ambassador\" award together with Sarah Novotny 2017. Lucas has e.g. shepherded kubeadm from inception to GA as a co-lead for SIG Cluster Lifecycle, ported Kubernetes to Raspberry Pi and multiple other platforms. Lucas is a CKA, runs a consulting company \"luxas labs\" for cloud native tech and has spoken at six KubeCons.",
  "talkPreviews": [
    ...,
    {
      "title": "Getting Started in the Kubernetes Community",
      "id": "getting-started-in-the-kubernetes-community",
      "views": 92,
      "stars": 1,
      "date": "2019-05-21T00:00:00Z",
      "link": "https://speakerdeck.com/luxas/getting-started-in-the-kubernetes-community",
      "dataID": "6816e10f104a44cebb0915b392cadd2d"
    },
    ...,
  ]
}

Get detailed information about a user's talks (all of them):

curl http://localhost:8080/api/talks/luxas

[
  ...,
  {
    "title": "Getting Started in the Kubernetes Community",
    "id": "getting-started-in-the-kubernetes-community",
    "views": 92,
    "stars": 1,
    "date": "2019-05-21T00:00:00Z",
    "link": "https://speakerdeck.com/luxas/getting-started-in-the-kubernetes-community",
    "dataID": "6816e10f104a44cebb0915b392cadd2d",
    "author": {
      "name": "Lucas Käldström",
      "handle": "luxas",
      "link": "https://speakerdeck.com/luxas",
      "avatarLink": "https://secure.gravatar.com/avatar/111ac0b31c0dc219c84ddadedc8e5f67?s=47"
    },
    "category": "Technology",
    "categoryLink": "https://speakerdeck.com/c/technology",
    "downloadLink": "https://speakerd.s3.amazonaws.com/presentations/6816e10f104a44cebb0915b392cadd2d/Lucas_Kaldstrom-Nikhita_Raghunath_-_May_21_-_Morning.pdf",
    "extraLinks": {
      "github.com": [
        "https://github.com/nikhita"
      ]
    },
    "hide": false
  },
  ...,
]

Get detailed information about one of the user's talks:

curl http://localhost:8080/api/talks/luxas/getting-started-in-the-kubernetes-community

[
  {
    "title": "Getting Started in the Kubernetes Community",
    "id": "getting-started-in-the-kubernetes-community",
    "views": 92,
    "stars": 1,
    "date": "2019-05-21T00:00:00Z",
    "link": "https://speakerdeck.com/luxas/getting-started-in-the-kubernetes-community",
    "dataID": "6816e10f104a44cebb0915b392cadd2d",
    "author": {
      "name": "Lucas Käldström",
      "handle": "luxas",
      "link": "https://speakerdeck.com/luxas",
      "avatarLink": "https://secure.gravatar.com/avatar/111ac0b31c0dc219c84ddadedc8e5f67?s=47"
    },
    "category": "Technology",
    "categoryLink": "https://speakerdeck.com/c/technology",
    "downloadLink": "https://speakerd.s3.amazonaws.com/presentations/6816e10f104a44cebb0915b392cadd2d/Lucas_Kaldstrom-Nikhita_Raghunath_-_May_21_-_Morning.pdf",
    "extraLinks": {
      "github.com": [
        "https://github.com/nikhita"
      ]
    },
    "hide": false
  }
]

For reference you can visit https://speakerdeck.com/luxas/getting-started-in-the-kubernetes-community to check where the data is coming from.

Geolocation

speakerdeck-api also has support for extensions, the extension that currently exists is LocationExtension (in ./location), which can using a Google Maps API key (with access to the Geocoding API) geolocate your talks just by you putting Location: <address> in the Speakerdeck talk description!

Enable it by giving the Google Maps key to the API:

$GOPATH/bin/speakerdeck-api -maps-api-key <API_KEY>

Now, notice the location field which is generated based on the Location: Fira Gran Via, Av. Joan Carles I, Barcelona, Spain string embedded in the talk description:

curl http://localhost:8080/api/talks/luxas/getting-started-in-the-kubernetes-community

[
  {
    "title": "Getting Started in the Kubernetes Community",
    "id": "getting-started-in-the-kubernetes-community",
    "views": 92,
    "stars": 1,
    "date": "2019-05-21T00:00:00Z",
    "link": "https://speakerdeck.com/luxas/getting-started-in-the-kubernetes-community",
    "dataID": "6816e10f104a44cebb0915b392cadd2d",
    "author": {
      "name": "Lucas Käldström",
      "handle": "luxas",
      "link": "https://speakerdeck.com/luxas",
      "avatarLink": "https://secure.gravatar.com/avatar/111ac0b31c0dc219c84ddadedc8e5f67?s=47"
    },
    "category": "Technology",
    "categoryLink": "https://speakerdeck.com/c/technology",
    "downloadLink": "https://speakerd.s3.amazonaws.com/presentations/6816e10f104a44cebb0915b392cadd2d/Lucas_Kaldstrom-Nikhita_Raghunath_-_May_21_-_Morning.pdf",
    "extraLinks": {
      "github.com": [
        "https://github.com/nikhita"
      ]
    },
    "hide": false,
    "location": {
      "requestedAddress": "Fira Gran Via, Av. Joan Carles I, Barcelona, Spain",
      "resolvedAddress": "Av. Joan Carles I, 64, 08908 L'Hospitalet de Llobregat, Barcelona, Spain",
      "lat": 41.3546878,
      "lng": 2.1277339
    }
  }
]

Library Usage

Check out the documentation on Godoc or pkg.go.dev!

License

MIT

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Author 

type Author struct {
	// Name describes the human-friendly name of the author
	Name string `json:"name"`

	// Handle describes the preferred nickname of the author. This is used in the URL
	Handle string `json:"handle"`

	// Link describes the link to the author's page
	Link string `json:"link"`

	// AvatarLink describes the link to the avatar of the author
	AvatarLink string `json:"avatarLink"`
}

Author describes the SD profile of the person that's created the presentations

type Location 

type Location struct {
	// RequestedAddress is populated from the original "requested" address mentioned in the
	// talk description (e.g. "TUAS-talo, Aalto University")
	RequestedAddress string `json:"requestedAddress"`

	// ResolvedAddress is populated by the Google Maps Geocoding API, and is the official street address
	// (or similar) of the place. In the above example: "Maarintie 8, 02150 Espoo, Finland"
	ResolvedAddress string `json:"resolvedAddress"`

	// Lat describes the latitude of the location
	Lat float64 `json:"lat"`

	// Lng describes the longitude of the location
	Lng float64 `json:"lng"`
}

Location describes a geographical location for the talk This struct is populated by the LocationExtension, and is set based on a "Location: <address>" string in the talk description. For instance, if you put "Location: TUAS-talo, Aalto University" in the talk description, Location will be populated with coordinates based on Google Maps data.

type Talk 

type Talk struct {
	// TalkPreview is embedded here as it contains all the data we want to display here, too
	TalkPreview

	// Date represents the talk presentation date
	Date time.Time `json:"date"`

	// Author describes the Speakerdeck profile of the person that's created the presentations
	Author Author `json:"author"`

	// Category is a string of Speakerdeck-specific categories you can choose from when uploading a
	// presentation, e.g. "Technology"
	Category string `json:"category"`

	// CategoryLink is the link to other presentations belonging to the same category, e.g.
	// https://speakerdeck.com/c/technology.
	CategoryLink string `json:"categoryLink"`

	// DownloadLink is the link from where you can download the underlying PDF
	DownloadLink string `json:"downloadLink"`

	// ExtraLinks contains parsed URLs found in the talk description, mapped by their domain name
	ExtraLinks map[string][]string `json:"extraLinks"`

	// Hide is set to true if the talk description contains a "Hide: true" string,
	// indicating it should not be visible for the API. If a talk has set Hide=true,
	// it will be returned as normal, and the user of the API can choose whether to ignore
	// or respect that pledge
	Hide bool `json:"hide"`

	// Location describes a geographical location for the talk
	// This field is populated by the LocationExtension, and is set based on
	// a "Location: <address>" string in the talk description. For instance,
	// if you put "Location: TUAS-talo, Aalto University" in the talk description,
	// Location will be populated with coordinates based on Google Maps data.
	Location *Location `json:"location,omitempty"`
}

Talk describes a presentation on Speakerdeck

func NewTalk 

func NewTalk() *Talk

NewTalk returns a new, empty talk object

type TalkPreview 

type TalkPreview struct {
	// Title describes the talk title
	Title string `json:"title"`

	// ID describes the URL-encoded descriptor for the talk, used in the URL as
	// https://speakerdeck.com/{user-handle}/{talk-id}. The ID is unique per user, but
	// not globally across users
	ID string `json:"id"`

	// Views describes how many views a talk has got
	Views uint32 `json:"views"`

	// Stars describes how many other users have starred this talk
	Stars uint32 `json:"stars"`

	// Link describes the link to the talk at Speakerdeck
	Link string `json:"link"`

	// DataID represents the key used to embed Speakerdeck presentations on an other website
	DataID string `json:"dataID"`
}

TalkPreview contains the information about a talk that can be seen on the user page

type TalkScraper 

type TalkScraper struct{}

TalkScraper implements scraper.Scraper

func (*TalkScraper) Hooks 

func (s *TalkScraper) Hooks() []scraper.Hook

Hooks returns mappings between DOM paths in the scraped web pages, and handler functions to extract data out of them

func (*TalkScraper) InitialData 

func (s *TalkScraper) InitialData() interface{}

InitialData returns the struct pointer passed around between the handler functions registered in Hooks() This pointer is passed as the second argument to all handlers. The handlers can cast it from interface{} to its real type, and modify its data.

func (*TalkScraper) Name 

func (s *TalkScraper) Name() string

Name returns the name of the TalkScraper

type Talks 

type Talks []Talk

Talks orders the Talk objects by time

func ScrapeTalks 

func ScrapeTalks(userHandle, talkID string, opts *scraper.ScrapeOptions) (Talks, error)

ScrapeTalks returns either one sepecific talk if both userHandle and talkID are set, or a set of all the users' talks in detail if only userHandle is set. In opts you can set extensions

func (Talks) Len 

func (p Talks) Len() int

Len implements sort.Interface

func (Talks) Less 

func (p Talks) Less(i, j int) bool

Less implements sort.Interface

func (Talks) Swap 

func (p Talks) Swap(i, j int)

Swap implements sort.Interface

type User 

type User struct {
	// Author describes the Speakerdeck profile of the person that's created the presentations
	Author Author `json:"author"`

	// Abstract contains a short description of the user
	Abstract string `json:"abstract"`

	// TalkPreviews is a list of TalkPreview objects, containing information about each talk
	// as seen on the user page
	TalkPreviews []TalkPreview `json:"talkPreviews"`
}

User represents a user on as browsed on the user page (i.e. https://speakerdeck.com/{user-handle})

func NewUser 

func NewUser() *User

NewUser creates a new User object

func ScrapeUser 

func ScrapeUser(userHandle string, opts *scraper.ScrapeOptions) (*User, error)

ScrapeUser returns an user object based on the given user handle. In opts, you may specify possible scraping extensions, or log levels.

type UserScraper 

type UserScraper struct{}

UserScraper implements scraper.Scraper

func (*UserScraper) Hooks 

func (s *UserScraper) Hooks() []scraper.Hook

Hooks returns mappings between DOM paths in the scraped web pages, and handler functions to extract data out of them

func (*UserScraper) InitialData 

func (s *UserScraper) InitialData() interface{}

InitialData returns the struct pointer passed around between the handler functions registered in Hooks() This pointer is passed as the second argument to all handlers. The handlers can cast it from interface{} to its real type, and modify its data.

func (*UserScraper) Name 

func (s *UserScraper) Name() string

Name returns the name of the UserScraper

Source Files

View all Source files

Directories

Path Synopsis
cmd
speakerdeck-api
The scraper package contains generic, high-level scraper functionality built on top of github.com/gocolly/colly In order to use it, create a struct (e.g.
 The scraper package contains generic, high-level scraper functionality built on top of github.com/gocolly/colly In order to use it, create a struct (e.g.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.