command module
Version: v0.0.0-...-5f5f8f3 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Feb 26, 2021 License: MIT Imports: 6 Imported by: 0


πŸ•· makescraper

Go Report Card

Create your very own web scraper and crawler using Go and Colly!

πŸ“š Table of Contents
  1. Project Structure
  2. Getting Started
  3. Deliverables
  4. Resources

Project Structure

πŸ“‚ makescraper
└── scrape.go

Getting Started

  1. Visit and create a new repository named makescraper.

  2. Run each command line-by-line in your terminal to set up the project:

    $ git clone
    $ cd makescraper
    $ git remote rm origin
    $ git remote add origin
    $ go mod download
  3. Open in your editor and replace all instances of YOUR_GITHUB_USERNAME with your GitHub username to enable the Go Report Card badge.


Complete each task in the order they appear. Use GitHub Task List syntax to update the task list.

  • IMPORTANT: Complete the Web Scraper Workflow worksheet distributed in class.
  • Create a struct to store your data.
  • Refactor the c.OnHTML callback on line 16 to use the selector(s) you tested while completing the worksheet.
  • Print the data you scraped to stdout.
Stretch Challenges
  • Add more fields to your struct. Extract multiple data points from the website. Print them to stdout in a readable format.
Serializing & Saving
  • Serialize the struct you created to JSON. Print the JSON to stdout to validate it.
  • Write scraped data to a file named output.json.
  • Add, commit, and push to GitHub.


Lesson Plans
Example Code
Serializing & Saving
  • JSON to Struct: Paste any JSON data and convert it into a Go structure that will support storing that data.
  • GoByExample - JSON: Covers Go's built-in support for JSON encoding and decoding to and from built-in and custom data types (structs).
  • GoByExample - Writing Files: Covers creating new files and writing to them.

Documentation ΒΆ

The Go Gopher

There is no documentation for this package.

Source Files ΒΆ

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL