malscraper

package module
v0.8.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 28, 2020 License: MIT Imports: 0 Imported by: 2

README


Build Status Coverage Status Go Report Card GitHub tag (latest SemVer) Documentation

go-malscraper is just another project scraping/parsing MyAnimeList website to a useful and easy-to-use data by using Go. It is a simple REST API that you can host yourself. It also provides the API library that you can use for your other Go projects.

Well, it is created to help people get MyAnimeList data without relying on MyAnimeList since they limited/disabled/closed their API. It's working as long as the web is up and the UI design stays the same so it can get the page sources and parse them.

go-malscraper is using PuerkitoBio's HTML DOM parser and inspired by Jikan's API library and my PHP Mal-Scraper library.

Features

  • Get anime information (details, characters, episodes, pictures, etc)
  • Get manga information (details, characters, pictures, recommendations, etc)
  • Get character information (details, pictures)
  • Get people information (details, pictures)
  • Get list of all anime/manga's genres
  • Get list of all anime/manga's producers/studios/licensors/magazines/serializations
  • Get anime/manga's recommendations
  • Get anime/manga's reviews
  • Search anime, manga, character and people
  • Get seasonal anime list
  • Get anime, manga, character and people top list
  • Get user information (profile, friends, histories, recommendations, reviews)
  • Get news list and details
  • Get featured article list and details
  • Get club list and details
  • Redis caching
  • Command-Line Interface (CLI)
  • Database support (PostgreSQL)
  • Dockerized containers
  • Kubernetes deployment

More will be coming soon...

Quick Start

With docker
docker pull rl404/malscraper
docker run -it --name malscraper -p 8005:8005 rl404/malscraper
With Go
go get github.com/rl404/go-malscraper
cd $GOPATH/src/github/rl404/go-malscraper
make

http://localhost:8005 is ready to use.

Feature Endpoint Example
Get anime detail information /v1/anime/:id /v1/anime/1
Get manga detail information /v1/manga/:id /v1/manga/1
Get character detail information /v1/character/:id /v1/character/1
Get manga detail information /v1/manga/:id /v1/manga/1
Get people detail information /v1/people/:id /v1/people/1
Get genres detail information /v1/genres/:type /v1/genres/anime
Get list of all anime producers /v1/producers /v1/producers
Get list of all anime recommendations /v1/recommendations/:type /v1/recommendations/anime
Get list of all anime reviews /v1/reviews/:type /v1/reviews/anime
Search anime /v1/search/anime /v1/search/anime?query=naruto
Get list of seasonal anime /v1/season /v1/season?year=2019&season=fall
Get list of top anime /v1/top/anime /v1/top/anime
Get user profile /v1/user/:user /v1/user/rl404
Get list of news /v1/news /v1/news
Get list of featured articles /v1/featured /v1/featured
Get list of clubs /v1/clubs /v1/clubs

For more detail information, please go to the documentation.

Contributing

  1. Clone it!
  2. Create your feature branch: git checkout -b my-new-feature
  3. Commit your changes: git commit -am 'Add some feature'
  4. Push to the branch: git push origin my-new-feature
  5. Submit a pull request.

Disclamer

go-malscraper is meant for educational purpose and personal usage only. Although there is no limit in using the API, do remember that every scraper method is accessing MyAnimeList page so use it responsibly according to MyAnimeList's Terms Of Service.

All data (including anime, manga, people, etc) and MyAnimeList logos belong to their respective copyrights owners. go-malscraper does not have any affiliation with content providers.

Additional Badges

Since I like badges, I try to collect as many badges as I can. :)

GitHub go.mod Go version Circle CI codecov CodeFactor Drone CI Appveyor CI

Gitpod ready-to-code

ForTheBadge built-with-love

License

MIT License

Copyright (c) 2020 Axel

Documentation

The Go Gopher

There is no documentation for this package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL