scraper

command module
v0.8.1-beta Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 23, 2015 License: MIT Imports: 43 Imported by: 0

README

scraper

An auto-scraper for EmulationStation written in Go using hashes. This currently works with NES, SNES, N64, GB, GBC, GBA, MD, SMS, 32X, GG, PCE, A2600, LNX, MAME(see below), Dreamcast(bin/gdi), PSX(bin/cue) ROMs.

How it Works

The script works by crawling a directory of ROM files looking for known extensions. When it finds a file it hashes the ROM data minus any headers or special file formatting with the goal of hashing only the data pulled from the original game. It compares this hash to a DB I've compiled to look up the correct game in theGamesDB.net. It downloads the metadata and builds the gamelist.xml file.

Installation

Make sure you have go version 1.2 or later installed.

$ go version
go version go1.2.1 linux/amd64

Fetch and build.

$ go get github.com/sselph/scraper
$ go build github.com/sselph/scraper

Usage

$ cd <rom directory>
$ scraper

ROMs will be scanned and a gamelist.xml file will be created. All images will be placed inside the images folder.

MAME

The scraper now supports MAME but using file names instead of hashing. Since it uses a different DB and lookup method, several of the command line flags no longer apply. When the -mame flag is used it disables all other databases and the mamedb only has the one size of image so the flags about thumbnails, gdb, ovgdb, etc don't do anything.

$ scraper -mame

You can choose your preference of image type with the mame_img flag. If you prefer marquees but want to fallback to titles then snapshots you can do:

$ scraper -mame -mame_img "m,t,s"

Command Line Flags

There are several command flags you can pass. To see a full list use -help

$ scraper -help

Raspberry Pi

At the time of writing this raspbian has an old version of go 1.0.2 so you can cross-compile on another system or download an unofficial go binary from http://dave.cheney.net/unofficial-arm-tarballs.

Build:
Rpi v1
$ GOARM=6 GOARCH=arm GOOS=linux go build github.com/sselph/scraper
Rpi v2
$ GOARM=7 GOARCH=arm GOOS=linux go build github.com/sselph/scraper
Install from my Binaries

Replace the release_name with the release like v0.6.3-beta

Rpi v1
$ wget https://github.com/sselph/scraper/releases/download/<release_name>/scraper_rpi.zip
$ sudo unzip scraper_rpi.zip scraper -d /usr/local/bin/
Rpi v2
$ wget https://github.com/sselph/scraper/releases/download/<release_name>/scraper_rpi2.zip
$ sudo unzip scraper_rpi2.zip scraper -d /usr/local/bin/
Usage

Add thumb_only can speed things up since the pi doesn't have a ton of memory.

Single System
$ cd ~/RetroPie/roms/<rom_dir>
$ scraper -thumb_only
All Systems
$ scraper -scrape_all -thumb_only

Used libraries

Package Description License
github.com/nfnt/resize resizes images ISC
github.com/kr/fs provides filesystem-related functions New BSD
github.com/mattn/go-sqlite3 provides sqlite MIT
github.com/mitchellh/go-homedir get user homedir MIT
github.com/syndtr/goleveldb provides leveldb Simplified BSD

Documentation

The Go Gopher

There is no documentation for this package.

Directories

Path Synopsis
Package gdb interacts with thegamedb.net's API.
Package gdb interacts with thegamedb.net's API.
rom
Package rom has helper functions for extracting rom data.
Package rom has helper functions for extracting rom data.
bin
gb
lnx
md
n64
nes
Package nes decodes .nes files
Package nes decodes .nes files
pce
sms
snes
Package snes decodes .sms and .sfc files.
Package snes decodes .sms and .sfc files.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL