gopa

README

GOPA, A Spider Written in Go.

Goal

• Light weight, low footprint, memory requirement should < 100MB
• Easy to deploy, no runtime or dependency required
• Easy to use, no programming or scripts ability needed, out of box features

Screenshoot

Build

Mac/Linux: Run make build to build the Gopa

Windows: Checkout this wiki page - How to build GOPA on windows

Download

Release

Start

After download/build the binary file, run ./gopa to start the Gopa

Run ./gopa -h to get the full list of command line options

Usage of ./bin/gopa:
  -config string
        the location of config file, default: gopa.yml (default "gopa.yml")
  -cpuprofile string
        write cpu profile to this file
  -daemon
        run in background as daemon
  -debug
        enable debug
  -log string
        the log level,options:trace,debug,info,warn,error, default: info (default "info")
  -log_path string
        the log path, default: log (default "log")
  -memprofile string
        write memory profile to this file
  -pidfile string
        pidfile path (only for daemon)
  -pprof string
        enable and setup pprof/expvar service, eg: localhost:6060 , the endpoint will be: http://localhost:6060/debug/pprof/ and http://localhost:6060/debug/vars

Stop

It's safety to press ctrl+c stop the current running Gopa, Gopa will handle the rest,saving the checkpoint, you may restore the job later,the world is still in your hand.

If you are running Gopa as daemon, you may stop it like this:

 kill -QUIT `pgrep gopa`

UI

• Search Console http://127.0.0.1:9001/
• Admin Console http://127.0.0.1:9001/admin/

API

• TBD

Contribution

You are sincerely and warmly welcomed to play with this project, from UI style to core features, or just a piece of document, welcome! let's make it better.

License

Released under the Apache License, Version 2.0 .