velobike-parser

README

Velobike Stations Parser

Velobike Parser parses data from velobike.ru stations map every minute.
Parsed data contains information about stations (like address, position, etc.) and their states (total and available places).

Usage

First of all, you need to set up environment variables

• DB_DIALECT: DBMS — sqlite3 or postgres
• DB_URI:
  • SQLite example: /data/velobike.db
  • PostgreSQL example:
    host=postgres port=5432 user=velobike password=velobike dbname=velobike sslmode=disable
• BACKUP_DIR: ./data — path to backup storage directory.

For docker-compose specific variables see Docker Compose section and the docker-compose.yml file.

Parsing mode

In parsing mode parser starts parsing data and saves it into the database: velobike-parser.

Import mode

If you need to restore backed up data, use import mode. \

$ velobike-parser import /path/to/backup.json

Export mode

If you need to export data, you have two options how to do that: you can backup all data (be careful) or data between two dates (inclusive).

$ velobike-parser export -from "2020-04-19 19:48 MSK" -to "2020-04-19 19:48 MSK" > backup.json
$ # or 
$ velobike-parser export -all > backup.json

Installation and running

Building a binary

Go 1.14 required. Clone velobike-parser repository and run:

$ go build -o main .
$ DB_URI="./data/velobike.db" DB_DIALECT=sqlite3 ./main
$ # or
$ BACKUP_DIR="./data" DB_URI="./data/velobike.db" DB_DIALECT=postgres ./main

Docker

$ docker build -t velobike-parser .
$ docker run -d \
    --name=velobike-parser \
    -e TZ="Europe/Moscow" \
    -e DB_URI="/data/velobike.db" \
    -v $(pwd)/data:/data \
    --restart=unless-stopped \
    velobike-parser

Docker Compose

Just run docker-compose up -d to start parser.
By default, it uses PostgreSQL for data storage and ./data directory as backup storage.

If you need to change volume, just set POSTGRES_VOLUME environment variable. By default, it creates a volume named postgres.

$ POSTGRES_VOLUME="./data/postgres" docker-compose up -d

You can connect to PostgreSQL database when containers are up.

Credentials:

• Host: localhost
• Port: 54321
• User: velobike
• Password: velobike
• Database: velobike

By default, docker-compose creates a volume postgres for PostgreSQL data. You can also explicitly set the volume or directory, e.g.:

$ POSTGRES_VOLUME="./data/postgres" docker-compose up

See docker-compose.yml for extra environment variables.

Notes

The parser doesn't support SQLite in-memory databases. SQLite databases do not support foreign key state → station.

---

Developed by Sergei Kolesnikov