density

README

High-density Point Maps

Render millions of points on a map.

Demo Site

This demo shows 77 million taxi pickups in NYC - from January to June 2015.

https://www.michaelfogleman.com/static/density/

Dependencies

• Go
• Cassandra

Download

go get github.com/fogleman/density

Loading Data

Cassandra is used to store large amounts of data. Data is clustered by (zoom, x, y) so that all of the points inside of a tile can quickly be fetched for rendering, even faster than PostGIS with an index.

First, create a new keyspace and table to house the data.

create keyspace density
    with replication = {
        'class': 'SimpleStrategy',
        'replication_factor': 1
    }
    and durable_writes = false;

create table points (
    zoom int,
    x int,
    y int,
    lat double,
    lng double,
    primary key ((zoom, x, y), lat, lng)
);

Next, load points into the database from a CSV file using the loader script.

go run loader/main.go < input.csv

Several command line options are available:

Flag	Default	Description
-keyspace	density	Cassandra keyspace to load into
-table	points	Cassandra table to load into
-lat	0	CSV column index of latitude values
-lng	1	CSV column index of longitude values
-zoom	18	Zoom level to use for binning points

Just run the loader whenever you need to insert more data.

Serving Tiles

Once the data is loaded into Cassandra, the tile server can be run for rendering tiles on the fly.

go run server/main.go

Flag	Default	Description
-keyspace	density	Cassandra keyspace to load from
-table	points	Cassandra table to load from
-zoom	18	Zoom level that was used for binning points
-port	5000	Tile server port number
-cache	cache	Directory for caching tile images

Serving Maps

A simple Leaflet map is provided to display a base map with the point tiles on top in a separate layer.

cd web
python -m SimpleHTTPServer

Then visit http://localhost:8000/ in your browser!

TODO

• tile rendering options
• multiple layers

Documentation

Index

• Constants
• func GetImage(url string) (image.Image, error)
• func Stitch(urlTemplate string, lat, lng float64, zoom, w, h int) (*image.NRGBA, error)
• func TileFloatXY(zoom int, lat, lng float64) (x, y float64)
• func TileLatLng(zoom, x, y int) (lat, lng float64)
• func TileXY(zoom int, lat, lng float64) (x, y int)
• type IntPoint
• type Kernel
  • func NewKernel(n int) Kernel
• type KernelItem
• type Renderer
  • func NewRenderer(host, keyspace, table string, baseZoom int) *Renderer
  • func (r *Renderer) Render(zoom, x, y int) (image.Image, bool)
• type Tile
  • func NewTile(zoom, x, y int) *Tile
  • func (tile *Tile) Add(lat, lng float64)
  • func (tile *Tile) Render(kernel Kernel, scale float64) (image.Image, bool)
• type TileLayer
  • func NewTileLayer(urlTemplate string) *TileLayer
  • func (layer *TileLayer) GetTile(z, x, y int) (image.Image, error)
  • func (layer *TileLayer) GetTiles(lat, lng float64, zoom, w, h int) (*image.NRGBA, error)

Constants

const TileSize = 256

Functions

func GetImage

func GetImage(url string) (image.Image, error)

func Stitch

func Stitch(urlTemplate string, lat, lng float64, zoom, w, h int) (*image.NRGBA, error)

func TileFloatXY

func TileFloatXY(zoom int, lat, lng float64) (x, y float64)

func TileLatLng

func TileLatLng(zoom, x, y int) (lat, lng float64)

func TileXY

func TileXY(zoom int, lat, lng float64) (x, y int)

Types

type IntPoint

type IntPoint struct {
	X, Y int
}

type Kernel

type Kernel []KernelItem

func NewKernel

func NewKernel(n int) Kernel

type KernelItem

type KernelItem struct {
	Dx, Dy int
	Weight float64
}

type Renderer

type Renderer struct {
	Cluster  *gocql.ClusterConfig
	Query    string
	BaseZoom int
}

func NewRenderer

func NewRenderer(host, keyspace, table string, baseZoom int) *Renderer

func (*Renderer) Render

func (r *Renderer) Render(zoom, x, y int) (image.Image, bool)

type Tile

type Tile struct {
	Zoom, X, Y int
	Grid       map[IntPoint]float64
	Points     int
}

func NewTile

func NewTile(zoom, x, y int) *Tile

func (*Tile) Add

func (tile *Tile) Add(lat, lng float64)

func (*Tile) Render

func (tile *Tile) Render(kernel Kernel, scale float64) (image.Image, bool)

type TileLayer

type TileLayer struct {
	URLTemplate string
}

func NewTileLayer

func NewTileLayer(urlTemplate string) *TileLayer

func (*TileLayer) GetTile

func (layer *TileLayer) GetTile(z, x, y int) (image.Image, error)

func (*TileLayer) GetTiles

func (layer *TileLayer) GetTiles(lat, lng float64, zoom, w, h int) (*image.NRGBA, error)