wof

package module
v0.0.16 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 19, 2024 License: BSD-3-Clause Imports: 4 Imported by: 0

README

wof-cli

Command-line tool for common Who's On First operations.

Tools

$> make cli
go build -mod vendor -ldflags="-s -w" -o bin/wof cmd/wof/main.go
wof
$> ./bin/wof -h
Usage: wof [CMD] [OPTIONS]
Valid commands are:
* emit
* export
* fmt
* geometry
* open
* pip
* property
* show
* uri
* validate

Important: The inputs and outputs for the wof tool have not been finalized yet, notably about how files are read and written if updated. You should expect change in the short-term.

wof emit
$> ./bin/wof emit -h
Emit one or more Who's On First records.
Usage:
	 ./bin/wof emit [options] path(N) path(N)
  -as-spr
    	Emit Who's On First records formatted as Standard Place Response (SPR) records.
  -as-spr-geojson
    	Emit Who's On First records as GeoJSON records where the 'properties' element is replaced by a Standard Place Response (SPR) representation of the record.
  -include-alt-geoms
    	Emit alternate geometry records. (default true)
  -iterator-uri string
    	A valid whosonfirst/go-whosonfirst-iterate/v2/emitter URI. If URI is "-" then this flag will be assigned a value of "file://" whose input will be the expanded URIs derived from additional arguments. (default "repo://")
  -query value
    	One or more {PATH}={REGEXP} parameters for filtering records.
  -query-mode string
    	Specify how query filtering should be evaluated. Valid modes are: ALL, ANY (default "ALL")
  -writer-uri string
    	A valid whosonfirst/go-writer.Writer URI. (default "jsonl://?writer=stdout://")

For example, emitting records as SPR results:

$> ./bin/wof emit -as-spr -writer-uri 'jsonl://?writer=stdout://' /usr/local/data/sfomuseum-data-maps/ 

{"edtf:cessation":"1985~","edtf:inception":"1985~","mz:is_ceased":1,"mz:is_current":0,"mz:is_deprecated":-1,"mz:is_superseded":0,"mz:is_superseding":0,"mz:latitude":37.616459,"mz:longitude":-122.386272,"mz:max_latitude":37.63100646804649,"mz:max_longitude":-122.37094362769881,"mz:min_latitude":37.60096637420677,"mz:min_longitude":-122.40407820844655,"mz:uri":"https://data.whosonfirst.org/136/039/131/3/1360391313.geojson","wof:belongsto":[],"wof:country":"US","wof:id":1360391313,"wof:lastmodified":1716594274,"wof:name":"SFO (1985)","wof:parent_id":-4,"wof:path":"136/039/131/3/1360391313.geojson","wof:placetype":"custom","wof:repo":"sfomuseum-data-maps","wof:superseded_by":[],"wof:supersedes":[]}
...and so on

Or with query filtering:

$> ./bin/wof emit -as-spr -query 'properties.wof:name=SFO \(2023\)' /usr/local/data/sfomuseum-data-maps/

{"edtf:cessation":"","edtf:inception":"2023-07~","mz:is_ceased":-1,"mz:is_current":-1,"mz:is_deprecated":-1,"mz:is_superseded":0,"mz:is_superseding":0,"mz:latitude":37.621284127293116,"mz:longitude":-122.38285759138246,"mz:max_latitude":37.642285759714994,"mz:max_longitude":-122.34578162574567,"mz:min_latitude":37.60153229886917,"mz:min_longitude":-122.40810153962025,"mz:uri":"https://data.whosonfirst.org/188/030/951/9/1880309519.geojson","wof:belongsto":[102527513,102191575,85633793,102087579,85922583,554784711,85688637,102085387],"wof:country":"US","wof:id":1880309519,"wof:lastmodified":1716594274,"wof:name":"SFO (2023)","wof:parent_id":-1,"wof:path":"188/030/951/9/1880309519.geojson","wof:placetype":"custom","wof:repo":"sfomuseum-data-maps","wof:superseded_by":[],"wof:supersedes":[]}

Or emitting records as FeatureCollection of GeoJSON-formatted SPR results (where the original geometry is preserved by the properties hash is replaced by that record's SPR) and piping the result to ogr2ogr:

$> ./bin/wof emit -as-spr -as-spr-geojson \
	-writer-uri 'featurecollection://?writer=stdout://' \
	/usr/local/data/sfomuseum-data-flights-2024-05 \
	
| ogr2ogr -f parquet flights2.parquet /vsistdin/

Or, iterating over a custom list of files:

$> wof emit -iterator-uri - -writer-uri 'featurecollection://?writer=stdout://' 1914563993 1914564157 1914564489 1914564345 | json_pp | grep 'wof:name'
2024/07/09 13:57:07 INFO time to index paths (4) 797.084µs
            "wof:name" : "Terminal 2",
            "wof:name" : "Terminal 1",
            "wof:name" : "International Terminal",
            "wof:name" : "Terminal 3",
wof export
$> ./bin/wof export -h
"Export-ify" one or more Who's On First records.
Usage:
	 ./bin/wof path(N) path(N)
  -stdout
    	Boolean flag signaling that updated records should be written to STDOUT. If false input files will be overwritten. (default true)
wof fmt
$> ./bin/wof fmt -h
Format one or more GeoJSON files according to Who's On First conventions.
Usage:
	 ./bin/wof path(N) path(N)
  -stdout
    	Boolean flag signaling that updated records should be written to STDOUT. If false input files will be overwritten. (default true)

For example:

$> ./bin/wof fmt ~/Desktop/test.geojson
{
  "id": 6102,
  "type": "Feature",
  "properties": {
    "ACCESS_": "PUBLIC",
    "USAGE": "STAIR"
  },
  "bbox": null,
  "geometry": {"coordinates":[[[-122.39515729496134,37.62304022215619,40],[-122.39511251529856,37.62305107289922,40],[-122.39508811697748,37.623056984354164,40],[-122.39509251970546,37.62306848116357,40],[-122.39510034575342,37.62308891418595,40],[-122.39516943471905,37.62307271147819,40],[-122.39515729496134,37.62304022215619,40]]],"type":"Polygon"}
}
wof geometry
$> ./bin/wof geometry -h
Display or update the geometry for one or more Who's On First records.
Usage:
	 ./bin/wof geometry [options] path(N) path(N)
  -action string
    	Valid options are: show, update. (default "show")
  -format string
    	The format of the source geometry (if -action update). Valid options are: geojson. (default "geojson")
  -source string
    	The data source from which to derive a geometry (if -action update). Valid options are: the path to a local file.
  -stdout
    	Boolean flag signaling that updated records should be written to STDOUT. If false input files will be overwritten.

For example:

$> wof geometry -action update -source /usr/local/data/t1.geojson /usr/local/data/sfomuseum-data-architecture/data/191/456/434/5/1914564345.geojson

Or:

$> wof geometry 1914564345
{"coordinates":[[[[-122.385635,37.611836],[-122.385693,37.611861],[-122.385712,37.611868],[-122.385789,37.611901],[-122.385836,37.61192],[-122.385915,37.611954],[-122.385906,37.611969],[-122.385902,37.611974],[-122.385902,37.611974],[-122.385867,37.612026],[-122.385867,37.612026],[-122.385853,37.612048],[-122.385846,37.612046],[-122.385754,37.612007],[-122.385572,37.61193],[-122.385446,37.61212] ... and so on
wof open
$> ./bin/wof open -h
Open one or more Who's On First documents in a custom editor.
Usage:
	 ./bin/wof path(N) path(N)
  -editor string
    	The editor to use for opening Who's On First records. If empty the value of the EDITOR environment variable will be used.
wof pip
$> ./bin/wof pip -h
Perform point-in-polygon and wof:hierarchy update operations on one or more Who's On First records.
Usage:
	 ./bin/wof path(N) path(N)
  -export
    	"Export-ify" each record after point-in-polygon operations are complete.
  -mapshaper-client-uri string
    	Optional URI to a sfomuseum/go-sfomuseum-mapshaper server instance used to derive point-in-polygon centroids. If absent then the centroid used to perform point-in-polygon operations will be determined using internal heuristics.
  -placetype string
    	Assign this value as the "wof:placetype" property before performing point-in-polygon operations.
  -spatial-database-uri string
    	A valid whosonfirst/go-whosonfirst-spatial/database URI. By default 'pmtiles://' and 'sqlite://' spatial database URIs are supported.
  -stdout
    	Boolean flag signaling that updated records should be written to STDOUT. (default true)

For example, imagine a shell script like this (in order to account for boring URL-escaping issues):

#!/bin/sh

TILES_URI=file:///usr/local/data
TILES_DATABASE=architecture
TILES_ZOOM=12
TILES_LAYER=architecture

ENC_TILES_URI=`urlescape ${TILES_URI}`

./bin/wof pip \
    -spatial-database-uri "pmtiles://?tiles=${ENC_TILES_URI}&database=${TILES_DATABASE}&zoom=${TILES_ZOOM}&layer=${TILES_LAYER}&enable-cache=true" \
    -mapshaper-client-uri http://localhost:8080 \
    -placetype venue \
    test.geojson

Running this command would yield something like this:

2024/05/20 15:34:27 INFO fetching architecture 0-16384
2024/05/20 15:34:27 INFO fetched architecture 0-16384
2024/05/20 15:34:27 INFO fetching architecture 127-89
2024/05/20 15:34:27 INFO fetched architecture 127-89
2024/05/20 15:34:27 INFO Time to create database path=/architecture/12/655/1585.mvt "spatial database uri"="sqlite://?dsn=modernc://mem" "count features"=628 time=183.835708ms
{
  "id": 6102,
  "type": "Feature",
  "properties": {
    "USAGE": "STAIR"
  ,"wof:placetype":"venue","wof:parent_id":102527513,"wof:country":"US","wof:hierarchy":[{"campus_id":102527513,"continent_id":102191575,"country_id":85633793,"county_id":102087579,"locality_id":85922583,"postalcode_id":554784711,"region_id":85688637},{"campus_id":102527513,"continent_id":102191575,"country_id":85633793,"county_id":102085387,"region_id":85688637}]},
  "bbox": null,
  "geometry": {"coordinates":[[[-122.39515729496134,37.62304022215619,40],[-122.39511251529856,37.62305107289922,40],[-122.39508811697748,37.623056984354164,40],[-122.39509251970546,37.62306848116357,40],[-122.39510034575342,37.62308891418595,40],[-122.39516943471905,37.62307271147819,40],[-122.39515729496134,37.62304022215619,40]]],"type":"Polygon"}
}

There are a few things to note:

  • The -placetype flag is a convenience to facilitate point-in-polygon operations without having to first update an input record.
  • By default the pip command neither "formats" or "exports" results. Although there is an -export flag to enable both it is your responsibility to ensure that input documents have all the necessary properties (for example "wof:name").
  • The pip command does NOT yet implement the logic of the py-mapzen-whosonfirst-hierarchy library. There is an open issue for this.
wof property

Print one or more properties for one or more Who's On First IDs.

$> wof property -h
Print one or more properties for one or more Who's On First IDs.
Usage:
	 wof path(N) path(N)
  -path value
    	One or more valid tidwall/gjson paths to extract from each document

For example:

$> wof property -path properties.wof:name 1796903597 1796889561 1796889543 1796889557 1796903629 1796889563 1796935715 1796935615
AirTrain Gargage G / BART Red Line
AirTrain Long-Term Parking Blue Line (Outbound)
AirTrain Garage G / BART Blue Line
AirTrain Westfield Road Station (Inbound)
Grand Hyatt Hotel Reception
Rental Car Center
International Terminal Main Hall Departures Door 2
San Francisco International Airport BART Station Platform

Eventually, this command will add functionality to export results for multiple IDs with multiple properties as CSV data.

wof show
$> ./bin/wof show -h
Display one or more Who's On First documents on a map.
Usage:
	 ./bin/wof path(N) path(N)
  -port int
    	The port number to listen for requests on (on localhost). If 0 then a random port number will be chosen.

For example:

$> ./bin/wof show montreal.geojson 
Records are viewable at http://localhost:63675

This should automatically open a new window in your default browser like this:

As of this writing there is minimal styling and little to no interactivity. That may (or may not) change. Right now this tool is principally just to be able to look at one or more Who's On First features on a map.

wof uri
$> ./bin/wof uri -h
Print the nested URI for one or more Who's On First IDs.
Usage:
	 ./bin/wof path(N) path(N)
  -prefix string
    	An optional prefix to append to the final URI.

For example:

$> ./bin/wof uri 1914650585
191/465/058/5/1914650585.geojson

$> cat `wof uri -prefix data 1914650737` | jq '.properties["wof:name"]'
"1H Student Art North"
wof validate
$> ./bin/wof validate -h
Validate one or more Who's On First documents.
Usage:
	 ./bin/wof path(N) path(N)

For example:

$> ./bin/wof validate test.geojson
2024/05/20 15:19:34 Failed to run 'validate' command, Failed to validate body for 'test.geojson', Failed to validate name, Failed to derive wof:name from body, Missing wof:name property

Or:

$> curl -s https://data.whosonfirst.org/102527513 | ./bin/wof validate -
2024/05/20 15:47:34 Failed to run 'validate' command, Failed to validate body for '-', Failed to validate EDTF, Failed to validate EDTF cessation date, Unrecognized EDTF string 'open' (Invalid or unsupported EDTF string)

If I download the record for SFO and manually change the edtf:cessation property from "open" to ".." and then run the tool again, everything validates.

$> ./bin/wof validate sfo.geojson

Note the default behaviour for successfull validation is to do nothing. That might change? Maybe?

Paths and URIs

The default behaviour for the wof command is to assume that all the URIs it is passed are paths on the local computer.

That being said all the tool also "expand" each path using the uris.ExpandURIsWithCallback method which allows for more sophisticated behaviour in the future.

Expansions

Currently there is only two supported "expansions":

  1. Treating bare numbers as Who's On First IDs, resolving them to their relative path and looking for that file in a parent "data" directory inside the current working directory. Basically it's a shortcut for resolving a Who's On First record to its GeoJSON representation inside a whosonfirst-data repository.
  2. URIs prefixed with repo:// will be treated as a whosonfirst/go-whosonfirst-iterate/v2 "repo" emitter URI and all the files in that repository will be processed.

Eventually there may be other "expansions" most notably support for the Go ./... syntax to process all the Who's On First records in the current working directory.

See also

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Commands 

func Commands() []string

Commands returns the list of command that have been registered.

func RegisterCommand 

func RegisterCommand(ctx context.Context, scheme string, init_func CommandInitializationFunc) error

RegisterCommand registers 'scheme' as a key pointing to 'init_func' in an internal lookup table used to create new `Command` instances by the `NewCommand` method.

Types

type Command 

type Command interface {
	Run(context.Context, []string) error
}

func NewCommand 

func NewCommand(ctx context.Context, cmd string) (Command, error)

NewCommand returns a new `Command` instance configured by 'cmd'. The value of 'uri' is parsed as a `url.URL` and its scheme is used as the key for a corresponding `CommandInitializationFunc` function used to instantiate the new `Command`. It is assumed that the scheme (and initialization function) have been registered by the `RegisterCommand` method.

type CommandInitializationFunc 

type CommandInitializationFunc func(ctx context.Context, cmd string) (Command, error)

CommandInitializationFunc is a function defined by individual command package and used to create an instance of that command

Source Files

View all Source files

Directories

Path Synopsis
app
wof
cmd
wof command
www

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.