tzf

package module
v0.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 24, 2022 License: MIT Imports: 14 Imported by: 0

README

TZF: a fast timezone finder for Go. Go Reference

NOTE: This package is also available in:

Quick Start

Go
// Use about 150MB memory for init, and 60MB after GC.
package main

import (
	"fmt"

	"github.com/deslittle/tzf"
)

func main() {
	finder, err := tzf.NewDefaultFinder()
	if err != nil {
		panic(err)
	}
	fmt.Println(finder.GetTimezoneName(116.6386, 40.0786))
}

If you need 100% accurate query result, use below to got a finder:

// Use about 900MB memory for init, and 660MB after GC.
package main

import (
	"fmt"

	"github.com/deslittle/tzf"
	tzfrel "github.com/deslittle/tzf-rel"
	"github.com/deslittle/tzf/pb"
	"google.golang.org/protobuf/proto"
)

func main() {
	input := &pb.Timezones{}

	// Full data, about 83.5MB
	dataFile := tzfrel.FullData

	if err := proto.Unmarshal(dataFile, input); err != nil {
		panic(err)
	}
	finder, _ := tzf.NewFinderFromPB(input)
	fmt.Println(finder.GetTimezoneName(116.6386, 40.0786))
}
CLI Tool
go install github.com/deslittle/tzf/cmd/tzf@latest
tzf -lng 116.3883 -lat 39.9289

Data

Original data download from https://github.com/evansiroky/timezone-boundary-builder .

Preprocessed probuf data can get from https://github.com/deslittle/tzf-rel which has Go's embed support. Those files are Protocol Buffers messages for more efficiency binary distribution like Python wheels, you can view the pb/tzinfo.proto or it's HTML format docs for the internal format info.

tzf's data pipeline can be drew as:

graph TD
    Raw[GeoJSON from evansiroky/timezone-boundary-builder]
    Full[Full: Probuf based data]
    Lite[Lite: smaller of Full data]
    Compressed[Compressed: Lite compressed via Polyline]
    Preindex[Tile based data]

    Finder[Finder: Polygon Based Finder]
    FuzzyFinder[FuzzyFinder: Tile based Finder]
    DefaultFinder[DefaultFinder: combine FuzzyFinder and Compressed Finder]

    tzfpy[tzfpy: tzf's Python binding]

    Raw --> |cmd/geojson2tzpb|Full
    Full --> |cmd/reducetzpb|Lite
    Lite --> |cmd/compresstzpb|Compressed
    Lite --> |cmd/preindextzpb|Preindex

    Full --> |tzf.NewFinderFromPB|Finder
    Lite --> |tzf.NewFinderFromPB|Finder
    Compressed --> |tzf.NewFinderFromCompressed|Finder --> |tzf.NewDefaultFinder|DefaultFinder
    Preindex --> |tzf.NewFuzzyFinderFromPB|FuzzyFinder --> |tzf.NewDefaultFinder|DefaultFinder

    DefaultFinder --> tzfpy

The full data(~80MB) could work anywhere but requires more memory usage.

The lite data(~10MB) doesn't work well in some edge places.

You can see points that results diff in this page.

If a little longer init time is acceptable, the compressed data(~5MB) which come from lite data will be more friendly for binary distribution.

The preindex data(~1.78MB) are many tiles. It's used inside the DefaultFinder, which built on FuzzyFinder, to reduce raycasting algorithm execution times.

Performance

Package tzf is designed for high performance geo queries related services like weather forecast API. And most queries could return in very limited time, averagely like 2000 nanoseconds.

Here is what have been done for performance improvements:

  1. Use preindexes to handle most queries, basically about 1000 nanoseconds
  2. Use RTree to filter candidate polygons instead of iter all polygons to reduce Ray Casting algorithm execution times
  3. Use a fine tuned Ray Casting algorithm package https://github.com/tidwall/geojson to check if polygon contain point

That's all. There is no black magics inside package tzf.

Benchmark run version https://github.com/deslittle/tzf/releases/tag/v0.10.0

goos: darwin
goarch: amd64
pkg: github.com/deslittle/tzf
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkDefaultFinder_GetTimezoneName_Random_WorldCities-16    	  443942	      2583 ns/op
BenchmarkFuzzyFinder_GetTimezoneName_Random_WorldCities-16      	 1580524	       769.3 ns/op
BenchmarkGetTimezoneName-16                                     	  184234	      6983 ns/op
BenchmarkGetTimezoneNameAtEdge-16                               	  145342	      8779 ns/op
BenchmarkGetTimezoneName_Random_WorldCities-16                  	  130540	      9247 ns/op
BenchmarkFullFinder_GetTimezoneName-16                          	  164649	      7256 ns/op
BenchmarkFullFinder_GetTimezoneNameAtEdge-16                    	  136508	      9382 ns/op
BenchmarkFullFinder_GetTimezoneName_Random_WorldCities-16       	   91137	     12385 ns/op
PASS
ok  	github.com/deslittle/tzf	19.073s

Thanks

Documentation

Overview

Package tzf is a package convert (lng,lat) to timezone.

Inspired by timezonefinder https://github.com/jannikmi/timezonefinder, fast python package for finding the timezone of any point on earth (coordinates) offline.

Index

Examples

Constants

This section is empty.

Variables

View Source 
var ErrNoTimezoneFound = errors.New("tzf: no timezone found")

Functions

func SetDropPBTZ 

func SetDropPBTZ(opt *Option)

SetDropPBTZ will make Finder not save github.com/deslittle/tzf/pb.Timezone in memory

Types

type DefaultFinder 

type DefaultFinder struct {
	// contains filtered or unexported fields
}

DefaultFinder is a finder impl combine both FuzzyFinder and Finder.

It's designed for performance first and allow some not so correct return at some area.

func NewDefaultFinder 

func NewDefaultFinder() (*DefaultFinder, error)

func (*DefaultFinder) GetTimezoneName 

func (f *DefaultFinder) GetTimezoneName(lng float64, lat float64) string
Example 
package main

import (
	"fmt"

	"github.com/deslittle/tzf"
)

func main() {
	finder, err := tzf.NewDefaultFinder()
	if err != nil {
		panic(err)
	}
	fmt.Println(finder.GetTimezoneName(116.6386, 40.0786))
}

Output:
Asia/Shanghai

func (*DefaultFinder) GetTimezoneNames 

func (f *DefaultFinder) GetTimezoneNames(lng float64, lat float64) ([]string, error)

func (*DefaultFinder) TimezoneNames 

func (f *DefaultFinder) TimezoneNames() []string

type Finder 

type Finder struct {
	// contains filtered or unexported fields
}

Finder is based on point-in-polygon search algo.

Memeory will use about 100MB if lite data and 1G if full data. Performance is very stable and very accuate.

func NewFinderFromCompressed 

func NewFinderFromCompressed(input *pb.CompressedTimezones, opts ...OptionFunc) (*Finder, error)

func NewFinderFromPB 

func NewFinderFromPB(input *pb.Timezones, opts ...OptionFunc) (*Finder, error)

func NewFinderFromRawJSON 

func NewFinderFromRawJSON(input *convert.BoundaryFile, opts ...OptionFunc) (*Finder, error)

func (*Finder) GetTimezone 

func (f *Finder) GetTimezone(lng float64, lat float64) (*pb.Timezone, error)

func (*Finder) GetTimezoneLoc 

func (f *Finder) GetTimezoneLoc(lng float64, lat float64) (*time.Location, error)
Example 
package main

import (
	"fmt"

	"github.com/deslittle/tzf"

	tzfrel "github.com/deslittle/tzf-rel"
	"github.com/deslittle/tzf/pb"
	"google.golang.org/protobuf/proto"
)

func main() {
	input := &pb.Timezones{}

	// Lite data, about 16.7MB
	dataFile := tzfrel.LiteData

	// Full data, about 83.5MB
	// dataFile := tzfrel.FullData

	if err := proto.Unmarshal(dataFile, input); err != nil {
		panic(err)
	}
	finder, _ := tzf.NewFinderFromPB(input)
	fmt.Println(finder.GetTimezoneLoc(116.6386, 40.0786))
}

Output:
Asia/Shanghai <nil>

func (*Finder) GetTimezoneName 

func (f *Finder) GetTimezoneName(lng float64, lat float64) string

GetTimezoneName will use alphabet order and return first matched result.

Example 
package main

import (
	"fmt"

	"github.com/deslittle/tzf"

	tzfrel "github.com/deslittle/tzf-rel"
	"github.com/deslittle/tzf/pb"
	"google.golang.org/protobuf/proto"
)

func main() {
	input := &pb.Timezones{}

	// Lite data, about 16.7MB
	dataFile := tzfrel.LiteData

	// Full data, about 83.5MB
	// dataFile := tzfrel.FullData

	if err := proto.Unmarshal(dataFile, input); err != nil {
		panic(err)
	}
	finder, _ := tzf.NewFinderFromPB(input)
	fmt.Println(finder.GetTimezoneName(116.6386, 40.0786))
}

Output:
Asia/Shanghai

func (*Finder) GetTimezoneNames 

func (f *Finder) GetTimezoneNames(lng float64, lat float64) ([]string, error)

func (*Finder) GetTimezoneShapeByName 

func (f *Finder) GetTimezoneShapeByName(name string) (*pb.Timezone, error)
Example 
package main

import (
	"fmt"

	"github.com/deslittle/tzf"

	tzfrel "github.com/deslittle/tzf-rel"
	"github.com/deslittle/tzf/pb"
	"google.golang.org/protobuf/proto"
)

func main() {
	input := &pb.Timezones{}

	// Lite data, about 16.7MB
	dataFile := tzfrel.LiteData

	if err := proto.Unmarshal(dataFile, input); err != nil {
		panic(err)
	}
	finder, _ := tzf.NewFinderFromPB(input)
	pbtz, err := finder.GetTimezoneShapeByName("Asia/Shanghai")
	fmt.Printf("%v %v\n", pbtz.Name, err)
}

Output:
Asia/Shanghai <nil>

func (*Finder) GetTimezoneShapeByShift 

func (f *Finder) GetTimezoneShapeByShift(shift int) ([]*pb.Timezone, error)
Example 
package main

import (
	"fmt"
	"strings"

	"sort"

	"github.com/deslittle/tzf"

	tzfrel "github.com/deslittle/tzf-rel"
	"github.com/deslittle/tzf/pb"
	"google.golang.org/protobuf/proto"
)

func main() {
	input := &pb.Timezones{}

	// Lite data, about 16.7MB
	dataFile := tzfrel.LiteData

	if err := proto.Unmarshal(dataFile, input); err != nil {
		panic(err)
	}
	finder, _ := tzf.NewFinderFromPB(input)
	pbtzs, _ := finder.GetTimezoneShapeByShift(28800)

	pbnames := make([]string, 0)
	for _, pbtz := range pbtzs {
		pbnames = append(pbnames, pbtz.Name)
	}
	sort.Strings(pbnames)

	fmt.Println(strings.Join(pbnames, ","))
}

Output:
Asia/Brunei,Asia/Choibalsan,Asia/Hong_Kong,Asia/Irkutsk,Asia/Kuala_Lumpur,Asia/Kuching,Asia/Macau,Asia/Makassar,Asia/Manila,Asia/Shanghai,Asia/Singapore,Asia/Taipei,Asia/Ulaanbaatar,Australia/Perth,Etc/GMT-8

func (*Finder) TimezoneNames 

func (f *Finder) TimezoneNames() []string

type FuzzyFinder 

type FuzzyFinder struct {
	// contains filtered or unexported fields
}

FuzzyFinder use a tile map to store timezone name. Data are made by github.com/deslittle/tzf/cmd/preindextzpb which powerd by github.com/deslittle/tzf/preindex.PreIndexTimezones.

func NewFuzzyFinderFromPB 

func NewFuzzyFinderFromPB(input *pb.PreindexTimezones) (*FuzzyFinder, error)

func (*FuzzyFinder) GetTimezoneName 

func (f *FuzzyFinder) GetTimezoneName(lng float64, lat float64) string
Example 
package main

import (
	"fmt"

	"github.com/deslittle/tzf"

	tzfrel "github.com/deslittle/tzf-rel"
	"github.com/deslittle/tzf/pb"
	"google.golang.org/protobuf/proto"
)

func main() {
	input := &pb.PreindexTimezones{}
	if err := proto.Unmarshal(tzfrel.PreindexData, input); err != nil {
		panic(err)
	}
	finder, _ := tzf.NewFuzzyFinderFromPB(input)
	fmt.Println(finder.GetTimezoneName(116.6386, 40.0786))
}

Output:
Asia/Shanghai

func (*FuzzyFinder) GetTimezoneNames 

func (f *FuzzyFinder) GetTimezoneNames(lng float64, lat float64) ([]string, error)

type Option 

type Option struct {
	DropPBTZ bool
}

type OptionFunc 

type OptionFunc = func(opt *Option)

Source Files

View all Source files

Directories

Path Synopsis
cmd
comparetzpb command
compresstzpb command
CLI tool to reduce polygon filesize
 CLI tool to reduce polygon filesize
geojson2tzpb command
CLI tool to convert GeoJSON based Timezone boundary to tzf's Probuf format.
 CLI tool to convert GeoJSON based Timezone boundary to tzf's Probuf format.
preindextzpb command
CLI tool to preindex timezone shape.
 CLI tool to preindex timezone shape.
reducetzpb command
CLI tool to reduce polygon filesize
 CLI tool to reduce polygon filesize
tzf command
tzf-cli tool for local query.
 tzf-cli tool for local query.
Package preindex
 Package preindex
Package reduce could reduce Polygon size both polygon lines and float precise.
 Package reduce could reduce Polygon size both polygon lines and float precise.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.