replication

package
v0.7.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 23, 2023 License: MIT Imports: 12 Imported by: 0

README

osm/replication Godoc Reference

Package replication handles fetching the Minute, Hour, Day and Changeset replication and the associated state value from Planet OSM.

For example, to fetch the current Minute replication state:

num, fullState, err := replication.CurrentMinuteState(ctx)

This is the data in http://planet.osm.org/replication/minute/state.txt updated every minute.

Once you know the change number you want, fetch the change using:

change, err := replication.Minute(ctx, num)

Finding sequences numbers by timestamp

It's also possible to find the sequence number by timestamp. These calls make multiple requests for state to find the one matching the given timestamp.

MinuteStateAt(ctx context.Context, timestamp time.Time) (MinuteSeqNum, *State, error)
HourStateAt(ctx context.Context, timestamp time.Time) (HourSeqNum, *State, error)
DayStateAt(ctx context.Context, timestamp time.Time) (DaySeqNum, *State, error)
ChangesetStateAt(ctx context.Context, timestamp time.Time) (ChangesetSeqNum, *State, error)

Documentation

Index

Constants

View Source 
const BaseURL = "https://planet.osm.org"

BaseURL defines the default planet server to hit.

View Source 
const DaySeqStart = DaySeqNum(1)

DaySeqStart is the beginning of valid day sequence data.

View Source 
const HourSeqStart = HourSeqNum(11)

HourSeqStart is the beginning of valid hour sequence data. Without deep inspection it looks like 1-10 are from July 2013.

View Source 
const MinuteSeqStart = MinuteSeqNum(4)

MinuteSeqStart is the beginning of valid minutely sequence data. The few before look to be way more than a minute. A quick looks says about 75, 57, 17 for 1, 2, 3 respectively.

Variables

View Source 
var DefaultDatasource = &Datasource{
	Client: &http.Client{
		Timeout: 30 * time.Minute,
	},
}

DefaultDatasource is the Datasource used by the package level convenience functions.

Functions

func ChangesetStateAt 

func ChangesetStateAt(ctx context.Context, timestamp time.Time) (ChangesetSeqNum, *State, error)

ChangesetStateAt will return the replication state/sequence number that contains data for the given timestamp. This would be the first replication state written after the timestamp. If the timestamp is after all current replication state the most recent will be returned. The caller can check for this case using state.Before(givenTimestamp).

This call can do 20+ requests to the binary search the replication states. Use sparingly or use a mirror. Delegates to the DefaultDatasource and uses its http.Client to make the request.

func Changesets 

func Changesets(ctx context.Context, n ChangesetSeqNum) (osm.Changesets, error)

Changesets returns the complete list of changesets for the given replication sequence. Delegates to the DefaultDatasource and uses its http.Client to make the request.

func CurrentChangesetState 

func CurrentChangesetState(ctx context.Context) (ChangesetSeqNum, *State, error)

CurrentChangesetState returns the current state of the changeset replication. Delegates to the DefaultDatasource and uses its http.Client to make the request.

func CurrentDayState 

func CurrentDayState(ctx context.Context) (DaySeqNum, *State, error)

CurrentDayState returns the current state of the daily replication. Delegates to the DefaultDatasource and uses its http.Client to make the request.

func CurrentHourState 

func CurrentHourState(ctx context.Context) (HourSeqNum, *State, error)

CurrentHourState returns the current state of the hourly replication. Delegates to the DefaultDatasource and uses its http.Client to make the request.

func CurrentMinuteState 

func CurrentMinuteState(ctx context.Context) (MinuteSeqNum, *State, error)

CurrentMinuteState returns the current state of the minutely replication. Delegates to the DefaultDatasource and uses its http.Client to make the request.

func Day 

func Day(ctx context.Context, n DaySeqNum) (*osm.Change, error)

Day returns the change diff for a given day. Delegates to the DefaultDatasource and uses its http.Client to make the request.

func DayStateAt 

func DayStateAt(ctx context.Context, timestamp time.Time) (DaySeqNum, *State, error)

DayStateAt will return the replication state/sequence number that contains data for the given timestamp. This would be the first replication state written after the timestamp. If the timestamp is after all current replication state the most recent will be returned. The caller can check for this case using state.Before(givenTimestamp).

This call can do 20+ requests to the binary search the replication states. Use sparingly or use a mirror. Delegates to the DefaultDatasource and uses its http.Client to make the request.

func Hour 

func Hour(ctx context.Context, n HourSeqNum) (*osm.Change, error)

Hour returns the change diff for a given hour. Delegates to the DefaultDatasource and uses its http.Client to make the request.

func HourStateAt 

func HourStateAt(ctx context.Context, timestamp time.Time) (HourSeqNum, *State, error)

HourStateAt will return the replication state/sequence number that contains data for the given timestamp. This would be the first replication state written after the timestamp. If the timestamp is after all current replication state the most recent will be returned. The caller can check for this case using state.Before(givenTimestamp).

This call can do 20+ requests to the binary search the replication states. Use sparingly or use a mirror. Delegates to the DefaultDatasource and uses its http.Client to make the request.

func Minute 

func Minute(ctx context.Context, n MinuteSeqNum) (*osm.Change, error)

Minute returns the change diff for a given minute. Delegates to the DefaultDatasource and uses its http.Client to make the request.

func MinuteStateAt 

func MinuteStateAt(ctx context.Context, timestamp time.Time) (MinuteSeqNum, *State, error)

MinuteStateAt will return the replication state/sequence number that contains data for the given timestamp. This would be the first replication state written after the timestamp. If the timestamp is after all current replication state the most recent will be returned. The caller can check for this case using state.Before(givenTimestamp).

This call can do 20+ requests to the binary search the replication states. Use sparingly or use a mirror. Delegates to the DefaultDatasource and uses its http.Client to make the request.

func NotFound 

func NotFound(err error) bool

NotFound will return try if the error from one of the methods was due to the file not found on the remote host.

Types

type ChangesetSeqNum 

type ChangesetSeqNum uint64

ChangesetSeqNum indicates the sequence of the changeset replication found here: https://planet.osm.org/replication/changesets/

func (ChangesetSeqNum) Dir 

func (n ChangesetSeqNum) Dir() string

Dir returns the directory of this data on planet osm.

func (ChangesetSeqNum) String 

func (n ChangesetSeqNum) String() string

String returns 'changeset/%d'.

func (ChangesetSeqNum) Uint64 

func (n ChangesetSeqNum) Uint64() uint64

Uint64 returns the seq num as a uint64 type.

type Datasource 

type Datasource struct {
	BaseURL string // will use package level BaseURL if empty
	Client  *http.Client
}

Datasource defines context around replication data requests.

func NewDatasource 

func NewDatasource(client *http.Client) *Datasource

NewDatasource creates a Datasource using the given client.

func (*Datasource) ChangesetState 

func (ds *Datasource) ChangesetState(ctx context.Context, n ChangesetSeqNum) (*State, error)

ChangesetState returns the state for the given changeset replication. There are no state files before 2007990. In that case a 404 error is returned.

func (*Datasource) ChangesetStateAt 

func (ds *Datasource) ChangesetStateAt(ctx context.Context, timestamp time.Time) (ChangesetSeqNum, *State, error)

ChangesetStateAt will return the replication state/sequence number that contains data for the given timestamp. This would be the first replication state written after the timestamp. If the timestamp is after all current replication state the most recent will be returned. The caller can check for this case using state.Before(givenTimestamp).

This call can do 20+ requests to the binary search the replication states. Use sparingly or use a mirror.

func (*Datasource) Changesets 

func (ds *Datasource) Changesets(ctx context.Context, n ChangesetSeqNum) (osm.Changesets, error)

Changesets returns the complete list of changesets in for the given replication sequence.

func (*Datasource) CurrentChangesetState 

func (ds *Datasource) CurrentChangesetState(ctx context.Context) (ChangesetSeqNum, *State, error)

CurrentChangesetState returns the current state of the changeset replication.

func (*Datasource) CurrentDayState 

func (ds *Datasource) CurrentDayState(ctx context.Context) (DaySeqNum, *State, error)

CurrentDayState returns the current state of the daily replication.

func (*Datasource) CurrentHourState 

func (ds *Datasource) CurrentHourState(ctx context.Context) (HourSeqNum, *State, error)

CurrentHourState returns the current state of the hourly replication.

func (*Datasource) CurrentMinuteState 

func (ds *Datasource) CurrentMinuteState(ctx context.Context) (MinuteSeqNum, *State, error)

CurrentMinuteState returns the current state of the minutely replication.

func (*Datasource) Day 

func (ds *Datasource) Day(ctx context.Context, n DaySeqNum) (*osm.Change, error)

Day returns the change diff for a given day.

func (*Datasource) DayState 

func (ds *Datasource) DayState(ctx context.Context, n DaySeqNum) (*State, error)

DayState returns the state of the given daily replication.

func (*Datasource) DayStateAt 

func (ds *Datasource) DayStateAt(ctx context.Context, timestamp time.Time) (DaySeqNum, *State, error)

DayStateAt will return the replication state/sequence number that contains data for the given timestamp. This would be the first replication state written after the timestamp. If the timestamp is after all current replication state the most recent will be returned. The caller can check for this case using state.Before(givenTimestamp).

This call can do 20+ requests to the binary search the replication states. Use sparingly or use a mirror.

func (*Datasource) Hour 

func (ds *Datasource) Hour(ctx context.Context, n HourSeqNum) (*osm.Change, error)

Hour returns the change diff for a given hour.

func (*Datasource) HourState 

func (ds *Datasource) HourState(ctx context.Context, n HourSeqNum) (*State, error)

HourState returns the state of the given hourly replication.

func (*Datasource) HourStateAt 

func (ds *Datasource) HourStateAt(ctx context.Context, timestamp time.Time) (HourSeqNum, *State, error)

HourStateAt will return the replication state/sequence number that contains data for the given timestamp. This would be the first replication state written after the timestamp. If the timestamp is after all current replication state the most recent will be returned. The caller can check for this case using state.Before(givenTimestamp).

This call can do 20+ requests to the binary search the replication states. Use sparingly or use a mirror.

func (*Datasource) Minute 

func (ds *Datasource) Minute(ctx context.Context, n MinuteSeqNum) (*osm.Change, error)

Minute returns the change diff for a given minute.

func (*Datasource) MinuteState 

func (ds *Datasource) MinuteState(ctx context.Context, n MinuteSeqNum) (*State, error)

MinuteState returns the state of the given minutely replication.

func (*Datasource) MinuteStateAt 

func (ds *Datasource) MinuteStateAt(ctx context.Context, timestamp time.Time) (MinuteSeqNum, *State, error)

MinuteStateAt will return the replication state/sequence number that contains data for the given timestamp. This would be the first replication state written after the timestamp. If the timestamp is after all current replication state the most recent will be returned. The caller can check for this case using state.Before(givenTimestamp).

This call can do 20+ requests to the binary search the replication states. Use sparingly or use a mirror.

type DaySeqNum 

type DaySeqNum uint64

DaySeqNum indicates the sequence of the daily diff replication found here: http://planet.osm.org/replication/day

func (DaySeqNum) Dir 

func (n DaySeqNum) Dir() string

Dir returns the directory of this data on planet osm.

func (DaySeqNum) String 

func (n DaySeqNum) String() string

String returns 'day/%d'.

func (DaySeqNum) Uint64 

func (n DaySeqNum) Uint64() uint64

Uint64 returns the seq num as a uint64 type.

type HourSeqNum 

type HourSeqNum uint64

HourSeqNum indicates the sequence of the hourly diff replication found here: http://planet.osm.org/replication/hour

func (HourSeqNum) Dir 

func (n HourSeqNum) Dir() string

Dir returns the directory of this data on planet osm.

func (HourSeqNum) String 

func (n HourSeqNum) String() string

String returns 'hour/%d'.

func (HourSeqNum) Uint64 

func (n HourSeqNum) Uint64() uint64

Uint64 returns the seq num as a uint64 type.

type MinuteSeqNum 

type MinuteSeqNum uint64

MinuteSeqNum indicates the sequence of the minutely diff replication found here: http://planet.osm.org/replication/minute

func (MinuteSeqNum) Dir 

func (n MinuteSeqNum) Dir() string

Dir returns the directory of this data on planet osm.

func (MinuteSeqNum) String 

func (n MinuteSeqNum) String() string

String returns 'minute/%d'.

func (MinuteSeqNum) Uint64 

func (n MinuteSeqNum) Uint64() uint64

Uint64 returns the seq num as a uint64 type.

type SeqNum 

type SeqNum interface {
	fmt.Stringer
	Dir() string
	Uint64() uint64
	// contains filtered or unexported methods
}

SeqNum is an interface type that includes MinuteSeqNum, HourSeqNum and DaySeqNum. This is an experiment to implement a sum type, a type that can be one of several things only.

type State 

type State struct {
	SeqNum        uint64    `json:"seq_num"`
	Timestamp     time.Time `json:"timestamp"`
	TxnMax        int       `json:"txn_max,omitempty"`
	TxnMaxQueried int       `json:"txn_max_queries,omitempty"`
}

State returns information about the current replication state.

func ChangesetState 

func ChangesetState(ctx context.Context, n ChangesetSeqNum) (*State, error)

ChangesetState returns the state for the given changeset replication. There are no state files before 2007990. In that case a 404 error is returned. Delegates to the DefaultDatasource and uses its http.Client to make the request.

func DayState 

func DayState(ctx context.Context, n DaySeqNum) (*State, error)

DayState returns the state of the given daily replication. Delegates to the DefaultDatasource and uses its http.Client to make the request.

func HourState 

func HourState(ctx context.Context, n HourSeqNum) (*State, error)

HourState returns the state of the given hourly replication. Delegates to the DefaultDatasource and uses its http.Client to make the request.

func MinuteState 

func MinuteState(ctx context.Context, n MinuteSeqNum) (*State, error)

MinuteState returns the state of the given minutely replication. Delegates to the DefaultDatasource and uses its http.Client to make the request.

type UnexpectedStatusCodeError 

type UnexpectedStatusCodeError struct {
	Code int
	URL  string
}

UnexpectedStatusCodeError is return for a non 200 or 404 status code.

func (*UnexpectedStatusCodeError) Error 

func (e *UnexpectedStatusCodeError) Error() string

Error returns an error message with some information.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.