README
¶
Generator of unique non-sequential short Ids
The package shortidenables the generation of short, fully unique,
non-sequential and by default URL friendly Ids at a rate of hundreds of thousand per second. It
guarantees uniqueness during the time period until 2050!
The package is heavily inspired by the node.js shortid library (see more detail below).
The easiest way to start generating Ids is:
fmt.Printf(shortid.Generate())
fmt.Printf(shortid.Generate())
The recommended one is to initialise and reuse a generator specific to a given worker:
sid, err := shortid.New(1, shortid.DefaultABC, 2342)
// then either:
fmt.Printf(sid.Generate())
fmt.Printf(sid.Generate())
// or:
shortid.SetDefault(sid)
// followed by:
fmt.Printf(shortid.Generate())
fmt.Printf(shortid.Generate())
Id Length
The standard Id length is 9 symbols when generated at a rate of 1 Id per millisecond, occasionally it reaches 11 (at the rate of a few thousand Ids per millisecond) and very-very rarely it can go beyond that during continuous generation at full throttle on high-performant hardware. A test generating 500k Ids at full throttle on conventional hardware generated the following Ids at the head and the tail (length > 9 is expected for this test):
-NDveu-9Q
iNove6iQ9J
NVDve6-9Q
VVDvc6i99J
NVovc6-QQy
VVoveui9QC
...
tFmGc6iQQs
KpTvcui99k
KFTGcuiQ9p
KFmGeu-Q9O
tFTvcu-QQt
tpTveu-99u
Life span
The package guarantees the generation of unique Ids with no collisions for 34 years (1/1/2016-1/1/2050) using the same worker Id within a single (although can be concurrent) application provided application restarts take longer than 1 millisecond. The package supports up to 32 workers all providing unique sequences from each other.
Implementation details
Although heavily inspired by the node.js shortid library this is not just a Go port. This implementation
- is safe to concurrency (test included);
- does not require any yearly version/epoch resets (test included);
- provides stable Id size over a the whole range of operation at the rate of 1ms (test included);
- guarantees no collisions: due to guaranteed fixed size of Ids between milliseconds and because multiple requests within the same ms lead to longer Ids with the prefix unique to the ms (tests included);
- supports 32 instead of 16 workers (test included)
The algorithm uses less randomness than the original node.js implementation, which permits to extend the life span as well as reduce and guarantee the length. In general terms, each Id has the following 3 pieces of information encoded: the millisecond since epoch (first 8 symbols, epoch: 1/1/2016), the worker Id (9th symbol), the running concurrent counter within the millisecond (only if required, spanning over all remaining symbols).
The element of randomness per symbol is 1/2 for the worker and the millisecond data and 0 for the counter. The original algorithm of the node.js library uses 1/4 throughout. Here 0 means no randomness, i.e. every value is encoded using a 64-base alphabet directly; 1/2 means one of two matching symbols of the supplied alphabet is used randomly, 1/4 one of four matching symbols. All methods accepting the parameters that govern the randomness are exported and can be used to directly implement an algorithm with e.g. more randomness, but with longer Ids and shorter life spans.
License and copyright
Copyright (c) 2016. Oleg Sklyar and teris.io. MIT license applies. All rights reserved.
Original algorithm: Copyright (c) 2015 Dylan Greene, contributors. The same MIT license applies. Many thanks to Dylan for putting together the original node.js library, which inspired this "port":
Seed computation: based on The Central Randomizer 1.3. Copyright (c) 1997 Paul Houle (houle@msc.cornell.edu)
Documentation
¶
Overview ¶
Package shortid enables the generation of short, unique, non-sequential and by default URL friendly Ids. The package is heavily inspired by the node.js https://github.com/dylang/shortid library.
Id Length ¶
The standard Id length is 9 symbols when generated at a rate of 1 Id per millisecond, occasionally it reaches 11 (at the rate of a few thousand Ids per millisecond) and very-very rarely it can go beyond that during continuous generation at full throttle on high-performant hardware. A test generating 500k Ids at full throttle on conventional hardware generated the following Ids at the head and the tail (length > 9 is expected for this test):
-NDveu-9Q iNove6iQ9J NVDve6-9Q VVDvc6i99J NVovc6-QQy VVoveui9QC ... tFmGc6iQQs KpTvcui99k KFTGcuiQ9p KFmGeu-Q9O tFTvcu-QQt tpTveu-99u
Life span ¶
The package guarantees the generation of unique Ids with zero collisions for 34 years (1/1/2016-1/1/2050) using the same worker Id within a single (although concurrent) application if application restarts take longer than 1 millisecond. The package supports up to 32 works, all providing unique sequences.
Implementation details ¶
Although heavily inspired by the node.js shortid library this is not a simple Go port. In addition it
- is safe to concurrency;
- does not require any yearly version/epoch resets;
- provides stable Id size over a long period at the rate of 1ms;
- guarantees no collisions (due to guaranteed fixed size of Ids between milliseconds and because multiple requests within the same ms lead to longer Ids with the prefix unique to the ms);
- supports 32 over 16 workers.
The algorithm uses less randomness than the original node.js implementation, which permits to extend the life span as well as reduce and guarantee the length. In general terms, each Id has the following 3 pieces of information encoded: the millisecond (first 8 symbols), the worker Id (9th symbol), running concurrent counter within the same millisecond, only if required, over all remaining symbols. The element of randomness per symbol is 1/2 for the worker and the millisecond and 0 for the counter. Here 0 means no randomness, i.e. every value is encoded using a 64-base alphabet; 1/2 means one of two matching symbols of the supplied alphabet, 1/4 one of four matching symbols. The original algorithm of the node.js module uses 1/4 throughout.
All methods accepting the parameters that govern the randomness are exported and can be used to directly implement an algorithm with e.g. more randomness, but with longer Ids and shorter life spans.
Index ¶
- Constants
- func Generate() (string, error)
- func MustGenerate() string
- func SetDefault(sid *Shortid)
- type Abc
- type Shortid
- func (sid *Shortid) Abc() Abc
- func (sid *Shortid) Epoch() time.Time
- func (sid *Shortid) Generate() (string, error)
- func (sid *Shortid) GenerateInternal(tm *time.Time, epoch time.Time) (string, error)
- func (sid *Shortid) MustGenerate() string
- func (sid *Shortid) String() string
- func (sid *Shortid) Worker() uint
Constants ¶
const DefaultABC = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_-"
DefaultABC is the default URL-friendly alphabet.
const Version = 1.1
Version defined the library version.
Variables ¶
This section is empty.
Functions ¶
func MustGenerate ¶
func MustGenerate() string
MustGenerate acts just like Generate, but panics instead of returning errors.
Types ¶
type Abc ¶
type Abc struct {
// contains filtered or unexported fields
}
Abc represents a shuffled alphabet used to generate the Ids and provides methods to encode data.
func MustNewAbc ¶
MustNewAbc acts just like NewAbc, but panics instead of returning errors.
func NewAbc ¶
NewAbc constructs a new instance of shuffled alphabet to be used for Id representation.
func (*Abc) Encode ¶
Encode encodes a given value into a slice of runes of length nsymbols. In case nsymbols==0, the length of the result is automatically computed from data. Even if fewer symbols is required to encode the data than nsymbols, all positions are used encoding 0 where required to guarantee uniqueness in case further data is added to the sequence. The value of digits [4,6] represents represents n in 2^n, which defines how much randomness flows into the algorithm: 4 -- every value can be represented by 4 symbols in the alphabet (permitting at most 16 values), 5 -- every value can be represented by 2 symbols in the alphabet (permitting at most 32 values), 6 -- every value is represented by exactly 1 symbol with no randomness (permitting 64 values).
func (*Abc) MustEncode ¶
MustEncode acts just like Encode, but panics instead of returning errors.
type Shortid ¶
type Shortid struct {
// contains filtered or unexported fields
}
Shortid type represents a short Id generator working with a given alphabet.
func GetDefault ¶
func GetDefault() *Shortid
GetDefault retrieves the default short Id generator initialised with the default alphabet, worker=0 and seed=1. The default can be overwritten using SetDefault.
func New ¶
New constructs an instance of the short Id generator for the given worker number [0,31], alphabet (64 unique symbols) and seed value (to shuffle the alphabet). The worker number should be different for multiple or distributed processes generating Ids into the same data space. The seed, on contrary, should be identical.
func (*Shortid) Epoch ¶
Epoch returns the value of epoch used as the beginning of millisecond counting (normally 2016-01-01 00:00:00 local time)
func (*Shortid) GenerateInternal ¶
GenerateInternal should only be used for testing purposes.
func (*Shortid) MustGenerate ¶
MustGenerate acts just like Generate, but panics instead of returning errors.
Source Files