clone

package
v1.1.15 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 26, 2024 License: Apache-2.0, MIT Imports: 12 Imported by: 0

README

go-clone: Clone any Go data structure deeply and thoroughly

Go Go Doc Go Report Coverage Status

Package clone provides functions to deep clone any Go data. It also provides a wrapper to protect a pointer from any unexpected mutation.

For users who use Go 1.18+, it's recommended to import github.com/huandu/go-clone/generic for generic APIs and arena support.

Clone/Slowly can clone unexported fields and "no-copy" structs as well. Use this feature wisely.

Install

Use go get to install this package.

go get github.com/huandu/go-clone

Usage

Clone and Slowly

If we want to clone any Go value, use Clone.

t := &T{...}
v := clone.Clone(t).(*T)
reflect.DeepEqual(t, v) // true

For the sake of performance, Clone doesn't deal with values containing pointer cycles. If we need to clone such values, use Slowly instead.

type ListNode struct {
    Data int
    Next *ListNode
}
node1 := &ListNode{
    Data: 1,
}
node2 := &ListNode{
    Data: 2,
}
node3 := &ListNode{
    Data: 3,
}
node1.Next = node2
node2.Next = node3
node3.Next = node1

// We must use `Slowly` to clone a circular linked list.
node := Slowly(node1).(*ListNode)

for i := 0; i < 10; i++ {
    fmt.Println(node.Data)
    node = node.Next
}
Generic APIs

Starting from go1.18, Go started to support generic. With generic syntax, Clone/Slowly and other APIs can be called much cleaner like following.

import "github.com/huandu/go-clone/generic"

type MyType struct {
    Foo string
}

original := &MyType{
    Foo: "bar",
}

// The type of cloned is *MyType instead of interface{}.
cloned := Clone(original)
println(cloned.Foo) // Output: bar

It's required to update minimal Go version to 1.18 to opt-in generic syntax. It may not be a wise choice to update this package's go.mod and drop so many old Go compilers for such syntax candy. Therefore, I decide to create a new standalone package github.com/huandu/go-clone/generic to provide APIs with generic syntax.

For new users who use Go 1.18+, the generic package is preferred and recommended.

Arena support

Starting from Go1.20, arena is introduced as a new way to allocate memory. It's quite useful to improve overall performance in special scenarios. In order to clone a value with memory allocated from an arena, there are new methods ArenaClone and ArenaCloneSlowly available in github.com/huandu/go-clone/generic.

// ArenaClone recursively deep clones v to a new value in arena a.
// It works in the same way as Clone, except it allocates all memory from arena.
func ArenaClone[T any](a *arena.Arena, v T) (nv T) 

// ArenaCloneSlowly recursively deep clones v to a new value in arena a.
// It works in the same way as Slowly, except it allocates all memory from arena.
func ArenaCloneSlowly[T any](a *arena.Arena, v T) (nv T)

Due to limitations in arena API, memory of the internal data structure of map and chan is always allocated in heap by Go runtime (see this issue).

Warning: Per discussion in the arena proposal, the arena package may be changed incompatibly or removed in future. All arena related APIs in this package will be changed accordingly.

Memory allocations and the Allocator

The Allocator is designed to allocate memory when cloning. It's also used to hold all customizations, e.g. custom clone functions, scalar types and opaque pointers, etc. There is a default allocator which allocates memory from heap. Almost all public APIs in this package use this default allocator to do their job.

We can control how to allocate memory by creating a new Allocator by NewAllocator. It enables us to take full control over memory allocation when cloning. See Allocator sample code to understand how to customize an allocator.

Let's take a closer look at the NewAllocator function.

func NewAllocator(pool unsafe.Pointer, methods *AllocatorMethods) *Allocator
  • The first parameter pool is a pointer to a memory pool. It's used to allocate memory for cloning. It can be nil if we don't need a memory pool.
  • The second parameter methods is a pointer to a struct which contains all methods to allocate memory. It can be nil if we don't need to customize memory allocation.
  • The Allocator struct is allocated from the methods.New or the methods.Parent allocator or from heap.

The Parent in AllocatorMethods is used to indicate the parent of the new allocator. With this feature, we can orgnize allocators into a tree structure. All customizations, including custom clone functions, scalar types and opaque pointers, etc, are inherited from parent allocators.

There are some APIs designed for convenience.

  • We can create dedicated allocators for heap or arena by calling FromHeap() or FromArena(a *arena.Arena).
  • We can call MakeCloner(allocator) to create a helper struct with Clone and CloneSlowly methods in which the type of in and out parameters is interface{}.
Mark struct type as scalar

Some struct types can be considered as scalar.

A well-known case is time.Time. Although there is a pointer loc *time.Location inside time.Time, we always use time.Time by value in all methods. When cloning time.Time, it should be OK to return a shadow copy.

Currently, following types are marked as scalar by default.

  • time.Time
  • reflect.Value

If there is any type defined in built-in package should be considered as scalar, please open new issue to let me know. I will update the default.

If there is any custom type should be considered as scalar, call MarkAsScalar to mark it manually. See MarkAsScalar sample code for more details.

Mark pointer type as opaque

Some pointer values are used as enumerable const values.

A well-known case is elliptic.Curve. In package crypto/tls, curve type of a certificate is checked by comparing values to pre-defined curve values, e.g. elliptic.P521(). In this case, the curve values, which are pointers or structs, cannot be cloned deeply.

Currently, following types are marked as scalar by default.

  • elliptic.Curve, which is *elliptic.CurveParam or elliptic.p256Curve.
  • reflect.Type, which is *reflect.rtype defined in runtime.

If there is any pointer type defined in built-in package should be considered as opaque, please open new issue to let me know. I will update the default.

If there is any custom pointer type should be considered as opaque, call MarkAsOpaquePointer to mark it manually. See MarkAsOpaquePointer sample code for more details.

Clone "no-copy" types defined in sync and sync/atomic

There are some "no-copy" types like sync.Mutex, atomic.Value, etc. They cannot be cloned by copying all fields one by one, but we can alloc a new zero value and call methods to do proper initialization.

Currently, all "no-copy" types defined in sync and sync/atomic can be cloned properly using following strategies.

  • sync.Mutex: Cloned value is a newly allocated zero mutex.
  • sync.RWMutex: Cloned value is a newly allocated zero mutex.
  • sync.WaitGroup: Cloned value is a newly allocated zero wait group.
  • sync.Cond: Cloned value is a cond with a newly allocated zero lock.
  • sync.Pool: Cloned value is an empty pool with the same New function.
  • sync.Map: Cloned value is a sync map with cloned key/value pairs.
  • sync.Once: Cloned value is a once type with the same done flag.
  • atomic.Value/atomic.Bool/atomic.Int32/atomic.Int64/atomic.Uint32/atomic.Uint64/atomic.Uintptr: Cloned value is a new atomic value with the same value.

If there is any type defined in built-in package should be considered as "no-copy" types, please open new issue to let me know. I will update the default.

Set custom clone functions

If default clone strategy doesn't work for a struct type, we can call SetCustomFunc to register a custom clone function.

SetCustomFunc(reflect.TypeOf(MyType{}), func(allocator *Allocator, old, new reflect.Value) {
    // Customized logic to copy the old to the new.
    // The old's type is MyType.
    // The new is a zero value of MyType and new.CanAddr() always returns true.
})

We can use allocator to clone any value or allocate new memory. It's allowed to call allocator.Clone or allocator.CloneSlowly on old to clone its struct fields in depth without worrying about dead loop.

See SetCustomFunc sample code for more details.

Clone atomic.Pointer[T]

As there is no way to predefine a custom clone function for generic type atomic.Pointer[T], cloning such atomic type is not supported by default. If we want to support it, we need to register a custom clone function manually.

Suppose we instantiate atomic.Pointer[T] with type MyType1 and MyType2 in a project, and then we can register custom clone functions like following.

import "github.com/huandu/go-clone/generic"

func init() {
    // Register all instantiated atomic.Pointer[T] types in this project.
    clone.RegisterAtomicPointer[MyType1]()
    clone.RegisterAtomicPointer[MyType2]()
}
Wrap, Unwrap and Undo

Package clone provides Wrap/Unwrap functions to protect a pointer value from any unexpected mutation. It's useful when we want to protect a variable which should be immutable by design, e.g. global config, the value stored in context, the value sent to a chan, etc.

// Suppose we have a type T defined as following.
//     type T struct {
//         Foo int
//     }
v := &T{
    Foo: 123,
}
w := Wrap(v).(*T) // Wrap value to protect it.

// Use w freely. The type of w is the same as that of v.

// It's OK to modify w. The change will not affect v.
w.Foo = 456
fmt.Println(w.Foo) // 456
fmt.Println(v.Foo) // 123

// Once we need the original value stored in w, call `Unwrap`.
orig := Unwrap(w).(*T)
fmt.Println(orig == v) // true
fmt.Println(orig.Foo)  // 123

// Or, we can simply undo any change made in w.
// Note that `Undo` is significantly slower than `Unwrap`, thus
// the latter is always preferred.
Undo(w)
fmt.Println(w.Foo) // 123

Performance

Here is the performance data running on my dev machine.

go 1.20.1
goos: darwin
goarch: amd64
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkSimpleClone-12       7164530        156.7 ns/op       24 B/op        1 allocs/op
BenchmarkComplexClone-12       628056         1871 ns/op     1488 B/op       21 allocs/op
BenchmarkUnwrap-12           15498139        78.02 ns/op        0 B/op        0 allocs/op
BenchmarkSimpleWrap-12        3882360        309.7 ns/op       72 B/op        2 allocs/op
BenchmarkComplexWrap-12        949654         1245 ns/op      736 B/op       15 allocs/op

License

This package is licensed under MIT license. See LICENSE for details.

Documentation

Overview

Package clone provides functions to deep clone any Go data. It also provides a wrapper to protect a pointer from any unexpected mutation.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Clone 

func Clone[T any](t T) T

Clone recursively deep clone v to a new value in heap. It assumes that there is no pointer cycle in v, e.g. v has a pointer points to v itself. If there is a pointer cycle, use Slowly instead.

Clone allocates memory and deeply copies values inside v in depth-first sequence. There are a few special rules for following types.

  • Scalar types: all number-like types are copied by value.
  • func: Copied by value as func is an opaque pointer at runtime.
  • string: Copied by value as string is immutable by design.
  • unsafe.Pointer: Copied by value as we don't know what's in it.
  • chan: A new empty chan is created as we cannot read data inside the old chan.

Unlike many other packages, Clone is able to clone unexported fields of any struct. Use this feature wisely.

func ComparablePtr 

func ComparablePtr[T comparable](s *T) (d *T)

func DecimalPtr 

func DecimalPtr(s *decimal.Decimal) (d *decimal.Decimal)

func GormModelPtr 

func GormModelPtr(s *gorm.Model) (d *gorm.Model)

func IsScalar 

func IsScalar(k reflect.Kind) bool

IsScalar returns true if k should be considered as a scalar type.

For the sake of performance, string is considered as a scalar type unless arena is enabled. If we need to deep copy string value in some cases, we can create a new allocator with custom isScalar function in which we can return false when k is reflect.String. 

// Create a new allocator which treats string as non-scalar type.
allocator := NewAllocator(nil, &AllocatorMethods{
	IsScalar: func(k reflect.Kind) bool {
		return k != reflect.String && IsScalar(k)
	},
})

func Map 

func Map[T Clonable[T], K comparable](s map[K]T) (d map[K]T)

func MarkAsOpaquePointer 

func MarkAsOpaquePointer(t reflect.Type)

MarkAsOpaquePointer marks t as an opaque pointer in heap allocator, so that all clone methods will copy t by value. If t is not a pointer, MarkAsOpaquePointer ignores t.

Here is a list of types marked as opaque pointers by default:

  • `elliptic.Curve`, which is `*elliptic.CurveParam` or `elliptic.p256Curve`;
  • `reflect.Type`, which is `*reflect.rtype` defined in `runtime`.

func MarkAsScalar 

func MarkAsScalar(t reflect.Type)

MarkAsScalar marks t as a scalar type in heap allocator, so that all clone methods will copy t by value. If t is not struct or pointer to struct, MarkAsScalar ignores t.

In the most cases, it's not necessary to call it explicitly. If a struct type contains scalar type fields only, the struct will be marked as scalar automatically.

Here is a list of types marked as scalar by default:

  • time.Time
  • reflect.Value

func RegisterAtomicPointer 

func RegisterAtomicPointer[T any]()

RegisterAtomicPointer registers a custom clone function for atomic.Pointer[T].

func SetCustomFunc 

func SetCustomFunc(t reflect.Type, fn Func)

SetCustomFunc sets a custom clone function for type t in heap allocator. If t is not struct or pointer to struct, SetCustomFunc ignores t.

If fn is nil, remove the custom clone function for type t.

func Slice 

func Slice[T Clonable[T], TS ~[]T](s TS) (d TS)

func SliceComparable 

func SliceComparable[T comparable, TS ~[]T](s TS) (d TS)

func SliceComparablePtr 

func SliceComparablePtr[T comparable, TS ~[]*T](s TS) (d TS)

func Slowly 

func Slowly[T any](t T) T

Slowly recursively deep clone v to a new value in heap. It marks all cloned values internally, thus it can clone v with cycle pointer.

Slowly works exactly the same as Clone. See Clone doc for more details.

func TimeLocationPtr 

func TimeLocationPtr(s *time.Location) (d *time.Location)

func TimePtr 

func TimePtr(s *time.Time) (d *time.Time)

func Undo 

func Undo[T any](t T)

Undo discards any change made in wrapped value. If v is not a wrapped value, nothing happens.

func Unwrap 

func Unwrap[T any](t T) T

Unwrap returns v's original value if v is a wrapped value. Otherwise, simply returns v itself.

func Wrap 

func Wrap[T any](t T) T

Wrap creates a wrapper of v, which must be a pointer. If v is not a pointer, Wrap simply returns v and do nothing.

The wrapper is a deep clone of v's value. It holds a shadow copy to v internally. 

t := &T{Foo: 123}
v := Wrap(t).(*T)               // v is a clone of t.
reflect.DeepEqual(t, v) == true // v equals t.
v.Foo = 456                     // v.Foo is changed, but t.Foo doesn't change.
orig := Unwrap(v)               // Use `Unwrap` to discard wrapper and return original value, which is t.
orig.(*T) == t                  // orig and t is exactly the same.
Undo(v)                         // Use `Undo` to discard any change on v.
v.Foo == t.Foo                  // Now, the value of v and t are the same again.

Types

type Allocator 

type Allocator struct {
	// contains filtered or unexported fields
}

Allocator is a utility type for memory allocation.

func FromHeap 

func FromHeap() *Allocator

FromHeap creates an allocator which allocate memory from heap.

func NewAllocator 

func NewAllocator(pool unsafe.Pointer, methods *AllocatorMethods) (allocator *Allocator)

NewAllocator creates an allocator which allocate memory from the pool. Both pool and methods are optional.

If methods.New is not nil, the allocator itself is created by calling methods.New.

The pool is a pointer to the memory pool which is opaque to the allocator. It's methods' responsibility to allocate memory from the pool properly.

func (*Allocator) Clone 

func (a *Allocator) Clone(val reflect.Value) reflect.Value

Clone recursively deep clone val to a new value with memory allocated from a.

func (*Allocator) CloneSlowly 

func (a *Allocator) CloneSlowly(val reflect.Value) reflect.Value

CloneSlowly recursively deep clone val to a new value with memory allocated from a. It marks all cloned values internally, thus it can clone v with cycle pointer.

func (*Allocator) MakeChan 

func (a *Allocator) MakeChan(t reflect.Type, buffer int) reflect.Value

MakeChan creates a new chan with buffer.

func (*Allocator) MakeMap 

func (a *Allocator) MakeMap(t reflect.Type, n int) reflect.Value

MakeMap creates a new map with minimum size n.

func (*Allocator) MakeSlice 

func (a *Allocator) MakeSlice(t reflect.Type, len, cap int) reflect.Value

MakeSlice creates a new zero-initialized slice value of t with len and cap.

func (*Allocator) MarkAsOpaquePointer 

func (a *Allocator) MarkAsOpaquePointer(t reflect.Type)

MarkAsOpaquePointer marks t as an opaque pointer so that all clone methods will copy t by value. If t is not a pointer, MarkAsOpaquePointer ignores t.

Here is a list of types marked as opaque pointers by default:

  • `elliptic.Curve`, which is `*elliptic.CurveParam` or `elliptic.p256Curve`;
  • `reflect.Type`, which is `*reflect.rtype` defined in `runtime`.

func (*Allocator) MarkAsScalar 

func (a *Allocator) MarkAsScalar(t reflect.Type)

MarkAsScalar marks t as a scalar type so that all clone methods will copy t by value. If t is not struct or pointer to struct, MarkAsScalar ignores t.

In the most cases, it's not necessary to call it explicitly. If a struct type contains scalar type fields only, the struct will be marked as scalar automatically.

Here is a list of types marked as scalar by default:

  • time.Time
  • reflect.Value

func (*Allocator) New 

func (a *Allocator) New(t reflect.Type) reflect.Value

New returns a new zero value of t.

func (*Allocator) SetCustomFunc 

func (a *Allocator) SetCustomFunc(t reflect.Type, fn Func)

SetCustomFunc sets a custom clone function for type t. If t is not struct or pointer to struct, SetCustomFunc ignores t.

If fn is nil, remove the custom clone function for type t.

type AllocatorMethods 

type AllocatorMethods struct {
	// Parent is the allocator which handles all unhandled methods.
	// If it's nil, it will be the default allocator.
	Parent *Allocator

	New       func(pool unsafe.Pointer, t reflect.Type) reflect.Value
	MakeSlice func(pool unsafe.Pointer, t reflect.Type, len, cap int) reflect.Value
	MakeMap   func(pool unsafe.Pointer, t reflect.Type, n int) reflect.Value
	MakeChan  func(pool unsafe.Pointer, t reflect.Type, buffer int) reflect.Value
	IsScalar  func(k reflect.Kind) bool
}

AllocatorMethods defines all methods required by allocator. If any of these methods is nil, allocator will use default method which allocates memory from heap.

type Clonable 

type Clonable[T any] interface {
	Clone() T
}

type Cloner 

type Cloner struct {
	// contains filtered or unexported fields
}

Cloner implements clone API with given allocator.

func MakeCloner 

func MakeCloner(allocator *Allocator) Cloner

MakeCloner creates a cloner with given allocator.

func (Cloner) Clone 

func (c Cloner) Clone(v interface{}) interface{}

Clone clones v with given allocator.

func (Cloner) CloneSlowly 

func (c Cloner) CloneSlowly(v interface{}) interface{}

CloneSlowly clones v with given allocator. It can clone v with cycle pointer.

type Func 

type Func func(allocator *Allocator, old, new reflect.Value)

Func is a custom func to clone value from old to new. The new is a zero value which `new.CanSet()` and `new.CanAddr()` is guaranteed to be true.

Func must update the new to return result.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.