README
¶
arena
Chunk-backed storage for values whose lifetime is one batch.
Values are copied in and handed back as views over the copy. Nothing is freed individually; the whole arena is rewound at once. That is what makes it cheap — storing a value is a bump within the current chunk, so a batch's worth of values costs a handful of chunk allocations instead of one allocation per value, and the collector has a handful of objects to track instead of millions.
go get github.com/JohanLindvall/arena
Requires Go 1.24 or newer.
Quick start
var a arena.StringArena // the zero value is usable, with 64 KiB chunks
for _, batch := range batches {
labels := make([]string, 0, len(batch))
for _, key := range batch {
// A copy, so whatever the decoder underneath reuses its buffer for next
// cannot show through.
labels = append(labels, a.Intern(key))
}
emit(labels)
// Every view above is dead from here, so the chunks can be handed out again.
a.Reset()
}
The two types
Arena[T] stores slices of any T. StringArena is an Arena[byte] with the
string entry points added — it exists because a method cannot narrow its
receiver's type parameter, so Intern cannot live on Arena itself. Everything
Arena[byte] does is promoted onto it, and the embedded field is addressable as
a.Arena for code that wants the plain arena.
a := arena.New[Sample](1 << 20) // 1 MiB chunks of Sample
s := arena.NewStringArena(4096) // 4 KiB chunks, plus Intern/StrRef/Str
f := arena.Make[float64](4096) // the same, as a value rather than a pointer
Storing a value
These four copy the value in, and differ only in what you get back. (Reserve,
below, is the one entry point that does not copy.)
| Call | Returns | Use it when |
|---|---|---|
Arena[T].Append([]T) []T |
a []T view |
you hold few enough views that pointers are free |
Arena[T].AppendRef([]T) Ref[T] |
a pointer-free descriptor | you retain a great many descriptors across the batch |
StringArena.Intern(string) string |
a string view |
the value is a string and you hold a bounded number |
StringArena.StrRef(string) Ref[byte] |
a pointer-free descriptor | the value is a string and you retain a great many |
Value resolves a Ref[T] back to a []T; Str resolves one back to a
string. They are the same descriptor, so the byte and string sides
interoperate freely. All four store through one path, so they agree on
everything except what they hand back.
Copies are shallow: pointer-bearing elements still refer to the original objects.
Intern copies each string on every call; it does not deduplicate equal strings.
Every view is one contiguous slice. A value that does not fit the current chunk starts a new chunk rather than being split, so nothing has to be reassembled on the way out.
Views stay valid until the next Reset or Release. Chunks are never
reallocated once allocated, so storing more values never invalidates a view
handed out earlier.
Building a value in place
Reserve(n) is Append without the copy: it hands back a []T of length 0 and
capacity exactly n, backed by the arena, for a value you are building
rather than one you already hold. A decoder that learns an array's length before
its elements reserves the backing once and appends into it, paying neither a
per-value allocation nor a copy out of a scratch buffer.
p := a.Reserve(len(row)) // len 0, cap exactly len(row)
for _, v := range row {
p = append(p, v) // fills the arena's own storage
}
The capacity is exactly n rather than the rest of the chunk, so overfilling
reallocates to the heap the way any other full slice does and can never write
over the neighbour. Everything else matches Append: the region is yours alone,
it stays valid until Reset or Release, and a reservation too large for a
chunk gets a chunk of its own. Reserve and the copying entry points share one
placement step and mix freely on one arena.
Reserve does not clear what it hands back — a caller about to fill the
region would pay for the zeroing twice. Before the first Reset every chunk
comes from make and so reads as zero; once Reset hands a chunk out again it
holds whatever the previous batch left. clear it if you read before writing,
or hand out a region you only partly fill.
Clear the full reservation, since the returned slice has length zero:
p := a.Reserve(n)
clear(p[:cap(p)])
p = p[:n] // safe to read all n elements, including unwritten ones
Why Ref exists
Ref[T] locates a value by chunk index and range instead of by pointer:
refs := make([]arena.Ref[byte], 0, len(lines))
for _, line := range lines {
refs = append(refs, a.StrRef(line))
}
...
for _, r := range refs {
if !r.Empty() {
w.WriteString(a.Str(r))
}
}
The extra indirection on every read buys two things. A []Ref[T] holds no
pointers, so it is allocated noscan and the garbage collector skips it entirely
— where a [][]byte or []string puts a pointer in every element and is walked
on every cycle. And a Ref is 12 bytes against a string header's 16 or a slice
header's 24, which for a caller that retains descriptors is the same saving
twice over. Both hold whatever T is: Ref[T] is 12 pointer-free bytes even
when T itself is full of pointers.
The type parameter is a phantom — it carries no field, and is there so a Ref
cannot be resolved against an arena of some other element type.
The zero Ref is the absent value: AppendRef and StrRef return it for empty
input, Empty reports it, Value resolves it to nil and Str to "".
What a Ref can address
Three int32s are what make a Ref small, and also what bound it. The bound is
2³¹−1 elements, not bytes, so it scales with the width of T — 2 GiB for an
Arena[byte], 16 GiB for an Arena[int64]. Two things can reach it: a single
value stored through AppendRef/StrRef, and an arena New was given a chunk
that wide. A third, 2³¹ chunks at once, is out of reach at any sane chunk size.
AppendRef and StrRef panic rather than let the conversion wrap, so crossing the bound
is loud. The check measures free: the uniform store path already caps the offset
at the chunk's own capacity, so only an oversized value or an over-wide chunk can
get near it, and the branch is never taken in ordinary use.
The check happens after storing the value. Recovering the panic leaves that
value counted in Size until the next rewind.
It is worth naming what the panic stands in for. Unguarded, a length between 2³¹
and 2³² wraps end negative, so Empty reports the value absent and Value
returns nil — the value is gone with nothing said. Past 2³² it wraps to a small
positive number and Value returns a truncated prefix.
Append and Intern carry no such limit — a slice or string header holds no
int32. The bound belongs to the descriptor, not to the arena, so they are what
to reach for when a value could approach it.
Rewinding
| Call | Size() afterwards |
Uniform chunks | Oversized chunks |
|---|---|---|---|
Reset() |
0 | kept, contents unchanged | dropped |
Release() |
0 | dropped | dropped |
Reset is for an owner about to run the next batch: re-allocating the chunks
would be the largest allocation it makes, so it does not. Release is for an
owner that outlives its bytes — a pooled writer parked empty between batches,
which should retain its per-value bookkeeping and none of the values.
Reset does not erase stored data. If T contains pointers, old elements in
reused chunks keep their targets alive until overwritten or the chunks are
released. Use Release when those targets should become collectible between
batches. Dropping a chunk makes it eligible for garbage collection once no
other references to it remain; it does not immediately return memory to the OS.
Two counters describe the arena, both in bytes whatever T is. Size is the
bytes stored since the last rewind, which is what you check against a payload
budget. It includes unfilled reservations, but excludes failed allocations.
Retained is the chunk capacity holding them, excluding chunk headers, allocator
overhead, and objects referenced by T. A burst grows the chunk list and Reset
never shrinks the uniform part of it, so Retained is the number a trim policy
should watch. Mid-batch it
counts any oversized chunks in flight; between batches it is the uniform
capacity that carried over.
Sizing the chunk
New[T](chunkBytes) takes a byte budget, not a count of elements, and
divides it down to a whole number of T (never below one, even with a zero or
negative budget). Make[T] is the
same thing as a value rather than a pointer, for an arena that lives as a field
or a local; StringArena{Arena: arena.Make[byte](n)} is what NewStringArena
builds. A wider T means
fewer of them per chunk, not a bigger chunk — so the 64 KiB zero value stays
sane for a 64-byte struct instead of quietly becoming 4 MiB.
Chunk size decides the tail waste: a value that does not fit the current chunk starts a new one and strands the remainder, so size the chunk to the values. Interning short strings is comfortable at the default; accumulating large rows or blocks wants something closer to 1 MiB.
A value larger than a whole chunk gets a chunk of its own, sized to fit it
exactly. Copying and reserving follow the same rule, so an oversized value
is Ref-addressable like any other. Reset drops those chunks instead of
recycling them — one was made to fit a single large value and is the wrong shape
for anything else, so keeping it would leave that value's memory in the reusable
set for as long as the arena lives. What survives a Reset is uniform, however
lumpy the batch that just ran was.
The rules
The caller must observe these rules:
- Not safe for concurrent use. One goroutine at a time, or your own lock.
- Every view and
Refexpires atResetorRelease. Nothing checks this. A view read afterwards sees whatever the next batch wrote there — and a region fromReserveis a view like any other. Resolve a nonemptyRefonly against the arena and batch that created it; misuse can return unrelated data or panic. - Views from
AppendandValueare read-only. Do not modify or append to them: their capacity can extend into neighbouring values, including bytes exposed as immutable strings byInternandStr.Reserveis the explicit exception: its region is yours to fill, with capacity limited to its size. - Do not copy an arena after first use. This includes
StringArenaand structs containing either type. Copies share storage but track placement independently, so they can overwrite each other's values. Pass a pointer.
What the arena saves is object count, not scan work: for a T that contains
pointers the chunks are still scanned, they are simply a handful of large
objects rather than one per value. For a pointer-free T — bytes included — the
chunks are noscan and the collector ignores them outright.
Run your tests under -race; this package's own suite does, on every supported
Go version, on Linux, macOS and Windows.
Benchmarks
go test -bench=. ./... — medians of five runs on one machine (Intel Core
Ultra 9 185H, Go 1.26, linux/amd64). Indicative, not a promise.
Interning against the obvious alternative, one heap-allocated copy per value, with both sides holding a batch of 4096 live — a copy that dies immediately gets stack-allocated and proves nothing:
| Value | StringArena.Intern |
One heap copy per value |
|---|---|---|
| 16 B | 6.6 ns/op, 0 allocs | 15.5 ns/op, 1 alloc |
| 256 B | 8.9 ns/op, 0 allocs | 60.1 ns/op, 1 alloc |
| 4 KiB | 152 ns/op, 0 allocs | 647 ns/op, 1 alloc |
The 4 KiB row moves by tens of percent between runs; the two smaller ones are
stable to a few percent. That batch occupies 16 MiB across 256 default chunks.
The first batch allocates them; subsequent batches reuse them through Reset.
The collector is the other half of the story. Holding 2²⁰ descriptors live and timing one full GC cycle over each form — same bytes in the arena either way, only the descriptor slice differs:
| Descriptors retained | Width | GC cycle |
|---|---|---|
[]Ref[byte] |
12 B, noscan | 254 µs |
[][]byte |
24 B, scanned | 2081 µs |
[]string |
16 B, scanned | 2393 µs |
The element type costs nothing. Storing 256 bytes per call through arenas of
different T, where the struct case also carries a pointer and so has scanned
chunks:
| Element type | Store |
|---|---|
byte |
9.0 ns/op, 0 allocs |
int64 |
8.8 ns/op, 0 allocs |
struct{int64; float64; string} |
10.1 ns/op, 0 allocs |
Documentation
Full API documentation is on pkg.go.dev, including runnable examples.
License
Documentation
¶
Overview ¶
Package arena provides chunk-backed storage for values whose lifetime is one batch.
Values are copied in and handed back as views over the copy; nothing is freed individually. The whole arena is rewound at once — Reset to reuse the chunks, Release to drop them — which is what makes it cheap: allocation is a bump within the current chunk, and a batch's worth of values costs a handful of chunk allocations rather than one per value.
Arena is generic in its element type, so it stores slices of any T. Strings are the one case the generic form cannot express, since a method cannot narrow its receiver's type parameter: StringArena is an Arena[byte] with the string entry points added.
It is not safe for concurrent use. Every view handed out must be dead before Reset or Release. Views from the copying entry points are read-only; regions from Reserve may be filled by the caller.
Example ¶
A batch-lifetime workload: intern the batch's values, use the views, then rewind the arena and run the next batch over the same chunks.
package main
import (
"fmt"
"github.com/JohanLindvall/arena"
)
func main() {
var a arena.StringArena // the zero value is usable, with 64 KiB chunks
for batch := range 2 {
labels := make([]string, 0, 3)
for _, key := range []string{"container_id", "level", "namespace"} {
// A copy, so whatever the decoder underneath reuses its buffer for
// next cannot show through.
labels = append(labels, a.Intern(key))
}
fmt.Printf("batch %d: %v, %d bytes interned\n", batch, labels, a.Size())
// Every view above is dead from here, so the chunks can be handed out again.
a.Reset()
}
}
Output: batch 0: [container_id level namespace], 26 bytes interned batch 1: [container_id level namespace], 26 bytes interned
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Arena ¶
type Arena[T any] struct { // contains filtered or unexported fields }
Arena copies slices of T into chunk-backed storage and hands back views over the copies. A view stays valid until Reset: chunks are never reallocated once allocated (a value that does not fit the current chunk goes into a fresh one), so storing more never invalidates earlier views. Reset rewinds the arena to reuse the existing chunks. An Arena must not be copied after first use and is not safe for concurrent use. Copies would share storage but track placement independently. References must be resolved against the original arena before its next Reset or Release.
The chunks the arena reuses are all one size. A value too large for one gets a chunk of its own instead, which Reset drops rather than recycling, so a single huge value cannot leave an oversized chunk sitting in the reusable set for the arena's lifetime.
What the arena saves is object count, not scan work: for a T that contains pointers the chunks are still scanned, they are simply a handful of large objects rather than one per value. For a pointer-free T the chunks are noscan and the collector ignores them outright. Reset does not clear reused chunks, so pointers in them keep their targets alive until overwritten or the chunks are released. Copies of T are shallow.
Example ¶
The element type is not limited to bytes. An arena over any T stores slices of T and hands back views over the copies, with the same one-batch lifetime.
package main
import (
"fmt"
"github.com/JohanLindvall/arena"
)
func main() {
type sample struct {
at int64
value float64
}
// The chunk size is a byte budget, divided down to a whole number of elements — so a
// wider T means fewer of them per chunk, not a bigger chunk.
a := arena.New[sample](4096) // 4 KiB, which is 256 samples
window := a.Append([]sample{{at: 1, value: 0.5}, {at: 2, value: 1.5}})
fmt.Println(window, "-", a.Size(), "bytes stored,", a.Retained(), "retained")
}
Output: [{1 0.5} {2 1.5}] - 32 bytes stored, 4096 retained
func Make ¶ added in v0.4.0
Make is New as a VALUE rather than a pointer, for an arena that lives as a field of a struct the caller already has, or as a local — where New's allocation buys nothing. The zero Arena is usable and needs neither, so reach for Make only to choose a chunk size: `Make[T](n)` is to `var a Arena[T]` what `New[T](n)` is to `new(Arena[T])`.
A StringArena takes its chunk size the same way, through the embedded field: `StringArena{Arena: Make[byte](n)}` is what NewStringArena builds.
Example ¶
Make is New without the allocation, for an arena that lives inside something the caller already has. The zero Arena needs no constructor at all; Make is how a struct field picks a chunk size other than the 64 KiB default.
package main
import (
"fmt"
"github.com/JohanLindvall/arena"
)
func main() {
type decoder struct {
coords arena.Arena[float64]
counts arena.Arena[int32]
}
d := decoder{
coords: arena.Make[float64](4096),
counts: arena.Make[int32](4096),
}
xs := d.coords.Append([]float64{0.5, 1.5})
ns := d.counts.Append([]int32{7})
fmt.Println(xs, ns, "-", d.coords.Retained()+d.counts.Retained(), "bytes retained")
}
Output: [0.5 1.5] [7] - 8192 bytes retained
func New ¶
New returns an arena with a chunk size of its own, given as a byte budget and rounded down to a whole number of T. Use it when the values are large relative to the default chunk — chunk size decides the tail waste, since a value that does not fit the current chunk starts a new one and strands the remainder. The capacity is at least one element, even for a zero or negative budget. No chunk is allocated until a value is stored.
Example ¶
Chunk size decides the tail waste, since a value that does not fit the current chunk starts a new one and strands the remainder. Size it to the values.
package main
import (
"fmt"
"github.com/JohanLindvall/arena"
)
func main() {
a := arena.New[byte](1 << 20) // 1 MiB, for values that would shred a 64 KiB chunk
row := a.Append([]byte("a row too long to pack well into a small chunk"))
fmt.Printf("%s (%d bytes retained)\n", row, a.Retained())
}
Output: a row too long to pack well into a small chunk (1048576 bytes retained)
func (*Arena[T]) Append ¶
func (a *Arena[T]) Append(v []T) []T
Append copies v into the arena and returns a stable view over the copy. The view stays valid until Reset or Release, and is always ONE contiguous slice: a value that does not fit the current chunk starts a new one rather than being split, so nothing has to be reassembled on the way out.
A value larger than a whole chunk gets a chunk of its own, which Reset drops rather than recycling — see Arena.
Empty input returns nil. The view is read-only: do not modify or append to it, since its capacity can extend into neighbouring values. Use Reserve to build a value in place.
func (*Arena[T]) AppendRef ¶
AppendRef copies v into the arena and returns a pointer-free Ref to the copy, for a caller that keeps many descriptors and does not want them scanned (see Ref). It stores exactly as Append does, the chunk of its own an oversized value gets included; the two differ only in what they hand back.
It panics if the copy lands somewhere a Ref cannot describe — see the limits on Ref. That is a loud stand-in for what the int32 conversion would otherwise do quietly, which is hand back a descriptor resolving to nothing or to the wrong bytes. Append has no such limit and is the way to store a value that large.
The check is after the copy, not before it, so a recovered panic leaves the value stored but undescribed — wasted space in the current batch, which the next Reset reclaims, and never a corrupt arena.
Example ¶
Ref is the descriptor form: a chunk index and a range, with no pointer in it. A []Ref is allocated noscan, so a caller holding millions of them across a batch costs the collector nothing, where the []byte Append returns would put a pointer in every element and turn that slice into scan work on every cycle.
package main
import (
"fmt"
"github.com/JohanLindvall/arena"
)
func main() {
var a arena.Arena[byte]
refs := make([]arena.Ref[byte], 0, 3)
for _, line := range []string{`{"n":1}`, `{"n":2}`, ""} {
refs = append(refs, a.AppendRef([]byte(line)))
}
for _, r := range refs {
if r.Empty() { // the zero Ref, which is what empty input returns
fmt.Println("(absent)")
continue
}
fmt.Printf("%s\n", a.Value(r))
}
}
Output: {"n":1} {"n":2} (absent)
func (*Arena[T]) Release ¶
func (a *Arena[T]) Release()
Release drops the chunks outright rather than rewinding them, for a caller that keeps the STRUCTURE around a batch but not the batch's bytes — a pooled writer parks itself drained and releases its arena first, so what is retained is the per-value bookkeeping and never the values. Every view handed out must be dead first, exactly as for Reset.
Example ¶
Reset and Release are both all-at-once rewinds, and differ only in what they keep.
package main
import (
"fmt"
"github.com/JohanLindvall/arena"
)
func main() {
a := arena.New[byte](4096)
a.Append(make([]byte, 3000))
// Reset resets Size and keeps the chunks without clearing their contents.
a.Reset()
fmt.Println("after Reset:", a.Size(), "bytes stored,", a.Retained(), "retained")
// Release drops the chunks too, for an owner that outlives its bytes — a pooled
// writer parked empty between batches.
a.Release()
fmt.Println("after Release:", a.Size(), "bytes stored,", a.Retained(), "retained")
}
Output: after Reset: 0 bytes stored, 4096 retained after Release: 0 bytes stored, 0 retained
func (*Arena[T]) Reserve ¶ added in v0.4.0
Reserve hands back room for n elements — a slice of length 0 and capacity exactly n, backed by arena storage — for a caller that fills the region itself rather than copying a value in. It is Append without the copy, and the two can be mixed freely on one arena. n <= 0 reserves nothing and returns nil.
Where Append is for a value the caller already has, Reserve is for one being built: a decoder that knows an array's length before its elements can reserve the backing once and append the elements into it, paying neither a per-value allocation nor a copy out of a scratch buffer.
The capacity is exactly n, not the rest of the chunk, so filling the region past n reallocates to the heap the way any other full slice does and can never write over the neighbouring value. The region is the caller's alone: no later Append or Reserve overlaps it, and the view stays valid until Reset or Release exactly as Append's does. A value larger than a whole chunk gets a chunk of its own, as in Append.
Contents ¶
Reserve does not clear what it hands back, since a caller that is about to fill the region would pay for the zeroing twice. In a chunk the arena has not yet reused — every chunk before the first Reset — the memory comes from make and so reads as zero; after a Reset hands the same chunk out again it holds whatever the previous batch left there. A caller that reads before writing, or that hands out a region it only partly fills, wants `clear(region[:cap(region)])` (or an arena it never Resets). Clearing the returned length-zero slice alone does nothing.
Size counts the whole reservation, filled or not: it is the room the batch has taken, which is what a caller bounding a payload is asking about. If allocating the region panics, the failed reservation does not count toward Size.
Example ¶
Reserve is Append without the copy, for a value that is being built rather than one the caller already has. A decoder that learns an array's length before its elements reserves the backing once and fills it in place — no allocation per value, and no copy out of a scratch buffer either.
package main
import (
"fmt"
"github.com/JohanLindvall/arena"
)
func main() {
a := arena.New[float64](4096)
// Three points, each backed by the same chunk rather than three make calls.
var points [][]float64
for _, row := range [][]float64{{1, 2, 3}, {4, 5, 6}, {7, 8, 9}} {
p := a.Reserve(len(row)) // length 0, capacity exactly len(row)
for _, v := range row {
p = append(p, v*10)
}
points = append(points, p)
}
fmt.Println(points, "-", a.Size(), "bytes stored,", a.Retained(), "retained")
}
Output: [[10 20 30] [40 50 60] [70 80 90]] - 72 bytes stored, 4096 retained
Example (Clear) ¶
Reset retains chunk contents. Clear the whole reservation before partially filling it: clearing Reserve's length-zero result alone would leave the old values in place.
package main
import (
"fmt"
"github.com/JohanLindvall/arena"
)
func main() {
var a arena.Arena[int]
a.Append([]int{7, 8, 9})
a.Reset()
p := a.Reserve(3)
clear(p[:cap(p)])
p = p[:3]
p[0] = 42
fmt.Println(p)
}
Output: [42 0 0]
func (*Arena[T]) Reset ¶
func (a *Arena[T]) Reset()
Reset rewinds the arena so its chunks are reused for the next round. The caller must ensure every view previously handed out is dead before calling Reset.
Oversized chunks are dropped rather than rewound. One was made to fit a single large value and is the wrong shape for anything else, so recycling it would leave that value's memory in the reusable set for as long as the arena lives. What survives a Reset is uniform, however lumpy the batch that just ran was.
Reused chunks are not cleared. If T contains pointers, those pointers keep their targets alive until overwritten or released, even though Size is zero. Release drops all chunks when retaining the previous batch's targets is undesirable. Reset invalidates Refs too.
func (*Arena[T]) Retained ¶
Retained reports the bytes the arena is holding — its chunk capacity, which is the real storage a trim policy should judge (a burst grows the chunk list; Reset keeps the uniform chunks). It excludes chunk headers, allocator overhead, and objects referenced by T.
Mid-batch this counts any oversized chunks in flight. Reset drops those, so what it reports between batches is the uniform capacity that carries over.
type Ref ¶
type Ref[T any] struct { // contains filtered or unexported fields }
Ref locates a value in the arena WITHOUT a pointer to it — a chunk index and a range within that chunk. Value resolves it. The type parameter is a phantom: it carries no field, and exists so a Ref cannot be resolved against an arena of some other element type.
The pointer-free part is the point, and it is worth the extra indirection on every read. A []Ref is allocated noscan, so the GC never walks it and a sort never pays a write barrier moving one; the []T form that Append returns puts a pointer in every element, and a caller holding millions of them turns its largest structure into scan work on every cycle. Measured on a production ingest service: the []byte form put runtime.gcBgMarkWorker at 11.1% of the process against 0.5% elsewhere in the tree. It is also 12 bytes against a slice header's 24, which for a caller that retains descriptors is the same saving twice over. Both hold whatever T is, pointer-bearing ones included.
The zero Ref is the ABSENT value: a stored value is never zero elements long (AppendRef returns the zero Ref for empty input), so end <= off cannot name a real one. A nonempty Ref belongs to the arena and batch that created it. Using it with another arena or after Reset or Release is invalid and may panic or return unrelated data; ownership and lifetime are not checked.
Limits ¶
Three int32s are what makes a Ref small, and they are also what bounds it. The bound is 2^31-1 ELEMENTS rather than bytes, so it scales with the width of T: 2 GiB for an Arena[byte], but 16 GiB for an Arena[int64]. Two things can reach it — a single value stored through AppendRef or StrRef, and an arena that New was given a chunk that wide. A third, 2^31 chunks at once, is out of reach at any sane chunk size.
AppendRef panics rather than let the conversion wrap, so crossing the bound is loud. It is worth knowing what the panic prevents: a length in [2^31, 2^32) wraps end negative, which Empty would report as ABSENT and Value resolve to nil, losing the value with nothing said; a length at or past 2^32 wraps to a small positive number, which Value would resolve to a truncated prefix.
Append and Intern carry no such limit. They hand back a slice or a string header, neither of which holds an int32 — the bound belongs to the descriptor, not to the arena — so they are what to reach for when a value could approach it.
type StringArena ¶
StringArena is an Arena[byte] with the string entry points added. It exists because a method cannot narrow its receiver's type parameter, so entry points that are only meaningful when the element type is byte cannot live on Arena itself.
Everything Arena[byte] does is promoted, so a StringArena stores raw bytes and hands out Ref[byte] descriptors too, and the embedded field is addressable as a.Arena for code that wants the plain arena. The zero value is usable, with 64 KiB chunks. Like Arena, StringArena must not be copied after first use and is not safe for concurrent use.
There are two ways to hold what it stores, matching the two on Arena. Intern hands back a string view, which is what a caller with a bounded number of live values wants; StrRef hands back a pointer-free descriptor that Str resolves, which is what a caller retaining a great many of them wants — a []Ref[byte] is 12 bytes per element and noscan, where a []string is 16 with a pointer in every one.
func NewStringArena ¶
func NewStringArena(chunkBytes int) *StringArena
NewStringArena returns a string arena with a chunk size of its own, as New does. Bytes and elements are the same thing here, so max(1, chunkBytes) is the chunk capacity.
func (*StringArena) Intern ¶
func (a *StringArena) Intern(s string) string
Intern copies s into the arena and returns a stable string view over the copy. A string larger than a whole chunk gets a chunk of its own, which Reset drops rather than recycling — see Arena.Append. Each call copies its input; equal strings are not deduplicated. The empty string stores nothing and returns "".
Example ¶
An interned view stays valid until Reset no matter how much is interned after it: a value that does not fit the current chunk starts a new chunk rather than growing one that already has views pointing into it.
package main
import (
"fmt"
"github.com/JohanLindvall/arena"
)
func main() {
a := arena.NewStringArena(4096)
first := a.Intern("still here")
for range 1000 {
a.Intern("filler, enough of it to force several new chunks")
}
fmt.Println(first, "-", a.Retained() > 4096, "chunks were added after it")
}
Output: still here - true chunks were added after it
func (*StringArena) Str ¶
func (a *StringArena) Str(r Ref[byte]) string
Str resolves r to a string view over the arena's copy — the string counterpart of Value. The absent descriptor resolves to the empty string. A nonempty r must belong to this arena's current batch; see Ref. The view stays valid until Reset or Release.
func (*StringArena) StrRef ¶
func (a *StringArena) StrRef(s string) Ref[byte]
StrRef copies s into the arena and returns a pointer-free descriptor for the copy — the string counterpart of AppendRef, resolved by Str. Reach for it over Intern when the batch retains enough strings for the descriptors themselves to matter (see Ref).
The empty string gives the zero Ref, which Str resolves back to "".
Like AppendRef, it panics if the stored range or chunk index exceeds the limits of Ref, including strings of 2 GiB or more. The check follows the copy, so a recovered panic leaves the value stored but undescribed. Intern has no such limit.
Example ¶
StrRef and Str are the descriptor form of Intern: the arena still owns the bytes, but what the caller retains is 12 pointer-free bytes rather than a string header.
package main
import (
"fmt"
"github.com/JohanLindvall/arena"
)
func main() {
a := arena.NewStringArena(4096)
refs := make([]arena.Ref[byte], 0, 3)
for _, s := range []string{"container_id", "level", ""} {
refs = append(refs, a.StrRef(s))
}
for _, r := range refs {
fmt.Printf("%q empty=%v\n", a.Str(r), r.Empty())
}
}
Output: "container_id" empty=false "level" empty=false "" empty=true
Source Files