README
¶
ferry/yaml
Load configuration from a YAML file into a Go struct, and write it back without wrecking the file.
go get github.com/onhotpath/ferry/driver/yaml
Loading and saving
Given config.yaml:
# the port the server listens on
port: 8080
label: "8080" # quoted, so it stays a string
debug: false
tags:
- a
owner: platform-team
and this struct:
type config struct {
Port int `ferry:"port"`
Label string `ferry:"label"`
Debug bool `ferry:"debug"`
Tags []string `ferry:"tags"`
}
load it, change two fields, and write it back:
cfg, err := ferry.Load[config](ctx, yaml.NewSource(path))
// port=8080 label="8080" debug=false tags=[a]
cfg.Debug = true
cfg.Tags = append(cfg.Tags, "b")
err = ferry.Dump(ctx, cfg, yaml.NewSink(path))
The file afterwards:
# the port the server listens on
port: 8080
label: "8080" # quoted, so it stays a string
debug: true
tags:
- a
- b
owner: platform-team
That is the Example in example_test.go, trimmed of its setup, so go test compiles it and compares that exact output.
Saving edits a file rather than replacing it
The comment survived, so did the key order, so did the quotes around "8080", and so did owner, which no field maps.
Only the keys your struct names are touched.
That is the reason to use this rather than marshalling a struct to YAML: a config file a person maintains stays a config file a person maintains.
What does not survive, all of them limits of the YAML writer rather than choices: a blank line between entries, an explicit --- marker, a file whose entire content is comments, the original indentation, and the column a trailing comment sat in.
A scalar's tag is not on that list.
A tag this package has no reading of its own for survives a save at a key your struct maps, so when: !!timestamp 2026-08-04 is saved back with its tag still on it.
Three cases replace it: a tag this package writes itself, a tag whose value is no longer the kind it was, which is stale in the way the old quoting would be, and a tag your own field declared - see A field can say what node type it is written as.
Such a tag is carried and not interpreted: !!timestamp 2026-08-04 and !mycompany:duration 30s both load as the string after the tag, whatever the tag says, and reach whichever codec your field declared.
A list or a map is replaced whole
The editing stops at a list or a map your struct maps, because what is in one comes from your value and not from your type.
Save []string{"x"} over
tags:
- a
- b
- c
and the file holds tags: [x], not [x, b, c].
A map that lost a key no longer holds that key either.
Anything else leaves the elements you dropped in the file, where they load straight back the next time out of a document that says something your value never did.
That was #220, and both legs of it were silent.
What stays is still yours. The entry is edited where it already sits rather than written out fresh, so its comments, its anchor and its tag all survive:
# the file
tags:
- &first x # keep me
note: untouched
is what saving []string{"x"} leaves of a three-element list whose first position carried that anchor and that comment.
A list is cut at the last position saved, which is what keeps the positions before it from being renumbered.
A leaf is the whole document
A value that is not a struct at all names one address, the document itself.
ferry.Dump(ctx, 8080, yaml.NewSink(path)) writes 8080, and ferry.Load[int] of that file reads it back.
Every leaf kind works, []byte included, and it is written under !!binary exactly as a byte field at a key would be.
Saving one over a file that already holds a mapping replaces the whole file: keep: me and other: 2 become 8080 and nothing else.
That is the rule above and not an exception to it.
The keys a save leaves alone are the ones no field of yours maps, and a value whose only address is the root leaves none.
A struct still patches, so nothing about the section above changes.
A struct's fields are not touched by this rule: a field your value leaves out is left exactly where it is, and so is every key no field of yours maps.
An anchor is kept, so an alias to it moves
There is exactly one case where a key no field of yours maps does not read back as it did.
host: &h localhost
other: *h
Save host as example and the file becomes:
host: &h example
other: *h
other's line is byte for byte what it was, and other now reads back as example rather than localhost.
That is what an anchor means: the operator who wrote other: *h said other is whatever host is, and the alias follows.
The alternative is worse.
Dropping the anchor leaves *h pointing at nothing, so the save reports success and writes a file that no reader can parse, including ferry's own load right afterwards.
That was #196.
It works the other way round too. A key your struct maps that is itself an alias is written through to the anchor it names:
base: &b 5432
port: *b
With port mapped, saving it as 5433 writes base: &b 5433 and leaves the port: *b line exactly as it was.
The value moves and the linkage survives.
Saving it as its own value instead would have written port: 5433 and quietly unshared the two, which was #198.
Two things follow from that.
A save refuses where your struct and the document disagree.
With both base and port above mapped, saving 1 and 2 asks the file to hold two values in one place.
It fails with ferry.ErrPlane naming the second address, and your file is left byte for byte as it was.
Saving the same value to both is fine, because that is what the document already says.
An alias naming a scalar is replaced where the address needs a container.
base: &b 5432
db: *b
other: *b
With db mapped as a struct, saving it writes db: with your fields under it and leaves base and other alone.
Following the alias would rewrite base into a mapping under other, which no field maps, and it would keep nothing: a scalar has no keys of its own for the reshape to lose.
Types survive the trip
A number stays a number, a quoted number stays a string, and null stays different from a key that is not there at all:
nul: null # an explicit null
empty: "" # an empty string, which is not the same thing
value: 8080 # a number, and "8080" would be a string
# a key that is simply missing is different again
A []byte field is saved as standard YAML !!binary, base64.
A number is read in YAML's own spellings, so 0x1F, 0o17, 0b101, 1_000, .inf and .nan all load, and a leading zero is octal as every YAML reader reads it: 017 is fifteen.
Saving writes spellings any YAML reader takes back, .inf and .nan among them.
A number you spelled your own way survives a save that did not change it, so mask: 0x1F stays 0x1F.
A field can say what node type it is written as
Keeping a tag the file already had is one thing.
Putting one there is another, and a save cannot guess it: what crosses ferry's boundary is a value and not a Go type, so wait: 30s says nothing about wanting !mycompany:duration.
yaml.Extension() is where a field says it.
Declare this package's struct tag key on a registry, annotate the field, and pass the registry to the call:
registry := ferry.MustRegistry(ferry.WithTagKeys(yaml.Extension()))
type config struct {
Wait string `ferry:"wait" yamlext:"node=!mycompany:duration"`
Port int `ferry:"port"`
}
cfg, err := ferry.Load[config](ctx, yaml.NewSource(path), ferry.WithRegistry(registry))
err = ferry.Dump(ctx, cfg, yaml.NewSink(path), ferry.WithRegistry(registry))
wait: 30s in, and out:
wait: !mycompany:duration 30s
port: 8080
That is ExampleExtension in example_test.go, trimmed of its setup.
The tag is written whether or not the file had one there, and a tag the file did have at that address loses to the declared one. A load needs nothing, because the value arrives as the text after the tag either way - which is what lets the annotation survive a load, a save and a second load with nothing lost.
Four things it refuses.
A tag that is not a tag: one not starting with !, one naming nothing after it, or one with a space in it.
A tag this package spells itself, !!str and !!int and the rest, because the value's own kind decides those.
The address of a struct, a list or a map, because a node tag names how one value is written and those are places rather than values - annotate the fields under them.
And a value this plane writes as a number, a boolean or bytes, refused when that value is written, because a scalar under a tag this plane does not read comes back as a string and the value would not survive the trip.
A null is written plainly rather than refused, so an optional field that happens to be unset does not fail a save.
The key is yamlext and not yaml, which is the key go-yaml's own marshaller reads: a field may carry both.
A registry that was not given the declaration reads none of it, and a save writes what it always wrote.
Loading and saving are separate types
yaml.NewSource(path) reads and yaml.NewSink(path) writes, so the path is written twice.
That is deliberate: code handed only a Source cannot save through it, and passing one to ferry.Dump does not compile rather than failing halfway through a write.
Saving is atomic, and durable if you ask
A save writes a temporary file beside yours and renames it into place once everything has been written. Nothing ever reads a half-written config, and no temporary file is ever left behind. A save that fails leaves your file byte for byte as it was. That is unconditional and there is no way to switch it off.
What is not unconditional is durability, which is a different promise and a far more expensive one. By default the replacement is handed to the operating system and lives in its cache until the kernel writes it out, exactly as any ordinary file write does, so a machine that loses power in that window comes back to the old document.
err = ferry.Dump(ctx, cfg, yaml.NewSink(path, yaml.Durable()))
yaml.Durable() buys the other promise: the new file's contents are flushed to the disk, and so is the directory entry that makes your path point at them, so a save that returned nil has reached the disk rather than the cache.
It costs a disk flush, which is usually more expensive than everything else in the save put together.
Windows has no way to flush a directory: there the contents are flushed and the durability of the rename is the filesystem's own business.
A durable save has one case where a save that failed has still replaced your file, and it is the flush that fails once the rename has landed.
It reports ferry.ErrPlane, because what could not be promised is that the replacement survives a crash, not that it happened.
A save refuses a file somebody else edited
A save is a merge into the document that is already there, so it reads the file, stages the replacement and renames it into place. An edit that lands in that window would be swapped away without a word.
So the save compares the file against what it read, one stat before the rename, and refuses with ferry.ErrPlane where it changed:
the file changed after this save read it, and saving now would discard that change:
load the file again, apply the same edits to what it holds now, and save again
Your file is left exactly as the other writer left it. The one edit the check cannot see is a rewrite that lands in the same modification-time tick and leaves the file's length alone.
Watching the file
Pass yaml.Watch to be called when the file changes underneath a source, which is how a process holding a loaded value learns to load a fresh one:
s := watch.New()
b, err := ferry.Bind[config](yaml.NewSource(path, yaml.Watch(ctx, 10*time.Millisecond, s.Changed)))
// ... an operator edits the file ...
seq, errf := watch.Values(ctx, s, b)
for cfg := range seq {
publish(cfg) // a reload is a load; publish it by replacement
}
That is ExampleWatch in example_test.go, trimmed of its setup and its plumbing.
It is opt-in, and it is the only thing in this package that runs on a goroutine of its own. Cancelling the context you gave it is what stops it. Looking is a stat every interval rather than an fsnotify subscription, so that watching a file costs this module no dependency, and the interval is yours to name.
Two sharp edges are worth reading before you wire it up, and the rest are in the package documentation.
Watching starts when the source is built, which is before ferry.Bind has handed back the binding to load through, so a change can land while there is nothing to load through yet.
watch.Signal is what keeps it: s.Changed records such a change instead of losing it, and watch.Values opens the stream with that reload.
And a panic in the callback takes the process down, exactly as it would on a goroutine you started yourself.
The loop this feeds is ferry/watch, and the whole of it is in the watch and reload guide.
One thing it cannot do
A Go string can hold any bytes; a YAML string has to be valid text.
So a string field holding bytes that are not valid UTF-8 cannot be saved to YAML at all.
Rather than mangling the value or inventing a private YAML tag for it, saving fails and names the field, and your file is left alone.
If you need to move arbitrary bytes through a YAML file, declare that field []byte instead, which is saved as !!binary.
This is a known limitation and is tracked in #157.
A node tag does not reach it: a string field carries no custom type to annotate, and which of your strings are not valid UTF-8 is not something you can know in advance.
More
The package documentation is the reference for everything above, and the design records behind it are in docs/adr/.
Documentation
¶
Overview ¶
Package yaml loads configuration from a YAML file into a Go struct, and writes it back without wrecking the file.
cfg, err := ferry.Load[Config](ctx, yaml.NewSource("config.yaml"))
err = ferry.Dump(ctx, cfg, yaml.NewSink("config.yaml"))
NewSource reads and NewSink writes, so the path is named twice. That is deliberate: code handed only a Source cannot save through it, and passing one to ferry.Dump does not compile rather than failing halfway through a write.
Saving edits the file rather than replacing it ¶
Only the keys your struct names are touched. Comments, key order, flow style, block scalars, the quoting of values ferry did not write and every key no field maps are all still there afterwards. That is the reason to use this rather than marshalling a struct to YAML: a config file a person maintains stays a config file a person maintains.
What does not survive is the YAML writer's limits rather than choices: a blank line between two entries, an explicit --- start marker, a file whose whole content is comments, the original indentation, which is two spaces on the way out, and the column a trailing comment sat in.
A scalar's tag is not on that list. A tag this package has no reading of its own for survives a save at a key your struct maps, so `when: !!timestamp 2026-08-04` is saved back with its tag still on it. Three cases replace it: a tag this package writes itself, a tag whose value is no longer the kind it was, which is stale in the way the old quoting would be, and a tag your own field declared through Extension.
A file holding a stream of several documents is refused rather than half-written, because an address names a place in one of them.
So is a file that spells one mapping key twice, and in both directions. The file says two things: a YAML reader takes the last spelling and a key is read here as the first, so a load would answer with a value most readers of that file do not see and a save would rewrite one occurrence and leave the other behind. The refusal names the key and both lines, and it lands before anything is written.
A list or a map is replaced whole ¶
The editing stops at a list or a map your struct maps, because what is in one comes from your value rather than from your type. Saving []string{"x"} over
tags: - a - b - c
leaves `tags: [x]` and not `[x, b, c]`, and a map that lost a key no longer holds that key in the file. Otherwise the elements you dropped would load straight back the next time, out of a file that says something your value never did.
What stays is still yours: the comments on it, the anchor on it and the tag it was written under all survive, because the entry is edited where it already sits rather than written out fresh. A list is cut at the last position saved, which is what keeps the positions before it from being renumbered.
A struct's fields are not touched by this. A field your value leaves out is left exactly where it is, and so is every key no field of yours maps.
An anchor is kept, so an alias to it moves ¶
An anchor you wrote on a value ferry replaces stays on it, and that is the one case where a key no field maps does not read back as it did. Given `host: &h localhost` and `other: *h`, saving host as "example" writes `host: &h example`, and `other` - whose line is byte for byte what it was - now reads back as "example". That is what an anchor means: other is whatever host is. Dropping the anchor instead would leave `*h` pointing at nothing and the file would no longer parse for any reader.
It works the other way round too. Saving a key that is itself an alias writes through to the anchor it names, so given `base: &b 5432` and `port: *b`, saving port as 5433 writes `base: &b 5433` and leaves the `port: *b` line alone. The value moves and the linkage survives, which is the same reading of an anchor.
Two consequences follow, and both are worth knowing before you write one.
A save refuses where your struct disagrees with the document about two keys that share an anchor. If `base` and `port` above are both mapped and you save 1 and 2, the file can hold only one of them, so the save fails with ferry.ErrPlane naming the second address and your file is left as it was. Saving the same value to both is fine.
An alias naming a scalar, at an address your struct says is a mapping or a list, is replaced rather than followed: given `base: &b 5432` and `db: *b`, saving db as a struct writes `db:` with your fields under it and leaves `base` alone. Following it would rewrite `base` itself under every other alias to it, and there is nothing to be gained, because a scalar has no keys of its own to keep.
A merge key is read, and written as an override ¶
A mapping that says `<<: *defaults` holds what defaults holds, and a load reads it that way. Given
defaults: &d host: localhost port: 1 db: <<: *d port: 5432
db loads as host "localhost" and port 5432: the key the mapping spells itself wins, the merge fills in the rest, and `<<` is never a key of your own. A map-typed field over db holds host and port and no member named `<<`. Written as a list, `<<: [*a, *b]`, the earlier source wins; a source that merges in turn is followed too.
A save does not write through one. Saving db's host writes a `host` key into db, which shadows the merged one, and leaves defaults exactly as it is - because writing through would move the value under every other mapping that merges defaults, which is not what one field of one struct asked for.
The one place that costs you something is a map or a list, which a save replaces whole. The mapping's own members are what the replacement keeps, so a map-typed field over db is written back with host and port spelled out and the `<<` line gone. The values are the ones your value held either way; the line that produced them is not kept, because keeping it would leave behind every inherited key the replacement meant to drop. Model a merged mapping as a struct if the `<<` line has to survive.
Types survive the trip ¶
YAML resolves a scalar's tag, so `port: 8080` is a number, `port: "8080"` is a string, `debug: true` is a boolean, `tags: null` is an explicit null, and a key that is simply not there is different again. Each of the five crosses ferry as its own kind and comes back spelled the way it went in. A []byte field is saved as standard YAML !!binary, base64.
A tag this package does not read is carried and not interpreted. `!!timestamp 2026-08-04` and `!mycompany:duration 30s` both load as the string after the tag, whatever the tag says, and reach whichever codec your field declared. So a field of a type that parses that text works today, and the tag itself changes nothing about how the value is read.
A field can say what node type it is written as ¶
Keeping a tag the file already had is one thing; putting one there is another, and a save cannot guess it: what crosses ferry's boundary is a value and not a Go type, so `wait: 30s` says nothing about wanting !mycompany:duration.
Extension is where a field says it. Declare this package's struct tag key on a registry, annotate the field, and pass the registry to the call:
var registry = ferry.MustRegistry(ferry.WithTagKeys(yaml.Extension()))
type Config struct {
Wait string `ferry:"wait" yamlext:"node=!mycompany:duration"`
}
err := ferry.Dump(ctx, cfg, yaml.NewSink(path), ferry.WithRegistry(registry))
The save writes `wait: !mycompany:duration 30s`, whether or not the file had a tag there, and a tag it did have at that address loses to the declared one. A load needs nothing: the value arrives as the text after the tag either way, which is what lets the annotation survive a load, a save and a second load with nothing lost. Read Extension for what it refuses.
Saving is atomic, and durable if you ask ¶
A save writes a temporary file beside yours and renames it into place once everything has been written, so nothing ever reads a half-written config, and no temporary file is ever left behind. A save that fails leaves your file byte for byte as it was. That is unconditional and there is no way to switch it off.
What is not unconditional is durability, which is a different promise and a far more expensive one. By default the replacement is handed to the operating system and lives in its cache until the kernel writes it out, exactly as any ordinary file write does, so a machine that loses power in that window comes back to the old document.
err = ferry.Dump(ctx, cfg, yaml.NewSink("config.yaml", yaml.Durable()))
Durable buys the other promise: the new file's contents are flushed to the disk, and so is the directory entry that makes your path point at them, so a save that returned nil has reached the disk rather than the cache. It costs a disk flush, which is usually more expensive than everything else in the save put together. Windows has no way to flush a directory: there the contents are flushed and the durability of the rename is the filesystem's own business.
A durable save has one case where a save that failed has still replaced your file, and it is the flush that fails once the rename has landed. It reports ferry.ErrPlane, because what could not be promised is that the replacement survives a crash, not that it happened.
A save refuses a file somebody else edited ¶
A save is a merge into the document that is already there, so it reads the file, stages the replacement and renames it into place. An edit landing in that window would be swapped away without a word, so the save compares the file against what it read - one stat, before the rename - and reports ferry.ErrPlane where it changed, leaving your file exactly as the other writer left it. Load again, apply the same change to what the file holds now, and save again.
The check is the file's length and modification time, so the one edit it cannot see is a rewrite in the same modification-time tick that leaves the length alone.
Watching the file ¶
Watch calls you back when the file changes underneath a source, which is how a process holding a loaded value learns to load a fresh one:
src := yaml.NewSource("config.yaml", yaml.Watch(ctx, time.Second, onChange))
func onChange(ctx context.Context) {
cfg, err := b.Load(ctx) // a reload is a load; publish it by replacement
}
It is opt-in and it is the only thing in this package that runs on a goroutine of its own: a source built without it touches the file only when a load asks it to. Cancelling the context you gave it is what stops it. Looking is a stat every interval rather than a subscription, so watching a file costs this module no dependency and the interval is yours to name.
Read Watch before wiring one up. Two of its sharp edges bite immediately: watching starts when the source is built, which is before ferry.Bind has handed back the binding your callback wants to load through, and a panic in the callback takes the process down exactly as it would on a goroutine you started yourself.
One thing it cannot do ¶
A Go string can hold any bytes; a YAML string has to be valid text. So a string field holding bytes that are not valid UTF-8 cannot be saved to YAML at all. Rather than mangling the value, the save fails with ferry.ErrValue and names the field, and your file is left alone. Declare that field []byte instead if you need to move arbitrary bytes through a YAML file.
The design records behind these decisions are in docs/adr/.
Example ¶
Example loads a hand-maintained config file, changes two fields and writes it back through the same path.
The output is the whole point: a dump is a merge into the document that is already there, so the comment, the key order, the quoting ferry did not touch and the key no field maps are all still in the file afterwards.
It is quoted in this package's README, which is why it is written to be read rather than to cover a branch.
package main
import (
"context"
"fmt"
"os"
"path/filepath"
"github.com/onhotpath/ferry"
"github.com/onhotpath/ferry/driver/yaml"
)
func main() {
type config struct {
Port int `ferry:"port"`
Label string `ferry:"label"`
Debug bool `ferry:"debug"`
Tags []string `ferry:"tags"`
}
const plane = `# the port the server listens on
port: 8080
label: "8080" # quoted, so it stays a string
debug: false
tags:
- a
owner: platform-team
`
path := writeExamplePlane(plane)
defer func() { _ = os.RemoveAll(filepath.Dir(path)) }()
ctx := context.Background()
cfg, err := ferry.Load[config](ctx, yaml.NewSource(path))
if err != nil {
fmt.Println(err)
return
}
fmt.Printf("port=%d label=%q debug=%v tags=%v\n\n", cfg.Port, cfg.Label, cfg.Debug, cfg.Tags)
cfg.Debug = true
cfg.Tags = append(cfg.Tags, "b")
if err := ferry.Dump(ctx, cfg, yaml.NewSink(path)); err != nil {
fmt.Println(err)
return
}
back, err := os.ReadFile(path)
if err != nil {
fmt.Println(err)
return
}
fmt.Print(string(back))
}
// writeExamplePlane puts the example's starting document in a directory of its
// own, because the sink stages its replacement beside the plane.
func writeExamplePlane(doc string) string {
dir, err := os.MkdirTemp("", "ferry-yaml-example")
if err != nil {
panic(err)
}
path := filepath.Join(dir, "config.yaml")
if err := os.WriteFile(path, []byte(doc), 0o600); err != nil {
panic(err)
}
return path
}
Output: port=8080 label="8080" debug=false tags=[a] # the port the server listens on port: 8080 label: "8080" # quoted, so it stays a string debug: true tags: - a - b owner: platform-team
Index ¶
Examples ¶
Constants ¶
const ExtensionKey = "yamlext"
ExtensionKey is the struct tag key Extension declares.
It is not "yaml", which is the key go-yaml's own marshaller reads: a field may carry both, and each library reads the key it owns.
Variables ¶
This section is empty.
Functions ¶
func Extension ¶
func Extension() ferry.KeyExtension
Extension declares this driver's struct tag key, for a registry to read beside ferry's own.
var Registry = ferry.MustRegistry(ferry.WithTagKeys(yaml.Extension()))
type Config struct {
Wait string `ferry:"wait" yamlext:"node=!mycompany:duration"`
}
err := ferry.Dump(ctx, cfg, yaml.NewSink(path), ferry.WithRegistry(Registry))
The vocabulary is one word. node=<tag> is the YAML tag the value at that address is written under, spelled the way a document spells it, with its leading ! and its type after it. A save then writes
wait: !mycompany:duration 30s
where it would have written `wait: 30s`, whether or not the file had a tag there, and a tag the file did have at that address loses to the one declared.
It changes nothing about a load, because nothing has to. A tag this package has no reading of its own for is carried and not interpreted, so the value arrives as the text after the tag and reaches whichever codec your field declared. That is also what makes the word survive a round trip, and it is the shape of its one limit: such a value comes back as a string, so the word belongs on a field this plane writes as one.
Four things it refuses, all of them at the save's Sink.Bind except the last:
- a tag that is not a tag: one not starting with !, one naming nothing after the !, or one with a space in it
- a tag this package spells itself, !!str and !!int and the rest, because the value's own kind decides those and a declaration cannot contradict it
- the address of a struct, a slice or a map, because a node tag names how one value is written and those are places rather than values
- a value this plane writes as a number, a boolean or bytes, refused when that value is written, since a tag this plane cannot read back would make the value return as a string
A null is the one value written without the tag rather than refused: there is no value there for a node type to describe, and the address reads back null either way.
A registry that was not given this declaration reads nothing: the key is then another library's business, the tags stay in your structs, and a save writes what it always wrote.
Example ¶
ExampleExtension declares this driver's own struct tag key, so a field can say what node type its value is written as.
The document going in carries no tag at either address, and the one coming out carries the one the field declared. That is the half a save could not do before: the boundary hands a driver a value and not a Go type, so nothing in a plain `wait: 30s` said this address wanted !mycompany:duration.
It is quoted in this package's README.
package main
import (
"context"
"fmt"
"os"
"path/filepath"
"github.com/onhotpath/ferry"
"github.com/onhotpath/ferry/driver/yaml"
)
func main() {
type config struct {
Wait string `ferry:"wait" yamlext:"node=!mycompany:duration"`
Port int `ferry:"port"`
}
registry := ferry.MustRegistry(ferry.WithTagKeys(yaml.Extension()))
path := writeExamplePlane("wait: 30s\nport: 8080\n")
defer func() { _ = os.RemoveAll(filepath.Dir(path)) }()
ctx := context.Background()
cfg, err := ferry.Load[config](ctx, yaml.NewSource(path), ferry.WithRegistry(registry))
if err != nil {
fmt.Println(err)
return
}
if err := ferry.Dump(ctx, cfg, yaml.NewSink(path), ferry.WithRegistry(registry)); err != nil {
fmt.Println(err)
return
}
back, err := os.ReadFile(path)
if err != nil {
fmt.Println(err)
return
}
fmt.Print(string(back))
}
// writeExamplePlane puts the example's starting document in a directory of its
// own, because the sink stages its replacement beside the plane.
func writeExamplePlane(doc string) string {
dir, err := os.MkdirTemp("", "ferry-yaml-example")
if err != nil {
panic(err)
}
path := filepath.Join(dir, "config.yaml")
if err := os.WriteFile(path, []byte(doc), 0o600); err != nil {
panic(err)
}
return path
}
Output: wait: !mycompany:duration 30s port: 8080
Types ¶
type Option ¶
type Option interface {
// contains filtered or unexported methods
}
Option is a setting handed to NewSink. The set is closed at one: Durable.
func Durable ¶
func Durable() Option
Durable makes a save survive a crash. The replacement's contents and the rename that makes your path point at them are both flushed to the disk before ferry.Dump returns.
err := ferry.Dump(ctx, cfg, yaml.NewSink("config.yaml", yaml.Durable()))
Without it a save is still atomic: the staged file is still renamed into place, so nothing ever reads a half-written config and a save that fails still leaves your file byte for byte as it was. What it is not is written out. The replacement sits in the operating system's cache until the kernel gets around to it, and a machine that loses power in that window comes back to the old document. That is what an ordinary file write gives you and it is the default here, because the usual reason to write a config file is that something is about to read it back.
It buys that at the price of a disk flush, which is the most expensive thing a save does and is usually more expensive than everything else in the save put together. Ask for it where losing the write would matter, and not by reflex.
Windows has no way to flush a directory, so a durable save there flushes the contents and leaves the durability of the rename to the filesystem.
One sharp edge: a flush that fails once the rename has landed is a save that failed with your file already replaced. It reports ferry.ErrPlane, because what could not be promised is that the replacement survives a crash, not that it happened.
type Sink ¶
type Sink struct {
// contains filtered or unexported fields
}
Sink writes a struct's fields into a YAML file.
A save is a merge into whatever document is already at the path: the keys your struct maps are replaced, and everything else - comments, key order, and every key no field of yours maps - is left as it was. That is what makes a hand-maintained config file survive being loaded and written back.
A list or a map your struct maps is the one place the merge stops, because it is replaced whole: a three-element list saved over with one element is one element afterwards, and a mapping saved over with a map that lost a key no longer holds that key. Anything a save like that keeps is still the operator's - its comments, its anchor and the node tag it was written under all survive - and a struct's fields are untouched by the rule, since a field your value leaves out is left exactly where it is rather than removed.
A value that is a leaf rather than a struct - an int, a string, a []byte - maps one address, the document itself, and saving one replaces the whole file: a file holding "keep: me" and "other: 2" is "8080" and nothing else after saving 8080 to it. That is the same rule as the one above and not an exception to it. There is no replace-or-patch switch to set here; a save always replaces what it writes, and the root is simply the one address with nothing above it left over to keep.
An anchor is the exception, and it is deliberate. A value ferry replaces keeps the anchor you wrote on it, so a key no field maps that aliases it reads back as the value just written: its line does not change and its value does. A key that is itself an alias is written through to the anchor it names, so that line does not change either and the value lands where the two share it.
The one refusal that follows is two keys your struct maps that share an anchor, saved with different values: the file can hold only one of them, so the save fails with ferry.ErrPlane and your file is left as it was.
The write is atomic. A temporary file beside yours is renamed into place once everything has been written, and a save that fails leaves your file byte for byte as it was with no temporary left behind.
A path that is a symlink is written through rather than replaced: the file the link names is the one a save reads, stages beside and renames over, and the link itself is left exactly as it is. A link that names a file which does not exist yet is followed all the same, and the save writes the file at the end of it. The link is followed once per save, so re-pointing it between two saves sends the second one to the new file.
A save that started before somebody else edited the file refuses rather than overwriting them. Because a save is a merge into the document it read, an edit that lands between the read and the rename would be silently dropped, so the save reports ferry.ErrPlane and leaves your file as it was: load again, apply your changes to what the file holds now, and save again. The check is the file's length and modification time, so the one edit it cannot see is a rewrite in the same modification-time tick that leaves the length alone.
It is not durable unless you ask. Pass Durable to flush the replacement to the disk before the save returns, and read that option before you do: it is the most expensive thing a save can be told to do.
It is a separate type from Source for the reason recorded there.
func NewSink ¶
NewSink returns a sink over the YAML file at path.
Pass Durable to flush the replacement to the disk before a save returns. The replacement is atomic either way.
It touches nothing, and in particular it does not check that the path can be written. A sink over an unwritable directory is legal to build, and the save refuses when it starts.
func (Sink) Bind ¶
func (s Sink) Bind(addrs *ferry.AddressSet) (ferry.OpenWriterFunc, error)
Bind builds no flat key from the address set, for the reason Source.Bind records.
What it does take from the set is the node tag declared at each address, where the registry this save resolves against was given Extension. A declaration this driver cannot honour is refused here, which is before the operator's file has been opened.
It does no I/O, so a file that cannot be written is not refused here. That refusal lands when the save starts, which is before anything has been written, rather than part way through.
type Source ¶
type Source struct {
// contains filtered or unexported fields
}
Source reads a struct's fields out of a YAML file.
It is a separate type from Sink rather than the other half of one, so a round trip names the path twice:
cfg, err := ferry.Load[Config](ctx, yaml.NewSource("config.yaml"))
err = ferry.Dump(ctx, cfg, yaml.NewSink("config.yaml"))
The repetition buys the refusal being a compile error: code handed only a Source cannot save through it, and nothing has to check at run time to say so.
func NewSource ¶
func NewSource(path string, opts ...SourceOption) Source
NewSource returns a source over the YAML file at path.
It touches nothing, and starts nothing, unless it is given Watch. The file is read when a load starts, so a source over a path that does not exist yet is legal to build, and a load through it sees a file holding no keys: every field takes its default, and a required field fails.
Pass Watch to be called when the file changes underneath the source. That is the one setting that does something before a load: it takes the file's current state here and polls from a goroutine of its own until the context it was given is done.
func (Source) Bind ¶
Bind builds no flat key from the address set, because this driver walks a document tree and two fields cannot collide on a path.
What it does take from the set is the shape each section's own members have. A struct's members are named and an array's are positions, so the document has to hold a mapping at the one and a sequence at the other, and a load through this binding refuses the other way round rather than reading an empty section out of it.
It does no I/O and cannot fail. A file that does not parse is reported when the load reads it, not here.
type SourceOption ¶
type SourceOption interface {
// contains filtered or unexported methods
}
SourceOption is a setting handed to NewSource. The set is closed at one: Watch.
It is a separate type from Option so that each constructor takes the settings that mean something to it, and the other way round is a compile error rather than a setting that is quietly ignored.
func Watch ¶
Watch calls onChange whenever the file changes underneath a source, so that a process holding a loaded value can load a fresh one.
b, err := ferry.Bind[Config](yaml.NewSource(path, yaml.Watch(ctx, time.Second, reload)))
func reload(ctx context.Context) {
cfg, err := b.Load(ctx) // a reload is a load
... // publish it by replacement, never by mutation
}
It is opt-in and it is the only thing in this package that runs on a goroutine of its own. A source built without it touches the file only when a load asks it to.
The watch begins when the source is built and ends when ctx is done, which is the only way to stop it: cancel the context you gave it, and the goroutine returns. The context reaches onChange as its argument, so a deadline, a cancellation and whatever the caller put in it are all in hand there.
Looking is a stat every interval. An interval of zero or less takes one second, and a nil onChange watches nothing.
Sharp edges, and they are the reason this is a callback and not a stream.
onChange runs on the watching goroutine and one call at a time. A callback that reloads inline is a slow one, and a slow callback delays the next look rather than running beside itself, so changes that land while it runs are one call afterwards rather than several. The Changed method of a Signal from github.com/onhotpath/ferry/watch returns immediately instead, which leaves the reload on the goroutine ranging the stream and this one free to keep looking.
A panic in onChange takes the process down, exactly as it would on a goroutine the caller started. Nothing here recovers it: there is no result to hand a failure back through, and a watch that swallowed the panic would leave a process that has silently stopped reloading. Recover inside the callback if a bug there should not be fatal.
Watching starts when the source is built, so it starts before ferry.Bind has handed back the binding the callback wants to load through, and a change can land while there is nothing yet to load through. A Signal from github.com/onhotpath/ferry/watch is what to pass here in that case: its Changed method records such a change rather than losing it, and the stream that opens afterwards begins with that reload, as the example in this package does.
A call says the file may have changed and nothing more. Load to find out what it holds now, which is correct whether the change was real, coalesced with another, or a touch that rewrote the same bytes.
A dump through the same path fires it, so a process that both watches and saves its own config hears its own writes.
One change is invisible: a rewrite landing in the same modification-time tick that leaves the length alone. Looking costs a stat, and a watch that hashed the file instead would read it whole every interval to catch a case an operator's editor does not produce.
Example ¶
ExampleWatch reloads a config file whenever an operator edits it.
The binding is held across the edit, the callback loads through it, and the value loaded before the edit is untouched by the one loaded after. That is the whole of reloading: a reload is a load, and publishing one means replacing a value rather than writing into a value somebody else is reading.
A server would keep the fresh value in an atomic pointer and swap it on every turn of the range.
package main
import (
"context"
"fmt"
"os"
"path/filepath"
"time"
"github.com/onhotpath/ferry"
"github.com/onhotpath/ferry/driver/yaml"
"github.com/onhotpath/ferry/watch"
)
func main() {
type config struct {
Port int `ferry:"port"`
}
path := writeExamplePlane("# the port the server listens on\nport: 8080\n")
defer func() { _ = os.RemoveAll(filepath.Dir(path)) }()
// Cancelling is what stops the watching goroutine, and it is the only thing
// that does.
ctx, stop := context.WithCancel(context.Background())
defer stop()
// Watching starts when the source is built, which is before Bind has handed
// back the binding to load through. The signal is what keeps a change that
// lands in that window: it records one, and the stream opens with it.
s := watch.New()
b, err := ferry.Bind[config](yaml.NewSource(path, yaml.Watch(ctx, 10*time.Millisecond, s.Changed)))
if err != nil {
fmt.Println(err)
return
}
held, err := b.Load(ctx)
if err != nil {
fmt.Println(err)
return
}
// The operator's own edit, landing while the process holds a loaded value.
if err := os.WriteFile(path, []byte("# the port the server listens on\nport: 443\n"), 0o600); err != nil {
fmt.Println(err)
return
}
seq, errf := watch.Values(ctx, s, b)
for cfg := range seq {
fmt.Printf("held: %d\n", held.Port)
fmt.Printf("reloaded: %d\n", cfg.Port)
break // one turn is enough for an example; a server keeps ranging
}
if err := errf(); err != nil {
fmt.Println(err)
}
}
// writeExamplePlane puts the example's starting document in a directory of its
// own, because the sink stages its replacement beside the plane.
func writeExamplePlane(doc string) string {
dir, err := os.MkdirTemp("", "ferry-yaml-example")
if err != nil {
panic(err)
}
path := filepath.Join(dir, "config.yaml")
if err := os.WriteFile(path, []byte(doc), 0o600); err != nil {
panic(err)
}
return path
}
Output: held: 8080 reloaded: 443
Source Files