README
¶
YAMLStar RapidYAML Parser
An optional RapidYAML parser for YAMLStar and go-yaml.
The shared library is implemented entirely in C++ and embeds no Go runtime.
The plugin implements ABI v2 transforms returning EDN parser events.
It contains no Glojure runtime.
The Go adapter uses purego to load its C ABI without cgo.
Versions through 0.1.2 embedded Go and could crash when loaded by Go hosts
on Intel macOS; use 0.1.3 or newer.
It requires a YAMLStar host with the yaml-parser=rapidyaml ABI v2 registration.
Release 0.1.0 used ABI v1 and cannot be loaded by the new host.
The shared manifest now declares :api "yaml-parser".
Hosts and adapters requiring that name reject older :api "parser"
libraries; rebuild or install an updated library together with the host.
yaml --plugin=yaml-parser=rapidyaml
The implementation name is rapidyaml and the plugin API is yaml-parser.
The repository and Python distribution are yamlstar-plugin-parser-rapidyaml.
Libraries use libyamlstar-plugin-parser-rapidyaml.so on Linux and
.dylib on macOS.
The native parser accepts no parser-specific options.
The Go adapter also accepts the installation option described below.
Build and test
make build
make test
make test-conformance
make fuzz
make benchmark
Makes installs project tools into .cache.
Building the native library requires a C++11 compiler, using GCC on Linux
or Clang on macOS.
Go is needed for the adapter and test suite, including the cgo test oracle.
Run project-dependent commands with make shell CMD='...'.
The build pins rapidyaml v0.16.0 at
f8ac8dd50f4f7916579d55a05ebf9c6488e52670.
This checkout includes the c4core v0.6.0 source snapshot; it has no submodules.
The C++ wrapper converts RapidYAML integer events directly to ABI v2 EDN.
The previous Go/YEBP v1 conversion is retained as a differential test oracle.
Tests compare native event bytes and error classifications against that oracle
for the pinned YAML test suite without changing its expected results.
Adapter and archive tests reject libraries containing a Go runtime.
go-yaml integration
The goyaml package exports New(options ...Option) and Register() error
for go-yaml.
The adapter loads this plugin's ABI v2 shared library at runtime and requires
no cgo in the consuming Go executable.
Linux, FreeBSD, and macOS hosts can load manually installed libraries.
Automatic installation supports the four Linux and macOS release platforms.
import (
"github.com/yamlstar/yamlstar-plugin-parser-rapidyaml/goyaml"
"go.yaml.in/yaml/v4"
)
var value any
err := yaml.Load([]byte("answer: 42"), &value,
yaml.WithPlugin(goyaml.New()))
Call goyaml.Register() during startup before parsing named configuration.
It registers yaml-parser=rapidyaml at goyaml.Version, preserving the native
Go parser as the default.
Registration and --help do not load or download a library.
The library is resolved on the first parse with RapidYAML.
The compiled-in go-yaml parser requires no parser shared library.
The adapter requires the native library version matching its release.
If missing, it downloads that exact release's archive and SHA256SUMS from
GitHub, verifies them, and installs the library atomically.
This requires no YAMLStar executable or external download/archive commands.
Existing incompatible libraries produce an error and are preserved.
Unknown configuration options, duplicate registration, and mismatched requested
versions also return errors.
Use goyaml.New(goyaml.WithInstall(false)) to disable downloads in Go code.
For named configuration, use:
plugin:
yaml-parser:
name: rapidyaml
install: false
YAMLSTAR_LIBRARY_PATH specifies library search directories.
When unset, the adapter searches beside the executable, its ../lib directory,
$HOME/.local/lib, /usr/local/lib, and /usr/lib.
An explicitly configured path replaces those defaults, including an empty path.
Missing libraries are installed into the first writable search directory.
Select another directory with YAMLSTAR_LIBRARY_PATH when an incompatible
library already exists.
After publishing the native 0.1.3 release assets and Go module tag v0.1.3,
build a go-yaml CLI once:
make cli \
PLUGIN-PACKAGE=github.com/yamlstar/yamlstar-plugin-parser-rapidyaml/goyaml@v0.1.3
./go-yaml --plugin=yaml-parser=rapidyaml -j < input.yaml
./go-yaml --plugin=yaml-parser=go-yaml -j < input.yaml
PLUGIN-PACKAGE links the adapter; runtime --plugin selects it.
The packages-only build retains native defaults.
To select RapidYAML as the build default, add PLUGIN=yaml-parser=rapidyaml.
No parser switch requires another build.
The Go module tag and native release tag refer to the same commit.
Archives retain unprefixed versions in their names and download URLs.
make test-goyaml tests the adapter with cgo disabled.
make test-goyaml-race runs its concurrency and installation tests with the
Go race detector.
Set GO_YAML_CLI to an adapter-enabled executable to run CLI regressions too.
Installer tests use local HTTP fixtures without contacting GitHub.
make test-goyaml-cli uses GO_YAML_DIR (default ../go-yaml) to test the
actual generic PLUGIN-PACKAGE build with a local adapter snapshot.
It stages sources, a file module proxy, and its executable under
.cache/integration, leaving the supplied checkout and executable unchanged.
To try a local build from this directory:
make build
YAMLSTAR_LIBRARY_PATH="$PWD/lib" yaml --plugin=yaml-parser=rapidyaml --no-install \
--eval 'a: [1, 2, 3]'
YAMLStar and go-yaml can share that same library directory.
For this coordinated workspace, build YAMLStar's CLI and shared host library
from the workspace root with make cli build-libyamlstar, then build go-yaml
with the wanted adapter packages once.
Until the adapters are published, stage local module snapshots under .cache
with an isolated file Go proxy and module cache, and supply their explicit
versions through PLUGIN-PACKAGE.
Keep local staging separate from public release commands.
Python
pip install yamlstar-plugin-parser-rapidyaml
import yamlstar
opts = yamlstar.Options().plugin(
yamlstar.parser('rapidyaml'))
ys = yamlstar.YAMLStar(opts)
assert ys.load('a: [1, 2, 3]') == {'a': [1, 2, 3]}
Wheels expose the parser-rapidyaml entry point in yamlstar.plugins.
They contain the ABI v2 shared library and manifest.
Other bindings select the same parser through their existing options mapping.
Integration tests exercise the Glojure CLI and Python binding.
Conformance
make test-conformance compares binary events against 402 fixtures in the
YAML test suite at 6ad3d2c62885d82fc349026c136ef560838fdf3d.
The baseline records nine upstream tab-separation differences: three valid
inputs rejected, three invalid inputs accepted, and three event mismatches.
See test/expected-events.txt for individual cases and explanations.
New differences and unexpected improvements fail the test for review.
The adapter does not fall back to another parser.
Tests cover scalar styles, tags, directives, explicit document markers,
anchors, aliases, buffer growth, malformed input, ownership, and concurrent
calls through the C ABI.
Input buffers are limited to less than 512 MiB by the integer-event adapter.
Output is owned by the plugin and released with yamlstar_plugin_v2_free.
Status 0 returns an EDN event vector.
Status 1 reports parse/configuration errors; status 2 reports ABI/internal errors.
Manifests use EDN; failures return plain UTF-8 error messages.
Performance
The initial full CLI comparison is slower than built-in go-yaml on the
three measured fixtures.
See Performance.md for results and reproduction commands.
Those historical measurements used ABI v1 binary transport.
The current ABI v2 adapter converts the internal binary events to EDN.
Assertions are disabled in all builds; parse-error handling remains enabled.
Release
See Releasing.md for PyPI trusted-publisher setup and recovery from a failed wheel publication.
make dist
make wheel
make test-wheel
make release v=0.1.3 d=1
make release v=0.1.3
Release archives use
yamlstar-plugin-parser-rapidyaml-VERSION-PLATFORM.tar.xz.
The release workflow builds Linux x64/aarch64 with a manylinux 2.28 baseline,
and macOS x64/arm64, validates archives and wheels, and publishes checksums.
The repository must exist under yamlstar/yamlstar-plugin-parser-rapidyaml.
Configure PyPI trusted publishing for distribution
yamlstar-plugin-parser-rapidyaml, workflow release.yaml, environment pypi.
Release from main, or pass a=1 to release from another branch.
The release helper creates both VERSION and vVERSION tags at the same
commit, checks them for conflicts, and updates the adapter's version together
with the native manifest and build metadata.
GitHub assets remain attached to the bare native tag.
The Go tag makes goyaml importable at a tagged module version.
Retries preserve both tags; changes after tagging require a new version.
Publish the matching native assets before advertising the adapter release.
No release or publication happens during normal builds or tests.
The integration workflow uses the YAMLStar branch selected by the repository
variable YAMLSTAR_REF, defaulting to a pinned main revision with ABI v2
RapidYAML support.
The go-yaml CLI integration uses a pinned host with the canonical parser API.
Third-party notices are included in licenses/ and release artifacts.
Documentation
Source Files
Directories