README
¶
xtoproto (csvtoproto, xmltoproto, etc.)
xtoproto is a library for automatically (1) inferring a protocol buffer
definition (a .proto file) from XML and CSV files, and (2) generating
runtime code that translates XML and CSV files to proto using the mappings
from (1).
Quickstart
The best way to get started is to try out xtoproto using the interactive, web-based playground hosted on Github.
More details about how to use xtoproto will be added soon.
Building
The project is buildable with Bazel and go build. Bazel is recommended because
the files needed for go build are only present in the release branches of the
repository (v0.0.6, v0.0.5, etc.).
bazel build //...
Playground
Try out xtoproto using the interactive, web-based playground hosted on Github. The playground uses a WebAssembly version of xtoproto and does not transmit the input example data to a remote server. Alternatively, you may start the playground on your workstation with this command, then navigate to http://localhost:8888/
bazel run //cmd/xtoproto_web -- --addr ":8888"
Development
gopls does not yet work with
bazel. In the meantime, it is
convenient to generate the .pb.go files used within this project so that gopls
can pick them up and make autocomplete work. To do this, issue the following
command from the root of the checked out xtoproto repository:
bazel run //releasing/generate_pb_go_files -- -output_dir $PWD/proto --alsologtostderr
Releasing
There is a script for generating the release. Run it from the a cloned repository with the following command.
git remote add google git@github.com:google/xtoproto.git
bazel run //releasing/make_release -- --workspace $PWD --branch_suffix v006c --tag v0.0.6
Disclaimer
This is not an official Google product.
This repository was created on June 29, 2020. We are incrementally migrating the code onto Github, and the project will not be functional until that migration is finished. This README will be updated with instructions about how to use the project once the migration is complete. In the mean time, feel free to browse the code.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
xtoproto
Program xtoproto infers .proto definitions from record-oriented files (CSV, XML, etc.).
|
Program xtoproto infers .proto definitions from record-oriented files (CSV, XML, etc.). |
|
Package codegen contains Go code generation facilities.
|
Package codegen contains Go code generation facilities. |
|
Package csvcoder decodes Go types from CSV records using struct tags, similar to encoding/csv and encoding/xml packages in the standard library.
|
Package csvcoder decodes Go types from CSV records using struct tags, similar to encoding/csv and encoding/xml packages in the standard library. |
|
Package csvinfer guesses the types of CSV columns and uses these to generate a CSVProtoMapping object that in turn may be used to generate a .proto definition and CSV-to-proto parser.
|
Package csvinfer guesses the types of CSV columns and uses these to generate a CSVProtoMapping object that in turn may be used to generate a .proto definition and CSV-to-proto parser. |
|
Package csvtoproto generates a .proto file and a .go file from a go/csv-to-proto mapping file.
|
Package csvtoproto generates a .proto file and a .go file from a go/csv-to-proto mapping file. |
|
Package csvtoprotoparse contains runtime functionality needed by code generated by the go/csv-to-proto tool.
|
Package csvtoprotoparse contains runtime functionality needed by code generated by the go/csv-to-proto tool. |
|
internal
|
|
|
protoreflectcmp
Package protoreflectcmp provides testing facilites for using the cmp package with protoreflect.
|
Package protoreflectcmp provides testing facilites for using the cmp package with protoreflect. |
|
Program playground is a web application intended to be compiled with WASM that infers .proto definitions from record-oriented files (CSV, XML, etc.).
|
Program playground is a web application intended to be compiled with WASM that infers .proto definitions from record-oriented files (CSV, XML, etc.). |
|
proto
|
|
|
Package protocp transforms one record-oriented format into another record-oriented format where the records are protocol buffers.
|
Package protocp transforms one record-oriented format into another record-oriented format where the records are protocol buffers. |
|
rdf
|
|
|
iri
Package iri contains facilities for working with Internationalized Resource Identifiers as specified in RFC 3987.
|
Package iri contains facilities for working with Internationalized Resource Identifiers as specified in RFC 3987. |
|
ntriples
Package ntriples parses the RDF triples formatted according to the W3C N-Triples format.
|
Package ntriples parses the RDF triples formatted according to the W3C N-Triples format. |
|
rdfxml
Package rdfxml parses RDF/XML documents according to the W3C standard.
|
Package rdfxml parses RDF/XML documents according to the W3C standard. |
|
rdfxml/generate_test_cases
Program generate_test_cases outputs a .go file with test cases based on the official w3c test case repository.
|
Program generate_test_cases outputs a .go file with test cases based on the official w3c test case repository. |
|
rdfxml/rdftestcases
Package rdftestcases contains go structs with the contents of the W3C's official RDF parser test cases.
|
Package rdftestcases contains go structs with the contents of the W3C's official RDF parser test cases. |
|
Package recordinfer guesses the types of record columns and uses these to generate a RecordProtoMapping object that in turn may be used to generate a .proto definition and record-to-proto parser.
|
Package recordinfer guesses the types of record columns and uses these to generate a RecordProtoMapping object that in turn may be used to generate a .proto definition and record-to-proto parser. |
|
releasing
|
|
|
generate_pb_go_files
Program generate_pb_go_files collects the generated go files from the bazel runfiles directory that match a given prefix and outputs those files to a destination directory; this may be used when .pb.go artifacts are needed to build xtoproto without bazel.
|
Program generate_pb_go_files collects the generated go files from the bazel runfiles directory that match a given prefix and outputs those files to a destination directory; this may be used when .pb.go artifacts are needed to build xtoproto without bazel. |
|
make_release
Program make_release helps make a release of xtoproto.
|
Program make_release helps make a release of xtoproto. |
|
Package service implements the XToProtoService gRPC service.
|
Package service implements the XToProtoService gRPC service. |
|
Package sexpr parses s-expressions in a manner similar to the Common Lisp reader.
|
Package sexpr parses s-expressions in a manner similar to the Common Lisp reader. |
|
form
Package form is an API for working with S-Expression objects.
|
Package form is an API for working with S-Expression objects. |
|
Package textcoder defines a registry of Go types and associated textual encoding/decoding functions.
|
Package textcoder defines a registry of Go types and associated textual encoding/decoding functions. |
|
Package textpos provides types and functions for working with line-based positions of text in a textual document.
|
Package textpos provides types and functions for working with line-based positions of text in a textual document. |
|
Package wirepath is an xpath-like means of representing a location within a protocol buffer message.
|
Package wirepath is an xpath-like means of representing a location within a protocol buffer message. |
|
Package xmlinfer attempts to infer protocol buffer definitions from a set of XML examples.
|
Package xmlinfer attempts to infer protocol buffer definitions from a set of XML examples. |