Documentation
¶
Overview ¶
Package eventlog reads the event log that progress.Log writes, version 1 of the format doc/event-log.md describes: what a view of the runs a program made reads, and what a server reads back from a command it ran as a child, to send on the child's progress.
Decode turns one line of a log back into what was written: the first line of a run, or one of its events as a progress.Event, with every text sanitised as a Bus sanitises it, so that a log something else wrote cannot act on a terminal or a page either.
A Tail reads a log as it grows, polled, and follows it as progress/cliprogress rotates it, and gives the whole lines it reads to a Feed. Forward is the Feed of a server that runs a command as a child, with the child's log in a file of its own: it gives the child's events to a progress.Sink, such as an mcpprogress.Notifier, once each. A Reducer is the Feed of a view of what a program did, as it does it: it folds the lines into runs, each the tree of the spans of its command.
A reader of version 1 leaves out the keys and the values it does not know, as the format asks, so that a log a later release of the kit writes, with a key or a value added, still reads: a key Decode does not know is not read, a value of a kind, a state, a status, a class, a unit, a stream or a flag it does not know is left out, and a line of a type it does not know is not decoded at all.
Index ¶
Examples ¶
Constants ¶
const MaxLine = 1 << 20
MaxLine is the longest line a Tail reads, in bytes, its newline included; a longer one is left out, whole, and the line after it read. The log writes no line nearly as long: every text of an event has its bound.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Feed ¶
type Feed interface {
// Line takes one whole line of the log, without its newline. b is
// the Tail's, and only good until Line returns.
Line(b []byte)
// Reset forgets every line taken: the log was cut short or made
// again, and is read again from its start, the rotated copy first.
Reset()
}
Feed takes the lines a Tail reads.
func Forward ¶
Forward returns a Feed that gives a sink the events of the lines it takes: what a server that runs a command as a child, with the child's event log in a file of its own, does to send on the child's progress, to an mcpprogress.Notifier for instance.
The Feed decodes each line, and leaves out those Decode refuses. It gives the sink each event once, in the order of its run's Seq: an event of a run whose Seq is not past the last it gave, as the log read again after a Reset has, is left out, so that Reset changes nothing. A sink that is a progress.TraceSink is told of each run once, at the first line of the run, before the run's events. The events of every run the log holds go to the sink; a log of its own for the child holds one.
The sink is called from the goroutine that polls the Tail, never while a Bus is locked, and is given the events as Decode gives them: without the text of a line of output, and with only the advances the log wrote.
Example ¶
A server runs a tool call's command as a child, with the child's event log in a file of its own, and sends on the child's progress as it polls the log, to an mcpprogress.Notifier. Here the child's Bus runs in the example, and the server polls once, after it has ended; a server polls every so often while the child runs, and once more after it.
package main
import (
"context"
"fmt"
"os"
"path/filepath"
"github.com/GSI-HPC/go-clikit/fanout"
"github.com/GSI-HPC/go-clikit/progress"
"github.com/GSI-HPC/go-clikit/progress/eventlog"
"github.com/GSI-HPC/go-clikit/progress/mcpprogress"
)
func main() {
dir, err := os.MkdirTemp("", "child")
if err != nil {
panic(err)
}
defer func() { _ = os.RemoveAll(dir) }()
path := filepath.Join(dir, "progress.jsonl")
// The child, run with its event log in path.
f, err := os.Create(path)
if err != nil {
panic(err)
}
log := progress.NewLog(f, progress.LogOptions{})
bus := progress.NewBus(progress.BusOptions{Sinks: []progress.Sink{log}, Program: "prog"})
fanout.Map(progress.WithBus(context.Background(), bus), []string{"exe1", "exe2", "exe3"},
fanout.MapOptions[string]{Step: "probe the nodes"},
func(context.Context, string) (struct{}, error) { return struct{}{}, nil })
bus.Close()
if err := log.Close(); err != nil {
panic(err)
}
_ = f.Close()
// The server; Send would send the notification on the call's
// session, and here keeps the last.
var last string
n := mcpprogress.New(mcpprogress.Options{
Program: "prog",
Send: func(progress, total float64, message string) error {
last = fmt.Sprintf("%v/%v %s", progress, total, message)
return nil
},
})
tail := eventlog.NewTail(path, eventlog.Forward(n))
if _, err := tail.Poll(); err != nil {
panic(err)
}
n.Close()
fmt.Println(last)
}
Output: 3/3 probe the nodes: 3/3 done
type Line ¶
type Line struct {
// Run names the run the line belongs to, the same on every line of
// the run: the lines of several runs may be in one log.
Run string
// Trace is the trace the run's spans belong to. On the first line of
// a run it says too where the trace came from, when another program
// handed it on; on an event, only its Trace is set.
Trace progress.TraceContext
// Begin says the line is the first of its run; Program and Version
// are then the program that wrote it and its version, and Event is
// zero.
Begin bool
Program, Version string
// Event is the event the line holds, as the Bus sent it to the log,
// but for the text of a line of output, which the log never holds.
Event progress.Event
// LeftOut counts the advances of the span the log did not write: on
// an advance, those since the span's previous advance it wrote, and
// on an end, all of the span's.
LeftOut int
}
Line is one line of an event log, decoded: the first line of a run, which says what the run is, or one of its events.
func Decode ¶
Decode reads one line of an event log, with or without its newline. It fails on a line that is not a JSON object, one of another version of the format than progress.LogVersion, one without a run, one of a type it does not know, an event without a time, and an event of a span without a span id, or with an id, of the span or of its parent, that is not 16 hexadecimal digits, not all zero. A key whose value is of the wrong type is left out, and the rest of the line read. The texts of the errors are no interface: a reader that skips the lines it cannot read needs none.
The texts of an event are sanitised and cut to their bounds as the Bus does it (see progress.Sanitize), and so are the run, the program and the version; a tracestate that progress.ParseTraceContext would not keep is left out, and so is a count, a size or a timeout below zero.
Example ¶
A reader decodes each line of a log, and skips those it cannot read.
package main
import (
"fmt"
"github.com/GSI-HPC/go-clikit/progress"
"github.com/GSI-HPC/go-clikit/progress/eventlog"
)
func main() {
for _, s := range []string{
`{"v":1,"type":"trace","run":"0123456789abcdef","trace":"4bf92f3577b34da6a3ce929d0e0e4736","program":"prog","version":"v1.2.3"}`,
`{"v":1,"run":"0123456789abcdef","trace":"4bf92f3577b34da6a3ce929d0e0e4736","seq":1,"time":"2026-09-25T12:00:00.000000000Z","type":"start","span":"b7e3a1c09d2f4e10","kind":"command","name":"exec","state":"running"}`,
`{"v":2,"run":"0123456789abcdef","type":"start"}`,
`{"v":1,"run":"0123456789abcdef","trace":"4bf92f3577b34da6a3ce929d0e0e4736","seq":2,"time":"2026-09-25T12:00:03.000000000Z","type":"end","span":"b7e3a1c09d2f4e10","kind":"command","name":"exec","state":"ended","status":"failed","class":"transport","err":"no route to host"}`,
} {
l, err := eventlog.Decode([]byte(s))
switch {
case err != nil:
fmt.Println("skipped:", err)
case l.Begin:
fmt.Println(l.Run, l.Program, l.Version)
case l.Event.Type == progress.TypeEnd:
e := l.Event
fmt.Println(e.Seq, e.Type, e.Kind, e.Name, e.Status, e.Class, e.Err)
default:
e := l.Event
fmt.Println(e.Seq, e.Type, e.Kind, e.Name, e.State)
}
}
}
Output: 0123456789abcdef prog v1.2.3 1 start command exec running skipped: a line of version 2 of the event log, not 1 2 end command exec failed transport no route to host
type Options ¶
type Options struct {
// Keep is how many runs the Reducer keeps: the newest, and of the
// older ones those that have not ended. 0 is 100.
Keep int
// MaxSpans is how many spans of a run the Reducer keeps; the spans
// that start after them are left out, and so are their events. 0 is
// 131072.
MaxSpans int
}
Options configure a Reducer.
type Reducer ¶
type Reducer struct {
// contains filtered or unexported fields
}
Reducer folds the lines of event logs into runs, each the tree of the spans of its command, for a view of what a program did, as it does it. The lines of several runs may come mixed, as when commands append to one log at once. A Reducer is a Feed, for a Tail. It is not safe for concurrent use.
Example ¶
A view of what a program did follows its event log with a Tail, folds it into runs with a Reducer, and draws the runs that changed. Here a Bus writes the log in the example, and the view polls once.
package main
import (
"context"
"errors"
"fmt"
"os"
"path/filepath"
"github.com/GSI-HPC/go-clikit/fanout"
"github.com/GSI-HPC/go-clikit/progress"
"github.com/GSI-HPC/go-clikit/progress/eventlog"
)
func main() {
dir, err := os.MkdirTemp("", "view")
if err != nil {
panic(err)
}
defer func() { _ = os.RemoveAll(dir) }()
path := filepath.Join(dir, "activity.jsonl")
// A command of the program, which appends to its log.
f, err := os.Create(path)
if err != nil {
panic(err)
}
log := progress.NewLog(f, progress.LogOptions{Run: "0123456789abcdef", Version: "v1.2.3"})
bus := progress.NewBus(progress.BusOptions{Sinks: []progress.Sink{log}, Program: "prog"})
ctx, cmd := progress.Start(progress.WithBus(context.Background(), bus), progress.KindCommand, "status")
fanout.Map(ctx, []string{"exe1", "exe2"}, fanout.MapOptions[string]{Step: "probe the nodes"},
func(_ context.Context, node string) (struct{}, error) {
if node == "exe2" {
return struct{}{}, errors.New("no route to host")
}
return struct{}{}, nil
})
cmd.End(nil)
bus.Close()
if err := log.Close(); err != nil {
panic(err)
}
_ = f.Close()
// The view.
red := eventlog.NewReducer(eventlog.Options{})
tail := eventlog.NewTail(path, red)
if _, err := tail.Poll(); err != nil {
panic(err)
}
fmt.Println(red.Changed())
for _, run := range red.Runs() {
fmt.Println(run.Program, run.Version, run.Root.Name, run.Root.Status)
for _, step := range run.Root.Children {
fmt.Println(" ", step.Kind, step.Name, step.Status, step.Err)
for _, target := range step.Children {
fmt.Println(" ", target.Kind, target.Node, target.Status)
}
}
}
}
Output: [0123456789abcdef] prog v1.2.3 status ok step probe the nodes failed 1 of 2 failed: exe2 target exe1 ok target exe2 failed
func NewReducer ¶
NewReducer returns a Reducer that holds no run.
func (*Reducer) Changed ¶
Changed returns the IDs of the runs that changed, or were dropped, since the last call, the oldest run first, and forgets them.
func (*Reducer) Line ¶
Line folds one line of a log in. A line Decode refuses is left out, and so is a line of a run the Reducer has dropped, and an event of a span that has ended.
func (*Reducer) Reset ¶
func (r *Reducer) Reset()
Reset forgets every run, as when the log was cut short and is read again from its start; Changed reports each of them.
func (*Reducer) Runs ¶
Runs returns the runs that have a command, the newest first. The Reducer never changes a Run it returned, but builds a run anew once it changes, so a Run may be kept and read from another goroutine; Runs returns the same Run again while it has not changed, its children shared, so a caller does not change what it was given.
type Run ¶
type Run struct {
// ID names the run, as the log does.
ID string
// Trace is the trace the run's spans belong to, and where it came
// from, as the run's first line says.
Trace progress.TraceContext
// Program and Version are the program that wrote the run and its
// version, as the run's first line says.
Program, Version string
// Root is the run's command, the first span of the kind
// progress.KindCommand, with the spans below it.
Root Span
// Last is the time of the run's latest event. A run that has not
// ended and has not changed for long was most likely killed: it will
// not end.
Last time.Time
// Suspended says the displays of the run are off the terminal, while
// the command asks a question there.
Suspended bool
}
Run is one run of a log as a Reducer folds it: what its first line says, and the tree of the spans of its command.
type Span ¶
type Span struct {
// ID is the span, and Parent the span it was started under; zero for
// the root.
ID, Parent progress.SpanID
// Kind, Name, Flags and State are the span's, as its latest event
// says.
Kind progress.Kind
Name string
Flags progress.Flags
State progress.State
// Fields are the span's fields, as its latest event says. Exit, when
// set, points to a value that is never written through.
progress.Fields
// Status, Class and Err say how the span ended.
Status progress.Status
Class progress.Class
Err string
// Queued is when the span started, queued or not; Started when its
// work started, zero while it is queued; Ended when it ended, zero
// while it has not.
Queued, Started, Ended time.Time
// Lines counts the lines of output the span had while a display
// asked for them, whose text the log never holds, and Dropped those
// that were not sent.
Lines, Dropped int
// Children are the spans started under this one, in the order they
// started. A span whose parent the log does not hold, such as one
// left out by Options.MaxSpans, is the root's.
Children []Span
}
Span is a span as its events left it.
type Tail ¶
type Tail struct {
// contains filtered or unexported fields
}
Tail reads an event log as it grows, polled: each Poll gives a Feed the whole lines written since the one before, and keeps a line still being written for the next. It is not safe for concurrent use.
It follows the log as progress/cliprogress rotates it, to its name with ".1" after it (decision 31). The first Poll reads that copy before the log, so that the runs before the last rotation count too, and every Poll after it what was written to the copy since, while it is the file it was: a command that had the log open when another rotated it goes on writing there. When the log has been moved to the copy, Poll reads the rest of it there and starts the new log from its start, and the Feed sees each line once. When the log is anything else than it was, a file shorter than what was read, one that starts with other bytes, or another file that is not the copy, Poll resets the Feed and reads the copy and the log again from their starts. A log that is missing is an empty one.
func (*Tail) Poll ¶
Poll gives the Feed the whole lines written since the last Poll, as the Tail's comment says, and reports whether it gave it any or reset it. A log or a copy that is missing is no error; one that cannot be opened or read is, and Poll then gives the Feed what it read before the error, and tries again on the next call.
Source Files