message

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 7, 2026 License: Apache-2.0 Imports: 9 Imported by: 1

Documentation

Overview

ClassAd serialization for HTCondor CEDAR protocol Based on HTCondor's classad_oldnew.cpp implementation

Package message provides serialization and deserialization of CEDAR protocol messages and frames.

This package implements the type serialization system used by HTCondor's CEDAR protocol, based on stream.cpp implementation.

Key concepts: - Frame: A single CEDAR protocol frame with fixed maximum size - Message: A logical message that may span multiple frames - Stream: Handles frame-level protocol and encryption

The type serialization follows HTCondor's exact binary format: - Integers: 64-bit values in network (big-endian) byte order - Doubles: Split into fractional and exponential parts for portability - Strings: Null-terminated with optional encryption length prefix - All types support bidirectional encoding/decoding

Index

Constants

View Source 
const (
	// FRAC_CONST used for double encoding - must match HTCondor's value
	FracConst = 2147483647 // 2^31 - 1
	// BinNullChar used to represent NULL strings
	BinNullChar = '\255'
	// IntSize is the number of bytes for integers on the wire
	IntSize = 8 // HTCondor sends 64-bit integers
	// MaxFrameSize maximum size for a single frame payload
	MaxFrameSize = 1024 * 1024 // 1MB frames
	// TargetFrameSize optimal frame size for network efficiency
	TargetFrameSize = 16 * 1024 // 16KB frames
)

Constants from HTCondor stream.cpp

Variables

This section is empty.

Functions

func ClassAdAttributeIsPrivateAny 

func ClassAdAttributeIsPrivateAny(name string) bool

ClassAdAttributeIsPrivateAny checks if an attribute is private (V1 or V2)

func ClassAdAttributeIsPrivateV1 

func ClassAdAttributeIsPrivateV1(name string) bool

ClassAdAttributeIsPrivateV1 checks if an attribute is private (V1) Based on HTCondor's ClassAdAttributeIsPrivateV1 function

func ClassAdAttributeIsPrivateV2 

func ClassAdAttributeIsPrivateV2(name string) bool

ClassAdAttributeIsPrivateV2 checks if an attribute is private (V2) Based on HTCondor's ClassAdAttributeIsPrivateV2 function

Types

type CodingDirection 

type CodingDirection int

CodingDirection represents the stream direction (encode vs decode) 

const (
	CodingEncode CodingDirection = iota
	CodingDecode
	CodingUnknown
)

type ErrStringSizeExceeded 

type ErrStringSizeExceeded struct {
	Length  int
	MaxSize int
}

ErrStringSizeExceeded is returned when a string exceeds the maximum allowed size The error includes the actual length and the maximum size limit

func (*ErrStringSizeExceeded) Error 

func (e *ErrStringSizeExceeded) Error() string

type Frame 

type Frame struct {
	// contains filtered or unexported fields
}

Frame represents a single CEDAR protocol frame containing raw bytes only Use Message for data serialization operations

func NewFrame 

func NewFrame() *Frame

NewFrame creates a new empty frame for writing

func NewFrameFromBytes 

func NewFrameFromBytes(data []byte) *Frame

NewFrameFromBytes creates a frame from existing bytes for reading

func (*Frame) Bytes 

func (f *Frame) Bytes() []byte

Bytes returns the message content as bytes

func (*Frame) GetBytes 

func (f *Frame) GetBytes(data []byte) (int, error)

GetBytes reads raw bytes from the message

func (*Frame) Len 

func (f *Frame) Len() int

Len returns the current message length

func (*Frame) PutBytes 

func (f *Frame) PutBytes(data []byte) (int, error)

PutBytes writes raw bytes to the message

func (*Frame) Reset 

func (f *Frame) Reset()

Reset clears the message buffer

func (*Frame) WriteTo 

func (f *Frame) WriteTo(w io.Writer) (int64, error)

WriteTo writes the frame content to a writer

type HTCondorVersion 

type HTCondorVersion struct {
	Major int
	Minor int
	Patch int
}

HTCondorVersion represents a HTCondor version for compatibility checks

func NewHTCondorVersion 

func NewHTCondorVersion(major, minor, patch int) *HTCondorVersion

NewHTCondorVersion creates a new version

func (*HTCondorVersion) BuiltSinceVersion 

func (v *HTCondorVersion) BuiltSinceVersion(major, minor, patch int) bool

BuiltSinceVersion checks if this version is >= the specified version

type Message 

type Message struct {
	// contains filtered or unexported fields
}

Message represents a streaming CEDAR protocol message that may span multiple frames Messages automatically read additional frames from the Stream as needed

func NewMessageForStream 

func NewMessageForStream(stream StreamInterface) *Message

NewMessageForStream creates a new message for writing to a stream

func NewMessageFromStream 

func NewMessageFromStream(stream StreamInterface) *Message

NewMessageFromStream creates a new message for reading from a stream

func (*Message) CodeChar 

func (m *Message) CodeChar(ctx context.Context, c *byte) error

func (*Message) CodeDouble 

func (m *Message) CodeDouble(ctx context.Context, value *float64) error

func (*Message) CodeFloat 

func (m *Message) CodeFloat(ctx context.Context, value *float32) error

func (*Message) CodeInt 

func (m *Message) CodeInt(ctx context.Context, value *int) error

func (*Message) CodeInt32 

func (m *Message) CodeInt32(ctx context.Context, value *int32) error

func (*Message) CodeInt64 

func (m *Message) CodeInt64(ctx context.Context, value *int64) error

func (*Message) CodeString 

func (m *Message) CodeString(ctx context.Context, s *string) error

func (*Message) FinishMessage 

func (m *Message) FinishMessage(ctx context.Context) error

FinishMessage completes the message by sending any remaining data with EOM flag

func (*Message) Finished 

func (m *Message) Finished() bool

Finished returns true if the message has been completely read

func (*Message) FlushFrame 

func (m *Message) FlushFrame(ctx context.Context, isEOM bool) error

FlushFrame sends the current buffer as a frame (for encoding)

func (*Message) GetBytes 

func (m *Message) GetBytes(ctx context.Context, numBytes int) ([]byte, error)

GetBytes reads the specified number of raw bytes from the message Reads across frame boundaries as needed

func (*Message) GetChar 

func (m *Message) GetChar(ctx context.Context) (byte, error)

GetChar reads a single byte, possibly across frame boundaries

func (*Message) GetClassAd 

func (m *Message) GetClassAd(ctx context.Context) (*classad.ClassAd, error)

GetClassAd reads a ClassAd from the streaming message

func (*Message) GetClassAdRaw added in v0.2.0 

func (m *Message) GetClassAdRaw(ctx context.Context) (string, error)

getClassAdFromMessageWithMaxSize reads a ClassAd from a Message with size limits maxSize limits the total bytes read for the entire ClassAd across all strings If maxSize is 0 or negative, no size limit is applied GetClassAdRaw reads a ClassAd off the wire and returns it as old-ClassAd text (newline-separated `Name = Value` lines, with MyType/TargetType appended as lines) WITHOUT parsing it into a *classad.ClassAd. The bytes consumed are identical to GetClassAd; only the AST-building step is skipped.

This is the efficient ingest path for callers that immediately re-encode the ad -- for example into a classad collections.Collection via UpdateOld, which streams the text straight to its wire form -- avoiding an AST that would only be discarded.

func (*Message) GetClassAdRawBody added in v0.2.0 

func (m *Message) GetClassAdRawBody(ctx context.Context, numExprs int) (string, error)

GetClassAdRawBody reads the body of a raw ClassAd whose leading expression count has already been consumed. This lets a caller reading a stream of ads on one connection first inspect that leading integer -- e.g. to tell a real ad (a small expression count) from a DC_NOP end-of-batch marker -- before committing to reading an ad. See GetClassAdRaw.

func (*Message) GetClassAdWithMaxSize 

func (m *Message) GetClassAdWithMaxSize(ctx context.Context, maxSize int) (*classad.ClassAd, error)

GetClassAdWithMaxSize reads a ClassAd from the streaming message with size limits maxSize limits the total bytes read for the entire ClassAd

func (*Message) GetDouble 

func (m *Message) GetDouble(ctx context.Context) (float64, error)

GetDouble reads a double using HTCondor's frexp/ldexp decoding, possibly across frame boundaries

func (*Message) GetFloat 

func (m *Message) GetFloat(ctx context.Context) (float32, error)

GetFloat reads a float, possibly across frame boundaries

func (*Message) GetInt 

func (m *Message) GetInt(ctx context.Context) (int, error)

GetInt reads an int from 64-bit value, possibly across frame boundaries

func (*Message) GetInt32 

func (m *Message) GetInt32(ctx context.Context) (int32, error)

GetInt32 reads an int32 from 64-bit value, possibly across frame boundaries

func (*Message) GetInt64 

func (m *Message) GetInt64(ctx context.Context) (int64, error)

GetInt64 reads an int64, possibly across frame boundaries

func (*Message) GetString 

func (m *Message) GetString(ctx context.Context) (string, error)

GetString reads a null-terminated string, possibly across frame boundaries

func (*Message) GetStringWithMaxSize 

func (m *Message) GetStringWithMaxSize(ctx context.Context, maxSize int) (string, error)

GetStringWithMaxSize reads a null-terminated string with a maximum size limit If the string exceeds maxSize, returns the truncated string (up to maxSize-1 bytes) along with an ErrStringSizeExceeded error to allow the caller to detect size violations For encrypted mode: null characters are preserved in the returned string For unencrypted mode: null characters terminate the string (as per protocol) If maxSize is 0 or negative, returns an empty string without reading any data SECURITY: Never buffers more than maxSize bytes to prevent DoS attacks

func (*Message) GetUint32 

func (m *Message) GetUint32(ctx context.Context) (uint32, error)

GetUint32 reads a uint32 from 64-bit value, possibly across frame boundaries

func (*Message) IsDecode 

func (m *Message) IsDecode() bool

IsDecode returns true if in decode mode

func (*Message) IsEncode 

func (m *Message) IsEncode() bool

IsEncode returns true if in encode mode

func (*Message) PutBytes 

func (m *Message) PutBytes(ctx context.Context, data []byte) error

PutBytes writes raw bytes to the message without length prefix Flushes frame if needed to accommodate the data For data larger than MaxFrameSize, splits across multiple frames

func (*Message) PutChar 

func (m *Message) PutChar(ctx context.Context, c byte) error

PutChar writes a single byte

func (*Message) PutClassAd 

func (m *Message) PutClassAd(ctx context.Context, ad *classad.ClassAd) error

PutClassAd writes a ClassAd to the streaming message

func (*Message) PutClassAdRaw added in v0.2.0 

func (m *Message) PutClassAdRaw(ctx context.Context, exprs []string, myType, targetType string) error

PutClassAdRaw writes a ClassAd to the wire from pre-rendered old-ClassAd expression strings ("Attr = Value") plus its MyType/TargetType values, WITHOUT a *classad.ClassAd. It is the send-side analogue of GetClassAdRaw: a caller that already holds the ad in a compact stored form (e.g. a collector streaming query results) can render the expression text directly and skip building an AST. The bytes written are identical to PutClassAd on the equivalent ad -- the expression count, each "Attr = Value" string, then MyType and TargetType.

func (*Message) PutClassAdRawBytes added in v0.2.0 

func (m *Message) PutClassAdRawBytes(ctx context.Context, exprs [][]byte, myType, targetType string) error

PutClassAdRawBytes is PutClassAdRaw with each "Attr = Value" expression given as a byte slice instead of a string, so a caller can render all of an ad's expressions into one reused buffer and stream them out with no per-expression allocation. The exprs slices are not modified and may alias a shared buffer.

func (*Message) PutClassAdWithOptions 

func (m *Message) PutClassAdWithOptions(ctx context.Context, ad *classad.ClassAd, config *PutClassAdConfig) error

PutClassAdWithOptions writes a ClassAd with options to the streaming message

func (*Message) PutDouble 

func (m *Message) PutDouble(ctx context.Context, value float64) error

PutDouble writes a double using HTCondor's frexp/ldexp encoding

func (*Message) PutFloat 

func (m *Message) PutFloat(ctx context.Context, value float32) error

PutFloat writes a float as double

func (*Message) PutInt 

func (m *Message) PutInt(ctx context.Context, value int) error

PutInt writes an int as 64-bit value

func (*Message) PutInt32 

func (m *Message) PutInt32(ctx context.Context, value int32) error

PutInt32 writes an int32 as 64-bit value

func (*Message) PutInt64 

func (m *Message) PutInt64(ctx context.Context, value int64) error

PutInt64 writes an int64

func (*Message) PutString 

func (m *Message) PutString(ctx context.Context, s string) error

PutString writes a string with null terminator If the string contains a null character, it is truncated at the first null and a null terminator is still appended

func (*Message) PutStringBytes added in v0.2.0 

func (m *Message) PutStringBytes(ctx context.Context, b []byte) error

PutStringBytes writes a byte slice as a CEDAR string (the wire form is identical to PutString(string(b))), but without allocating a string from b. It lets a caller stream many strings out of one reused scratch buffer -- e.g. a collector rendering each ad's "Attr = Value" expressions into a shared buffer -- with no per-string allocation. b is not modified.

func (*Message) PutUint32 

func (m *Message) PutUint32(ctx context.Context, value uint32) error

PutUint32 writes a uint32 as 64-bit value

func (*Message) SkipClassAdRaw added in v0.2.0 

func (m *Message) SkipClassAdRaw(ctx context.Context) error

SkipClassAdRaw reads and discards one raw ClassAd off the stream -- the leading expression count, that many expression strings, then MyType and TargetType -- without allocating any of its text. It is the drain counterpart of GetClassAdRaw, for a caller that must consume an ad to stay framed on the connection but does not need its contents: counting query results, skipping ads that fail a cheap pre-filter, or a throughput benchmark that isolates the sender from client-side decode cost.

func (*Message) SkipString added in v0.2.0 

func (m *Message) SkipString(ctx context.Context) error

SkipString reads and discards one CEDAR string, matching GetString's framing (encrypted: an int32 length prefix followed by that many bytes; plaintext: bytes up to a null terminator) but allocating nothing for the value.

type PutClassAdConfig 

type PutClassAdConfig struct {
	Options        PutClassAdOptions
	Whitelist      []string         // If provided, only these attributes will be sent
	EncryptedAttrs []string         // Attributes that should be encrypted (not implemented)
	PeerVersion    *HTCondorVersion // Peer version for compatibility checks
}

PutClassAdConfig provides configuration for ClassAd serialization

type PutClassAdOptions 

type PutClassAdOptions int

PUT_CLASSAD options that control how ClassAds are serialized These mirror HTCondor's putClassAd options from classad_oldnew.cpp 

const (
	PutClassAdNone              PutClassAdOptions = 0
	PutClassAdNoTypes           PutClassAdOptions = 1 << 0 // Don't send MyType/TargetType
	PutClassAdNoPrivate         PutClassAdOptions = 1 << 1 // Exclude private attributes
	PutClassAdServerTime        PutClassAdOptions = 1 << 2 // Add ATTR_SERVER_TIME
	PutClassAdNonBlocking       PutClassAdOptions = 1 << 3 // Non-blocking mode (not used in this impl)
	PutClassAdNoExpandWhitelist PutClassAdOptions = 1 << 4 // Don't expand whitelist references
)

type StreamInterface 

type StreamInterface interface {
	ReadFrame(ctx context.Context) ([]byte, bool, error) // data, isEOM, error
	WriteFrame(ctx context.Context, data []byte, isEOM bool) error
	IsEncrypted() bool
}

StreamInterface defines the interface needed from Stream to read/write frames

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.