ccb

package
v0.5.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 12, 2026 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Overview

Package ccb implements the wire protocol for HTCondor's Condor Connection Broker (CCB). It is shared by CCB clients (requesters), CCB listeners (targets that register to be reachable), and CCB servers (brokers).

The protocol is described in the HTCondor C++ sources src/ccb/ccb_client.cpp, ccb_listener.cpp and ccb_server.cpp. All control messages are a single CEDAR message carrying one ClassAd. The reverse-connect "hello" is a raw command integer followed by a ClassAd in one message (no security handshake), so it can be delivered to an ordinary CEDAR command socket.

Index

Constants

View Source 
const (
	CommandRegister         = commands.CCB_REGISTER           // 67
	CommandRequest          = commands.CCB_REQUEST            // 68
	CommandReverseConnect   = commands.CCB_REVERSE_CONNECT    // 69
	CommandProxyConnect     = commands.CCB_PROXY_CONNECT      // 82 (outbound tunneling)
	CommandGetTunnelAddress = commands.CCB_GET_TUNNEL_ADDRESS // 83 (master->CCB tunnel address)
	CommandAlive            = commands.ALIVE                  // 441
)

Command integers (mirrors condor_commands.h via the commands package).

View Source 
const (
	AttrCommand     = "Command"
	AttrCCBID       = "CCBID"
	AttrClaimID     = "ClaimId"
	AttrRequestID   = "RequestID"
	AttrMyAddress   = "MyAddress"
	AttrResult      = "Result"
	AttrErrorString = "ErrorString"
	AttrName        = "Name"

	// Tunneling extension (new). Attribute names are proposed by the CCB tunneling
	// design (§4.5) and must be pinned jointly with the C++ implementation, like
	// the CCB_GET_TUNNEL_ADDRESS command int.
	AttrSubsys     = "Subsys"     // master -> CCB: requesting subsystem (debugging)
	AttrCCBAddress = "CCBAddress" // CCB -> master: derived tunnel address

	// Streaming/proxy extension (new; ignored by stock HTCondor).
	AttrCCBStreaming            = "CCBStreaming"            // server -> peer: capability advertisement
	AttrCCBStreamingRequired    = "CCBStreamingRequired"    // requester -> server: this request needs proxying
	AttrProxyMode               = "ProxyMode"               // server -> requester: reply will be proxied on this socket
	AttrCCBStreamingUnsupported = "CCBStreamingUnsupported" // server -> requester: typed "not supported" failure

	// Recursive inbound tunnel (Model 1) + audit. Must match the C++ wire names
	// (src/condor_includes/condor_attributes.h).
	//
	// AttrCCBRoute carries the remaining downstream CCBIDs after this hop's target
	// (space-separated; empty/absent at the final hop). When present, the target is
	// itself a downstream CCB and is asked to recurse: set up the next hop, then
	// reverse-connect and relay. AttrCCBOriginalRequester (stamped by the entry
	// broker from the *authenticated* client) and AttrCCBPriorHop (each forwarder's
	// own address) are the audit trail, forwarded for logging only.
	AttrCCBRoute             = "CCBRoute"
	AttrCCBOriginalRequester = "CCBOriginalRequester"
	AttrCCBPriorHop          = "CCBPriorHop"

	// AttrCCBTTL is the decrementing outbound-proxy time-to-live (like TCP): the
	// originator sets it, each forwarding broker decrements it and refuses at 0.
	AttrCCBTTL = "CCBTTL"
)

ClassAd attribute names (must match condor_attributes.h exactly).

Variables

View Source 
var OutboundMinVersion = StreamingMinVersion

OutboundMinVersion is the minimum broker $CondorVersion$ assumed to support the outbound CCB_PROXY_CONNECT tunneling command. A requester version-gates the broker before sending command 82 so it fails fast against an old broker that has no handler for it, rather than tripping a confusing mid-protocol error. Tunneling ships alongside streaming, so this tracks StreamingMinVersion.

View Source 
var StreamingMinVersion = version.CondorVersion{Major: 25, Minor: 12, Sub: 0}

StreamingMinVersion is the minimum broker $CondorVersion$ that is assumed to support streaming/proxy mode when no explicit capability flag is available. golang-ccb advertises a version at or above this; older C++ CCB servers fall below it, so a private requester fails fast instead of sending a request the old server would mishandle.

Functions

func AdBool 

func AdBool(ad *classad.ClassAd, name string) (bool, bool)

AdBool returns a boolean attribute and whether it was present.

func AdInt 

func AdInt(ad *classad.ClassAd, name string) (int64, bool)

AdInt returns an integer attribute and whether it was present.

func AdString 

func AdString(ad *classad.ClassAd, name string) string

AdString returns a string attribute, or "" if absent.

func ContactString 

func ContactString(brokerAddr string, ccbid uint64) string

ContactString builds a CCB contact "<brokerAddr>#<ccbid>" (broker address without angle brackets), matching CCBServer::CCBIDToContactString.

func Dial 

func Dial(ctx context.Context, contacts []addresses.CCBContact, opts DialOptions) (net.Conn, error)

Dial reaches a target daemon through the CCB broker(s) named by contacts, using connection reversal, and returns a net.Conn connected to the target (the reverse-connect hello already consumed and validated). The caller then runs the normal CEDAR client handshake over the returned connection.

When more than one broker contact is given, Dial uses a Happy-Eyeballs-style algorithm: it tries brokers in randomized order, starting an attempt to the next broker after Stagger has elapsed (or immediately if an outstanding attempt has already failed), and returns the first connection that succeeds. Remaining attempts are cancelled. This bounds the latency impact of a slow or dead CCB server in a multi-CCB pool. Each attempt uses its own connect id, so concurrent attempts to different brokers do not collide.

func GenerateConnectID 

func GenerateConnectID() (string, error)

GenerateConnectID returns a fresh connect id: 20 random bytes rendered as 40 hex characters, matching CCBClient's connect-id generation.

func GetTunnelAddress added in v0.4.0 

func GetTunnelAddress(ctx context.Context, brokerAddr string, sec *security.SecurityConfig, subsys string) (string, error)

GetTunnelAddress asks a CCB broker (an inside CCB) for its derived tunnel address -- the "<outside>#<inside_id>" contact it obtained by registering upstream (§4.5). A master uses this in the off-host CCB deployment (Model 2) to learn the CCB_ADDRESS to inject into child daemons, when it cannot read the CCB's address file locally. The command is authenticated (DAEMON on the server); subsys names the requesting subsystem (debugging only).

func NewAd 

func NewAd(fields map[string]any) *classad.ClassAd

NewAd builds a ClassAd from a map of attribute values. Supported value types are string, int, int64, uint64, and bool.

func OutboundConnect added in v0.4.0 

func OutboundConnect(ctx context.Context, broker, target string, opts OutboundOptions) (net.Conn, error)

OutboundConnect asks a broker to dial target on the requester's behalf and splice, returning a net.Conn carrying an opaque byte relay to target. It is the client side of CCB tunneling's outbound mode: a daemon whose network forbids outbound TCP reaches target through its OUTBOUND_CCB_ADDRESS broker.

broker is a broker Sinful ("host:port", or a shared-port "host:port?sock=name"). target is the destination Sinful the broker should dial; the broker validates it against its allow-list. Unlike CCB_REQUEST there is NO CCBID -- the target is addressed by Sinful, which is the protocol-level distinction (§4.1).

The returned conn is a raw relay: there is no reverse-connect hello and no ProxyMode echo. The requester IS the CEDAR connector, so the caller must run a normal, full CEDAR client handshake to the real target over the returned conn (wrapping it in a fresh stream, which resets MAC/MD state and tears down the broker-session crypto). That end-to-end handshake -- auth, integrity, encryption -- rides opaquely through the broker, which never holds its keys.

func ReadControlAd 

func ReadControlAd(ctx context.Context, s *stream.Stream) (*classad.ClassAd, error)

ReadControlAd reads a single control message (one ClassAd) from the stream.

func ReadReverseConnectAd 

func ReadReverseConnectAd(ctx context.Context, msg *message.Message, cmd int) (*classad.ClassAd, error)

ReadReverseConnectAd reads the ClassAd of a reverse-connect hello when the command integer has already been consumed from msg (e.g. by a dispatching server). It validates that cmd is CCB_REVERSE_CONNECT.

func SplitBrokerList 

func SplitBrokerList(s string) []string

SplitBrokerList splits a CCB broker list as written in configuration (e.g. the value of CCB_ADDRESS), which is separated by commas and/or whitespace, into individual broker addresses.

func WriteControlAd 

func WriteControlAd(ctx context.Context, s *stream.Stream, ad *classad.ClassAd) error

WriteControlAd writes a single control message (one ClassAd terminated by end-of-message) on the stream. Used for CCB_REGISTER, CCB_REQUEST forwarding, result reports, heartbeats and replies.

func WriteReverseConnect 

func WriteReverseConnect(ctx context.Context, s *stream.Stream, connectID, requestID, myAddr string) error

WriteReverseConnect writes the raw reverse-connect hello: the CCB_REVERSE_CONNECT command integer followed by a ClassAd, in one message. The ClassAd carries ClaimId (the connect id), RequestID and MyAddress.

Types

type BrokerDialer added in v0.4.0 

type BrokerDialer func(ctx context.Context, brokerAddr string) (net.Conn, error)

BrokerDialer establishes a raw connection to a CCB broker over a non-default carrier -- e.g. a filesystem/yamux tunnel (see golang-ccb transport/fstun) -- instead of the default TCP/shared-port dial. When set on a ListenerConfig or OutboundOptions, the returned conn is wrapped in a CEDAR stream exactly as a TCP conn would be, so the rest of the protocol is carrier-agnostic. brokerAddr is provided for logging/routing; a point-to-point carrier may ignore it (there is only one peer). It must honor ctx for cancellation.

type ConnHandler 

type ConnHandler func(conn net.Conn, meta InboundMeta)

ConnHandler is called with each inbound reverse connection a Listener accepts on behalf of the registered daemon. By the time it is called the reverse-connect hello has been sent; the connection is now an ordinary inbound CEDAR command socket on which the caller is the server (the remote requester will drive DC_AUTHENTICATE). The handler owns the conn and must close it. meta carries any tunnel route/audit forwarded by the broker.

type DialOptions 

type DialOptions struct {
	// Security is the credential/config used to authenticate to the CCB
	// broker. Its Command is overridden with CCB_REQUEST. Required.
	Security *security.SecurityConfig

	// MyAddress, if set, is the reverse-connect address advertised to the
	// target (HTCondor sinful, e.g. "<1.2.3.4:5678>"). If empty, the address
	// of the local reverse-connect listener is used. Ignored in proxy mode.
	MyAddress string

	// ProxyReturnAddr, if set, switches the dial to streaming/proxy mode: the
	// requester is itself private, so instead of listening it asks the broker
	// to proxy. The value is the requester's own CCB sinful (carrying a ccbid),
	// sent as MyAddress so the broker recognizes proxy mode.
	ProxyReturnAddr string

	// RequireStreaming makes proxy mode mandatory: if the broker does not
	// support streaming, the dial fails fast rather than falling back.
	RequireStreaming bool

	// TargetDesc is a human-readable description of the target (debugging).
	TargetDesc string

	// ListenAddr is the bind address for the reverse-connect listener in
	// standard mode (default ":0"). Ignored when SharedPortEndpoint is set.
	ListenAddr string

	// SharedPortEndpoint, if set, makes a standard-mode requester accept the
	// target's reverse connection through a shared-port endpoint (a Unix socket
	// that a condor_shared_port daemon forwards connections to) instead of
	// opening its own TCP listen socket. This lets a requester whose only
	// inbound path is a shared port still be reached by the reverse connection.
	// Ignored in proxy mode (ProxyReturnAddr set).
	SharedPortEndpoint *SharedPortEndpointConfig

	// Timeout bounds the whole dial (default 30s).
	Timeout time.Duration

	// Stagger is the Happy-Eyeballs delay between starting attempts to
	// successive brokers (default 250ms). A new broker attempt is also started
	// immediately whenever an outstanding attempt fails. Set to a negative
	// value to force fully-sequential dialing (one broker at a time).
	Stagger time.Duration
}

DialOptions configures a CCB reverse-connect dial.

type InboundMeta added in v0.5.0 

type InboundMeta struct {
	Route             string
	OriginalRequester string
	PriorHop          string
}

InboundMeta carries the routing/audit fields from the reverse-connect request (forwarded by the requesting broker) to the ConnHandler. For a recursive inbound tunnel (Model 1), a non-empty Route means the handler is an intermediate CCB and must set up the next hop and relay rather than serve a command locally; Route is the space-separated remaining downstream CCBIDs after this hop. OriginalRequester and PriorHop are the audit trail (see the CCBRoute/CCBOriginalRequester/CCBPriorHop attributes).

type Listener 

type Listener struct {
	// contains filtered or unexported fields
}

Listener maintains a persistent registration with one or more CCB brokers and services reverse-connect requests from any of them. Each broker is handled by an independent registration (its own connection, heartbeat, and reconnect loop), so one broker going down does not affect the others.

func NewListener 

func NewListener(cfg ListenerConfig) *Listener

NewListener creates a Listener for all configured brokers.

func (*Listener) BrokerSupportsStreaming 

func (l *Listener) BrokerSupportsStreaming() bool

BrokerSupportsStreaming reports whether any currently-registered broker advertised streaming support. Returns false if no broker is registered.

func (*Listener) Contact 

func (l *Listener) Contact() string

Contact returns a single assigned contact (the first registered broker's), for convenience in the common single-broker case. Empty until registered.

func (*Listener) ContactList 

func (l *Listener) ContactList() string

ContactList returns Contacts joined by spaces, i.e. the value to use for the "ccbid" parameter of a sinful string.

func (*Listener) Contacts 

func (l *Listener) Contacts() []string

Contacts returns the CCB contact strings ("addr#id") currently assigned by the brokers this listener is registered with, one per registered broker. Together they are the ccbid list to advertise in the daemon's sinful. Brokers that are not currently registered are omitted.

func (*Listener) NumRegistered 

func (l *Listener) NumRegistered() int

NumRegistered returns how many of the configured brokers are currently registered.

func (*Listener) Run 

func (l *Listener) Run(ctx context.Context) error

Run registers with every broker and services requests until ctx is cancelled, reconnecting (preserving each ccbid via its reconnect cookie) on failure. It returns after all broker loops have stopped.

type ListenerConfig 

type ListenerConfig struct {
	// BrokerAddrs are the CCB broker addresses ("host:port", brackets
	// optional) to register with. A daemon in a pool with multiple CCB
	// servers registers with all of them, and the union of the assigned
	// contacts (see Contacts) forms the ccbid list in the daemon's sinful.
	BrokerAddrs []string

	// BrokerAddr is a convenience for the single-broker case. If set, it is
	// appended to BrokerAddrs.
	BrokerAddr string

	// Security authenticates registration to the broker. Its Command is
	// overridden with CCB_REGISTER. Required.
	Security *security.SecurityConfig

	// Handler receives each reverse-connected inbound connection. Required.
	Handler ConnHandler

	// Name identifies this daemon to the broker (debugging only).
	Name string

	// HeartbeatInterval is how often to send ALIVE (default 1200s, min 30s).
	HeartbeatInterval time.Duration

	// ReconnectInterval is how long to wait before re-registering after a
	// broker connection drops (default 60s).
	ReconnectInterval time.Duration

	// DialTimeout bounds reverse-connect dials to requesters (default 30s).
	DialTimeout time.Duration

	// Dial, when set, reaches the broker(s) over a non-default carrier (see
	// BrokerDialer) instead of TCP -- used by an inside CCB whose upstream link
	// is a tunnel. It is used for BOTH the persistent registration dial and the
	// reverse-connect dials the broker asks for (in streaming mode those target
	// the broker's own address, i.e. the same carrier peer). nil ⇒ default
	// TCP/shared-port. A carrier is point-to-point, so this is only sensible with
	// a single broker in the list.
	Dial BrokerDialer
}

ListenerConfig configures a CCB Listener (a daemon registering itself with one or more brokers so private peers can reach it).

type OutboundOptions added in v0.4.0 

type OutboundOptions struct {
	// Security authenticates to the broker; its Command is overridden with
	// CCB_PROXY_CONNECT. Required. The broker enforces DAEMON authorization, so
	// this must carry a credential the broker accepts at that level.
	Security *security.SecurityConfig

	// Name is an optional debugging identity for the requester (sent as Name).
	Name string

	// Timeout bounds the whole exchange, including the broker's outbound dial to
	// the target (default 30s).
	Timeout time.Duration

	// Dial, when set, reaches the broker over a non-default carrier (see
	// BrokerDialer) instead of TCP -- used by an inside CCB that forwards the
	// proxy request to its next hop over a tunnel. nil ⇒ default TCP/shared-port.
	Dial BrokerDialer

	// TTL, when > 0, is sent to the broker as the outbound hop budget (CCBTTL).
	// The originator sets it; each forwarding broker decrements and refuses at 0,
	// bounding the chain length. 0 ⇒ omit (broker applies its own default).
	TTL int
}

OutboundOptions configures an outbound proxy (tunnel) connect.

type OutboundUnsupportedError added in v0.4.0 

type OutboundUnsupportedError struct {
	Broker  string
	Version string
}

OutboundUnsupportedError indicates the broker cannot honor CCB_PROXY_CONNECT (too old, or the outbound-proxy handler is disabled).

func (*OutboundUnsupportedError) Error added in v0.4.0 

func (e *OutboundUnsupportedError) Error() string

type SharedPortEndpointConfig 

type SharedPortEndpointConfig struct {
	// SharedPortAddr is the "host:port" of the shared_port daemon that fronts
	// this endpoint. It is used to build the advertised return address
	// "<host:port?sock=NAME>" so the target reverse-connects through the shared
	// port. Required.
	SharedPortAddr string

	// SocketDir is the directory the shared_port daemon watches for endpoint
	// sockets (its DAEMON_SOCKET_DIR). The endpoint socket is created here so
	// the daemon can forward connections to it. If empty, the socket is created
	// in a temporary directory -- only useful when the shared_port daemon shares
	// that filesystem location, so prefer setting it explicitly.
	SocketDir string

	// SocketName is the shared-port id (the sock= value) to register. If empty,
	// a random ("anonymous") name is generated.
	SocketName string
}

SharedPortEndpointConfig configures a requester's incoming reverse-connect port when it is managed by a condor_shared_port daemon rather than a private TCP listen socket.

type StreamingUnsupportedError 

type StreamingUnsupportedError struct {
	Broker  string
	Version string
}

StreamingUnsupportedError indicates the broker cannot honor a required streaming/proxy request.

func (*StreamingUnsupportedError) Error 

func (e *StreamingUnsupportedError) Error() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.