parser

README

Generated Documentation and Embedded Help Texts

Many parts of the parser package include special consideration for generation or other production of user-facing documentation. This includes interactive help messages, generated documentation of the set of available functions, or diagrams of the various expressions.

Generated documentation is produced and maintained at compile time, while the interactive, contextual help is returned at runtime.

We equip the generated parser with the ability to report contextual help in two circumstances:

• when the user explicitly requests help with the HELPTOKEN (current syntax: standalone "??")
• when the user makes a grammatical mistake (e.g. INSERT sometable INTO(x, y) ...)

We use the docgen tool to produce the generated documentation files that are then included in the broader (handwritten) published documentation.

Help texts embedded in the grammar

The help is embedded in the grammar using special markers in yacc comments, for example:

// %Help: HELPKEY - shortdescription
// %Category: SomeCat
// %Text: whatever until next %marker at start of line, or non-comment.
// %SeeAlso: whatever until next %marker at start of line, or non-comment.
// %End (optional)

The "HELPKEY" becomes the map key in the generated Go map.

These texts are extracted automatically by help.awk and converted into a Go data structure in help_messages.go.

Support in the parser

Primary mechanism - LALR error recovery

The primary mechanism is leveraging error recovery in LALR parsers using the special error token [1] [2]: when an unexpected token is encountered, the LALR parser will pop tokens on the stack until the prefix matches a grammar rule with the special "error" token (if any). If such a rule exists, its action is used to reduce and the erroneous tokens are discarded.

This mechanism is used both when the user makes a mistake, and when the user inserts the HELPTOKEN in the middle of a statement. When present in the middle of a statement, HELPTOKEN is considered an error and triggers the error recovery.

We use this for contextual help by providing error rules that generate a contextual help text during LALR error recovery.

For example:

backup_stmt:
  BACKUP targets TO string_or_placeholder opt_as_of_clause opt_incremental opt_with_options
  {
    $$.val = &Backup{Targets: $2.targetList(), To: $4.expr(), IncrementalFrom: $6.exprs(), AsOf: $5.asOfClause(), Options: $7.kvOptions()}
  }
| BACKUP error { return helpWith(sqllex, `BACKUP`) }

In this example, the grammar specifies that if the BACKUP keyword is followed by some input tokens such that the first (valid) grammar rule doesn't apply, the parser will "recover from the error" by backtracking up until the point it only sees BACKUP on the stack followed by non-parsable tokens, at which points it takes the error rule and executes its action.

The action is return helpWith(...). What this does is:

• halts parsing (the generated parser executes all actions in a big loop; a return interrupts this loop);
• makes the parser return with an error (the helpWith function returns non-zero);
• extends the parsing error message with a help text; this help text can subsequently be exploited in a client to display the help message in a friendly manner.

Code generation

Since the pattern "{ return helpWith(sqllex, ...) }" is common, we also implement a shorthand syntax based on comments, for example:

backup_stmt:
   ...
| BACKUP error // SHOW HELP: BACKUP

The special comment syntax "SHOW HELP: XXXX" is substituted by means of an auxiliary script (replace_help_rules.awk) into the form explained above.

Secondary mechanism - explicit help token

The mechanism described above works both when the user make a grammatical error and when they place the HELPTOKEN in the middle of a statement, rendering it invalid.

However for contextual help this is not sufficient: what happens if the user requests HELPTOKEN at a position in the grammar where everything before is a complete, valid SQL input?

For example: DELETE FROM foo ?

When encountering this input, the LALR parser will see DELETE FROM foo first, then reduce using the DELETE action because everything up to this point is a valid DELETE statement. When the HELPTOKEN is encountered, the statement has already been completed and the LALR parser doesn't 'know' any more that it was in the context of a DELETE statement.

If we try to place an error-based recovery rule at the top-level:

stmt:
  alter_stmt
| backup_stmt
| ...
| delete_stmt
| ...
| error { ??? }

This wouldn't work: the code inside the error action cannot "observe" the tokens observed so far and there would be no way to know whether the error should be about DELETE, or instead about ALTER, BACKUP, etc.

So in order to handle HELPTOKEN after a valid statement, we must place it in a rule where the context is still available, that is before the statement's grammar rule is reduced.

Where would that be? Suppose we had a simple statement rule:

somesimplestmt:
  SIMPLE DO SOMETHING { $$ = reduce(...) }
| SIMPLE error { help ... }

We could extend with:

somesimplestmt:
  SIMPLE DO SOMETHING { $$ = reduce(...) }
| SIMPLE DO SOMETHING HELPTOKEN { help ... }
| SIMPLE error { help ... }

(the alternative also works:

somesimplestmt:
  SIMPLE DO SOMETHING { $$ = reduce(...) }
| SIMPLE DO SOMETHING error { help ... }
| SIMPLE error { help ... }

)

That is all fine and dandy, but in SQL we have statements with many alternate forms, for example:

alter_rename_table_stmt:
  ALTER TABLE relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name { ... }

To add complementary handling of the help token at the end of valid statements we could, but would hate to, duplicate all the rules:

alter_rename_table_stmt:
  ALTER TABLE relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE relation_expr RENAME TO qualified_name HELPTOKEN { help ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name HELPTOKEN { help ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name HELPTOKEN { help ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name HELPTOKEN { help ... }

This duplication is horrendous (not to mention hard to maintain), so instead we should attempt to factor the help token in a context where it is still known that we are dealing just with that statement.

The following works:

alter_rename_table_stmt:
  real_alter_rename_table_stmt { $$ = $1 }
| real_alter_rename_table_stmt HELPTOKEN { help ... }

real_alter_rename_table_stmt:
  ALTER TABLE relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name { ... }

Or does it? Without anything else, yacc complains with a "shift/reduce conflict". The reason is coming from the ambiguity: when the parsing stack contains everything sufficient to match a real_alter_rename_table_stmt, there is a choice between reducing the simple form alter_rename_table_stmt: real_alter_rename_table_stmt, or shifting into the more complex form alter_rename_table_stmt: real_alter_rename_table_stmt HELPTOKEN.

This is another form of the textbook situation when yacc is used to parse if-else statements in a programming language: the rule stmt: IF cond THEN body | IF cond THEN body ELSE body is ambiguous (and yields a shift/reduce conflict) for exactly the same reason.

The solution here is also straight out of a textbook: one simply informs yacc of the relative priority between the two candidate rules. In this case, when faced with a neutral choice, we encourage yacc to shift. The particular mechanism is to tell yacc that one rule has a higher priority than another.

It just so happens however that the yacc language only allows us to set relative priorities of tokens, not rules. And here we have a problem, of the two rules that need to be prioritized, only one has a token to work with (the one with HELPTOKEN). Which token should we prioritize for the other?

Conveniently yacc knows about this trouble and offers us an awkward, but working solution: we can tell it "use for this rule the same priority level as an existing token, even though the token is not part of the rule". The syntax for this is rule %prec TOKEN.

We can then use this as follows:

alter_rename_table_stmt:
  real_alter_rename_table_stmt           %prec LOWTOKEN { $$ = $1 }
| real_alter_rename_table_stmt HELPTOKEN %prec HIGHTOKEN { help ... }

We could create two new pseudo-tokens for this (called LOWTOKEN and HIGHTOKEN) however conveniently we can also reuse otherwise valid tokens that have known relative priorities. We settled in our case on VALUES (low priority) and UMINUS (high priority).

Code generation

With the latter mechanism presented above the pattern

rule:
  somerule           %prec VALUES
| somerule HELPTOKEN %prec UMINUS { help ...}`

becomes super common, so we automate it with the following special syntax:

rule:
  somerule // EXTEND WITH HELP: XXX

And the code replacement in replace_help_rules.awk expands this to the form above automatically.

Generated Documentation

Documentation of the SQL functions and operators is generated by the docgen utility, using make generate PKG=./docs/.... The markdown-formatted files are kept in docs/generated/sql and should be re-generated whenever the functions/operators they document change, and indeed if regenerating produces a diff, a CI failure is expected.

References

1. https://www.gnu.org/software/bison/manual/html_node/Error-Recovery.html
2. http://stackoverflow.com/questions/9796608/error-handling-in-yacc

Documentation

Overview

Package parser contains exposes a SQL parser for cockroach.

Index

• Constants
• Variables
• func GetTypeFromCastOrCollate(expr tree.Expr) (tree.ResolvableTypeReference, error)
• func GetTypeFromValidSQLSyntax(sql string) (tree.ResolvableTypeReference, error)
• func GetTypeReferenceFromName(typeName tree.Name) (tree.ResolvableTypeReference, error)
• func NakedIntTypeFromDefaultIntSize(defaultIntSize int32) *types.T
• func Parse(sql string) (statements.Statements, error)
• func ParseExpr(sql string) (tree.Expr, error)
• func ParseExprWithInt(sql string, nakedIntType *types.T) (tree.Expr, error)
• func ParseExprs(sql []string) (tree.Exprs, error)
• func ParseFunctionName(sql string) (*tree.UnresolvedObjectName, error)
• func ParseOne(sql string) (statements.Statement[tree.Statement], error)
• func ParseOneRetainComments(sql string) (statements.Statement[tree.Statement], error)
• func ParseOneWithInt(sql string, nakedIntType *types.T) (statements.Statement[tree.Statement], error)
• func ParseQualifiedTableName(sql string) (*tree.TableName, error)
• func ParseTableName(sql string) (*tree.UnresolvedObjectName, error)
• func ParseTablePattern(sql string) (tree.TablePattern, error)
• func ParseWithInt(sql string, nakedIntType *types.T) (statements.Statements, error)
• func PopulateErrorDetails(tokID int32, lastTokStr string, lastTokPos int32, lastErr error, lIn string) error
• func RunShowSyntax(ctx context.Context, stmt string, ...)
• func SplitFirstStatement(sql string) (pos int, ok bool)
• type HelpMessage
  • func (h *HelpMessage) Format(w io.Writer)
  • func (h *HelpMessage) String() string
• type HelpMessageBody
• type Parser
  • func (p *Parser) Parse(sql string) (statements.Statements, error)
  • func (p *Parser) ParseWithInt(sql string, nakedIntType *types.T) (statements.Statements, error)
• type TokenString
  • func Tokens(sql string) (tokens []TokenString, ok bool)

Constants

const ABORT = 57363

const ABSOLUTE = 57364

const ACCESS = 57365

const ACTION = 57366

const ADD = 57367

const ADMIN = 57368

const AFTER = 57369

const AGGREGATE = 57370

const ALL = 57371

const ALTER = 57372

const ALWAYS = 57373

const ANALYSE = 57374

const ANALYZE = 57375

const AND = 57376

const AND_AND = 57377

const ANNOTATE_TYPE = 57379

const ANY = 57378

const ARRAY = 57380

const AS = 57381

const ASC = 57382

const ASENSITIVE = 57385

const ASYMMETRIC = 57386

const AS_JSON = 57383

const AS_LA = 58051

const AT = 57387

const ATOMIC = 57388

const ATTRIBUTE = 57389

const AT_AT = 57384

const AUTHORIZATION = 57390

const AUTOMATIC = 57391

const AVAILABILITY = 57392

const AVOID_FULL_SCAN = 57393

const BACKUP = 57394

const BACKUPS = 57395

const BACKWARD = 57396

const BATCH = 57397

const BCONST = 57348

const BEFORE = 57398

const BEGIN = 57399

const BETWEEN = 57400

const BIDIRECTIONAL = 57401

const BIGINT = 57402

const BIGSERIAL = 57403

const BINARY = 57404

const BIT = 57405

const BITCONST = 57349

const BOOLEAN = 57407

const BOTH = 57408

const BOX2D = 57409

const BUCKET_COUNT = 57406

const BUNDLE = 57410

const BY = 57411

const BYPASSRLS = 57412

const CACHE = 57413

const CALL = 57414

const CALLED = 57415

const CANCEL = 57416

const CANCELQUERY = 57417

const CAPABILITIES = 57418

const CAPABILITY = 57419

const CASCADE = 57420

const CASE = 57421

const CAST = 57422

const CBRT = 57423

const CHANGEFEED = 57424

const CHAR = 57425

const CHARACTER = 57426

const CHARACTERISTICS = 57427

const CHECK = 57428

const CHECK_FILES = 57429

const CLOSE = 57430

const CLUSTER = 57431

const CLUSTERS = 57432

const CLUSTER_ALL = 58059

const COALESCE = 57433

const COLLATE = 57434

const COLLATION = 57435

const COLUMN = 57436

const COLUMNS = 57437

const COMMENT = 57438

const COMMENTS = 57439

const COMMIT = 57440

const COMMITTED = 57441

const COMPACT = 57442

const COMPLETE = 57443

const COMPLETIONS = 57444

const CONCAT = 57445

const CONCURRENTLY = 57446

const CONFIGURATION = 57447

const CONFIGURATIONS = 57448

const CONFIGURE = 57449

const CONFLICT = 57450

const CONNECTION = 57451

const CONNECTIONS = 57452

const CONSTRAINT = 57453

const CONSTRAINTS = 57454

const CONTAINED_BY = 58061

const CONTAINS = 57455

const CONTROLCHANGEFEED = 57456

const CONTROLJOB = 57457

const CONVERSION = 57458

const CONVERT = 57459

const COPY = 57460

const COST = 57462

const COS_DISTANCE = 57461

const COVERING = 57463

const CREATE = 57464

const CREATEDB = 57465

const CREATELOGIN = 57466

const CREATEROLE = 57467

const CROSS = 57468

const CSV = 57469

const CUBE = 57470

const CURRENT = 57471

const CURRENT_CATALOG = 57472

const CURRENT_DATE = 57473

const CURRENT_ROLE = 57475

const CURRENT_SCHEMA = 57474

const CURRENT_TIME = 57476

const CURRENT_TIMESTAMP = 57477

const CURRENT_USER = 57478

const CURSOR = 57479

const CYCLE = 57480

const DATA = 57481

const DATABASE = 57482

const DATABASES = 57483

const DATE = 57484

const DAY = 57485

const DEALLOCATE = 57492

const DEBUG_IDS = 57486

const DEC = 57487

const DECIMAL = 57488

const DECLARE = 57493

const DEFAULT = 57489

const DEFAULTS = 57490

const DEFERRABLE = 57494

const DEFERRED = 57495

const DEFINER = 57491

const DELETE = 57496

const DELIMITER = 57497

const DEPENDS = 57498

const DESC = 57499

const DESTINATION = 57500

const DETACHED = 57501

const DETAILS = 57502

const DISABLE = 57503

const DISCARD = 57504

const DISTANCE = 57505

const DISTINCT = 57506

const DO = 57507

const DOMAIN = 57508

const DOT_DOT = 57355

const DOUBLE = 57509

const DROP = 57510

const EACH = 57511

const ELSE = 57512

const ENABLE = 57513

const ENCODING = 57514

const ENCRYPTED = 57515

const ENCRYPTION_INFO_DIR = 57516

const ENCRYPTION_PASSPHRASE = 57517

const END = 57518

const ENUM = 57519

const ENUMS = 57520

const ERROR = 57362

const ESCAPE = 57521

const EXCEPT = 57522

const EXCLUDE = 57523

const EXCLUDING = 57524

const EXECUTE = 57526

const EXECUTION = 57527

const EXISTS = 57525

const EXPERIMENTAL = 57528

const EXPERIMENTAL_AUDIT = 57531

const EXPERIMENTAL_FINGERPRINTS = 57529

const EXPERIMENTAL_RELOCATE = 57532

const EXPERIMENTAL_REPLICA = 57530

const EXPIRATION = 57533

const EXPLAIN = 57534

const EXPORT = 57535

const EXTENSION = 57536

const EXTERNAL = 57537

const EXTRACT = 57538

const EXTRACT_DURATION = 57539

const EXTREMES = 57540

const FAILURE = 57541

const FALSE = 57542

const FAMILY = 57543

const FCONST = 57351

const FETCH = 57544

const FETCHTEXT = 57546

const FETCHTEXT_PATH = 57548

const FETCHVAL = 57545

const FETCHVAL_PATH = 57547

const FILES = 57549

const FILTER = 57550

const FIRST = 57551

const FLOAT = 57552

const FLOAT4 = 57553

const FLOAT8 = 57554

const FLOORDIV = 57555

const FOLLOWING = 57556

const FOR = 57557

const FORCE = 57558

const FORCE_INDEX = 57559

const FORCE_INVERTED_INDEX = 57560

const FORCE_NOT_NULL = 57561

const FORCE_NULL = 57562

const FORCE_QUOTE = 57563

const FORCE_ZIGZAG = 57564

const FOREIGN = 57565

const FORMAT = 57566

const FORWARD = 57567

const FREEZE = 57568

const FROM = 57569

const FULL = 57570

const FUNCTION = 57571

const FUNCTIONS = 57572

const GENERATED = 57573

const GENERATED_ALWAYS = 58052

const GENERATED_BY_DEFAULT = 58053

const GEOGRAPHY = 57574

const GEOMETRY = 57575

const GEOMETRYCOLLECTION = 57579

const GEOMETRYCOLLECTIONM = 57580

const GEOMETRYCOLLECTIONZ = 57581

const GEOMETRYCOLLECTIONZM = 57582

const GEOMETRYM = 57576

const GEOMETRYZ = 57577

const GEOMETRYZM = 57578

const GLOBAL = 57583

const GOAL = 57584

const GRANT = 57585

const GRANTEE = 57586

const GRANTS = 57587

const GREATER_EQUALS = 57357

const GREATEST = 57588

const GROUP = 57589

const GROUPING = 57590

const GROUPS = 57591

const HASH = 57593

const HAVING = 57592

const HEADER = 57594

const HELPTOKEN = 58065

const HIGH = 57595

const HISTOGRAM = 57596

const HOLD = 57597

const HOUR = 57598

const ICONST = 57350

const IDENT = 57346

const IDENTITY = 57599

const IF = 57600

const IFERROR = 57601

const IFNULL = 57602

const IGNORE_FOREIGN_KEYS = 57603

const ILIKE = 57604

const IMMEDIATE = 57605

const IMMEDIATELY = 57606

const IMMUTABLE = 57607

const IMPORT = 57608

const IN = 57609

const INCLUDE = 57610

const INCLUDE_ALL_SECONDARY_TENANTS = 57612

const INCLUDE_ALL_VIRTUAL_CLUSTERS = 57613

const INCLUDING = 57611

const INCREMENT = 57614

const INCREMENTAL = 57615

const INCREMENTAL_LOCATION = 57616

const INDEX = 57620

const INDEXES = 57621

const INDEX_AFTER_ORDER_BY_BEFORE_AT = 57627

const INDEX_BEFORE_NAME_THEN_PAREN = 57626

const INDEX_BEFORE_PAREN = 57625

const INET = 57617

const INET_CONTAINED_BY_OR_EQUALS = 57618

const INET_CONTAINS_OR_EQUALS = 57619

const INHERITS = 57622

const INITIALLY = 57624

const INJECT = 57623

const INNER = 57628

const INOUT = 57629

const INPUT = 57630

const INSENSITIVE = 57631

const INSERT = 57632

const INSTEAD = 57633

const INT = 57634

const INTEGER = 57635

const INTERSECT = 57636

const INTERVAL = 57637

const INTERVAL_SIMPLE = 58063

const INTO = 57638

const INTO_DB = 57639

const INVERTED = 57640

const INVISIBLE = 58032

const INVOKER = 57641

const IS = 57642

const ISERROR = 57643

const ISNULL = 57644

const ISOLATION = 57645

const JOB = 57646

const JOBS = 57647

const JOIN = 57648

const JSON = 57649

const JSONB = 57650

const JSON_ALL_EXISTS = 57652

const JSON_SOME_EXISTS = 57651

const KEY = 57653

const KEYS = 57654

const KMS = 57655

const KV = 57656

const LABEL = 57657

const LANGUAGE = 57658

const LAST = 57659

const LATERAL = 57660

const LATEST = 57661

const LC_COLLATE = 57663

const LC_CTYPE = 57662

const LEADING = 57664

const LEAKPROOF = 57667

const LEASE = 57665

const LEAST = 57666

const LEFT = 57668

const LESS = 57669

const LESS_EQUALS = 57356

const LEVEL = 57670

const LIKE = 57671

const LIMIT = 57672

const LINESTRING = 57673

const LINESTRINGM = 57674

const LINESTRINGZ = 57675

const LINESTRINGZM = 57676

const LIST = 57677

const LOCAL = 57678

const LOCALITY = 57679

const LOCALTIME = 57680

const LOCALTIMESTAMP = 57681

const LOCKED = 57682

const LOGICAL = 57683

const LOGICALLY = 57684

const LOGIN = 57685

const LOOKUP = 57686

const LOW = 57687

const LSHIFT = 57688

const MATCH = 57689

const MATERIALIZED = 57690

const MAXVALUE = 57693

const MERGE = 57691

const METHOD = 57694

const MINUTE = 57695

const MINVALUE = 57692

const MODE = 57698

const MODIFYCLUSTERSETTING = 57696

const MODIFYSQLCLUSTERSETTING = 57697

const MONTH = 57699

const MOVE = 57700

const MULTILINESTRING = 57701

const MULTILINESTRINGM = 57702

const MULTILINESTRINGZ = 57703

const MULTILINESTRINGZM = 57704

const MULTIPOINT = 57705

const MULTIPOINTM = 57706

const MULTIPOINTZ = 57707

const MULTIPOINTZM = 57708

const MULTIPOLYGON = 57709

const MULTIPOLYGONM = 57710

const MULTIPOLYGONZ = 57711

const MULTIPOLYGONZM = 57712

const MaxInt = int(MaxUint >> 1)

const MaxUint = ^uint(0)

const NAME = 57714

const NAMES = 57715

const NAN = 57713

const NATURAL = 57716

const NEG_INNER_PRODUCT = 57717

const NEVER = 57718

const NEW = 57719

const NEW_DB_NAME = 57720

const NEW_KMS = 57721

const NEXT = 57722

const NO = 57723

const NOBYPASSRLS = 57724

const NOCANCELQUERY = 57725

const NOCONTROLCHANGEFEED = 57726

const NOCONTROLJOB = 57727

const NOCREATEDB = 57728

const NOCREATELOGIN = 57729

const NOCREATEROLE = 57730

const NODE = 57731

const NOLOGIN = 57732

const NOMODIFYCLUSTERSETTING = 57733

const NONE = 57739

const NONVOTERS = 57740

const NOREPLICATION = 57734

const NORMAL = 57741

const NOSQLLOGIN = 57735

const NOT = 57742

const NOTHING = 57743

const NOTHING_AFTER_RETURNING = 57744

const NOTNULL = 57745

const NOT_EQUALS = 57358

const NOT_LA = 58048

const NOT_REGIMATCH = 57361

const NOT_REGMATCH = 57359

const NOVIEWACTIVITY = 57746

const NOVIEWACTIVITYREDACTED = 57747

const NOVIEWCLUSTERSETTING = 57748

const NOWAIT = 57749

const NO_FULL_SCAN = 57738

const NO_INDEX_JOIN = 57736

const NO_ZIGZAG_JOIN = 57737

const NULL = 57750

const NULLIF = 57751

const NULLS = 57752

const NULLS_LA = 58049

const NUMERIC = 57753

const OF = 57754

const OFF = 57755

const OFFSET = 57756

const OID = 57757

const OIDS = 57758

const OIDVECTOR = 57759

const OLD = 57760

const OLD_KMS = 57761

const ON = 57762

const ONLY = 57763

const ON_LA = 58057

const OPERATOR = 57778

const OPT = 57764

const OPTION = 57765

const OPTIONS = 57766

const OR = 57767

const ORDER = 57768

const ORDINALITY = 57769

const OTHERS = 57770

const OUT = 57771

const OUTER = 57772

const OVER = 57773

const OVERLAPS = 57774

const OVERLAY = 57775

const OWNED = 57776

const OWNER = 57777

const PARALLEL = 57779

const PARENT = 57780

const PARTIAL = 57781

const PARTITION = 57782

const PARTITIONS = 57783

const PASSWORD = 57784

const PAUSE = 57785

const PAUSED = 57786

const PER = 57787

const PERMISSIVE = 57788

const PHYSICAL = 57789

const PLACEHOLDER = 57352

const PLACEMENT = 57790

const PLACING = 57791

const PLAN = 57792

const PLANS = 57793

const POINT = 57794

const POINTM = 57795

const POINTZ = 57796

const POINTZM = 57797

const POLICIES = 57798

const POLICY = 57799

const POLYGON = 57800

const POLYGONM = 57801

const POLYGONZ = 57802

const POLYGONZM = 57803

const POSITION = 57804

const POSTFIXOP = 58062

const PRECEDING = 57805

const PRECISION = 57806

const PREPARE = 57807

const PREPARED = 57808

const PRESERVE = 57809

const PRIMARY = 57810

const PRIOR = 57811

const PRIORITY = 57812

const PRIVILEGES = 57813

const PROCEDURAL = 57814

const PROCEDURE = 57815

const PROCEDURES = 57816

const PUBLIC = 57817

const PUBLICATION = 57818

const QUERIES = 57819

const QUERY = 57820

const QUOTE = 57821

const RANGE = 57822

const RANGES = 57823

const READ = 57824

const REAL = 57825

const REASON = 57826

const REASSIGN = 57827

const RECURRING = 57829

const RECURSIVE = 57828

const REDACT = 57830

const REF = 57831

const REFERENCES = 57832

const REFERENCING = 57833

const REFRESH = 57834

const REGCLASS = 57835

const REGIMATCH = 57360

const REGION = 57836

const REGIONAL = 57837

const REGIONS = 57838

const REGNAMESPACE = 57839

const REGPROC = 57840

const REGPROCEDURE = 57841

const REGROLE = 57842

const REGTYPE = 57843

const REINDEX = 57844

const RELATIVE = 57845

const RELEASE = 57854

const RELOCATE = 57846

const REMOVE_PATH = 57847

const REMOVE_REGIONS = 57848

const RENAME = 57849

const REPEATABLE = 57850

const REPLACE = 57851

const REPLICATED = 57852

const REPLICATION = 57853

const RESET = 57855

const RESET_ALL = 58054

const RESTART = 57856

const RESTORE = 57857

const RESTRICT = 57858

const RESTRICTED = 57859

const RESTRICTIVE = 57860

const RESUME = 57861

const RETENTION = 57862

const RETRY = 57866

const RETURN = 57864

const RETURNING = 57863

const RETURNS = 57865

const REVISION_HISTORY = 57867

const REVOKE = 57868

const RIGHT = 57869

const ROLE = 57870

const ROLES = 57871

const ROLE_ALL = 58055

const ROLLBACK = 57872

const ROLLUP = 57873

const ROUTINES = 57874

const ROW = 57875

const ROWS = 57876

const RSHIFT = 57877

const RULE = 57878

const RUNNING = 57879

const SAVEPOINT = 57880

const SCANS = 57881

const SCATTER = 57882

const SCHEDULE = 57883

const SCHEDULES = 57884

const SCHEMA = 57886

const SCHEMAS = 57888

const SCHEMA_ONLY = 57887

const SCONST = 57347

const SCROLL = 57885

const SCRUB = 57889

const SEARCH = 57890

const SECOND = 57891

const SECONDARY = 57892

const SECURITY = 57893

const SELECT = 57894

const SEQUENCE = 57895

const SEQUENCES = 57896

const SERIALIZABLE = 57897

const SERVER = 57898

const SERVICE = 57899

const SESSION = 57900

const SESSIONS = 57901

const SESSION_USER = 57902

const SET = 57903

const SETOF = 57904

const SETS = 57905

const SETTING = 57906

const SETTINGS = 57907

const SET_TRACING = 58060

const SHARE = 57908

const SHARED = 57909

const SHOW = 57910

const SIMILAR = 57911

const SIMPLE = 57912

const SIZE = 57913

const SKIP = 57914

const SKIP_LOCALITIES_CHECK = 57915

const SKIP_MISSING_FOREIGN_KEYS = 57916

const SKIP_MISSING_SEQUENCES = 57917

const SKIP_MISSING_SEQUENCE_OWNERS = 57918

const SKIP_MISSING_UDFS = 57920

const SKIP_MISSING_VIEWS = 57919

const SMALLINT = 57921

const SMALLSERIAL = 57922

const SNAPSHOT = 57923

const SOME = 57924

const SOURCE = 57925

const SPLIT = 57926

const SQL = 57927

const SQLLOGIN = 57928

const SQRT = 57955

const STABLE = 57929

const START = 57930

const STATE = 57931

const STATEMENT = 57932

const STATEMENTS = 57957

const STATISTICS = 57933

const STATUS = 57934

const STDIN = 57935

const STDOUT = 57936

const STOP = 57937

const STORAGE = 57942

const STORE = 57943

const STORED = 57944

const STORING = 57945

const STRAIGHT = 57938

const STREAM = 57939

const STRICT = 57940

const STRING = 57941

const SUBJECT = 57946

const SUBSCRIPTION = 57956

const SUBSTRING = 57947

const SUPER = 57948

const SUPPORT = 57949

const SURVIVAL = 57951

const SURVIVE = 57950

const SYMMETRIC = 57952

const SYNTAX = 57953

const SYSTEM = 57954

const TABLE = 57958

const TABLES = 57959

const TABLESPACE = 57960

const TEMP = 57961

const TEMPLATE = 57962

const TEMPORARY = 57963

const TENANT = 57964

const TENANTS = 57966

const TENANT_ALL = 58058

const TENANT_NAME = 57965

const TESTING_RELOCATE = 57967

const TEXT = 57968

const THEN = 57969

const THROTTLING = 57976

const TIES = 57970

const TIME = 57971

const TIMESTAMP = 57973

const TIMESTAMPTZ = 57974

const TIMETZ = 57972

const TO = 57975

const TRACE = 57978

const TRACING = 57992

const TRAILING = 57977

const TRANSACTION = 57979

const TRANSACTIONS = 57980

const TRANSFER = 57981

const TRANSFORM = 57982

const TREAT = 57983

const TRIGGER = 57984

const TRIGGERS = 57985

const TRIM = 57986

const TRUE = 57987

const TRUNCATE = 57988

const TRUSTED = 57989

const TYPE = 57990

const TYPEANNOTATE = 57354

const TYPECAST = 57353

const TYPES = 57991

const UMINUS = 58064

const UNBOUNDED = 57993

const UNCOMMITTED = 57994

const UNIDIRECTIONAL = 57995

const UNION = 57996

const UNIQUE = 57997

const UNKNOWN = 57998

const UNLISTEN = 57999

const UNLOGGED = 58000

const UNSAFE_RESTORE_INCOMPATIBLE_VERSION = 58001

const UNSET = 58006

const UNSPLIT = 58002

const UNTIL = 58007

const UPDATE = 58003

const UPDATES_CLUSTER_MONITORING_METRICS = 58004

const UPSERT = 58005

const USE = 58008

const USER = 58009

const USERS = 58010

const USER_ALL = 58056

const USING = 58011

const UUID = 58012

const VALID = 58013

const VALIDATE = 58014

const VALUE = 58015

const VALUES = 58016

const VARBIT = 58017

const VARCHAR = 58018

const VARIABLES = 58023

const VARIADIC = 58019

const VARYING = 58024

const VECTOR = 58020

const VERIFY_BACKUP_TABLE_DATA = 58021

const VIEW = 58022

const VIEWACTIVITY = 58025

const VIEWACTIVITYREDACTED = 58026

const VIEWCLUSTERMETADATA = 58028

const VIEWCLUSTERSETTING = 58029

const VIEWDEBUG = 58027

const VIRTUAL = 58030

const VIRTUAL_CLUSTER = 58037

const VIRTUAL_CLUSTER_NAME = 58036

const VISIBILITY = 58033

const VISIBLE = 58031

const VOLATILE = 58034

const VOTERS = 58035

const WHEN = 58038

const WHERE = 58039

const WINDOW = 58040

const WITH = 58041

const WITHIN = 58042

const WITHOUT = 58043

const WITH_LA = 58050

const WORK = 58044

const WRITE = 58045

const YEAR = 58046

const ZONE = 58047

Variables

var AllHelp = func(h map[string]HelpMessageBody) string {

	cmds := make(map[string][]string)
	for c, details := range h {
		if details.Category == "" {
			continue
		}
		cmds[details.Category] = append(cmds[details.Category], c)
	}

	// Ensure the result is deterministic.
	var categories []string
	for c, l := range cmds {
		categories = append(categories, c)
		sort.Strings(l)
	}
	sort.Strings(categories)

	// Compile the final help index.
	var buf bytes.Buffer
	w := tabwriter.NewWriter(&buf, 0, 0, 1, ' ', 0)
	for _, cat := range categories {
		fmt.Fprintf(w, "%s:\n", cases.Title(language.English, cases.NoLower).String(cat))
		for _, item := range cmds[cat] {
			fmt.Fprintf(w, "\t\t%s\t%s\n", item, h[item].ShortDescription)
		}
		fmt.Fprintln(w)
	}
	_ = w.Flush()
	return buf.String()
}(helpMessages)

AllHelp contains an overview of all statements with help messages. For example, displayed in the CLI shell with \h without additional parameters.

var HelpMessages = func(h map[string]HelpMessageBody) map[string]HelpMessageBody {
	appendSeeAlso := func(newItem, prevItems string) string {

		if prevItems != "" {
			return newItem + "\n  " + prevItems
		}
		return newItem
	}
	reformatSeeAlso := func(seeAlso string) string {
		return strings.Replace(
			strings.Replace(seeAlso, ", ", "\n  ", -1),
			"WEBDOCS", docs.URLBase, -1)
	}
	srcMsg := h["<SOURCE>"]
	srcMsg.SeeAlso = reformatSeeAlso(strings.TrimSpace(srcMsg.SeeAlso))
	selectMsg := h["<SELECTCLAUSE>"]
	selectMsg.SeeAlso = reformatSeeAlso(strings.TrimSpace(selectMsg.SeeAlso))
	for k, m := range h {
		m = h[k]
		m.ShortDescription = strings.TrimSpace(m.ShortDescription)
		m.Text = strings.TrimSpace(m.Text)
		m.SeeAlso = strings.TrimSpace(m.SeeAlso)

		if strings.Contains(m.Text, "<source>") && k != "<SOURCE>" {
			m.Text = strings.TrimSpace(m.Text) + "\n\n" + strings.TrimSpace(srcMsg.Text)
			m.SeeAlso = appendSeeAlso(srcMsg.SeeAlso, m.SeeAlso)
		}

		if strings.Contains(m.Text, "<selectclause>") && k != "<SELECTCLAUSE>" {
			m.Text = strings.TrimSpace(m.Text) + "\n\n" + strings.TrimSpace(selectMsg.Text)
			m.SeeAlso = appendSeeAlso(selectMsg.SeeAlso, m.SeeAlso)
		}

		if strings.Contains(m.Text, "<tablename>") {
			m.SeeAlso = appendSeeAlso("SHOW TABLES", m.SeeAlso)
		}
		m.SeeAlso = reformatSeeAlso(m.SeeAlso)
		h[k] = m
	}
	return h
}(helpMessages)

HelpMessages is the registry of all help messages, keyed by the top-level statement that they document. The key is intended for use via the \h client-side command.

var ParseDoBlockFn func(tree.DoBlockOptions) (tree.DoBlockBody, error)

ParseDoBlockFn allows the SQL parser to parse a PL/pgSQL DO block body.

Functions

func GetTypeFromCastOrCollate

func GetTypeFromCastOrCollate(expr tree.Expr) (tree.ResolvableTypeReference, error)

GetTypeFromCastOrCollate returns the type of the given tree.Expr. The method assumes that the expression is either tree.CastExpr or tree.CollateExpr (which wraps the tree.CastExpr).

func GetTypeFromValidSQLSyntax

func GetTypeFromValidSQLSyntax(sql string) (tree.ResolvableTypeReference, error)

GetTypeFromValidSQLSyntax retrieves a type from its SQL syntax. The caller is responsible for guaranteeing that the type expression is valid SQL (or handling the resulting error). This includes verifying that complex identifiers are enclosed in double quotes, etc.

func GetTypeReferenceFromName

func GetTypeReferenceFromName(typeName tree.Name) (tree.ResolvableTypeReference, error)

GetTypeReferenceFromName turns a type name into a type reference. This supports only “simple” (single-identifier) references to built-in types, when the identifer has already been parsed away from the input SQL syntax.

func NakedIntTypeFromDefaultIntSize

func NakedIntTypeFromDefaultIntSize(defaultIntSize int32) *types.T

NakedIntTypeFromDefaultIntSize given the size in bits or bytes (preferred) of how a "naked" INT type should be parsed returns the corresponding integer type.

func Parse

func Parse(sql string) (statements.Statements, error)

Parse parses a sql statement string and returns a list of Statements.

func ParseExpr

func ParseExpr(sql string) (tree.Expr, error)

ParseExpr parses a SQL scalar expression. The caller is responsible for ensuring that the input is, in fact, a valid SQL scalar expression — the results are undefined if the string contains invalid SQL syntax.

func ParseExprWithInt

func ParseExprWithInt(sql string, nakedIntType *types.T) (tree.Expr, error)

ParseExprWithInt parses a SQL scalar expression, using the given type when INT is used as type name in the SQL syntax. The caller is responsible for ensuring that the input is, in fact, a valid SQL scalar expression — the results are undefined if the string contains invalid SQL syntax.

func ParseExprs

func ParseExprs(sql []string) (tree.Exprs, error)

ParseExprs parses a comma-delimited sequence of SQL scalar expressions. The caller is responsible for ensuring that the input is, in fact, a comma-delimited sequence of SQL scalar expressions — the results are undefined if the string contains invalid SQL syntax.

func ParseFunctionName

func ParseFunctionName(sql string) (*tree.UnresolvedObjectName, error)

ParseFunctionName parses a function name. The function name must contain one or more name parts, using the full input SQL syntax: each name part containing special characters, or non-lowercase characters, must be enclosed in double quote. The name may not be an invalid function name (the caller is responsible for guaranteeing that only valid function names are provided as input).

func ParseOne

func ParseOne(sql string) (statements.Statement[tree.Statement], error)

ParseOne parses a sql statement string, ensuring that it contains only a single statement, and returns that Statement. ParseOne will always interpret the INT and SERIAL types as 64-bit types, since this is used in various internal-execution paths where we might receive bits of SQL from other nodes. In general, we expect that all user-generated SQL has been run through the ParseWithInt() function.

func ParseOneRetainComments

func ParseOneRetainComments(sql string) (statements.Statement[tree.Statement], error)

ParseOneRetainComments is similar to ParseOne, but it retains scanned comments in the returned statement's Comment field.

func ParseOneWithInt

func ParseOneWithInt(
	sql string, nakedIntType *types.T,
) (statements.Statement[tree.Statement], error)

ParseOneWithInt is similar to ParseOn but interprets the INT and SERIAL types as the provided integer type.

func ParseQualifiedTableName

func ParseQualifiedTableName(sql string) (*tree.TableName, error)

ParseQualifiedTableName parses a possibly qualified table name. The table name must contain one or more name parts, using the full input SQL syntax: each name part containing special characters, or non-lowercase characters, must be enclosed in double quote. The name may not be an invalid table name (the caller is responsible for guaranteeing that only valid table names are provided as input).

func ParseTableName

func ParseTableName(sql string) (*tree.UnresolvedObjectName, error)

ParseTableName parses a table name. The table name must contain one or more name parts, using the full input SQL syntax: each name part containing special characters, or non-lowercase characters, must be enclosed in double quote. The name may not be an invalid table name (the caller is responsible for guaranteeing that only valid table names are provided as input).

func ParseTablePattern

func ParseTablePattern(sql string) (tree.TablePattern, error)

ParseTablePattern parses a table pattern. The table name must contain one or more name parts, using the full input SQL syntax: each name part containing special characters, or non-lowercase characters, must be enclosed in double quote. The name may not be an invalid table name (the caller is responsible for guaranteeing that only valid table names are provided as input). The last part may be '*' to denote a wildcard.

func ParseWithInt

func ParseWithInt(sql string, nakedIntType *types.T) (statements.Statements, error)

ParseWithInt parses a sql statement string and returns a list of Statements. The INT token will result in the specified TInt type.

func PopulateErrorDetails

func PopulateErrorDetails(
	tokID int32, lastTokStr string, lastTokPos int32, lastErr error, lIn string,
) error

PopulateErrorDetails properly wraps the "last error" field in the lexer.

func RunShowSyntax

func RunShowSyntax(
	ctx context.Context,
	stmt string,
	report func(ctx context.Context, field, msg string),
	reportErr func(ctx context.Context, err error),
)

RunShowSyntax analyzes the syntax and reports its structure as data for the client. Even an error is reported as data.

Since errors won't propagate to the client as an error, but as a result, the usual code path to capture and record errors will not be triggered. Instead, the caller can pass a reportErr closure to capture errors instead. May be nil.

func SplitFirstStatement

func SplitFirstStatement(sql string) (pos int, ok bool)

SplitFirstStatement returns the length of the prefix of the string up to and including the first semicolon that separates statements. If there is no including the first semicolon that separates statements. If there is no semicolon, returns ok=false.

Types

type HelpMessage

type HelpMessage struct {
	// Command is set if the message is about a statement.
	Command string
	// Function is set if the message is about a built-in function.
	Function string

	// HelpMessageBody contains the details of the message.
	HelpMessageBody
}

HelpMessage describes a contextual help message.

func (*HelpMessage) Format

func (h *HelpMessage) Format(w io.Writer)

Format prints out details about the message onto the specified output stream.

func (*HelpMessage) String

func (h *HelpMessage) String() string

String implements the fmt.String interface.

type HelpMessageBody

type HelpMessageBody struct {
	Category         string
	ShortDescription string
	Text             string
	SeeAlso          string
}

HelpMessageBody defines the body of a help text. The messages are structured to facilitate future help navigation functionality.

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser wraps a scanner, parser and other utilities present in the parser package.

func (*Parser) Parse

func (p *Parser) Parse(sql string) (statements.Statements, error)

Parse parses the sql and returns a list of statements.

func (*Parser) ParseWithInt

func (p *Parser) ParseWithInt(sql string, nakedIntType *types.T) (statements.Statements, error)

ParseWithInt parses a sql statement string and returns a list of Statements. The INT token will result in the specified TInt type.

type TokenString

type TokenString struct {
	TokenID int32
	Str     string
}

TokenString is the unit value returned by Tokens.

func Tokens

func Tokens(sql string) (tokens []TokenString, ok bool)

Tokens decomposes the input into lexical tokens.