natural-lsp

README

natural-lsp

A Language Server Protocol implementation for Software AG Natural — a Go-based LSP server with a hand-written parser delivering navigation, completion, references, hover, call hierarchy, and semantic highlighting for Natural codebases on any LSP-capable editor.

Natural is a 4GL language widely deployed on IBM z/OS mainframes, typically alongside COBOL, Adabas, and IMS. natls (Java, MIT) is the existing parser-based LSP server for Natural. natural-lsp is a Go alternative built around a hand-written lexer and recursive-descent parser, with config-driven library mapping independent of NaturalONE project files and a git-safe content-hash cache.

It operates on filesystem-based Natural sources — the .NSx object files used by NaturalONE / SPoD — rather than objects stored only in the mainframe Natural/Adabas library system. Natural that lives solely on the mainframe must be exported to files before it can be indexed.

Project status: v1.0.0 — feature-complete for a 1.0 release. The LSP capability surface (navigation, declaration/type-definition, references, workspace symbols, document outline, hover, code lens, push + pull diagnostics, completion, signature help, call hierarchy, semantic tokens, document links, document highlight) is implemented and reviewed. The LSP lifecycle (initialize/shutdown/exit) and Content-Length framing are implemented (features 01–03). textDocument/didOpen, textDocument/didChange, textDocument/didClose, and workspace/didChangeWatchedFiles are wired to an in-memory document store; an fsnotify-based workspace watcher re-analyzes files on disk change (feature 04). Workspace indexing and persistent cache are now implemented (feature 05) with content-hash invalidation for git-safe cache. A hand-written lexer and recursive-descent parser producing an AST with ranged syntax diagnostics is implemented (feature 00) behind the Analyzer interface. Per-file call/dependency extraction is now implemented (feature 06): CALLNAT, PERFORM, INCLUDE, FETCH, and RUN produce relationship edges with caller context (static vs. dynamic distinguished). Cross-file resolution of those edges is now implemented (feature 07): the steplib chain (current library → declared steplibs → SYSTEM, non-transitive), explicit-library bypass for RUN 'pgm' 'lib', inline-before-external PERFORM, INCLUDE→copycode binding, and flat-namespace ambiguity diagnostics — exposed as a workspace-index capability. Adabas data-access extraction is now implemented (feature 08): READ/FIND/GET (read relationships) and STORE plus record-form UPDATE/DELETE (write relationships) against DDMs, DEFINE DATA variable/parameter definitions, and DEFINE WORK FILE definitions. Embedded-SQL extraction is now implemented (feature 08b): native SQL and PROCESS SQL DDM read/write edges, CALLDBPROC call edges, and host-variable references. This is per-file extraction only — cross-file binding (name→DDM/host-var resolution across the steplib chain, and the record-form UPDATE/DELETE enclosing-loop file binding) remains future work. Program-structure extraction is now implemented (feature 09): a per-object hierarchical symbol tree (object root, data sections + fields, subroutines, maps, DDM references) with source ranges, backing document outline / workspace symbols / hover. Navigation & symbol search is now implemented (feature 10): the first shipped LSP providers — textDocument/definition (FR-24), textDocument/references (FR-25), and workspace/symbol (FR-26). The running server now builds and holds a workspace index + resolution set and updates them incrementally on document/watched-file changes. Document outline is now implemented (feature 11): the textDocument/documentSymbol provider (FR-27) renders feature 09's hierarchical symbol tree as a nested DocumentSymbol[], served from the open-document buffer so it tracks unsaved edits. Hover is now implemented (feature 12): the textDocument/hover provider (FR-28) shows module metadata (name, location, inbound/outbound counts), subroutine parameter signatures on PERFORM targets, and DDM field details on data-access statements — the latter backed by a new .NSD DDM field parser. Code lens is now implemented (feature 13): the textDocument/codeLens provider (FR-29) renders an inbound call-count lens and a table-write-summary lens above each object, both activating find-references to reveal the call/write sites; enabled by default and disableable via enable_code_lens. Diagnostics are now published (feature 14): the server pushes textDocument/publishDiagnostics (FR-30 parse errors, FR-31 ambiguous resolution), aggregating the parser's ranged syntax diagnostics with the resolver's flat-namespace ambiguity warnings and tracking edits across didOpen/didChange/didClose/watched-file changes (clearing via an empty array on fix, close, or delete). Modeled gaps (dynamic/unresolved references) are never diagnostics (FR-17). All planned LSP providers are now wired (navigation, document outline, hover, code lens, diagnostics, completion, signature help, call hierarchy). Rich symbol detail & VIEW OF binding is now implemented (feature 28): the document outline gained a typed record layout (field type/level/array/REDEFINE detail), and VIEW OF declarations are parsed, shown with their DDM, and their fields decode to the DDM's logical fields — a bare view field shows the inherited type and go-to-definition reaches the .NSD field. Extends documentSymbol/definition/hover with no new capability. Semantic tokens are now implemented (feature 29): a semanticTokensProvider (full + range) drives AST-aware highlighting — keyword/comment/string/number/operator lexically, plus variable/parameter/call-target/DDM-view-and-field/system-variable classification with declaration/definition/readonly/modification/defaultLibrary modifiers — computed on demand behind the Analyzer seam with no cache-format change (full/delta deferred). (Feature 35) the classifier was then made O(n log n) — a byte-identical fix removing an O(n²) latency cliff (a large file went from ~2.6 s to ~12 ms per request). Pull diagnostics are now implemented (feature 30): the LSP 3.17 pull model (textDocument/diagnostic + workspace/diagnostic, diagnosticProvider capability) serves the same content as the feature-14 push path (byte-identical); push is suppressed for pull-capable clients and a workspace/diagnostic/refresh prompts re-pull of cross-file diagnostics, so nothing double-reports (FR-57, server-layer only, no cache-format change). Declaration & type-definition navigation is now implemented (feature 31): textDocument/declaration (a declarationProvider, delegating to the definition path — Natural has no header/impl split) and textDocument/typeDefinition (a typeDefinitionProvider that jumps a VIEW OF field to the .NSD DDM field that types it, via the feature-28 view→DDM binding and the steplib chain); scalar-only fields with no DDM type object return empty (FR-17). Thin providers over features 10/27/28 resolution — server-layer only, no cache-format change (FR-58). Document links are now implemented (feature 32): a documentLinkProvider renders each resolved CALLNAT/INCLUDE/FETCH/RUN/external-PERFORM target span as a clickable link to the resolved object's file — an always-visible presentation of the same navigation go-to-definition serves. Dynamic/unresolved/ambiguous targets (and inline same-file PERFORMs) get no link (FR-17). Server-layer only, no cache-format change (FR-59). DEFINE DATA … USING data-area navigation is now implemented (feature 36, bug #58): go-to-definition (and declaration/references/document-link) on the data-area name in a DEFINE DATA {LOCAL|PARAMETER|GLOBAL} USING <name> clause now navigates to the .NSL/.NSA/.NSG object via the steplib chain — previously the one module reference with no go-to-definition. A new USES_DATA_AREA edge kind bumps the cache format to 0.10.0 (one-time rebuild). Pre-built binaries for Linux/macOS/Windows are published on GitHub Releases (with checksums); you can also go install or build from source (see Installation).

A full independent assessment (2026-07-14) — docs/assessment-2026-07-14.md — confirmed the core is solid but identified five known issues, re-planned as features 19–23. See Known issues below before relying on completion details, non-VS-Code editors, or large-workspace performance.

---

Features

The capabilities below define the target feature set for the first stable release.

Navigation (providers shipped — feature 10)

• Jump to definition for CALLNAT, FETCH, RUN, and PERFORM targets — shipped
• Find all references to a subroutine, program, or DDM field across the workspace — shipped
• Workspace symbol search by program name or subroutine — shipped

Hover (provider shipped — feature 12)

• Program metadata: module name, location, inbound call count (+ a single outbound-dependency count) — shipped
• Subroutine signatures on PERFORM targets (inline and external .NSS), from the target's PARAMETER section — shipped
• DDM field names and types on data-access statements, parsed from the referenced .NSD — shipped (physical Adabas/IMS runtime metadata remains out of scope; an un-indexed DDM shows an honest "unavailable" message rather than fabricated fields)

Document outline (provider shipped — feature 11; typed detail + VIEW OF — feature 28)

• Full symbol tree: DEFINE DATA sections, subroutines, maps, DDM references — the underlying hierarchical symbol model (feature 09, FileAnalysis.Structure) and the textDocument/documentSymbol provider that renders it as a nested DocumentSymbol[] are both shipped. The outline is served from the open-document buffer, so it reflects unsaved edits.
• Typed record layout (feature 28): each field carries a Detail string — verbatim type (A26/P9,2/(A) DYNAMIC), array index ranges ((1:10), (1:5,1:10), (1:*) — never "OCCURS"), and REDEFINE overlays (<type> REDEFINE <target>, FILLER nX gaps) — shipped
• VIEW OF binding (feature 28): level view-name VIEW [OF] ddm-name is parsed and shown as a VIEW OF <ddm> node; a bare view field inherits its type from the referenced .NSD and go-to-definition on it jumps to the DDM field declaration (view name → its own line; a view-local REDEFINE sub-field → the view's own line) — shipped (a TYPE: SQL / DB2-backed DDM is not yet parsed, so its view fields list but don't type-resolve — a recorded limit)

Diagnostics (provider shipped — feature 14)

• Parse errors surfaced as diagnostics at the offending position, with a message identifying the construct — shipped (FR-30); blank lines and comments produce none
• Ambiguous resolution (a name matching objects in more than one library with no library map) reported as a distinct warning-severity diagnostic naming the candidates — shipped (FR-31); declaring a disambiguating library map removes it
• Diagnostics track edits and config (open-document didChange and external watched-file changes), clearing when the cause is fixed — shipped; modeled outcomes (dynamic/unresolved references) are never diagnostics — shipped (FR-17)
• Both delivery models: push (textDocument/publishDiagnostics, feature 14) and the LSP 3.17 pull model (textDocument/diagnostic + workspace/diagnostic, feature 30) — shipped (FR-57); pull content is byte-identical to push, and push is suppressed for pull-capable clients (with workspace/diagnostic/refresh prompting re-pull of cross-file diagnostics) so nothing double-reports

Workspace indexing (full implementation shipped)

• Open-document tracking via in-memory store (textDocument/didOpen|didChange|didClose) — shipped
• On-disk file change detection via fsnotify + workspace/didChangeWatchedFiles — shipped
• Per-file extraction of static CALLNAT 'LITERAL' calls — shipped (cross-file resolution of the edges — shipped, feature 07: steplib chain, explicit-library bypass, flat-namespace ambiguity diagnostics)
• INCLUDE / copycode dependency tracking — shipped
• Dynamic CALLNAT #VARIABLE calls flagged as unresolved with caller context preserved — shipped
• Persistent cache across sessions (sub-second startup after first index) — shipped

LSP protocol compliance

• textDocument/definition — shipped (feature 10)
• textDocument/declaration (a declarationProvider mirroring definition — call/transfer/subroutine targets and variable use → DEFINE DATA) — shipped (feature 31)
• textDocument/typeDefinition (a typeDefinitionProvider jumping a VIEW OF field to the .NSD DDM field that types it; scalar-only fields → empty) — shipped (feature 31)
• textDocument/documentLink (a documentLinkProvider — resolved CALLNAT/INCLUDE/FETCH/RUN/external-PERFORM targets as clickable links to the resolved object; dynamic/unresolved/ambiguous → no link) — shipped (feature 32)
• textDocument/references — shipped (feature 10)
• workspace/symbol — shipped (feature 10)
• textDocument/hover — shipped (feature 12)
• textDocument/completion (module names, subroutine names, DDM field names) — shipped (feature 16; the detail/sortText wire defect was fixed in feature 19)
• textDocument/signatureHelp (parameter interfaces at call sites) — shipped (feature 17)
• textDocument/callHierarchy (prepare + incoming/outgoing call panels) — shipped (feature 18)
• textDocument/documentSymbol — shipped (feature 11)
• textDocument/codeLens (call counts, table write summaries) — shipped (feature 13)
• textDocument/publishDiagnostics (parse errors, ambiguous resolution) — shipped (feature 14)
• textDocument/diagnostic + workspace/diagnostic (LSP 3.17 pull model over the same content; diagnosticProvider capability, push suppressed for pull clients, workspace/diagnostic/refresh on republish) — shipped (feature 30)
• window/workDoneProgress (indexing progress on first run) — shipped (FR-32, feature 21; gated on the client's window.workDoneProgress capability)
• window/logMessage + the trace/$/setTrace/$/logTrace handshake (LSP-native logging/tracing; --log-level stderr flag) — shipped (feature 26)
• textDocument/definition / references extended to data variables and SQL host variables (jump to the DEFINE DATA declaration, same-file and cross-file via USING data areas / SQL DDM names through the steplib chain) — shipped (feature 27)
• textDocument/documentHighlight (highlight a symbol's occurrences in the file) — shipped (feature 27)
• textDocument/semanticTokens (full + range; server-driven, AST-aware highlighting — keywords, comments, strings, numbers, operators lexically, plus variables/parameters/call targets/DDM views & fields/system variables semantically, with declaration/definition/readonly/modification/defaultLibrary modifiers) — shipped (feature 29; full/delta deferred)

---

Known issues (2026-07-14 assessment)

An independent full-project assessment — live stdio wire probes, four specialist reviews, and LSP 3.17 spec verification — is recorded in docs/assessment-2026-07-14.md. The lifecycle, position encoding, capability set, diagnostics semantics, robustness (FR-43), and the Analyzer seam all verified clean. Five issues were confirmed and re-planned:

1. Completion results are corrupted on the wire — FIXED in feature 19. CompletionItem.detail/sortText no longer serialize as {}; all provider results now marshal via the json/v2 path (gojson.Marshal/MarshalJSONTo), guarded by wire-bytes tests and a regression check against reintroducing stdlib json.Marshal.
2. The server ignores rootUri/workspaceFolders — FIXED in feature 20. The server now negotiates the workspace root from the initialize handshake (workspaceFolders → rootUri → cwd sentinel walk-up), so any LSP client works regardless of the server's launch directory. An empty or unresolved workspace is surfaced via an stderr warning and a window/showMessage notification rather than silently returning nothing.
3. No indexing progress reporting — FIXED in feature 21. The server sends window/workDoneProgress create → begin → report (N/M files) → end during indexing when the client advertises the capability.
4. The cold index build blocks all requests — FIXED in feature 21. The initial index build runs on a background goroutine, so the editor stays responsive; requests degrade to null/empty until the index is ready, and edits made during the build are replayed into it.
5. Performance/scale claims are unmeasured — ADDRESSED in feature 22. A benchmark suite (just bench, off the default gate) now records NFR-1/2/3/4 verdicts (measure-and-record; see the plan's Results section), and the two per-query hot spots were fixed: workspace/symbol and references no longer re-read files from disk (~46×/~34× faster via an in-memory line-width table), and the completion name index is cached (~87× faster). Honest caveat: measured at ≤4k objects and extrapolated to tens of thousands; warm-start latency at very large scale remains a future optimization.

Secondary items were addressed in feature 23: the Go module was renamed to github.com/dkrieg/natural-lsp so the documented go install resolves, and scripts/smoke.sh's no-argument binary resolution was fixed. (A Homebrew tap remains deferred — see the installation section.)

---

Parser-based extraction

natural-lsp uses a hand-written lexer and recursive-descent parser for Natural, modeled on natls (the Java reference implementation). The parser produces a proper AST that enables completion, signature help, call hierarchy, real syntax diagnostics, and accurate symbol tables — features that regex extraction cannot deliver reliably.

The parser also handles embedded SQL: native Natural SQL statements (SELECT/SELECT SINGLE, INSERT, SQL-form UPDATE/DELETE, MERGE, COMMIT, ROLLBACK, CALLDBPROC, READ RESULT SET) are parsed into the AST with syntax diagnostics, accepting both the structured (END-SELECT/END-RESULT) and reporting-mode (LOOP) loop terminators, and PROCESS SQL bodies are held as a single opaque, unparsed <<…>> span. Embedded-SQL extraction (feature 08b) then walks that AST and emits the relationships those statements carry: DDM read edges (SELECT/SELECT SINGLE/READ RESULT SET), DDM write edges (INSERT/SQL-UPDATE/ SQL-DELETE/MERGE), a CALLDBPROC call edge, and host-variable references — including a colon-mandatory scan of the PROCESS SQL opaque body (in-body table names are pass-through text and are not bound). What remains is cross-file resolution — binding those extracted SQL table names to DDMs and host-variables to their declarations across the steplib chain.

Two kinds of analysis gap are handled separately, and neither is dropped silently:

• Unresolvable references — e.g. CALLNAT #VARIABLE, whose target cannot be determined statically — are noted as unresolvable with the call site preserved, so they appear in find-references and outline rather than disappearing. (Implemented for CALLNAT/FETCH/RUN as *_DYNAMIC edges that retain the call site; cross-file resolution (feature 07) binds resolvable references and records these as Unresolved(dynamic) outcomes that keep the call site.)
• Parse errors — source the parser cannot interpret — are surfaced as LSP diagnostics so they are visible in the editor, not silently discarded.

The parser sits behind the Analyzer interface so the backend can evolve (e.g. to a tree-sitter grammar) without touching the LSP layer.

---

Installation

Pre-built binary (recommended)

Download the appropriate binary for your platform from GitHub Releases:

natural-lsp-linux-amd64
natural-lsp-linux-arm64
natural-lsp-darwin-amd64
natural-lsp-darwin-arm64
natural-lsp-windows-amd64.exe

Each release also publishes a checksums.txt (SHA-256 of every binary) — verify your download against it. Place the binary somewhere on your PATH:

# Linux / macOS
chmod +x natural-lsp-linux-amd64
mv natural-lsp-linux-amd64 /usr/local/bin/natural-lsp

# Verify
natural-lsp --version

Build from source

Requires Go 1.26+.

git clone https://github.com/dkrieg/natural-lsp
cd natural-lsp
go build -o natural-lsp ./cmd/natural-lsp

go install

The module path is github.com/dkrieg/natural-lsp, so once a release tag is published this resolves:

go install github.com/dkrieg/natural-lsp/cmd/natural-lsp@latest

(@latest requires at least one pushed release tag; before the first release, install from a clone: git clone https://github.com/dkrieg/natural-lsp && cd natural-lsp && go build -o natural-lsp ./cmd/natural-lsp.)

A native OS package-manager channel (Homebrew tap, Scoop) is future work — it needs a dedicated tap repository and a committed release cadence, deferred until the release line stabilizes. The pre-built binaries and go install cover installation in the meantime.

Verify an install

Any binary — pre-built, built-from-source, or go installed — can be smoke-checked with the bundled script, which runs --version and a minimal initialize → initialized → shutdown → exit stdio round-trip:

scripts/smoke.sh "$(command -v natural-lsp)"

(The underlying one-liner the script builds on is natural-lsp --stdio < /dev/null, which must start and exit cleanly on EOF.)

Logging & tracing

The server surfaces its own activity through the standard LSP channels, so you can see what it's doing inside the editor (the JetBrains/LSP4IJ Logs tab, the VS Code output channel, etc.):

• window/logMessage — operational events: index build begin/end (with a warm-cache-hit vs. rebuild signal), a build-end summary of any skipped files, resolution ambiguities, and request errors. Always emitted, severity-tagged (Info/Warning/Error).
• $/logTrace — per-request RPC tracing, driven by the editor's Trace setting (off / messages / verbose); off by default. The server honors the trace value from initialize and the $/setTrace notification, so changing the Trace level takes effect without a restart. verbose payloads are size-capped.

Independently, a --log-level CLI flag controls the server's own stderr verbosity (this is separate from the LSP trace level, which the editor governs):

natural-lsp --stdio --log-level debug   # error | warn | info (default) | debug

An unrecognized value falls back to the default level with an actionable message (never crashes).

---

Editor setup

Workspace root: the server negotiates the workspace root from the initialize handshake — workspaceFolders first, then the (deprecated) rootUri, then a .natural-lsp.toml sentinel walk-up from the server's working directory as a fallback. Any LSP client that opens a folder (VS Code, Neovim, Zed, Helix, JetBrains) works regardless of where the natural-lsp process is launched from. If no root can be established or the workspace has no indexable Natural files, the server logs an actionable warning and sends a window/showMessage notification rather than silently indexing nothing.

VS Code

The companion extension lives in this repo under editors/vscode/. It is distributed as a .vsix (not on the VS Code Marketplace). Each GitHub Release publishes a platform-specific .vsix that bundles the natural-lsp server binary — download the one matching your OS/arch and the extension works with no separate server install (feature 37):

# From the GitHub Release, pick your platform: linux-x64 | linux-arm64 | darwin-x64 | darwin-arm64 | win32-x64
code --install-extension natural-lsp-vscode-<version>-<platform>.vsix

To build one from source instead (packages the server binary you place in editors/vscode/bin/, or none — then it falls back to PATH):

cd editors/vscode
npm ci
npm run package                 # produces a natural-lsp-vscode-<version>.vsix
code --install-extension natural-lsp-vscode-<version>.vsix

The extension launches the server automatically when a Natural source file (.NSP, .NSN, .NSS, .NSC, .NSM, .NSL, .NSG, .NSA, .NSH, .NSD, .NS4, .NS7, .NS3, .NS8, .NST) is opened. It resolves the server in this order: the naturalLsp.serverPath setting (if set) → the bundled binary shipped in the .vsix (if present) → natural-lsp on your PATH. So a platform-specific .vsix needs no configuration, and a source-built/generic .vsix still works when natural-lsp is on PATH.

To point at a specific binary (e.g. a dev build), add to .vscode/settings.json — this overrides the bundled binary:

{
  "naturalLsp.serverPath": "/path/to/natural-lsp"
}

The extension version tracks the server's release line. See editors/vscode/README.md for development, testing, and packaging details.

Neovim

Neovim needs two things: a natural filetype for .NSx files (Neovim has no built-in detection for them) and an LSP client that launches natural-lsp --stdio for that filetype. The root_markers/root_dir settings below set the workspace root the server honors (via workspaceFolders/rootUri), with the .natural-lsp.toml sentinel as the marker.

1. Associate .NSx files with the natural filetype (required for either LSP form below — without it the filetypes/filetype gate never matches):

vim.filetype.add({
  pattern = {
    ['.*%.[nN][sS].'] = 'natural',  -- matches .NSP/.NSN/…/.NST (any case)
  },
})

The equivalent autocmd form, if you prefer:

vim.api.nvim_create_autocmd({ 'BufRead', 'BufNewFile' }, {
  pattern = { '*.NS*', '*.ns*' },
  command = 'set filetype=natural',
})

2a. Neovim 0.11+ (recommended — built-in vim.lsp.config/vim.lsp.enable):

vim.lsp.config('natural_lsp', {
  cmd = { 'natural-lsp', '--stdio' },
  filetypes = { 'natural' },
  root_markers = { '.natural-lsp.toml', '.git' },
})
vim.lsp.enable('natural_lsp')

2b. Alternative — nvim-lspconfig (still supported; requires Neovim 0.11.3+):

require('lspconfig').configs['natural_lsp'] = {
  default_config = {
    cmd = { 'natural-lsp', '--stdio' },
    filetypes = { 'natural' },
    root_dir = require('lspconfig.util').root_pattern(
      '.natural-lsp.toml', '.git'
    ),
  }
}
require('lspconfig').natural_lsp.setup({})

Verify (against docs/plans/features/15-editor-clients/sample-workspace/):

• Server-side smoke first: scripts/smoke.sh "$(command -v natural-lsp)" passes.
• Open HELLO.NSP from the sample workspace; :set filetype? reports filetype=natural and :LspInfo (or :checkhealth lsp) shows natural_lsp attached.
• Caret on CALLGREET → vim.lsp.buf.definition() (default grd/gd) navigates to CALLGREET.NSN; on SAYHELLO it navigates to SAYHELLO.NSS.

Zed

Zed's settings.json can associate file extensions with a language and point a known language at a custom server binary, but it cannot define a brand-new language (nor bind a language server to one) on its own — Zed only runs a language server for a language it already knows, and adding a new language requires a Zed language extension. A first-party Natural Zed extension (which would register the natural language, its Tree-sitter grammar, and the natural-lsp server binding) is future work.

In the meantime you can associate the .NSx extensions and declare the server so they are ready once a natural language is available. Add to your settings.json:

{
  "file_types": {
    "Natural": [
      "NSP", "NSN", "NSS", "NSC", "NSM", "NSL", "NSG", "NSA",
      "NSH", "NSD", "NS4", "NS7", "NS3", "NS8", "NST"
    ]
  },
  "lsp": {
    "natural-lsp": {
      "binary": {
        "path": "natural-lsp",
        "arguments": ["--stdio"]
      }
    }
  },
  "languages": {
    "Natural": {
      "language_servers": ["natural-lsp"]
    }
  }
}

The server itself launches identically to every other editor (natural-lsp --stdio) and detects the workspace root via the .natural-lsp.toml sentinel.

Verify:

• Server-side smoke: scripts/smoke.sh "$(command -v natural-lsp)" passes (the automatable lower bound — full in-editor navigation depends on the pending Natural language extension above).
• .NSx files open and are associated with the Natural language entry (once the language extension is installed, go-to-definition on the sample workspace's CALLGREET/SAYHELLO navigates to their definitions).

Helix (languages.toml)

[[language]]
name = "natural"
scope = "source.natural"
file-types = ["NSP", "NSN", "NSS", "NSC", "NSM", "NSL", "NSG", "NSA", "NSH", "NSD", "NS4", "NS7", "NS3", "NS8", "NST"]
language-servers = ["natural-lsp"]

[language-server.natural-lsp]
command = "natural-lsp"
args = ["--stdio"]

This registers a natural language for all 15 .NSx file types and launches natural-lsp --stdio. Helix does not pass explicit root markers here — the server itself finds the workspace root by walking up for the .natural-lsp.toml sentinel, so run Helix from within your Natural project.

Verify (against docs/plans/features/15-editor-clients/sample-workspace/):

• Server-side smoke: scripts/smoke.sh "$(command -v natural-lsp)" passes.
• Open HELLO.NSP; :lsp-workspace-command / the status line shows natural-lsp active, and hx --health lists the natural language.
• Caret on CALLGREET → goto definition (gd) navigates to CALLGREET.NSN; on SAYHELLO it navigates to SAYHELLO.NSS.

JetBrains IDEs (IntelliJ, PyCharm, …)

JetBrains does not auto-discover LSP servers the way VS Code does. The recommended route — which works in all JetBrains IDEs including the free Community editions — is the free LSP4IJ plugin. This repo ships an importable LSP4IJ server template (all 15 Natural file types; programArgs set to natural-lsp --stdio, with a windows variant natural-lsp.exe --stdio) so setup is reproducible:

1. Install LSP4IJ from the JetBrains Marketplace.
2. Settings/Preferences → Languages & Frameworks → Language Servers → [+] → Template → Import from custom template… and select editors/jetbrains/lsp4ij-template/.
3. Open the project root containing your .natural-lsp.toml sentinel and open a Natural file.

Full step-by-step instructions (including a manual by-hand alternative and a Verify checklist) are in editors/jetbrains/README.md.

The native JetBrains LSP API (com.intellij.platform.lsp) is an alternative, but it requires a paid/Ultimate-tier IDE and a custom plugin — LSP4IJ is the simpler, more portable path.

---

Workspace configuration

The server locates the workspace root by walking up from the opened file (or the LSP initialize workspace root) looking for a .natural-lsp.toml sentinel file. Place this file at your Natural codebase root.

To generate a fully-commented starter config with every key shown at its default:

natural-lsp --init > .natural-lsp.toml

All keys are optional — the server applies defaults for any key you omit. The full schema:

# .natural-lsp.toml

[workspace]
# Object types to index (defaults shown). The default set covers all 15
# recognized Natural constructs. Exact extensions depend on how your objects
# were exported — adjust to match your tooling.
extensions = [
  # Core types
  ".NSP",  # program
  ".NSN",  # subprogram
  ".NSS",  # external subroutine
  ".NSC",  # copycode (INCLUDE targets)
  ".NSM",  # map
  ".NSL",  # local data area
  ".NSG",  # global data area
  ".NSA",  # parameter data area
  ".NSH",  # helproutine
  ".NSD",  # DDM
  # Extended types
  ".NS4",  # class (NaturalX)
  ".NS7",  # function (user-defined)
  ".NS3",  # dialog (Natural for Windows)
  ".NS8",  # adapter (Natural Ajax)
  ".NST",  # text
]

# Map non-standard extensions to their construct type. Useful when files were
# exported with a different suffix (e.g. legacy tools using .NAT for programs).
# Valid values: program, subprogram, externalsubroutine, copycode, map,
#   localdataarea, globaldataarea, parameterdataarea, helproutine, ddm,
#   class, function, dialog, adapter, text
# [workspace.extension_types]
# ".NAT" = "program"

# Directories to exclude from indexing
exclude = ["archive", "backup", ".git"]

# Maximum file size to index (bytes)
max_file_size = 5_000_000

[cache]
# Where to write the workspace index cache
# Defaults to .natural-lsp-cache/ at workspace root
path = ".natural-lsp-cache"

[analysis]
# Treat CALLNAT #VARIABLE as an unresolved external dependency
# rather than an error. Default: true
flag_dynamic_calls = true

# Minimum token length to consider a string literal a potential
# module name in dynamic CALLNAT resolution heuristics
dynamic_call_min_length = 6

# When true, the server advertises and serves code lens providers
# (inbound call counts, table write summaries). Default: true
enable_code_lens = true

[resolution]
# Natural resolves CALLNAT / PERFORM / FETCH targets by walking a steplib
# chain — current library first, then each steplib in order, then SYSTEM —
# NOT by file path. The same module name can exist in multiple libraries,
# so the search order is what disambiguates.
#
# Map workspace directories to Natural libraries and declare each library's
# steplib search order. If no libraries are declared, the server treats the
# whole workspace as a single flat namespace and emits a diagnostic when a
# name resolves ambiguously.
[[resolution.library]]
name = "MYAPP"
path = "src/MYAPP"
steplibs = ["COMMON", "SYSTEM"]

[[resolution.library]]
name = "COMMON"
path = "src/COMMON"

---

Workspace indexing

On first open, the server indexes the entire workspace. Progress is reported via window/workDoneProgress — your editor will show a status bar indicator:

Natural LSP: Indexing workspace… 1,243 / 2,891 files (43%)

The completed index is serialized to .natural-lsp-cache/ (gitignored by default) as gzip-compressed compact JSON — a compact encoding that keeps the cache a small fraction of its raw size (~100× smaller than indented JSON on a representative corpus). Subsequent startups load from cache and re-analyze only files whose content hash has changed since the last run (content hashing rather than mtime keeps the cache valid across git checkouts; a cache-format version forces a full rebuild on upgrade). The figures below are design targets, not measured benchmarks — cold index time is expected to scale roughly linearly with codebase size:

Codebase size	First index	Subsequent startup
500 files	~3s	<1s
5,000 files	~25s	<1s
30,000 files	~3min	<1s

---

Supported Natural constructs

Call relationships

Construct	Resolution	Edge type
CALLNAT 'LITERAL'	Static — resolved to definition	CALLS
CALLNAT #VARIABLE	Dynamic — surfaced as unresolvable	CALLS_DYNAMIC
FETCH 'LITERAL'	Static — navigation edge	NAVIGATES_TO
FETCH #VARIABLE	Dynamic — surfaced as unresolvable	NAVIGATES_TO_DYNAMIC
RUN 'LITERAL'	Static — navigation edge	NAVIGATES_TO
RUN #VARIABLE	Dynamic — surfaced as unresolvable	NAVIGATES_TO_DYNAMIC
PERFORM subroutine	Local scope first, then external	PERFORMS
INCLUDE copycode	Resolved to copycode file	INCLUDES

A literal CALLNAT/FETCH/RUN target containing an & runtime-substitution placeholder (e.g. CALLNAT 'PRG&LANG') is treated as dynamic — it produces the *_DYNAMIC edge rather than a false static edge to a non-existent object. RUN 'PGM' library-id records the named target library on the edge.

Data access (extraction shipped — feature 08)

Construct	Extracted
READ / FIND / GET	File/DDM name, read relationship — shipped (GET SAME → no edge)
STORE	File/DDM name, write relationship — shipped
record UPDATE / DELETE	Write relationship at the site, no file name (bound from the enclosing loop — resolution deferred, OQ-4) — shipped
DEFINE DATA	Variable declarations + parameter interfaces (level/type/dimensions/section kind, REDEFINE nesting) — shipped
DEFINE WORK FILE	Work-file definitions (number + name; dynamic names recorded verbatim) — shipped
.NSD DDM files	Field name/type/level, group nesting, MU/PE arrays — parsed from the exported DDM report into FileAnalysis.Definitions (fixed-column line-scanner) — shipped (feature 12; DB short-name/descriptor/suppression columns dropped, SQL DDMs out of scope)

Scope is Adabas-style data access against DDMs.

Embedded-SQL access (extraction shipped — feature 08b)

Construct	Extracted
SELECT / SELECT SINGLE	FROM table → DDM read relationship — shipped
READ RESULT SET	Read-access site (result-set handle is not a DDM → empty name) — shipped
INSERT / SQL-UPDATE / SQL-DELETE / MERGE	Target table → DDM write relationship — shipped
CALLDBPROC	Stored-procedure call edge (literal → CALLS; variable/&-placeholder → CALLS_DYNAMIC) — shipped
PROCESS SQL	ddm-name → DDM read edge; opaque <<…>> body scanned for colon host-vars (in-body table names not bound) — shipped
host variables (native + opaque body)	HostVarRef references (bare-or-colon in native clauses; colon-mandatory in the opaque body) — shipped

SQL table operands are .NSD DDM names (the same namespace as Adabas). Per-file extraction only — binding these SQL-sourced DDM/host-var references to their definitions (cross-file resolution) is future work.

Program structure (extraction shipped — feature 09)

Each object is walked into a per-object hierarchical symbol tree (FileAnalysis.Structure), the backbone for document outline, workspace symbols, and hover. Every node carries a source Range (the whole construct) and a SelectionRange (the name token), mirroring LSP DocumentSymbol.

Symbol kind	Source
object (root)	the object itself (name = file base without extension) — shipped
data section	each DEFINE DATA section (LOCAL/PARAMETER/…), fields nested as children incl. REDEFINE — shipped
data field	DEFINE DATA items (level/array/REDEFINE nesting) — shipped
subroutine	DEFINE SUBROUTINE — shipped
map	DEFINE MAP (with its fields) — shipped
DDM reference	named READ/FIND/GET/STORE + SQL table references — shipped

Children are deterministically source-ordered; fields are grouped into their owning section by source-range containment (so multiple same-kind sections keep their own fields). Deferred: inline-vs-external subroutine distinction, type-specific members for helproutines/classes/functions, and INCLUDE copycode as an outline node. Both the workspace/symbol provider over this tree (feature 10) and the per-file textDocument/documentSymbol (outline) provider that renders it (feature 11) are shipped.

Program structure

Construct	Symbol kind
Program file root	Program
DEFINE SUBROUTINE	Subroutine
DEFINE DATA LOCAL/GLOBAL/PARAMETER	DataSection
DEFINE MAP / .NSM files	Map
DDM references	DDM

---

Architecture

cmd/natural-lsp/
  main.go                  Binary entrypoint — stdio LSP server

internal/
  config/
    config.go              .natural-lsp.toml parsing, defaults, validation,
                           workspace-root discovery, library map

  server/
    server.go              LSP lifecycle + dispatch; builds/holds the workspace
                           index + resolution; incremental update on change
    position.go            model<->protocol position/range conversion (ADR-008)
    cursor.go              cursor position -> reference-site (edge/data-access) lookup
    definition.go          textDocument/definition provider
    references.go          textDocument/references provider (reverse sweep)
    workspace_symbols.go   workspace/symbol provider (Structure name search)
    document_symbols.go    textDocument/documentSymbol provider (outline; store-first)
    degradation.go         per-file analyze + graceful-degradation helpers
    handlers.go            (stub) package doc + TODO
    progress.go            (stub) window/workDoneProgress helpers — TODO
    diagnostics.go         (stub) diagnostic aggregation/publish — TODO

  document/
    store.go               In-memory document store (didOpen/didChange/didClose)
    sync.go                File watcher for workspace files

  workspace/
    index.go               Cross-file symbol table
    cache.go               Serialize/deserialize index to disk
    resolution.go          Steplib-chain resolution: current library → ordered
                           steplibs → system; library map; ambiguity diagnostics

  model/
    model.go               Analyzer output types (FileAnalysis, symbols, edges,
                           data access, definitions, work files)
                           — the contract shared by analysis, workspace, server;
                           free of backend internals

  analysis/
    analyzer.go            Analyzer interface (the replaceable-backend seam)
    natural/
      analyzer.go          Parser-based extraction pipeline (hand-written
                           lexer + recursive-descent parser)
      hover.go             Hover content builders
      calls.go             CALLNAT / FETCH / RUN / PERFORM extraction
                           (produces unresolved references; see resolution.go)
      data.go              DEFINE DATA / READ / FIND / GET / STORE /
                           record UPDATE|DELETE / DEFINE WORK FILE extraction
      sql.go               Embedded-SQL extraction (DDM edges, CALLDBPROC,
                           host-var refs, PROCESS SQL opaque-body scan)
      structure.go         Program-structure extraction (per-object
                           hierarchical symbol tree for outline/symbols/hover)

editors/
  vscode/                  VS Code companion extension (TypeScript)
  jetbrains/               JetBrains integration (LSP4IJ config / plugin)
                           Neovim / Zed / Helix are configured via the docs above

testdata/
  workspace/               Sanitized Natural programs for integration tests
                           (include multi-library cases for resolution)
  *.NSP                    Unit test fixtures per construct

Extraction vs. resolution. Per-file extraction (analysis/natural/) produces unresolved references with caller context. Cross-file resolution (workspace/resolution.go) walks the steplib chain and the configured library map to bind those references to definitions — keeping the highest-risk logic out of the parser backend and behind the workspace index.

---

Development

Required software

All tooling is cross-platform — install via your OS package manager or the official instructions linked below. Versions are specified, not install commands, so this is OS-independent.

Tool	Version	Purpose	Install
Go	1.26 or newer	build and test the server	https://go.dev/doc/install
just	1.0 or newer	task runner for the dev commands below	https://just.systems/man/en/packages.html
Git	any recent	version control and the pre-push hook	https://git-scm.com/downloads
actionlint	optional	lint the GitHub Actions workflow locally	https://github.com/rhysd/actionlint/blob/main/docs/install.md

After cloning, enable the pre-push gate once:

just install-hooks   # configures the pre-push hook to run `just verify`

Common tasks

just --list             # list all recipes
just verify             # full gate: gofmt + vet + build + unit (-race) + integration tests
just test               # unit tests with the race detector
just test-integration   # integration tests (builds the binary, runs the `integration` build tag)
just build              # build the server binary
./natural-lsp --stdio < /dev/null   # smoke test: should print initialize response shape

just verify is the single gate that runs locally (via the pre-push hook) and in CI — so if it passes locally, CI should pass. There is no need to memorize the underlying go commands; just --list is the entry point.

Releases

Releases are cut by maintainers from the GitHub Actions → Release → Run workflow button (a manual workflow_dispatch). Enter the version tag (e.g. v1.2.3); the workflow runs the full just verify gate, cross-compiles every platform via just release, then creates the git tag and a GitHub Release with the binaries and checksums.txt attached. Dispatch it from the main commit you intend to release.

To produce the same artifacts locally (into dist/):

just release v1.2.3

Adding a test case

When you encounter a Natural construct that the analyzer handles incorrectly:

1. Create a minimal .NSP file in testdata/ that reproduces the issue
2. Write a unit test in internal/analysis/natural/analyzer_test.go asserting the expected extraction
3. Fix the analyzer
4. The testdata file becomes a permanent regression fixture

---

Known limitations

• Library / steplib resolution is configuration-driven (see [resolution]). Without a declared library map, the workspace is treated as a single flat namespace, and modules sharing a name across libraries cannot be disambiguated.
• Dynamic CALLNAT #VARIABLE calls cannot be statically resolved. The call site is retained so they appear in find-references and outline rather than disappearing silently.
• Adabas verbs (READ, FIND, GET against Adabas files) are extracted structurally, and DDM field definitions are parsed from exported .NSD files (feature 12) so hover can show field name/type. The physical/runtime Adabas DDM metadata (occurrence counts, storage) is still not resolved, and cross-file resolution of DDM references is future work. IMS segment metadata requires external configuration.
• Natural preprocessor macros and code generation constructs may not extract correctly.
• Column-sensitive syntax (fixed-format Natural) is handled for common patterns; unusual legacy formatting may produce incomplete extraction rather than errors.

---

License

MIT. See LICENSE.

---

Contributing

Issues and PRs welcome. If you encounter a Natural construct that the analyzer mishandles, opening an issue with a minimal reproducer is the most useful contribution. Testdata fixtures of sanitized (non-proprietary) Natural code that exercise edge cases are particularly valuable.