wcgi

package module
v0.0.0-...-78872e6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 24, 2023 License: MIT Imports: 27 Imported by: 0

README

MIT licensed CI status Report Support me on Patreon

WCGI for Caddy

Package wcgi implements “WCGI” (CGI for WASI binaries) for Caddy 2, a modern, full-featured, easy-to-use web server, by embedding the wazero WebAssembly runtime.

It has been forked from the caddy-cgi plugin.

Documentation

Requirements

  • This module needs to be installed (obviously).

    Refer to the Caddy documentation on how to build Caddy with plugins/modules.

  • The directive needs to be registered in the Caddyfile:

    {
    order wcgi before respond
}

    Adjust the order as needed. Putting WCGI before other response-writing handlers should be a sane default, since the WCGI module is typically used with a specific matcher and will then take over completely. Manipulating the request after the application responds is unlikely to be necessary.

Basic Syntax

The basic wcgi directive lets you add a handler in the current caddy router location with a given script and optional arguments. The matcher is a default caddy matcher that is used to restrict the scope of this directive. The directive can be repeated any reasonable number of times. Here is the basic syntax:

wcgi [matcher] exec [args...]

For example:

wcgi /report /app/handlers/report.wasm

When a request such as https://example.com/report or https://example.com/report/weekly arrives, the wcgi middleware will detect the match and run /app/handlers/report.wasm. The root directory exposed to the runtime will by default be the same as the working directory of Caddy itself, but you’ll probably want to change that using the advanced syntax below.

The environment variables PATH_INFO and QUERY_STRING are populated and passed to the script automatically. There are a number of other standard CGI variables included that are described below. If you need to pass any special environment variables or allow any environment variables that are part of Caddy's process to pass to your script, you will need to use the advanced directive syntax described below.

Beware that in Caddy v2 it is (currently) not possible to separate the path left of the matcher from the full URL. Therefore if you require your WCGI program to know the SCRIPT_NAME, make sure to pass that explicitly:

wcgi /script.cgi* /path/to/app.wasm someargument {
  script_name /script.cgi
}
Advanced Syntax

In order to specify custom environment variables, pass along one or more environment variables known to Caddy, or specify more than one match pattern for a given rule, you will need to use the advanced directive syntax. That looks like this:

wcgi [matcher] exec [args...] {
    compile_cache cache_path
    script_name subpath
    dir working_directory
    env key1=val1 [key2=val2...]
    pass_env key1 [key2...]
    pass_all_env
    buffer_limit <size>
    unbuffered_output
    inspect
}

For example,

wcgi /sample/report* /app/report.wasm {
    compile_cache /var/tmp/caddy
    script_name /sample/report
    dir /app/data/report
    env DB=/db/main.db SECRET=/secret CGI_LOCAL=
    pass_env HOME UID
}

The compile_cache subdirective enables ahead-of-time compilation with caching (using the provided cache location), speeding up the performance of the application at the cost of some storage used for caching the compiled native code.

The script_name subdirective helps the cgi module to separate the path to the script from the (virtual) path afterwards (which shall be passed to the script).

env can be used to define a list of key=value environment variable pairs that shall be passed to the script. pass_env can be used to define a list of environment variables of the Caddy process that shall be passed to the script.

If your application runs properly at the command line but fails to run from Caddy it is possible that certain environment variables may be missing.

The pass_all_env subdirective instructs Caddy to pass each environment variable it knows about to the WCGI excutable. This is rather silly with how sandbox-focused the whole WASI thing is, but it’s a leftover from caddy-cgi which didn’t really have a huge reason to get deleted.

buffer_limit is used when a http request has Transfer-Endcoding: chunked. The Go CGI Handler refused to handle these kinds of requests, see https://github.com/golang/go/issues/5613. In order to work around this the chunked request is buffered by caddy and sent to the WCGI application as a whole with the correct CONTENT_LENGTH set. The buffer_limit setting marks a threshold between buffering in memory and using a temporary file. Every request body smaller than the buffer_limit is buffered in-memory. It accepts all formats supported by go-humanize. Default: 4MiB.
(An example of this is git push if the objects to push are larger than the http.postBuffer)

With the unbuffered_output subdirective it is possible to instruct the handler to flush output from the application as soon as possible. By default, the output is buffered into chunks before it is being written to optimize the network usage and allow to determine the Content-Length. When unbuffered, bytes will be written as soon as possible. This will also force the response to be written in chunked encoding.

Troubleshooting

If you run into unexpected results with the plugin, you are able to examine the environment in which your application runs. To enter inspection mode, add the subdirective inspect to your WCGI configuration block. This is a development option that should not be used in production. When in inspection mode, the plugin will respond to matching requests with a page that displays variables of interest. In particular, it will show the replacement value of {match} and the environment variables to which your CGI application has access.

This information can be used to diagnose problems with how a CGI application is called.

To return to operation mode, remove or comment out the inspect subdirective.

Interpreters Example

This small example demonstrates how to configure various scripting interpreters, precompiled WASI binaries for which can be obtained from the WebAssembly Language Runtimes project.

wcgi /php-page /home/val/Downloads/php-cgi-8.2.6-slim.wasm {
  env SCRIPT_FILENAME=page.php
  dir /tmp/scripts
  compile_cache /tmp/owo
}

wcgi /ruby-page /home/val/Downloads/ruby-3.2.2-slim.wasm page.rb {
  dir /tmp/scripts
  compile_cache /tmp/owo
}

wcgi /python-page /home/val/Downloads/python-3.11.4.wasm page.py {
  dir /tmp/scripts
  compile_cache /tmp/owo
}

Note the different ways to pass the script name (PHP uses SCRIPT_FILENAME). Now, with the following scripts:

<?php
print('<h1>Python version:'.$_GET['py']."</h1>\n");
print('<h1>Ruby version:'.$_GET['rb']."</h1>\n");
print('<h1>PHP version:'.phpversion()."</h1>\n");
?>

puts "Location: /php-page?rb=#{RUBY_VERSION}&#{ENV['QUERY_STRING']}"
puts ''

import sys
from urllib.parse import quote
print("Location: /ruby-page?py={}\n".format(quote(sys.version)))

You can run through them something like this:

$ time curl -L localhost:8080/python-page
<h1>Python version:3.11.4 (tags/v3.11.4:d2340ef, Jul 14 2023, 11:21:53) [Clang 16.0.0 ]</h1>
<h1>Ruby version:3.2.2</h1>
<h1>PHP version:8.2.6</h1>

        1.12 real         0.00 user         0.00 sys

Interpreters aren’t super-fast to run currently with wazero’s optimizing compiler not yet having been finished and enabled. Expect much better performance for compiled programs.

Go Source Example

This small example demonstrates how to write a CGI program in Go and compile it to the WASI target. The use of a bytes.Buffer makes it easy to report the content length in the CGI header.

package main

import (
    "bytes"
    "fmt"
    "os"
    "time"
)

func main() {
    var buf bytes.Buffer

    fmt.Fprintf(&buf, "Server time at %s is %s\n",
        os.Getenv("SERVER_NAME"), time.Now().Format(time.RFC1123))
    fmt.Println("Content-type: text/plain")
    fmt.Printf("Content-Length: %d\n\n", buf.Len())
    buf.WriteTo(os.Stdout)
}

To compile to the WASI target, specify it in environment variables:

GOOS=wasip1 GOARCH=wasm go build -o servertime.wasm servertime.go

Now, the following directive in your Caddy file will make it available:

wcgi /servertime /path/to/servertime.wasm

Documentation

Overview

Package wcgi implements “WCGI” (CGI for WASI binaries) for Caddy 2, a modern, full-featured, easy-to-use web server, by embedding the wazero WebAssembly runtime.

It has been forked from the caddy-cgi plugin.

Documentation

Requirements

- This module needs to be installed (obviously). 

Refer to the Caddy documentation on how to build Caddy with
plugins/modules.

- The directive needs to be registered in the Caddyfile: 

    {
        order wcgi before respond
    }

Adjust the order as needed. Putting WCGI before other
response-writing handlers should be a sane default, since the WCGI
module is typically used with a specific matcher and will then take
over completely. Manipulating the request after the application
responds is unlikely to be necessary.

Basic Syntax

The basic wcgi directive lets you add a handler in the current caddy router location with a given script and optional arguments. The matcher is a default caddy matcher that is used to restrict the scope of this directive. The directive can be repeated any reasonable number of times. Here is the basic syntax: 

wcgi [matcher] exec [args...]

For example: 

wcgi /report /app/handlers/report.wasm

When a request such as https://example.com/report or https://example.com/report/weekly arrives, the wcgi middleware will detect the match and run /app/handlers/report.wasm. The root directory exposed to the runtime will by default be the same as the working directory of Caddy itself, but you’ll probably want to change that using the advanced syntax below.

The environment variables PATH_INFO and QUERY_STRING are populated and passed to the script automatically. There are a number of other standard CGI variables included that are described below. If you need to pass any special environment variables or allow any environment variables that are part of Caddy's process to pass to your script, you will need to use the advanced directive syntax described below.

Beware that in Caddy v2 it is (currently) not possible to separate the path left of the matcher from the full URL. Therefore if you require your WCGI program to know the SCRIPT_NAME, make sure to pass that explicitly: 

wcgi /script.cgi* /path/to/app.wasm someargument {
  script_name /script.cgi
}

Advanced Syntax

In order to specify custom environment variables, pass along one or more environment variables known to Caddy, or specify more than one match pattern for a given rule, you will need to use the advanced directive syntax. That looks like this: 

wcgi [matcher] exec [args...] {
    compile_cache cache_path
    script_name subpath
    dir working_directory
    env key1=val1 [key2=val2...]
    pass_env key1 [key2...]
    pass_all_env
    buffer_limit <size>
    unbuffered_output
    inspect
}

For example, 

wcgi /sample/report* /app/report.wasm {
    compile_cache /var/tmp/caddy
    script_name /sample/report
    dir /app/data/report
    env DB=/db/main.db SECRET=/secret CGI_LOCAL=
    pass_env HOME UID
}

The compile_cache subdirective enables ahead-of-time compilation with caching (using the provided cache location), speeding up the performance of the application at the cost of some storage used for caching the compiled native code.

The script_name subdirective helps the cgi module to separate the path to the script from the (virtual) path afterwards (which shall be passed to the script).

env can be used to define a list of key=value environment variable pairs that shall be passed to the script. pass_env can be used to define a list of environment variables of the Caddy process that shall be passed to the script.

If your application runs properly at the command line but fails to run from Caddy it is possible that certain environment variables may be missing.

The pass_all_env subdirective instructs Caddy to pass each environment variable it knows about to the WCGI excutable. This is rather silly with how sandbox-focused the whole WASI thing is, but it’s a leftover from caddy-cgi which didn’t really have a huge reason to get deleted.

buffer_limit is used when a http request has Transfer-Endcoding: chunked. The Go CGI Handler refused to handle these kinds of requests, see https://github.com/golang/go/issues/5613. In order to work around this the chunked request is buffered by caddy and sent to the WCGI application as a whole with the correct CONTENT_LENGTH set. The buffer_limit setting marks a threshold between buffering in memory and using a temporary file. Every request body smaller than the buffer_limit is buffered in-memory. It accepts all formats supported by go-humanize. Default: 4MiB. (An example of this is git push if the objects to push are larger than the http.postBuffer)

With the unbuffered_output subdirective it is possible to instruct the handler to flush output from the application as soon as possible. By default, the output is buffered into chunks before it is being written to optimize the network usage and allow to determine the Content-Length. When unbuffered, bytes will be written as soon as possible. This will also force the response to be written in chunked encoding.

Troubleshooting

If you run into unexpected results with the plugin, you are able to examine the environment in which your application runs. To enter inspection mode, add the subdirective inspect to your WCGI configuration block. This is a development option that should not be used in production. When in inspection mode, the plugin will respond to matching requests with a page that displays variables of interest. In particular, it will show the replacement value of {match} and the environment variables to which your CGI application has access.

This information can be used to diagnose problems with how a CGI application is called.

To return to operation mode, remove or comment out the inspect subdirective.

Interpreters Example

This small example demonstrates how to configure various scripting interpreters, precompiled WASI binaries for which can be obtained from the WebAssembly Language Runtimes project. 

wcgi /php-page /home/val/Downloads/php-cgi-8.2.6-slim.wasm {
  env SCRIPT_FILENAME=page.php
  dir /tmp/scripts
  compile_cache /tmp/owo
}

wcgi /ruby-page /home/val/Downloads/ruby-3.2.2-slim.wasm page.rb {
  dir /tmp/scripts
  compile_cache /tmp/owo
}

wcgi /python-page /home/val/Downloads/python-3.11.4.wasm page.py {
  dir /tmp/scripts
  compile_cache /tmp/owo
}

Note the different ways to pass the script name (PHP uses SCRIPT_FILENAME). Now, with the following scripts: 

<?php
print('<h1>Python version:'.$_GET['py']."</h1>\n");
print('<h1>Ruby version:'.$_GET['rb']."</h1>\n");
print('<h1>PHP version:'.phpversion()."</h1>\n");
?>

puts "Location: /php-page?rb=#{RUBY_VERSION}&#{ENV['QUERY_STRING']}"
puts ''

import sys
from urllib.parse import quote
print("Location: /ruby-page?py={}\n".format(quote(sys.version)))

You can run through them something like this: 

$ time curl -L localhost:8080/python-page
<h1>Python version:3.11.4 (tags/v3.11.4:d2340ef, Jul 14 2023, 11:21:53) [Clang 16.0.0 ]</h1>
<h1>Ruby version:3.2.2</h1>
<h1>PHP version:8.2.6</h1>

        1.12 real         0.00 user         0.00 sys

Interpreters aren’t super-fast to run currently with wazero’s optimizing compiler not yet having been finished and enabled. Expect much better performance for compiled programs.

Go Source Example

This small example demonstrates how to write a CGI program in Go and compile it to the WASI target. The use of a bytes.Buffer makes it easy to report the content length in the CGI header. 

package main

import (
    "bytes"
    "fmt"
    "os"
    "time"
)

func main() {
    var buf bytes.Buffer

    fmt.Fprintf(&buf, "Server time at %s is %s\n",
        os.Getenv("SERVER_NAME"), time.Now().Format(time.RFC1123))
    fmt.Println("Content-type: text/plain")
    fmt.Printf("Content-Length: %d\n\n", buf.Len())
    buf.WriteTo(os.Stdout)
}

To compile to the WASI target, specify it in environment variables: 

GOOS=wasip1 GOARCH=wasm go build -o servertime.wasm servertime.go

Now, the following directive in your Caddy file will make it available: 

wcgi /servertime /path/to/servertime.wasm

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type WCGI 

type WCGI struct {
	// Name of executable script or binary
	Executable string `json:"executable"`
	// Working directory (default, current Caddy working directory)
	WorkingDirectory string `json:"dir,omitempty"`
	// The script path of the uri.
	ScriptName string `json:"script_name,omitempty"`
	// Arguments to submit to executable
	Args []string `json:"args,omitempty"`
	// Environment key value pairs (key=value) for this particular app
	Envs []string `json:"env,omitempty"`
	// Environment keys to pass through for all apps
	PassEnvs []string `json:"pass_env,omitempty"`
	// True to pass all environment variables to CGI executable
	PassAll bool `json:"pass_all_env,omitempty"`
	// True to return inspection page rather than call CGI executable
	Inspect bool `json:"inspect,omitempty"`
	// Size of the in memory buffer to buffer chunked transfers
	// if this size is exceeded a temporary file is used
	BufferLimit int64 `json:"buffer_limit,omitempty"`
	// If set, output from the CGI script is immediately flushed whenever
	// some bytes have been read.
	UnbufferedOutput bool `json:"unbuffered_output,omitempty"`
	// Compile cache directory (can be shared, default none)
	CompileCache string `json:"compile_cache,omitempty"`
	// contains filtered or unexported fields
}

WCGI implements a CGI handler that executes WASI binaries following the CGI protocol, passing parameters via environment variables and evaluating the response as the HTTP response.

func (WCGI) CaddyModule 

func (c WCGI) CaddyModule() caddy.ModuleInfo

CaddyModule returns the module info

func (*WCGI) Provision 

func (c *WCGI) Provision(ctx caddy.Context) error

Provision initializes the module

func (*WCGI) ServeHTTP 

func (c *WCGI) ServeHTTP(w http.ResponseWriter, r *http.Request, next caddyhttp.Handler) error

func (*WCGI) UnmarshalCaddyfile 

func (c *WCGI) UnmarshalCaddyfile(d *caddyfile.Dispenser) error

UnmarshalCaddyfile implements caddyfile.Unmarshaler.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.