README
¶
objdeliv
This is a universal temporary large object delivery service.
When some internal artifacts generated by various micro-services need to be sent to the user, you could use objdeliv as a delivery service.
It provides consistent and non-consistent storage drivers, high performance, extremely low memory footprint and very simple and easy-to-implement protocol compared to storing the stuff to Redis or just pipelining them between services.
Installation
First install golang building environment.
Then run:
go -o objdeliv build cmd/main.go
Move objdeliv to $PATH and you can use it now.
If you have just installed, you can use just to build and run, just build to build, just run to run and just build-cross to build for different targets.
Example
Run the following code in your shell and you get a objdeliv server up, running on port 24032, with a LocalDriver which stores all the objects in /tmp/objdeliv as it storage and have a default object expire time of 2 hours (7200 seconds).
go run cmd/main.go -l :24032 -d LocalDriver -o '{"path": "/tmp/objdeliv/"}' -e 7200
Now put an object to it. Example code:
<?php
$sock = fsockopen("127.0.0.1:24032");
$write = fn ($data) => @fwrite($sock, $data);
$write("CONNECT /new-object?expire=60&id=e45d7d04-52b6-473d-bc78-aa40f4b07e73 HTTP/1.1\r\n\r\n");
$write(<<<EOF
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Document</title>
</head>
<body>
<h1>My Website</h1>
<p>This is a test.</p>
</body>
</html>
EOF);
var_dump(json_decode(fgets($sock), true));
fclose($sock);
You will get a result of something like:
array(2) {
["status"]=>
string(7) "success"
["id"]=>
string(36) "e45d7d04-52b6-473d-bc78-aa40f4b07e73"
}
And now navigate to http://localhost:24032/get-object?id=e45d7d04-52b6-473d-bc78-aa40f4b07e73&auto-release=true&content-type=text/html. You will see the object you uploaded!
Now just refresh the page. You set auto-release to true, so the object is now release and cannot be accessed again!
Configure
Three ways can be used to configure objdeliv.
Generally, the priority is: command arguments > environment vars > configure file.
1. Using Environment Variables
| Name | Description | Example |
|---|---|---|
OBJDELIV_LISTEN |
Listen address | [::]:8080 |
OBJDELIV_DRIVER |
Storage driver name | LocalDriver or MemoryDriver |
OBJDELIV_DRIVER_OPTIONS |
Storage driver options, in JSON | {"path": "/tmp/objdeliv/"} |
OBJDELIV_DEFAULT_EXPIRE |
Default expire time, in seconds | 7200 |
2. Using Command Arguments
GLOBAL OPTIONS:
--config value, -c value, --conf value Specify the config file path (default: "./configure.yml") [$OBJDELIV_CONFIG]
--listen value, -l value, --addr value Listen address [$OBJDELIV_LISTEN]
--driver value, -d value Specify the storage driver [$OBJDELIV_DRIVER]
--driver-options value, -o value Storage driver options, in JSON [$OBJDELIV_DRIVER_OPTIONS]
--default-expire value, --expire value, -e value Default object expire time, -1 for unlimited. [$OBJDELIV_DEFAULT_EXPIRE]
--help, -h show help (default: false)
3. Using Configure File
Use YAML.
An example:
listen: :24032
driver: MemoryDriver
driver-options:
path: ./dev-temp
default-expire: 7200
4. Storage Driver Options
4.1 MemoryDriver
No options needed.
4.2 LocalDriver
Only one, path, stands for the storage path. If not exist, the program will automatically create it.
Protocol
1. Create an object
This is not standard HTTP!
First client establishes a TCP connection and send a package "CONNECT /new-object HTTP/1.1\r\n\r\n" to the server.
Note: like HTTP, you can send query params in the URL. You can send: id and expire.
If you send id, you need to promise it is a valid UUID string. We will use that instead of generating a new one.
Then the server responses in JSON, with two fields, status and id. status should be success. The id is a UUID string, used to identify the object.
Then you just write your stuff into the socket connection and it works. When finished, just feel free to close the connection.
A sample SDK in PHP:
<?php
class ObjDelivClient
{
protected $conn;
protected string $id = '';
public function __construct(string $addr, ?string $id = null, ?int $expire = null)
{
$this->conn = fsockopen($addr);
$endpoint = '/new-object?';
if ($id !== null) {
$endpoint .= 'id=' . urlencode($id);
if ($expire !== null) $endpoint .= '&';
}
if ($expire !== null) {
$endpoint .= 'expire=' . $expire;
}
$pkt = "CONNECT $endpoint HTTP/1.1\r\n\r\n";
fwrite($this->conn, $pkt);
$this->id = json_decode(fgets($this->conn), 1)['id'];
}
public function getID(): string
{
return $this->id;
}
public function write(string $bytes): int
{
return fwrite($this->conn, $bytes);
}
public function __destruct()
{
fclose($this->conn);
}
}
2. Get an Object
Just GET /get-object?id=[THE_ID]&auto-release=[true/false]&content-type=[CONTENT_TYPE]&content-disposition=[CONTENT_DISPOSITION]&pragma=[PRAGMA]&cache-control=[CACHE_CONTROL]&expires=[EXPIRES]
If auto-release is true, after the request the object will be released.
The other parameters are all HTTP response headers that could be customized.
3. Release an Object
Just GET /release-object?id=[THE_ID].
Response is in JSON format.
4. Set Expire Time for an Object
Just GET /set-expire?id=[THE_ID]&expire=[EXPIRE_TIME_IN_SECONDS].
Response is in JSON format.
Directories
¶
| Path | Synopsis |
|---|---|
|
Command objdeliv is the objdeliv server
|
Command objdeliv is the objdeliv server |
|
Package server implements the objdeliv server
|
Package server implements the objdeliv server |
|
Package storage provides utilities and facades to interact with storage engines
|
Package storage provides utilities and facades to interact with storage engines |
|
drivers
Package drivers includes several storage drivers for objdeliv
|
Package drivers includes several storage drivers for objdeliv |
|
drivers/local
Package local implements a local storage driver for objdeliv
|
Package local implements a local storage driver for objdeliv |
|
drivers/memory
Package memory implements a memory storage driver for objdeliv
|
Package memory implements a memory storage driver for objdeliv |