(IN DEVELOPMENT)
Backupper
The tool is for backuping file from FTP/SFTP source and save it another FTP/SFTP backup destination.
Features
- Schedule backuping with CRON expression in config file
- Download files from FTP/SFTP sources
- Executing commands on source server (SFTP only)
- Upload files to FTP/SFTP destinations
- Upload files to Telegram with Bot API (max 50 MB files)
- Limit the file count in target folder (if limit is reached, oldest backups will be deleted)
- Limit the total file size in target folder (if limit is reached, oldest backups will be deleted)
- Limit the target folder by duration (oldest backups than date will be deleted)
Config
Main config file is config.json
in the root folder of the project.
Also you can use config.example.json
as a template.
Main Structure
{
"timezone": "Europe/Istanbul",
"dateFormat": "2006-01-02__15-04-05",
"backups": []
}
Key |
Description |
Type |
timezone |
TZ identifier |
string |
dateFormat |
the format of the date to be added to the file name (Golang time format) |
string |
logLevel |
Log level (debug, info, warn, error, dpanic, panic, fatal) |
string |
backups |
Backup schedules |
array |
Backup Schedule Structure
{
"name": "backup-title",
"cronExpr": "*/60 * * * * *",
"callbackUrl": "http://example.com/callback",
"deleteLocal": true,
"source": {
"type": "sftp",
"info": {
"host": "127.0.0.1",
"port": 22,
"user": "root",
"pass": "",
"privateKeyFile": "~/.ssh/custom_id_rsa",
"privateKeyPass": "",
"variables": {
"foo": "bar",
"bar": 123,
"baz": true
},
"beforeCommands": [
"echo $foo > /tmp/backupper/$BACKUP_ID/foo.txt",
"echo $bar > /tmp/backupper/$BACKUP_ID/bar.txt",
"echo $baz > /tmp/backupper/$BACKUP_ID/baz.txt",
"cd /opt/foo/bar",
"zip -r /tmp/backupper/$BACKUP_ID/foo_backup.zip ."
],
"downloads": [
"foo_backup.zip",
"foo.txt",
"bar.txt",
"baz.txt"
],
"afterCommands": [
"rm -rf /tmp/backupper/$BACKUP_ID/"
]
}
},
"destination": {
"type": "ftp",
"deleteAfterUpload": true,
"info": {
"host": "backup-ftp.example.com",
"port": 21,
"user": "root",
"pass": "p4ssw0rd",
"target": "/up/backups/foo/",
"limitByCount": 3,
"LimitBySize": 1073741824,
"limitByDate": "2 DAYS"
}
}
}
Key |
Description |
Type |
name |
Name of the backup schedule |
string |
cronExpr |
CRON expression (also supports cronSeconds) |
string |
callbackUrl |
Callback URL to be called after backup process is completed |
string |
deleteLocal |
Delete local files after upload process is completed |
bool |
source |
Source server information |
object |
destination |
Destination server information |
object |
Source
Key |
Description |
Type |
type |
Source server type (ftp/sftp) |
string |
info |
Source server information |
object |
Source Info (FTP)
Key |
Description |
Type |
host |
FTP server host |
string |
port |
FTP server port |
int |
user |
FTP server username |
string |
pass |
FTP server password |
string |
downloads |
Files to be downloaded from FTP server (only files, not directories) |
array |
Source Info (SFTP)
Key |
Description |
Type |
host |
SFPT server host |
string |
port |
SFTP server port |
int |
user |
SFTP server username |
string |
pass |
SFTP server password |
string |
privateKeyFile |
Private key file path |
string |
passphrase |
Private key passphrase |
string |
variables |
Custom variables to be used in SSH commands |
object |
beforeCommands |
SSH Commands to be executed before download process |
array |
downloads |
Files to be downloaded from SFTP server (only files, not directories) |
array |
afterCommands |
SSH Commands to be executed after download process |
array |
SFTP - SSH Command Variables
Variable |
Description |
Type |
$BACKUP_ID |
Unique ID of the backup process (generated by the tool) (Nano unix timestamp) |
string |
$BACKUP_NAME |
Name of the backup schedule |
string |
Destination
Key |
Description |
Type |
type |
Destination server type (ftp/sftp/telegram_bot) |
string |
deleteAfterUpload |
Delete files after upload process is completed |
bool |
info |
Destination server information |
object |
Destination Info (Telegram with Bot API) (max 50 MB files)
Key |
Description |
Type |
token |
Telegram bot token from @BotFather |
string |
chatID |
Telegram chat ID (channel/group) or public username |
string |
Destination Info (FTP)
Key |
Description |
Type |
host |
FTP server host |
string |
port |
FTP server port |
int |
user |
FTP server username |
string |
pass |
FTP server password |
string |
target |
Target folder on FTP server |
string |
limitByCount |
Limit the file count in target folder |
int |
limitBySize |
Limit the total file size in target folder (bytes) |
int |
limitByDate |
Limit the target folder by duration (duration format) |
string |
Destination Info (SFTP)
Key |
Description |
Type |
host |
SFTP server host |
string |
port |
SFTP server port |
int |
user |
SFTP server username |
string |
pass |
SFTP server password |
string |
privateKeyFile |
Private key file path |
string |
passphrase |
Private key passphrase |
string |
target |
Target folder on SFTP server |
string |
limitByCount |
Limit the file count in target folder |
int |
limitBySize |
Limit the total file size in target folder (bytes) |
int |
limitByDate |
Limit the target folder by duration (duration format) |
string |
Format |
Date Range |
5 MINUTES |
All files before 5 minutes |
1 HOUR |
All files before 1 hour |
2 DAYS |
All files before 2 days |
3 WEEKS |
All files before 3 weeks |
4 MONTHS |
All files before 4 months |
5 YEARS |
All files before 5 years |
Multiple durations can be used together. (e.g. 1 HOUR 30 MINUTES)
Callback Post Data
{
"backup_date": "2023-07-20T15:27:50+03:00",
"backup_destination": "sftp",
"backup_destination_result": {
"totalUploadedFiles": 1,
"totalUploadedSize": 5820073
},
"backup_duration": "19.7989235s",
"backup_id": "1689856070674044500",
"backup_name": "test-backup",
"backup_source": "sftp",
"backup_ts": 1689856070
}
Key |
Description |
Type |
backup_date |
Backup date (RFC3339) |
string |
backup_destination |
Destination server type (ftp/sftp) |
string |
backup_destination_result |
Destination server upload result |
object |
backup_duration |
Backup duration (time.Duration string) |
string |
backup_id |
Unique ID of the backup process (generated by the tool) (Nano unix timestamp) |
int |
backup_name |
Name of the backup schedule |
string |
backup_source |
Source server type (ftp/sftp) |
string |
backup_ts |
Backup timestamp (Unix seconds) |
int |
Used Modules