go-oom-guard
It creates eventfd and register a notifier by given threshold in CGroup.
Then it sends SIGINT or SIGTERM to top memory used process.
Problem
Postgres is unable to control memory usage by backends.
CGroup OOM Killer triggered by some query causes whole database restart.
How to mitigate it?
Possible solutions
- Just monitor memory usage, make it less impact, recommendations by CrunchyData
- LD_PRELOAD custom library, solution by TimescaleDB Cloud
- Register cgroup memory event before OOM triggers. A lot of solutions, for example Tencent Cloud
This repo is example of #3
Prerequisites
- Docker
- python3, psycopg2
- psql
- Go
Demo
terminal #1:
$ make build
$ make docker
$ make docker-run
terminal #2:
$ docker exec -i postgres_oom_guarded ./go-oom-guard
terminal #3:
make test-oom-execute
make test-oom-parse
and the try it without running go-oom-guard
, stop terminal #2, and run commands in terminal #3 again.
Cautions
It's only POC, don't use it on production AS IS!
TODOs
Links