mongosync
mongosync is a lightweight and extremely fast utility that copies differential data between two mongodb instance.
It does not delete any data if it is deleted in the source database.
Feature Matrix
Feature |
State |
Comment |
Insane multithreaded performance |
✔ |
Truly insane |
DB Creation |
✔ |
|
Collection Creation |
✔ |
|
Differential Document Creation |
✔ |
|
Single DB Scope |
✔ |
|
Single Collection Scope |
✔ |
|
Batch Uploads |
✔ |
Every Insert or Replace requests is batched |
Custom IDs (including objects) |
✔ |
It supports pretty much anything possible |
Multiple workers per collections |
✔ |
Enabled by default, but workers are set to 1, increase for gains |
Configuration in file |
✔ |
Obligatory, extremely finetunable |
No-check if exist |
✔ |
Config argument: no_find |
Date-based filtering |
✔ |
Of course ! |
Lightweigth on RAM |
✔ |
It's not a memory hog |
Lightweigth on CPU |
❌ |
Hopefully you have a lot of cores :) |
Installation
mongosync requires go 1.19, you can download go here: Downloads - The Go Programming Language
Then, do this:
#> go install github.com/sherweb/mongosync@v1.10.0
Test with this:
#> mongosync
mongosync is an utility to sync two different mongodb instances
Usage:
mongosync [flags]
mongosync [command]
Available Commands:
completion Generate the autocompletion script for the specified shell
copy copy data from one mongodb instance to another
generate-config Generates a config with a given source and destination
help Help about any command
Flags:
-h, --help help for mongosync
Usage
First, you must run the following
$> mongosync generate-config --source mongodb://user:pass@url:port --destination mongodb://user:pass@url:port
This will output a config.yml, you can see a detailed sample with explanations on what all switches do here: sample-config.yml
YOU MUST EDIT THE CONFIG FILE !
To copy the configuration you just modified, run this:
$> mongosync copy --config config.yml
Caveats
- Does not support index copy
- Does not support any kind of views! Remove them from the config!!
- Error handling isn't amazing. Yet !
Contributing
To contribute, feel free to open PRs and/or issues