mongo-solr

README

Sciveyor Mongo-to-Solr Bridge

Building

1. Check out the code, including the submodules: git clone --recurse-submodules ...
2. Install statik: go get github.com/rakyll/statik
3. Generate static schema blob: go generate
4. Build: go build
5. Run: ./mongo-solr ...

Using

What you need:

1. A MongoDB server, with a collection of documents that follow the schema spelled out here. FIXME: At some point in the future, this server will become more complex, with support for other collections carrying information about disambiguated authors, journals, and institutions. That support is not currently available in this tool.
2. A Solr server, pre-loaded with the schema described here. (FIXME: Not currently available for public consumption. Watch this space; it needs more debugging.)

The script performs a three-step synchronization, extremely simply.

1. For each document in the MongoDB database:
   1. If it is present in the Solr database, but either its version or its dataSourceVersion parameters have changed, delete and re-create it in the Solr database.
   2. If it is not present in the Solr database, create it.
2. For each document in the Solr database:
   1. If it is not present in the Mongo database, delete it.

Notably, this is not a proper atomic synchronization. Documents are deleted and re-created, not partially updated (in Solr's terminology, we do not use "atomic updates"). We also do not detect any changes other than in the two version parameters. Version numbers must be bumped to trigger a sync. (This is an intentional policy choice.)

To use it, then, you need to specify the following parameters:

./mongo-solr \
  -mongo-address mongodb://localhost \
  -mongo-database YourDatabase \
  -mongo-collection documents \
  -solr-address http://localhost:8983/solr \
  -solr-collection sciveyor

All of those options should be fairly self-explanatory.

Options

• -verbose: By default, basic information about the sync will be printed to the console. To see much more information (including printed dumps of the IDs present in both the Mongo and Solr databases), pass the -verbose flag.

Changelog

• v0.2: Store all the date values in documents as ISODate in MongoDB.
• v0.1: Initial support for only the parameters mentioned in the JSON document schema.

License

The code here is copyright © 2021 Charles H. Pence, and released under the GNU GPL v3.