mongoimport

README

Mongoimport

CLI and go library for importing data from CSV, JSON or XML files into MongoDB.

go run github.com/grittaweisheit/mongoimport/cmd/mongoimport csv <path-to-csv>

You can also download pre built binaries from the releases page.

For a list of options, run

go run github.com/grittaweisheit/mongoimport/cmd/mongoimport csv --help

Usage as a library

Using the tool as a standalone CLI tool is great for quick loading of a few files. However, you might need more fine-grained control over what files are imported into which collection or perform additional pre/post processing (e.g. parsing timestamps). For this use case, we offer a very extensivle and modular API for configuring your imports.

import "github.com/grittaweisheit/mongoimport"

// example t.b.a

For more examples, see examples/.

Development

All commits are automatically built and tested on travis-ci.com. In order to pass the required checks it is strongly recommended to install the repositories pre commit hooks (assuming you are in the repositories root):

pip install pre-commit invoke bump2version
pre-commit install

The pre commit hooks will run a number of go tools. Try to run pre-commit run --all-files and check for missing tools. You might need to install some of those:

go get -u golang.org/x/lint/golint
go get -u github.com/fzipp/gocyclo

In order to be found, make sure to include $GOPATH/bin in your $PATH ($GOPATH has to be set of course like export GOPATH="$HOME/go for example).

Before committing, pre-commit is run to make sure all checks pass!

Documentation

Index

• Constants
• type Datasource
• type Import
  • func (i Import) Start() (ImportResult, error)
• type ImportResult
• type MongoConnection
  • func (c *MongoConnection) Client() (*mongo.Client, error)
• type PostLoadHook
• type PreDumpHook
• type UpdateFilterHook

Constants

const Version = "0.1.8"

Version is incremented using bump2version

Types

type Datasource

type Datasource struct {
	Files           []string
	DatabaseName    string
	Collection      string
	Loader          loaders.Loader
	PostLoad        PostLoadHook
	PreDump         PreDumpHook
	UpdateFilter    UpdateFilterHook
	EmptyCollection bool
	Sanitize        bool
}

Datasource ...

type Import

type Import struct {
	Connection   *MongoConnection
	Data         []*Datasource
	IgnoreErrors bool
}

Import ...

func (Import) Start

func (i Import) Start() (ImportResult, error)

Start ...

type ImportResult

type ImportResult struct {
	File       string
	Collection string
	Succeeded  int
	Failed     int
	Elapsed    time.Duration
	// contains filtered or unexported fields
}

ImportResult ...

type MongoConnection

type MongoConnection struct {
	DatabaseName string
	User         string
	Password     string
	Host         string
	Port         int
}

MongoConnection ...

func (*MongoConnection) Client

func (c *MongoConnection) Client() (*mongo.Client, error)

Client ...

type PostLoadHook

type PostLoadHook func(loaded map[string]interface{}) (interface{}, error)

PostLoadHook ...

type PreDumpHook

type PreDumpHook func(loaded interface{}) (interface{}, error)

PreDumpHook ...

type UpdateFilterHook

type UpdateFilterHook func(loaded interface{}) (interface{}, error)

UpdateFilterHook ...