large-json-splitter

README

Large JSON Splitter

Large JSON Splitter is a Go library that reads JSON data as a stream and extracts objects, saving them as separate JSON files.

The generated JSON files will be placed in subdirectories under the original input file's location.

Input

// Original sample.json
{
  "stringValue": "value",
  "intValue": 1,
  "boolValue": true,
  "arrayValue": [
    "value1",
    "value2"
  ],
  
  "objectValue": {
    "innerStringValue": "value",
    "innerIntValue": 1,
    "innerBoolValue": true,
    "innerArrayValue": [
      "value1",
      "value2"
    ]
  }
  
}

Output

├── sample
│   ├── objectValue
│   │   └── objectValue.json
│   └── sample.json
└── sample.json

// sample.json with objectValue key removed
{
  "arrayValue": [
    "value1",
    "value2"
  ],
  "boolValue": true,
  "intValue": 1,
  "stringValue": "value"
}

// objectValue.json
{
  "innerArrayValue": [
    "value1",
    "value2"
  ],
  "innerBoolValue": true,
  "innerIntValue": 1,
  "innerStringValue": "value"
}

Installation

You can install this library using go get:

go get github.com/KENKEN0803/large-json-splitter

Usage

The large-json-splitter library provides both a command-line executable and a Go function for splitting JSON files.

Executable Usage

Executables can be found on release page. You can use the executable as follows:

./large-json-splitter --i ./data/141.json --d "  "

• --input, --i: (Required) Specifies the input JSON file to process.
• --indent, --d: (Optional) Specifies the indent string to use when writing the output JSON files. If not specified, the output JSON files will be minified.

Go Library Usage

package main

import (
    "github.com/KENKEN0803/large-json-splitter"
    "fmt"
)

func main() {
    inputPath := "sample.json"
    
    if err := largeJsonSplitter.SplitJson(inputPath, ""); err != nil {
        fmt.Printf("Error: %v\n", err)
    }
}

GitHub Repository

Find the source code and additional documentation on the GitHub Repository

License

This project is licensed under the MIT License. See the LICENSE file for details.