springytools

README

LibGuides Tools

A Golang package for working with LibGuides exported XML.

Table of contents

• Introduction
• Installation
• Known issues and limitations
• Getting help
• License
• Authors and history
• Acknowledgments

Introduction

There is a periodic need to work with exported LibGuide XML in Caltech Library. This is a golang package for working with the explorted data. Go provides a robust may of mapping simple data structures to and from XML (or JSON). This makes working with XML very easy in a consistant fashion. It seem time to move beyond my usual Bash/sed/python scripts.

One program is currrently provided with springytools, lgxml2sjon which converts a LibGuides XML export file into JSON.

Installation

This is a Golang package. It requires Go 1.16.x or better and can be installed by cloning the source repository on GitHub. jq is a JSON Query tool written by Stephen Dolan and is available form https://stedolan.github.io/jq/ (it is used to view and validate the JSON output before installing the tool lgxml2json)

1. clone the repository
2. change into the clone directory
3. test
4. build the command line tool lgxml2json
5. use lgxml2json and test output with jq

• Replace "LibGuides_export_XXXX.xml" with the file path to your exported LibGuides XML file

1. install lgxml2json

git clone git@github.com:caltechlibrary/springytools
cd springytools
go test
go build cmd/lgxml2json.go
./lgxml2sjon LibGuides_export_XXXX.xml export.json
jq . export.json
go install cmd/lgxml2json.go

Known issues and limitations

This library is currently written to perform the LibGuide link analysis. It only provides the commands I needed to do the data anaysis. It will grow as needed.

The exported XML output from the LibGuides may not be valid UTF-8. UTF-8 encoding is required to successfully parse the export file. Looking at the raw XML markup in vim I noticed a number of control code sequences. This corresponded to the errors on parsing the unsanitized XML file. The problem characters appear as ^A, ^K, ^L, ^S, ^C, ^R. These maybe non-UTF-8 characters embedded as UTF-8 when the rich text documents were pasted in via the LibGuides edit UI. My hunch is these were pasted in/imported from Word documents. Remove the offending characters allowed the export to parse successfully. These edits are distructive as some of the codes probably prepresent UTF-8 characters used in non-English European names or termonology.

Getting help

File an issue on GitHub.

License

Software produced by the Caltech Library is Copyright © 2021 California Institute of Technology. This software is freely distributed under a BSD/MIT type license. Please see the LICENSE file for more information.

Authors and history

• R. S. Doiel, Software Developer, Digital Library Development, Caltech Library

Acknowledgments

This work was funded by the California Institute of Technology Library.

(If this work was also supported by other organizations, acknowledge them here. In addition, if your work relies on software libraries, or was inspired by looking at other work, it is appropriate to acknowledge this intellectual debt too.)

Documentation

Overview

libguides.go implements the data structures for working with with LibGuides exported XML.

Index

• type Account
• type Asset
• type Box
• type Customer
• type Group
• type Guide
• type LibGuides
  • func (lg *LibGuides) FromXML(src []byte) error
  • func (lg *LibGuides) ToJSON() ([]byte, error)
• type Owner
• type Page
• type Pane
• type Site
• type Subject
• type Tag
• type Vendor

Types

type Account

type Account struct {
	Id        int    `xml:"id" json:"id"`
	Email     string `xml:"email" json:"email"`
	FirstName string `xml:"first_name" json:"first_name"`
	LastName  string `xml:"last_name" json:"last_name"`
	Title     string `xml:"title" json:"title"`
	Nickname  string `xml:"nickname" json:"nickname"`
	Signature string `xml:"signature" json:"signature"`
	Image     string `xml:"image" json:"image"`
	Address   string `xml:"address" json:"address"`
	Phone     string `xml:"phone" json:"phone"`
	Skype     string `xml:"skype" json:"skype"`
	Website   string `xml:"website" json:"website"`
	Created   string `xml:"created" json:"created"`
	Updated   string `xml:"updated" json:"updated"`
}

type Asset

type Asset struct {
	Id   int    `xml:"id" json:"id"`
	Name string `xml:"name" json:"name"`
	Type string `xml:"type" json:"type"`
	// Description contains HTML encoded text, double encoding existing encoded text
	Description string `xml:"description" json:"descriptoin"`
	Url         string `xml:"url" json:"url"`
	Owner       Owner  `xml:"owner" json:"owner"`
	MapId       string `xml:"map_id" json:"map_id"`
	Position    int    `xml:"position" json:"position"`
	Created     string `xml:"created" json:"created"`
	Updated     string `xml:"updated" json:"updated"`
}

type Box

type Box struct {
	XMLName  xml.Name `xml:"box" json:"box"`
	Id       int      `xml:"id" json:"id"`
	Name     string   `xml:"name" json:"name"`
	Type     string   `xml:"type" json:"type"`
	MapId    string   `xml:"map_id" json:"map_id"`
	Column   int      `xml:"column" json:"column"`
	Position int      `xml:"position" json:"position"`
	Hidden   int      `xml:"hidden" json:"hidden"`
	Created  string   `xml:"created" json:"created"`
	Updated  string   `xml:"updated" json:"updated"`
	Assets   []*Asset `xml:"assets>asset" json:"assets"`
	Panes    []*Pane  `xml:"panes>pane,omitempty" json:"panes,omitempty"`
}

type Customer

type Customer struct {
	XMLName  xml.Name `xml:"customer" json:"-"`
	Id       int      `xml:"id" json:"id"`
	Type     string   `xml:"type" json:"type"`
	Name     string   `xml:"name" json:"name"`
	Url      string   `xml:"url" json:"url"`
	City     string   `xml:"city" json:"city"`
	State    string   `xml:"state" json:"state"`
	Country  string   `xml:"country" json:"country"`
	TimeZone string   `xml:"time_zone" json:"time_zone"`
	Created  string   `xml:"created" json:"created"`
	Updated  string   `xml:"updated" json:"updated"`
}

type Group

type Group struct {
	Id          int    `xml:"id" json:"id"`
	Type        string `xml:"type" json:"type"`
	Name        string `xml:"name" json:"name"`
	Url         string `xml:"url" json:"url"`
	Description string `xml:"description" json:"description"`
	Password    string `xml:"password" json:"password"`
	Created     string `xml:"created" json:"created"`
	Updated     string `xml:"updated" json:"updated"`
}

type Guide

type Guide struct {
	Id          int        `xml:"id" json:"id"`
	Type        string     `xml:"type" json:"type"`
	Name        string     `xml:"name" json:"name"`
	Description string     `xml:"description" json:"descriptoin"`
	Url         string     `xml:"url" json:"url"`
	Owner       Owner      `xml:"owner" json:"owner"`
	Group       Group      `xml:"group" json:"group"`
	Redirect    string     `xml:"redirect" json:"redirect"`
	Status      string     `xml:"status" json:"status"`
	Created     string     `xml:"created" json:"created"`
	Updated     string     `xml:"updated" json:"updated"`
	Modified    string     `xml:"modified" json:"modified"`
	Published   string     `xml:"published" json:"published"`
	Subjects    []*Subject `xml:"subjects>subject" json:"subjects"`
	Tags        []*Tag     `xml:"tags>tag" json:"tags"`
	Pages       []*Page    `xml:"pages>page" json:"pages"`
}

type LibGuides

type LibGuides struct {
	XMLName  xml.Name   `json:"-"`
	Customer *Customer  `xml:"customer" json:"customer"`
	Site     *Site      `xml:"site" json:"site"`
	Accounts []*Account `xml:"accounts>account" json:"accounts"`
	Groups   []*Group   `xml:"groups>group" json:"groups"`
	Subjects []*Subject `xml:"subjects>subject" json:"subjects"`
	Tags     []*Tag     `xml:"tags>tag" json:"tags"`
	Vendors  []*Vendor  `xml:"vendors>vendor" json:"vendors"`
	Guides   []*Guide   `xml:"guides>guide" json:"guides"`
}

func (*LibGuides) FromXML

func (lg *LibGuides) FromXML(src []byte) error

FromXML takes a LibGuides Object, []bytes of XML source populates the LibGuides object and returns any error.

func (*LibGuides) ToJSON

func (lg *LibGuides) ToJSON() ([]byte, error)

ToJSON takes a LibGuides object and renders JSON output and error

type Owner

type Owner struct {
	XMLName   xml.Name `xml:"owner" json:"owner"`
	Id        int      `xml:"id" json:"id"`
	Email     string   `xml:"email" json:"email"`
	FirstName string   `xml:"first_name" json:"first_name"`
	LastName  string   `xml:"last_name" json:"last_name"`
	Image     string   `xml:"image" json:"image"`
}

type Page

type Page struct {
	Id           int    `xml:"id" json:"id"`
	Name         string `xml:"name" json:"name"`
	Description  string `xml:"description" json:"descriptoin"`
	Url          string `xml:"url" json:"url"`
	Redirect     string `xml:"redirect" json:"redirect"`
	SourcePageId int    `xml:"source_page_id" json:"source_page_id"`
	ParentPageId int    `xml:"parent_page_id" json:"parent_page_id"`
	Position     int    `xml:"position" json:"position"`
	Hidden       int    `xml:"hidden" json:"hidden"`
	Created      string `xml:"created" json:"created"`
	Updated      string `xml:"updated" json:"updated"`
	Modified     string `xml:"modified" json:"modified"`
	Boxes        []*Box `xml:"boxes>box" json:"boxes"`
}

type Pane

type Pane struct {
	Assets []*Asset `xml:"assets>asset" json:"assets"`
}

type Site

type Site struct {
	XMLName xml.Name `xml:"site" json:"-"`
	Id      int      `xml:"id" json:"jd"`
	Type    string   `xml:"type" json:"type"`
	Name    string   `xml:"name" json:"name"`
	Domain  string   `xml:"domain" json:"domain"`
	Admin   string   `xml:"admin" json:"admin"`
	Created string   `xml:"created" json:"created"`
	Updated string   `xml:"updated" json:"updated"`
}

type Subject

type Subject struct {
	Id   int    `xml:"id" json:"id"`
	Name string `xml:"name" json:"name"`
	Url  string `xml:"url" json:"url"`
}

type Tag

type Tag struct {
	Id   int    `xml:"id" json:"id"`
	Name string `xml:"name" json:"name"`
}

type Vendor

type Vendor struct {
	Id   int    `xml:"id" json:"id"`
	Name string `xml:"name" json:"name"`
}