sopsearch

README

Sop Search

Description

This program takes the SOP documents from the openshift repository and indexes them into an elasticsearch container as a way of making it easier to find an SOP document. This information is then displayed in a web interface which allows you to easily search for the SOP document you want.

How to Use

After launching the UI, simply type into the searchbox to find results. You can also choose to filter by the tags (aka the possible location of your document), author, and the name of the SOP to help narrow your search. You can further choose to sort your results by the last updated or by the most relevant.

Building the Docker Images

To build the images used in deployment.yml:

The Elasticsearch Image

1. build the image using the dockerfile in the elasticsearch folder
2. push image to your repo

The Sop Search Image

1. build the image using the dockerfile
2. push image to your repo

The UI Image

The UI image is slightly different as you have to make sure to update the build folder each time you change anything in the UI.

1. make sure your build folder has been updated
   • npm run build
   • if above fails, try npm install or npm update and then trying the above command again
2. build the image using dockerfile in the ui folder
3. push image to your repo

Deploying

1. Create the ssh secret yml file

apiVersion: v1
data:
  ssh-privatekey:
  ssh-publickey:
kind: Secret
metadata:
  name: ssh
  namespace: sop-search
type: Opaque

1. Create the configmap file

apiVersion: v1
kind: ConfigMap
metadata:
  name: configmap
  namespace: sop-search
data:
  time: "5" #number of minutes before restarting routine
  elastic: http://localhost:9200 #location of elasticsearch data
  repourl: https://github.com/openshift/ops-sop-search/blob/master/ #repo url used for creating links
  reponame: ops-sop-search #name of the repository you're indexing
  gitscript: script.sh #location of the shell script
  giturl: git@github.com:openshift/ops-sop-search.git #clone with ssh

1. Create the service_account, role, and role_binding (using files from deploy folder)
2. Create the services (one for the UI and one for the elasticsearch using files from deploy folder)
3. Create the routes (one for UI and one for elasticsearch using files from deploy folder)
4. Deploy the deployment file
5. Access the application via the web address given in the UI route

Documentation

Index

• func FindFiles(root, pattern string) ([]string, error)
• func GetKubeClient() (client.Client, error)
• func GitClone(script string, repo string) error
• func GitLog(filepath string) ([]byte, error)
• func GitPull(script string) ([]byte, error)
• func IndexSOP(indexer Indexer, sops map[string]string) error
• func NeedReIndex(config Config) error
• func RunIndex(index bool, config Config) error
• func ScanForFiles(path string, config Config) ([]MDFile, []ADFile, error)
• func ToBulkJSON(s []Sop) (map[string]string, error)
• type ADFile
• type Config
  • func GetConfig(kubecli client.Client) (Config, error)
• type Dates
  • func GetAuthorsAndDates(data []byte) ([]string, Dates, error)
• type ElasticClient
  • func NewElasticClient(addresses []string, username, password string) (ElasticClient, error)
  • func (ec *ElasticClient) CreateOrUpdateIndex(index, documentID, body string) error
• type Indexer
• type MDFile
• type Sop
  • func ToBulkSOP(mf []MDFile, af []ADFile) ([]Sop, error)

Functions

func FindFiles

func FindFiles(root, pattern string) ([]string, error)

FindFiles takes a path to a directory and a pattern (in this case, the type of file) and looks through the entire directory and adds all the paths for the files that match the pattern into a slice of strings which it returns.

func GetKubeClient

func GetKubeClient() (client.Client, error)

GetKubeClient creates a Kubernetes client which can be used to access the ConfigMap stored on the oc environment.

func GitClone

func GitClone(script string, repo string) error

GitClone performs a git clone of repo contained in the repo string. The script string containts the location of the shell script that performs the git clone.

func GitLog

func GitLog(filepath string) ([]byte, error)

GitLog performs a git log of a file specified by the filepath string. It then returns the output of the command.

func GitPull

func GitPull(script string) ([]byte, error)

GitPull performs a git pull of the repo. The script string contains the location of the shell script that performs the git pull. It will then return the output of the git pull to determine if the repo was actually updated or not.

func IndexSOP

func IndexSOP(indexer Indexer, sops map[string]string) error

IndexSOP takes the map holding the SOPs and indexes them or updates the index that already exists.

func NeedReIndex

func NeedReIndex(config Config) error

NeedReIndex performs a git pull to determine if a re-indexing needs to occur. If the repo is already up to date, then it will do a RunIndex with false. If it's not, then it wil return RunIndex with true.

func RunIndex

func RunIndex(index bool, config Config) error

RunIndex performs the indexing. If the index bool is true, then this function will perform the indexing routine. If the bool is false, then it won't index. The Config object is used to get the elasticsearch url. Once it finishes the indexing routine, it will print out how much time it took to index the SOP documents.

func ScanForFiles

func ScanForFiles(path string, config Config) ([]MDFile, []ADFile, error)

ScanForFiles scans the ops-sop directory and finds files ending in .md or .asciidoc and then it will take all of those files and put them inside of a MDFile or ADFile object respectively. It also performs a GitLog to find authors and dates for files, and will grab other data from the files to put into the object.

func ToBulkJSON

func ToBulkJSON(s []Sop) (map[string]string, error)

ToBulkJSON takes all the Sop objects and puts them into the map that will be used when calling IndexSOP after encoding the Sop into JSON. The name of the SOP will be the key and the json encoded sop as the value for the key-value pair.

Types

type ADFile

type ADFile struct {
	Path         string
	Name         string
	Content      string
	Author       []string
	CreationDate time.Time
	LastUpdated  time.Time
	Tags         []string
	Link         string
}

ADFile is the struct for a asciidoc file

type Config

type Config struct {
	Time       int
	ElasticURL string
	GitURL     string
	RepoName   string
	GitScript  string
	RepoURL    string
}

Config struct holds all the important values used across the program.

func GetConfig

func GetConfig(kubecli client.Client) (Config, error)

GetConfig uses a Kubernetes client and then gets the configmap object. Then it puts the data from that configmap object into the Config object which can then be used and accessed across the whole program.

type Dates

type Dates struct {
	Oldest time.Time
	Newest time.Time
}

Dates contains the creation date of the SOP document and the date it was last updated

func GetAuthorsAndDates

func GetAuthorsAndDates(data []byte) ([]string, Dates, error)

GetAuthorsAndDates takes the slice of bytes (data) which contains the git log of a file and then goes through and finds all the unique authors of the file as well as the creation date and date the file was last updated and then returns that information.

type ElasticClient

type ElasticClient struct {
	// contains filtered or unexported fields
}

ElasticClient struct holds the elasticsearch client

func NewElasticClient

func NewElasticClient(addresses []string, username, password string) (ElasticClient, error)

NewElasticClient creates a new elastic client which lets SOPs be indexed into the elasticsearch cluster.

func (*ElasticClient) CreateOrUpdateIndex

func (ec *ElasticClient) CreateOrUpdateIndex(index, documentID, body string) error

CreateOrUpdateIndex will instantiate a request object and then perform the request. In this particular case, it will create or update each index for an SOP.

type Indexer

type Indexer interface {
	CreateOrUpdateIndex(index, documentID, body string) error
}

Indexer interface which allows indexer to create or update an index

type MDFile

type MDFile struct {
	Path         string
	Name         string
	Content      string
	Author       []string
	CreationDate time.Time
	LastUpdated  time.Time
	Tags         []string
	Link         string
}

MDFile is the struct for a markdown file

type Sop

type Sop struct {
	Name         string    `json:"name"`
	Path         string    `json:"path"`
	Author       []string  `json:"author"`
	CreationDate time.Time `json:"creationDate"`
	LastUpdated  time.Time `json:"lastUpdated"`
	Commit       string    `json:"commit"`
	Content      string    `json:"content"`
	Tags         []string  `json:"tags"`
	Link         string    `json:"links"`
}

Sop struct contains all the information that will be in the index for the sop document

func ToBulkSOP

func ToBulkSOP(mf []MDFile, af []ADFile) ([]Sop, error)

ToBulkSOP takes a slice of MDFile objects and a slice of ADFile objects and turns both into SOP objects and then returns a slice containing all of them as SOP objects.