tests

README

System Test Plan

General Guidelines

• Create a single testing package to hold all system tests.
• Establish a Test User so that generated files are in one directory.
• Give each test a project name that is unique and descriptive.
• Add info on memory usage and duration to log.
• Put request.yaml files into a directory, name each file using RequestName + StepName.
• Make bibleId a variable in each test.
• Give each request.yaml a func Test{RequestName}{StepName}.
• Each func can process the request with a variable list of BibleIds.
• There is a system wide list of BibleId's to test as well as an optional list in each test.
• There is a test harness that submits request to the server or cli.
• The test harness will always move output to a specific test output directory.
• Each test func will have some kind of verification, at least a record or line count.
• Each test function will have a comment that describes what manual verification is recommended.

Terminology

• Plain Text refers to the DBP plain text dataset.
• Plain Text Edit refers to DBP plain text with headings extracted from USX.
• USX Text Edit refers to USX text edited to contain only content in Script.
• Script refers to Excel spreadsheets used in process of recording Audio Bible.
• Whisper is a Speech to Text program produced by OpenAI.
• FastText is a Word Encoding program produced by Meta.
• Aeneas is a Timestamp generating program.
• Librosa is an MFCC generating program.
• Gonum/float is a go library for floating point
• Pandas is a python dataset manipulation tool frequently used by AI model researchers.

Script level text tests

• 01 Load Plain Text Script from BB-API, CSV output

• 02 Load Plain Text Edit from BB-API, JSON output

• 03 Load USX Text Edit from BB-API, CSV output

• 04 Load Script Using File, Sqlite output

• 05 Compare Plain Text to Plain Text Edit, HTML output

• 06 Compare Plain Text Edit to USX Edit, HTML output

• 07 Compare USX Edit to Script, HTML output (Mark Scott use case)

• 08 Compare Plain Text Edit to Script, HTML output

• Note: Part of the purpose of these comparison tests is to verify that each kind of data parsing is correct.

Script level audio tests

• 09 Load Audio from BB-API do Whisper speech to text, output the text as JSON

• 10 Compare text to USXEdit, output HTML (Mark Scott use case)

• 11 Load Audio from POST, do speech to text, output JSON (OBT use case)

• 12 Reuse request with second and third passage, output JSON (OBT use case)

• 13 Load Project Text with Plain Text Edit, because it is verse aligned. Load time stamps from BB-API, Output CSV

• 14 Load same Text, and timestamp with aeneas, compare the timestamps using gonum/float to find in difference they are equal (float.SubTo, float.Abs, float.Max. And, I need to look at Sandeep’s method

• 15 Load Script and script timestamps, compare with above case where the script is verse aligned.

• 16 Output sql script to insert a fileset's timestamp data into DBP (Jon's OBT use case)

• Note: The comparison code to do this has not yet been written (5/1/24)

• 17 Load audios and compute timestamps and mfcc. Verification is limited to record counts. output CSV

• 18 Repeat and output JSON

• 19 Load CSV file into Pandas

• 20 Load Json file into Pandas

Word level text tests

• Using existing Plain Text Edit, generate words, validate word split, fast text words, generate CSV
• Using existing USX Text Edit, generate words, validate word split, fast text words, generate JSON
• Load CSV file into Pandas
• Load Json file into Pandas

Word level audio tests

• Load audio into Word Plain Text Test, timestamp words using script level timestamps, generate MFCC, output CSV
• Load audio into Word USX Text Edit, timestamp words using script level timestamps, generate MFCC, output JSON
• Load CSV output into Pandas
• Load JSON output into Pandas

Full Test

• Load USX Text Edit, Audio, timestamp, compute MFCC, word Split, timestamp words, compute MFCC of words, fast text encode words, output JSON.
• Output CSV on same database.

Error Tests

• List here a number of error conditions, output error

Documentation

Index

• Constants
• func APITestUtility(request string, tests []APITest, t *testing.T)
• func CLIExec(requestYaml string, t *testing.T) (string, string)
• func DirectSqlTest(request string, tests []SqliteTest, t *testing.T) string
• func DirectTestUtility(requestYaml string, tests []CtlTest, t *testing.T)
• func ExtractFilename(stdout string) string
• func FCBHDatasetExec(requestYaml string, t *testing.T) (string, string)
• func NumCVSFileLines(filename string, t *testing.T) int
• func NumCVSLines(content []byte, t *testing.T) int
• func NumFileLines(filename string, t *testing.T) int
• func NumHTMLFileLines(filename string, t *testing.T) int
• func NumJSONFileLines(filename string, t *testing.T) int
• func NumJSONLines(content []byte, t *testing.T) int
• func SelectScalarInt(conn *sql.DB, sql string, t *testing.T) int
• type APITest
• type CtlTest
• type SqliteTest

Constants

const (
	HOST       = `http://localhost:8080/`
	UPLOADHOST = `http://localhost:7777/upload`
	OUTPUT     = `/Users/gary/FCBH2024/systemtest/`
)

Functions

func APITestUtility

func APITestUtility(request string, tests []APITest, t *testing.T)

func CLIExec

func CLIExec(requestYaml string, t *testing.T) (string, string)

func DirectSqlTest

func DirectSqlTest(request string, tests []SqliteTest, t *testing.T) string

DirectSqlTest requires a sqlite database as output to perform tests on the result

func DirectTestUtility

func DirectTestUtility(requestYaml string, tests []CtlTest, t *testing.T)

func ExtractFilename

func ExtractFilename(stdout string) string

func FCBHDatasetExec

func FCBHDatasetExec(requestYaml string, t *testing.T) (string, string)

func NumCVSFileLines

func NumCVSFileLines(filename string, t *testing.T) int

func NumCVSLines

func NumCVSLines(content []byte, t *testing.T) int

func NumFileLines

func NumFileLines(filename string, t *testing.T) int

func NumHTMLFileLines

func NumHTMLFileLines(filename string, t *testing.T) int

func NumJSONFileLines

func NumJSONFileLines(filename string, t *testing.T) int

func NumJSONLines

func NumJSONLines(content []byte, t *testing.T) int

func SelectScalarInt

func SelectScalarInt(conn *sql.DB, sql string, t *testing.T) int

Types

type APITest

type APITest struct {
	BibleId  string
	Expected int
	Diff     int
}

type CtlTest

type CtlTest struct {
	BibleId   string
	Name      string
	TextNtId  string
	AudioNTId string
	TextType  request.MediaType
	Language  string
	Expected  int
}

type SqliteTest

type SqliteTest struct {
	Query string
	Count int
}