eyes

package module
v0.0.0-...-949532f Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 21, 2020 License: MIT Imports: 26 Imported by: 0

README

eyes-golang

Eyes of the Community golang backend. go install ==> places library in $GOPATH/pkg/linux_amd64/github.com/dgnabasik/eyes.a The .env file must be source'ed for the eyes library to function properly. Accesses PostgreSQL database.

Documentation

Index

Constants

View Source 
const (
	LotsOfStars = "***********************************************************************************"
)

constants

View Source 
const (
	PrefixTimeOffset = "T11:45:26.371Z"
)

PrefixTimeOffset used for time formatting.

Variables

This section is empty.

Functions

func CheckErr 

func CheckErr(err error)

CheckErr database error handler.

func CreateKeyValuePairs 

func CreateKeyValuePairs(m map[string]interface{}) string

CreateKeyValuePairs func map[string]interface{}

func CreateSharedLinkURLs 

func CreateSharedLinkURLs(commonFilename string) error

CreateSharedLinkURLs func calls Dropbox API for all files with commonFilename. NOT USED! 

curl -X POST https://api.dropboxapi.com/2/sharing/create_shared_link_with_settings \
  -H "Authorization: Bearer ywHtOwT7ZB0AAAAAAABbgoC0gaLDHAKrQ2rs3LWyGrcyme6u7tOZiD0F0oIbprzR" \
  -H "Content-Type: application/json" \
  -d "{\"path\": \"/086d3dd999a7abab.jpg\", \"settings\": {\"requested_visibility\": \"public\"}}"

func CurrentTimeString 

func CurrentTimeString() string

CurrentTimeString func

func FetchTranscription 

func FetchTranscription(jobID string) (string, error)

FetchTranscription func returns transcriptions string. Command: curl -XGET https://api.temi.com/v1/jobs/sd541295393/transcript?version=machine -H "Accept: text/plain" -H "Authorization: Bearer <api-token>" Response: Speaker 1: 00:05 text

func FormatDate 

func FormatDate(t time.Time) string

FormatDate func as yyyy-MM-dd

func GetDatabaseConnectionString 

func GetDatabaseConnectionString() string

GetDatabaseConnectionString func.

func GetDatabaseReference 

func GetDatabaseReference() (*sql.DB, error)

GetDatabaseReference func. SET search_path TO flyingmonkeys, public; The sql.Open function should be called just once. It is rarely necessary to close a DB.

func GetDateRangeFromMonthyear 

func GetDateRangeFromMonthyear(monthYear string) (time.Time, time.Time)

GetDateRangeFromMonthyear returns 2 start and end of month dates. Assumes lowercase monthYear.

func GetDummyID 

func GetDummyID() int32

GetDummyID func avoids "Error:Field validation for 'ID' failed on the 'required' tag"

func GetLastFileModTime 

func GetLastFileModTime(filename string) (time.Time, error)

GetLastFileModTime func returns last file mod time.

func GetSharedLinkURLs 

func GetSharedLinkURLs(path string) ([]string, error)

GetSharedLinkURLs func may need a second before dropbox URL is available after file has been copied to shared folder. Lists files/folders from EYES_TARGETURLPREFIX.

func IndexOf 

func IndexOf(a []string, str string) int

IndexOf func returns index of complete string in array.

func IndexOfSubstring 

func IndexOfSubstring(a []string, substr string) int

IndexOfSubstring func returns first index that contains substring in array.

func MonitorFileSystem 

func MonitorFileSystem(tableController TableController, folderToMonitor string, action int16)

MonitorFileSystem scans for existing and new image files. image filename format: IMG_20200926_130139.jpg action=0 audio filename format: recording-20200926-130140.mp3 action=1 transcription format: recording-20200926-130140.txt action=2

func NoRowsReturned 

func NoRowsReturned(err error) bool

NoRowsReturned func

func ParseFilenameAsDateTime 

func ParseFilenameAsDateTime(fn string) (time.Time, error)

ParseFilenameAsDateTime inputs have format 20200926_152432 for image and 20200926-152402 for audio.

func ProcessExistingFiles 

func ProcessExistingFiles(tableController TableController)

ProcessExistingFiles assumes that all necessary files exist in their original locations, not their target locations.

func ProcessTranscription 

func ProcessTranscription(audioFile string) (string, error)

ProcessTranscription func returns Transcription filename. audioFile MUST be a dropbox URL!!

func RandomHex 

func RandomHex(n int) (string, error)

RandomHex func: use n=12 for [aidata.keycode]

func ReadBinaryFile 

func ReadBinaryFile(path string) ([]byte, error)

ReadBinaryFile func

func ReadTextFileAsArray 

func ReadTextFileAsArray(path string) ([]string, error)

ReadTextFileAsArray reads a whole file into memory and returns a slice of its lines.

func ReadTextFileAsString 

func ReadTextFileAsString(path string) (string, error)

ReadTextFileAsString reads entire file into string. Removes first line and any empty lines. Perform redaction here?

func ReconcileExistingFiles 

func ReconcileExistingFiles(tableController TableController) error

ReconcileExistingFiles func inserts orphaned groups (which may need Dropbox links) from EYES_TARGETURLPREFIX into database.

func SingleFileMonitor 

func SingleFileMonitor(filename string)

SingleFileMonitor func

func WriteTextFileAsString 

func WriteTextFileAsString(path string, str string) error

WriteTextFileAsString func

Types

type AIdata 

type AIdata struct {
	ID                int32     `json:"id" xml:"id" form:"id" binding:"required"`
	KeyCode           string    `json:"keycode" xml:"keycode" form:"keycode" binding:"required"`
	RecorderCode      string    `json:"recordercode" xml:"recordercode" form:"recordercode" binding:"required"`
	InterviewDate     time.Time `json:"interviewdate" xml:"interviewdate" form:"interviewdate" binding:"required"`
	InterviewLocation string    `json:"interviewlocation" xml:"interviewlocation" form:"interviewlocation" binding:"required"`
	LanguageCode      string    `json:"languagecode" xml:"languagecode" form:"languagecode" binding:"required"`
	ImageData         string    `json:"imagedata" xml:"imagedata" form:"imagedata" binding:"required"` // 200
	AudioData         string    `json:"audiodata" xml:"audiodata" form:"audiodata" binding:"required"` // 200
	TransData         string    `json:"transdata" xml:"transdata" form:"transdata" binding:"required"` // 200
	Transcribed       string    `json:"transcribed" xml:"transcribed" form:"transcribed" binding:"required"`
	UpVotes           int32     `json:"upvotes" xml:"upvotes" form:"upvotes" binding:"required"`
	DownVotes         int32     `json:"downvotes" xml:"downvotes" form:"downvotes" binding:"required"`
	Modified          int32     `json:"modified" xml:"modified" form:"modified" binding:"required"`
	DateUpdated       time.Time `json:"dateupdated" xml:"dateupdated" form:"dateupdated" binding:"required"`
}

AIdata struct.

type AIdataInterface 

type AIdataInterface interface {
	GetAIdata() ([]AIdata, error)
	GetAIdataFiltered(langcode string, monthYear string, recorder string) ([]AIdata, error)
	GetAIdataByKeycode(keycode string) (AIdata, error)
	GetAIdataCount() (int64, error)
	GetDistinctKeycodes() ([]string, error)
	CreateAIdata(aidata AIdata) (AIdata, error)
	UpdateAIvote(keycode string, direction bool) error
	DeleteAIdataByKeycode(keycode string) error
}

AIdataInterface repository interface

type AIdataRepository 

type AIdataRepository struct {
	DB      *sql.DB
	DBStore AIdataInterface
}

AIdataRepository struct

func (*AIdataRepository) CreateAIdata 

func (r *AIdataRepository) CreateAIdata(aidata AIdata) (AIdata, error)

CreateAIdata method.

func (*AIdataRepository) DeleteAIdataByKeycode 

func (r *AIdataRepository) DeleteAIdataByKeycode(keycode string) error

DeleteAIdataByKeycode method.

func (*AIdataRepository) GetAIdata 

func (r *AIdataRepository) GetAIdata() ([]AIdata, error)

GetAIdata method returns everything.

func (*AIdataRepository) GetAIdataByKeycode 

func (r *AIdataRepository) GetAIdataByKeycode(keycode string) ([]AIdata, error)

GetAIdataByKeycode method returns single row as array. Exact lowercase match on keycode.

func (*AIdataRepository) GetAIdataCount 

func (r *AIdataRepository) GetAIdataCount() (int64, error)

GetAIdataCount method returns -1 upon error.

func (*AIdataRepository) GetAIdataFiltered 

func (r *AIdataRepository) GetAIdataFiltered(langcode string, monthYear string, recorder string) ([]AIdata, error)

GetAIdataFiltered method uses yyyy-mm-dd date formats. Filter by LanguageCode and InterviewDate month/year ordered by UpVotes.

func (*AIdataRepository) GetDistinctKeycodes 

func (r *AIdataRepository) GetDistinctKeycodes() ([]string, error)

GetDistinctKeycodes method returns distinct Keycodes.

func (*AIdataRepository) UpdateAIvote 

func (r *AIdataRepository) UpdateAIvote(keycode string, direction string) error

UpdateAIvote method updates only UpVotes, DownVotes, DateUpdated.

type Authorization 

type Authorization struct {
	Auth0Domain       string `json:"auth0domain" binding:"required"`
	Auth0ClientID     string `json:"auth0clientid" binding:"required"`
	Auth0Audience     string `json:"auth0audience" binding:"required"`
	Auth0Callback     string `json:"auth0callback" binding:"required"`
	Auth0ClientSecret string `json:"auth0clientsecret" binding:"required"`
}

Authorization struct

func GetAuthorizationValues 

func GetAuthorizationValues() Authorization

GetAuthorizationValues func

type FileGroup 

type FileGroup struct {
	CommonFilename string `json:"commonfilename" binding:"required"`
	ImageFilename  string `json:"imagefilename" binding:"required"`
	AudioFilename  string `json:"audiofilename" binding:"required"`
	TransFilename  string `json:"transfilename" binding:"required"`
}

FileGroup struct

func GetDropboxURLs 

func GetDropboxURLs(commonFilename string) ([]FileGroup, error)

GetDropboxURLs func. The Dropbox API call returns preview links, so have to replace prefix URL. Lists files from EYES_TARGETURLPREFIX. Return actual Dropbox file URLs, each with its unique 16-char prefix, for storage of URL into database. Filter using commonFilename. If len(commonFilename) == 0 then do not filter. https://www.dropbox.com/s/36zqs6l7kmt9o74/086d3dd999a7abab.jpg?dl=0 ==> https://dl.dropboxusercontent.com/s/36zqs6l7kmt9o74/086d3dd999a7abab.jpg?dl=0

type MetaData 

type MetaData struct {
	RecorderCode      string `json:"recordercode" binding:"required"`
	LanguageCode      string `json:"languagecode" binding:"required"`
	InterviewLocation string `json:"interviewlocation" binding:"required"`
}

MetaData struct

func GetMetaData 

func GetMetaData(tableController TableController) (MetaData, error)

GetMetaData func depends upon env.var EYES_RECORDERCODE

type Operation 

type Operation struct {
	ID                int32     `json:"id" xml:"id" form:"id" binding:"required"`
	RecorderName      string    `json:"recordername" xml:"recordername" form:"recordername" binding:"required"`
	RecorderCode      string    `json:"recordercode" xml:"recordercode" form:"recordercode" binding:"required"`
	ImageRecorder     string    `json:"imagerecorder" xml:"imagerecorder" form:"imagerecorder"`
	AudioRecorder     string    `json:"audiorecorder" xml:"audiorecorder" form:"audiorecorder"`
	Certificate       string    `json:"certificate" xml:"certificate" form:"certificate"`
	InterviewLocation string    `json:"interviewlocation" xml:"interviewlocation" form:"interviewlocation" binding:"required"`
	LanguageCode      string    `json:"languagecode" xml:"languagecode" form:"languagecode" binding:"required"`
	DateUpdated       time.Time `json:"dateupdated" xml:"dateupdated" form:"dateupdated" binding:"required"`
}

Operation struct

type OperationInterface 

type OperationInterface interface {
	GetOperation() ([]Operation, error)
	CreateOperation(op Operation) (Operation, error)
	UpdateOperation(op Operation) error
	GetRecorderCodes() ([]string, error)
}

OperationInterface repository interface

type OperationRepository 

type OperationRepository struct {
	DB      *sql.DB
	DBStore OperationInterface
}

OperationRepository struct

func (*OperationRepository) CreateOperation 

func (r *OperationRepository) CreateOperation(op Operation) (Operation, error)

CreateOperation method needs to test for unique recorderCode.

func (*OperationRepository) GetOperation 

func (r *OperationRepository) GetOperation() ([]Operation, error)

GetOperation method. Nullable: imageRecorder, audioRecorder, certificate.

func (*OperationRepository) GetRecorderCodes 

func (r *OperationRepository) GetRecorderCodes() ([]string, error)

GetRecorderCodes method (distinct)

func (*OperationRepository) UpdateOperation 

func (r *OperationRepository) UpdateOperation(op Operation) error

UpdateOperation method

type SourceFolders 

type SourceFolders struct {
	ImageSourceFolder string `json:"imagesourcefolder" binding:"required"` // EYES_IMAGESOURCEFOLDER
	AudioSourceFolder string `json:"audiosourcefolder" binding:"required"` // EYES_AUDIOSOURCEFOLDER
	TransSourceFolder string `json:"transsourcefolder" binding:"required"` // EYES_TRANSSOURCEFOLDER
	TargetURLprefix   string `json:"targeturlprefix" binding:"required"`   // EYES_TARGETURLPREFIX
}

SourceFolders struct

func GetSourceFolders 

func GetSourceFolders() SourceFolders

GetSourceFolders func

type TableController 

type TableController struct {
	ARepository AIdataRepository
	ORepository OperationRepository
}

TableController struct

func InitializeController 

func InitializeController() (TableController, error)

InitializeController func: defer is set in routing::InitializeRoutes()

func (*TableController) CreateAIdata 

func (c *TableController) CreateAIdata(ctx *gin.Context)

CreateAIdata method. Need to test for unique KeyCode.

func (*TableController) CreateOperation 

func (c *TableController) CreateOperation(ctx *gin.Context)

CreateOperation method.

func (*TableController) GetAIdata 

func (c *TableController) GetAIdata(ctx *gin.Context)

GetAIdata method

func (*TableController) GetAIdataByKeycode 

func (c *TableController) GetAIdataByKeycode(ctx *gin.Context)

GetAIdataByKeycode method returns 0..1 rows as array.

func (*TableController) GetAIdataCount 

func (c *TableController) GetAIdataCount(ctx *gin.Context)

GetAIdataCount method

func (*TableController) GetAIdataFiltered 

func (c *TableController) GetAIdataFiltered(ctx *gin.Context)

GetAIdataFiltered method parameterized.

func (*TableController) GetDistinctKeycodes 

func (c *TableController) GetDistinctKeycodes() ([]string, error)

GetDistinctKeycodes func does not use gin.Context

func (*TableController) GetOperation 

func (c *TableController) GetOperation(ctx *gin.Context)

GetOperation method

func (*TableController) GetRecorderCodes 

func (c *TableController) GetRecorderCodes(ctx *gin.Context)

GetRecorderCodes method

func (*TableController) UpdateAIvote 

func (c *TableController) UpdateAIvote(ctx *gin.Context)

UpdateAIvote method uses unique 16-char KeyCode.

func (*TableController) UpdateOperation 

func (c *TableController) UpdateOperation(ctx *gin.Context)

UpdateOperation method uses unique RecorderCode.

type TemiJobStatus 

type TemiJobStatus struct {
	ID        string `json:"id" binding:"required"`
	Status    string `json:"status" binding:"required"`
	CreatedOn string `json:"created_on" binding:"required"` // 2020-10-17T22:17:30.71 not time.Time
	Metadata  string `json:"metadata" binding:"required"`
}

TemiJobStatus struct

func CreateTranscriptionJob 

func CreateTranscriptionJob(audioFile string) (TemiJobStatus, error)

CreateTranscriptionJob func returns job id. Command: curl -XPOST -i https://api.temi.com/v1/jobs -H "Content-Type: application/json" -d '{"media_url": "https://www.youtube.com/watch?v=xK2vinLqFzw"}' -H "Authorization: Bearer <Api Key>" Response: {"id":"sd541295393","status":"in_progress","created_on":"2020-10-09T10:02:19.823","metadata":"meta content"}

type TemiPayload 

type TemiPayload struct {
	MediaURL string `json:"media_url"`
}

TemiPayload struct

type TemiTranscriptionJob 

type TemiTranscriptionJob struct {
	ID        string `json:"id" binding:"required"`
	Status    string `json:"status" binding:"required"`
	CreatedOn string `json:"created_on" binding:"required"` // 2020-10-17T22:17:30.71 not time.Time
	WebURL    string `json:"web_url" binding:"required"`
	Duration  int    `json:"duration_seconds" binding:"required"`
	Name      string `json:"name" binding:"required"`
	Metadata  string `json:"metadata" binding:"required"`
}

TemiTranscriptionJob struct

func GetTranscriptionStatus 

func GetTranscriptionStatus(jobID string) (TemiTranscriptionJob, error)

GetTranscriptionStatus func. If status="transcribed" it returns text URL in web_url field. Command: curl -XGET https://api.temi.com/v1/jobs/sd541295393 -H "Authorization: Bearer <api-token>" Response: {"id":"sd541295393","status":"transcribed","created_on":"2020-10-09T10:02:19.823","web_url":"https://www.temi.com/editor/file/541295393","duration_seconds":76,"name":"recording-20201008-195051","metadata":"meta content"}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.