crontinuous

package module
v1.1.10 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 15, 2024 License: MIT Imports: 18 Imported by: 0

README

Vulcan Crontinuous

A cron based scheduler to execute Vulcan scans and digest report generation.

To run execute:

go build cmd/vulcan-crontinuous
./cmd/vulcan-crontinuous/vulcan-crontinuous -c _resources/config/local.toml

Exposed API

The exposed API is very simple. It exposes two group of endpoints to handle schedules for scans and reports.

Scan scheduling

  • Get a snapshot of the current scheduled cron jobs.

    GET to /entries

    The endpoint will return a response like this.

 [
    {
        "program_id": "44a57d24-2a23-41a0-a986-2f11a68e9e8b",
        "team_id":"461a62aa-6e1c-11e8-802e-4c32758b498f",
        "cron_spec":"15 * * * *"
    },
    {
        "program_id": "8491b4c9-efd1-4ea0-bd83-a627edb61b65",
        "id":"561a62aa-6e1c-11e8-802e-4c32758b498f",
        "cron_spec":"15 * * * *"
    }
]

  • Get a snapshot of the current scheduled cron jobs for a program.

    GET to /entries/:programID

    The endpoint will return a response like this.

{
    "program_id": "44a57d24-2a23-41a0-a986-2f11a68e9e8b",
    "team_id":"461a62aa-6e1c-11e8-802e-4c32758b498f",
    "cron_spec":"15 * * * *"
}

  • Create or update a cron job.

    POST to /settings/:programID/:teamID with a json payload in the body like this:

 {
     "str" : "* * * * * *"
 }

This will create a new cron job that will schedule a scan associated with the given program ID.

If the program ID already exists it will replace the schedule with the new passed cron string.

  • Bulk set.

    POST to /entries/ with a json payload in the body like this:

 [
     {
      "str" : "* * * * * *",
      "program_id":"global_default",
      "team_id":"a_team_id"
      "overwrite": true/false
     },
     {
      "str" : "* * * * * *",
      "program_id":"global_default",
      "team_id":"a_team_id"
      "overwrite": true/false
     }
 ]

This will create a new cron job for each item defined in the array only
if no other schedule for the same program exists, unless the 'overwrite' param
is set to true (default if omitted in the payload is false), in that case the
existent job is overwritten

  • Delete a schedule.

    DELETE to: /entries/:programID .

    The end point will return 200 if the entry was deleted and 400 if the entry was not found.

Report scheduling

  • Get a snapshot of the current scheduled report cron jobs.

    GET to /report/entries

    The endpoint will return a response like this.

 [
    {
        "team_id":"461a62aa-6e1c-11e8-802e-4c32758b498f",
        "cron_spec":"15 * * * *"
    },
    {
        "id":"561a62aa-6e1c-11e8-802e-4c32758b498f",
        "cron_spec":"15 * * * *"
    }
]

  • Get a snapshot of the current scheduled report cron jobs for a team.

    GET to /report/entries/:teamID

    The endpoint will return a response like this.

{
    "team_id":"461a62aa-6e1c-11e8-802e-4c32758b498f",
    "cron_spec":"15 * * * *"
}

  • Create or update a report cron job.

    POST to /report/settings/:teamID with a json payload in the body like this:

 {
     "str" : "* * * * * *"
 }

This will create a new cron job that will schedule a report associated with the given team ID.

If the team ID already exists it will replace the schedule with the new passed cron string.

  • Bulk set.

    POST to /report/entries/ with a json payload in the body like this:

 [
     {
      "str" : "* * * * * *",
      "team_id":"a_team_id"
      "overwrite": true/false
     },
     {
      "str" : "* * * * * *",
      "team_id":"a_team_id"
      "overwrite": true/false
     }
 ]

This will create a new report cron job for each item defined in the array only
if no other schedule for the same program exists, unless the 'overwrite' param
is set to true (default if omitted in the payload is false), in that case the
existent job is overwritten

  • Delete a schedule.

    DELETE to: /report/entries/:teamID .

    The end point will return 200 if the entry was deleted and 400 if the entry was not found.

Docker execute

Those are the variables you have to use:

Variable Description Sample
PORT 8081
AWS_REGION eu-west-1
AWS_S3_ENDPOINT AWS SDK S3 endpoint http://localhost:9000
PATH_STYLE Access bucket through path instead hostname false
CRONTINUOUS_BUCKET vulcan-crontinuous-local-bucket
VULCAN_API http://localhost:8080/api
VULCAN_USER User to interact with Vulcan API when creating scans vulcan-scheduler@vulcan.com
VULCAN_TOKEN Vulcan API authorization token TOKEN
ENABLE_TEAMS_WHITELIST_SCAN Flag to enable whitelist on scan scheduling false
TEAMS_WHITELIST_SCAN List of whitelisted team IDs for scan scheduling []
ENABLE_TEAMS_WHITELIST_REPORT Flag to enable whitelist on report scheduling false
TEAMS_WHITELIST_REPORT List of whitelisted team IDs for report scheduling []
RANDOMIZE_CRON_MINUTE_SUFFIXES Coma separated list of program suffixes to randomize the cron minute
RANDOMIZE_CRON_MINUTE_INTERVAL Specifies the interval range to randomize the cron minute. From 0 to 59 59 
docker build . -t vc

# Use the default config.toml customized with env variables.
docker run --env-file ./local.env vc

# Use custom config.toml
docker run -v `pwd`/custom.toml:/app/config.toml vc

Documentation

Index

Constants

View Source 
const (
	MaxRandomizeCronMinuteInterval int = 59

	ScanCronType CronType = iota
	ReportCronType
)
View Source 
const (
	S3ReportsCrontabFilename = "reportsCrontab.json"
)
View Source 
const (
	S3ScansCrontabFilename = "crontab.json"
)

Variables

View Source 
var (
	// ErrScheduleNotFound is returned by DeleteSchedule method if the id for the schedule is not found.
	ErrScheduleNotFound = errors.New("ErrorScheduleNotFound")

	// ErrMalformedSchedule indicates the given cron spec is invalid.
	ErrMalformedSchedule = errors.New("ErrorMalformedSchedule")

	// ErrMalformedEntry indicates the given entry is invalid.
	ErrMalformedEntry = errors.New("ErrorMalformedEntry")

	// ErrInvalidCronType indicates the given cron type is invalid.
	ErrInvalidCronType = errors.New("ErrInvalidCronType")
)

Functions

This section is empty.

Types

type Config 

type Config struct {
	Bucket                     string
	EnableTeamsWhitelistScan   bool
	TeamsWhitelistScan         []string
	EnableTeamsWhitelistReport bool
	TeamsWhitelistReport       []string

	RandomizeCronMinuteProgramSuffixes string
	RandomizeCronMinuteInterval        int
}

Config holds the information required by the Crontinuous

type CronEntry 

type CronEntry interface {
	GetID() string
	GetCronSpec() string
}

type CronType 

type CronType int

type Crontinuous 

type Crontinuous struct {
	// contains filtered or unexported fields
}

Crontinuous implements the logic for storing and executing programs.

func NewCrontinuous 

func NewCrontinuous(cfg Config, logger *logrus.Logger,
	scanCreator ScanCreator, scanCronStore ScanCronStore,
	reportSender ReportSender, reportCronStore ReportCronStore) *Crontinuous

NewCrontinuous creates a new instance of the crontinuous service.

func (*Crontinuous) BulkCreate 

func (c *Crontinuous) BulkCreate(typ CronType, entries []CronEntry, overwriteSettings []bool) error

BulkCreate tests for each specified entry if an entry with the same programID exists. If it exists and overwrite setting for that entry is set to false the method does nothing. If it doesn't exist or overwrite setting is set to true, the method creates/overwrites the entry.

func (*Crontinuous) GetEntries 

func (c *Crontinuous) GetEntries(typ CronType) ([]CronEntry, error)

GetEntries returns a snapshot of the current entries.

func (*Crontinuous) GetEntryByID 

func (c *Crontinuous) GetEntryByID(typ CronType, ID string) (CronEntry, error)

GetEntryByID returns a snapshot of the current entries.

func (*Crontinuous) RemoveEntry 

func (c *Crontinuous) RemoveEntry(typ CronType, ID string) error

RemoveEntry remove an existing entry.

func (*Crontinuous) SaveEntry 

func (c *Crontinuous) SaveEntry(typ CronType, entry CronEntry) error

SaveEntry adds a new entry to the crontab.

func (*Crontinuous) Start 

func (c *Crontinuous) Start() error

Start reads the cron entries from store, s3 by now, and initializes all the entries.

func (*Crontinuous) Stop 

func (c *Crontinuous) Stop()

Stop signals the command processor to stop processing commands and wait for it to exit.

type ReportCronStore 

type ReportCronStore interface {
	GetReportEntries() (map[string]ReportEntry, error)
	SaveReportEntries(entries map[string]ReportEntry) error
}

type ReportEntry 

type ReportEntry struct {
	TeamID   string `json:"team_id"`
	CronSpec string `json:"cron_spec"`
}

ReportEntry defines the data stored by a report cron entry.

func (ReportEntry) GetCronSpec 

func (e ReportEntry) GetCronSpec() string

func (ReportEntry) GetID 

func (e ReportEntry) GetID() string

type ReportSender 

type ReportSender interface {
	SendReport(teamID string) error
}

ReportSender defines the service needed by the crontinuos component in order to trigger digest reports generation and sending.

type S3CronStore 

type S3CronStore struct {
	// contains filtered or unexported fields
}

func NewS3CronStore 

func NewS3CronStore(bucket, scanCronKey, reportCronKey string, s3Client s3iface.S3API) *S3CronStore

func (*S3CronStore) GetReportEntries 

func (s *S3CronStore) GetReportEntries() (map[string]ReportEntry, error)

func (*S3CronStore) GetScanEntries 

func (s *S3CronStore) GetScanEntries() (map[string]ScanEntry, error)

func (*S3CronStore) SaveReportEntries 

func (s *S3CronStore) SaveReportEntries(entries map[string]ReportEntry) error

func (*S3CronStore) SaveScanEntries 

func (s *S3CronStore) SaveScanEntries(entries map[string]ScanEntry) error

type ScanCreator 

type ScanCreator interface {
	CreateScan(scanID, teamID string) error
}

ScanCreator defines the services needed by the crontinuos component in order to create scans.

type ScanCronStore 

type ScanCronStore interface {
	GetScanEntries() (map[string]ScanEntry, error)
	SaveScanEntries(entries map[string]ScanEntry) error
}

type ScanEntry 

type ScanEntry struct {
	ProgramID string `json:"program_id"`
	TeamID    string `json:"team_id"`
	CronSpec  string `json:"cron_spec"`
}

ScanEntry defines the data stored by a scan cron entry.

func (ScanEntry) GetCronSpec 

func (e ScanEntry) GetCronSpec() string

func (ScanEntry) GetID 

func (e ScanEntry) GetID() string

type ScanRequest 

type ScanRequest struct {
	ProgramID     string    `json:"program_id"`
	ScheduledTime time.Time `json:"scheduled_time"`
	RequestedBy   string    `json:"requested_by"`
}

ScanRequest contains the payload to send to the API scan endpoint.

type VulcanClient 

type VulcanClient struct {
	VulcanAPI   string
	VulcanUser  string
	VulcanToken string

	Log *logrus.Logger
}

VulcanClient provides functionality for interacting with the vulcan-api.

func (*VulcanClient) CreateScan 

func (c *VulcanClient) CreateScan(scanID, teamID string) error

CreateScan creates a scan by calling vulcan-api

func (*VulcanClient) SendReport 

func (c *VulcanClient) SendReport(teamID string) error

SendReport triggers a report sending operation by calling vulcan-api.

Source Files

View all Source files

Directories

Path Synopsis
cmd
vulcan-crontinuous
vulcan-crontinuous/commands

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.