checker

package
v0.0.0-...-2d6eab0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 1, 2020 License: GPL-3.0 Imports: 19 Imported by: 0

README

STARTTLS Check

Evaluates an @mail domain on how secure its TLS settings are. First retrieves all MX records for the domain, then performs a series of checks on each discovered hostname's port 25.

If $HOSTNAME environment variable is set, this is used in the SMTP hello.

What does it check?

For each hostname found via a MX lookup, we check:

  • Can connect (over SMTP) on port 25
  • STARTTLS support
  • Presents a valid certificate
  • TLS version up-to-date
  • Secure TLS ciphers

Build

As a library

go get github.com/efforg/starttls-backend/checker

or if you want to use it as a bin command

go get github.com/efforg/starttls-backend/checker/cmd/starttls-check

NOTE: many ISPs block outbound port 25 to mitigate botnet e-mail spam. If you are on a residential IP, you might not be able to run this tool!

API

The most important API that we provide is checker.CheckDomain(domain string, mxHostnames []string) DomainResult; which performs all associated checks for a particular domain.

This first performs an MX lookup, then performs checks on each of the resulting hostnames. The Status of DomainResult is inherited from the check status of the MX records with the highest priority. So, the Status is set to Success only when all high priority hostnames also have the Success status.

The reason we only require the highest-priority mailservers to pass is because many deploy dummy mailservers as a spam mitigation.

We do, however, provide the check information for the additional hostnames-- they just don't affect the status of the primary domain check.

Command Line Usage

starttls-check -domain <email domain>

For instance, running ./starttls-check -domain gmail.com will check for the TLS configurations (over SMTP) on port 25 for all the MX domains for gmail.com.

Results

From a preliminary STARTTLS scan on the top 1000 alexa domains, performed 3/8/2018, we found:

  • 20.19% of 421 unique MX hostnames don't support STARTTLS.
  • 36.01% of the servers which support STARTTLS didn't present valid certificates.
    • We're not sure how to define valid certificates. On manual inspection, although many certificates are self-signed, it seems that many of these certs are issued for other subdomains owned by the same entity.

Seems like an improvement from results in 2014, but we can do better!

TODO

  • Check DANE
  • Present recommendations for issues
  • Tests

Documentation

Index

Constants

View Source 
const (
	Connectivity     = "connectivity"
	STARTTLS         = "starttls"
	Version          = "version"
	Certificate      = "certificate"
	MTASTS           = "mta-sts"
	MTASTSText       = "mta-sts-text"
	MTASTSPolicyFile = "mta-sts-policy-file"
	PolicyList       = "policylist"
)

IDs for checks that can be run

View Source 
const (
	// TopDomainsSource labels aggregated scans of the top million domains.
	TopDomainsSource = "TOP_DOMAINS"
	// LocalSource labels aggregated scan data for users of the web frontend.
	LocalSource = "LOCAL"
)

Variables

This section is empty.

Functions

func PolicyMatches 

func PolicyMatches(mx string, patterns []string) bool

PolicyMatches return true iff a given mx matches an array of patterns. It is modelled after PolicyMatches in Appendix B of the MTA-STS RFC 8641. Also used to validate hostnames on the STARTTLS Everywhere policy list.

Types

type AggregatedScan 

type AggregatedScan struct {
	Time              time.Time
	Source            string
	Attempted         int
	WithMXs           int
	MTASTSTesting     int
	MTASTSTestingList []string
	MTASTSEnforce     int
	MTASTSEnforceList []string
}

AggregatedScan compiles aggregated stats across domains. Implements ResultHandler.

func (*AggregatedScan) HandleDomain 

func (a *AggregatedScan) HandleDomain(r DomainResult)

HandleDomain adds the result of a single domain scan to aggregated stats.

func (AggregatedScan) PercentMTASTS 

func (a AggregatedScan) PercentMTASTS() float64

PercentMTASTS returns the fraction of domains with MXs that support MTA-STS, represented as a float between 0 and 1.

func (AggregatedScan) TotalMTASTS 

func (a AggregatedScan) TotalMTASTS() int

TotalMTASTS returns the number of domains supporting test or enforce mode.

type Checker 

type Checker struct {
	// Timeout specifies the maximum timeout for network requests made during
	// checks.
	// If nil, a default timeout of 10 seconds is used.
	Timeout time.Duration

	// Cache specifies the hostname scan cache store and expire time.
	// If `nil`, then scans are not cached.
	Cache *ScanCache

	// CheckHostname defines the function that should be used to check each hostname.
	// If nil, FullCheckHostname (all hostname checks) will be used.
	CheckHostname func(string, string, time.Duration) HostnameResult
	// contains filtered or unexported fields
}

A Checker is used to run checks against SMTP domains and hostnames.

func (*Checker) CheckCSV 

func (c *Checker) CheckCSV(domains *csv.Reader, resultHandler ResultHandler, domainColumn int)

CheckCSV runs the checker on a csv of domains, processing the results according to resultHandler.

func (*Checker) CheckDomain 

func (c *Checker) CheckDomain(domain string, expectedHostnames []string) DomainResult

CheckDomain performs all associated checks for a particular domain. First performs an MX lookup, then performs subchecks on each of the resulting hostnames.

The status of DomainResult is inherited from the check status of the MX records with highest priority. This check succeeds only if the hostname checks on the highest priority mailservers succeed. 

`domain` is the mail domain to perform the lookup on.
`expectedHostnames` is the list of expected hostnames.
  If `expectedHostnames` is nil, we don't validate the DNS lookup.

type DomainResult 

type DomainResult struct {
	// Domain being checked against.
	Domain string `json:"domain"`
	// Message if a failure or error occurs on the domain lookup level.
	Message string `json:"message,omitempty"`
	// Status of this check, inherited from the results of preferred hostnames.
	Status DomainStatus `json:"status"`
	// Results of this check, on each hostname.
	HostnameResults map[string]HostnameResult `json:"results"`
	// The list of hostnames which will impact the Status of this result.
	// It discards mailboxes that we can't connect to.
	PreferredHostnames []string `json:"preferred_hostnames"`
	// Expected MX hostnames supplied by the caller of CheckDomain.
	MxHostnames []string `json:"mx_hostnames,omitempty"`
	// Result of MTA-STS checks
	MTASTSResult *MTASTSResult `json:"mta_sts"`
	// Extra global results
	ExtraResults map[string]*Result `json:"extra_results,omitempty"`
}

DomainResult wraps all the results for a particular mail domain.

func NewSampleDomainResult 

func NewSampleDomainResult(domain string) DomainResult

NewSampleDomainResult returns a sample successful domain result for testing. This is exported so other packages can use it in their integration tests.

func (DomainResult) Class 

func (d DomainResult) Class() string

Class satisfies raven's Interface interface. https://github.com/getsentry/raven-go/issues/125

type DomainStatus 

type DomainStatus int32

DomainStatus indicates the overall status of a single domain. 

const (
	DomainSuccess            DomainStatus = 0
	DomainWarning            DomainStatus = 1
	DomainFailure            DomainStatus = 2
	DomainError              DomainStatus = 3
	DomainNoSTARTTLSFailure  DomainStatus = 4
	DomainCouldNotConnect    DomainStatus = 5
	DomainBadHostnameFailure DomainStatus = 6
)

In order of precedence.

type HostnameResult 

type HostnameResult struct {
	*Result
	Domain    string    `json:"domain"`
	Hostname  string    `json:"hostname"`
	Timestamp time.Time `json:"-"`
}

HostnameResult wraps the results of a security check against a particular hostname.

func FullCheckHostname 

func FullCheckHostname(domain string, hostname string, timeout time.Duration) HostnameResult

FullCheckHostname performs a series of checks against a hostname for an email domain. `domain` is the mail domain that this server serves email for. `hostname` is the hostname for this server.

func NoopCheckHostname 

func NoopCheckHostname(domain string, hostname string, _ time.Duration) HostnameResult

NoopCheckHostname returns a fake error result containing `domain` and `hostname`.

type MTASTSResult 

type MTASTSResult struct {
	*Result
	Policy string // Text of MTA-STS policy file
	Mode   string
	MXs    []string
}

MTASTSResult represents the result of a check for inbound MTA-STS support.

func MakeMTASTSResult 

func MakeMTASTSResult() *MTASTSResult

MakeMTASTSResult constructs a base result object and returns its pointer.

func (MTASTSResult) MarshalJSON 

func (m MTASTSResult) MarshalJSON() ([]byte, error)

MarshalJSON prevents MTASTSResult from inheriting the version of MarshalJSON implemented by Result.

type Result 

type Result struct {
	Name     string             `json:"name"`
	Status   Status             `json:"status"`
	Messages []string           `json:"messages,omitempty"`
	Checks   map[string]*Result `json:"checks,omitempty"`
}

Result is the result of a singular check. It's agnostic to the nature of the check performed, and simply stores a reference to the check's name, a summary of what the check should do, as well as any error, failure, or warning messages associated.

func MakeResult 

func MakeResult(name string) *Result

MakeResult constructs a base result object and returns its pointer.

func (Result) Description 

func (r Result) Description() string

Description returns the full-text name of a check.

func (*Result) Error 

func (r *Result) Error(format string, a ...interface{}) *Result

Error adds an error message to this check result. The Error status will override any other existing status for this check. Typically, when a check encounters an error, it stops executing.

func (*Result) Failure 

func (r *Result) Failure(format string, a ...interface{}) *Result

Failure adds a failure message to this check result. The Failure status will override any Status other than Error. Whenever Failure is called, the entire check is failed.

func (Result) MarshalJSON 

func (r Result) MarshalJSON() ([]byte, error)

MarshalJSON writes Result to JSON. It adds status_text and description to the output.

func (Result) StatusText 

func (r Result) StatusText() string

StatusText returns the text version of the Result Status

func (*Result) Success 

func (r *Result) Success() *Result

Success simply sets the status of Result to a Success. Status is set if no other status has been declared on this check.

func (*Result) Warning 

func (r *Result) Warning(format string, a ...interface{}) *Result

Warning adds a warning message to this check result. The Warning status only supercedes the Success status.

type ResultHandler 

type ResultHandler interface {
	HandleDomain(DomainResult)
}

ResultHandler processes domain results. It could print them, aggregate them, write the to the db, etc.

type ScanCache 

type ScanCache struct {
	ScanStore
	ExpireTime time.Duration
}

ScanCache wraps a scan storage object. When calling GetScan, only returns a scan if there was made in the last ExpireTime window

func MakeSimpleCache 

func MakeSimpleCache(expiryTime time.Duration) *ScanCache

MakeSimpleCache creates a cache with a SimpleStore backing it.

func (*ScanCache) GetHostnameScan 

func (c *ScanCache) GetHostnameScan(hostname string) (HostnameResult, error)

GetHostnameScan retrieves the scan from underlying storage if there is one present within the cached time window.

func (*ScanCache) PutHostnameScan 

func (c *ScanCache) PutHostnameScan(hostname string, result HostnameResult) error

PutHostnameScan puts in a scan.

type ScanStore 

type ScanStore interface {
	GetHostnameScan(string) (HostnameResult, error)
	PutHostnameScan(string, HostnameResult) error
}

ScanStore is an interface for using and retrieving scan results.

type SimpleStore 

type SimpleStore struct {
	// contains filtered or unexported fields
}

SimpleStore is simple HostnameResult storage backed by map.

func (*SimpleStore) GetHostnameScan 

func (s *SimpleStore) GetHostnameScan(hostname string) (HostnameResult, error)

GetHostnameScan wraps a map get. Returns error if not present in map.

func (*SimpleStore) PutHostnameScan 

func (s *SimpleStore) PutHostnameScan(hostname string, result HostnameResult) error

PutHostnameScan wraps a map set. Can never return error.

type Status 

type Status int32

Status is an enum encoding the status of the overall check. 

const (
	Success Status = 0
	Warning Status = 1
	Failure Status = 2
	Error   Status = 3
)

Values for Result Status

func SetStatus 

func SetStatus(oldStatus Status, newStatus Status) Status

SetStatus the resulting status of combining old & new. The order of priority for CheckStatus goes: Error > Failure > Warning > Success

Source Files

View all Source files

Directories

Path Synopsis
cmd
starttls-check

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.