gocensys

README

gocensys

gocensys is a simple Go package for accessing the Censys API.

Successful API queries return native Go structs that can be used immediately, with no need for type assertions.

gocensys implements endpoints defined in the documentation: https://www.censys.io/api. More detailed information about the behavior of each particular endpoint can be found at the official documentation.

Installation

$ go get -u github.com/abadojack/gocensys

Usage

	import "github.com/abadojack/gocensys"

Authentication

HTTP basic auth is required using the API ID and secret that are shown under My Account.

  api := gocensys.NewCensysAPI("your-uid", "your-secret", nil)

Queries

Executing queries on an authenticated CensysAPI struct is simple.

queryJob, err := api.StartQueryJob("select p110.pop3.ssl_2.certificate.parsed.issuer.province from ipv4.20160708 limit 1000;")
if err != nil {
	log.Fatal(err)
}

fmt.Println(queryJob.JobID)

Licence

gocensys is free software licenced under the GNU LGPL licence. Details are provided in the LICENCE file.

Documentation

Overview

Package gocensys provides structs and functions for accessing version 1 of the Censys API.

Successful API queries return native Go structs that can be used immediately, with no need for type assertions.

Authentication

If you already have the uid and secret for your user, creating the client is simple:

api := gocensys.NewCensysAPI("your-uid", "your-secret", nil)

Queries

Executing queries on an authenticated CensysAPI struct is simple.

series, _ := api.GetSeries()

Endpoints

gocensys implements most of the endpoints defined in the Censys API documentation: https://www.censys.io/api/v1.

More detailed information about the behavior of each particular endpoint can be found at the official Censys API documentation.

Index

• Constants
• type APIError
  • func (apiErr APIError) Error() string
• type CensysAPI
  • func NewCensysAPI(uid, secret string, httpClient *http.Client) *CensysAPI
  • func (c CensysAPI) CreateReport(index string, query map[string]interface{}) (Report, error)
  • func (c CensysAPI) GetExportJobStatus(jobID string) (ExportJobStatus, error)
  • func (c CensysAPI) GetQueryJobResults(jobID, page string) (QueryJobResults, error)
  • func (c CensysAPI) GetQueryJobStatus(jobID string) (QueryJob, error)
  • func (c CensysAPI) GetQuerySeries() (QuerySeries, error)
  • func (c CensysAPI) GetQuerySeriesDetails(series string) (QuerySeriesDetails, error)
  • func (c CensysAPI) GetSeries() (RawData, error)
  • func (c CensysAPI) Search(index string, query map[string]interface{}) (SearchResults, error)
  • func (c CensysAPI) StartExportJob(data map[string]interface{}) (ExportJob, error)
  • func (c CensysAPI) StartQueryJob(query string) (QueryJob, error)
  • func (c CensysAPI) ViewDocument(index, id string) (interface{}, error)
  • func (c CensysAPI) ViewResult(series, result string) (ScanResult, error)
  • func (c CensysAPI) ViewSeries(series string) (RawSeriesDetail, error)
• type ExportJob
• type ExportJobStatus
• type File
• type Files
• type LatestResult
• type PrimarySeries
• type PrimarySeriesDetail
• type QueryJob
• type QueryJobResults
• type QuerySeries
• type QuerySeriesDetails
• type RawData
• type RawSeries
• type RawSeriesDetail
• type Report
• type Result
• type ScanResult
• type SearchResults

Constants

const (
	// BaseURL represents the Censys version 1 API URL.
	BaseURL = "https://www.censys.io/api/v1"
)

Types

type APIError

type APIError struct {
	ErrorCode int
	ErrorStr  string
	Request   *http.Request
}

APIError represents an error response

func (APIError) Error

func (apiErr APIError) Error() string

ApiError supports the error interface

type CensysAPI

type CensysAPI struct {
	// HTTPClient is the HTTP client that will be used in the API requests.
	HTTPClient *http.Client

	// UID represents your API key.
	UID string

	// Secret is required for accessing the API.
	Secret string
}

CensysAPI represents a censys API client used for accessing available endpoints.

func NewCensysAPI

func NewCensysAPI(uid, secret string, httpClient *http.Client) *CensysAPI

NewCensysAPI takes user specific API ID (uid), secret and returns a CensysAPI struct for that user.

func (CensysAPI) CreateReport

func (c CensysAPI) CreateReport(index string, query map[string]interface{}) (Report, error)

CreateReport lets you run aggregate reports on the breakdown of a field in a result set analogous to the "Build Report".

func (CensysAPI) GetExportJobStatus

func (c CensysAPI) GetExportJobStatus(jobID string) (ExportJobStatus, error)

GetExportJobStatus lets you retrieve information about a previously submitted job.

func (CensysAPI) GetQueryJobResults

func (c CensysAPI) GetQueryJobResults(jobID, page string) (QueryJobResults, error)

GetQueryJobResults allows you to retrieve results of a query after it has completed successfully.

func (CensysAPI) GetQueryJobStatus

func (c CensysAPI) GetQueryJobStatus(jobID string) (QueryJob, error)

GetQueryJobStatus allows you to determine whether a job has completed.

func (CensysAPI) GetQuerySeries

func (c CensysAPI) GetQuerySeries() (QuerySeries, error)

GetQuerySeries returns a list of series that can queried through the SQL interface.

func (CensysAPI) GetQuerySeriesDetails

func (c CensysAPI) GetQuerySeriesDetails(series string) (QuerySeriesDetails, error)

GetQuerySeriesDetails returns details about a series, including the list of tables and schema for the series.

func (CensysAPI) GetSeries

func (c CensysAPI) GetSeries() (RawData, error)

GetSeries endpoint returns a data on the types of scans censys regularly performs ("series").

func (CensysAPI) Search

func (c CensysAPI) Search(index string, query map[string]interface{}) (SearchResults, error)

Search endpoint allows searches against the current data in the IPv4, Top Million Websites, and Certificates indexes using the same search syntax as the primary site. The endpoint returns a paginated result set of hosts (or websites or certificates) that match the search.

func (CensysAPI) StartExportJob

func (c CensysAPI) StartExportJob(data map[string]interface{}) (ExportJob, error)

StartExportJob lets you submit an export job.

func (CensysAPI) StartQueryJob

func (c CensysAPI) StartQueryJob(query string) (QueryJob, error)

StartQueryJob lets you to start a new asynchronous query job.

func (CensysAPI) ViewDocument

func (c CensysAPI) ViewDocument(index, id string) (interface{}, error)

ViewDocument used to fetch full document once its ID is known (e.g., IP address or domain).

func (CensysAPI) ViewResult

func (c CensysAPI) ViewResult(series, result string) (ScanResult, error)

ViewResult endpoint returns data on a particular scan ("result"), as found in the Get Series or View Series endpoints.

func (CensysAPI) ViewSeries

func (c CensysAPI) ViewSeries(series string) (RawSeriesDetail, error)

ViewSeries endpoint returns data censys has about a particular series — a scan of the same protocol and destination across time—including the list of scans.

type ExportJob

type ExportJob struct {
	Status        string `json:"status"`
	Configuration struct {
		Format    string `json:"format"`
		Compress  bool   `json:"compress"`
		Headers   bool   `json:"headers"`
		Delimiter bool   `json:"delimiter"`
		Flatten   bool   `json:"flatten"`
		Query     string `json:"query"`
	} `json:"configuration"`
	JobID string `json:"job_id"`
}

ExportJob represents an export job.

type ExportJobStatus

type ExportJobStatus struct {
	Status     string `json:"status"`
	Statistics struct {
		BackendTime interface{} `json:"backend_time"`
		ResultCount interface{} `json:"result_count"`
	} `json:"statistics"`
	JobID         string `json:"job_id"`
	Configuration struct {
		Format    string `json:"format"`
		Compress  bool   `json:"compress"`
		Headers   bool   `json:"headers"`
		Delimiter bool   `json:"delimiter"`
		Flatten   bool   `json:"flatten"`
		Query     string `json:"query"`
	} `json:"configuration"`
	DownloadPaths []string `json:"download_paths"`
}

ExportJobStatus represents the status of a submitted job.

type File

type File struct {
	CompressedSize              int         `json:"compressed_size"`
	CompressedSha256Fingerprint interface{} `json:"compressed_sha256_fingerprint"`
	CompressedDownloadPath      string      `json:"compressed_download_path"`
	Sha256Fingerprint           string      `json:"sha256_fingerprint"`
	DownloadPath                string      `json:"download_path"`
	FileType                    string      `json:"file_type"`
	Schema                      interface{} `json:"schema"`
	CompressionType             string      `json:"compression_type"`
	Size                        int         `json:"size"`
}

File represents details about a downloadable file.

type Files

type Files struct {
	ZtagMetadata     File `json:"ztag-metadata"`
	ZteeZgrabUpdates File `json:"ztee-zgrab-updates"`
	ZgrabMetadata    File `json:"zgrab-metadata"`
	ZgrabResults     File `json:"zgrab-results"`
	ZmapLog          File `json:"zmap-log"`
	ZgrabLog         File `json:"zgrab-log"`
	ZteeZmapUpdates  File `json:"ztee-zmap-updates"`
	ZmapResults      File `json:"zmap-results"`
	ZmapMetadata     File `json:"zmap-metadata"`
	ZtagLog          File `json:"ztag-log"`
}

Files represents the different kinds of files that can be downloaded.

type LatestResult

type LatestResult struct {
	Timestamp  string `json:"timestamp"`
	Name       string `json:"name"`
	DetailsURL string `json:"details_url"`
}

LatestResult represents details the latest scan results.

type PrimarySeries

type PrimarySeries struct {
	AllX509Certificates       PrimarySeriesDetail `json:"All X.509 Certificates"`
	IPv4Snapshots             PrimarySeriesDetail `json:"IPv4 Snapshots"`
	AlexaTop1MillionSnapshots PrimarySeriesDetail `json:"Alexa Top 1 Million Snapshots"`
}

PrimarySeries represents primary series of the scan.

type PrimarySeriesDetail

type PrimarySeriesDetail struct {
	Description  string       `json:"description"`
	DetailsURL   string       `json:"details_url"`
	LatestResult LatestResult `json:"latest_result"`
	ID           string       `json:"id"`
	Name         string       `json:"name"`
}

PrimarySeriesDetail represents detials about the primary series of the scan.

type QueryJob

type QueryJob struct {
	Status        string `json:"status"`
	Configuration struct {
		Query string `json:"query"`
	} `json:"configuration"`
	JobID string `json:"job_id"`
	Error string `json:"error"`
}

QueryJob represents an SQL query job.

type QueryJobResults

type QueryJobResults struct {
	Statistics struct {
		Rows          float64 `json:"rows"`
		Pages         float64 `json:"pages"`
		DataProcessed float64 `json:"data_processed"`
	} `json:"statistics"`
	Configuration struct {
		Query  string `json:"query"`
		Page   int    `json:"page"`
		Schema struct {
			Fields []struct {
				Type string `json:"type"`
				Name string `json:"name"`
				Mode string `json:"mode"`
			} `json:"fields"`
		} `json:"schema"`
	} `json:"configuration"`
	Rows []interface{} `json:"rows"`
}

QueryJobResults represents results of a query after it has completed successfully.

type QuerySeries

type QuerySeries struct {
	Series []string `json:"series"`
}

QuerySeries represents series that can queried through the SQL interface.

type QuerySeriesDetails

type QuerySeriesDetails struct {
	Definition struct {
		Fields []struct {
			Type   string `json:"type"`
			Name   string `json:"name"`
			Mode   string `json:"mode"`
			Fields []struct {
				Type string `json:"type"`
				Name string `json:"name"`
				Mode string `json:"mode"`
			} `json:"fields,omitempty"`
		} `json:"fields"`
	} `json:"definition"`
	Tables []string `json:"tables"`
}

QuerySeriesDetails represents details about a series, including the list of tables and schema for the series.

type RawData

type RawData struct {
	PrimarySeries PrimarySeries `json:"primary_series"`
	RawSeries     RawSeries     `json:"raw_series"`
}

RawData represents raw data that can be downloaded from Censys.

type RawSeries

type RawSeries struct {
	Two2SSHBannerFullIpv4             RawSeriesDetail `json:"22-ssh-banner-full_ipv4"`
	Two5SMTPStarttlsAlexaTop1Mil      RawSeriesDetail `json:"25-smtp-starttls-alexa_top1mil"`
	One10Pop3StarttlsFullIpv4         RawSeriesDetail `json:"110-pop3-starttls-full_ipv4"`
	Four43HTTPSSsl3FullIpv4           RawSeriesDetail `json:"443-https-ssl_3-full_ipv4"`
	Four43HTTPSHeartbleedFullIpv4     RawSeriesDetail `json:"443-https-heartbleed-full_ipv4"`
	One911FoxDeviceIDFullIpv4         RawSeriesDetail `json:"1911-fox-device_id-full_ipv4"`
	Four43HTTPSDheFullIpv4            RawSeriesDetail `json:"443-https-dhe-full_ipv4"`
	ZeroIcmpEchoRequestFullIpv4       RawSeriesDetail `json:"0-icmp-echo_request-full_ipv4"`
	Four65SmtpsSsl2FullIpv4           RawSeriesDetail `json:"465-smtps-ssl_2-full_ipv4"`
	Nine95Pop3SRsaExportFullIpv4      RawSeriesDetail `json:"995-pop3s-rsa_export-full_ipv4"`
	Two0000Dnp3StatusFullIpv4         RawSeriesDetail `json:"20000-dnp3-status-full_ipv4"`
	One02S7SzlFullIpv4                RawSeriesDetail `json:"102-s7-szl-full_ipv4"`
	Four43HTTPSHeartbleedAlexaTop1Mil RawSeriesDetail `json:"443-https-heartbleed-alexa_top1mil"`
	Two5SMTPDheExport1SampleIpv4      RawSeriesDetail `json:"25-smtp-dhe_export-1%_sample_ipv4"`
	Two5SMTPSsl2FullIpv4              RawSeriesDetail `json:"25-smtp-ssl_2-full_ipv4"`
	Four43HTTPSDheAlexaTop1Mil        RawSeriesDetail `json:"443-https-dhe-alexa_top1mil"`
	Eight0HTTPGetFullIpv4             RawSeriesDetail `json:"80-http-get-full_ipv4"`
	Nine93ImapsSsl2FullIpv4           RawSeriesDetail `json:"993-imaps-ssl_2-full_ipv4"`
	Nine93ImapsTLSFullIpv4            RawSeriesDetail `json:"993-imaps-tls-full_ipv4"`
	Two3TelnetBannerFullIpv4          RawSeriesDetail `json:"23-telnet-banner-full_ipv4"`
	Four43HTTPSDheExportAlexaTop1Mil  RawSeriesDetail `json:"443-https-dhe_export-alexa_top1mil"`
	Two5SMTPRsaExportFullIpv4         RawSeriesDetail `json:"25-smtp-rsa_export-full_ipv4"`
	Eight0HTTPOpenProxyFullIpv4       RawSeriesDetail `json:"80-http-open_proxy-full_ipv4"`
	Eight0HTTPGetAlexaTop1Mil         RawSeriesDetail `json:"80-http-get-alexa_top1mil"`
	One43ImapSsl2FullIpv4             RawSeriesDetail `json:"143-imap-ssl_2-full_ipv4"`
	Four43HTTPSSsl2FullIpv4           RawSeriesDetail `json:"443-https-ssl_2-full_ipv4"`
	Nine93ImapsRsaExportFullIpv4      RawSeriesDetail `json:"993-imaps-rsa_export-full_ipv4"`
	One10Pop3Ssl2FullIpv4             RawSeriesDetail `json:"110-pop3-ssl_2-full_ipv4"`
	Four43HTTPSRsaExportAlexaTop1Mil  RawSeriesDetail `json:"443-https-rsa_export-alexa_top1mil"`
	Four43HTTPSTLSFullIpv4            RawSeriesDetail `json:"443-https-tls-full_ipv4"`
	Four7808BacnetDeviceIDFullIpv4    RawSeriesDetail `json:"47808-bacnet-device_id-full_ipv4"`
	One43ImapStarttlsFullIpv4         RawSeriesDetail `json:"143-imap-starttls-full_ipv4"`
	Five3DNSLookupFullIpv4            RawSeriesDetail `json:"53-dns-lookup-full_ipv4"`
	Four43HTTPSRsaExport1SampleIpv4   RawSeriesDetail `json:"443-https-rsa_export-1%_sample_ipv4"`
	Seven547CwmpGetFullIpv4           RawSeriesDetail `json:"7547-cwmp-get-full_ipv4"`
	Two5SMTPStarttlsFullIpv4          RawSeriesDetail `json:"25-smtp-starttls-full_ipv4"`
	Two1FtpBannerFullIpv4             RawSeriesDetail `json:"21-ftp-banner-full_ipv4"`
	Four43HTTPSDheExport1SampleIpv4   RawSeriesDetail `json:"443-https-dhe_export-1%_sample_ipv4"`
	Four65SmtpsTLSFullIpv4            RawSeriesDetail `json:"465-smtps-tls-full_ipv4"`
	Five02ModbusDeviceIDFullIpv4      RawSeriesDetail `json:"502-modbus-device_id-full_ipv4"`
	Nine95Pop3STLSFullIpv4            RawSeriesDetail `json:"995-pop3s-tls-full_ipv4"`
	Four43HTTPSRsaExportFullIpv4      RawSeriesDetail `json:"443-https-rsa_export-full_ipv4"`
	Four43HTTPSSsl3AlexaTop1Mil       RawSeriesDetail `json:"443-https-ssl_3-alexa_top1mil"`
	Four43HTTPSDheExportFullIpv4      RawSeriesDetail `json:"443-https-dhe_export-full_ipv4"`
	Four43HTTPSHeartbleed1SampleIpv4  RawSeriesDetail `json:"443-https-heartbleed-1%_sample_ipv4"`
	Nine93ImapsDheExportFullIpv4      RawSeriesDetail `json:"993-imaps-dhe_export-full_ipv4"`
	Four43HTTPSTLSAlexaTop1Mil        RawSeriesDetail `json:"443-https-tls-alexa_top1mil"`
	Nine95Pop3SSsl2FullIpv4           RawSeriesDetail `json:"995-pop3s-ssl_2-full_ipv4"`
}

RawSeries represents raw series of the scan.

type RawSeriesDetail

type RawSeriesDetail struct {
	Subprotocol  string       `json:"subprotocol"`
	Desription   string       `json:"desription"`
	Protocol     string       `json:"protocol"`
	Name         string       `json:"name"`
	DetailsURL   string       `json:"details_url"`
	LatestResult LatestResult `json:"latest_result"`
	Destination  string       `json:"destination"`
	ID           string       `json:"id"`
	Port         int          `json:"port"`
}

RawSeriesDetail represents details about the raw series of the scan.

type Report

type Report struct {
	Status  string `json:"status"`
	Results []struct {
		Key      string `json:"key"`
		DocCount int    `json:"doc_count"`
	} `json:"results"`
	Metadata struct {
		Count            int    `json:"count"`
		BackendTime      int    `json:"backend_time"`
		NonnullCount     int    `json:"nonnull_count"`
		OtherResultCount int    `json:"other_result_count"`
		Buckets          int    `json:"buckets"`
		ErrorBound       int    `json:"error_bound"`
		Query            string `json:"query"`
	} `json:"metadata"`
}

Report represents a report on the breakdown of a field in the result set of a query.

type Result

type Result struct {
	HistoricalResult []LatestResult `json:"historical"`
	LatestResult     LatestResult   `json:"latest_result"`
}

Result represents the general scan results which include the latest results and historical results.

type ScanResult

type ScanResult struct {
	Files  Files       `json:"files"`
	TaskID interface{} `json:"task_id"`
	Series struct {
		ID   string `json:"id"`
		Name string `json:"name"`
	} `json:"series"`
	PrimaryFile File   `json:"primary_file"`
	Timestamp   string `json:"timestamp"`
	ID          string `json:"id"`
	Metadata    struct {
	} `json:"metadata"`
}

ScanResult represents details about all the scan results.

type SearchResults

type SearchResults struct {
	Status   string        `json:"status"`
	Results  []interface{} `json:"results"`
	Metadata struct {
		Count       int    `json:"count"`
		Query       string `json:"query"`
		BackendTime int    `json:"backend_time"`
		Page        int    `json:"page"`
		Pages       int    `json:"pages"`
	} `json:"metadata"`
}

SearchResults represents the response from a search request.