xray

README

XRay has been reimplemented in Rust and expanded with more features, this repository is LEGACY code. Use https://github.com/evilsocket/legba

XRAY

XRay is a tool for network OSINT gathering, its goal is to make some of the initial tasks of information gathering and network mapping automatic.

How Does it Work?

XRay is a very simple tool, it works this way:

1. It'll bruteforce subdomains using a wordlist and DNS requests.
2. For every subdomain/ip found, it'll use Shodan to gather open ports and other intel.
3. If a ViewDNS API key is provided, for every subdomain historical data will be collected.
4. For every unique ip address, and for every open port, it'll launch specific banner grabbers and info collectors.
5. Eventually the data is presented to the user on the web ui.

Grabbers and Collectors

• HTTP Server, X-Powered-By and Location headers.
• HTTP and HTTPS robots.txt disallowed entries.
• HTTPS certificates chain ( with recursive subdomain grabbing from CN and Alt Names ).
• HTML title tag.
• DNS version.bind. and hostname.bind. records.
• MySQL, SMTP, FTP, SSH, POP and IRC banners.

Notes

Shodan API Key

The shodan.io API key parameter ( -shodan-key KEY ) is optional, however if not specified, no service fingerprinting will be performed and a lot less information will be shown (basically it just gonna be DNS subdomain enumeration).

ViewDNS API Key

If a ViewDNS API key parameter ( -viewdns-key KEY ) is passed, domain historical data will also be retrieved.

Anonymity and Legal Issues

The software will rely on your main DNS resolver in order to enumerate subdomains, also, several connections might be directly established from your host to the computers of the network you're scanning in order to grab banners from open ports. Technically, you're just connecting to public addresses with open ports (and there's no port scanning involved, as such information is grabbed indirectly using Shodan API), but you know, someone might not like such behaviour.

If I were you, I'd find a way to proxify the whole process ... #justsaying

Building a Docker image

To build a Docker image with the latest version of XRay:

git clone https://github.com/evilsocket/xray.git
cd xray
docker build -t xraydocker .

Once built, XRay can be started within a Docker container using the following:

docker run --rm -it -p 8080:8080 xraydocker xray -address 0.0.0.0 -shodan-key shodan_key_here -domain example.com 

Manual Compilation

Make sure you are using Go >= 1.7, that your installation is working properly, that you have set the $GOPATH variable and you have appended $GOPATH/bin to your $PATH.

Then:

go get github.com/evilsocket/xray
cd $GOPATH/src/github.com/evilsocket/xray/
make

You'll find the executable in the build folder.

Usage

Usage: xray -shodan-key YOUR_SHODAN_API_KEY -domain TARGET_DOMAIN
Options:
  -address string
        IP address to bind the web ui server to. (default "127.0.0.1")
  -consumers int
        Number of concurrent consumers to use for subdomain enumeration. (default 16)
  -domain string
        Base domain to start enumeration from.
  -port int
        TCP port to bind the web ui server to. (default 8080)
  -preserve-domain
        Do not remove subdomain from the provided domain name.
  -session string
        Session file name. (default "<domain-name>-xray-session.json")
  -shodan-key string
        Shodan API key.
  -viewdns-key string
        ViewDNS API key.
  -wordlist string
        Wordlist file to use for enumeration. (default "wordlists/default.lst")

Example:

# xray -shodan-key yadayadayadapicaboo... -viewdns-key foobarsomethingsomething... -domain fbi.gov

____  ___
\   \/  /
 \     RAY v 1.0.0b
 /    by Simone 'evilsocket' Margaritelli
/___/\  \
      \_/

@ Saving session to fbi.gov-xray-session.json
@ Web UI running on http://127.0.0.1:8080/

License

XRay was made with ♥ by Simone Margaritelli and it's released under the GPL 3 license.

The files in the wordlists folder have been taken from various open source tools accross several weeks and I don't remember all of them. If you find the wordlist of your project here and want to be mentioned, feel free to open an issue or send a pull request.

Documentation

Index

• Variables
• func GetSessionFileName(domain string) string
• func LineReader(filename string) (chan string, error)
• func SetupGrabbers(gs []Grabber)
• type CertSH
  • func NewCertSH() *CertSH
  • func (me *CertSH) GetSubDomains(c *Context) []string
• type Context
  • func GetContext() *Context
  • func MakeContext(domain string, session_file string, consumers int, wordlist string, ...) *Context
  • func (c *Context) GetSubDomain(domain string) string
  • func (c *Context) StartGrabbing(t *Target)
• type Grabber
• type HistoryEntry
• type Machine
  • func NewMachine(consumers int, filename string, session *Session, run_handler RunHandler, ...) *Machine
  • func (m *Machine) AddInput(input string)
  • func (m *Machine) Start() error
  • func (m *Machine) UpdateStats()
  • func (m *Machine) Wait()
• type Pool
  • func NewPool(session *Session) *Pool
  • func (p *Pool) Add(t *Target)
  • func (p *Pool) Find(address string) *Target
  • func (p *Pool) FlushSession(stats *Statistics)
  • func (p *Pool) Sorted() []string
  • func (p *Pool) WasRestored() bool
• type Query
• type Response
• type Result
• type ResultHandler
• type RunHandler
• type Session
  • func NewSession(filename string) *Session
  • func (s *Session) Flush(stats *Statistics)
• type Statistics
• type Target
  • func NewTarget(address string, domain string) *Target
  • func (t *Target) AddDomain(domain string) bool
  • func (t *Target) SortedBanners() []string
• type ViewDNS
  • func NewViewDNS(apikey string) *ViewDNS
  • func (d *ViewDNS) GetHistory(domain string) []HistoryEntry

Variables

var (
	Grabbers []Grabber
)

var SessionDefaultFilename = "<domain-name>-xray-session.json"

Functions

func GetSessionFileName

func GetSessionFileName(domain string) string

func LineReader

func LineReader(filename string) (chan string, error)

LineReader will accept the name of a file and offset as argument and will return a channel from which lines can be read one at a time.

func SetupGrabbers

func SetupGrabbers(gs []Grabber)

Types

type CertSH

type CertSH struct {
}

func NewCertSH

func NewCertSH() *CertSH

func (*CertSH) GetSubDomains

func (me *CertSH) GetSubDomains(c *Context) []string

type Context

type Context struct {
	Domain  string
	Bruter  *Machine
	Session *Session
	Pool    *Pool
	Shodan  *shodan.Client
	VDNS    *ViewDNS
	CSH     *CertSH
}

func GetContext

func GetContext() *Context

func MakeContext

func MakeContext(domain string, session_file string, consumers int, wordlist string, shodan_token string, viewdns_token string, run_handler RunHandler, res_handler ResultHandler) *Context

func (*Context) GetSubDomain

func (c *Context) GetSubDomain(domain string) string

func (*Context) StartGrabbing

func (c *Context) StartGrabbing(t *Target)

type Grabber

type Grabber interface {
	Name() string
	Grab(port int, t *Target)
}

type HistoryEntry

type HistoryEntry struct {
	Address  string `json:"ip"`
	Location string `json:"location"`
	ISP      string `json:"owner"`
	Updated  string `json:"lastseen"`
}

type Machine

type Machine struct {
	// Runtime statistics.
	Stats Statistics
	// contains filtered or unexported fields
}

The main object.

func NewMachine

func NewMachine(consumers int, filename string, session *Session, run_handler RunHandler, res_handler ResultHandler) *Machine

Builds a new machine object, if consumers is less or equal than 0, CPU*2 will be used as default value.

func (*Machine) AddInput

func (m *Machine) AddInput(input string)

func (*Machine) Start

func (m *Machine) Start() error

Start the machine.

func (*Machine) UpdateStats

func (m *Machine) UpdateStats()

func (*Machine) Wait

func (m *Machine) Wait()

Wait for all jobs to be completed.

type Pool

type Pool struct {
	sync.RWMutex

	Session *Session
	// contains filtered or unexported fields
}

func NewPool

func NewPool(session *Session) *Pool

func (*Pool) Add

func (p *Pool) Add(t *Target)

func (*Pool) Find

func (p *Pool) Find(address string) *Target

func (*Pool) FlushSession

func (p *Pool) FlushSession(stats *Statistics)

func (*Pool) Sorted

func (p *Pool) Sorted() []string

func (*Pool) WasRestored

func (p *Pool) WasRestored() bool

type Query

type Query struct {
	// contains filtered or unexported fields
}

type Response

type Response struct {
	Records []HistoryEntry `json:"records"`
}

type Result

type Result struct {
	Query    Query    `json:"query"`
	Response Response `json:"response"`
}

type ResultHandler

type ResultHandler func(result interface{})

This is where positive results are handled.

type RunHandler

type RunHandler func(line string) interface{}

This is where the main logic goes.

type Session

type Session struct {
	Stats   *Statistics
	Targets map[string]*Target
	// contains filtered or unexported fields
}

func NewSession

func NewSession(filename string) *Session

func (*Session) Flush

func (s *Session) Flush(stats *Statistics)

type Statistics

type Statistics struct {
	// Time the execution started
	Start time.Time
	// Time the execution finished
	Stop time.Time
	// Total duration of the execution
	Total time.Duration
	// Total number of inputs from the wordlist
	Inputs uint64
	// Executions per second
	Eps float64
	// Total number of executions
	Execs uint64
	// Total number of executions with positive results.
	Results uint64
	// % of progress as: ( execs / inputs ) * 100.0
	Progress float64
}

This structure contains some runtime statistics.

type Target

type Target struct {
	sync.Mutex

	Address   string
	Hostnames []string
	Domains   []string
	Banners   map[string]string
	Info      *shodan.Host
	History   map[string][]HistoryEntry
	// contains filtered or unexported fields
}

func NewTarget

func NewTarget(address string, domain string) *Target

func (*Target) AddDomain

func (t *Target) AddDomain(domain string) bool

func (*Target) SortedBanners

func (t *Target) SortedBanners() []string

type ViewDNS

type ViewDNS struct {
	// contains filtered or unexported fields
}

func NewViewDNS

func NewViewDNS(apikey string) *ViewDNS

func (*ViewDNS) GetHistory

func (d *ViewDNS) GetHistory(domain string) []HistoryEntry