httpx

module

v1.1.0 Latest Latest Go to latest Published: Jun 24, 2021 License: MIT

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/projectdiscovery/httpx

Links

Open Source Insights

README ¶

Features • Installation • Usage • Running httpx • Notes • Join Discord

httpx is a fast and multi-purpose HTTP toolkit allow to run multiple probers using retryablehttp library, it is designed to maintain the result reliability with increased threads.

Features

Simple and modular code base making it easy to contribute.
Fast And fully configurable flags to probe mutiple elements.
Supports multiple HTTP based probings.
Smart auto fallback from https to http as default.
Supports hosts, URLs and CIDR as input.
Handles edge cases doing retries, backoffs etc for handling WAFs.

Supported probes:-

Probes	Default check	Probes	Default check
URL	true	IP	true
Title	true	CNAME	true
Status Code	true	Raw HTTP	false
Content Length	true	HTTP2	false
TLS Certificate	true	HTTP 1.1 Pipeline	false
CSP Header	true	Virtual host	false
Location Header	true	CDN	false
Web Server	true	Path	false
Web Socket	true	Ports	false
Response Time	true	Request method	false

Installation Instructions

httpx requires go1.14+ to install successfully. Run the following command to get the repo -

GO111MODULE=on go get -v github.com/projectdiscovery/httpx/cmd/httpx

Usage

httpx -h

This will display help for the tool. Here are all the switches it supports.

👉 httpx help menu 👈

Usage of ./httpx:

  -H value
      Custom Header
  -allow value
      Allowlist ip/cidr
  -body string
      Request Body
  -cdn
      Check if domain's ip belongs to known CDN (akamai, cloudflare, ..)
  -cname
      Output first cname
  -content-length
      Extracts content length
  -content-type
      Extracts content-type
  -csp-probe
      Send HTTP probes on the extracted CSP domains
  -debug
      Debug mode
  -deny value
      Denylist ip/cidr
  -extract-regex string
      Extract Regex
  -fc string
      Filter status code
  -filter-regex string
      Filter Regex
  -filter-string string
      Filter String
  -fl string
      Filter content length
  -follow-host-redirects
      Only follow redirects on the same host
  -follow-redirects
      Follow Redirects
  -http-proxy string
      HTTP Proxy, eg http://127.0.0.1:8080
  -http2
      HTTP2 probe
  -include-chain
      Show Raw HTTP Chain In Output (-json only)
  -include-response
      Show Raw HTTP Response In Output (-json only)
  -ip
      Output target ip
  -json
      JSON Output
  -l string
      File containing domains
  -location
      Extracts location header
  -match-regex string
      Match Regex
  -match-string string
      Match string
  -max-response-body-size int
      Maximum response body size (default 2147483647)
  -mc string
      Match status code
  -method
      Output method
  -ml string
      Match content length
  -no-color
      No Color
  -no-fallback
      If HTTPS on port 443 is successful on default configuration, probes also port 80 for HTTP
  -o string
      File to write output to (optional)
  -path string
      Request path/file (example '/api')
  -paths string
      Command separated paths or file containing one path per line (example '/api/v1,/apiv2')
  -pipeline
      HTTP1.1 Pipeline
  -ports value
      ports range (nmap syntax: eg 1,2-10,11)
  -random-agent
      Use randomly selected HTTP User-Agent header value
  -request string
      File containing raw request
  -response-in-json
      Show Raw HTTP Response In Output (-json only) (deprecated)
  -response-time
      Output the response time
  -retries int
      Number of retries
  -silent
      Silent mode
  -sr
      Save response to file (default 'output')
  -srd string
      Save response directory (default "output")
  -stats
      Enable statistic on keypress (terminal may become unresponsive till the end)
  -status-code
      Extracts status code
  -store-chain
      Save chain to file (default 'output')
  -tech-detect
      Perform wappalyzer based technology detection
  -threads int
      Number of threads (default 50)
  -timeout int
      Timeout in seconds (default 5)
  -title
      Extracts title
  -tls-grab
      Perform TLS data grabbing
  -tls-probe
      Send HTTP probes on the extracted TLS domains
  -unsafe
      Send raw requests skipping golang normalization
  -verbose
      Verbose Mode
  -version
      Show version of httpx
  -vhost
      Check for VHOSTs
  -vhost-input
      Get a list of vhosts as input
  -web-server
      Extracts server header
  -websocket
      Prints out if the server exposes a websocket
  -x string
      Request Methods, use ALL to check all verbs ()

Running httpX

Running httpx with stdin

This will run the tool against all the hosts and subdomains in hosts.txt and returns URLs running HTTP webserver.

▶ cat hosts.txt | httpx 

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   / 
 / / / / /_/ /_/ /_/ /   |  
/_/ /_/\__/\__/ .___/_/|_|   v1.0  
             /_/            

    projectdiscovery.io

[WRN] Use with caution. You are responsible for your actions
[WRN] Developers assume no liability and are not responsible for any misuse or damage.

https://mta-sts.managed.hackerone.com
https://mta-sts.hackerone.com
https://mta-sts.forwarding.hackerone.com
https://docs.hackerone.com
https://www.hackerone.com
https://resources.hackerone.com
https://api.hackerone.com
https://support.hackerone.com

Running httpx with file input

This will run the tool against all the hosts and subdomains in hosts.txt and returns URLs running HTTP webserver.

▶ httpx -l hosts.txt -silent

https://docs.hackerone.com
https://mta-sts.hackerone.com
https://mta-sts.managed.hackerone.com
https://mta-sts.forwarding.hackerone.com
https://www.hackerone.com
https://resources.hackerone.com
https://api.hackerone.com
https://support.hackerone.com

Running httpx with CIDR input

▶ echo 173.0.84.0/24 | httpx -silent

https://173.0.84.29
https://173.0.84.43
https://173.0.84.31
https://173.0.84.44
https://173.0.84.12
https://173.0.84.4
https://173.0.84.36
https://173.0.84.45
https://173.0.84.14
https://173.0.84.25
https://173.0.84.46
https://173.0.84.24
https://173.0.84.32
https://173.0.84.9
https://173.0.84.13
https://173.0.84.6
https://173.0.84.16
https://173.0.84.34

Running httpx with subfinder

▶ subfinder -d hackerone.com | httpx -title -tech-detect -status-code -follow-redirects

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   /
 / / / / /_/ /_/ /_/ /   |
/_/ /_/\__/\__/ .___/_/|_|
             /_/              v1.0.6

    projectdiscovery.io

Use with caution. You are responsible for your actions
Developers assume no liability and are not responsible for any misuse or damage.
https://mta-sts.managed.hackerone.com [404] [Page not found · GitHub Pages] [Varnish,GitHub Pages,Ruby on Rails]
https://mta-sts.hackerone.com [404] [Page not found · GitHub Pages] [Varnish,GitHub Pages,Ruby on Rails]
https://mta-sts.forwarding.hackerone.com [404] [Page not found · GitHub Pages] [GitHub Pages,Ruby on Rails,Varnish]
https://docs.hackerone.com [200] [HackerOne Platform Documentation] [Ruby on Rails,jsDelivr,Gatsby,React,webpack,Varnish,GitHub Pages]
https://support.hackerone.com [301,302,301,200] [HackerOne] [Cloudflare,Ruby on Rails,Ruby]
https://resources.hackerone.com [301,301,404] [Sorry, no Folders found.]

📋 Notes

As default, httpx checks for HTTPS probe and fall-back to HTTP only if HTTPS is not reachable.
For printing both HTTP/HTTPS results, no-fallback flag can be used.
Custom scheme for ports can be defined, for example -ports http:443,http:80,https:8443
vhost, http2, pipeline, ports, csp-probe, tls-probe and path are unique flag with different probes.
Unique flags should be used for specific use cases instead of running them as default with other flags.
When using json flag, all the information (default probes) included in the JSON output.

Thanks

httpx is made with 🖤 by the projectdiscovery team. Community contributions have made the project what it is. See the Thanks.md file for more details. Do also check out these similar awesome projects that may fit in your workflow:

Probing feature is inspired by @tomnomnom/httprobe work ❤

Directories ¶

Path	Synopsis
cmd
httpx
common
customheader Package customheader contains all the funcionality to deal with Custom Global Headers	Package customheader contains all the funcionality to deal with Custom Global Headers
customlist Package customlist contains all the funcionality to deal with Custom Target List	Package customlist contains all the funcionality to deal with Custom Target List
customports Package customport contains all the funcionality to deal with HTTP ports	Package customport contains all the funcionality to deal with HTTP ports
fileutil Package fileutil contains all the funcionality related to deal with files	Package fileutil contains all the funcionality related to deal with files
httputilz Package httputilz contains all the funcionality related to common HTTP operations, dump, define methods...	Package httputilz contains all the funcionality related to common HTTP operations, dump, define methods...
httpx Package httpx containst the httpx common funcionality	Package httpx containst the httpx common funcionality
slice Package slice contains a set of utilities to deal with slices	Package slice contains a set of utilities to deal with slices
stringz Package stringz contains a set of utilities to deal with strings	Package stringz contains a set of utilities to deal with strings
runner Package runner executes the enumeration process.	Package runner executes the enumeration process.

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL