README
iprepd
iprepd is a centralized reputation daemon that can be used to store reputation information for various objects such as IP addresses and retrieve reputation scores for the objects.
The project initially focused on managing reputation information for only IP addresses, but has since been expanded to allow reputation tracking for other types of values such as account names or email addresses.
The daemon provides an HTTP API for requests, and uses a Redis server as the backend storage mechanism. Multiple instances of the daemon can be deployed using the same Redis backend.
Note: Support for legacy endpoints, such as GET /127.0.0.1
has been discontinued. Typed
endpoints must now be used. For more information see the API documentation below. Some minor
modification to client code is required to convert from the old endpoints.
Configuration
Configuration is done through the configuration file, by default ./iprepd.yaml
. The location
can be overridden with the -c
flag.
See iprepd.yaml.sample for an example configuration.
Building the Docker image
./write_version_json.sh
docker build -t iprepd:latest .
Docker images are also published.
docker pull mozilla/iprepd:latest
docker run -ti --rm -v `pwd`/iprepd.yaml:/app/iprepd.yaml mozilla/iprepd:latest
API
Authentication
iprepd supports two forms of authentication. Clients can authenticate to the service using either standard API keys, or by using Hawk authentication.
Standard API key authentication can be configured in the configuration file in the apikey
(for
read/write) and ROapikey
(for a read-only user) sections under the auth
section in the
configuration file. To use API key authentication, clients should send an Authorization
header
that is of format APIKey <apikey>
.
Similarly, Hawk authentication is configured under the hawk
and ROhawk
sections in the
configuration file. To use Hawk authentication clients need to include the hawk authentication
header in the Authorization
header when making a request.
Endpoints
GET /type/ip/10.0.0.1
Request the reputation for an object of a given type. Responds with 200 and a JSON document describing the reputation if found. Responds with a 404 if the object is unknown to iprepd, or is in the exceptions list.
The current supported object types are ip
for an IP address and email
for an
email address.
The response body may include a decayafter
element if the reputation for the address was changed
with a recovery suppression applied. If the timestamp is present, it indicates the time after which
the reputation for the address will begin to recover.
{
"object": "10.0.0.1",
"type": "ip",
"reputation": 75,
"reviewed": false,
"lastupdated": "2018-04-23T18:25:43.511Z"
}
DELETE /type/ip/10.0.0.1
Deletes the reputation entry for the requested object of the specified type.
PUT /type/ip/10.0.0.1
Sets a reputation score for the specified object of the specified type. A reputation JSON
document must be provided with the request body. The reputation
field must be provided
in the document. The reviewed field can be included and set to true to toggle the reviewed
field for a given reputation entry.
Note that if the reputation decays back to 100, if the reviewed field is set on the entry it will toggle back to false.
The reputation will begin to decay back to 100 immediately for the address based on the decay
settings in the configuration file. If it is desired that the reputation should not decay for a
period of time, the decayafter
field can be set with a timestamp to indicate when the reputation
decay logic should begin to be applied for the entry.
{
"object": "10.0.0.1",
"type": "ip",
"reputation": 75
}
PUT /violations/type/ip/10.0.0.1
Applies a violation penalty to the specified object of the specified type.
If an unknown violation penalty is submitted, this endpoint will still return 200, but the error will be logged.
If desired, suppress_recovery
can be included in the request body and set to an integer which
indicates the number of seconds that must elapse before the reputation for this entry will begin
to decay back to 100. If this setting is not included, the reputation will begin to decay
immediately. If the violation is being applied to an existing entry, the suppress_recovery
field
will only be applied if the existing entry has no current recovery suppression, or the specified
recovery suppression time frame would result in a time in the future beyond which the entry
currently has. If suppress_recovery
is included it must be less than 1209600
(14 days).
{
"object": "10.0.0.1",
"type": "ip",
"violation": "violation1"
}
PUT /violations/type/ip
Applies a violation penalty to a multiple objects of a given type.
If an unknown violation penalty is submitted, this endpoint will still return 200, but the error will be logged.
Request body[
{"object": "10.0.0.1", "type": "ip", "violation": "violation1"},
{"object": "10.0.0.2", "type": "ip", "violation": "violation1"},
{"object": "10.0.0.3", "type": "ip", "violation": "violation2"}
]
GET /violations
Returns violations configured in iprepd in a JSON document.
Response body[
{"name": "violation1", "penalty": 5, "decreaselimit": 50},
{"name": "violation2", "penalty": 25, "decreaselimit": 0},
]
GET /dump
Returns all reputation entries.
Note: This makes use of the KEYS redis command, which is known to be very slow. Use with care.
Response body[
{"ip": "10.0.0.1", "reputation": 75, "reviewed": false, "lastupdated": "2018-04-23T18:25:43.511Z"},
{"ip": "10.0.0.2", "reputation": 50, "reviewed": false, "lastupdated": "2018-04-23T18:31:27.457Z"},
{"ip": "10.0.20.2", "reputation": 25, "reviewed": false, "lastupdated": "2018-04-23T17:22:42.230Z"},
]
GET /__heartbeat__
Service heartbeat endpoint.
GET /__lbheartbeat__
Service heartbeat endpoint.
GET /__version__
Return version data.
Acknowledgements
The API design and overall concept for this project are based on work done in Tigerblood.
Documentation
Index ¶
- Constants
- func StartDaemon(confpath string)
- type Client
- func (c *Client) ApplyViolation(vr *ViolationRequest) error
- func (c *Client) BatchApplyViolation(typ string, vrs []ViolationRequest) error
- func (c *Client) DeleteReputation(objectType, object string) error
- func (c *Client) Dump() ([]Reputation, error)
- func (c *Client) GetReputation(objectType, object string) (*Reputation, error)
- func (c *Client) GetViolations() ([]Violation, error)
- func (c *Client) Heartbeat() (bool, error)
- func (c *Client) LBHeartbeat() (bool, error)
- func (c *Client) SetReputation(r *Reputation) error
- func (c *Client) Version() (*VersionResponse, error)
- type Reputation
- type VersionResponse
- type Violation
- type ViolationRequest
Constants ¶
const ( // TypeIP is the object type for IP addresses TypeIP = "ip" // TypeEmail is the object type for email addresses TypeEmail = "email" )
Variables ¶
Functions ¶
func StartDaemon ¶
func StartDaemon(confpath string)
StartDaemon starts a new instance of iprepd using configuration file confpath.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client is the iprepd service client
func (*Client) ApplyViolation ¶
func (c *Client) ApplyViolation(vr *ViolationRequest) error
ApplyViolation submits a ViolationRequest to iprepd
func (*Client) BatchApplyViolation ¶
func (c *Client) BatchApplyViolation(typ string, vrs []ViolationRequest) error
BatchApplyViolation submits a batch of ViolationRequests to iprepd
func (*Client) DeleteReputation ¶
DeleteReputation deletes the reputation of a given object and type
func (*Client) Dump ¶
func (c *Client) Dump() ([]Reputation, error)
Dump retrieves all reputation entries
func (*Client) GetReputation ¶
func (c *Client) GetReputation(objectType, object string) (*Reputation, error)
GetReputation fetches the reputation of a given object and type
func (*Client) GetViolations ¶
GetViolations gets all existing violations on the server
func (*Client) LBHeartbeat ¶
LBHeartbeat checks whether an IPrepd LB is healthy / reachable
func (*Client) SetReputation ¶
func (c *Client) SetReputation(r *Reputation) error
SetReputation updates the reputation of a given object and type to a given score
func (*Client) Version ¶
func (c *Client) Version() (*VersionResponse, error)
Version retrieves the version of the IPrepd deployment
type Reputation ¶
type Reputation struct { // Object is the object associated with the reputation entry. For example // if the type is "ip", object will be an IP address. Object string `json:"object"` // Type describes the type of object the reputation entry is for Type string `json:"type"` // Reputation is the reputation score for the object, ranging from 0 to // 100 where 100 indicates no violations have been applied to it. Reputation int `json:"reputation"` // Reviewed is true if the entry has been manually reviewed, this flag indicates // a firm confidence in the entry. Reviewed bool `json:"reviewed"` // LastUpdated indicates when a reputation was last either set manually or via // a violation on this entry LastUpdated time.Time `json:"lastupdated"` // DecayAfter is used to temporarily stop reputation recovery until after the // current time has passed the time indicated by DecayAfter. This can be used // to for example enforce a mandatory minimum reputation decrease for an object // for a set period of time. DecayAfter time.Time `json:"decayafter,omitempty"` }
Reputation stores information related to the reputation of a given object
func (*Reputation) Validate ¶
func (r *Reputation) Validate() error
Validate performs validation of a Reputation type.
type VersionResponse ¶
type VersionResponse struct { Commit string `json:"commit"` Version string `json:"version"` Source string `json:"source"` Build string `json:"build"` }
VersionResponse is the response payload from the /__version__ endpoint
type Violation ¶
type Violation struct { // Name of violation as specified in iprepd cfg Name string `json:"name"` // Penalty is how many points a reputation will be decreased by if this // violation is submitted for an object Penalty int `json:"penalty"` // DecreaseLimit is the lowest possible value this violation will decrease a // reputation to. Since the same violation can be applied multiple times to // the same object, this can be used to place a lower bound on the total decrease. DecreaseLimit int `json:"decreaselimit"` }
Violation describes a violation penalty that can be applied to an object.
type ViolationRequest ¶
type ViolationRequest struct { // The violation name to be applied Violation string `json:"violation,omitempty"` // The object the violation should be applied to. Object string `json:"object,omitempty"` // The type of object (e.g., ip). Type string `json:"type,omitempty"` // An optional recovery suppression value in seconds. If set, it indicates the // number of seconds which must pass before the reputation for the object will // begin to recover. SuppressRecovery int `json:"suppress_recovery,omitempty"` // The IP field supports reverse compatibility with older clients. It is essentially // the same thing as passing an IP address in the object field, with a type set to // ip. IP string `json:"ip,omitempty"` }
ViolationRequest represents the structure used to apply a violation to a given object. This structure is used as the basis for unmarshaling requests to violation handlers in the API.
func (*ViolationRequest) Fixup ¶
func (v *ViolationRequest) Fixup(typestr string)
Fixup is used to convert legacy format violations
func (*ViolationRequest) Validate ¶
func (v *ViolationRequest) Validate() error
Validate performs validation of a ViolationRequest type
Source Files
Directories
Path | Synopsis |
---|---|
cmd/iprepd |