log

module
Version: v0.0.0-...-b042743 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 12, 2017 License: MIT

README

DarkMetrix Log

A fast network logging system powered by Apache Kafka.

Architecture

image

Quick start

Preperation

An Apache Kafka need to be installed previously and create a topic.

./kafka-topics.sh --zookeeper localhost:2181 --create --replication-factor 1 --partitions 10 --topic net_log

See DarkMetrix/log/agent or DarkMetrix/log/server to get more information about Kafka setting.

Install from source
git clone git@github.com:DarkMetrix/log.git 
Build agent & run (You will need a root user)
$cd DarkMetrix/log/agent/src
$go build -o ../bin/dm_log_agent
$../admin/start.sh
Build server & run
$cd DarkMetrix/log/server/src
$go build -o ../bin/dm_log_server
$../admin/start.sh

Protocol

We use google/protobuf as the protocol, so you shuold use protobuf whatever programming language you use to marshal the log and send via unix domain socket.

log.proto
syntax = "proto2";

package log_proto;

//Log package
message LogPackage
{
    optional string project = 1;    //Project name(Also used as the folder name)
    optional string service = 2;    //Service name(Also used in the log file name)
    optional uint32 level = 3;      //Log level(INFO = 0, WARNING = 1, ERROR = 2, FATAL = 3)
    optional bytes  log = 4;        //Log content
};

Configuration

All configuration file is in json.

agent

See DarkMetrix/log/agent to get more information.

server

See DarkMetrix/log/server to get more information.

Lisense

DarkMetrix Log

MIT license

Dependencies

Directories

Path Synopsis
agent
src
client
Package log_proto is a generated protocol buffer package.
Package log_proto is a generated protocol buffer package.
server
src

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL