odam

package module
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 7, 2020 License: Apache-2.0 Imports: 17 Imported by: 0

README

ODaM - Object Detection and Monitoring

GoDoc Sourcegraph Go Report Card GitHub tag

v0.1.0

ODaM is project aimed to do monitoring such as: pedestrian detection and counting, vehicle detection and counting, speed estimation of objects, sending detected objects to gRPC server for detailed analysis.

It's written on Go with a lot of CGO.

Work in progress

Notice that gRPC integration uses 'license_plate_recognition' protobuff (see gRPC server here https://github.com/LdDl/license_plate_recognition)

Table of Contents

About

ODaM is tool for doing monitoring via Darknet's neural network called Yolo V4 (paper: https://arxiv.org/abs/2004.10934).

It's built on top of go-darknet which uses AlexeyAB's fork of Darknet. For doing computer vision stuff and video reading GoCV is used.

Installation

notice: targeted for Linux users (no Windows/OSX instructions currenlty)

Need to enable CUDA (GPU) in every installation step where it's possible.

  1. Install CUDA (we recommend version 10.2)

    wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget http://developer.download.nvidia.com/compute/cuda/10.2/Prod/local_installers/cuda-repo-ubuntu1804-10-2-local-10.2.89-440.33.01_1.0-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1804-10-2-local-10.2.89-440.33.01_1.0-1_amd64.deb
sudo apt-key add /var/cuda-repo-10-2-local-10.2.89-440.33.01/7fa2af80.pub
sudo apt-get update
sudo apt-get -y install cuda
echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:LD_LIBRARY_PATH'  >> ~/.bashrc
source ~/.bashrc

  2. Install cuDNN (we recommend version v7.6.5 (November 18th, 2019), for CUDA 10.2) Go to NVIDIA's site and download *.deb package. After downloading *.deb package install it:

    sudo dpkg -i libcudnn7_7.6.5.32-1+cuda10.2_amd64.deb
sudo dpkg -i libcudnn7-dev_7.6.5.32-1+cuda10.2_amd64.deb
sudo dpkg -i libcudnn7-doc_7.6.5.32-1+cuda10.2_amd64.deb

    Do not forget to check if cuDNN installed properly:

    cp -r /usr/src/cudnn_samples_v7/ $HOME
cd  $HOME/cudnn_samples_v7/mnistCUDNN
make clean && make
./mnistCUDNN
cd -

  3. Install AlexeyAb's fork of Darknet

    git clone https://github.com/AlexeyAB/darknet
cd ./darknet
# Enable GPU acceleration
sed 's/GPU=0/GPU=1/' ./Makefile
# Enable cuDNN
sed 's/CUDNN=0/CUDNN=1/' ./Makefile
# Prepare *.so
sed 's/LIBSO=0/LIBSO=1/' ./Makefile
make
# Copy *.so to /usr/lib + /usr/include (or /usr/local/lib + /usr/local/include)
sudo cp libdarknet.so /usr/lib/libdarknet.so && sudo cp include/darknet.h /usr/include/darknet.h
# sudo cp libdarknet.so /usr/local/lib/libdarknet.so && sudo cp include/darknet.h /usr/local/include/darknet.h

  4. Go bindings for Darknet - instructions link

  5. GoCV - instructions link.

  6. Blob tracking library - instructions link

  7. If you want to use gRPC client-server model: gRPC - instructions link

    You need to implement your gRPC server as following proto-file: https://github.com/LdDl/odam/blob/master/yolo_grpc.proto.

    In case of my needs I need to detect license plates on vehicles and do OCR on server-side: you can take a look on https://github.com/LdDl/license_plate_recognition for gRPC server example

After steps above done:

go get github.com/LdDl/odam
go install github.com/LdDl/odam

Check if executable available

odam -h

and you will see something like this:

Usage of ./odam:
-settings string
        Path to application's settings (default "conf.json")

Usage

notice: targeted for Linux users (no Windows/OSX instructions currenlty)
  • Prepare neural network stuff
    • Download YOLO's weights, configuration file and *.names file. Your way may warry, but here is our script: download_data.sh 
      ./download_data_v3.sh
    • Make sure there is link to *.names file in YOLO's configuration file: 
      [yolo]
mask = 0,1,2
anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326
classes=80
num=9
jitter=.3
ignore_thresh = .7
truth_thresh = 1
random=1
names = coco.names # <<========= here is the link to 'coco.names' file
  • Prepare configuration file for application. Example of file: conf.json. Description of fields:
{
    "video_settings": { # Video input settings
        "source": "rtsp://127.0.0.1:554/h264", # Link to RTSP stream
        "width": 1920, # Width of image in video source
        "height": 1080, # Height of image in video source
        "reduced_width": 640, # Desired width of image (for imshow and MJPEG streaming, also reduces inference time (processing > accuracy) for neural network)
        "reduced_height": 360, # Desired height of image (for imshow and MJPEG streaming, also reduces inference time (processing > accuracy) for neural network)
        "camera_id": "f2abe45e-aad8-40a2-a3b7-0c610c0f3dda" # Unique ID for video source (useful for 'client-server' model)
    },
    "neural_network_settings": { # YOLO neural network settings
        "darknet_cfg": "yolov3.cfg", # Path to configuration.file
        "darknet_weights": "yolov3.weights", # Path to weights wile
        "darknet_classes": "coco.names", # Path to *.names file (labels of objects)
        "conf_threshold": 0.2, # Confidence threshold
        "nms_threshold": 0.4, # NMS threshold (postprocessing)
        "target_classes": ["car", "motorbike", "bus", "train", "truck"] # What classes you want to detect (if you want to use public dataset, but ignore some classes)
    },
    "cuda_settings":{ # CUDA settings, currently useless
        "enable": true # CUDA settings, currently useless
    },
    "mjpeg_settings":{ # MJPEG streaming settings
        "imshow_enable": false, # Do you want to enable imshow() feature (useful for testing purposes)
        "enable": true, # Do you want to enable this feature?
        "port": 35678 # Listening port fo connections
    },
    "grpc_settings": { # gRPC 'client-server' model settings
        "enable": true, # Do you want to enable this feature?
        "server_ip": "localhost", # gRPC server's IP
        "server_port": 50051 # gRPC server's listening port
    },
    "tracker_settings": { # Tracked settings
        "lines_settings":[
            {
                "line_id": 1, # Unique ID for line id (useful for 'client-server' model)
                "begin": [150, 800], # [X1,Y1], start point of line (usually, left side)
                "end": [1600, 800], # [X2,Y2], end point of line (usually, right side)
                "direction": "to_detector", # Direction of line (possible values: 'to_detector' and 'from_detector')
                "detect_classes": ["car", "motorbike", "bus", "train", "truck"], # What classes must be cropped (as detected objects) that were captured by detection line.
                "rgba": [255, 0, 0, 0] # Color of detection line
            }
        ],
        "draw_track_settings": { # Tracker drawing settings (for WOW effect in imshow() or MJPEG streaming)
            "bbox_settings": { # Setting for bounding boxes (detected objects)
                "rgba": [255, 255, 0, 0], # Color of bounding box border
                "thickness": 2 # Thickness as is
            },
            "centroid_settings": { Setting for centroid of bounding boxes
                "rgba": [255, 0, 0, 0], # Color of circle
                "radius": 4, # Radius of circle
                "thickness": 2 # Thickness as is
            },
            "text_settings": { Setting for text above bounding boxes
                "rgba": [0, 255, 0, 0], # Text color
                "scale": 1.2, # Size of text
                "thickness": 2, # Thickness as is
                "font": "hershey_plain"
            }
        }
    },
    "matpprof_settings": { # pprof for GoCV. Useful for debugging
        "enable": true # Do you want to enable this feature?
    }
}
  • Run 
    odam --settings=conf.json

Screenshots

  • gocv.Imshow() output:

  • MJPEG streaming output:

Support

If you have troubles or questions please open an issue. Feel free to make PR's (we do not have contributing guidelines currently, but we will someday)

Roadmap

Please see ROADMAP.md

Dependencies

License

You can check it here

Developers

cpllbstr https://github.com/cpllbstr

LdDl https://github.com/LdDl

Pavel7824 https://github.com/Pavel7824

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func FixRectForOpenCV 

func FixRectForOpenCV(r *image.Rectangle, cols, rows int)

func RegisterSTYoloServer 

func RegisterSTYoloServer(s *grpc.Server, srv STYoloServer)

func Round 

func Round(v float64) int

Round - Round float64 to int

Types

type AppSettings 

type AppSettings struct {
	VideoSettings         VideoSettings         `json:"video_settings"`
	NeuralNetworkSettings NeuralNetworkSettings `json:"neural_network_settings"`
	CudaSettings          CudaSettings          `json:"cuda_settings"`
	MjpegSettings         MjpegSettings         `json:"mjpeg_settings"`
	GrpcSettings          GrpcSettings          `json:"grpc_settings"`
	TrackerSettings       TrackerSettings       `json:"tracker_settings"`
	MatPPROFSettings      MatPPROFSettings      `json:"matpprof_settings"`
}

AppSettings Settings for application

func NewSettings 

func NewSettings(fname string) (*AppSettings, error)

NewSettings Create new AppSettings from content of configuration file

type BBoxSettings 

type BBoxSettings struct {
	RGBA      [4]uint8 `json:"rgba"`
	Thickness int      `json:"thickness"`
}

BBoxSettings Options for detection rectangle

type CamInfo 

type CamInfo struct {
	CamId                string           `protobuf:"bytes,1,opt,name=cam_id,json=camId,proto3" json:"cam_id,omitempty"`
	Timestamp            int64            `protobuf:"varint,2,opt,name=timestamp,proto3" json:"timestamp,omitempty"`
	Image                []byte           `protobuf:"bytes,3,opt,name=image,proto3" json:"image,omitempty"`
	Detection            *Detection       `protobuf:"bytes,4,opt,name=detection,proto3" json:"detection,omitempty"`
	VirtualLine          *VirtualLineInfo `protobuf:"bytes,5,opt,name=virtual_line,json=virtualLine,proto3" json:"virtual_line,omitempty"`
	XXX_NoUnkeyedLiteral struct{}         `json:"-"`
	XXX_unrecognized     []byte           `json:"-"`
	XXX_sizecache        int32            `json:"-"`
}

func (*CamInfo) Descriptor 

func (*CamInfo) Descriptor() ([]byte, []int)

func (*CamInfo) GetCamId 

func (m *CamInfo) GetCamId() string

func (*CamInfo) GetDetection 

func (m *CamInfo) GetDetection() *Detection

func (*CamInfo) GetImage 

func (m *CamInfo) GetImage() []byte

func (*CamInfo) GetTimestamp 

func (m *CamInfo) GetTimestamp() int64

func (*CamInfo) GetVirtualLine added in v0.2.0 

func (m *CamInfo) GetVirtualLine() *VirtualLineInfo

func (*CamInfo) ProtoMessage 

func (*CamInfo) ProtoMessage()

func (*CamInfo) Reset 

func (m *CamInfo) Reset()

func (*CamInfo) String 

func (m *CamInfo) String() string

func (*CamInfo) XXX_DiscardUnknown 

func (m *CamInfo) XXX_DiscardUnknown()

func (*CamInfo) XXX_Marshal 

func (m *CamInfo) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*CamInfo) XXX_Merge 

func (m *CamInfo) XXX_Merge(src proto.Message)

func (*CamInfo) XXX_Size 

func (m *CamInfo) XXX_Size() int

func (*CamInfo) XXX_Unmarshal 

func (m *CamInfo) XXX_Unmarshal(b []byte) error

type CentroidSettings 

type CentroidSettings struct {
	RGBA      [4]uint8 `json:"rgba"`
	Radius    int      `json:"radius"`
	Thickness int      `json:"thickness"`
}

CentroidSettings Options for center of detection rectangle

type CudaSettings 

type CudaSettings struct {
	Enable bool `json:"enable"`
}

CudaSettings CUDA settings

type DetectedObject 

type DetectedObject struct {
	Rect       image.Rectangle
	Classname  string
	Confidence float32
}

DetectedObject Store detected object info

type DetectedObjects 

type DetectedObjects []*DetectedObject

DetectedObjects Just alias to slice of DetectedObject

type Detection 

type Detection struct {
	XLeft                int32    `protobuf:"varint,1,opt,name=x_left,json=xLeft,proto3" json:"x_left,omitempty"`
	YTop                 int32    `protobuf:"varint,2,opt,name=y_top,json=yTop,proto3" json:"y_top,omitempty"`
	Height               int32    `protobuf:"varint,3,opt,name=height,proto3" json:"height,omitempty"`
	Width                int32    `protobuf:"varint,4,opt,name=width,proto3" json:"width,omitempty"`
	XXX_NoUnkeyedLiteral struct{} `json:"-"`
	XXX_unrecognized     []byte   `json:"-"`
	XXX_sizecache        int32    `json:"-"`
}

func (*Detection) Descriptor 

func (*Detection) Descriptor() ([]byte, []int)

func (*Detection) GetHeight 

func (m *Detection) GetHeight() int32

func (*Detection) GetWidth 

func (m *Detection) GetWidth() int32

func (*Detection) GetXLeft 

func (m *Detection) GetXLeft() int32

func (*Detection) GetYTop 

func (m *Detection) GetYTop() int32

func (*Detection) ProtoMessage 

func (*Detection) ProtoMessage()

func (*Detection) Reset 

func (m *Detection) Reset()

func (*Detection) String 

func (m *Detection) String() string

func (*Detection) XXX_DiscardUnknown 

func (m *Detection) XXX_DiscardUnknown()

func (*Detection) XXX_Marshal 

func (m *Detection) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Detection) XXX_Merge 

func (m *Detection) XXX_Merge(src proto.Message)

func (*Detection) XXX_Size 

func (m *Detection) XXX_Size() int

func (*Detection) XXX_Unmarshal 

func (m *Detection) XXX_Unmarshal(b []byte) error

type DrawTrackSettings 

type DrawTrackSettings struct {
	// Drawing options for detection rectangle
	BBoxSettings BBoxSettings `json:"bbox_settings"`
	// Drawing options for center of detection rectangle
	CentroidSettings CentroidSettings `json:"centroid_settings"`
	// Drawing options for text in top left corner of detection rectangle
	TextSettings TextSettings `json:"text_settings"`
}

DrawTrackSettings Drawing settings for MJPEG/imshow

type FrameData 

type FrameData struct {
	ImgSource gocv.Mat //  Source image
	ImgScaled gocv.Mat // Scaled image
	ImgSTD    image.Image
}

FrameData Wrapper around gocv.Mat

func NewFrameData 

func NewFrameData() *FrameData

NewFrameData Simplify creation of FrameData

func (*FrameData) Close 

func (fd *FrameData) Close()

Close Simplify memory management for each gocv.Mat of FrameData

func (*FrameData) Preprocess 

func (fd *FrameData) Preprocess(width, height int) error

type GrpcSettings 

type GrpcSettings struct {
	Enable     bool   `json:"enable"`
	ServerIP   string `json:"server_ip"`
	ServerPort int    `json:"server_port"`
}

GrpcSettings gRPC-server address

type LinesSetting 

type LinesSetting struct {
	LineID        int      `json:"line_id"`
	Begin         [2]int   `json:"begin"`
	End           [2]int   `json:"end"`
	Direction     string   `json:"direction"`
	DetectClasses []string `json:"detect_classes"`
	RGBA          [4]uint8 `json:"rgba"`
	// Exported, but not from JSON
	VLine *VirtualLine `json:"-"`
}

LinesSetting Virtual lines

type MatPPROFSettings 

type MatPPROFSettings struct {
	Enable bool `json:"enable"`
}

MatPPROFSettings pprof settings of gocv.Mat

type MjpegSettings 

type MjpegSettings struct {
	ImshowEnable bool `json:"imshow_enable"`
	Enable       bool `json:"enable"`
	Port         int  `json:"port"`
}

MjpegSettings settings for output

type NeuralNetworkSettings 

type NeuralNetworkSettings struct {
	DarknetCFG     string `json:"darknet_cfg"`
	DarknetWeights string `json:"darknet_weights"`
	// DarknetClasses string   `json:"darknet_classes"`
	ConfThreshold float64  `json:"conf_threshold"`
	NmsThreshold  float64  `json:"nms_threshold"`
	TargetClasses []string `json:"target_classes"`
}

NeuralNetworkSettings Neural network

type Response 

type Response struct {
	Message              string   `protobuf:"bytes,1,opt,name=message,proto3" json:"message,omitempty"`
	Warning              string   `protobuf:"bytes,2,opt,name=warning,proto3" json:"warning,omitempty"`
	Error                string   `protobuf:"bytes,3,opt,name=error,proto3" json:"error,omitempty"`
	XXX_NoUnkeyedLiteral struct{} `json:"-"`
	XXX_unrecognized     []byte   `json:"-"`
	XXX_sizecache        int32    `json:"-"`
}

func (*Response) Descriptor 

func (*Response) Descriptor() ([]byte, []int)

func (*Response) GetError 

func (m *Response) GetError() string

func (*Response) GetMessage 

func (m *Response) GetMessage() string

func (*Response) GetWarning 

func (m *Response) GetWarning() string

func (*Response) ProtoMessage 

func (*Response) ProtoMessage()

func (*Response) Reset 

func (m *Response) Reset()

func (*Response) String 

func (m *Response) String() string

func (*Response) XXX_DiscardUnknown 

func (m *Response) XXX_DiscardUnknown()

func (*Response) XXX_Marshal 

func (m *Response) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Response) XXX_Merge 

func (m *Response) XXX_Merge(src proto.Message)

func (*Response) XXX_Size 

func (m *Response) XXX_Size() int

func (*Response) XXX_Unmarshal 

func (m *Response) XXX_Unmarshal(b []byte) error

type STYoloClient 

type STYoloClient interface {
	SendDetection(ctx context.Context, in *CamInfo, opts ...grpc.CallOption) (*Response, error)
}

STYoloClient is the client API for STYolo service.

For semantics around ctx use and closing/ending streaming RPCs, please refer to https://godoc.org/google.golang.org/grpc#ClientConn.NewStream.

func NewSTYoloClient 

func NewSTYoloClient(cc grpc.ClientConnInterface) STYoloClient

type STYoloServer 

type STYoloServer interface {
	SendDetection(context.Context, *CamInfo) (*Response, error)
}

STYoloServer is the server API for STYolo service.

type TextSettings 

type TextSettings struct {
	RGBA      [4]uint8 `json:"rgba"`
	Scale     float64  `json:"scale"`
	Thickness int      `json:"thickness"`
	Font      string   `json:"font"` // Possible values are: hershey_simplex, hershey_plain, hershey_duplex, hershey_complex, hershey_triplex, hershey_complex_small, hershey_script_simplex, hershey_script_cddomplex, italic
}

TextSettings Options for text in top left corner of detection rectangle

type TrackerSettings 

type TrackerSettings struct {
	LinesSettings     []LinesSetting    `json:"lines_settings"`
	DrawTrackSettings DrawTrackSettings `json:"draw_track_settings"`

	// Exported, but not from JSON
	DrawOptions *blob.DrawOptions `json:"-"`
}

TrackerSettings Object tracker settings

type UnimplementedSTYoloServer 

type UnimplementedSTYoloServer struct {
}

UnimplementedSTYoloServer can be embedded to have forward compatible implementations.

func (*UnimplementedSTYoloServer) SendDetection 

func (*UnimplementedSTYoloServer) SendDetection(ctx context.Context, req *CamInfo) (*Response, error)

type VideoSettings 

type VideoSettings struct {
	Source        string `json:"source"`
	Width         int    `json:"width"`
	Height        int    `json:"height"`
	ReducedWidth  int    `json:"reduced_width"`
	ReducedHeight int    `json:"reduced_height"`
	CameraID      string `json:"camera_id"`

	// Exported, but not from JSON
	ScaleX float64 `json:"-"`
	ScaleY float64 `json:"-"`
}

VideoSettings Settings for video

type VirtualLine 

type VirtualLine struct {
	LeftPT    image.Point
	RightPT   image.Point
	Color     color.RGBA
	Direction bool
}

VirtualLine Detection line attributes

func (*VirtualLine) Draw 

func (vline *VirtualLine) Draw(img *gocv.Mat)

Draw Draw virtual line on image

type VirtualLineInfo added in v0.2.0 

type VirtualLineInfo struct {
	Id                   int32    `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
	LeftX                int32    `protobuf:"varint,2,opt,name=left_x,json=leftX,proto3" json:"left_x,omitempty"`
	LeftY                int32    `protobuf:"varint,3,opt,name=left_y,json=leftY,proto3" json:"left_y,omitempty"`
	RightX               int32    `protobuf:"varint,4,opt,name=right_x,json=rightX,proto3" json:"right_x,omitempty"`
	RightY               int32    `protobuf:"varint,5,opt,name=right_y,json=rightY,proto3" json:"right_y,omitempty"`
	XXX_NoUnkeyedLiteral struct{} `json:"-"`
	XXX_unrecognized     []byte   `json:"-"`
	XXX_sizecache        int32    `json:"-"`
}

func (*VirtualLineInfo) Descriptor added in v0.2.0 

func (*VirtualLineInfo) Descriptor() ([]byte, []int)

func (*VirtualLineInfo) GetId added in v0.2.0 

func (m *VirtualLineInfo) GetId() int32

func (*VirtualLineInfo) GetLeftX added in v0.2.0 

func (m *VirtualLineInfo) GetLeftX() int32

func (*VirtualLineInfo) GetLeftY added in v0.2.0 

func (m *VirtualLineInfo) GetLeftY() int32

func (*VirtualLineInfo) GetRightX added in v0.2.0 

func (m *VirtualLineInfo) GetRightX() int32

func (*VirtualLineInfo) GetRightY added in v0.2.0 

func (m *VirtualLineInfo) GetRightY() int32

func (*VirtualLineInfo) ProtoMessage added in v0.2.0 

func (*VirtualLineInfo) ProtoMessage()

func (*VirtualLineInfo) Reset added in v0.2.0 

func (m *VirtualLineInfo) Reset()

func (*VirtualLineInfo) String added in v0.2.0 

func (m *VirtualLineInfo) String() string

func (*VirtualLineInfo) XXX_DiscardUnknown added in v0.2.0 

func (m *VirtualLineInfo) XXX_DiscardUnknown()

func (*VirtualLineInfo) XXX_Marshal added in v0.2.0 

func (m *VirtualLineInfo) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*VirtualLineInfo) XXX_Merge added in v0.2.0 

func (m *VirtualLineInfo) XXX_Merge(src proto.Message)

func (*VirtualLineInfo) XXX_Size added in v0.2.0 

func (m *VirtualLineInfo) XXX_Size() int

func (*VirtualLineInfo) XXX_Unmarshal added in v0.2.0 

func (m *VirtualLineInfo) XXX_Unmarshal(b []byte) error

Source Files

View all Source files

Directories

Path Synopsis
cmd
odam

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.