godge

README

Godge

Godge is a self-hosted online judge for meetups and workshops. It adds a competitive environment to the workshop/meetup.

Why I built Godge?

Read the blog post : http://blog.mbassem.com/2017/03/12/godge/.

What's the difference between godge and other online judges?

1- The obvious difference is that godge is self hosted. It's not intended to be running for public internet audience. It targets meetup/workshop attendees.

2- With Godge you can test every submission the way you want. Other online judges like (spoj and codeforces) only test that the user's submission printed the correct output. When you are teaching Go for instance, you want to test that attendees can create command line flags, that they can start a webserver and respond 200 to your requests, etc. Godge allows you do so.

3- Other online judges only allow you to submit a single file. Godge allows you test the project as a whole, with its subpackages, config files, assets, etc.

How to use it?

As a meetup/workshop Host

1- Install docker (> 1.22).

2- Implement the tasks that you want your attendees to solve using the Godge package. An example for two tasks that tests that attendees can print output to stdout and that they can use flags correctly.

package main

import (
	"fmt"
	"log"
	"math/rand"
	"time"

	"github.com/MohamedBassem/godge"
)

func runAndCompareOutput(sub *godge.Submission, args []string, want string) error {
	if err := sub.Executor.Execute(args); err != nil {
		return err
	}
	defer sub.Executor.Stop()
	<-sub.Executor.DieEvent()
	got, err := sub.Executor.Stdout()
	if err != nil {
		return err
	}
	if got != want {
		return fmt.Errorf("want: %v, got: %v", want, got)
	}
	return nil
}

var tasks = []godge.Task{
	{
		Name: "HelloWorld",
		Desc: "Your program should print 'Hello World!' to stdout.",
		Tests: []godge.Test{
			{
				Name: "PrintsHelloWorld",
				Func: func(sub *godge.Submission) error {
					return runAndCompareOutput(sub, []string{}, "Hello World!")
				},
			},
		},
	},
	{
		Name: "Flags",
		Desc: "The binary should accept the '--name' flag and prints 'Hello $name!' to stdout.",
		Tests: []godge.Test{
			{
				Name: "PrintsHelloJudge",
				Func: func(sub *godge.Submission) error {
					return runAndCompareOutput(sub, []string{"--name", "Judge"}, "Hello Judge!")
				},
			},
			{
				Name: "PrintsHelloUser",
				Func: func(sub *godge.Submission) error {
					return runAndCompareOutput(sub, []string{"--name", sub.Username}, fmt.Sprintf("Hello %v!", sub.Username))
				},
			},
		},
	},
}

func main() {
	rand.Seed(time.Now().UnixNano())
	server, err := godge.NewServer(":8080", "unix:///var/run/docker.sock", "tmp.sqlite")
	if err != nil {
		log.Fatal(err)
	}
	for _, t := range tasks {
		server.RegisterTask(t)
	}
	log.Fatal(server.Start())
}

3- Share with your attendees the address of the server. You can host it on the local network or on a public server.

As an Attendee

1- Download the Godge command line client.

$ go get -u github.com/MohamedBassem/godge/cmd/godge

2- Register a new account on the server.

$ godge --address <addr> register --username <username> --password <password>

3- List the available tasks.

$ godge --address <addr> tasks
HelloWorld: Your program should print 'Hello World!' to stdout.
=============================
Flags: The binary should accept the '--name' flag and prints 'Hello $name!' to stdout.
=============================

4- Work on one of the tasks and then submit it.

$ godge --address <addr> submit --task <task> --language <lang> --username <username> --password <password>
2017/03/12 19:04:58 Will submit /private/tmp/tmp
2017/03/12 19:04:58 Done zipping /private/tmp/tmp
2017/03/12 19:05:00 You submission passed!

5- Check the scoreboard at http://<addr>/scoreboard.

A Live Demo

How It Works

Submissions run in a separate container. The container is determined based on the language. Godge offers an abstract API to interact with the container (start, stop, fetch stdout, ..).

Go

The command line client, zips the whole "main" package (and its subpackages) and sends it to the server. The server then uses the image golang:1.8 to build and run the package.

Known Issues / Future Work

1- Currently Go is the only supported language.

2- All the data (scoreboard, tasks and users) are stored in the server's memory. All the data will be lost if the server is restarted. Although it's not a problem for meetups or workshop as they are short by nature, it would be nice to persist this info in a sqlite database for example.

3- The Execute function should allow opening ports in the container to be able to test web servers for example.

4- Currently a single goroutine executes the submissions sequentially. It would be nice to run multiple submissions in parallel. [Easy Fix]

##Contribution

Your contributions and ideas are welcomed through issues and pull requests.

##License

Copyright (c) 2017, Mohamed Bassem. (MIT License)

See LICENSE for more info.

Documentation

Overview

Package godge is used to build online judges for workshops and meetups. You define the tasks of the workshop along with the tests for each task. Users submit their submission with the command line client which then get judged and displayed on the scoreboard.

An example for testing that the users are able to print to stdout and use flags: https://github.com/MohamedBassem/godge/blob/master/example/example.go

Index

• type ErrorResponse
• type Errors
  • func (e Errors) Error() string
  • func (e Errors) ErrorOrNil() error
• type Executor
• type GoExecutor
  • func (b *GoExecutor) DieEvent() chan struct{}
  • func (g *GoExecutor) Execute(args []string) error
  • func (b *GoExecutor) ReadFileFromContainer(path string) (string, error)
  • func (b *GoExecutor) StartEvent() chan struct{}
  • func (b *GoExecutor) Stderr() (string, error)
  • func (b *GoExecutor) Stdout() (string, error)
  • func (b *GoExecutor) Stop() error
• type RegisterRequest
• type Server
  • func NewServer(address string, dockerAddress string, dbpath string) (*Server, error)
  • func (s *Server) RegisterTask(t Task)
  • func (s *Server) Start() error
• type Submission
  • func (s *Submission) UnmarshalJSON(d []byte) error
• type SubmissionResponse
• type Task
• type Test

Types

type ErrorResponse

type ErrorResponse struct {
	Error string `json:"error"`
}

ErrorResponse represents an error that's returned in the http request in case of a non success code. It's exposed to be used by the command line client.

type Errors

type Errors []error

Errors represents a collection of errors.

func (Errors) Error

func (e Errors) Error() string

Error returns the error string.

func (Errors) ErrorOrNil

func (e Errors) ErrorOrNil() error

ErrorOrNil returns nill if there is no error, otherwise returns the error.

type Executor

type Executor interface {

	// Excutes the submitted code with the provided arguments.
	Execute(args []string) error
	// Reads a certain file from the container's workspace.
	ReadFileFromContainer(path string) (string, error)
	// Returns the contents of the stdout of the container.
	Stdout() (string, error)
	// Returns the contents for the stderr of the container.
	Stderr() (string, error)
	// Stops the running binary.
	Stop() error
	// A channels that gets signaled when the container starts.
	StartEvent() chan struct{}
	// A channels that gets signaled when the container dies.
	DieEvent() chan struct{}
	// contains filtered or unexported methods
}

Executor is used to interact with the submission.

type GoExecutor

type GoExecutor struct {

	// A zip archive containing the "main" package to be executed.
	PackageArchive []byte `json:"packageArchive"`
	// contains filtered or unexported fields
}

GoExecutor implements the Executor interface. It's used in the submit request when the language is Go. You won't deal with the GoExecutor directly, it's only exposed to be used by the command line client.

func (*GoExecutor) DieEvent

func (b *GoExecutor) DieEvent() chan struct{}

DieEvent returns a channel that gets signaled when the container dies.

func (*GoExecutor) Execute

func (g *GoExecutor) Execute(args []string) error

Execute executes the Go main package submitted with the given arguments.

func (*GoExecutor) ReadFileFromContainer

func (b *GoExecutor) ReadFileFromContainer(path string) (string, error)

ReadFileFromContainer reads a certain file from the container's workspace. The path is relative to the container's workdir.

func (*GoExecutor) StartEvent

func (b *GoExecutor) StartEvent() chan struct{}

StartEvent returns a channel that gets signaled when the container starts.

func (*GoExecutor) Stderr

func (b *GoExecutor) Stderr() (string, error)

Stderr returns the content of the stderr of the container.

func (*GoExecutor) Stdout

func (b *GoExecutor) Stdout() (string, error)

Stdout returns the content of the stdout of the container.

func (*GoExecutor) Stop

func (b *GoExecutor) Stop() error

Stop stops the running binary.

type RegisterRequest

type RegisterRequest struct {
	Username string `json:"username"`
	Password string `json:"password"`
}

RegisterRequest represents the registeration request. It's exposed to be used by the command line client.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server holds all the information related to a single instance of the judge. It's used to register Tasks and start the HTTP server.

func NewServer

func NewServer(address string, dockerAddress string, dbpath string) (*Server, error)

NewServer creates a new instance of the judge. It takes the address that the judge will listen to and the address of the address daemon (e.g. unix:///var/run/docker.sock). NewServer returns an error if it fails to connect to the docker daemon or with the sqlite db.

func (*Server) RegisterTask

func (s *Server) RegisterTask(t Task)

RegisterTask registers a new task in the server.

func (*Server) Start

func (s *Server) Start() error

Start starts the http server and the goroutine responsible for processing the submissions.

type Submission

type Submission struct {

	// The language of the submission.
	Language string `json:"language"`
	// The task this submission is sent to.
	TaskName string `json:"taskName"`
	// The username of the submitter.
	Username string `json:"username"`
	// The executor interface to deal with the submission.
	Executor Executor `json:"submission"`
	// contains filtered or unexported fields
}

Submission is the input of the user defined task tests.

func (*Submission) UnmarshalJSON

func (s *Submission) UnmarshalJSON(d []byte) error

UnmarshalJSON is a custom JSON unmarshaller. It's used mainly to create a new executor instance based on the language field of the submission.

type SubmissionResponse

type SubmissionResponse struct {
	Passed bool   `json:"passed"`
	Error  string `json:"error"`
}

SubmissionResponse is the response returned back by the server in response to the submission request. It's exposed to be used by the command line client.

type Task

type Task struct {
	// The name of the task that the user will use to submit their submission.
	Name string `json:"name"`
	// A description of what's required in order to pass the task.
	Desc string `json:"desc"`
	// A group of tests that a submission needs to pass in order to pass the task.
	Tests []Test `json:"-"`
}

Task defines a group of related tests. The user needs to pass all the tests to pass the task and get its point on the scoreboard.

type Test

type Test struct {
	// The name of the test, which will be returned to the user when the test
	// fails.
	Name string
	// The actuall test. It takes a submission as an input (along with its excutor)
	// and should return a descriptive error when the submission don't pass the test.
	Func func(*Submission) error
}

Test defines on of the tests of a certain task.