planout

package module
v0.0.0-...-c4d175e Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 8, 2016 License: Apache-2.0 Imports: 10 Imported by: 0

README

Build Status

(Multi Variate Testing) interpreter for PlanOut code written in Golang

What what ?

PlanOut is a framework for providing randomised parameter assignment for controlling parameters and defaults used in code. It exists as a combination of both a generalised methodology and as a DSL for constructing online field experiments.

An excellent introduction can be found both in the original research, Designing and Deploying Online Field Experiments (Bakshy, Eckles and Bernstein), and in the following lecture https://www.youtube.com/watch?v=Ayd4sqPH2DE.

So what is this ?

This is an interpreter that provides the basic functionality for running PlanOut interpreter code, allowing for integrating experiments into GoLang applications.

This is not a full implementation of a complete PlanOut stack, as such it lacks the compiler needed to turn PlanOut DSL into the interpreter code, as well as the general test management tooling needed.

Much of the additional tooling for PlanOut can be found the in original project.

This code will however run PlanOut programs in an idiomatic Golang fashion.

How to run a basic experiment ?

Here's an example program that consumes compiled PlanOut code and executes the associated experiment using the Golang interpreter.

package main

import (
	"encoding/json"
	"fmt"
	"io/ioutil"
	"math/rand"
	"github.com/URXtech/planout-golang"
)

// Helper function to generate random string.
func generateString() string {
	s := make([]byte, 10)
	for j := 0; j < 10; j++ {
		s[j] = 'a' + byte(rand.Int()%26)
	}
	return string(s)
}

func main() {
	// Read PlanOut code from file on disk.
	data, _ := ioutil.ReadFile("test/simple_ops.json")

	// The PlanOut code is expected to use json.
	// This format is the same as the output of
	// the PlanOut compiler webapp
	// http://facebook.github.io/planout/demo/planout-compiler.html
	var js map[string]interface{}
	json.Unmarshal(data, &js)

	// Set the necessary input parameters required to run
	// the experiments. For instance, simple_ops.json expects
	// the value for 'userid' to be set.
	params := make(map[string]interface{})
	params["experiment_salt"] = "expt"
	params["userid"] = generateString()

	// Construct an instance of the Interpreter object.
	// Initialize Salt and set Inputs to params.
	expt := &planout.Interpreter{
		Salt: "global_salt",
		Evaluated:      false,
		Inputs:         params,
		Outputs:        map[string]interface{}{},
		Overrides:      map[string]interface{}{},
        Code: js,
	}
	
	// Call the Run() method on the Interpreter instance.
	// The output of the run will contain the dictionary 
	// of variables and associated values that were evaluated
	// as part of the experiment.
	output, ok := expt.Run()
	if !ok {
		fmt.Println("Failed to run the experiment")
	} else {
		fmt.Printf("Params: %v\n", params)
	}
	
	fmt.Println(output)
}

Suppose we want to run the following experiment:

id = uniformChoice(choices=[1, 2, 3, 4], unit=userid);

The PlanOut code generated by the compiler looks like:

{
  "op": "seq",
  "seq": [
    {
      "op": "set",
      "var": "id",
      "value": {
        "choices": {
          "op": "array",
          "values": [
            1,
            2,
            3,
            4
          ]
        },
        "unit": {
          "op": "get",
          "var": "userid"
        },
        "op": "uniformChoice"
      }
    }
  ]
}

Each execution of the above experiment will result in setting the variable 'id'. The output to stdout will look like:

Params: map[experiment_salt:expt userid:noocavzddw salt:id id:2]
Params: map[experiment_salt:expt userid:cuncjyqmmz salt:id id:1]

How to run a experiments in an allocated namespace ?

This example consumes multiple compiled PlanOut experiments and executes within a namespace.

package main

func main() {
	js1 := readTest("test/simple_ops.json")
	js2 := readTest("test/random_ops.json")
	js3 := readTest("test/simple.json")

	inputs := make(map[string]interface{})
	inputs["userid"] = "test-id"

	n := planout.NewSimpleNamespace("simple_namespace", 100, "userid", inputs)
	n.AddExperiment("simple ops", js1, 10)
	n.AddExperiment("random ops", js2, 10)
	n.AddExperiment("simple", js3, 80)

    out, ok = := n.Run()
}

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func FisherYatesShuffle

func FisherYatesShuffle(inputs []interface{}, nhash ...uint64)

Types

type Interpreter

type Interpreter struct {
	Name                       string
	Salt                       string
	Inputs, Outputs, Overrides map[string]interface{}
	Code                       interface{}
	Evaluated, InExperiment    bool
	ParameterSalt              string
	OperatorOverrides          map[string]Operator
}

func (*Interpreter) Evaluate

func (interpreter *Interpreter) Evaluate(code interface{}) interface{}

func (*Interpreter) Get

func (interpreter *Interpreter) Get(name string) (interface{}, bool)

func (*Interpreter) GetOperator

func (interpreter *Interpreter) GetOperator(expr interface{}) (Operator, bool)

func (*Interpreter) Run

func (interpreter *Interpreter) Run(force ...bool) (map[string]interface{}, bool)

type Namespace

type Namespace interface {
	AddExperiment(name string, code map[string]interface{}, segments int) error
	RemoveExperiment(name string)
}

type Operator

type Operator interface {
	Execute(map[string]interface{}, *Interpreter) interface{}
}

type PlanOutCode

type PlanOutCode interface {
	Run() (map[string]interface{}, bool)
}

type SimpleNamespace

type SimpleNamespace struct {
	Name        string
	PrimaryUnit string
	NumSegments int
	Inputs      map[string]interface{}
	// contains filtered or unexported fields
}

func NewSimpleNamespace

func NewSimpleNamespace(name string, numSegments int, primaryUnit string, inputs map[string]interface{}) SimpleNamespace

func (*SimpleNamespace) AddDefaultExperiment

func (n *SimpleNamespace) AddDefaultExperiment(defaultExperiment *Interpreter)

func (*SimpleNamespace) AddExperiment

func (n *SimpleNamespace) AddExperiment(name string, interpreter *Interpreter, segments int) error

func (*SimpleNamespace) RemoveExperiment

func (n *SimpleNamespace) RemoveExperiment(name string) error

func (*SimpleNamespace) Run

func (n *SimpleNamespace) Run() *Interpreter

type TypedMap

type TypedMap struct {
	// contains filtered or unexported fields
}

func NewTypedMap

func NewTypedMap(data map[string]interface{}) *TypedMap

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL