om

README

om ... Optimized Multileaving

References

1. Schuth, Anne, et al. "Multileaved comparisons for fast online evaluation." Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. ACM, 2014.

2. Manabe, Tomohiro, et al. "A comparative live evaluation of multileaving methods on a commercial cqa search." Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 2017.

Documentation

Index

• type OptimizedMultiLeaving
  • func (o *OptimizedMultiLeaving) CalcInsensitivityAndBias(rks []intergo.Ranking, res []intergo.Res, creditLabel int, alpha float64) (float64, float64)
  • func (o *OptimizedMultiLeaving) GetCredit(rankingIdx int, itemId interface{}, idToPlacements []map[interface{}]int, ...) float64
  • func (o *OptimizedMultiLeaving) GetIdToPlacementMap(rks []intergo.Ranking) []map[interface{}]int
  • func (o *OptimizedMultiLeaving) GetInterleavedRanking(num int, rks ...intergo.Ranking) ([]intergo.Res, error)

Types

type OptimizedMultiLeaving

type OptimizedMultiLeaving struct {
	NumSampling int
	CreditLabel int
	Alpha       float64
}

func (*OptimizedMultiLeaving) CalcInsensitivityAndBias

func (o *OptimizedMultiLeaving) CalcInsensitivityAndBias(rks []intergo.Ranking, res []intergo.Res, creditLabel int, alpha float64) (float64, float64)

func (*OptimizedMultiLeaving) GetCredit

func (o *OptimizedMultiLeaving) GetCredit(rankingIdx int, itemId interface{}, idToPlacements []map[interface{}]int, creditLabel int, isSameRankingIdx bool) float64

func (*OptimizedMultiLeaving) GetIdToPlacementMap

func (o *OptimizedMultiLeaving) GetIdToPlacementMap(rks []intergo.Ranking) []map[interface{}]int

func (*OptimizedMultiLeaving) GetInterleavedRanking

func (o *OptimizedMultiLeaving) GetInterleavedRanking(num int, rks ...intergo.Ranking) ([]intergo.Res, error)

GetInterleavedRanking ... get a Interleaved ranking sampled from a set of interleaved rankings generated by `prefixConstraintSampling` method. Note that the way of the sampling is different from the original paper [Schuth, Anne, et al.,2014] where they solved LP with the unbiased constraint. We omit the unbiased constraint and only take `sensitivity` into account. Then we sample a ranking according to calculated sensitivities defined by equation (1) in [Manabe, Tomohiro, et al., 2017]