README
¶
OONI Probe Client Library and CLI
The Open Observatory of Network Interference (OONI) is a non-profit free software project that aims to empower decentralized efforts in documenting Internet censorship around the world.
This repository contains core OONI tools written in Go:
-
the CLI client (cmd/ooniprobe);
-
the test helper server (internal/cmd/oohelperd);
-
the mobile library (pkg/oonimkall);
-
the OONI Probe engine (inside internal).
Every top-level directory in this repository contains an explanatory README file. You
may also notice that some internal packages live under internal/engine
while most others are top-level. This is part of a long-standing refactoring started when we merged
https://github.com/ooni/probe-engine into this repository. We'll slowly
ensure that all packages inside engine are moved out of it and inside internal.
Semantic versioning policy
The mobile library is a public package for technical reasons. Go mobile tools require
a public package to build from. Yet, we don't consider API breakages happening in
such a package to be sufficient to bump our major version number. For us, the mobile
library is just a mean to implement OONI Probe Android and OONI Probe iOS. We'll
only bump the major version number if we change ./cmd/ooniprobe's CLI.
License
SPDX-License-Identifier: GPL-3.0-or-later
User setup
Please, follow the instructions at ooni.org/install/cli
to install ooniprobe. If we do not support your use case, please let us know. Once
ooniprobe is installed, try ooniprobe help to get interactive help.
Reporting issues
Report issues at github.com/ooni/probe.
Please, make sure you add the ooni/probe-cli label.
Build instructions
Be sure you have:
-
the golang version mentioned inside the GOVERSION file;
-
a C compiler (Mingw-w64 for Windows).
Caveats
As of 2022-08-22, building with go1.19 will not include Psiphon as a dependency. Fixing this issue is TODO(https://github.com/ooni/probe/issues/2222).
ooniprobe
Ooniprobe is the official CLI client. Compile using:
go build -v -ldflags "-s -w" ./cmd/ooniprobe
This will generate a binary called ooniprobe in the current directory.
miniooni
Miniooni is the experimental OONI client used for research. Compile using:
go build -v -ldflags "-s -w" ./internal/cmd/miniooni
This will generate a binary called miniooni in the current directory.
oohelperd
Oohelperd is the test helper server. Compile using:
go build -v -ldflags "-s -w" ./internal/cmd/oohelperd
This will generate a binary called oohelperd in the current directory.
Specifications
Every nettest (aka experiment) implemented in this repository has a companion spec in ooni/spec.
Contributing
Please, see CONTRIBUTING.md.
Updating dependencies
go get -t -u -v ./... && go mod tidy
Releasing
Create an issue according to the routine release template and perform any item inside the check-list.
We build releases using Makefile, which requires GNU make. Run
make help for detailed usage.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
ooniprobe/internal/autorun
Package autorun contains code to manage automatic runs
|
Package autorun contains code to manage automatic runs |
|
ooniprobe/internal/enginex
Package enginex contains ooni/probe-engine extensions.
|
Package enginex contains ooni/probe-engine extensions. |
|
ooniprobe/internal/log/handlers/cli/progress
Package progress provides a simple terminal progress bar.
|
Package progress provides a simple terminal progress bar. |
|
ooniprobe/internal/log/handlers/syslog
Package syslog contains a syslog handler.
|
Package syslog contains a syslog handler. |
|
ooniprobe/internal/oonitest
Package oonitest contains code used for testing.
|
Package oonitest contains code used for testing. |
|
internal
|
|
|
atomicx
Package atomicx extends sync/atomic.
|
Package atomicx extends sync/atomic. |
|
bytecounter
Package bytecounter contains code to track the number of bytes sent and received by a probe.
|
Package bytecounter contains code to track the number of bytes sent and received by a probe. |
|
cmd/apitool
Command apitool is a simple tool to fetch individual OONI measurements.
|
Command apitool is a simple tool to fetch individual OONI measurements. |
|
cmd/getresources
Command getresources downloads the resources
|
Command getresources downloads the resources |
|
cmd/jafar
Jafar is a censorship simulation tool used for testing OONI.
|
Jafar is a censorship simulation tool used for testing OONI. |
|
cmd/jafar/badproxy
Package badproxy implements misbehaving proxies.
|
Package badproxy implements misbehaving proxies. |
|
cmd/jafar/flagx
Package flagx contains extensions for the standard library flag package.
|
Package flagx contains extensions for the standard library flag package. |
|
cmd/jafar/httpproxy
Package httpproxy contains a censoring HTTP proxy.
|
Package httpproxy contains a censoring HTTP proxy. |
|
cmd/jafar/iptables
Package iptables contains code for managing firewall rules.
|
Package iptables contains code for managing firewall rules. |
|
cmd/jafar/resolver
Package resolver contains a censoring DNS resolver.
|
Package resolver contains a censoring DNS resolver. |
|
cmd/jafar/tlsproxy
Package tlsproxy contains a censoring TLS proxy.
|
Package tlsproxy contains a censoring TLS proxy. |
|
cmd/jafar/uncensored
Package uncensored contains code used by Jafar to evade its own censorship efforts by taking alternate routes.
|
Package uncensored contains code used by Jafar to evade its own censorship efforts by taking alternate routes. |
|
cmd/miniooni
Command miniooni is a simple binary for research and QA purposes with a CLI interface similar to MK and OONI Probe v2.x.
|
Command miniooni is a simple binary for research and QA purposes with a CLI interface similar to MK and OONI Probe v2.x. |
|
cmd/oohelper
Command oohelper contains a simple command line client for the Web Connectivity test helper.
|
Command oohelper contains a simple command line client for the Web Connectivity test helper. |
|
cmd/oohelperd
Command oohelperd implements the Web Connectivity test helper.
|
Command oohelperd implements the Web Connectivity test helper. |
|
cmd/oonireport
Command oonireport uploads reports stored on disk to the OONI collector.
|
Command oonireport uploads reports stored on disk to the OONI collector. |
|
cmd/printversion
Command printversion prints the current version of this repository.
|
Command printversion prints the current version of this repository. |
|
engine
Package engine contains the engine API.
|
Package engine contains the engine API. |
|
engine/experiment/dash
Package dash implements the DASH network experiment.
|
Package dash implements the DASH network experiment. |
|
engine/experiment/dnscheck
Package dnscheck contains the DNS check experiment.
|
Package dnscheck contains the DNS check experiment. |
|
engine/experiment/dnsping
Package dnsping is the experimental dnsping experiment.
|
Package dnsping is the experimental dnsping experiment. |
|
engine/experiment/example
Package example contains a simple example experiment.
|
Package example contains a simple example experiment. |
|
engine/experiment/fbmessenger
Package fbmessenger contains the Facebook Messenger network experiment.
|
Package fbmessenger contains the Facebook Messenger network experiment. |
|
engine/experiment/hhfm
Package hhfm contains the HTTP Header Field Manipulation network experiment.
|
Package hhfm contains the HTTP Header Field Manipulation network experiment. |
|
engine/experiment/hirl
Package hirl contains the HTTP Invalid Request Line network experiment.
|
Package hirl contains the HTTP Invalid Request Line network experiment. |
|
engine/experiment/httphostheader
Package httphostheader contains the HTTP host header network experiment.
|
Package httphostheader contains the HTTP host header network experiment. |
|
engine/experiment/ndt7
Package ndt7 contains the ndt7 network experiment.
|
Package ndt7 contains the ndt7 network experiment. |
|
engine/experiment/psiphon
Package psiphon implements the psiphon network experiment.
|
Package psiphon implements the psiphon network experiment. |
|
engine/experiment/quicping
Package quicping implements the quicping network experiment.
|
Package quicping implements the quicping network experiment. |
|
engine/experiment/riseupvpn
Package riseupvpn contains the RiseupVPN network experiment.
|
Package riseupvpn contains the RiseupVPN network experiment. |
|
engine/experiment/run
Package run contains code to run other experiments.
|
Package run contains code to run other experiments. |
|
engine/experiment/signal
Package signal contains the Signal network experiment.
|
Package signal contains the Signal network experiment. |
|
engine/experiment/simplequicping
Package simplequicping is the experimental simplequicping experiment.
|
Package simplequicping is the experimental simplequicping experiment. |
|
engine/experiment/sniblocking
Package sniblocking contains the SNI blocking network experiment.
|
Package sniblocking contains the SNI blocking network experiment. |
|
engine/experiment/stunreachability
Package stunreachability contains the STUN reachability experiment.
|
Package stunreachability contains the STUN reachability experiment. |
|
engine/experiment/tcpping
Package tcpping is the experimental tcpping experiment.
|
Package tcpping is the experimental tcpping experiment. |
|
engine/experiment/telegram
Package telegram contains the Telegram network experiment.
|
Package telegram contains the Telegram network experiment. |
|
engine/experiment/tlsping
Package tlsping is the experimental tlsping experiment.
|
Package tlsping is the experimental tlsping experiment. |
|
engine/experiment/tlstool
Package tlstool contains a TLS tool that we are currently using for running quick and dirty experiments.
|
Package tlstool contains a TLS tool that we are currently using for running quick and dirty experiments. |
|
engine/experiment/tlstool/internal
Package internal contains the implementation of tlstool.
|
Package internal contains the implementation of tlstool. |
|
engine/experiment/tor
Package tor contains the tor experiment.
|
Package tor contains the tor experiment. |
|
engine/experiment/torsf
Package torsf contains the torsf experiment.
|
Package torsf contains the torsf experiment. |
|
engine/experiment/urlgetter
Package urlgetter implements a nettest that fetches a URL.
|
Package urlgetter implements a nettest that fetches a URL. |
|
engine/experiment/vanillator
Package vanillator contains the vanilla_tor experiment.
|
Package vanillator contains the vanilla_tor experiment. |
|
engine/experiment/webconnectivity
Package webconnectivity implements OONI's Web Connectivity experiment.
|
Package webconnectivity implements OONI's Web Connectivity experiment. |
|
engine/experiment/webconnectivity/internal
Package internal contains internal code.
|
Package internal contains internal code. |
|
engine/experiment/whatsapp
Package whatsapp contains the WhatsApp network experiment.
|
Package whatsapp contains the WhatsApp network experiment. |
|
engine/geolocate
Package geolocate implements IP lookup, resolver lookup, and geolocation.
|
Package geolocate implements IP lookup, resolver lookup, and geolocation. |
|
engine/mockable
Package mockable contains mockable objects
|
Package mockable contains mockable objects |
|
engine/netx
Package netx contains code to perform network measurements.
|
Package netx contains code to perform network measurements. |
|
engine/probeservices
Package probeservices contains code to contact OONI probe services.
|
Package probeservices contains code to contact OONI probe services. |
|
engine/probeservices/testorchestra
Package testorchestra helps with testing the OONI orchestra API.
|
Package testorchestra helps with testing the OONI orchestra API. |
|
engine/sessionresolver
Package sessionresolver contains the resolver used by the session.
|
Package sessionresolver contains the resolver used by the session. |
|
experiment/webconnectivity
Package webconnectivity implements the web_connectivity experiment.
|
Package webconnectivity implements the web_connectivity experiment. |
|
fsx
Package fsx contains io/fs extensions.
|
Package fsx contains io/fs extensions. |
|
geoipx
Package geoipx contains code to use the embedded MaxMind-like databases.
|
Package geoipx contains code to use the embedded MaxMind-like databases. |
|
httpapi
Package httpapi contains code for calling HTTP APIs.
|
Package httpapi contains code for calling HTTP APIs. |
|
httpx
Package httpx contains http extensions.
|
Package httpx contains http extensions. |
|
humanize
Package humanize is like dustin/go-humanize.
|
Package humanize is like dustin/go-humanize. |
|
kvstore
Package kvstore implements model.KeyValueStore.
|
Package kvstore implements model.KeyValueStore. |
|
legacy/assetsdir
Package assetsdir contains code to cleanup the assets dir.
|
Package assetsdir contains code to cleanup the assets dir. |
|
logx
Package logx contains logging extensions
|
Package logx contains logging extensions |
|
measurex
Package measurex contains measurement extensions.
|
Package measurex contains measurement extensions. |
|
measurexlite
Package measurexlite contains measurement extensions.
|
Package measurexlite contains measurement extensions. |
|
mlablocate
Package mlablocate contains a locate.measurementlab.net client implementing v1 of the locate API.
|
Package mlablocate contains a locate.measurementlab.net client implementing v1 of the locate API. |
|
mlablocatev2
Package mlablocatev2 implements m-lab locate services API v2.
|
Package mlablocatev2 implements m-lab locate services API v2. |
|
model
Package model contains the shared interfaces and data structures.
|
Package model contains the shared interfaces and data structures. |
|
model/mocks
Package mocks contains mocks for internal/model interfaces.
|
Package mocks contains mocks for internal/model interfaces. |
|
multierror
Package multierror contains code to manage multiple errors.
|
Package multierror contains code to manage multiple errors. |
|
netxlite
Package netxlite contains network extensions.
|
Package netxlite contains network extensions. |
|
netxlite/filtering
Package filtering allows to implement self-censorship.
|
Package filtering allows to implement self-censorship. |
|
netxlite/quictesting
Package quictesting contains code useful to test QUIC.
|
Package quictesting contains code useful to test QUIC. |
|
oonirun
Package oonirun contains code to run OONI experiments.
|
Package oonirun contains code to run OONI experiments. |
|
platform
Package platform allows you to obtain the platform name.
|
Package platform allows you to obtain the platform name. |
|
ptx
Package ptx contains code to use pluggable transports.
|
Package ptx contains code to use pluggable transports. |
|
randx
Package randx contains math/rand extensions.
|
Package randx contains math/rand extensions. |
|
registry
Package registry contains a registry of all the available experiments.
|
Package registry contains a registry of all the available experiments. |
|
runtimex
Package runtimex contains runtime extensions.
|
Package runtimex contains runtime extensions. |
|
shellx
Package shellx runs external commands.
|
Package shellx runs external commands. |
|
stuninput
Package stuninput contains stun targets as well as code to format such targets according to various conventions.
|
Package stuninput contains stun targets as well as code to format such targets according to various conventions. |
|
testingx
Package testingx contains code useful for testing.
|
Package testingx contains code useful for testing. |
|
torlogs
Package torlogs contains code to read tor logs.
|
Package torlogs contains code to read tor logs. |
|
tracex
Package tracex performs measurements using tracing.
|
Package tracex performs measurements using tracing. |
|
tunnel
Package tunnel allows to create tunnels to speak with OONI backends and other services.
|
Package tunnel allows to create tunnels to speak with OONI backends and other services. |
|
tunnel/mocks
Package mocks contains mocks for tunnel.
|
Package mocks contains mocks for tunnel. |
|
tutorial/experiment/torsf/chapter01
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/experiment/torsf/chapter02
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/generator
Command generator generates or re-generates the tutorial chapters.
|
Command generator generates or re-generates the tutorial chapters. |
|
tutorial/measurex/chapter01
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/measurex/chapter02
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/measurex/chapter03
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/measurex/chapter04
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/measurex/chapter05
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/measurex/chapter06
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/measurex/chapter07
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/measurex/chapter08
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/measurex/chapter09
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/measurex/chapter10
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/measurex/chapter11
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/measurex/chapter12
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/measurex/chapter13
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/measurex/chapter14
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/netxlite/chapter01
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/netxlite/chapter02
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/netxlite/chapter03
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/netxlite/chapter04
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/netxlite/chapter05
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/netxlite/chapter06
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/netxlite/chapter07
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
tutorial/netxlite/chapter08
-=-=- StartHere -=-=-
|
-=-=- StartHere -=-=- |
|
version
Package version contains version information
|
Package version contains version information |
|
pkg
|
|
|
oonimkall
Package oonimkall implements APIs used by OONI mobile apps.
|
Package oonimkall implements APIs used by OONI mobile apps. |
Click to show internal directories.
Click to hide internal directories.