Skip to content

OONI Probe CLI v3.21.0
Compare
Choose a tag to compare
@github-actions github-actions released this 27 Mar 12:29
v3.21.0
This tag was signed with the committer’s verified signature. The key has expired.
bassosimone Simone Basso
GPG key ID: 80B691277733D95B
Expired
Learn about vigilant mode.
df7f2fd

This is a stable release. On a high-level, this is what changed since 3.19:

  • ✨ 🐛 🚧 Web Connectivity LTE improvements
  • ✨ 🐛 🚧 Web Connectivity Test Helper improvements
  • 🐛 TorSf experiment: fixed data quality issue
  • ✨ Add the ./script/go.bash script to help development
  • 🚧 OONI Run v2: more flexible Engine<->App interface
  • ✨ Adding Wikimedia DNS-over-HTTPS resolver
  • 🚧 Removing the run experiment
  • 🚧 Improve SummaryKeys management
  • 🚧 Updating dependencies
  • 🚧 Minor Miscellaneous Maintenance
  • 🐛 Minor Miscellaneous Bug Fixes

More in detail:

✨ 🐛 🚧 Web Connectivity LTE improvements

We made sure that Web Connectivity LTE handles internationalized domain names correctly:

We improved Web Connectivity LTE to correctly handle malformed URLs:

We ensured that we correctly handle measurements containing loopback addresses:

  • fix(webconnectivitylte): handle measurements with loopback addrs by @bassosimone in #1462

We fixed a bug where Web Connectivity LTE was not correctly counting bytes sent and received:

We ensured that Web Connectivity LTE correctly handle cases of websites with missing A/AAAA addresses:

  • fix(webconnectivitylte): handle domains w/o A/AAAA records by @bassosimone in #1473

We fixed a bug where Web Connectivity LTE was not including the network events:

We updated Web Connectivity LTE's "classic" analysis engine to generate the extended blocking flags, such that it is now possible to always use the "classic" engine. In practical term, this implied improving the minipipeline package and making sure the "classic" engine, which already generates v0.4-compatible tests keys, could also generate the test keys that were previously generated by the "orig" engine. After this change, we removed the now-unneded "orig" analysis engine.

We added more test Web Connectivity LTE and minipipeline test cases:

We improved Web Connectivity LTE to correctly handle more blocking=null accessible=null cases:

We improved (and modified) the implementation of StreamAllContext and moved it inside the netxlite package:

We implemented support for SummaryKeys in Web Connectivity LTE:

We factored Web Connectivity LTE algorithms into the webconnectivityalgo package and added support for cycling through several DNS-over-UDP resolvers, thus implementing a feature request demanded by users in the SEA region:

We did work to reduce to ~zero the churn caused by regenerating minipipeline test cases, after we realized the non-determinism in generating the test cases was going to cause too many large and ~hard-to-review properly diffs:

We ensured that Web Connectivity LTE correctly includes the client_resolver field:

We ensured that Web Connectivity LTE correctly uses the tcptls_experiment tag when needed:

  • fix(webconnectivitylte): add classic and tcptls_experiment tags by @bassosimone in #1502

We implemented additional, misc changes to improve the codebase

  • refactor(webconnectivitylte): use NewHTTPTransportWithOptions by @bassosimone in #1494
  • doc(webconnectivitylte): clarify http_transaction_{start,end} semantics by @bassosimone in #1495

✨ 🐛 🚧 Web Connectivity Test Helper improvements

We implemented protection against oohelperd overload, which can be caused by synchronized clients:

  • ✨ feat(oohelperd): protect against overload and add metrics by @bassosimone in #1442

We implemented logs and tests to make sure oohelperd does not try connecting to bogons:

  • 🐛 fix(oohelperd): make sure endpoints don't connect to 127.0.0.1 by @bassosimone in #1463

We removed technical debt in the oohelperd, which was making GVisor-based integration tests tricky. Specifically, we ensured we have a single constructor for oohelperd and that such a constructor depends on a *netxlite.Netx. In turn, this allowed us to use the exact same constructor for running oohelperd in production and in integration testing, thus allowing for less fragile integration testing.

  • 🐛 fix(oohelperd,netemx): construct equivalent HTTPTransports by @bassosimone in #1464
  • 🚧 cleanup(netxlite): remove first-order implicit-Netx wrappers by @bassosimone in #1465
  • 🚧 refactor(oohelperd): depend on netxlite.Netx only by @bassosimone in #1466
  • 🚧 refactor(oohelperd,netmx): reduce construction diff to zero by @bassosimone in #1467
  • 🚧 refactor(netemx,oohelperd): use oohelperd.NewHandler constructor by @bassosimone in #1468

We modified how we deploy oohelperd. We're now using a docker build.

  • oohelperd: move prometheus metrics behind HTTP basic auth + docker image build + codepipeline buildspec by @hellais in #1520

🐛 TorSf experiment: fixed data quality issue

We fixed a data quality issue (ooni/probe#2692) that was causing all torsf runs to fail:

✨ Add the ./script/go.bash script to help development

Following a community request, we implemented the ./script/go.bash script to allow anyone to build ooniprobe and other tools implemented by this repository using the correct version of Go. This works as long as the user has go >= 1.15 installed.

🚧 OONI Run v2: more flexible Engine<->App interface

We refactored the interface between the OONI Engine and mobile apps, such that we provide more control over the request sent and the responses received to the mobile apps. In turn, this allows to evolve OONI Run v2 faster.

✨ Adding Wikimedia DNS-over-HTTPS resolver

Since this release, we use Wikimedia's DNS-over-HTTPS resolver as one of the candidates DNS resolvers used when communicating with the OONI backend or when we are performing other control-plane operations (i.e., finding out the probe IP).

🚧 Removing the run experiment

The run experiment has never been documented and had only been used to write some research papers. As such, we felt comfortable with removing this experiment when its implementation was preventing refactoring SummaryKeys.

🚧 Improve SummaryKeys management

We realized that our SummaryKeys implementation was very fragile, so we refactored it to improve robustness:

🚧 Updating dependencies

The following pull requests updated dependencies used by ooniprobe:

🚧 Minor Miscellaneous Maintenance

The following pull requests contain minor, miscellaneous maintenance-related changes:

🐛 Minor Miscellaneous Bug Fixes

The following pull requests contain minor, miscellaneous bug fixes:

Contributors

bassosimone, hellais, and 3 other contributors
Assets 45
Loading