Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Basic SMTP/IMAP probe: look for bad TLS and StartTLS downgrade attacks #989

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

ooninoob
Copy link
Contributor

@ooninoob ooninoob commented Nov 23, 2022

Checklist

  • I have read the contribution guidelines
  • reference issue for this pull request: probe#1648
  • if you changed anything related how experiments work and you need to reflect these changes in the ooni/spec repository, please link to the related ooni/spec pull request: TODO
  • if you change code inside an experiment, make sure you bump its version number

Description

As part of the InternetBorders hackathon, i am submitting a work-in-progress pull request to add SMTP/IMAP TLS blocking and StartTLS downgrade attacks detection support in miniooni. I was mentored by @hellais during the hackathon, who introduced me to the many flavors of ooni APIs.

I am aware that the contribution guidelines favor small patches, however these two small modules consist of mostly the same code, to the point that i introduced a new helper library for plaintext TCP + StartTLS. Please let me know if i should submit three pull requests instead.

There is still a moderate amount of work to do before merging, and to be honest i'm a little ashamed of the state of this pull request. I would never have submitted yet if not for the hackathon deadline. I am however ready to dedicate at least 3 more days of my time full-time in order to get this pull request merged, given sufficient/guidance.

The biggest blocker to merging that i have no idea how to fix is a futex deadlock in the StartTLS code that i introduced at some point during refactoring. See TODO for more information.

Input

The smtp probe expects a smtp://hostname[:port] input for StartTLS or smtps://hostname[:port] for direct TLS. If no port is provided, ports 587 for StartTLS and 465 for direct TLS are implied.

The imap probe expects a imap://hostname[:port] input for StartTLS or imaps://hostname[:port] for direct TLS. If no port is provided, ports 143 for StartTLS and 993 for direct TLS are implied.

Output

Github says PR body is too long so i will post sample output/report and test results in comments

IMAP has the following data in the report's test keys:

  • hostname: the hostname the probe was run against
  • queries: ooni-compliant DNS data, as specified in df-002-dnst.md
  • runs: a list of individual test runs
  • failure: global failure string

Where an individual test run contains:

  • failure: (failure: a failure aborting this individual run)
  • failed_step: a unique identifier for the code point where the failure was introduced
  • ip: the remote IP address
  • port: the remote port
  • tcp_connect: TCP connect DATA, as specified in df-005-tcpconnect.md
  • tls_handshake: a single TLS handshake DATA, not yet compliant with df-006-tlshandshake.md (see TODO)

The IMAP failed_step can be:

  • imap_wait_capability: failed to read plaintext welcome message from IMAP server
  • imap_wrong_capability: the welcome message from IMAP server does not contain "CAPABILITY" string
  • imap_noop_N: failed to write to the connection to generate NoOp traffic (failed at iteration N)
  • tcp_connect: failed to establish plaintext connection to remote server
  • tls_handshake: failed to establish TLS handshake
  • starttls_wait_ok: failed to read server's response after sending StartTLS message

SMTP has the following data in the report's test keys:

  • hostname: the hostname the probe was run against
  • queries: ooni-compliant DNS data, as specified in df-002-dnst.md
  • runs: a list of individual test runs
  • failure: global failure string

Where an individual test run contains:

  • failure: (failure: a failure aborting this individual run)
  • failed_step: a unique identifier for the code point where the failure was introduced
  • ip: the remote IP address
  • port: the remote port
  • tcp_connect: TCP connect DATA, as specified in df-005-tcpconnect.md
  • tls_handshake: a single TLS handshake DATA, not yet compliant with df-006-tlshandshake.md (see TODO)

The SMTP failed_step can be:

  • smtp_init: failed to start the SMTP client
  • smtp_tls_ehlo: failed to perform a EHLO command against the server over encrypted connection
  • smtp_plaintext_ehlo: failed to perform a EHLO command against the server over plaintext connection
  • smtp_noop_N: failed to write to the connection to generate NoOp traffic (failed at iteration N)
  • tcp_connect: failed to establish plaintext connection to remote server
  • tls_handshake: failed to establish TLS handshake
  • starttls_wait_ok: failed to read server's response after sending StartTLS message

TODO

  • TCPSessionModel.TLSHandshake takes a list of handshakes (and is renamed accordingly) to conform to df-006-tlshandshake.md
  • Write more tests for multiple TLS/StartTLS failure types?
  • Write specifications (when reviewers are happy with the proposed data format)
  • Fix broken StartTLS due to deadlock (more below)

The only hard blocker that may require some active help from a more experience gopher is the broken StartTLS. It seemed to be working fine previously (will submit previous code in a further comment) until i refactored everything. Right now it just deadlocks until timeout 5 minutes later (why 5 minutes and not 1 minute timeout set in run context?). Running under strace reveals some issue related to FUTEX.

I suspect it may have to do with introducing a reader to check for StartTLS ready message, but i have exactly zero experience with golang (or golang networked programming) so i don't have any idea where to start investigating. The issue may be obvious to a more experienced gopher as the code is not so convoluted.

@ooninoob
Copy link
Contributor Author

ooninoob commented Nov 23, 2022

IMAP output

Only direct TLS output is presented here as StartTLS code is currently broken (see TODO in PR description).

Example report:

{
  "annotations": {
    "architecture": "amd64",
    "engine_name": "ooniprobe-engine",
    "engine_version": "3.17.0-alpha",
    "platform": "linux"
  },
  "data_format_version": "0.2.0",
  "input": "imaps://imap.riseup.net",
  "measurement_start_time": "2022-11-23 16:40:54",
  "probe_asn": "AS20766",
  "probe_cc": "FR",
  "probe_ip": "127.0.0.1",
  "probe_network_name": "Association \"Gitoyen\"",
  "report_id": "",
  "resolver_asn": "AS15169",
  "resolver_ip": "172.217.33.194",
  "resolver_network_name": "Google LLC",
  "software_name": "miniooni",
  "software_version": "3.17.0-alpha",
  "test_keys": {
    "hostname": "imap.riseup.net",
    "queries": [
      {
        "answers": [
          {
            "asn": 16652,
            "as_org_name": "Riseup Networks",
            "answer_type": "A",
            "ipv4": "198.252.153.22",
            "ttl": null
          },
          {
            "asn": 16652,
            "as_org_name": "Riseup Networks",
            "answer_type": "A",
            "ipv4": "198.252.153.21",
            "ttl": null
          },
          {
            "answer_type": "CNAME",
            "hostname": "mail.riseup.net",
            "ttl": null
          }
        ],
        "engine": "getaddrinfo",
        "failure": null,
        "hostname": "imap.riseup.net",
        "query_type": "ANY",
        "resolver_hostname": null,
        "resolver_port": null,
        "resolver_address": "",
        "t0": 6.9017e-05,
        "t": 0.392044849
      }
    ],
    "runs": [
      {
        "tcp_connect": [
          {
            "ip": "198.252.153.22",
            "port": 993,
            "status": {
              "failure": null,
              "success": true
            },
            "t0": 0.392142465,
            "t": 0.578600071
          }
        ],
        "tls_handshakes": {
          "network": "tcp",
          "address": "198.252.153.22:993",
          "cipher_suite": "TLS_AES_256_GCM_SHA384",
          "failure": null,
          "negotiated_protocol": "",
          "no_tls_verify": false,
          "peer_certificates": [
            {
              "data": "MIIFejCCBGKgAwIBAgISA/RH55OA6DN+ZSHjmgnopqIzMA0GCSqGSIb3DQEBCwUAMDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQDEwJSMzAeFw0yMjExMjMwMjI4NTBaFw0yMzAyMjEwMjI4NDlaMBoxGDAWBgNVBAMTD21haWwucmlzZXVwLm5ldDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALwsvpcklfIBAZxmRrudIxZ2XghRgzbG5n4Eqx9DWh2jINg7I0zTgoYI0hS3hCtU4zu3kekn0KJ9IMPIdeWufk3QWKPDWkmrQD1tpawuvITk9WgzvDlEEKeLyD8U0uKqJeT93zc+C7XYAwhSJB1sTqykpH3pAexo5B1mphODSOSuz00RAXXQQSG5LZIZ++2PfROahxd9/g2kAJzYA2ugmPSTyMY14kk9YoHbB/2+KC0GG320m9vRiPjediW2Vkdq157rfXObl5u8Vz0yH/7xq/2U7+XGhCkg6LHV46bQQIidEiQ4wJ4GE8+DT8qXeflhKnRYzb4scPFdCx465IxramcCAwEAAaOCAqAwggKcMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQUWTOo8S6HmnYRfwWHLkWk4WWZeVcwHwYDVR0jBBgwFoAUFC6zF7dYVsuuUAlA5h+vnYsUwsYwVQYIKwYBBQUHAQEESTBHMCEGCCsGAQUFBzABhhVodHRwOi8vcjMuby5sZW5jci5vcmcwIgYIKwYBBQUHMAKGFmh0dHA6Ly9yMy5pLmxlbmNyLm9yZy8wcAYDVR0RBGkwZ4IQZmV3czEucmlzZXVwLm5ldIIQZmV3czIucmlzZXVwLm5ldIIPaW1hcC5yaXNldXAubmV0gg9tYWlsLnJpc2V1cC5uZXSCDnBvcC5yaXNldXAubmV0gg9zbXRwLnJpc2V1cC5uZXQwTAYDVR0gBEUwQzAIBgZngQwBAgEwNwYLKwYBBAGC3xMBAQEwKDAmBggrBgEFBQcCARYaaHR0cDovL2Nwcy5sZXRzZW5jcnlwdC5vcmcwggEEBgorBgEEAdZ5AgQCBIH1BIHyAPAAdgB6MoxU2LcttiDqOOBSHumEFnAyE4VNO9IrwTpXo1LrUgAAAYSih/goAAAEAwBHMEUCIQDKfy3g+wuEe1ngugHXtPR4ocXK/lWAiUtQihVoitfc6AIgHARn8FRkaevO0hrtnkleQ6dobI9SXdva5n6QBP/1iAUAdgDoPtDaPvUGNTLnVyi8iWvJA9PL0RFr7Otp4Xd9bQa9bgAAAYSih/oBAAAEAwBHMEUCIQDOzm0w7maB6IwA8jaat0i/+zZaRKvFwb+yENPee+SgsQIgR/1jytWIaUzk9Yp1lbmQKh8GmnY0PIEJWj9WpVlOy74wDQYJKoZIhvcNAQELBQADggEBAHzzszgg2p2oB5hARn6J7mUSrVUuR3H7wAcCPdOrQA3ZnOGjGg0Pq1T0eV5z+NII+ijOZh5dnWjFqpcv7U1DWfZe4gfme96djylB95AmdbcUyb3eS/+jpwSqKlkXBQBZmWhFUyIDi9BNA7UAKNATmlG+sn9vr0H4aQ9+otYy4BDpdrizyNXGhX4wQhUKMT25aYZScNb0sgsjQAXaAVZv9CEz2nN/P6zYuer/DXTZ51wPf4orBTam7xFmAiGhedSSQ59W6jDIlAekBfMC/ZJxYC5mY7eS+x3dcXrtzFgmRfKad4+XZDNt6ZUeJ6ncKuROE7dqJ0srsBY3ktX3OLTdoDI=",
              "format": "base64"
            },
            {
              "data": "MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAwTzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2VhcmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjAwOTA0MDAwMDAwWhcNMjUwOTE1MTYwMDAwWjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3MgRW5jcnlwdDELMAkGA1UEAxMCUjMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC7AhUozPaglNMPEuyNVZLD+ILxmaZ6QoinXSaqtSu5xUyxr45r+XXIo9cPR5QUVTVXjJ6oojkZ9YI8QqlObvU7wy7bjcCwXPNZOOftz2nwWgsbvsCUJCWH+jdxsxPnHKzhm+/b5DtFUkWWqcFTzjTIUu61ru2P3mBw4qVUq7ZtDpelQDRrK9O8ZutmNHz6a4uPVymZ+DAXXbpyb/uBxa3Shlg9F8fnCbvxK/eG3MHacV3URuPMrSXBiLxgZ3Vms/EY96Jc5lP/Ooi2R6X/ExjqmAl3P51T+c8B5fWmcBcUr2Ok/5mzk53cU6cG/kiFHaFpriV1uxPMUgP17VGhi9sVAgMBAAGjggEIMIIBBDAOBgNVHQ8BAf8EBAMCAYYwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYfr52LFMLGMB8GA1UdIwQYMBaAFHm0WeZ7tuXkAXOACIjIGlj26ZtuMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcwAoYWaHR0cDovL3gxLmkubGVuY3Iub3JnLzAnBgNVHR8EIDAeMBygGqAYhhZodHRwOi8veDEuYy5sZW5jci5vcmcvMCIGA1UdIAQbMBkwCAYGZ4EMAQIBMA0GCysGAQQBgt8TAQEBMA0GCSqGSIb3DQEBCwUAA4ICAQCFyk5HPqP3hUSFvNVneLKYY611TR6WPTNlclQtgaDqw+34IL9fzLdwALduO/ZelN7kIJ+m74uyA+eitRY8kc607TkC53wlikfmZW4/RvTZ8M6UK+5UzhK8jCdLuMGYL6KvzXGRSgi3yLgjewQtCPkIVz6D2QQzCkcheAmCJ8MqyJu5zlzyZMjAvnnAT45tRAxekrsu94sQ4egdRCnbWSDtY7kh+BImlJNXoB1lBMEKIq4QDUOXoRgffuDghje1WrG9ML+Hbisq/yFOGwXD9RiX8F6sw6W4avAuvDszue5L3sz85K+EC4Y/wFVDNvZo4TYXao6Z0f+lQKc0t8DQYzk1OXVu8rp2yJMC6alLbBfODALZvYH7n7do1AZls4I9d1P4jnkDrQoxB3UqQ9hVl3LEKQ73xF1OyK5GhDDX8oVfGKF5u+decIsH4YaTw7mP3GFxJSqv3+0lUFJoi5Lc5da149p90IdshCExroL1+7mryIkXPeFM5TgO9r0rvZaBFOvV2z0gp35Z0+L4WPlbuEjN/lxPFin+HlUjr8gRsI3qfJOQFy/9rKIJR0Y/8Omwt/8oTWgy1mdeHmmjk7j1nYsvC9JSQ6ZvMldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqXnLRbwHOoq7hHwg==",
              "format": "base64"
            },
            {
              "data": "MIIFYDCCBEigAwIBAgIQQAF3ITfU6UK47naqPGQKtzANBgkqhkiG9w0BAQsFADA/MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMTDkRTVCBSb290IENBIFgzMB4XDTIxMDEyMDE5MTQwM1oXDTI0MDkzMDE4MTQwM1owTzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2VhcmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCt6CRz9BQ385ueK1coHIe+3LffOJCMbjzmV6B493XCov71am72AE8o295ohmxEk7axY/0UEmu/H9LqMZshftEzPLpI9d1537O4/xLxIZpLwYqGcWlKZmZsj348cL+tKSIG8+TA5oCu4kuPt5l+lAOf00eXfJlII1PoOK5PCm+DLtFJV4yAdLbaL9A4jXsDcCEbdfIwPPqPrt3aY6vrFk/CjhFLfs8L6P+1dy70sntK4EwSJQxwjQMpoOFTJOwT2e4ZvxCzSow/iaNhUd6shweU9GNx7C7ib1uYgeGJXDR5bHbvO5BieebbpJovJsXQEOEO3tkQjhb7t/eo98flAgeYjzYIlefiN5YNNnWe+w5ysR2bvAP5SQXYgd0FtCrWQemsAXaVCg/Y39W9Eh81LygXbNKYwagJZHduRze6zqxZXmidf3LWicUGQSk+WT7dJvUkyRGnWqNMQB9GoZm1pzpRboY7nn1ypxIFeFntPlF4FQsDj43QLwWyPntKHEtzBRL8xurgUBN8Q5N0s8p0544fAQjQMNRbcTa0B7rBMDBcSLeCO5imfWCKoqMpgsy6vYMEG6KDA0Gh1gXxG8K28Kh8hjtGqEgqiNx2mna/H2qlPRmP6zjzZN7IKw0KKP/32+IVQtQi0Cdd4Xn+GOdwiK1O5tmLOsbdJ1Fu/7xk9TNDTwIDAQABo4IBRjCCAUIwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAQYwSwYIKwYBBQUHAQEEPzA9MDsGCCsGAQUFBzAChi9odHRwOi8vYXBwcy5pZGVudHJ1c3QuY29tL3Jvb3RzL2RzdHJvb3RjYXgzLnA3YzAfBgNVHSMEGDAWgBTEp7Gkeyxx+tvhS5B1/8QVYIWJEDBUBgNVHSAETTBLMAgGBmeBDAECATA/BgsrBgEEAYLfEwEBATAwMC4GCCsGAQUFBwIBFiJodHRwOi8vY3BzLnJvb3QteDEubGV0c2VuY3J5cHQub3JnMDwGA1UdHwQ1MDMwMaAvoC2GK2h0dHA6Ly9jcmwuaWRlbnRydXN0LmNvbS9EU1RST09UQ0FYM0NSTC5jcmwwHQYDVR0OBBYEFHm0WeZ7tuXkAXOACIjIGlj26ZtuMA0GCSqGSIb3DQEBCwUAA4IBAQAKcwBslm7/DlLQrt2M51oGrS+o44+/yQoDFVDC5WxCu2+b9LRPwkSICHXM6webFGJueN7sJ7o5XPWioW5WlHAQU7G75K/QosMrAdSW9MUgNTP52GE24HGNtLi1qoJFlcDyqSMo59ahy2cI2qBDLKobkx/J3vWraV0T9VuGWCLKTVXkcGdtwlfFRjlBz4pYg1htmf5X6DYO8A4jqv2Il9DjXA6USbW1FzXSLr9Ohe8Y4IWS6wY7bCkjCWDcRQJMEhg76fsO3txE+FiYruq9RUWhiF1myv4Q6W+CyBFCDfvp7OOGAN6dEOM4+qR9sdjoSYKEBpsr6GtPAQw4dy753ec5",
              "format": "base64"
            }
          ],
          "server_name": "imap.riseup.net",
          "t0": 0.578655355,
          "t": 0.795748072,
          "tags": [],
          "tls_version": "TLSv1.3"
        },
        "failure": "",
        "failed_step": "",
        "ip": "198.252.153.22",
        "port": "993"
      },
      {
        "tcp_connect": [
          {
            "ip": "198.252.153.21",
            "port": 993,
            "status": {
              "failure": null,
              "success": true
            },
            "t0": 1.139660482,
            "t": 1.298815516
          }
        ],
        "tls_handshakes": {
          "network": "tcp",
          "address": "198.252.153.21:993",
          "cipher_suite": "TLS_AES_256_GCM_SHA384",
          "failure": null,
          "negotiated_protocol": "",
          "no_tls_verify": false,
          "peer_certificates": [
            {
              "data": "MIIFejCCBGKgAwIBAgISA/RH55OA6DN+ZSHjmgnopqIzMA0GCSqGSIb3DQEBCwUAMDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQDEwJSMzAeFw0yMjExMjMwMjI4NTBaFw0yMzAyMjEwMjI4NDlaMBoxGDAWBgNVBAMTD21haWwucmlzZXVwLm5ldDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALwsvpcklfIBAZxmRrudIxZ2XghRgzbG5n4Eqx9DWh2jINg7I0zTgoYI0hS3hCtU4zu3kekn0KJ9IMPIdeWufk3QWKPDWkmrQD1tpawuvITk9WgzvDlEEKeLyD8U0uKqJeT93zc+C7XYAwhSJB1sTqykpH3pAexo5B1mphODSOSuz00RAXXQQSG5LZIZ++2PfROahxd9/g2kAJzYA2ugmPSTyMY14kk9YoHbB/2+KC0GG320m9vRiPjediW2Vkdq157rfXObl5u8Vz0yH/7xq/2U7+XGhCkg6LHV46bQQIidEiQ4wJ4GE8+DT8qXeflhKnRYzb4scPFdCx465IxramcCAwEAAaOCAqAwggKcMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQUWTOo8S6HmnYRfwWHLkWk4WWZeVcwHwYDVR0jBBgwFoAUFC6zF7dYVsuuUAlA5h+vnYsUwsYwVQYIKwYBBQUHAQEESTBHMCEGCCsGAQUFBzABhhVodHRwOi8vcjMuby5sZW5jci5vcmcwIgYIKwYBBQUHMAKGFmh0dHA6Ly9yMy5pLmxlbmNyLm9yZy8wcAYDVR0RBGkwZ4IQZmV3czEucmlzZXVwLm5ldIIQZmV3czIucmlzZXVwLm5ldIIPaW1hcC5yaXNldXAubmV0gg9tYWlsLnJpc2V1cC5uZXSCDnBvcC5yaXNldXAubmV0gg9zbXRwLnJpc2V1cC5uZXQwTAYDVR0gBEUwQzAIBgZngQwBAgEwNwYLKwYBBAGC3xMBAQEwKDAmBggrBgEFBQcCARYaaHR0cDovL2Nwcy5sZXRzZW5jcnlwdC5vcmcwggEEBgorBgEEAdZ5AgQCBIH1BIHyAPAAdgB6MoxU2LcttiDqOOBSHumEFnAyE4VNO9IrwTpXo1LrUgAAAYSih/goAAAEAwBHMEUCIQDKfy3g+wuEe1ngugHXtPR4ocXK/lWAiUtQihVoitfc6AIgHARn8FRkaevO0hrtnkleQ6dobI9SXdva5n6QBP/1iAUAdgDoPtDaPvUGNTLnVyi8iWvJA9PL0RFr7Otp4Xd9bQa9bgAAAYSih/oBAAAEAwBHMEUCIQDOzm0w7maB6IwA8jaat0i/+zZaRKvFwb+yENPee+SgsQIgR/1jytWIaUzk9Yp1lbmQKh8GmnY0PIEJWj9WpVlOy74wDQYJKoZIhvcNAQELBQADggEBAHzzszgg2p2oB5hARn6J7mUSrVUuR3H7wAcCPdOrQA3ZnOGjGg0Pq1T0eV5z+NII+ijOZh5dnWjFqpcv7U1DWfZe4gfme96djylB95AmdbcUyb3eS/+jpwSqKlkXBQBZmWhFUyIDi9BNA7UAKNATmlG+sn9vr0H4aQ9+otYy4BDpdrizyNXGhX4wQhUKMT25aYZScNb0sgsjQAXaAVZv9CEz2nN/P6zYuer/DXTZ51wPf4orBTam7xFmAiGhedSSQ59W6jDIlAekBfMC/ZJxYC5mY7eS+x3dcXrtzFgmRfKad4+XZDNt6ZUeJ6ncKuROE7dqJ0srsBY3ktX3OLTdoDI=",
              "format": "base64"
            },
            {
              "data": "MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAwTzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2VhcmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjAwOTA0MDAwMDAwWhcNMjUwOTE1MTYwMDAwWjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3MgRW5jcnlwdDELMAkGA1UEAxMCUjMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC7AhUozPaglNMPEuyNVZLD+ILxmaZ6QoinXSaqtSu5xUyxr45r+XXIo9cPR5QUVTVXjJ6oojkZ9YI8QqlObvU7wy7bjcCwXPNZOOftz2nwWgsbvsCUJCWH+jdxsxPnHKzhm+/b5DtFUkWWqcFTzjTIUu61ru2P3mBw4qVUq7ZtDpelQDRrK9O8ZutmNHz6a4uPVymZ+DAXXbpyb/uBxa3Shlg9F8fnCbvxK/eG3MHacV3URuPMrSXBiLxgZ3Vms/EY96Jc5lP/Ooi2R6X/ExjqmAl3P51T+c8B5fWmcBcUr2Ok/5mzk53cU6cG/kiFHaFpriV1uxPMUgP17VGhi9sVAgMBAAGjggEIMIIBBDAOBgNVHQ8BAf8EBAMCAYYwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYfr52LFMLGMB8GA1UdIwQYMBaAFHm0WeZ7tuXkAXOACIjIGlj26ZtuMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcwAoYWaHR0cDovL3gxLmkubGVuY3Iub3JnLzAnBgNVHR8EIDAeMBygGqAYhhZodHRwOi8veDEuYy5sZW5jci5vcmcvMCIGA1UdIAQbMBkwCAYGZ4EMAQIBMA0GCysGAQQBgt8TAQEBMA0GCSqGSIb3DQEBCwUAA4ICAQCFyk5HPqP3hUSFvNVneLKYY611TR6WPTNlclQtgaDqw+34IL9fzLdwALduO/ZelN7kIJ+m74uyA+eitRY8kc607TkC53wlikfmZW4/RvTZ8M6UK+5UzhK8jCdLuMGYL6KvzXGRSgi3yLgjewQtCPkIVz6D2QQzCkcheAmCJ8MqyJu5zlzyZMjAvnnAT45tRAxekrsu94sQ4egdRCnbWSDtY7kh+BImlJNXoB1lBMEKIq4QDUOXoRgffuDghje1WrG9ML+Hbisq/yFOGwXD9RiX8F6sw6W4avAuvDszue5L3sz85K+EC4Y/wFVDNvZo4TYXao6Z0f+lQKc0t8DQYzk1OXVu8rp2yJMC6alLbBfODALZvYH7n7do1AZls4I9d1P4jnkDrQoxB3UqQ9hVl3LEKQ73xF1OyK5GhDDX8oVfGKF5u+decIsH4YaTw7mP3GFxJSqv3+0lUFJoi5Lc5da149p90IdshCExroL1+7mryIkXPeFM5TgO9r0rvZaBFOvV2z0gp35Z0+L4WPlbuEjN/lxPFin+HlUjr8gRsI3qfJOQFy/9rKIJR0Y/8Omwt/8oTWgy1mdeHmmjk7j1nYsvC9JSQ6ZvMldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqXnLRbwHOoq7hHwg==",
              "format": "base64"
            },
            {
              "data": "MIIFYDCCBEigAwIBAgIQQAF3ITfU6UK47naqPGQKtzANBgkqhkiG9w0BAQsFADA/MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMTDkRTVCBSb290IENBIFgzMB4XDTIxMDEyMDE5MTQwM1oXDTI0MDkzMDE4MTQwM1owTzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2VhcmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCt6CRz9BQ385ueK1coHIe+3LffOJCMbjzmV6B493XCov71am72AE8o295ohmxEk7axY/0UEmu/H9LqMZshftEzPLpI9d1537O4/xLxIZpLwYqGcWlKZmZsj348cL+tKSIG8+TA5oCu4kuPt5l+lAOf00eXfJlII1PoOK5PCm+DLtFJV4yAdLbaL9A4jXsDcCEbdfIwPPqPrt3aY6vrFk/CjhFLfs8L6P+1dy70sntK4EwSJQxwjQMpoOFTJOwT2e4ZvxCzSow/iaNhUd6shweU9GNx7C7ib1uYgeGJXDR5bHbvO5BieebbpJovJsXQEOEO3tkQjhb7t/eo98flAgeYjzYIlefiN5YNNnWe+w5ysR2bvAP5SQXYgd0FtCrWQemsAXaVCg/Y39W9Eh81LygXbNKYwagJZHduRze6zqxZXmidf3LWicUGQSk+WT7dJvUkyRGnWqNMQB9GoZm1pzpRboY7nn1ypxIFeFntPlF4FQsDj43QLwWyPntKHEtzBRL8xurgUBN8Q5N0s8p0544fAQjQMNRbcTa0B7rBMDBcSLeCO5imfWCKoqMpgsy6vYMEG6KDA0Gh1gXxG8K28Kh8hjtGqEgqiNx2mna/H2qlPRmP6zjzZN7IKw0KKP/32+IVQtQi0Cdd4Xn+GOdwiK1O5tmLOsbdJ1Fu/7xk9TNDTwIDAQABo4IBRjCCAUIwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAQYwSwYIKwYBBQUHAQEEPzA9MDsGCCsGAQUFBzAChi9odHRwOi8vYXBwcy5pZGVudHJ1c3QuY29tL3Jvb3RzL2RzdHJvb3RjYXgzLnA3YzAfBgNVHSMEGDAWgBTEp7Gkeyxx+tvhS5B1/8QVYIWJEDBUBgNVHSAETTBLMAgGBmeBDAECATA/BgsrBgEEAYLfEwEBATAwMC4GCCsGAQUFBwIBFiJodHRwOi8vY3BzLnJvb3QteDEubGV0c2VuY3J5cHQub3JnMDwGA1UdHwQ1MDMwMaAvoC2GK2h0dHA6Ly9jcmwuaWRlbnRydXN0LmNvbS9EU1RST09UQ0FYM0NSTC5jcmwwHQYDVR0OBBYEFHm0WeZ7tuXkAXOACIjIGlj26ZtuMA0GCSqGSIb3DQEBCwUAA4IBAQAKcwBslm7/DlLQrt2M51oGrS+o44+/yQoDFVDC5WxCu2+b9LRPwkSICHXM6webFGJueN7sJ7o5XPWioW5WlHAQU7G75K/QosMrAdSW9MUgNTP52GE24HGNtLi1qoJFlcDyqSMo59ahy2cI2qBDLKobkx/J3vWraV0T9VuGWCLKTVXkcGdtwlfFRjlBz4pYg1htmf5X6DYO8A4jqv2Il9DjXA6USbW1FzXSLr9Ohe8Y4IWS6wY7bCkjCWDcRQJMEhg76fsO3txE+FiYruq9RUWhiF1myv4Q6W+CyBFCDfvp7OOGAN6dEOM4+qR9sdjoSYKEBpsr6GtPAQw4dy753ec5",
              "format": "base64"
            }
          ],
          "server_name": "imap.riseup.net",
          "t0": 1.298879237,
          "t": 1.453138571,
          "tags": [],
          "tls_version": "TLSv1.3"
        },
        "failure": "",
        "failed_step": "",
        "ip": "198.252.153.21",
        "port": "993"
      }
    ],
    "failure": ""
  },
  "test_name": "imap",
  "test_runtime": 1.81085194,
  "test_start_time": "2022-11-23 16:40:52",
  "test_version": "0.0.1"
}
Example output:

$ ./miniooni -n -i "imaps://imap.riseup.net" imap
[      0.000070]  Current time: 2022-11-23 17:40:51 CET
[      0.000140]  miniooni home directory: $HOME/.miniooni
[      0.000434]  Looking up OONI backends; please be patient...
[      0.118630]  sessionresolver: https://dns.google/dns-query... ok
[      0.215206]  session: using probe services: {Address:https://api.ooni.io Type:https Front:}
[      0.215225]  Looking up your location; please be patient...
[      0.215252]  iplookup: using cloudflare
[      0.257615]  sessionresolver: https://dns.google/dns-query... ok
2022/11/23 17:40:52 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.
[      0.402298]  sessionresolver: http3://dns.google/dns-query... ok
[      0.402525]  - country: FR
[      0.402539]  - network: Association "Gitoyen" (AS20766)
[      0.402547]  - resolver's IP: 172.217.33.194
[      0.402555]  - resolver's network: Google LLC (AS15169)
[      0.402589]  [1/1] running with input: imaps://imap.riseup.net
[      0.402621]  Resolving DNS for imap.riseup.net
[      0.794714]  Finished DNS for imap.riseup.net: [198.252.153.22 198.252.153.21]
[      0.794733]  Dialing to 198.252.153.22:993
[      0.981213]  Running direct TLS mode to 198.252.153.22:993
[      0.981228]  Starting TLS handshake with 198.252.153.22:993
[      1.198376]  Handshake succeeded
[      1.198388]  Retrieving existing connection
[      1.198397]  Reusing TLS connection
[      1.198403]  Starting IMAP query
[      1.542027]  Finished starting IMAP
[      1.542044]  Trying to generate more no-op traffic
[      1.542071]  NoOp Iteration 1
[      1.542107]  NoOp Iteration 2
[      1.542124]  NoOp Iteration 3
[      1.542145]  NoOp Iteration 4
[      1.542160]  NoOp Iteration 5
[      1.542172]  NoOp Iteration 6
[      1.542200]  NoOp Iteration 7
[      1.542213]  NoOp Iteration 8
[      1.542223]  NoOp Iteration 9
[      1.542232]  NoOp Iteration 10
[      1.542241]  Successfully generated no-op traffic
[      1.542251]  Dialing to 198.252.153.21:993
[      1.701427]  Running direct TLS mode to 198.252.153.21:993
[      1.701445]  Starting TLS handshake with 198.252.153.21:993
[      1.855763]  Handshake succeeded
[      1.855783]  Retrieving existing connection
[      1.855789]  Reusing TLS connection
[      1.855795]  Starting IMAP query
[      2.212937]  Finished starting IMAP
[      2.212971]  Trying to generate more no-op traffic
[      2.212983]  NoOp Iteration 1
[      2.213088]  NoOp Iteration 2
[      2.213110]  NoOp Iteration 3
[      2.213134]  NoOp Iteration 4
[      2.213155]  NoOp Iteration 5
[      2.213168]  NoOp Iteration 6
[      2.213177]  NoOp Iteration 7
[      2.213195]  NoOp Iteration 8
[      2.213212]  NoOp Iteration 9
[      2.213222]  NoOp Iteration 10
[      2.213281]  Successfully generated no-op traffic
[      2.214634]  saving measurement to disk
[      2.214883]  experiment: recv   0.00  byte, sent   0.00  byte
[      2.215070]  sessionresolver: [{"URL":"https://dns.google/dns-query","Score":1},{"URL":"https://cloudflare-dns.com/dns-query","Score":1},{"URL":"http3://dns.google/dns-query","Score":1},{"URL":"http3://cloudflare-dns.com/dns-query","Score":1.3065307323292716e-8},{"URL":"https://mozilla.cloudflare-dns.com/dns-query","Score":1.0000000000000005e-8},{"URL":"http3://mozilla.cloudflare-dns.com/dns-query","Score":9.999999999999979e-9},{"URL":"https://dns.quad9.net/dns-query","Score":9.996899376120266e-9},{"URL":"system:///","Score":0}]
[      2.215140]  whole session: recv   7.23 kbyte, sent   2.06 kbyte

Example of TLS blocking with patched jafar:

  • sudo echo "127.0.0.1 imap.riseup.net" > /etc/hosts
  • sudo ./jafar -tls-proxy-address "127.0.0.1:993" -tls-proxy-block "imap" -tls-proxy-outbound-port "993"
Example report:

{
  "annotations": {
    "architecture": "amd64",
    "engine_name": "ooniprobe-engine",
    "engine_version": "3.17.0-alpha",
    "platform": "linux"
  },
  "data_format_version": "0.2.0",
  "input": "imaps://imap.riseup.net",
  "measurement_start_time": "2022-11-23 16:56:53",
  "probe_asn": "AS20766",
  "probe_cc": "FR",
  "probe_ip": "127.0.0.1",
  "probe_network_name": "Association \"Gitoyen\"",
  "report_id": "",
  "resolver_asn": "AS15169",
  "resolver_ip": "172.217.45.133",
  "resolver_network_name": "Google LLC",
  "software_name": "miniooni",
  "software_version": "3.17.0-alpha",
  "test_keys": {
    "hostname": "imap.riseup.net",
    "queries": [
      {
        "answers": [
          {
            "answer_type": "A",
            "ipv4": "127.0.0.1",
            "ttl": null
          },
          {
            "answer_type": "CNAME",
            "hostname": "imap.riseup.net",
            "ttl": null
          }
        ],
        "engine": "getaddrinfo",
        "failure": null,
        "hostname": "imap.riseup.net",
        "query_type": "ANY",
        "resolver_hostname": null,
        "resolver_port": null,
        "resolver_address": "",
        "t0": 2.9858e-05,
        "t": 0.000193664
      }
    ],
    "runs": [
      {
        "tcp_connect": [
          {
            "ip": "127.0.0.1",
            "port": 993,
            "status": {
              "failure": null,
              "success": true
            },
            "t0": 0.000240459,
            "t": 0.000352533
          }
        ],
        "tls_handshakes": {
          "network": "tcp",
          "address": "127.0.0.1:993",
          "cipher_suite": "",
          "failure": "unknown_failure: remote error: tls: internal error",
          "negotiated_protocol": "",
          "no_tls_verify": false,
          "peer_certificates": [],
          "server_name": "imap.riseup.net",
          "t0": 0.000392685,
          "t": 0.000940963,
          "tags": [],
          "tls_version": ""
        },
        "failure": "unknown_failure: remote error: tls: internal error",
        "failed_step": "tls_handshake",
        "ip": "127.0.0.1",
        "port": "993"
      }
    ],
    "failure": "unknown_failure: remote error: tls: internal error"
  },
  "test_name": "imap",
  "test_runtime": 0.000998997,
  "test_start_time": "2022-11-23 16:56:53",
  "test_version": "0.0.1"
}
Example output:

$ ./miniooni -n -i "imaps://imap.riseup.net" imap
[      0.000460]  Current time: 2022-11-23 17:56:52 CET
[      0.000504]  miniooni home directory: $HOME/.miniooni
[      0.000631]  Looking up OONI backends; please be patient...
2022/11/23 17:56:52 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.
[      0.154063]  sessionresolver: http3://dns.google/dns-query... ok
[      0.292586]  session: using probe services: {Address:https://api.ooni.io Type:https Front:}
[      0.292600]  Looking up your location; please be patient...
[      0.292623]  iplookup: using cloudflare
[      0.446195]  sessionresolver: https://dns.google/dns-query... ok
[      0.604106]  sessionresolver: http3://dns.google/dns-query... ok
[      0.604278]  - country: FR
[      0.604286]  - network: Association "Gitoyen" (AS20766)
[      0.604291]  - resolver's IP: 172.217.45.133
[      0.604296]  - resolver's network: Google LLC (AS15169)
[      0.604320]  [1/1] running with input: imaps://imap.riseup.net
[      0.604340]  Resolving DNS for imap.riseup.net
[      0.604550]  Finished DNS for imap.riseup.net: [127.0.0.1]
[      0.604559]  Dialing to 127.0.0.1:993
[      0.604685]  Running direct TLS mode to 127.0.0.1:993
[      0.604695]  Starting TLS handshake with 127.0.0.1:993
[      0.605284]  unknown_failure: remote error: tls: internal error
[      0.605734]  saving measurement to disk
[      0.605823]  experiment: recv   0.00  byte, sent   0.00  byte
[      0.605983]  sessionresolver: [{"URL":"https://dns.google/dns-query","Score":1},{"URL":"https://cloudflare-dns.com/dns-query","Score":1},{"URL":"http3://dns.google/dns-query","Score":1},{"URL":"http3://cloudflare-dns.com/dns-query","Score":1.3065307323292716e-8},{"URL":"https://mozilla.cloudflare-dns.com/dns-query","Score":1.0000000000000005e-8},{"URL":"http3://mozilla.cloudflare-dns.com/dns-query","Score":9.999999999999979e-9},{"URL":"https://dns.quad9.net/dns-query","Score":9.996899376120266e-9},{"URL":"system:///","Score":0}]
[      0.606046]  whole session: recv   7.23 kbyte, sent   2.06 kbyte

@ooninoob
Copy link
Contributor Author

SMTP output

Example report:

{
  "annotations": {
    "architecture": "amd64",
    "engine_name": "ooniprobe-engine",
    "engine_version": "3.17.0-alpha",
    "platform": "linux"
  },
  "data_format_version": "0.2.0",
  "input": "smtps://smtp.riseup.net",
  "measurement_start_time": "2022-11-23 16:59:46",
  "probe_asn": "AS20766",
  "probe_cc": "FR",
  "probe_ip": "127.0.0.1",
  "probe_network_name": "Association \"Gitoyen\"",
  "report_id": "",
  "resolver_asn": "AS15169",
  "resolver_ip": "172.217.44.131",
  "resolver_network_name": "Google LLC",
  "software_name": "miniooni",
  "software_version": "3.17.0-alpha",
  "test_keys": {
    "hostname": "smtp.riseup.net",
    "queries": [
      {
        "answers": [
          {
            "asn": 16652,
            "as_org_name": "Riseup Networks",
            "answer_type": "A",
            "ipv4": "198.252.153.21",
            "ttl": null
          },
          {
            "asn": 16652,
            "as_org_name": "Riseup Networks",
            "answer_type": "A",
            "ipv4": "198.252.153.22",
            "ttl": null
          },
          {
            "answer_type": "CNAME",
            "hostname": "mail.riseup.net",
            "ttl": null
          }
        ],
        "engine": "getaddrinfo",
        "failure": null,
        "hostname": "smtp.riseup.net",
        "query_type": "ANY",
        "resolver_hostname": null,
        "resolver_port": null,
        "resolver_address": "",
        "t0": 3.9506e-05,
        "t": 0.313091826
      }
    ],
    "runs": [
      {
        "tcp_connect": [
          {
            "ip": "198.252.153.21",
            "port": 465,
            "status": {
              "failure": null,
              "success": true
            },
            "t0": 0.31319894,
            "t": 0.480454542
          }
        ],
        "tls_handshakes": {
          "network": "tcp",
          "address": "198.252.153.21:465",
          "cipher_suite": "TLS_AES_256_GCM_SHA384",
          "failure": null,
          "negotiated_protocol": "",
          "no_tls_verify": false,
          "peer_certificates": [
            {
              "data": "MIIFejCCBGKgAwIBAgISA/RH55OA6DN+ZSHjmgnopqIzMA0GCSqGSIb3DQEBCwUAMDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQDEwJSMzAeFw0yMjExMjMwMjI4NTBaFw0yMzAyMjEwMjI4NDlaMBoxGDAWBgNVBAMTD21haWwucmlzZXVwLm5ldDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALwsvpcklfIBAZxmRrudIxZ2XghRgzbG5n4Eqx9DWh2jINg7I0zTgoYI0hS3hCtU4zu3kekn0KJ9IMPIdeWufk3QWKPDWkmrQD1tpawuvITk9WgzvDlEEKeLyD8U0uKqJeT93zc+C7XYAwhSJB1sTqykpH3pAexo5B1mphODSOSuz00RAXXQQSG5LZIZ++2PfROahxd9/g2kAJzYA2ugmPSTyMY14kk9YoHbB/2+KC0GG320m9vRiPjediW2Vkdq157rfXObl5u8Vz0yH/7xq/2U7+XGhCkg6LHV46bQQIidEiQ4wJ4GE8+DT8qXeflhKnRYzb4scPFdCx465IxramcCAwEAAaOCAqAwggKcMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQUWTOo8S6HmnYRfwWHLkWk4WWZeVcwHwYDVR0jBBgwFoAUFC6zF7dYVsuuUAlA5h+vnYsUwsYwVQYIKwYBBQUHAQEESTBHMCEGCCsGAQUFBzABhhVodHRwOi8vcjMuby5sZW5jci5vcmcwIgYIKwYBBQUHMAKGFmh0dHA6Ly9yMy5pLmxlbmNyLm9yZy8wcAYDVR0RBGkwZ4IQZmV3czEucmlzZXVwLm5ldIIQZmV3czIucmlzZXVwLm5ldIIPaW1hcC5yaXNldXAubmV0gg9tYWlsLnJpc2V1cC5uZXSCDnBvcC5yaXNldXAubmV0gg9zbXRwLnJpc2V1cC5uZXQwTAYDVR0gBEUwQzAIBgZngQwBAgEwNwYLKwYBBAGC3xMBAQEwKDAmBggrBgEFBQcCARYaaHR0cDovL2Nwcy5sZXRzZW5jcnlwdC5vcmcwggEEBgorBgEEAdZ5AgQCBIH1BIHyAPAAdgB6MoxU2LcttiDqOOBSHumEFnAyE4VNO9IrwTpXo1LrUgAAAYSih/goAAAEAwBHMEUCIQDKfy3g+wuEe1ngugHXtPR4ocXK/lWAiUtQihVoitfc6AIgHARn8FRkaevO0hrtnkleQ6dobI9SXdva5n6QBP/1iAUAdgDoPtDaPvUGNTLnVyi8iWvJA9PL0RFr7Otp4Xd9bQa9bgAAAYSih/oBAAAEAwBHMEUCIQDOzm0w7maB6IwA8jaat0i/+zZaRKvFwb+yENPee+SgsQIgR/1jytWIaUzk9Yp1lbmQKh8GmnY0PIEJWj9WpVlOy74wDQYJKoZIhvcNAQELBQADggEBAHzzszgg2p2oB5hARn6J7mUSrVUuR3H7wAcCPdOrQA3ZnOGjGg0Pq1T0eV5z+NII+ijOZh5dnWjFqpcv7U1DWfZe4gfme96djylB95AmdbcUyb3eS/+jpwSqKlkXBQBZmWhFUyIDi9BNA7UAKNATmlG+sn9vr0H4aQ9+otYy4BDpdrizyNXGhX4wQhUKMT25aYZScNb0sgsjQAXaAVZv9CEz2nN/P6zYuer/DXTZ51wPf4orBTam7xFmAiGhedSSQ59W6jDIlAekBfMC/ZJxYC5mY7eS+x3dcXrtzFgmRfKad4+XZDNt6ZUeJ6ncKuROE7dqJ0srsBY3ktX3OLTdoDI=",
              "format": "base64"
            },
            {
              "data": "MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAwTzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2VhcmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjAwOTA0MDAwMDAwWhcNMjUwOTE1MTYwMDAwWjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3MgRW5jcnlwdDELMAkGA1UEAxMCUjMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC7AhUozPaglNMPEuyNVZLD+ILxmaZ6QoinXSaqtSu5xUyxr45r+XXIo9cPR5QUVTVXjJ6oojkZ9YI8QqlObvU7wy7bjcCwXPNZOOftz2nwWgsbvsCUJCWH+jdxsxPnHKzhm+/b5DtFUkWWqcFTzjTIUu61ru2P3mBw4qVUq7ZtDpelQDRrK9O8ZutmNHz6a4uPVymZ+DAXXbpyb/uBxa3Shlg9F8fnCbvxK/eG3MHacV3URuPMrSXBiLxgZ3Vms/EY96Jc5lP/Ooi2R6X/ExjqmAl3P51T+c8B5fWmcBcUr2Ok/5mzk53cU6cG/kiFHaFpriV1uxPMUgP17VGhi9sVAgMBAAGjggEIMIIBBDAOBgNVHQ8BAf8EBAMCAYYwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYfr52LFMLGMB8GA1UdIwQYMBaAFHm0WeZ7tuXkAXOACIjIGlj26ZtuMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcwAoYWaHR0cDovL3gxLmkubGVuY3Iub3JnLzAnBgNVHR8EIDAeMBygGqAYhhZodHRwOi8veDEuYy5sZW5jci5vcmcvMCIGA1UdIAQbMBkwCAYGZ4EMAQIBMA0GCysGAQQBgt8TAQEBMA0GCSqGSIb3DQEBCwUAA4ICAQCFyk5HPqP3hUSFvNVneLKYY611TR6WPTNlclQtgaDqw+34IL9fzLdwALduO/ZelN7kIJ+m74uyA+eitRY8kc607TkC53wlikfmZW4/RvTZ8M6UK+5UzhK8jCdLuMGYL6KvzXGRSgi3yLgjewQtCPkIVz6D2QQzCkcheAmCJ8MqyJu5zlzyZMjAvnnAT45tRAxekrsu94sQ4egdRCnbWSDtY7kh+BImlJNXoB1lBMEKIq4QDUOXoRgffuDghje1WrG9ML+Hbisq/yFOGwXD9RiX8F6sw6W4avAuvDszue5L3sz85K+EC4Y/wFVDNvZo4TYXao6Z0f+lQKc0t8DQYzk1OXVu8rp2yJMC6alLbBfODALZvYH7n7do1AZls4I9d1P4jnkDrQoxB3UqQ9hVl3LEKQ73xF1OyK5GhDDX8oVfGKF5u+decIsH4YaTw7mP3GFxJSqv3+0lUFJoi5Lc5da149p90IdshCExroL1+7mryIkXPeFM5TgO9r0rvZaBFOvV2z0gp35Z0+L4WPlbuEjN/lxPFin+HlUjr8gRsI3qfJOQFy/9rKIJR0Y/8Omwt/8oTWgy1mdeHmmjk7j1nYsvC9JSQ6ZvMldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqXnLRbwHOoq7hHwg==",
              "format": "base64"
            },
            {
              "data": "MIIFYDCCBEigAwIBAgIQQAF3ITfU6UK47naqPGQKtzANBgkqhkiG9w0BAQsFADA/MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMTDkRTVCBSb290IENBIFgzMB4XDTIxMDEyMDE5MTQwM1oXDTI0MDkzMDE4MTQwM1owTzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2VhcmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCt6CRz9BQ385ueK1coHIe+3LffOJCMbjzmV6B493XCov71am72AE8o295ohmxEk7axY/0UEmu/H9LqMZshftEzPLpI9d1537O4/xLxIZpLwYqGcWlKZmZsj348cL+tKSIG8+TA5oCu4kuPt5l+lAOf00eXfJlII1PoOK5PCm+DLtFJV4yAdLbaL9A4jXsDcCEbdfIwPPqPrt3aY6vrFk/CjhFLfs8L6P+1dy70sntK4EwSJQxwjQMpoOFTJOwT2e4ZvxCzSow/iaNhUd6shweU9GNx7C7ib1uYgeGJXDR5bHbvO5BieebbpJovJsXQEOEO3tkQjhb7t/eo98flAgeYjzYIlefiN5YNNnWe+w5ysR2bvAP5SQXYgd0FtCrWQemsAXaVCg/Y39W9Eh81LygXbNKYwagJZHduRze6zqxZXmidf3LWicUGQSk+WT7dJvUkyRGnWqNMQB9GoZm1pzpRboY7nn1ypxIFeFntPlF4FQsDj43QLwWyPntKHEtzBRL8xurgUBN8Q5N0s8p0544fAQjQMNRbcTa0B7rBMDBcSLeCO5imfWCKoqMpgsy6vYMEG6KDA0Gh1gXxG8K28Kh8hjtGqEgqiNx2mna/H2qlPRmP6zjzZN7IKw0KKP/32+IVQtQi0Cdd4Xn+GOdwiK1O5tmLOsbdJ1Fu/7xk9TNDTwIDAQABo4IBRjCCAUIwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAQYwSwYIKwYBBQUHAQEEPzA9MDsGCCsGAQUFBzAChi9odHRwOi8vYXBwcy5pZGVudHJ1c3QuY29tL3Jvb3RzL2RzdHJvb3RjYXgzLnA3YzAfBgNVHSMEGDAWgBTEp7Gkeyxx+tvhS5B1/8QVYIWJEDBUBgNVHSAETTBLMAgGBmeBDAECATA/BgsrBgEEAYLfEwEBATAwMC4GCCsGAQUFBwIBFiJodHRwOi8vY3BzLnJvb3QteDEubGV0c2VuY3J5cHQub3JnMDwGA1UdHwQ1MDMwMaAvoC2GK2h0dHA6Ly9jcmwuaWRlbnRydXN0LmNvbS9EU1RST09UQ0FYM0NSTC5jcmwwHQYDVR0OBBYEFHm0WeZ7tuXkAXOACIjIGlj26ZtuMA0GCSqGSIb3DQEBCwUAA4IBAQAKcwBslm7/DlLQrt2M51oGrS+o44+/yQoDFVDC5WxCu2+b9LRPwkSICHXM6webFGJueN7sJ7o5XPWioW5WlHAQU7G75K/QosMrAdSW9MUgNTP52GE24HGNtLi1qoJFlcDyqSMo59ahy2cI2qBDLKobkx/J3vWraV0T9VuGWCLKTVXkcGdtwlfFRjlBz4pYg1htmf5X6DYO8A4jqv2Il9DjXA6USbW1FzXSLr9Ohe8Y4IWS6wY7bCkjCWDcRQJMEhg76fsO3txE+FiYruq9RUWhiF1myv4Q6W+CyBFCDfvp7OOGAN6dEOM4+qR9sdjoSYKEBpsr6GtPAQw4dy753ec5",
              "format": "base64"
            }
          ],
          "server_name": "smtp.riseup.net",
          "t0": 0.480503605,
          "t": 1.6728580050000001,
          "tags": [],
          "tls_version": "TLSv1.3"
        },
        "failure": "",
        "failed_step": "",
        "ip": "198.252.153.21",
        "port": "465"
      },
      {
        "tcp_connect": [
          {
            "ip": "198.252.153.22",
            "port": 465,
            "status": {
              "failure": null,
              "success": true
            },
            "t0": 4.184200913,
            "t": 4.390252046
          }
        ],
        "tls_handshakes": {
          "network": "tcp",
          "address": "198.252.153.22:465",
          "cipher_suite": "TLS_AES_256_GCM_SHA384",
          "failure": null,
          "negotiated_protocol": "",
          "no_tls_verify": false,
          "peer_certificates": [
            {
              "data": "MIIFejCCBGKgAwIBAgISA/RH55OA6DN+ZSHjmgnopqIzMA0GCSqGSIb3DQEBCwUAMDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQDEwJSMzAeFw0yMjExMjMwMjI4NTBaFw0yMzAyMjEwMjI4NDlaMBoxGDAWBgNVBAMTD21haWwucmlzZXVwLm5ldDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALwsvpcklfIBAZxmRrudIxZ2XghRgzbG5n4Eqx9DWh2jINg7I0zTgoYI0hS3hCtU4zu3kekn0KJ9IMPIdeWufk3QWKPDWkmrQD1tpawuvITk9WgzvDlEEKeLyD8U0uKqJeT93zc+C7XYAwhSJB1sTqykpH3pAexo5B1mphODSOSuz00RAXXQQSG5LZIZ++2PfROahxd9/g2kAJzYA2ugmPSTyMY14kk9YoHbB/2+KC0GG320m9vRiPjediW2Vkdq157rfXObl5u8Vz0yH/7xq/2U7+XGhCkg6LHV46bQQIidEiQ4wJ4GE8+DT8qXeflhKnRYzb4scPFdCx465IxramcCAwEAAaOCAqAwggKcMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQUWTOo8S6HmnYRfwWHLkWk4WWZeVcwHwYDVR0jBBgwFoAUFC6zF7dYVsuuUAlA5h+vnYsUwsYwVQYIKwYBBQUHAQEESTBHMCEGCCsGAQUFBzABhhVodHRwOi8vcjMuby5sZW5jci5vcmcwIgYIKwYBBQUHMAKGFmh0dHA6Ly9yMy5pLmxlbmNyLm9yZy8wcAYDVR0RBGkwZ4IQZmV3czEucmlzZXVwLm5ldIIQZmV3czIucmlzZXVwLm5ldIIPaW1hcC5yaXNldXAubmV0gg9tYWlsLnJpc2V1cC5uZXSCDnBvcC5yaXNldXAubmV0gg9zbXRwLnJpc2V1cC5uZXQwTAYDVR0gBEUwQzAIBgZngQwBAgEwNwYLKwYBBAGC3xMBAQEwKDAmBggrBgEFBQcCARYaaHR0cDovL2Nwcy5sZXRzZW5jcnlwdC5vcmcwggEEBgorBgEEAdZ5AgQCBIH1BIHyAPAAdgB6MoxU2LcttiDqOOBSHumEFnAyE4VNO9IrwTpXo1LrUgAAAYSih/goAAAEAwBHMEUCIQDKfy3g+wuEe1ngugHXtPR4ocXK/lWAiUtQihVoitfc6AIgHARn8FRkaevO0hrtnkleQ6dobI9SXdva5n6QBP/1iAUAdgDoPtDaPvUGNTLnVyi8iWvJA9PL0RFr7Otp4Xd9bQa9bgAAAYSih/oBAAAEAwBHMEUCIQDOzm0w7maB6IwA8jaat0i/+zZaRKvFwb+yENPee+SgsQIgR/1jytWIaUzk9Yp1lbmQKh8GmnY0PIEJWj9WpVlOy74wDQYJKoZIhvcNAQELBQADggEBAHzzszgg2p2oB5hARn6J7mUSrVUuR3H7wAcCPdOrQA3ZnOGjGg0Pq1T0eV5z+NII+ijOZh5dnWjFqpcv7U1DWfZe4gfme96djylB95AmdbcUyb3eS/+jpwSqKlkXBQBZmWhFUyIDi9BNA7UAKNATmlG+sn9vr0H4aQ9+otYy4BDpdrizyNXGhX4wQhUKMT25aYZScNb0sgsjQAXaAVZv9CEz2nN/P6zYuer/DXTZ51wPf4orBTam7xFmAiGhedSSQ59W6jDIlAekBfMC/ZJxYC5mY7eS+x3dcXrtzFgmRfKad4+XZDNt6ZUeJ6ncKuROE7dqJ0srsBY3ktX3OLTdoDI=",
              "format": "base64"
            },
            {
              "data": "MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAwTzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2VhcmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjAwOTA0MDAwMDAwWhcNMjUwOTE1MTYwMDAwWjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3MgRW5jcnlwdDELMAkGA1UEAxMCUjMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC7AhUozPaglNMPEuyNVZLD+ILxmaZ6QoinXSaqtSu5xUyxr45r+XXIo9cPR5QUVTVXjJ6oojkZ9YI8QqlObvU7wy7bjcCwXPNZOOftz2nwWgsbvsCUJCWH+jdxsxPnHKzhm+/b5DtFUkWWqcFTzjTIUu61ru2P3mBw4qVUq7ZtDpelQDRrK9O8ZutmNHz6a4uPVymZ+DAXXbpyb/uBxa3Shlg9F8fnCbvxK/eG3MHacV3URuPMrSXBiLxgZ3Vms/EY96Jc5lP/Ooi2R6X/ExjqmAl3P51T+c8B5fWmcBcUr2Ok/5mzk53cU6cG/kiFHaFpriV1uxPMUgP17VGhi9sVAgMBAAGjggEIMIIBBDAOBgNVHQ8BAf8EBAMCAYYwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYfr52LFMLGMB8GA1UdIwQYMBaAFHm0WeZ7tuXkAXOACIjIGlj26ZtuMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcwAoYWaHR0cDovL3gxLmkubGVuY3Iub3JnLzAnBgNVHR8EIDAeMBygGqAYhhZodHRwOi8veDEuYy5sZW5jci5vcmcvMCIGA1UdIAQbMBkwCAYGZ4EMAQIBMA0GCysGAQQBgt8TAQEBMA0GCSqGSIb3DQEBCwUAA4ICAQCFyk5HPqP3hUSFvNVneLKYY611TR6WPTNlclQtgaDqw+34IL9fzLdwALduO/ZelN7kIJ+m74uyA+eitRY8kc607TkC53wlikfmZW4/RvTZ8M6UK+5UzhK8jCdLuMGYL6KvzXGRSgi3yLgjewQtCPkIVz6D2QQzCkcheAmCJ8MqyJu5zlzyZMjAvnnAT45tRAxekrsu94sQ4egdRCnbWSDtY7kh+BImlJNXoB1lBMEKIq4QDUOXoRgffuDghje1WrG9ML+Hbisq/yFOGwXD9RiX8F6sw6W4avAuvDszue5L3sz85K+EC4Y/wFVDNvZo4TYXao6Z0f+lQKc0t8DQYzk1OXVu8rp2yJMC6alLbBfODALZvYH7n7do1AZls4I9d1P4jnkDrQoxB3UqQ9hVl3LEKQ73xF1OyK5GhDDX8oVfGKF5u+decIsH4YaTw7mP3GFxJSqv3+0lUFJoi5Lc5da149p90IdshCExroL1+7mryIkXPeFM5TgO9r0rvZaBFOvV2z0gp35Z0+L4WPlbuEjN/lxPFin+HlUjr8gRsI3qfJOQFy/9rKIJR0Y/8Omwt/8oTWgy1mdeHmmjk7j1nYsvC9JSQ6ZvMldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqXnLRbwHOoq7hHwg==",
              "format": "base64"
            },
            {
              "data": "MIIFYDCCBEigAwIBAgIQQAF3ITfU6UK47naqPGQKtzANBgkqhkiG9w0BAQsFADA/MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMTDkRTVCBSb290IENBIFgzMB4XDTIxMDEyMDE5MTQwM1oXDTI0MDkzMDE4MTQwM1owTzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2VhcmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCt6CRz9BQ385ueK1coHIe+3LffOJCMbjzmV6B493XCov71am72AE8o295ohmxEk7axY/0UEmu/H9LqMZshftEzPLpI9d1537O4/xLxIZpLwYqGcWlKZmZsj348cL+tKSIG8+TA5oCu4kuPt5l+lAOf00eXfJlII1PoOK5PCm+DLtFJV4yAdLbaL9A4jXsDcCEbdfIwPPqPrt3aY6vrFk/CjhFLfs8L6P+1dy70sntK4EwSJQxwjQMpoOFTJOwT2e4ZvxCzSow/iaNhUd6shweU9GNx7C7ib1uYgeGJXDR5bHbvO5BieebbpJovJsXQEOEO3tkQjhb7t/eo98flAgeYjzYIlefiN5YNNnWe+w5ysR2bvAP5SQXYgd0FtCrWQemsAXaVCg/Y39W9Eh81LygXbNKYwagJZHduRze6zqxZXmidf3LWicUGQSk+WT7dJvUkyRGnWqNMQB9GoZm1pzpRboY7nn1ypxIFeFntPlF4FQsDj43QLwWyPntKHEtzBRL8xurgUBN8Q5N0s8p0544fAQjQMNRbcTa0B7rBMDBcSLeCO5imfWCKoqMpgsy6vYMEG6KDA0Gh1gXxG8K28Kh8hjtGqEgqiNx2mna/H2qlPRmP6zjzZN7IKw0KKP/32+IVQtQi0Cdd4Xn+GOdwiK1O5tmLOsbdJ1Fu/7xk9TNDTwIDAQABo4IBRjCCAUIwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAQYwSwYIKwYBBQUHAQEEPzA9MDsGCCsGAQUFBzAChi9odHRwOi8vYXBwcy5pZGVudHJ1c3QuY29tL3Jvb3RzL2RzdHJvb3RjYXgzLnA3YzAfBgNVHSMEGDAWgBTEp7Gkeyxx+tvhS5B1/8QVYIWJEDBUBgNVHSAETTBLMAgGBmeBDAECATA/BgsrBgEEAYLfEwEBATAwMC4GCCsGAQUFBwIBFiJodHRwOi8vY3BzLnJvb3QteDEubGV0c2VuY3J5cHQub3JnMDwGA1UdHwQ1MDMwMaAvoC2GK2h0dHA6Ly9jcmwuaWRlbnRydXN0LmNvbS9EU1RST09UQ0FYM0NSTC5jcmwwHQYDVR0OBBYEFHm0WeZ7tuXkAXOACIjIGlj26ZtuMA0GCSqGSIb3DQEBCwUAA4IBAQAKcwBslm7/DlLQrt2M51oGrS+o44+/yQoDFVDC5WxCu2+b9LRPwkSICHXM6webFGJueN7sJ7o5XPWioW5WlHAQU7G75K/QosMrAdSW9MUgNTP52GE24HGNtLi1qoJFlcDyqSMo59ahy2cI2qBDLKobkx/J3vWraV0T9VuGWCLKTVXkcGdtwlfFRjlBz4pYg1htmf5X6DYO8A4jqv2Il9DjXA6USbW1FzXSLr9Ohe8Y4IWS6wY7bCkjCWDcRQJMEhg76fsO3txE+FiYruq9RUWhiF1myv4Q6W+CyBFCDfvp7OOGAN6dEOM4+qR9sdjoSYKEBpsr6GtPAQw4dy753ec5",
              "format": "base64"
            }
          ],
          "server_name": "smtp.riseup.net",
          "t0": 4.39030345,
          "t": 4.742412218,
          "tags": [],
          "tls_version": "TLSv1.3"
        },
        "failure": "",
        "failed_step": "",
        "ip": "198.252.153.22",
        "port": "465"
      }
    ],
    "failure": ""
  },
  "test_name": "smtp",
  "test_runtime": 7.153666512,
  "test_start_time": "2022-11-23 16:59:38",
  "test_version": "0.0.1"
}
Example output:

$ ./miniooni -n -i "smtps://smtp.riseup.net" smtp
[      0.000326]  Current time: 2022-11-23 17:59:38 CET
[      0.000363]  miniooni home directory: $HOME/.miniooni
[      0.000476]  Looking up OONI backends; please be patient...
2022/11/23 17:59:38 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.
[      0.112820]  sessionresolver: http3://dns.google/dns-query... ok
[      0.233255]  session: using probe services: {Address:https://api.ooni.io Type:https Front:}
[      0.233277]  Looking up your location; please be patient...
[      0.233303]  iplookup: using cloudflare
[      0.292353]  sessionresolver: http3://dns.google/dns-query... ok
[      0.447327]  sessionresolver: http3://dns.google/dns-query... ok
[      0.447525]  - country: FR
[      0.447535]  - network: Association "Gitoyen" (AS20766)
[      0.447541]  - resolver's IP: 172.217.44.131
[      0.447549]  - resolver's network: Google LLC (AS15169)
[      0.447601]  [1/1] running with input: smtps://smtp.riseup.net
[      0.447627]  Resolving DNS for smtp.riseup.net
[      0.760779]  Finished DNS for smtp.riseup.net: [198.252.153.21 198.252.153.22]
[      0.760796]  Dialing to 198.252.153.21:465
[      0.928074]  Running direct TLS mode to 198.252.153.21:465
[      0.928088]  Starting TLS handshake with 198.252.153.21:465
[      2.120489]  Handshake succeeded
[      2.120504]  Retrieving existing connection
[      2.120509]  Reusing TLS connection
[      2.120518]  Initializing SMTP client
[      2.459708]  Starting SMTP EHLO
[      2.608812]  Finished SMTP EHLO
[      2.608826]  Trying to generate more no-op traffic
[      2.608834]  NoOp Iteration 1
[      2.757679]  NoOp Iteration 2
[      2.993018]  NoOp Iteration 3
[      3.197764]  NoOp Iteration 4
[      3.402707]  NoOp Iteration 5
[      3.607850]  NoOp Iteration 6
[      3.812655]  NoOp Iteration 7
[      4.017062]  NoOp Iteration 8
[      4.222409]  NoOp Iteration 9
[      4.431981]  NoOp Iteration 10
[      4.631774]  Successfully generated no-op traffic
[      4.631796]  Dialing to 198.252.153.22:465
[      4.837870]  Running direct TLS mode to 198.252.153.22:465
[      4.837888]  Starting TLS handshake with 198.252.153.22:465
[      5.190039]  Handshake succeeded
[      5.190052]  Retrieving existing connection
[      5.190057]  Reusing TLS connection
[      5.190071]  Initializing SMTP client
[      5.533317]  Starting SMTP EHLO
[      5.684826]  Finished SMTP EHLO
[      5.684840]  Trying to generate more no-op traffic
[      5.684849]  NoOp Iteration 1
[      5.838907]  NoOp Iteration 2
[      5.988120]  NoOp Iteration 3
[      6.172832]  NoOp Iteration 4
[      6.372601]  NoOp Iteration 5
[      6.577030]  NoOp Iteration 6
[      6.787039]  NoOp Iteration 7
[      6.987647]  NoOp Iteration 8
[      7.199339]  NoOp Iteration 9
[      7.399888]  NoOp Iteration 10
[      7.601166]  Successfully generated no-op traffic
[      7.602301]  saving measurement to disk
[      7.602535]  experiment: recv   0.00  byte, sent   0.00  byte
[      7.602682]  sessionresolver: [{"URL":"https://dns.google/dns-query","Score":1},{"URL":"https://cloudflare-dns.com/dns-query","Score":1},{"URL":"http3://dns.google/dns-query","Score":1},{"URL":"http3://cloudflare-dns.com/dns-query","Score":1.3065307323292716e-8},{"URL":"https://mozilla.cloudflare-dns.com/dns-query","Score":1.0000000000000005e-8},{"URL":"http3://mozilla.cloudflare-dns.com/dns-query","Score":9.999999999999979e-9},{"URL":"https://dns.quad9.net/dns-query","Score":9.996899376120266e-9},{"URL":"system:///","Score":0}]
[      7.602749]  whole session: recv   7.23 kbyte, sent   2.06 kbyte

Example of TLS blocking with #987:

sudo echo "127.0.0.1 smtp.riseup.net" > /etc/hosts
sudo ./jafar -tls-proxy-address "127.0.0.1:465" -tls-proxy-block "imap" -tls-proxy-outbound-port "465"
Example report:

{
  "annotations": {
    "architecture": "amd64",
    "engine_name": "ooniprobe-engine",
    "engine_version": "3.17.0-alpha",
    "platform": "linux"
  },
  "data_format_version": "0.2.0",
  "input": "smtps://smtp.riseup.net",
  "measurement_start_time": "2022-11-23 17:02:42",
  "probe_asn": "AS20766",
  "probe_cc": "FR",
  "probe_ip": "127.0.0.1",
  "probe_network_name": "Association \"Gitoyen\"",
  "report_id": "",
  "resolver_asn": "AS15169",
  "resolver_ip": "172.253.10.132",
  "resolver_network_name": "Google LLC",
  "software_name": "miniooni",
  "software_version": "3.17.0-alpha",
  "test_keys": {
    "hostname": "smtp.riseup.net",
    "queries": [
      {
        "answers": [
          {
            "answer_type": "A",
            "ipv4": "127.0.0.1",
            "ttl": null
          },
          {
            "answer_type": "CNAME",
            "hostname": "smtp.riseup.net",
            "ttl": null
          }
        ],
        "engine": "getaddrinfo",
        "failure": null,
        "hostname": "smtp.riseup.net",
        "query_type": "ANY",
        "resolver_hostname": null,
        "resolver_port": null,
        "resolver_address": "",
        "t0": 3.5108e-05,
        "t": 0.000177662
      }
    ],
    "runs": [
      {
        "tcp_connect": [
          {
            "ip": "127.0.0.1",
            "port": 465,
            "status": {
              "failure": null,
              "success": true
            },
            "t0": 0.000229193,
            "t": 0.000352798
          }
        ],
        "tls_handshakes": {
          "network": "tcp",
          "address": "127.0.0.1:465",
          "cipher_suite": "",
          "failure": "unknown_failure: remote error: tls: internal error",
          "negotiated_protocol": "",
          "no_tls_verify": false,
          "peer_certificates": [],
          "server_name": "smtp.riseup.net",
          "t0": 0.000403952,
          "t": 0.000897951,
          "tags": [],
          "tls_version": ""
        },
        "failure": "unknown_failure: remote error: tls: internal error",
        "failed_step": "tls_handshake",
        "ip": "127.0.0.1",
        "port": "465"
      }
    ],
    "failure": "unknown_failure: remote error: tls: internal error"
  },
  "test_name": "smtp",
  "test_runtime": 0.000971074,
  "test_start_time": "2022-11-23 17:02:42",
  "test_version": "0.0.1"
}
Example output:

$ ./miniooni -n -i "smtps://smtp.riseup.net" smtp
[      0.000035]  Current time: 2022-11-23 18:02:41 CET
[      0.000072]  miniooni home directory: $HOME/.miniooni
[      0.000184]  Looking up OONI backends; please be patient...
[      0.107067]  sessionresolver: https://dns.google/dns-query... ok
[      0.213210]  session: using probe services: {Address:https://api.ooni.io Type:https Front:}
[      0.213226]  Looking up your location; please be patient...
[      0.213246]  iplookup: using cloudflare
[      0.260491]  sessionresolver: https://dns.google/dns-query... ok
2022/11/23 18:02:42 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.
[      0.827062]  sessionresolver: http3://dns.google/dns-query... ok
[      0.827277]  - country: FR
[      0.827287]  - network: Association "Gitoyen" (AS20766)
[      0.827294]  - resolver's IP: 172.253.10.132
[      0.827300]  - resolver's network: Google LLC (AS15169)
[      0.827325]  [1/1] running with input: smtps://smtp.riseup.net
[      0.827351]  Resolving DNS for smtp.riseup.net
[      0.827542]  Finished DNS for smtp.riseup.net: [127.0.0.1]
[      0.827553]  Dialing to 127.0.0.1:465
[      0.827696]  Running direct TLS mode to 127.0.0.1:465
[      0.827708]  Starting TLS handshake with 127.0.0.1:465
[      0.828250]  unknown_failure: remote error: tls: internal error
[      0.828688]  saving measurement to disk
[      0.828779]  experiment: recv   0.00  byte, sent   0.00  byte
[      0.828949]  sessionresolver: [{"URL":"https://dns.google/dns-query","Score":1},{"URL":"https://cloudflare-dns.com/dns-query","Score":1},{"URL":"http3://dns.google/dns-query","Score":1},{"URL":"http3://cloudflare-dns.com/dns-query","Score":1.3065307323292716e-8},{"URL":"https://mozilla.cloudflare-dns.com/dns-query","Score":1.0000000000000005e-8},{"URL":"http3://mozilla.cloudflare-dns.com/dns-query","Score":9.999999999999979e-9},{"URL":"https://dns.quad9.net/dns-query","Score":9.996899376120266e-9},{"URL":"system:///","Score":0}]
[      0.829035]  whole session: recv   7.23 kbyte, sent   2.06 kbyte


port := ""

if parsed.Port() == "" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps instead of setting a default port, we should throw an error due to a malformed input.

Since we generally use the input column for searching and filtering through measurements, a measurement run with as input imap://gmail.com will not be grouped together with a measurement with input imap://gmail.com:143.

ServerName: config.host,
}

runner := &tcprunner.TCPRunner{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should eventually put something like this into the standard measurement library. I think putting it as an experiment package is sub-optimal.

@bassosimone where do you think something like this could go?

Copy link
Contributor

@bassosimone bassosimone Nov 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should eventually put something like this into the standard measurement library. I think putting it as an experiment package is sub-optimal.

@bassosimone where do you think something like this could go?

Based on a quick skim through the package implementation itself, this code seems to me an abstraction built on top of measurexlite. It seems to support well the use case of this experiment and may support alike experiments, so I think it's fine to give it a toplevel-internal directory name such as ./internal/tcprunner.

BTW, the fact that @ooninoob felt the need to write a support package is very precious information. Our previous assumption was that people were going to write experiments using measurexlite primitives. However, it seems this level of abstraction is too low to really be satisfactory. Because we also determined the declarative API of urlgetter is also not flexible enough, it is clear to me there is need to invest some extra time to provide a more comprehensive abstraction on top of measurexlite to support several use cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be useful if i wrote about my personal feelings as a newcomer regarding the internal APIs? I could open a new ticket about this. Please note that i'm still not confident about what makes a good data structure for the test keys when it comes to testing multiple things (several hostnames and/or IP/port combos, as in these probes and the DHT/Bittorrent probes) and that consideration very much affects how an internal API should feel like.

I don't know what is a good answer and i for sure can't come up with specific guidelines, but it feels to me like there's different ways to approach the testkeys organization from a high-level perspective:

  • global testkeys for everything (linear queries/tcp results/etc)
  • global testkeys to store one invididual testkeys for each hostname (including DNS results)
  • global testkeys (for DNS results) storing one individual testkeys for each IP resolved
  • global testkeys storing one individual testkeys for each hostname, each storing one sub-individual testkeys for each resolved IP

It would sure be possible to have explicit guidelines about this, but just for starting to develop things we could have high-level APIs for several of those modes of operation. I think exposing a failed_step/failed_run as part of those high-level APIs would also bring value to help produce better test results (and better debugging abilities) without having each probe reinvent the wheel.

That is my personal opinion that probes should focus on their probing logic (networking/protocol stuff) and not have to know/care about ooni internals and how data is magically collected and presented. This would for sure lower the barrier for contribution for new probes, although i would understand if the ooni project willingly prefers to keep the barrier high and the implementation details explicit. I just felt from talking to @hellais and from reading your comment that it may not be a conscious design decision to keep me (and other future contributors) buried under ooni internal abstractions and that all this error-prone boilerplate may have been accidental ;-)

Copy link
Contributor

@bassosimone bassosimone Dec 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ooninoob thank you for your very detailed reply!

Some comments inline:

different ways to approach the testkeys organization from a high-level perspective:

  • global testkeys for everything (linear queries/tcp results/etc)
  • global testkeys to store one invididual testkeys for each hostname (including DNS results)
  • global testkeys (for DNS results) storing one individual testkeys for each IP resolved
  • global testkeys storing one individual testkeys for each hostname, each storing one sub-individual testkeys for each resolved IP

I am not sure sure about what you mean here. Are you complaining about the data format being inconsistent across experiments or are you referring to the APIs available for writing experiments?

probes should focus on their probing logic (networking/protocol stuff) and not have to know/care about ooni internals and how data is magically collected and presented

I agree with the need to allow a programmer to focus on the network experiment logic. I find it to insightful to see you would like our documentation to spend less time about how data is collected and presented. I see why the current documentation focuses on that and why you rightfully say you should be (too much) concerned with that.

lower the barrier for contribution for new probes, although i would understand if the ooni project willingly prefers to keep the barrier high and the implementation details explicit

We previously had a high level API, which we now have deprecated. The reason why we deprecated it is that it was and is a burden to maintain in light of having a number of experiments to maintain. Each experiment has some set of semi-conflicting requirements, which shapes and puts pressure on such API.

You ended up using directly the internal primitives (aka "step-by-step") that we determined are "better", where better is defined in terms of the kind of experiments (and follow-up experiments) we can write. We were not sure whether there was a need to add another API on top of that or not. Your experience tells us that we definitely need to offer some kind of API for people to write experiments that is more abstract.

it may not be a conscious design decision to keep me (and other future contributors) buried under ooni internal abstractions and that all this error-prone boilerplate may have been accidental

Yeah, it's not a conscious decision. I suppose it's more of a learning process where, on the one end, we need to adapt our tools for measuring censorship and detecting which low-level step failed and possibly try to run some follow-up test. On the other end, we also need to provide contributors with easy to use abstractions. There's definitely a time lag in there and it's also true that it's difficult to know the direction without feedback, so thank you for your feedback!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Oh, @ooninoob, if you have time to chat about this, perhaps we can have a chat on Slack and then distill the result of the chat in some issue or design document to improve the contributing experience.)

TCPConnect []*model.ArchivalTCPConnectResult `json:"tcp_connect"`
TLSHandshake *model.ArchivalTLSOrQUICHandshakeResult `json:"tls_handshakes"`
Failure string `json:"failure"`
FailureStep string `json:"failed_step"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
FailureStep string `json:"failed_step"`
FailedStep string `json:"failed_step"`

// FailedStep TCPSessionModel
func (itk *IndividualTestKeys) FailedStep(failure string, step string) {
itk.Failure = failure
itk.FailureStep = step
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
itk.FailureStep = step
itk.FailedStep = step

TCPConnect []*model.ArchivalTCPConnectResult `json:"tcp_connect"`
TLSHandshake *model.ArchivalTLSOrQUICHandshakeResult `json:"tls_handshakes"`
Failure string `json:"failure"`
FailureStep string `json:"failed_step"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
FailureStep string `json:"failed_step"`
FailedStep string `json:"failed_step"`

// FailedStep TCPSessionModel
func (itk *IndividualTestKeys) FailedStep(failure string, step string) {
itk.Failure = failure
itk.FailureStep = step
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
itk.FailureStep = step
itk.FailedStep = step

func (itk *IndividualTestKeys) FailedStep(failure string, step string) {
itk.Failure = failure
itk.FailureStep = step
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you really need setters for the individualTestKeys attributes. You can just set them directly in the body of the experiment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I introduced this helper method to be sure i wouldn't miss one of the two keys when reporting an error. Is this unreasonable? Also, the field is named FailureStep precisely because this method FailedStep creates a naming conflict. I could rename it of course if you think it's wise to keep this helper method.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding to what @hellais said:

The use case where you need setters I can think of is the one in which you're running operations in parallel and therefore want to protect the test keys with a mutex. In such a case, I suggest this pattern:

type TestKeys struct {
  Foo string `json:"foo"`

  mu *sync.Mutex // not exported because lowercase
}

func (tk *TestKeys) setFoo(value string) {
  defer tk.mu.Unlock()
  tk.mu.Lock()
  tk.Foo = value
}

For cases in which you're running operations in parallel, I would recommend setting keys directly.

return nil
}

func testIMAP(s *tcprunner.TCPSession, noop uint8) bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very bike-sheddy comment, but maybe these functions should not be named with the test prefix to avoid confusion with the Test functions inside of a unit/integration test.

Port string
TLS bool
RawConn *net.Conn
TLSConn *net.Conn
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed out of band the issue about the deadlock in the startTLS method. While it's still unclear to me exactly what the root cause is, one problem in the current code is that these RawConn and TLSConn should not be pointers (net.Conn is an interface), but an actual copy to the relevant connection.

net.Conn makes some guarantees about it being possible to call methods on them also concurrently, which means it has in it some locking to deal with concurrency and is probably what is leading to the deadlock.

I haven't fully understood where exactly this is happening, but I suspect it has something to do with the fact that there is this proxy object managing the two connections.

My suggestion would be to refactor the code to get rid of this proxy object and use the net.Conn objects directly. You don't even need to handle the special case for startTLS, as you can conditionally perform startTLS on the plaintext connection and swap it out for the original net.Conn once you have an encrypted tunnel.

That should simplify the code and hopefully make it more clear where the deadlock is happening.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So i just pushed a new commit with this suggestion. Unfortunately the code is still deadlocking. strace log:


[      5.571522]  Initializing SMTP client
epoll_pwait(3, [], 128, 0, NULL, 0)     = 0
write(5, "\0", 1)                       = 1
futex(0x2482168, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
futex(0x2482168, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
epoll_pwait(3, [], 128, 0, NULL, 0)     = 0
futex(0x2482168, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
epoll_pwait(3, [], 128, 0, NULL, 0)     = 0
nanosleep({tv_sec=0, tv_nsec=3000}, NULL) = 0
write(5, "\0", 1)                       = 1
futex(0x2482168, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
epoll_pwait(3, [], 128, 0, NULL, 0)     = 0
futex(0xc000088148, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x2482168, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
futex(0x2482168, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
futex(0x2482168, FUTEX_WAIT_PRIVATE, 0, NULL

@hellais
Copy link
Member

hellais commented Nov 25, 2022

I have reviewed the PR and done some light testing of it. I think once we resolve the deadlocking issue this is pretty much feature complete WRT being a functioning test.

The other remaining bits that we will have to address before we can merge it into master are:

  1. Do a more thorough review of the data format specification with the rest of the team to ensure it's solid
  2. Write a complete test specification for it over here: https://github.com/ooni/spec (see example spec: https://github.com/ooni/spec/blob/master/nettests/ts-000-example.md)

I think that WRT the hackathon, the challenge is to be considered complete, as it has for sure met what I think can be expected from somebody to do in a few days of work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants