Browser remote connections test

A basic study was made on how some web browsers compare in terms of network metadata leakage. Asking questions like: What do we learn when we fully intercept a local browser and inspect the traffic sent in and out as part of just starting it? How efficient are their different approaches in ontent filtering? How much telemetry is going on? What can someone on the other end of the Carbon Fibre learn about us?

A testing framework was built and used in order to make the results reproducible and extensible.

The ambition is that others will be able to take inspiration and perhaps even directly building on this in order to allow users to understand their own software and make educated decisions.

Methodology

Environment

The tests were carried out with a set of containers:

• Proxy. Debian 13.3 with:

• Two network interfaces

• internal facing the browser

• public connected to internet

• mitmproxy terminating TLS with a self-signed certificate

• Browser. Debian 13.3 with:

• The self-signed cert injected in system CA trust store

• Browser under test running

• Latest release installed from public official download

• Using Proxy for outgoing connections and DNS

• Proxy’s mitmproxy configured as outgoing http(s) proxy

• Other processes and system services turned off to a minimum; no other processes to taint results

This should allow us to consider the impact of browser choice with regards to:

• Fingerprintability and metadata leakage from ISP perspective (tcpdump data)
• Fingerprintability from DNS operator perspective (tcpdump + DNS data)
• Fingerprintability and metadata leakage from any integrated third-party services the browser is using (mitmproxy data)

Tested browsers

To be as representative as possible, we use official releases from the projects. The exception is Debian firefox-esr which is treated as a separate browser from upstream Firefox.

• Firefox 147.0.1
• Debian firefox-esr 140.7.0esr-1
• Konform Browser 140.7.0esr-105
• LibreWolf 147.0.1
• Mullvad Browser 15.0.4
• Zen Browser 1.18b Browsers that I also wanted to test and might include later

These not being tested was simply a question of time.

• Brave 1.86.142 starts fine but keeps crashing in the container environment. Will need a little debugging.

• Floorp 12.10.2

• GNU IceCat 140.7.0-gnu1

• Waterfox 6.6.8

• Chromium

• Ungoogled Chromium Not considered

• Google Chrome: Not open source and prior experience already disqualified it

• In future might be interesting to include for reference next to (Ungoogled) Chromium

• Microsoft Edge: Not open source and prior experience already disqualified it

Browser configuration

As far as possible we want to be sampling a representative stock configuration and keep customizations and tweaking at a minimum. The following changes have been made for the sake of enabling the study:

• Use mitmproxy as proxy

• Trust mitmproxy CA cert and silently accept TLS validation errors

• Allow preinstalling system extensions from disk

• Motivation: LibreWolf will download and install uBlock Origin from addons.mozilla.org on first launch. For some ominous reason, the browser does not even attempt to download it over the proxy. We work around this by sideloading the same version from local filesystem in a similar fashion instead.

• Mullvad Browser does not seem to have the same issue and downloads the NoScript .xpi happily. Perhaps Tor Browser tightened up some proxy leak there?

Aside from making things measurable for ourselves, we think these changes should have minimal impact on results.

Limitations

Since we utilize a configured proxy to intercept traffic, we rely on browsers consistently respecting and using configuration. If the browser bypasses the proxy due to either a bug or nefariousness, that will not be detected in this setup. Doing the full transparent interception using rootless podman networking _should_ be possible but it is messy. tcpdump could be set up on the running host. The single proxy approach was chosen for its simplicity.

We saw one potential examples for this: Requests for the following didn’t show up in the proxy and resulted in the feature being unavailable:

• Language translation models for full-page translations (All Firefox browsers)

• uBlock Origin download (LibreWolf)

• See Browser configuration.

The numbers listed under Results can be misleading. To really understand the impact of the requests and responses behind them, we will need to look closer at the actual data inside and be a bit less lazy with our statistics. Hopefully those who find interest in colorful graphs and intuitive visualizations will be able to derive them from examples given.

The testing protocol is a limited scenario. It was chosen to start with a useful baseline against which more involved testing scenarios like "type into the search bar" or "click Accept" can be compared with, as well as a benchmark on base efficiency.

Users are encouraged to try reproducing tests with their browser and see how their custom configuration affects results.

This is intended strictly for study, educational, and testing purposes. Using the provided setup for performing critical tasks would be unsafe and insecure. Using it for bad things is bad, mkay?

This is limited to Linux browsers. Testing mobile browsers would be very interesting but require an entirely different and more demanding setup.

Testing procedure

Basics test environment usage

The browser is started via podman compose. The browser to test is configured with the MITM_BROWSER environment variable.

MITM_BROWSER=firefox-esr podman compose up --force-recreate --build

The below protocol was carried out in each browser.

When Ctrl+C is pressed in the terminal, the browser will quit and request logs should have appeared under ./data/proxy-dumps.

$ ls -lh data/proxy-dumps/firefox-esr/
total 20M
-rw-r--r-- 164K keylog.txt
-rw-r--r--  12M mitmproxy-browser.har      # JSON HAR log
-rw-r--r-- 8.4M mitmproxy-browser.streams  # raw mitmproxy flow data

Protocol 1: Clean

• If onboarding screen appears:

• Click through and choose in order of preference: "Finish", "Skip", "Reject", "Don’t agree", "Not now", "Later". Never agree to opt-in agreements.

• If popups or tabs are auto-opened:

• Scroll to bottom without interacting with the page or content

• If asked to set as default browser: Agree

• Then close

• Open about:preferences

• Scroll to bottom of every view.

• Change Search Suggestions settings back and forth, leaving them in the same state we found them in

• Change Search Engine to Bing, if available.

• Close

• If nags/notifications are demanding your attention:

• Click on them

• Handle like popups

• If Privacy Policy and Terms of Agreement links are shown:

• Scroll to bottom but close and don’t click to agree

• Don’t other call-to-actions

• Don’t "Sign In" or "Sign Up"

• Finally: Press Ctrl+T

• Move the mouse around a little bit without clicking on anything

• Wait at least 1 minute

• Quit browser

Results

Slicing the .har logs with jq is convenient to get a feeling for things.

Number of requests

$ for dir in $(find -maxdepth 1 -mindepth 1 -type d); do cat $dir/mitmproxy-browser.har | jq -r '.log.entries|map(.request.url).[]' | wc -l; echo $dir; done | xargs -n2  | sort -k2  | tr -d './'
119 firefox
81 firefox-esr
0 konform
7 librewolf
30 mullvad-browser
62 zen-browser

Accumulated POST (upload) body sizes

$ for dir in $(find -maxdepth 1 -mindepth 1 -type d); do cat $dir/mitmproxy-browser.har | jq -r '.log.entries|map(.request.bodySize)|flatten|add' ; echo $dir; done | xargs -n2  | sort -k2  | tr -d './'
23766 firefox
225 firefox-esr
0 konform
0 librewolf
0 mullvad-browser
55532 zen-browser

Number of WebSocket messages

$ for dir in $(find -maxdepth 1 -mindepth 1 -type d); do  cat $dir/mitmproxy-browser.har | jq -r '.log.entries|map(select(._webSocketMessages)|._webSocketMessages)|flatten|length' ; echo $dir ; done | xargs -n2 | sort -k2 | tr -d './'
2 firefox
2 firefox-esr
0 konform
0 librewolf
0 mullvad-browser
2 zen-browser

Accumulated WebSocket message byte sizes

user@ddev:~/src/browser-network-privacy-test/data/proxy-dumps$ for dir in $(find -maxdepth 1 -mindepth 1 -type d); do  cat $dir/mitmproxy-browser.har | jq -j '.log.entries|map(._webSocketMessages|select(.)|map(.data))|flatten|.[]' | wc -c ; echo $dir ; done | xargs -n2 | sort -k2 | tr -d './'
171 firefox
171 firefox-esr
0 konform
0 librewolf
0 mullvad-browser
361 zen-browser

Number of cookies

$ for dir in $(find -maxdepth 1 -mindepth 1 -type d); do  cat $dir/mitmproxy-browser.har | jq -r '.log.entries|map([.response.cookies, .response.cookies])|flatten|length' ; echo $dir ; done | xargs -n2 | sort -k2 | tr -d './'
0 firefox
0 firefox-esr
0 konform
4 librewolf
0 mullvad-browser
8 zen-browser

What telemetry endpoints are accessed even if we never opted in?

$ cat firefox/mitmproxy-browser.har | jq '.log.entries|map(.request.url).[]' -r  | sort | uniq | grep -F telemetry.
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/addons/1/82fb762c-6929-4925-bf41-0985c4329d88
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/addons/1/c3ba83d1-8f94-4a53-b02a-507c658443d8
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/baseline/1/9bd4daef-78fb-4b35-939b-070a794efab4
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/health/1/a0049bee-07ba-4fcf-923c-4f3ca491dc61
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/messaging-system/1/221aa8fd-587f-4a4d-84a9-1c9af18652b8
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/messaging-system/1/26aefcea-6e42-44ab-9752-4d73d5566e59
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/messaging-system/1/2cb4aa53-9298-4ac2-8830-9b8d3501fc00
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/messaging-system/1/620b1732-3e0d-4b87-a7f1-f6c7e570c49f
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/messaging-system/1/65ea53c9-1a01-4e7f-acba-19cbc60665fe
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/messaging-system/1/8bac12b3-bd85-40b2-a84f-ee211ad7d265
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/messaging-system/1/e0dc1c42-80de-4da8-a1e7-ab550a686a63
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/messaging-system/1/f7e68193-9d21-4c3d-b29a-04934d33a30f
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/messaging-system/1/f83761dc-a507-496c-bf0a-88bce2e4c0a1
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/messaging-system/1/fb4a69da-b6f3-4f26-ac63-2f005224cd07
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/messaging-system/1/fb9c6e5d-3f7d-45cc-800e-7b4171c49c73
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/newtab/1/6d5d86f0-39e3-4800-8122-90ec0a9b6399
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/newtab/1/d890313e-0b1b-4009-b56f-aabc04dbe384
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/nimbus-targeting-context/1/50868b40-189a-4140-a617-8d7a6a50bc6c
https://incoming.telemetry.mozilla.org/submit/firefox-desktop/usage-reporting/1/55a4528e-d6e7-4177-8195-e9db465762f3
https://telemetry.us.transcend.io/collect

How sensitive are those GUIDs? Look into the request bodies to find out!