Kushal Das

FOSS and life. Kushal Das talks here.

kushal76uaid62oup5774umh654scnu5dwzh4u2534qxhcbi4wbab3ad.onion

Use DoH over Tor for your Qubes system

I was using my dns-tor-proxy tool in the AppVMs in my QubesOS system. But, at the same time I was trying to figure out how to make it the default DNS system for the whole Qubes.

ahf provided me a shell script showing how he is forwarding the DNS requests to a VPN interface. I modified the same so that all of standard DNS queries become DoH queries over the Tor network.

Setting up sys-firewall

In the following example, I am setting up the sys-firewall service VM. All other AppVMs connected to this VM as netvm will be use dns-tor-proxy without any modification.

Make sure that the template for sys-firewall has the latest Tor installed. You can get it from the official Tor repository.

Download (or build) the latest dns-tor-proxy 0.3.0 release, and put the file (as executable) in /rw/config/ directory.

Next, modify the /rw/config/rc.local file & add the following lines.

systemctl start tor
sh /rw/config/dns.sh
/rw/config/dns-tor-proxy --doh &

As you can see, we are executing another script at /rw/config/dns.sh, which has the following content. Remember to modify the DNS value to the right IP for your sys-firewall vm.


#!/bin/sh

QUBES_DNS_SERVERS="10.139.1.1 10.139.1.2"
DNS=10.137.0.x

# accept DNS requests from the other vms

iptables -I INPUT -i vif+ -p udp --dport 53 -j ACCEPT
iptables -I INPUT -i vif+ -p tcp --dport 53 -j ACCEPT

# Clean up our NAT firewall rules.
iptables --flush PR-QBS --table nat

# We take incoming traffic on TCP and UDP port 53 and forward to
# our DNS server.
for QUBES_DNS_SERVER in ${QUBES_DNS_SERVERS} ; do
    iptables --append PR-QBS --table nat --in-interface vif+ --protocol tcp --destination "${QUBES_DNS_SERVER}" --dport 53 --jump DNAT --to-destination "${DNS}":53
    iptables --append PR-QBS --table nat --in-interface vif+ --protocol udp --destination "${QUBES_DNS_SERVER}" --dport 53 --jump DNAT --to-destination "${DNS}":53
done

# Log *other* DNS service connections. This part is optional, but ensures that
# you can monitor if one of your VM's is making any traffic on port 53 with
# either TCP or UDP. If you want to log *every* DNS "connection", including the
# ones to QUBES_DNS_SERVERS, you can either move these commands up before the
# for-loop in this file or change the --apend option to be an --insert instead.
iptables --append PR-QBS --table nat --in-interface vif+ --protocol tcp --dport 53 --jump LOG --log-level 1 --log-prefix 'DNS Query: '
iptables --append PR-QBS --table nat --in-interface vif+ --protocol udp --dport 53 --jump LOG --log-level 1 --log-prefix 'DNS Query: '

Now, restart your sys-firewall vm. And you are all set for your DNS queries.

Introducing pyage-rust, a Python module for age encryption

age is a simple, modern and secure file encryption tool, it was designed by @Benjojo12 and @FiloSottile.

An alternative interoperable Rust implementation is available at github.com/str4d/rage

pyage-rust is a Python module for age, this is built on top of the Rust crate. I am not a cryptographer, and I prefer to keep this important thing to the specialists :)

pyage-rust demo

Installation

I have prebuilt wheels for both Linux and Mac, Python 3.7 and Python 3.8.

python3 -m pip install pyage-rust==0.1.0

Please have a look at the API documentation to learn how to use the API.

Building pyage-rust

You will need the nightly rust toolchain (via https://rustup.rs).

python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install requirements-dev.txt
maturin develop && maturin build

Missing items in the current implementation

I am yet to add ways to use ssh keys or alias features, but those will come slowly in the future releases.

Another try at a new Python module for OpenPGP aka johnnycanencrypt

Using OpenPGP from Python is a pain. There are various documentation/notes on the Internet explaining why, including the famous one from isis agora lovecraft where they explained why they changed the module name to pretty_bad_protocol.

sequoia-pgp is a Rust project to do OpenPGP from scratch in Rust, and as library first approach. You can see the status page to see how much work is already done.

Using this and Pyo3 project I started writing an experimental Python module for OpenPGP called Johnny Can Encrypt.

I just did an release of 0.1.0. Here is some example code.

>>> import johnnycanencrypt as jce
>>> j = jce.Johnny("secret.asc")
>>> data = j.encrypt_bytes("kushal 🐍".encode("utf-8"))
>>> print(data)
-----BEGIN PGP MESSAGE-----

wcFMAwhsWpR1vDokAQ//UQGjrmmPLP0Td8pELf8XZEPh6fY9Xad6XHH6vQjGwvjG
36kK8ejRqyLbZpwVOO1FUfiZt6AyaeIEeEagoolMxmFl67mWBHsw5Z2NUPhydAwJ
EX+VdFn6CtRzQ0xG3T7rOCrsR50COO13gc4fIAn7Rxj1DyjqlFvur10FNnxRm0iJ
jnOwPnWVWKwoROzevfQd1Oef0n4nbkDUuyrS9oHSRFhFF/9I9bGtJhho0VIIcmFG
YVkhR0+QROTZ4edKotUg0R3UUfHmfwT0XcybGWMG/8Nh3W8pYuxwDdtbSMNDZzxu
o9TdpLrgoRIkhyGmuYWrURrRN1hmce5B6XOagWu7aKL7pFhP7Vd6LLoliDwY4G6x
1yKHbSo/1FEof7WBDujCksuVedUO8Gs9giitR/p/U9PBadeyiW0CKTYiTURkkNiF
g79lVfbmM54eZ9zmU+PraVNpekYXvH3o+lvXd5T39mo4Y/dv2fDCjo2aiZ/bE56Q
yn/0Hhmj2ikrsUk3NcuhO4zxt+VLctJt+lfk+R2hal+6NTaRkREdprPp4ltAt/bm
8xwBmqp+FDdxGgY+ItJkG69vsIf4WpPsvBI37fVbeYqrPsaz9NGlz2QKdfQvaH7j
R7rgxf24H2FjbbyNuHF3tJJa4Kfpnhq4nkxA/EdRP9JcVm/X568jLayTLyJGmrbS
PAHlVMLSBXQDApkY+5Veu3teRR5M2BLPr7X/gfeNnTlbZ4kF5S+E+0bjTjrz+6oo
dcsnTYxmcAm9hdPjng==
=1IYb
-----END PGP MESSAGE-----

>>> result = j.decrypt_bytes(data.encode("utf-8"), "mysecretpassword")
>>> print(result.decode("utf-8"))
kushal 🐍

The readme of the project page has build instruction, and more details about available API calls. We can create new keypair (RSA4096). We can encrypt/decrypt bytes and files. We can also sign/verify bytes/files. The code does not have much checks for error handling, this is super early stage.

You will need nettle (on Fedora) and libnettle on Debian (and related development packages) to build it successfully.

I published wheels for Debian Buster (Python3.7), and Fedora 32 (Python3.8).

Issues in providing better wheels for pip install

The wheels are linked against system provided nettle library. And every distribution has a different version. Means even if I am building a python3.7 wheel on Debian, it will not work on Fedora. I wish to find a better solution to this in the coming days.

As I said earlier in this post, this is just starting of this project. It will take time to mature for production use. And because of Sequoia, we will have better defaults of cipher/hash options.

dns-tor-proxy 0.2.0 aka DoH release

I just now released 0.2.0 of the dns-tor-proxy tool. The main feature of this release is DNS over HTTPS support. At first I started writing it from scratch, and then decided to use modified code from the amazing dns-over-https project instead.

Demo

demo of the DoH support in the tool

✦ ❯ ./dns-tor-proxy -h
Usage of ./dns-tor-proxy:
      --doh                 Use DoH servers as upstream.
      --dohaddress string   The DoH server address. (default "https://mozilla.cloudflare-dns.com/dns-query")
  -h, --help                Prints the help message and exists.
      --port int            Port on which the tool will listen. (default 53)
      --proxy string        The Tor SOCKS5 proxy to connect locally, IP:PORT format. (default "127.0.0.1:9050")
      --server string       The DNS server to connect IP:PORT format. (default "1.1.1.1:53")
  -v, --version             Prints the version and exists.
Make sure that your Tor process is running and has a SOCKS proxy enabled.

Now you can pass --doh flag to enable DoH server usage, by default it will use https://mozilla.cloudflare-dns.com/dns-query. But you can pass any server using --dohaddress flag. I found the following servers are working well over Tor.

  • https://doh.libredns.gr/dns-query
  • https://doh.powerdns.org
  • https://dns4torpnlfs2ifuz2s2yf3fc7rdmsbhm6rw75euj35pac6ap25zgqad.onion/dns-query
  • https://dnsforge.de/dns-query

The release also has a binary executable for Linux x86_64. You can verify the executable using the signature file available in the release page.

Introducing dns-tor-proxy, a new way to do all of your DNS calls over Tor

dns-tor-proxy is a small DNS server which you can run in your local system along with the Tor process. It will use the SOCKS5 proxy provided from Tor, and route all of your DNS queries over encrypted connections via Tor.

By default the tool will use 1.1.1.1 (from Cloudflare) as the upstream server, but as the network calls will happen over Tor, this will provide you better privacy than using directly.

In this first release I am only providing source packages, maybe in future I will add binaries so that people can download and use them directly.

Demo

In the following demo I am building the tool, running it at port 5300, and then using dig to find the IP addresses for mirrors.fedoraproject.org and python.org.

demo of dns tor proxy

The -h flag will show you all the available configurable options.

./dns-tor-proxy -h

Usage of ./dns-tor-proxy:
  -h, --help            Prints the help message and exists.
      --port int        Port on which the tool will listen. (default 53)
      --proxy string    The Tor SOCKS5 proxy to connect locally,  IP:PORT format. (default "127.0.0.1:9050")
      --server string   The DNS server to connect IP:PORT format. (default "1.1.1.1:53")
  -v, --version         Prints the version and exists.
Make sure that your Tor process is running and has a SOCKS proxy enabled.

Setting up a personal DoH server

DoH is a hot discussion point in both the privacy and DNS people. There are many criticisms, including encryption support of the clients or still trusting a third party. There is an excellent talk from Bert Hubert on this topic.

In this post, we will learn how to set up our own personal DoH server. I am not posting any tips on the IPTABLES rules, you should be able to add those based on what all services you run on the server.

We will use unbound as the recursive DNS server in our setup. In the server, we can easily install it via the OS package management (apt/dnf).

Getting the root name servers' details

We should get a fresh copy of the root name servers' details, and then you can have a cron job every six months to get a fresh copy.

curl --output /etc/unbound/root.hints https://www.internic.net/domain/named.cache

Setting up unbound

I am using the following configuration

# Unbound configuration file for Debian.
#
# See the unbound.conf(5) man page.
#
# See /usr/share/doc/unbound/examples/unbound.conf for a commented
# reference config file.
#
# The following line includes additional configuration files from the
# /etc/unbound/unbound.conf.d directory.
include: "/etc/unbound/unbound.conf.d/*.conf"
server:
        root-hints: "/etc/unbound/root.hints"
        interface: 127.0.0.1
        access-control: 0.0.0.0/0 allow
        use-syslog: yes

Then you can get the configuration checked via the unbound-checkconf command.

# unbound-checkconf
unbound-checkconf: no errors in /etc/unbound/unbound.conf
systemctl enable unbound
systemctl start unbound  

Setup Nginx along with certbot

Setup Nginx and use the certbot tool to get the SSL certificate. Here is the configuration I am using, the main point to notice the upstream section.

upstream dns-backend {
    server 127.0.0.1:8053;
    keepalive 30;
}
server {
    listen 80;
    listen [::]:80;

    location /.well-known/acme-challenge {
        alias /var/www/leftover;
    }
    root /var/www/html;

    server_name yourdomain;
    return 301 https://$host;
}

server {
    listen 443 ssl http2;

    # if you wish, you can use the below line for listen instead
    # which enables HTTP/2
    # requires nginx version >= 1.9.5
    # listen 443 ssl http2;

    server_name yourdomain;
    index index.html;

    ssl_certificate /etc/letsencrypt/live/yourdomain/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain/privkey.pem;

    # Turn on OCSP stapling as recommended at
    # https://community.letsencrypt.org/t/integration-guide/13123
    # requires nginx version >= 1.3.7
    ssl_stapling on;
    ssl_stapling_verify on;

    # modern configuration. tweak to your needs.
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
    ssl_prefer_server_ciphers on;


    # Uncomment this line only after testing in browsers,
    # as it commits you to continuing to serve your site over HTTPS
    # in future
    # add_header Strict-Transport-Security "max-age=31536000";

    rewrite ^(.*).php https://www.youtube.com/watch?v=dQw4w9WgXcQ last;
    # maintain the .well-known directory alias for renewals
    location /.well-known {

        alias /var/www/yourdomain/.well-known;
    }

        location / {
            root   /var/www/html;
            index  index.html;
        }
    location /dns-query {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;
        proxy_set_header Connection "";
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_redirect off;
        proxy_set_header        X-Forwarded-Proto $scheme;
        proxy_read_timeout 86400;
        proxy_pass http://dns-backend/dns-query;
    }

}

Then start the Nginx server.

Setting up the DoH server

You will have to setup golang as we will have to compile the project. First step is always the git clone :)

git clone https://github.com/m13253/dns-over-https.git
cd dns-over-https
make
make install 

As next step we will modify the configuration file of the dns-over-https tool at /etc/dns-over-https/doh-server.conf. Now, there are many different configuration options available, I am using only a small part of it. Check the github repo for an uptodate commented configuration example.

listen = [
    "127.0.0.1:8053",
    "[::1]:8053"
]
local_addr = ""
cert = ""
key = ""
path = "/dns-query"
upstream = [
        "udp:127.0.0.1:53"
]
timeout = 10
tries = 3
verbose = false
log_guessed_client_ip = false

We are asking the tool to talk to the unbound running on the same server. Next, we can start and enable the service. Remember to check the logs for any errors due to typos.

systemctl restart doh-server
systemctl enable doh-server

Testing the setup

You can test the setup by making a call using curl command and using Python's json module to give you a readable output.

curl -s "https://yourdomain/dns-query?name=dgplug.org.org&type=A" | python3 -m json.tool

You can now use the server in the General -> Network Settings section of Firefox.

setting up firefox

Access Riseup email over Onion service

Email service (📧) is another excellent example that can be accessed safely over Tor Onion services. This is in particular useful in places where people in power do not like their citizens accessing privacy-focused email providers. I know, you must be thinking about your own country, but no worries, we all are in the same place :)

In this post, I will explain how one can access their emails via IMAP, and send using SMTP over onion services. I am taking Riseup as an example because they provide this option to the users, and also because I personally use their service. This document assumes that you already have tor service running on your system.

Riseup Tor Onion services address

Riseup and Tor

Riseup has a page listing all the Onion service addresses they provide. You can also verify the signed address from the signed file in the same page. For the rest of this post, we will use 5gdvpfoh6kb2iqbizb37lzk2ddzrwa47m6rpdueg2m656fovmbhoptqd.onion as the address for both IMAP and SMTP services. In the normal Internet, those are imap.riseup.net and smtp.riseup.net.

Getting the SSL certificate for the service for verification

Riseup uses Let's Encrypt for the SSL certificates. We have to pin them for the above-mentioned onion address so that we can use them in our system.

mkdir -p ~/.cert
torify openssl s_client -connect 5gdvpfoh6kb2iqbizb37lzk2ddzrwa47m6rpdueg2m656fovmbhoptqd.onion:993 -showcerts 2>&1 < /dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | sed -ne '1,/-END CERTIFICATE-/p' > ~/.cert/riseuponion.pem

openssl x509 -in .cert/riseuponion.pem -noout -sha256 -fingerprint
SHA256 Fingerprint=C6:BB:7B:04:97:54:05:65:76:81:4D:56:22:CE:50:6C:91:53:D3:3E:27:95:CC:C9:B8:B7:19:A5:E9:31:7D:15

The first command fetches the SSL certification from the given onion addresses, and stores it in the ~/.cert/riseuponion.pem file. The second command gives us the fingerprint for the same. You can verify these values by running the command against imap.riseup.net:993 and comparing the values.

By the way, remember that these values will change every 3 months (like any other Let's Encrypt certificate).

Setting up mbsync for IMAP access of the emails

I prefer to use the mbsync command from the imap package. The following the configuration for the same.

IMAPAccount riseup
# Address to connect to
Host 5gdvpfoh6kb2iqbizb37lzk2ddzrwa47m6rpdueg2m656fovmbhoptqd.onion
Port 993
User <my full email address without angle brakets>
PassCmd "/usr/bin/pass riseup"
# Use SSL
AuthMechs PLAIN
SSLType IMAPS
SSLVersions TLSv1 TLSv1.1 TLSv1.2
CertificateFile /home/kdas/.cert/riseuponion.pem

IMAPStore riseup-remote
Account riseup

MaildirStore riseup-local
# The trailing "/" is important
Path ~/.imap-mail/riseup/
Inbox ~/.imap-mail/riseup/Inbox

Channel riseup
Master :riseup-remote:
Slave :riseup-local:
# Exclude certain things
# Or include everything
Patterns *
# Automatically create missing mailboxes, both locally and on the server
Create Both
# Save the synchronization state files in the relevant directory
SyncState *

You can notice that I am using the CertificateFile key to point to the certificate we downloaded previously.

Now, I can sync the emails using the torify along with the regular mbsync command.

torify mbsync -a riseup 

Setting up msmtp to send emails

The following is my msmtp configuration

# riseup
account riseup
host 5gdvpfoh6kb2iqbizb37lzk2ddzrwa47m6rpdueg2m656fovmbhoptqd.onion
port 587
auth on
proxy_host 127.0.0.1
proxy_port 9050
tls on
tls_fingerprint C6:BB:7B:04:97:54:05:65:76:81:4D:56:22:CE:50:6C:91:53:D3:3E:27:95:CC:C9:B8:B7:19:A5:E9:31:7D:15
user <my full email address without angle brakets>
passwordeval "/usr/bin/pass riseup"
maildomain riseup.net
from <my full email address without angle brakets>

One thing to notice that msmtp actually allows us to directly mention the tor socks proxy details in the configuration file. And then in my mutt configuration, I mentioned

set sendmail="/usr/bin/msmtp -a riseup"

Onion service v2 deprecation timeline

On Monday June 15, the developers of the Tor Project announced the initial plan for the deprecation of Onion services v2. You can identify v2 addresses easily as they are only 16 character long, where as the v3 addresses are 56 character long.

Why?

The v2 services used RSA1024, where as v3 uses ed25519, means better cryptography. We can also have offline keys for the onion service. You can read all other benefits in the v3 spec.

Timeline

According to the email to the list, the following the current timeline:

  • On 2020-09-15 with 0.4.4.x release Tor will start informing v2 onion service operators that v2 is deprecated.
  • On 2021-07-15 with 0.4.6.x release Tor will stop supporting v2 onion addresses, and all related source code will be removed.
  • On 2021-10-15 there will be a new stable version release which will disable using v2 onion services on the Tor network.

How can you prepare as an Onion service provider?

If you are using/providing any v2 onion service, you should enable v3 service for the same service. This will help you to test your v3 configuration while keeping the v2 on, and then you can retire your v2 address. If you need help in setting authenticated v3 service, you can follow this blog post. I wrote another post which explains how can you generate the keys using Python cryptography module.

Read the full announcement in the list archive.

Curious case of image based email signatures and Kmail

We already talk about why HTML emails are bad, but that is the default in most of the email service providers. HTML emails means some code is getting executed and rendered on your system. Maybe on a browser, or on a desktop email client.

Many people do not use any HTML tag in their emails, but then they have fancy email signatures. A lot of time they have fancy image generated on a website and they use the generated image URL as signature. This means every time someone opened the email (with HTML rendering on) the third party company will be able to track those usages. We don't know what happens next to all of these tracking information.

Last week I was trying out various desktop email clients available on Fedora 32, and noticed a strange thing on Kmail/Kontact, the email client of KDE. I run my Unoon tool to monitor all processes for any network connection on system. And, suddenly it popped a notification about Kmail connecting to mysignatures.io. I was surprised for a second, as Kmail also disables loading of any remote resource (say images) and does not render HTML email by default.

Screenshot of Unoon

Then I figured that if I click on reply button (the compose window), it fetches the image from the signature (or any <img> tag). This means the HTML is getting rendered somehow, even if it is not showing to the user. After I filed a bug upstream, I also pinged my friend ADE. He helped to reproduce it and also find more details on the same. Now, we are waiting for a fix. I hope this does not involve JS execution during that internal rendering.

I also checked for same behavior in Thunderbid, and it does not render in similar way.

Onion location and Onion names in Tor Browser 9.5

Yesterday the Tor Browser 9.5 was released. I am excited about this release for some user-focused updates.

Onion-Location header

If your webserver provides this one extra header Onion-Location, the Tor Browser will ask the user if they want to visit the onion site itself. The user can even mark to visit every such onion site by default. See it in action here.

To enable this, in Apache, you need a configuration line like below for your website’s configuration.

Onion location demo

Header set Onion-Location "http://your-onion-address.onion%{REQUEST_URI}s"

Remember to enable rewrite module.

For nginx, add the following in your server configuration.

add_header Onion-Location http://<your-onion-address>.onion$request_uri;

URL which we can remember aka onion names

This is the first proof of concept built along with Freedom of the Press Foundation (yes, my team) and HTTPS Everywhere to help people to use simple names for onion addresses. For example, below, you can see that I typed theintercept.securedrop.tor.onion on the browser, and that took us to The Intercept’s SecureDrop address.

Onion name