Kushal Das

FOSS and life. Kushal Das talks here.

kushal76uaid62oup5774umh654scnu5dwzh4u2534qxhcbi4wbab3ad.onion

Introducing pyarti, python module for the Tor Project

python3 -m pip install pyarti

pyarti is a Python module written in Rust using Arti from the Tor Project. Right now pyarti is in the initial stage, and you can create a SOCKS5 proxy object, and pass any Python connection via it. The following example creates a default proxy at port 9150 , and then verifies that the connection works. Finally we fetch a web page and print the text.

from pyarti import OnionProxy
import httpx
from httpx_socks import SyncProxyTransport

URL = "https://httpbin.org/get"

p = OnionProxy()
assert p.verify(blocking=True)
# Now we can use the proxy
transport = SyncProxyTransport.from_url("socks5://127.0.0.1:9150")
with httpx.Client(transport=transport) as client:
    res = client.get(URL)
    assert res.status_code == 200
    print(res.text)

In the coming months I am hoping to add more detailed API to the module. For now you can read the documentation.

Johnnycanencrypt 0.9.0 release

3 days ago I released Johnnycanencrypt 0.9.0. Here is the changelog:

- Adds `setuptools-rust` as build system.
- Key.uids now contains the certification details of each user id.
- `merge_keys` in rjce now takes a force boolean argument.
- `certify_key` can sign/certify another key by both card and on disk primary key.

The first biggest change is related to build system, now we are using setuptools-rust to build. This change happened as dkg is working towards packaging the module for Debian.

The other big change is about certifying someone's key. We can use the primary key (either on disk or on Yubikey) to do the signing.

k = ks.certify_key(
    my_key,
    k,
    ["Kushal Das <kushaldas@gmail.com>", "Kushal Das <kushal@fedoraproject.org>"],
    jce.SignatureType.PositiveCertification,
    password=password,
)

In the above example I am signing two user ids of the key k using my_key with a PositiveCertification.

johnnycanencrypt 0.7.0 released

Today I released Johnnycanencrypt 0.7.0. It has breaking change of some function names.

  • create_newkey renamed to create_key
  • import_cert renamed to import_key

But, the major work done are in few different places:

  • Handling errors better, no more normal Rust panics, instead providing better Python exceptions as CryptoError.
  • We can now sign bytes/files in both detached & in normal compressed binary form.
  • Signature can be done via smartcards, and verification works as usual.

In the Github release page you can find an OpenPGP signature, which you can use to verify the release. You can also verify via sigstore.

SIGSTORE_LOGLEVEL=debug python -m sigstore verify --cert-email mail@kushaldas.in --cert-oidc-issuer https://github.com/login/oauth johnnycanencrypt-0.7.0.tar.gz
DEBUG:sigstore._cli:parsed arguments Namespace(subcommand='verify', certificate=None, signature=None, cert_email='mail@kushaldas.in', cert_oidc_issuer='https://github.com/login/oauth', rekor_url='https://rekor.sigstore.dev', staging=False, files=[PosixPath('johnnycanencrypt-0.7.0.tar.gz')])
DEBUG:sigstore._cli:Using certificate from: johnnycanencrypt-0.7.0.tar.gz.crt
DEBUG:sigstore._cli:Using signature from: johnnycanencrypt-0.7.0.tar.gz.sig
DEBUG:sigstore._cli:Verifying contents from: johnnycanencrypt-0.7.0.tar.gz
DEBUG:sigstore._verify:Successfully verified signing certificate validity...
DEBUG:sigstore._verify:Successfully verified signature...
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): rekor.sigstore.dev:443
DEBUG:urllib3.connectionpool:https://rekor.sigstore.dev:443 "POST /api/v1/index/retrieve/ HTTP/1.1" 200 85
DEBUG:urllib3.connectionpool:https://rekor.sigstore.dev:443 "GET /api/v1/log/entries/362f8ecba72f4326972bc321d658ba3c9197b29bb8015967e755a97e1fa4758c13222bc07f26d27c HTTP/1.1" 200 None
DEBUG:sigstore._verify:Successfully verified Rekor entry...
OK: johnnycanencrypt-0.7.0.tar.gz

I took 8 months for this release, now time to write some tools to use it in more places :)

Tor sysadmin 101 workshop for new relay operators

Tor log

On 4th June, at 19:00 UTC, we are doing an online workshop to help out new relay operators. If you ever wanted to help the Tor Project, or just curious about what is required to become a relay/bridge operator, you should join into the workshop.

The workshop is specially geared towards folks who are new to the land of Internet facing services. You will get to chat with many other operators and people from the Tor Project, and ask any doubts you have.

Register for the event, and share the news at your local groups/lists. Ask your friends to join :)

Securing verybad web application with only systemd

In my last blog post I talked about verybad web application. It has multiple major security holes, which allows anyone to do remote code execution or read/write files on a server. Look at the source code to see what all you can do.

I am running one instance in public http://verybad.kushaldas.in:8000/, and then I asked twitter to see if anyone can get access. Only difference is that this service has some of the latest security mitigation from systemd on a Fedora 35 box.

The service is up for a few days now, a few people tried for hours. One person managed to read the verybad.service file after a few hours of different tries. This allowed me to look into other available options from systemd. Rest of the major protections are coming from DynamicUser=yes configuration in systemd. This enables multiple other protections (which can not be turned off). Like:

  • SUID/SGID files can not be created or executed
  • Temporary filesystem is private to the service
  • The entire file system hierarchy is mounted read-only except a few places

systemd can also block exec mapping of shared libraries or executables. This way we can block any random command execution, but still allow the date command to execute.

Please have a look at the man page and learn about many options systemd now provides. I am finding this very useful as it takes such small amount of time to learn and use. The credit goes to Lennart and rest of the maintainers.

Oh, just in case you are wondering, for a real service you should enable this along with other existing mechanisms, like SELinux or AppArmor.

Introducing Very Bad Web Application

I am planning to add a few chapters on securing services in my Linux Command Line book. But, to make it practical & hands on, I needed one real application which the readers can deploy and secure. I needed something simple, say one single binary so that it becomes easier to convert it into a proper systemd service.

I decided to write one in Rust :) This also helps to showcase that one can write totally insecure code even in Rust (or any other language). Let me introduce Very Bad Web application. The README contains the build instructions. The index page shows the available API.

Issues in the service

The service has the following 3 major issues:

  • Directory traversal
  • Arbitrary file read
  • Remote code execution

I am currently updating the systemd (services) chapter in my book to show how to secure the service using only the features provided by the latest systemd. In future I will also have chapters on SELinux and AppArmor and learn how to secure the service using those two options.

If you think I should add some other nice security holes in this application, please feel free to suggest :)

eduGAIN Key Signing Ceremony

eduGAIN is a interfederation joining academic identity federations around the world, including over 4781 identify providers & 3519 service providers. The project was initiated by by the GÉANT research and education networking community in Europe.

On 8th of March, there was a key signing ceremony at Sunet office. This was my first chance to be able to attend one such ceremony in real life :)

hardware

The day before there was test run where I acted like a participant plugging in various cables/Yubikeys. A specific APU based airgapped box was used to generate the key, and it was talking over a serial port. Which in turn was split into two parts, one on a Mac where Björn Mattsson was doing the actual typing of the commands, and the other side was connected to a dot matrix printer. The printer printed every command & output of the ceremony. Apparently there were only 3 such paper rolls were available in whole of Stockholm.

Representants for the eduGAIN service flew in from different parts of EU as witness (there were many more online witnesses) and also participated in the ceremony. We also had many people present in the room (including Leif & I). Leif started describing the steps as they happened. At the end there were two copies of the keys & passphrase were made, and both the copies went to vaults of separate countries. The required material was also synced with the HSM cluster with the help of a super long cable :)

working on HSM

At the end of the ceremony the witnesses signed the printer pages containing the output.

signing the paper

This was a fun but important event for me to watch. The keys are generated for the next 20 years, so a lot of things will change in the world by the time we will have to do it again :) You can watch it all on Youtube.

Targeted WebID for privacy in Solid

In my last post I talked about the privacy issues from static public WebID in Solid. In this post I am trying to explain a way to preserve privacy, I will later submit a proposal (after figuring out how to) to change/update the original SPECs as required.

Targeted WebID for each unique client

Instead of returning the same unique WebID, the OP can return targeted WebID based on the client asking for the information. This will remain the same for every unique client and user, and can also be computed in future. This way every service accessing a Solid Pod server, will see a different unique URL for WebID, and those can not be used to co-relate the information.

We will have to update the OP (IDP) so that either it itself can calculate (or ask a different service) for the unique WebID every time.

Below I modified the official example flow to show (in step 19 and 20) how this can be achieved.

Sequence diagram

This brings in the question of how the user will learn/see all the available/used WebIDs for themselves.

That can be done by marking one client as the primary viewer/editor for the user, you can think it like a wallet. This solid application will be able to get the original unique WebID, and using that in the user's pod the wallet can find all the issued WebIDs. This goes into the implementation details of the pod server. Maybe all targeted WebIDs (& related pods) will be stored in a different namespace, maybe not.

I will write more in the next post.

Solid Project, WebID and privacy

In my last post I mentioned about Solid Project, and while digging more into it I got more questions on privacy issues. Let us break it down from beginning:

What is Solid Project & WebID?

Solid is a specification that lets people store their data securely in decentralized data stores called Pods. Pods are like secure personal web servers for data.

If you dig into the the actual specification, you will find this paragraph about WebID.

In line with Linked Data principles, a WebID is a HTTP URI that, when dereferenced, resolves to a profile document that is structured data in an RDF 1.1 format. This profile document allows people to link with others to grant access to identity resources as they see fit. WebIDs underpin Solid and are used as a primary identifier for Users in this specification.

One person can have more than one WebID (say one for work, one for personal details, one from Government). And the services will use the WebID you provide, or provided by your digital wallet (some Solid application running somewhere) which in turn comes from a Government provided service. The WebIDs provided by an agency (private or government) can be verified based on the issuer.

Now this WebID is the unique thing in the Solid world, the core of the Linked Data. If one service can get the WebID for someone and identify the person, they (or any other service) can corelate the same WebID usage in all other services. You don’t need magical code, just find the unique WebID usage.

For government issued WebID, this becomes even easier, as the person has no choice of providing the ID. Instead whatever mechanism the agency use to identify, will provide the same WebID every time (after identifying to the IDP service). One similar usage is documented in flow diagram here.

In my mind this is a privacy nightmare. The WebID spec has section about security considerations, but nothing about privacy implications.

One way of dealing with this could be having a separate service providing random (but unique to each to application asking for the resource based on aud) pseudo WebIDs to the IDP, and IDP provides it back to the client (wallet). I will write a separate blog post with sequence diagram to explain it better. Maybe it will work, maybe not.

Using Python to access a Solid Pod

solid logo

From the project website:

Solid is a specification that lets people store their data securely in decentralized data stores called Pods. Pods are like secure personal web servers for your data.

We can host these Pods in personal servers or at any provider. Everything is tied up based on the user identity, called WebID. It is an HTTP URI described as RDF document.

You can decide who/what can access your data. Applications/humans can use Solid authentication and identify to the Pod server to access the data using open protocols.

How to get a Pod?

The website lists current vendors who provide Pod services. If you want to play around locally, you can run community server on your local system. Or just create an account at solidcommunity.net, and use that to learn more.

Using Python with Solid

We already have a solid-file Python module. You can install it via pip in a virtual environment.

$ python3 -m venv .venv
$ source .venv/bin/activate
$ python3 -m pip install solid-file

For the rest of the example code, I am going to use my Pod at the solidcommunity.net, feel free to replace the username/password and URL values in the code as required.

USERNAME = "kushaldas"
PASSWORD = "******************************************"

IDP = 'https://solidcommunity.net'
POD_ENDPOINT = "https://kushaldas.solidcommunity.net/public/"

from solid.auth import Auth
from solid.solid_api import SolidAPI

auth = Auth()
api = SolidAPI(auth)
auth.login(IDP, USERNAME, PASSWORD)

Here we are importing the module, creating an Auth object and identify using username and password.

Then we will check if a folder exist or not (it does not exist yet), and create the folder in this case.

folder_url = f"{POD_ENDPOINT}/languages/"
if not api.item_exists(folder_url):
    print(api.create_folder(folder_url))

The output is <Response [201 Created]>.

Next, we create two text files.

data = io.BytesIO("I ❤️ 🦀".encode("utf-8"))
file_url = f"{folder_url}hello.txt"
print(api.put_file(file_url, data, 'text/plain'))
data = io.BytesIO(b"Already 10 years of SOPA blackout")
msg_url = f"{folder_url}message.txt"
print(api.put_file(msg_url, data, 'text/plain'))

We can then list all of the items under our subfolder.

folder_data = api.read_folder(folder_url)
files = "\n".join(list(map(lambda x: x.name, folder_data.files)))
print(f'Files in the folder: \n{files}')

We can then try to read one of the files we just now created.

resp = api.get(file_url)
print(f"**{resp.text}**")

Output:

Files in the folder: 
hello.txt
message.txt

Why am I looking at Solid?

Solid as the specification is evolving along with the community. One usecase for any government organization would be if the citizens can control the access to their own data, and people can authorize who gets to access their data. Solid can become answer to that question. The specification is loose enough to allow building things easily on top of it.

I would love to see Python as a major part of this ecosystem. The solid-file project maintainers are doing a great job. But, we need more related projects, including proper examples of various usecases. Maybe a server too.