Kushal Das

FOSS and life. Kushal Das talks here.


Adding dunder methods to a Python class written in Rust

Last week I did two rounds of my Creating Python modules in Rust workshop. During the second session on Sunday, someone asked if we can create standard dunder methods, say __str__ or __repr__. I never did that before, and during the session I tried to read the docs and implement it. And I failed :)

Later I realized that I should have read the docs carefully. To add those methods, we will have to implement PyObjectProtocol for the Rust structure.

impl PyObjectProtocol for Ros {
    fn __repr__(&self) -> PyResult<String> {
        let cpus = self.sys.get_processors().len();
        let repr = format!("<Ros(CPUS: {})>", cpus);

    fn __str__(&self) -> PyResult<String> {
        let cpus = self.sys.get_processors().len();
        let repr = format!("<Ros(CPUS: {})>", cpus);
>>> from randomos import Ros
>>> r = Ros()
>>> r
<Ros (CPUS: 8)>
>>> str(r)
'<Ros (CPUS: 8)>'

This code example is in the ros-more branch of the code.

Workshop on writing Python modules in Rust April 2020

I am conducting 2 repeat sessions for a workshop on "Writing Python modules in Rust".

The first session is on 16th April, 1500 UTC onwards, and the repeat session will be on 18th April 0900 UTC. More details can be found in this issue.

You don't have to have any prior Rust knowledge. I will be providing working code, and we will go very slowly to have a working Python module with useful functions in it.

If you are planning to attend or know anyone else who may want to join, then please point them to the issue link.

Introducing Tumpa, to make OpenPGP simple with smartcards

Generating OpenPGP keys in an offline air-gapped system and then moving them into a smart card is always a difficult task for me. To remember the steps and command-line options of gpg2 correctly and then following them in the same order is difficult, and I had trouble enough number of times in doing so when I think about someone who is not into the command line that much, how difficult these steps are for them.

While having a chat with Saptak a few weeks ago, we came up with the idea of writing a small desktop tool to help. I started adding more features into my Johnnycanencrypt for the same. The OpenPGP operations are possible due to the amazing Sequoia project.

Introducing Tumpa

The work on the main application started during the holiday break, and today I am happy to release 0.1.0 version of Tumpa to make specific OpenPGP operations simple to use. It uses Johnnycanencrypt inside, and does not depend on the gpg.

Here is a small demo of the application running in a Tails (VM) environment. I am creating a new OpenPGP key with encryption and signing subkeys, and then putting them into a Yubikey. We are also setting the card holder's name via our tool.

Tumpa demo

We can also reset any Yubikey with just a click.

Reset Yubikey

You can download the Debian Buster package for Tails from the release page from Github. You can run from the source in Mac or Fedora too. But, if you are doing any real key generation, then you should try to do it in an air-gapped system.

You can install the package as dpkg -i ./tumpa_0.1.0+buster+nmu1_all.deb inside of Tails.

What are the current available features?

  • We can create a new OpenPGP key along with selected subkeys using Curve25519. By default, the tool will add three years for the expiration of the subkeys.
  • We can move the subkeys to a smart card. We tested only against Yubikeys as that is what we have.
  • We can set the name and public key URL on the card.
  • We can set the user pin and the admin pin of the smart card
  • We can reset a Yubikey.
  • We can export the public key for a selected key.

What is next?

A lot of work :) This is just the beginning. There are a ton of features we planned, and we will slowly add those. The UI also requires a lot of work and touch from a real UX person.

The default application will be very simple to use, and we will also have many advanced features, say changing subkey expiration dates, creating new subkeys, etc. for the advanced users.

We are also conducting user interviews (which takes around 20 minutes of time). If you have some time to spare to talk to us and provide feedback, please feel free to ping us via Twitter/mastodon/IRC.

We are available on #tumpa channel on Freenode. Come over and say hi :)

There are a lot of people I should thank for this release. Here is a quick list at random. Maybe I miss many names here, but you know that we could not do this without your help and guidance.

  • Sequoia team for all the guidance on OpenPGP.
  • Milosch Meriac for providing the guidance (and a ton of hardware).
  • Vincent Breitmoser, for keep explaining OpenKeyChain codebase to me to understand smart card operations
  • Anwesha Das for fixing the CI failures for Johnnycanencrypt, and documentation PRs.
  • Harlo and Micah, for all the amazing input for months.
  • Saptak Sengupta for being the amazing co-maintainer.

Johnnycanencrypt 0.4.0 released

Last night I released 0.4.0 of johnnycanencrypt module for OpenPGP in Python. This release has one update in the creating new key API. Now, we can pass one single UID as a string, or multiple in a list, or even pass None to the key creation method. This means we can have User ID-less certificates, which sequoia-pgp allows.

I also managed to fix the bug so that users can use pip to install the latest release from https://pypi.org.

You will need the rust toolchain, I generally install from https://rustup.rs.

For Fedora

sudo dnf install nettle clang clang-devel nettle-devel python3-devel

For Debian/Ubuntu

sudo apt install -y python3-dev libnettle6 nettle-dev libhogweed4 python3-pip python3-venv clang

Remember to upgrade your pip version inside of the virtual environment if you are in Buster.

For macOS

Install nettle via brew.

Installing the package

❯ python3 -m pip install johnnycanencrypt
Collecting johnnycanencrypt
  Downloading https://files.pythonhosted.org/packages/50/98/53ae56eb208ebcc6288397a66cf8ac9af5de53b8bbae5fd27be7cd8bb9d7/johnnycanencrypt-0.4.0.tar.gz (128kB)
     |████████████████████████████████| 133kB 6.4MB/s
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: johnnycanencrypt
  Building wheel for johnnycanencrypt (PEP 517) ... done
  Created wheel for johnnycanencrypt: filename=johnnycanencrypt-0.4.0-cp37-cp37m-macosx_10_7_x86_64.whl size=1586569 sha256=41ab04d3758479a063a6c42d07a15684beb21b1f305d2f8b02e820cb15853ae1
  Stored in directory: /Users/kdas/Library/Caches/pip/wheels/3f/63/03/8afa8176c89b9afefc11f48c3b3867cd6dcc82e865c310c90d
Successfully built johnnycanencrypt
Installing collected packages: johnnycanencrypt
Successfully installed johnnycanencrypt-0.4.0
WARNING: You are using pip version 19.2.3, however version 20.2.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Now, you can import the module inside of your virtual environment :)

Note: In the future, I may change the name of the module to something more meaningful :)

Fixing errors on my blog's feed

For the last few weeks, my blog feed was not showing up in the Fedora Planet. While trying to figure out what is wrong, Nirik pointed me to the 4 errors in the feed according to the W3C validator. If you don't know, I use a self developed Rust application called khata for my static blog. This means I had to fix these errors.

  • Missing guid, just adding the guid to the feed items solved this.
  • Relative URLs, this had to be fixed via the pulldown_cmark parser.
  • Datetime error as error said "not RFC822" value. I am using chrono library, and was using to_rfc2822 call. Now, creating by hand with format RFC822 value.
  • There is still one open issue dependent on the upstream fix.

The changes are in the git. I am using a build from there. I will make a release after the final remaining issue is fixed.

Oh, I also noticed how bad the code looks now as I can understand Rust better :)

Also, the other Planets, like Python and Tor, are still working for my feed.

Updates from Johnnycanencrypt development in last few weeks

In July this year, I wrote a very initial Python module in Rust for OpenPGP, Johnnycanencrypt aka jce. It had very basic encryption, decryption, signing, verification, creation of new keys available. It uses https://sequoia-pgp.org library for the actual implementation.

I wanted to see if I can use such a Python module (which does not call out to the gpg2 executable) in the SecureDrop codebase.

First try (2 weeks ago)

Two weeks ago on the Friday, when I sat down to see if I can start using the module, within a few minutes, I understood it was not possible. The module was missing basic key management, more more refined control over creation, or expiration dates.

On that weekend, I wrote a KeyStore using file-based keys as backend and added most of the required functions to try again.

The last Friday

I sat down again; this time, I had a few friends (including Saptak, Nabarun) on video along with me, and together we tried to plug the jce inside SecureDrop container for Focal. After around 4 hours, we had around 5 failing tests (from 32) in the crypto-related tests. Most of the basic functionality was working, but we are stuck for the last few tests. As I was using the file system to store the keys (in simple .sec or .pub files), it was difficult to figure out the existing keys when multiple processes were creating/deleting keys in the same KeyStore.

Next try via a SQLite based KeyStore

Next, I replaced the KeyStore with an SQLite based backend. Now multiple processes can access the keys properly. With a few other updates, now I have only 1 failing test (where I have to modify the test properly) in that SecureDrop Focal patch.

While doing this experiment, I again found the benefits of writing the documentation of the library as I developed. Most of the time, I had to double-check against it to make sure that I am doing the right calls. I also added one example where one can verify the latest (10.0) Tor Browser download via Python.

In case you already use OpenPGP encryption in your tool/application, or you want to try it, please give jce a try. Works on Python3.7+. I tested on Linux and macOS, and it should work on Windows too. I have an issue open on that, and if you know how to do that, please feel free to submit a PR.