Kushal Das

FOSS and life. Kushal Das talks here.

kushal76uaid62oup5774umh654scnu5dwzh4u2534qxhcbi4wbab3ad.onion

Documentation of Puppet code using sphinx

Sphinx is the primary documentation tooling for most of my projects. I use it for the Linux command line book too. Last Friday while in a chat with Leif about documenting all of our puppet codebase, I thought of mixing these too.

Now puppet already has a tool to generate documentation from it's code, called puppet strings. We can use that to generate markdown output and then use the same in sphix for the final HTML output.

I am using https://github.com/simp/pupmod-simp-simplib as the example puppet code as it comes with good amount of reference documentation.

Install puppet strings and the dependencies

$ gem install yard puppet-strings

Then cloning puppet codebase.

$ git clone https://github.com/simp/pupmod-simp-simplib

Finally generating the initial markdown output.

$ puppet strings generate --format markdown --out simplib.md
Files                     161
Modules                   3 (3 undocumented)
Classes                   0 (0 undocumented)
Constants                 0 (0 undocumented)
Attributes                0 (0 undocumented)
Methods                   5 (0 undocumented)
Puppet Tasks              0 (0 undocumented)
Puppet Types              7 (0 undocumented)
Puppet Providers          8 (0 undocumented)
Puppet Plans              0 (0 undocumented)
Puppet Classes            2 (0 undocumented)
Puppet Data Type Aliases  73 (0 undocumented)
Puppet Defined Types      1 (0 undocumented)
Puppet Data Types         0 (0 undocumented)
Puppet Functions          68 (0 undocumented)
 98.20% documented

sphinx setup

python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install sphinx myst_parser

After that create a standard sphinx project or use your existing one, and update the conf.py with the following.

extensions = ["myst_parser"]
source_suffix = {
    '.rst': 'restructuredtext',
    '.txt': 'markdown',
    '.md': 'markdown',
}

Then copy over the generated markdown from the previous step and use sed command to update the title of the document to something better.

$ sed -i '1 s/^.*$/SIMPLIB Documenation/' simplib.md

Don't forget to add the simplib.md file to your index.rst and then build the HTML documentation.

$ make html

We can still improve the markdown generated by the puppet strings command, have to figure out simpler ways to do that part.

Example output

SBOM and vulnerability scanning

Software Bill of Materials became one of the latest buzzword. A lot of people and companies talking about it like a magical thing, if you use it then all of your security problems will be solved, just like what happened with Blockchain!!.

Though a hand full of projects (or companies building those projects) focused on the actual tooling part. Things we can use and see some useful output than blogposts/presentations with fancy graphics.

In this post we will try to see how can we use these tools today (2023/09/20).

SBOM currently comes in two major flavors, SPDX aka Software Package Data Index and CycloneDX. There are existing tooling to convert in between.

Syft

We will use syft from Anchore to generate our SBOM(s).

This tool can generate from various sources, starting from container images to Python projects, RPM/Debian dbs, Rust or Go projects.

Let us generate the SBOM for a Debian 12 VM.

$ syft /var/lib/dpkg -o spdx-json=server.spdx.json --source-name debian12 
 ✔ Indexed file system                                                                                         /var/lib/dpkg
 ✔ Cataloged packages              [395 packages]  

For for a Rust project:

$ syft /home/kdas/code/johnnycanencrypt/Cargo.lock -o spdx-json=jce.spdx.json
 ✔ Indexed file system                                                                      /home/kdas/code/johnnycanencrypt
 ✔ Cataloged packages              [203 packages]

We generated the SBOMs. Now this should solve the security issues, isn't?

SBOM joke

I found the above in Matthew Martin's timeline.

Grype

This is where Grype comes handy, it is a vulnerability scanner for container images and filesystems and works with the SBOM(s) generated by syft.

$ grype jce.spdx.json 
 ✔ Vulnerability DB                [updated]  
 ✔ Scanned for vulnerabilities     [1 vulnerability matches]  
   ├── by severity: 0 critical, 0 high, 1 medium, 0 low, 0 negligible
   └── by status:   1 fixed, 0 not-fixed, 0 ignored 
NAME  INSTALLED  FIXED-IN  TYPE        VULNERABILITY        SEVERITY 
time  0.1.45     0.2.23    rust-crate  GHSA-wcg3-cvx6-7396  Medium

And:

grype server.spdx.json 
 ✔ Vulnerability DB                [no update available]  
 ✔ Scanned for vulnerabilities     [178 vulnerability matches]  
   ├── by severity: 6 critical, 136 high, 34 medium, 2 low, 0 negligible
   └── by status:   0 fixed, 178 not-fixed, 0 ignored 
NAME     INSTALLED     FIXED-IN  TYPE  VULNERABILITY     SEVERITY 
file     1:5.44-3                      CVE-2007-1536     High      
git      1:2.39.2-1.1                  CVE-2020-5260     High      
gnupg    2.2.40-1.1                    CVE-2022-3515     Critical  
gnupg    2.2.40-1.1                    CVE-2022-34903    Medium    
gnupg    2.2.40-1.1                    CVE-2022-3219     Low       
openssl  3.0.9-1                       CVE-2023-4807     High      
openssl  3.0.9-1                       CVE-2023-3817     Medium    
openssl  3.0.9-1                       CVE-2023-2975     Medium    
openssl  3.0.9-1                       CVE-2023-1255     Medium    
perl     5.36.0-7                      CVE-2023-31486    High      
perl     5.36.0-7                      CVE-2023-31484    High      
vim      2:9.0.1378-2                  CVE-2022-3520     Critical  
vim      2:9.0.1378-2                  CVE-2022-0318     Critical  
vim      2:9.0.1378-2                  CVE-2017-6350     Critical  
vim      2:9.0.1378-2                  CVE-2017-6349     Critical  
vim      2:9.0.1378-2                  CVE-2017-5953     Critical  
vim      2:9.0.1378-2                  CVE-2023-4781     High      
vim      2:9.0.1378-2                  CVE-2023-4752     High      

<snipped>

Now it is on your team members to decide how to react to information we gather from these tools. The tools themselves will not solve the problems at hand. You have to decide the update steps and if that is at all required or not.

Also please remember, there is and will be a lot of false positives (not in Grype output yet, but other tools in the SBOM ecosystem). The projects (I am talking about in general most of the tooling in this field) are trying hard to reduce these, but not possible always to remove every such edge case.

My talk in RustNL 2023

starting of the conference

On May 10th, I attended my first ever Rust conference, RustNL 2023. I reached there the night before. My talk was the 3rd one in the morning.

My talk

The title of my talk was Using Rust to write Python modules, and my main plan was to inform developers in the crowd to think about python developers as their API/library users. I demoed Tumpa to showcase what can be achieved to help the final end users.

The next 2 talks after mine also had Python in the theme. You should check out all the talks from the conference.

Book signing queue

I also managed to meet Mara Bos and get a copy of the book signed. Thank you so much.

I found the conference very tightly organized. The venue being on top of a library and centrally located was also very useful. The funniest incident was to find milk in the lunch menu, that was a first for me.

I also managed to meet some friends whom I only knew from Internet and met other Fedora friends after around 8 years.

I am hoping to be able to participate next year too.

Thank you Mikko

Thank you Mikko Hyppönen for spending me this punch card along with your signature.

Book front cover with the punchcard

It was difficult to figure out a trip to Finland to get the book signed by him, but he is kind enough to send me a signature to keep :)

Book cover with the signed punchcard

Fixing missing yubikey trouble on fedora 38

From the time I updated to Fedora 38, I am having trouble with my Yubikey. If I remove the key, just plugging it back does not help. gpg can not detect it.

$ gpg --card-status 
gpg: selecting card failed: No such device
gpg: OpenPGP card not available: No such device

The only way to get it working is restarting the pcscd service, again & again.

As Heiko pointed out, this is the trouble between pcscd and scdaemon, the second one comes via gnupg package in Fedora.

To solve the issue, first I tried the following

$ echo disable-ccid >> ~/.gnupg/scdaemon.conf
$ gpgconf --reload gpg-agent

Then I figured that I have opensc package installed, just removing that one and then a reboot solved the trouble for me.

$ sudo dnf remove opensc -y