This comics explains the birth of the project Tunir. When I started working in the Fedora Cloud SIG, I volunteered myself to help with the testing of the Cloud images. We have very clear guidelines about what to test, and how to test.

Basically, you have to boot up a cloud image (or the atomic image) in a Cloud or on your local computer, run a few commands in sequence, and check the output. For the first few times it was fun to do, but slowly I found it is difficult for me. Being a super lazy programmer, I thought why not use the computer to do this job, it is not music theory :) The birth of Tunir was the result of the conversation between me and even lazier me. I just had to convert the shell commands into some Python3 based unittest cases.

At beginning it could only handle the qcow2 images of cloud base and atomic image. But while working with two week atomic change, I found out that we need do the same for Vagrant based images. It is important to remember that Fedora project generates two different kind of Vagrant boxes, one for libvirt, another one for standard Virtualbox based .box file. So Tunir got the power to execute tests on any given Vagrant image. Using these features Tunir is being consumed by the Autocloud project, where we automatically test Fedora cloud and Atomic image builds. I should not forget to mention that Tunir can also connect to a remote system, and execute the tests there. Did I mention that Tunir is doing all of these with only one JSON file containing the job description, and one text file containing the commands used for the actual tests? Tunir started as a tool for a developer, who does not want to spend time in configuration, it remains in the same way.

Until a couple of releases back, Tunir actually had two types of tests in it, one type of commands which will return zero as a success value, we could also mark a set of commands as the ones which will return non-zero exit code. Right now Tunir can have a third set of commands, the non-gating tests, these commands may pass, or they may fail. But Tunir will continue executing the tests accordingly.

I wrote Tunir to test the actual OS images, but it is generic enough that it can be used to test any application. You can easily configure it to download all the dependencies of your project every time on a clean cloud/Vagrant image, and then it will build and test your application (or the full application stack along with configuration files).

Last week I have added a new weapon in Tunir's arsenal. It can now test using a given AWS AMI ID. If you run your application on AWS cloud platform, you now can use Tunir to test it there. This code is in a separate branch, but will be merged in the master this week. Following the similar style Praveen Kumar submitted a patch using which Tunir can run tests on an Openstack Cloud. We will work on it a little bit more before merging it to master branch.

You can view the tests already written for Fedora in this repository. Thanks to Trishna Guha, and Farhaan Bukhsh we now have many more test cases. I have written a document explaining how to write more tests, I have another document which explains how to debug failed tests.

What is in the future? The best way to predict your future is to create it. We have the features which we, and our users use regularly. Tunir is still simple enough for anyone to understand in less than 10 minutes. If you just want to say hi, or you are looking forward for any new feature, come to #fedora-cloud on freenode.

Fedora Cloud Working Group maintains two different types of Cloud images, Cloud base image (which is a minimal standard Fedora build for Cloud), and also the Fedora atomic image (from Project Atomic). We also maintain, and release the Vagrant images for the same. We are on our way to keep releasing updated Atomic images every two weeks, and updated cloud images once every month. This means we need more help in testing the images.

Autocloud is a new service running in Fedora Infrastructure which automatically tests the cloud images. In a last post I wrote about Autocloud, but it does not test every aspect of the images. We are yet to be in a state where we can say that 100(s) of tests are keeping eye on the images automatically. We need the support from the community in both writing new tests, and also testing the images manually. This is a great way not only to help the community, but also a chance to learn about the latest container and cloud technologies. You will be not only working with the Cloud WG, but will also be in touch with Fedora QA, and Fedora Infra group. If you are interested feel free to add your name in this list, or ping me (nick name kushal) on #fedora-cloud channel.

During Fedora 23 release cycle as part of Two week atomic image, we have developed, and deployed a new service in Fedora Infrastructure, called Autocloud. In simple words this services listens to fedmsg messages for successful koji builds of cloud base, and atomic images. When found, it downloads those images, and test them locally using Tunir. It tests the standard qcow2 images, and also the box files for vagrant. Yes, we test both libvirt, and Virtualbox based vagrant images (using tunir).

If you want to learn about how it is deployed, please visit this page. This link shows the output of currently running tests again a Fedora 23 atomic image. We do need help on writing new tests, and also doing the actual tests manually. My next post will have more details on that.

Day one is the first day of main event. I was late to wake up, but somehow managed to reach the venue around 8:30am. Had a quick breakfast, and then moved into the Red Hat booth. Sankarshan, Alfred, Soni were already there. I don't know the exact reason, but the booth managed to grab the attention of all the people in the venue. It was over crowded :) While the students were much more interested in stickers, and other goodies, many came forward to ask about internship options, and future job opportunities. Alfred did an excellent job in explaining the details to the participants. The crowd was in booth even though the keynote of day one had started. I missed most of keynote as many people kept coming in the booth, and they had various questions.

At 11:30AM, we had our annual Dgplug meetup on the staircase of the venue. It is already known as "dgplug staircase meeting".We are a virtual group for the most part of the year, but PyCon is the time when we try to meet and discuss various things face to face. Though this year many new members were busy volunteering, so they missed the meeting. We mostly discussed about the summer training, what went wrong, how can we improve etc. Here are two tweets about the same from others.

I had only one talk in my mind (other than the keynotes) which I wanted to attend. It was "Pretty printing in Python" by Shakthi Kannan, a talk about a 3D printer made from the parts available in India, and his experiments with it in both software, hardware level. It was an excellent talk.

During lunch time Anwesha and Py (my daughter) came in. This was Py's first conference. She really enjoyed it, happily roamed around with Sankarshan, and Chandan for a long time. Here is a family photo in the Red Hat booth.

Rest of the second half I spent mostly talking with the people in and around the booth. "What do you think about working in Red Hat and upstream projects? What is the difference you feel than working for any other big names?" this question led to an interesting discussion with few SymPy developers. I hope I managed to explain my viewpoints to them properly. That answer anyway should get it's own blog post :)

This year we also had a childcare facility in the event, and it was amazing. So many kids were inside through out the both days of the event. The parents were extremely happy about the facility. Thank you organisers for making this available to the participants with kids.

I came down to Bangalore few days back to attend PyCon India 2015. This event is as usual the best place to meet most of my FOSS friends in India. This time, the day 0 aka the workshop day also had dev-sprints running in the first floor of the venue.

I reached the venue around 8:50AM, after registration went straight to the breakfast place in the back. The food was good as usual. Only problem of this venue is about not having a good coffee shop nearby. The events for the day started at 9AM. When Sayan introduced dev sprints to all of the participants, most of them were first timers in PyCon and also firt timer in a dev-sprint. After all the other mentors introduced their projects, I talked about CPython.

The people who were sprinting with me were mostly very much new comers into any upstream project. So instead of solving bugs directly we spent the time explaining/learning about the process of reviewing a patch. How to start digging deep into the codebase. We took one bug with a patch (randomly), and then everyone tried to apply and test the patch as much as they could. The biggest issue was about not having a stable network connection, but we somehow managed to go through most of the steps.

In between ntoll also joined us in the sprints. It was fun to see him describing timezones using a tea cup as globe. We spent a lot of time talking to various students through out the day. Later at night we had a dinner with many of current and past dgplug summer training students.

If you are coming to PyCon today or tomorrow, please visit the Red Hat booth in the event, we will have lot of upstream contributors in the booth who can help you in jump starting your upstream contribution.

Last week I have released tunir 0.8. In this release I have fixed few bugs related to better logging. But most of the work was wnet on vagrant support. We now support both Virtualbox, and libvirt based vagrant images. There is a new key in the job.json file called provider, if you give the value virtualbox, then tunir will use vagrant along with Virtualbox, otherwise it will try to use vagrant-libvirt provider (which is default for our usage). There is separate page in the docs which explains the configurations.

Last week Dusty started looking into few bugs related to cloud base image. His first work was on the locale issues coming up in the base image. If you login to the system, you will see many warnings like:

-bash: warning: setlocale: LC_CTYPE: cannot change locale (en_US.UTF-8): No such file or directory
-bash: warning: setlocale: LC_COLLATE: cannot change locale (en_US.UTF-8): No such file or directory
-bash: warning: setlocale: LC_MESSAGES: cannot change locale (en_US.UTF-8): No such file or directory
-bash: warning: setlocale: LC_NUMERIC: cannot change locale (en_US.UTF-8): No such file or directory
-bash: warning: setlocale: LC_TIME: cannot change locale (en_US.UTF-8): No such file or directory

Last Monday I tested the patch locally first, and then pushed it to the spin-kickstarts . His next work was related to the cloud-base-vagrant image, where we removed extlinux, and used grub for booting, you can view the commit here.

I also worked on broken i386 cloud image, it was not getting built for last few weeks. While trying to build the last known broken kickstart locally, I found the dracut was on a loop, it never went to the next stage. With the help Ian McLeod, the amazing upstream author of Imagefactory, we chased down the issue in #anaconda, and Kevin (nirik) confirmed that Oz requires more RAM to build the i386 image. We tested that locally, Dennis will update the koji builders to get the issue fixed. If you are trying to build the i386 version of the Cloud image locally, remember to increase libvirt memory in /etc/oz/oz.cfg Rest of my time mostly went on tunir and autocloud project. There will be few more blog posts in coming days on them.

During FUDCon, I heard that later in the year we might get a Django Girls workshop in Pune. If you never heard about Django Girls before, here is a quote from the website:

"Django Girls is a non-profit organization that empowers and helps women to organize free, one-day programming workshops by providing tools, resources and support. It was born in July 2014 in Berlin and started by two Polish girls: Ola Sitarska and Ola Sendecka."

I also found that we are going host it in the Red Hat Pune office. As soon as the call for mentors came out, I applied, and got chosen to become a mentor for the workshop.

Last Saturday was the actual workshop day. Going to office only after four hours of sleep was difficult, but I somehow managed to do so with enough amount of coffee :). I met Shagufta (the organizer) of the workshop there, later Priyanka also reached the venue. There were in total five mentors, and around twenty-seven participants. We started a bit late as we had to wait for a few participants.

In my group I had five students, one just finished college this year, two were in their final years, and one second year student. The last student was from third year mechanical department. Python was a new language for them, and two students used Windows. <sarcasm>As my knowledge about Windows is way better</sarcasm>, we installed Fedora on Virtualbox in those two computers. The tutorial for the workshop is available on Github. At the beginning of the session, I told that our biggest problem will be typing mistakes (as it is in any workshop). We learned few basic commands, Python language syntax, and then moved into Django. We also used the excellent https://www.pythonanywhere.com to deploy our code. We went with a slow pace as we have to keep debugging our typing mistakes :) Even though we did not finish the whole tutorial, I showed them how to use IRC, and get help there. I also suggested that we can do a few more sessions if they want, they just have to come with a date when all of them can come down once again. Oh here is a tip, if your students want to copy-paste code from the book, ask them to do so from the markdown source files, or from rendered HTML files in github.

The workshop ended with a group photoshoot. There will be another workshop in Mumbai in the coming months.

Today I have released Tunir 0.7. Tunir is a simple CI which developers can even use in their laptops. There are few major changes in this release. The first one is about no database support. Tunir itself will not save any data anywhere, this also means --stateless command line argument is now unnecessary. Even if you do not pass that option, tunir will print out the output of the tests on STDOUT.

Second, and the biggest change is the ability to test on Vagrant boxes using vagrant-libvirt plugin. On a Fedora 22 system, you can install vagrant by the following command.

$ sudo dnf install vagrant-libvirt

The following is an example job configuration on Vagrant. This of course assumes that you have already downloaded that box file somewhere in the local filesystem. In case you have not noticed, we are now generating vagrant images for both Cloud base, and Atomic image in Fedora Project. You can download them from here.

{
  "name": "fedora",
  "type": "vagrant",
  "image": "/home/Fedora-Cloud-Atomic-Vagrant-22-20150521.x86_64.vagrant-libvirt.box",
  "ram": 2048,
  "user": "vagrant",
  "port": "22"
}

I have already built the rpms for Fedora, they are right now in the testing repo.

I was working on adding Vagrant based tests in tunir, and hit an issue without any informative error message.

$ sudo vagrant up
Bringing machine 'tunirserver' up with 'libvirt' provider...
/usr/share/gems/gems/fog-1.23.0/lib/fog/libvirt/requests/compute/list_volumes.rb:32:in `info': Call to virStorageVolGetInfo failed: Storage volume not found: no storage vol with matching path '/tmp/orbit-kdas' (Libvirt::RetrieveError)
    from /usr/share/gems/gems/fog-1.23.0/lib/fog/libvirt/requests/compute/list_volumes.rb:32:in `volume_to_attributes'
    from /usr/share/gems/gems/fog-1.23.0/lib/fog/libvirt/requests/compute/list_volumes.rb:10:in `block (2 levels) in list_volumes'
    from /usr/share/gems/gems/fog-1.23.0/lib/fog/libvirt/requests/compute/list_volumes.rb:9:in `each'
    from /usr/share/gems/gems/fog-1.23.0/lib/fog/libvirt/requests/compute/list_volumes.rb:9:in `block in list_volumes'
    from /usr/share/gems/gems/fog-1.23.0/lib/fog/libvirt/requests/compute/list_volumes.rb:44:in `block in raw_volumes'
    from /usr/share/gems/gems/fog-1.23.0/lib/fog/libvirt/requests/compute/list_volumes.rb:42:in `each'
    from /usr/share/gems/gems/fog-1.23.0/lib/fog/libvirt/requests/compute/list_volumes.rb:42:in `raw_volumes'
    from /usr/share/gems/gems/fog-1.23.0/lib/fog/libvirt/requests/compute/list_volumes.rb:8:in `list_volumes'
    from /usr/share/gems/gems/fog-1.23.0/lib/fog/libvirt/models/compute/volumes.rb:11:in `all'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.0.26/lib/vagrant-libvirt/action/handle_box_image.rb:43:in `block in call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.0.26/lib/vagrant-libvirt/action/handle_box_image.rb:40:in `synchronize'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.0.26/lib/vagrant-libvirt/action/handle_box_image.rb:40:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/builtin/handle_box.rb:56:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.0.26/lib/vagrant-libvirt/action/handle_storage_pool.rb:24:in `block in call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.0.26/lib/vagrant-libvirt/action/handle_storage_pool.rb:17:in `synchronize'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.0.26/lib/vagrant-libvirt/action/handle_storage_pool.rb:17:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.0.26/lib/vagrant-libvirt/action/set_name_of_domain.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:95:in `block in finalize_action'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/builder.rb:116:in `call'
    from /usr/share/vagrant/lib/vagrant/action/runner.rb:66:in `block in run'
    from /usr/share/vagrant/lib/vagrant/util/busy.rb:19:in `busy'
    from /usr/share/vagrant/lib/vagrant/action/runner.rb:66:in `run'
    from /usr/share/vagrant/lib/vagrant/action/builtin/call.rb:53:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.0.26/lib/vagrant-libvirt/action/connect_libvirt.rb:18:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/builder.rb:116:in `call'
    from /usr/share/vagrant/lib/vagrant/action/runner.rb:66:in `block in run'
    from /usr/share/vagrant/lib/vagrant/util/busy.rb:19:in `busy'
    from /usr/share/vagrant/lib/vagrant/action/runner.rb:66:in `run'
    from /usr/share/vagrant/lib/vagrant/machine.rb:214:in `action_raw'
    from /usr/share/vagrant/lib/vagrant/machine.rb:191:in `block in action'
    from /usr/share/vagrant/lib/vagrant/environment.rb:516:in `lock'
    from /usr/share/vagrant/lib/vagrant/machine.rb:178:in `call'
    from /usr/share/vagrant/lib/vagrant/machine.rb:178:in `action'
    from /usr/share/vagrant/lib/vagrant/batch_action.rb:82:in `block (2 levels) in run'

Found a bug, but that did not give any information on how to fix it. Then in the middle of night (around 1am) Dusty Mabe gave the right solution :). I have to refresh all the available storage pools in libvirt. It had nothing to do with Vagrant, but I wish that the error message was a little bit less cryptic. In my case I did the following.

$  sudo virsh pool-list
 Name                 State      Autostart 
-------------------------------------------
 default              active     yes       
 tmp                  active     yes 

Then refreshing each pool manually.

$  sudo virsh pool-refresh tmp
Pool tmp refreshed
$  sudo virsh pool-refresh default
Pool default refreshed