Kushal Das

FOSS and life. Kushal Das talks here.

kushal76uaid62oup5774umh654scnu5dwzh4u2534qxhcbi4wbab3ad.onion

One week with Fedora Atomic in production

I was using containers for over a year in my personal servers. I was running a few services in those. For the last one week, I moved all my personal servers into Fedora Atomic, and running more number of services in those.

Server hardware & OS

These are actually VM(s) with couple of GB(s) of RAM, and a few CPU(s). I installed using the Fedora Atomic ISO image (get it from here) over virt-manager.

The containers inside

You can find all the Dockerfiles etc in the repo. Note: I still have to clean up a few of those.

Issues faced

In day zero the first box I installed, stopped printing anything on STDOUT, after a reboot I upgraded with atomic host upgrade command. I never had any other problem still now. So, try to stay updated.

Building my own rpm-ostree repo

My next target was to compose my own rpm-ostree repo. I used Patrick's workstation repo files for the same. In my fork I added couple of files for my own tree, and the build script. The whole work is done on a Fedora 24 container. You can view the repo here. This whole thing is exposed via another apache container. I will explain more about the steps in a future blog post.

What is next?

First step is clean up my old Dockerfiles. I will add up any future service as containers in those boxes. Even though we are automatically testing our images using Autocloud, using this in my production environment will help me find bugs in more convenient manner.

Event report: Flock 2016

This year's Flock was help in Krakow, Poland, from 2nd to 5th August. /me, and Sayan started our journey on 30th from Pune, and we reached Krakow finally on 31st afternoon. Ratnadeep joined us from the Frankfurt. Patrick was my roommate this time, he reached few hours after we reached.

Day -1

Woke up early, and started welcoming people near the hotel entrance. Slowly the whole hotel was filling up with Flock attendees. Around noon, few of us decided to visit the Oskar Schindler's Enamel Factory. This is place I had in my visit list for a long time (most of those are historical places), finally managed to check that off. Then walked back to city center, and finally back to the hotel.

Started meeting a lot more people in the hotel lobby. The usual stay up till late night continued in this conference too. Only issue was about getting up early, somehow I could not wake up early, and write down the daily reports as I did last year.

Day 0

Managed to reach breakfast with enough time to eat before the keynote starts. Not being able to find good espresso was an issue, but Amanda later pointed me to the right place. I don't know how she manages to do this magic everytime, she can really remove/fix any blocker for any project :)

Received the conference badge, and other event swags from registration desk. This one is till date the most beautiful badge I have seen. Mathew gave his keynote on "The state of Fedora". Among many other important stats he shared, one point was really noticeable for me. For every single Red Hat employee who contributes to Fedora, there are at least two contributors from the community. This is a nice sign of a healthy community.

After the keynote I started attending the hallway tracks as usual. I went to this conference with a long of topics I need discuss with various people. I managed to do all of those talks over the 4 days of the event. Got tons of input about my work, and about the project ideas. Now this is the time to make those suggestions into solid contributions.

Later I went into the "The state of Fedora-infra" talk. This was important to me personally, as this gives as easy way to revisit all the infrastructure work going on. Later in the day I attended Fedora Magazine, and university outreach talk.

In the evening there was "Tour of Krakow", but Fedora Engineering team had a team dinner. As this is only time when all of us meet physically. Food was once again superb.

Day 1

As I mentioned before it was really difficult to wake up. But somehow managed to do that, and reached downstairs before the keynote started. Scratch was mentioned in the keynote as tool they use. Next was usual hallway talks, in the second half I attended the diversity panel talk, and then moved to Pagure talk. I knew that there were a huge list new cool features in Pagure, but learning about them directly from the upstream author is always a different thing. Pingou's slides also contained many friends' names, which is always a big happy thing :)

My talk on Testing containers using Tunir was one of the last talk of the day. You can go through the whole presentation, and if you want to see any of the demos, click on those slides. That will open a new tab with a shell console, type as you normally type in any shell (you can type any char), and press Enter as required. I use Tunir to test my personal containers which I run on production. This talk I tried to various such use cases.

At night we went out the river cruising. Before coming back few of us visited the famous Wawel Dragon. I also met Scot Collier for the first time. It is always to meet people with whom you work regularly over internet.

Day 2

It started with lightening talks. I spoke for few minutes about dgplug summer training. You can find the list of talks here. After this in the same room we had "Meet the FAmSCo" session. I managed to meet Gerold, Kanarip, Fabian after 9 years in this Flock. Christoph Wickert took notes, and discussed the major action items in last week's FAmSCo IRC meeting too. Next I attended "Infrastructure workshop", after that as usual hallway tracks for me. I was looking forward to have a chat with Dodji Seketeli about his, and Sinny's work related about ABI stability. Later at night few of us managed to stay up till almost 5AM working :)

Day 3

Last day of Flock 2016. Even after the early morning bed time, somehow managed to pull myself out of the bed, and came down to the lobby. Rest of the day I spent by just talking to people. Various project ideas, demos of ongoing work, working on future goals.

Conclusion

Personally I had a long list of items which I wanted to talk to various people. I think I managed to cross off all of those. Got enough feedback to work on items. In the coming days I will blog on those action items. Meanwhile you can view the photos from the event.

Fedora Atomic 24 is now available

Just in case you missed the news, Adam Miller already announced the availability of Fedora Atomic release based on Fedora 24. You can get it from the usual place. Dusty already uploaded the same into Atlas for Vagrant. You can try it out by the following.

    $ vagrant init fedora/24-atomic-host; vagrant up

As Adam mentioned in his mail, we are sorry for the delay, but we will keep improving the process. Thank you everyone for helping us with this release.

Fedora mirror at home with improved hardware

It was always a dream to have a fully functional Fedora mirror in the local network which I can use. I tried many times before, mostly with copying rpms from office, carrying them around in hard drive, etc. But never managed to setup a working mirror which will just work (even though setting it up was not that difficult). My house currently has 3 different network (from 3 different providers) and at any point of time 1 of them stays down 😔

Hardware

If you remember my post on home storage, I was using Banana Pi(s). They are still very nice, and Fedora runs on them properly, but they were not very powerful, things like rsync was crawling on them. This PyCon, I received Minnowboard Turbot from John Hawley(Thanks a lot once again). It took time to get them setup (as I don't have a monitor with HDMI, I had to steal the TV from the front room), they are finally up in my own production environment. Installation of Fedora was super easy, just used the latest Fedora 24 from a live USB stick, and I was ready to go.

In the picture above you can see two of those running, you can also see a Banana Pi in the back.

Syncing up the mirror

Now for my work, I mostly need x86_64, and nothing else (I update my ARM boards, but not regularly). So following the tips in #fedora-noc channel from smooge, and puiterwijk, and some tips from this wiki page, I started rsyncing the 24GA. This was around 55GB, and took me some days to get it in. Mean while Chandan helped me by syncing the updates repo. Right now I have a cron job which syncs the update repo every night.

Remember to add the following your Apache virtualhost configuration

  AddType application/octet-stream .iso
  AddType application/octet-stream .rpm

Event report: Fedora 24 release party Pune

Last Saturday we had the Fedora 24 release party in Pune. This was actually done along with our regular Fedora meetup, and in the same location. We had a few new faces this time. But most of the regular attendees attended the meetup.

Chandan Kumar started the day with a presentation about new features in Fedora 24. We tried to see a few those in our laptops. During the discussions Parag , and Siddhesh pointed how important is self learning for us. Siddhesh also talked about a few project ideas on Glibc. In one of the previous meet, many of the attendees sent PR to the Fedora Autocloud testcases. We talked about those for few minutes. From yesterday, I am actually merging them in master.

As a group we decided to work on to make a modern version of glibc documentation. There is no git repo yet, but I will provide the links when we have something to show. As a group our goal is to do more upstream contribution. One thing I noticed that most of the attendees were dgplug summer training participants.

Thank you Patrick

Patrick

This post is say thank you to Patrick Uiterwijk, who is part of Fedora Infrastructure sysadmin team (along with nirik, and smooge). He is one of our silent warrior who keeps the Fedora Infrastructure running. Many may not heard about him, unless you came down to any of our admin or development IRC channels, and asked for help. I personally ask him for help in many different areas, starting for basic sysadmin questions, to complex deployment issues, to programming ideas. He is one of the hackers I know who can code in assembly to the web frontend as required. He is our main force behind Fedora Infrastructure Cloud, and spam fighting :) He is also our new Fedora Infrastructure Security Officer. If any of our new and young contributor thinks that she/he is too young to do things, you may want to look at Patrick's work. He started contributing back in 2012, and he is right now 24 :)

So, thank you once again for all the help.

Event report: Fedora Cloud FAD 2016

Around a month back the Fedora Cloud Working Group met in Raleigh for two days for Cloud FAD. The goal of the meet was to agree about the future we want, to go through major action items for the coming releases. I reached Raleigh one day before, Adam Miller was my room mate for this trip. Managed to meet Spot after a long time, this was my first visit to mothership :) I also managed to meet my new teammate Randy Barlow.

Adam took the lead in the event, we first went through the topics from the FAD wiki page. Then arranged them as in the order we wanted to discuss.

Documentation was the first item. Everyone in the room agreed that it is the most important part of the project, we communicate with our users using the documentation. If a project can provide better documentation with clear examples, it will be able to attract more users to itself. Last year I have started a repo to have documents available in the faster manner, but that did not work out well. Jared volunteered, and also suggested to have an open repo where people can submit documents without worrying much about format. He will help to convert to the right format as required. We also noted few important examples/documents we want to see. Feel free to write about any of these and submit a pull request to the repo.

Automated testing was the next import point. We went through the current state of tests in Autocloud project. Most of our tests there came from various bugs filed against the images/tools. Dusty was very efficient in creating the corresponding issues on github from the RH bugzilla, which then in turn was converted into proper Python3 unittest based test case by the volunteers. Next we moved to the automated testing of layered image builds. Tflink provided valuable input on this discussion. We talked SPC(s), how the maintainers of the images will be responsible for the tests. For the tests related to the Cloud/Atomic images, the Cloud Working Group will be responsible.

There were few super long discussions related Fedora, Atomic, and Openshift projects. Many members from the Openshift development team also joined in the discussion. Results of these discussions will be seen in the mailing lists (a lot of content for this blog post). We also discussed about Vagrant, and related tools people are using or creating. With help from Randy, I managed to package vagrant-digitalocean plugin during the FAD. There will be a Fedora magazine post with more details on the same.

We also agreed upon having monthly updated base images. We still have to find out a few missing points related to the same so that we can have a smooth updated release.

Public cloud provider outreach was one of the last points we discussed. We have to pick different providers one by one, and have to make sure that we can provide updated releases to them for the consumption the users. The point of more documentation came up in this discussion too.

Summer training 2016 is on

The 9th edition of dgplug summer training started few weeks back. This year in the IRC channel (#dgplug on freenode) we saw around 186+ nicks participating in the sessions. Till now we have went through communication guidelines, IRC, mailing list how to, a text editor ( Vim in this case), blogging, basic bash commands, a few more bit advanced bash commands. We also learned about reStructured Text, and Sphinx. We also managed to live demos to all students from the mentor's terminal.

Even though most of the participants are new to Linux, we can see a better quality of questions coming from them, better point is that most of them are not afraid about asking questions. Which is a very good sign to see in the training. Few days back Trishna Guha took the first guest session of this year, she talked about her experience in the summer training last year, and how she has become an upstream contributor. If you read the logs, then you can see people asked all shorts of questions to her.

We also have a feed aggregator up where you can read the blog posts of the participants. I still have to add a few more blogs to this as many sent in the feed URLs much later.

If you are an upstream contributor, and you want to encourage this group of new participants, please drop me a note. We would love to see more upstream folks talking to these new/upcoming contributors. The sessions run on IRC, it can be in a time when you are available. So just ping me if you are interested to take a session :)

Day -1 of PyCon US 2016

I reached Portland two days back, was happy to see that Remy helped us to find a hotel just opposite to the conference center. As I am typing this, I can see the empty roads, and the main doors of the venue. Yesterday also I woke up by 5AM, the virtue of being in a place 12:30 hours apart from your standard timezone :) After writing the article about Microbit support in Fedora (it is still under review) I moved to the conference venue. It was very easy to find the staff room. As usual Doug,Lvh,Eric were already there, later I met Ewa, and then met Betsy for the first time. While discussing security practices when I asked, Lvh pointed out that getting golang vendored sources in the source code repository and then not updating them later, is more of a software engineering problem than a security problem as he sees.

Later in the day few of us walked towards downtown to get some lunch, and then we decided to visit Powell's again. Mike Pirnat gave some valuable insights about children books, and card games :) After coming back to the venue, we saw more people are coming in. Rami was the first person to meet, PyCon is really a great place to meet old friends. Later I met Glyph, Hynek, Guido, Harry, Mike Muller, Graham Dumpleton, and Yarko (I don't think I will be able to write down people's names like this from tomorrow, the list will be too long :D ). I also found PyDanny, and Audrey coming in for the registration. It was good see her walking around after the big surgery.

Guido pointed out the xkcd comics about the Python patch. The discussion then moved to regex, legal side of the software field. I also had a chat about how CentOS is helping CPython with CI effort. It turns out that Guido knew about CentOS, but never looked deep into what is that :) He was seems be happy about help CentOS is doing by providing the beefy boxes to do build+test super fast. I am actually going to talk about that today in the Language Summit.

Later at night a large group of us moved to a nearby Ethiopian place for dinner. The food was great, we had more people coming in for the next few hours. Ntoll also came in and joined our table. Came back to the hotel room around 22:30, and tried to sleep early so that I can wake up in time. Today I will be in the registration desk from 7AM just like other years, and then I will move into the Language Summit. See you all in PyCon :)

dgplug summer training student Tosin Damilare James Animashaun

Your name (blog/twitter) and what do you do

My name is Tosin Damilare James Animashaun (IRC: acetakwas; Twitter: @acetakwas; Github: github.com/takwas, Blog: takwas.github.io).

I am currently a part-time Software Engineering student at NIIT, Lagos. I am also a co-founder at a startup called Krohx. We are in the final stages of deploying a job/hiring web platform that targets seekers of short-term jobs while also easing the process of getting service delivery for the hirers.

With my involvements with the burgeoning Python community in Nigeria, I recently got elected as secretary for the organization.

How did you learn about the training? (++ My Experience) I fail to recall

exactly how I learnt about the community now, but I remember joining the mailing list and going on from there. This was in fact my first time of joining a mailing list. Upon learning about the #dgplug IRC channel, I promptly enlisted the channel to my IRC client's favourites.

I was so enthusiastic about this, as I had no prior experience of an online tutoring session. The feeling was truly exhilarating.

Having come from a Java background, I was really interested in learning the Python programming language as soon as I caught on waves with. At the time, I had volunteered to write some scripts to help automate some of the manual processing of NIIT's annual scholarship exam records/documents at my school for that year. I accomplished that using Java, but in hind-sight, I feel I could have saved a lot of time if I had utilised Python in the situation.

This was a blog post I wrote as part of tasks given to us. I have failed to maintain that blog though. I now write here

How this training changed (if at all) your life? Python is surely the more

beginner-friendly language and it's making its rounds these days. Every programmer should have it in their arsenal. One of the things I love most about the language is it versatility in application.

No doubt, I have continued to improve with Python everyday, but more so in my programming journey. I have become more and more familiar with various mainstream concepts and tools. The idea of contributing to open-source can be overwhelming at first, but following the path laid out in this course, one would be surprised to learn how easy it could be.

I have volunteered to mentor attendees at a Django Girls workshop, which held in Lagos, Nigeria. I picked-up Django in a week prior to the event, because frankly, there wasn't much of a learning curve to it since I already used Flask, a different Python-based web development framework.

Flask was a framework I first got introduced to at the summer training, but I did not revisit it until months later. As a backend web-developer, I now do most of my work in Flask.

In my experience, there is no better or faster way to grow, than to collaborate; and that is what open source preaches. Note that growth here is not only about the project, but the growth applies to each contributing individual. Heck, even if you do not contribute, you will grow by reading good open-source code -- this I can attest to. And joining the dgplug community has surely made open-source a more approachable endeavour. # #

Have you contributed to any upstream project(s)? If yes, then details.

While (and a bit disappointingly) I have yet to contribute to an upstream project, I do have quite a number of them lined up. Most probably, the first would be to the official Slack Client written in Python. Having recently developed a bot, which was used in conducting the election of committee members of the Python Users Nigeria Group, I do have a number of changes I plan to push to the aforementioned API library.

With this forked project, I have also tried my hands at revamping the architecture of the official dgplug bot ekanora, which is used in running the summer training sessions.

Some other projects I'd like to contribute to include:

  • https://github.com/StellarCN/py-stellar-base
  • https://github.com/getpelican/pelican-blog

Any tips for the next batch of participants. (++Final words)

I am not sure I am in the best position to answer this. However, I would advise that you endeavour to remain focused through this; simply keep at it. Even if your experience at the summer camp doesn't go as you expect, you should definitely not count it loss. Instead, make it a reason to put in more effort and grow as you go.

Depending on your personal activity schedule, you might miss one/two classes. But that shouldn't deter you from continuing the program. The logs are available for covering such lapses. Also, you could consult the summer training docs to get a hang of the agenda, so you can anticipate classes beforehand and be better prepared.

While I participated in the training, there were times when there was no power supply, and I had to use my phone. Thankfully, there are IRC clients for mobile phones (I recommend AndChat).

The class sessions have a few rules, and the community would love it if you played by them. Not to worry, they are no stringent rules; just there to keep everyone behaving. For instance, the use of SMS-speak is discouraged.

You should keep notes beside you when attending the sessions. Writing is a good way to memorize things in my opinion. Also, although most clients would have this by default, you should ensure your IRC client is configured to log conversations. Going back over what you've learnt would do you great good as you mightn't be able to keep up with the speed of the class at times.

If you have some time before program begins, two things I'd advise you become familiar with are: - Your OS (Linux, preferably Fedora or Ubuntu) - An IRC client (I suggest HexChat)

I learned quite a lot from the summer training. I was really obsessed with attending whenever I could. Although coming from a different country, the timing wasn't favourable, often meeting me in the middle of the day's activities -- school especially. I made efforts to invite a few friends to join on -- one or two did, but the determination to keep on was obviously lacking, which reminds me of a statement I heard a while back that reads something like this:

"If 40 people begin a course like this together, only about 5 get to
finish."

That in my experience is very often the case. Be aware that a lot of people are on this bandwagon. The question to ask yourself, is "Do I want to be among the surviving commutants at the end of this journey?" Unless you plan to experiment with this experience, if your answer is yes, then I congratulate you as you begin the journey that could potentially kick-start your software engineering career/journey.

Know this, free and open source software is for the good of all. The open source community is wide and continues to grow. It welcomes all and sundry, but it poses a few hurdles to sieve the wheat from the chaff. In the end, the gains are worth it.