Kushal Das4

FOSS and life. Kushal Das talks here.

kushal76uaid62oup5774umh654scnu5dwzh4u2534qxhcbi4wbab3ad.onion

Tor Browser and Selenium

Many of us use Python Selenium to do functional testing of our websites or web applications. We generally test against Firefox and Google Chrome browser on the desktop. But, there is also a lot of people who uses Tor Browser (from Tor Project) to browse the internet and access the web applications.

In this post we will see how can we use the Tor Browser along with Selenium for our testing.

Setting up the environment

First step is to download and verify, and then extract the Tor Browser somewhere in your system. Next, download and extract geckodriver 0.17.0 somewhere in the path. For the current series of Tor Browsers, you will need this particular version of the geckodriver.

We will use pipenv to create the Python virtualenv and also to install the dependencies.

$ mkdir tortests
$ cd tortests
$ pipenv install selenium tbselenium
$ pipenv shell

The tor-browser-selenium is Python library required for Tor Browser Selenium tests.

Example code

import unittest
from time import sleep
from tbselenium.tbdriver import TorBrowserDriver


class TestSite(unittest.TestCase):
    def setUp(self):
        # Point the path to the tor-browser_en-US directory in your system
        tbpath = '/home/kdas/.local/tbb/tor-browser_en-US/'
        self.driver = TorBrowserDriver(tbpath, tbb_logfile_path='test.log')
        self.url = "https://check.torproject.org"

    def tearDown(self):
        # We want the browser to close at the end of each test.
        self.driver.close()

    def test_available(self):
        self.driver.load_url(self.url)
        # Find the element for success
        element = self.driver.find_element_by_class_name('on')
        self.assertEqual(str.strip(element.text),
                         "Congratulations. This browser is configured to use Tor.")
        sleep(2)  # So that we can see the page


if __name__ == '__main__':
    unittest.main()

In the above example, we are connecting to the https://check.torproject.org and making sure that it informs we are connected over Tor. The tbpath variable in the setUp method contains the path to the Tor Browser in my system.

You can find many other examples in the source repository.

Please make sure that you test web application against Tor Browser, having more applications which can run smoothly on top of the Tor Browser will be a great help for the community.

PyQt5 thread example

PyQt is the Python binding for Qt library. To write Qt5 code, we use PyQt5 module. Like many others, my first introduction to GUI application development was using PyQt. Back in foss.in 2005 a talk from Sirtaj introduced me to PyQt, and later fall in love with it.

I tried to help in a GUI application after 8 years (I think), a lot of things have changed in between. But, Qt/PyQt still seems to be super helpful when it comes to ease of development. Qt has one of the best documentation out there for any Open Source project.

Many students start developing GUI tools by replacing one of the command line tool they use. Generally the idea is very simple, take some input in the GUI, and then process it (using a subprocess call) on a button click, and then show the output. The subprocess call happens over a simple method, means the whole GUI gets stuck till the function call finishes. We can fix this issue by using a QThread. In the below example, we will just write a frontend for git clone command and then will do the same using QThread.

Setting up project directory

I have used qt creator to create a simple MainWindow form and saved it as mainwindow.ui in the project directory. Then, used pipenv to create a virtualenv and also installed the pyqt5 module. Next, used the pyuic5 command to create a Python file from UI file.

The code does not have error checks, the subprocess documentation should give you enough details about how to add them.

Doing git clone without any thread

The following code creates a temporary directory, and then git clones any given git repository into that.

#!/usr/bin/python3

import sys
import tempfile
import subprocess
from PyQt5 import QtWidgets

from mainwindow import Ui_MainWindow


class ExampleApp(QtWidgets.QMainWindow, Ui_MainWindow):

    def __init__(self, parent=None):
        super(ExampleApp, self).__init__(parent)
        self.setupUi(self)
        # Here we are telling to call git_clone method when
        # someone clicks on the pushButton.
        self.pushButton.clicked.connect(self.git_clone)

    # Here is the actual method which does git clone
    def git_clone(self):
        git_url = self.lineEdit.text()  # Get the git URL
        tmpdir = tempfile.mkdtemp()  # Creates a temporary directory
        cmd = "git clone {0} {1}".format(git_url, tmpdir)
        subprocess.check_output(cmd.split())  # Execute the command
        self.textEdit.setText(tmpdir)  # Show the output to the user


def main():
    app = QtWidgets.QApplication(sys.argv)
    form = ExampleApp()
    form.show()
    app.exec_()


if __name__ == '__main__':
    main()

Doing git clone with a thread

In the below example we added a new CloneThread class, it has a run method, which gets called when the thread starts. At the end of the run, we are emitting a signal to inform the main thread that the git clone operation has finished.

#!/usr/bin/python3

import sys
import tempfile
import subprocess
from PyQt5 import QtWidgets
from PyQt5.QtCore import QThread, pyqtSignal

from mainwindow import Ui_MainWindow


class CloneThread(QThread):
    signal = pyqtSignal('PyQt_PyObject')

    def __init__(self):
        QThread.__init__(self)
        self.git_url = ""

    # run method gets called when we start the thread
    def run(self):
        tmpdir = tempfile.mkdtemp()
        cmd = "git clone {0} {1}".format(self.git_url, tmpdir)
        subprocess.check_output(cmd.split())
        # git clone done, now inform the main thread with the output
        self.signal.emit(tmpdir)


class ExampleApp(QtWidgets.QMainWindow, Ui_MainWindow):

    def __init__(self, parent=None):
        super(ExampleApp, self).__init__(parent)
        self.setupUi(self)
        self.pushButton.setText("Git clone with Thread")
        # Here we are telling to call git_clone method when
        # someone clicks on the pushButton.
        self.pushButton.clicked.connect(self.git_clone)
        self.git_thread = CloneThread()  # This is the thread object
        # Connect the signal from the thread to the finished method
        self.git_thread.signal.connect(self.finished)

    def git_clone(self):
        self.git_thread.git_url = self.lineEdit.text()  # Get the git URL
        self.pushButton.setEnabled(False)  # Disables the pushButton
        self.textEdit.setText("Started git clone operation.")  # Updates the UI
        self.git_thread.start()  # Finally starts the thread

    def finished(self, result):
        self.textEdit.setText("Cloned at {0}".format(result))  # Show the output to the user
        self.pushButton.setEnabled(True)  # Enable the pushButton


def main():
    app = QtWidgets.QApplication(sys.argv)
    form = ExampleApp()
    form.show()
    app.exec_()


if __name__ == '__main__':
    main()

The example looks like the above GIF. You can find the source code here. You can find a bigger example in the journalist_gui of the SecureDrop project.

Fedora 28 template is available on QubesOS

In case you missed the news, Fedora 28 is now available as a template in Qubes OS 4.0. Fedora 26 will end of life on 2018-06-01, means this is a good time for everyone to upgrade. Use the following command in your dom0 to install the template. The template is more than 1GB in size, means it will take some time to download.

$ sudo qubes-dom0-update qubes-template-fedora-28

After installation, remember to start the template, and update, and also install all the required applications there. Next step would be to use this template everywhere.

Btw, we do have the latest Python 3.6.5 in Fedora 28 :)

$ python3
Python 3.6.5 (default, Mar 29 2018, 18:20:46)
[GCC 8.0.1 20180317 (Red Hat 8.0.1-0.19)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

SecureDrop development sprint in PyCon 2018

SecureDrop will take part in PyCon US development sprints (from 14th to 17th May). This will be first time for the SecureDrop project to present in the sprints.

If you never heard of the project before, SecureDrop is an open source whistleblower submission system that media organizations can install to securely accept documents from anonymous sources. Currently, dozens of news organizations including The Washington Post, The New York Times, The Associated Press, USA Today, and more, use SecureDrop to preserve the anonymous tipline in an era of mass surveillance. SecureDrop is installed on-premises in the news organizations, and journalists and source both use a web application to interact with the system. It was originally coded by the late Aaron Swartz and is now managed by Freedom of the Press Foundation.

How to prepare for the sprints

The source code of the project is hosted on Github.

The web applications, administration CLI tool, and a small Qt-based GUI are all written in Python. We use Ansible heavily for the orchestration. You can setup the development environment using Docker. This section of the documentation is a good place to start.

A good idea would be to create the initial Docker images for the development before the sprints. We have marked many issues for PyCon Sprints and also there are many documentation issues.

Another good place to look is the tests directorty. We use pytest for most of our test cases. We also have Selenium based functional tests.

Where to find the team?

Gitter is our primary communication platform. During the sprint days, we will in the same room of the CPython development (as I will be working on both).

So, if you are in PyCon sprints, please visit us to know more and maybe, start contributing to the project while in sprints.