pySAML2

pySAML2 is a pure Python implementation of SAML2.0. Roland Hedberg is the original author, and now he maintains the project along with the other team members of Identity Python project.

For the rest of the examples in this book, I will be using Python3.10 in a virtual environment (on Fedora). But, you should be able to follow along in any modern Linux distribution (or on Mac or in WSL).

Creating our own service provider (SP)

We will start with creating a new project directory for our SP. We will also need a virtual environment for the same.

$ mkdir sp
$ cd sp
$ python3 -m venv .venv
$ source .venv/bin/activate
$ python3 -m pip install wheel

We are going to use Flask as the web framework for SP. Let us install a few other packages too.

$ python3 -m pip install flask ipdb pysaml2

Then copy over the certificate & the key file from home directory (we created them in the TLS chapter).

$ cp ~/localhost+2.pem ./backend.pem
$ cp ~/localhost+2-key.pem ./backend.key

Create test IdP

We will use the example provided by pysaml2. For that we will clone the repository, create a virtualenv for the same, install all required packages, and also copy over the key & certificate we created before.

$ git clone https://github.com/IdentityPython/pysaml2/
$ cd pysaml2/examples/idp2
$ python3 -m venv .venv
$ source .venv/bin/activate
$ python3 -m pip install wheel
$ cd ../..
$ python3 -m pip install .
$ python3 -m pip install mako cherrypy
$ cp ~/localhost+2.pem ./server.pem
$ cp ~/localhost+2-key.pem ./server.key
$ cp idp_conf.py.example idp_conf.py

In the last command, we copied the example IdP configuration. We will make three small modifications in this file. First, we mentioned that we want to provide signed response and also changed the server key & certificate files path to use the key & certificate we generated. Also the SP metadata file path is now commented.

--- idp_conf.py.example	2022-07-15 08:31:54.442424007 +0200
+++ idp_conf.py	2022-07-21 14:06:34.328494860 +0200
@@ -69,6 +69,8 @@
         },
         "idp": {
             "name": "Rolands IdP",
+            "sign_response": True,
+            "sign_assertion": True,
             "endpoints": {
                 "single_sign_on_service": [
                     ("%s/sso/redirect" % BASE, BINDING_HTTP_REDIRECT),
@@ -111,10 +113,10 @@
         },
     },
     "debug": 1,
-    "key_file": full_path("pki/mykey.pem"),
-    "cert_file": full_path("pki/mycert.pem"),
+    "key_file": full_path("./server.key"),
+    "cert_file": full_path("./server.pem"),
     "metadata": {
-        "local": [full_path("../sp-wsgi/sp.xml")],
+        "local": [full_path("./sp.xml")],
     },
     "organization": {
         "display_name": "Rolands Identiteter",

Next, we will have to regenerate the metadata for the IdP.

$ make_metadata.py idp_conf.py > idp.xml
[2022-07-20 13:37:20,260] [DEBUG] [saml2.assertion.__init__] policy restrictions: None
[2022-07-20 13:37:20,260] [DEBUG] [saml2.assertion.__init__] policy restrictions: None
[2022-07-20 13:37:20,260] [DEBUG] [saml2.assertion.__init__] policy restrictions: {'default': {'lifetime': {'minutes': 15}, 'attribute_restrictions': None, 'name_form': 'urn:oasis:names:tc:SAML:2.0:attrname-format:uri', 'entity_categories': None}}

Every time we make any change to the configuration, we will have to regenerate the metadata. Same goes for the SP.

Service provider configuration

The configuration of the pysaml2 is the core part of the whole learning process. And I should admit that my brain totally broke when I looked at these at first. Even though the documentation explains each configuration value, it was still a lot. Also this is the first time I had to configure a library/module to use. Before now I always to configure applications, but never a library at this detailed level.

After the first few failed attempts, Ivan showed me an easier way. Instead of writing the configuration from scratch, I started with an example configuration from the SATOSA project.

I am also using the code from Satosa, that way I can very easily parse the configuration from the YAML file and use it.

saml2_backend.yaml

---
module: satosa.backends.saml2.SAMLBackend
name: Saml2
config:
  idp_blacklist_file: /path/to/denylist.json

  acr_mapping:
    "": default-LoA
    "https://accounts.google.com": LoA1

  # disco_srv must be defined if there is more than one IdP in the metadata specified above
  disco_srv: http://disco.example.com

  entityid_endpoint: true
  mirror_force_authn: no
  memorize_idp: no
  use_memorized_idp_when_force_authn: no
  send_requester_id: no
  enable_metadata_reload: no

  sp_config:
    name: "Demo SP written in Python"
    description: "Our amazing SP"
    key_file: backend.key
    cert_file: backend.crt
    organization: {display_name: Example Identities, name: Example Identities Org., url: 'http://www.example.com'}
    contact_person:
    - {contact_type: technical, email_address: technical@example.com, given_name: Technical}
    - {contact_type: support, email_address: support@example.com, given_name: Support}

    metadata:
      local: [idp.xml]

    entityid: http://localhost:5000/proxy_saml2_backend.xml
    accepted_time_diff: 60
    service:
      sp:
        ui_info:
          display_name:
            - lang: en
              text: "SP Display Name"
          description:
            - lang: en
              text: "SP Description"
          information_url:
            - lang: en
              text: "http://sp.information.url/"
          privacy_statement_url:
            - lang: en
              text: "http://sp.privacy.url/"
          keywords:
            - lang: se
              text: ["Satosa", "SP-SE"]
            - lang: en
              text: ["Satosa", "SP-EN"]
          logo:
            text: "http://sp.logo.url/"
            width: "100"
            height: "100"
        authn_requests_signed: true
        want_response_signed: true
        want_assertion_signed: true
        allow_unknown_attributes: true
        allow_unsolicited: true
        endpoints:
          assertion_consumer_service:
            - [http://localhost:5000/acs/post, 'urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST']
            - [http://localhost:5000/acs/redirect, 'urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect']
          discovery_response:
          - [<base_url>/<name>/disco, 'urn:oasis:names:tc:SAML:profiles:SSO:idp-discovery-protocol']
        name_id_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:transient'
        # A name_id_format of 'None' will cause the authentication request to not
        # include a Format attribute in the NameIDPolicy.
        # name_id_format: 'None'
        name_id_format_allow_create: true

We are interested only the values in the sp_config dictionary here. We mentioned the key file and the certificate file. We also defined the URL for callback (via POST or redirect ) in the endpoints as assertion_consumer_service.

The meatadata contains various service related metadata, in our case we are mentioning an idp.xml file. Let us copy over this file from our test IdP.

$ cp ../../pysaml2/example/idp2/idp.xml .

The flask application (SP) will run locally with default flask app port 5000. So, we will have to define the routes for all the URLs mentioned here.

We will need the satosa installed in the virtual environment so that we can reuse that code.

$ python3 -m pip install satosa

app.py

Next, is our main Flask application. Please open the current code in a browser tab. We will go through this code and try to understand it better.

We are first reading the YAMl file for configuration values and then creating a configuration object.

with open("./saml2_backend.yaml") as fobj:
    sp_config = SPConfig().load(yaml.load(fobj, SafeLoader)["config"]["sp_config"])


sp = Saml2Client(sp_config)

Next, we have a rndstr function to create random string.

def rndstr(size=16, alphabet=""):
    """
    Returns a string of random ascii characters or digits
    :type size: int
    :type alphabet: str
    :param size: The length of the string
    :param alphabet: A string with characters.
    :return: string
    """
    rng = random.SystemRandom()
    if not alphabet:
        alphabet = string.ascii_letters[0:52] + string.digits
    return type(alphabet)().join(rng.choice(alphabet) for _ in range(size))

We also have a function to find the IdP's entity ID, called get_idp_entity_id. In our code it is using the global object sp.

def get_idp_entity_id():
    """
    Finds the entity_id for the IDP
    :return: the entity_id of the idp
    """

    idps = sp.metadata.identity_providers()
    only_idp = idps[0]
    entity_id = only_idp

    return entity_id

We have our index page route as hello_world function. This currently shows a login link.

@app.route("/")
def hello_world():
    return '<p><a href="/login/">Login</a></p>'

We also need a /metadata/ endpoint, this will help us to generate the metadata (XML format) for the SP. We need to pass this metadata information to our IdP before hand. We are using the create_metadata_string function from the pysaml2 module.

@app.route("/metadata/")
def metadata():
    metadata_string = create_metadata_string(
        None, sp.config, 4, None, None, None, None, None
    ).decode("utf-8")
    return Response(metadata_string, mimetype="text/xml")

We will come back to this metadata later in the chapter/book. Meanwhile, we look into the most complex function in our code (yet), the /login/ route.

@app.route("/login/")
def login():
    try:
        acs_endp, response_binding = sp.config.getattr("endpoints", "sp")[
            "assertion_consumer_service"
        ][0]
        relay_state = rndstr()
        entity_id = get_idp_entity_id()
        req_id, binding, http_args = sp.prepare_for_negotiated_authenticate(
            entityid=entity_id,
            response_binding=response_binding,
            relay_state=relay_state,
        )
        if binding == BINDING_HTTP_REDIRECT:
            headers = dict(http_args["headers"])
            return redirect(str(headers["Location"]), code=303)

        return Response(http_args["data"], headers=http_args["headers"])
    except Exception as e:
        print(e)

Here, first we get the response_binding value from our configuration, the default value for this is BINDING_HTTP_POST, that is 'urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST'. You will also notice that the function again returns a binding value. The function itself checks for available options from IdP, and returns the correct binding back. For our test IdP it actually returns BINDING_HTTP_REDIRECT, so our code calls redirect function with status code 303. The Location header value contains all required SAML data in encoded format.

This is what it looks like:

{'data': [],
 'headers': [('Location',
              'https://localhost:8088/sso/redirect?SAMLRequest=nVdZj9rYEn7vX4HII%2Bp4YzOazuh4xRgb78Z%2BGXnDC8YGH%2B%2B%2FftzdSTqJckdzBwmJqlPL91XVqSP%2B%2BLO%2F5bM2qmBaFi9z7DM6%2F%2FPL0x8FRHegqZNCix5NBOvZZFXA3aR%2BmTdVsSs9mE6id4vgrg52OpCOO%2FwzurtXZV0GZT7%2F7oD9s4MHYVTVU%2B4PD%2FxlntT1fYcgXdd97ojPZRUjOIqiCEoik1EI0%2FjTfCYwL%2FM0fKblaISbwh1q806mZNLNZ9Y3NlOCyQ7CJhIKWHtFPalQHH9GN884amD4jiB22Nadz5iJYlp49ZvXa3I4Zc%2FLwMuTEta7LbrdIhCWSBWFaRUF9XymfCVKpUWYFvE%2Fk%2FTfjeBubxjKs3LSjfkMfCNOlwVsblGlR1WbBpGpHb%2Fz%2F0CweqXvBRC5T9Jb9jYNo0qeUr3MmehWznRl1lVpXUfFLC1mylAnU02%2FTI3Edm8VqGZcWd28%2Bp%2BRvmqmol7eTHdRUaf1MP%2FyezhTr%2FvhL%2Bjdcvwv3wuuURF%2BntrzB%2FKR8zU%2FvtPTeKptU0UzIXyZf5ew%2BcdxFArFpXyXaa8oi3RKlY5vHZGiiUs4A3lcTgST2%2F8YDwzB0NfxeI764DnAlsWnOfJL%2Fn8Z6adBq6D3DBMP%2BxZMiy5RFRVBNDM14WX%2B6bcj%2BG5qVF4BX0sJf5H%2FPwRR0UZ5eY%2FCZ%2FiNyDcw%2Fz7i76uD%2FA4mk8bTdfgvtfqxTu9RLC9voi%2Fb6%2BZGW7wRrvsNlWS2Lmx0zeTPegtf3jH8aP2u%2BV7nr%2FKvU%2FK9qe9Ogo%2FWnB7eKJDd95ZYDf7ysBFpRJOWZge5QVizqGKKEVJ1anvmdV32F55LEKv04Q0Lc%2FNUEWxuUhcsoQQhkLZxKpWSjnjtIs0RJTkiSHURo6LnL01ubOP7xipCdM%2FYe58G8nUBkScXD1cNeoTQjkWtR%2Ff3fiGQJbdVXZXVUl0A9GD5jrPdbFqH6x0Hi629NpKjXYYPI3TTp7ErWaItYEDl0ul6zu93z1hyvR0Z3GZjr1fnA5lrTtF2toaKx%2Fahk7cer9wq89bZpS3IJxS1zDQOJT0iWSRb19eztTawTeZhjkQ9KLLmiD0kuYDblqFnQ%2F8mFzhaqUeEgxxdt%2BqTTpy0u6LeZBQ8CvuoYIJNS%2BDl5aMDP1T8rQtiNHy05LxCScarvQ%2BJfl1xl%2Bku19EXSRBYj6FpUNMx6AQKxIIGjkR475wwcU2CRolQLI6dFzodozoHsXSFpA1koLJHSgVd7MB%2Bn4GAimWLAqXB2Vjl4NYQ0tTVPbu5z5Odb1vNJDOqwXYSveQBZrJ01xUeriUBSkEflxOfn6ISajPZZ47dX30cSzx7GYtsjQV4kvg0xTpn6S4ZQSerDmOpKsPQq9rD5Two1Pg%2FxaKWZ8YQBolhx%2Bm7lBgVlbHyVYdOv191xFddZ49sIdHmO%2FZEiny7ztzzARV4LQ9vFvQJqnbt1SS%2F4tHu7i3PnLOWSwbojvEb3iNDh5XLc6PK962Dc9DjyTGkV18xa61vc3c%2FBaVOWKPHc1BgJ07DtYtjNpUAytP6g9cFn2BUlgKqCcBSoJgOvJ6LoJx6pzIh3NS26njr4NCrVedb10tWEvQhzxEd1MLdHGQ3XTlRiYar9pD2W9xoVfNkZ%2B2Fb%2BPONEbGshKBw5qLqIIFRykx4nhDwrdieTXgGtMP%2BxMpp%2FUtpYvuxkCKXtnxVi7iJr9dRLzcVpY5tMMQrGxSAYj%2BQE16uyKOG1oxlIcqKZsTbg%2FN4mIf1z69ujAqeZ%2FmA1c8es2g9qZAyQVLXFV3GdNJdHUc092v0lPdg3tVn%2FNlnFDj2DpGhx6GZdMnY7dJy5C0eWHvnvoizJYx9AX8sQ8ieZvs7zVw7Q1dcEa6n9bLPshvyxQxdY8S2j0ukcTx6iC80d2dMZZuRnfKG2zkff5kTOOxCW8iMzx8EEsUAHwWZhQqgbf5DZmOpZBOZac7w5UMMF7nf6%2BbLMMAkYrjiopZjlKD6QRc3s8klmeAHVPq2IeLx7UqO00EbF6IFIY6vfToOpQB8M1WY9mDAbI4zn%2BYZ31P7bdgwrFnwe8%2B7O%2FuJxPHPAXoFXtoOLklM5nhfR%2B11jnRdebp8CBL%2FcEyaCrBlaiIK22IvXW4jBbZKWu0dbDRaz6iUa5BrqOCkTiaINbYaCyBCeFePmD2gin5UwYvkmX7D23ZuduUp7bn9Exh8sN83Fp6URN0uCzZgPEP25OxPRd8IeVw9IlNVbjp8IAZTgz6mq0BgRH0SsU94lAddHyAWrhVWJXg7kyHkA8VQffa9noC55bUyP7Sc0t%2BPSZiY4XjyCq5iyZ7oTJDgQqVI48LFwWnFR7WxuoRItPGxbZHJx%2F0nhhLSk1Ryn5IGdoNcqfTC0o8OE1wnPa2yHgafTkZ6wt39Kh6BHXvX%2FNrcuJSzI2YR8M1eIIj8ma9ykw7txfrfr2UeLeRPfmBysdpTyzrq5sCVLFWUMAGRljlp8KD%2BVqu1I1zuxAufbyITYt6Qz2Wh8oBAYdICgUYnRX05JrFDWgj9QCglrl60yUcGVv4WcGxVtZ988Ce96vkXsXn%2Fuotxa23MrFKHmvRGSkDC9VsRDL%2FNG0NMjx5NG%2BobrCMvz0Wv27%2FD%2B37C4H89Hr8%2FL68yT%2F%2F%2F%2Fjy9Dc%3D&RelayState=w55JXDjMjii6zPmk&SigAlg=http%3A%2F%2Fwww.w3.org%2F2000%2F09%2Fxmldsig%23rsa-sha1&Signature=jdwo4NH3Hna4v3GRkKF55H%2Ftlv3OOxfDMkyxQJ%2BauR3bQ8aHiYerpBNITVzSyQFqXcv8Cj2uBeeG%2BEBbVuXsW1fx0LHtJuJw03NxuDnxGZnGaAg8XrEBbwOOCtVat7irAIzG9EanDyslAs1YxM4VQtaoXqlBVDr0ZqVzfWHtNAByfEDJAgR%2B7XMlUkZ8iwLDneT9vAnIj%2BZFsnAl%2BphJMOzoK2zR%2BFw81WndrwSbEdOMSTeONOpckMrQwOP2GGx9T5jehf3e2CikZFN03MAhuY1lP0tlUSdP3nQqPweh2aBD1MlC5mYlijZrmZ1L79fPcRY9UgWXEouw84B%2BmPe0Gg%3D%3D')],
 'method': 'GET',
 'status': 303,
 'url': 'https://localhost:8088/sso/redirect'}

The last two routes are for step 5 of our flow diagram, in this step we validate the response we receive.

@app.route("/acs/post", methods=["POST"])
def acs_post():
    outstanding_queries = {}
    binding = "urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"
    authn_response = sp.parse_authn_request_response(
        request.form["SAMLResponse"], binding, outstanding=outstanding_queries
    )
    #ipdb.set_trace()
    return str(authn_response.ava)

@app.route("/acs/redirect", methods=["GET"])
def acs_redirect():
    outstanding_queries = {}
    binding = BINDING_HTTP_REDIRECT
    authn_response = sp.parse_authn_request_response(
        request.form["SAMLResponse"], binding, outstanding=outstanding_queries
    )
    return str(authn_response.ava)

The sp.parse_authn_request_response validates the response, in future we will put this function inside of a try-except block to handle any failure or to deal with bad response. These functions are supposed to redirect the user (after marking them as logged in user) to the actual resource. But, for now we will just print all the validated information we received from the IdP.

Starting the SP Flask application

Do the following in the same directory of SP (remember to activate the virtual environment first).

$ flask run
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:5000 (Press CTRL+C to quit)

Right now the server is listening on port 5000.

Now, let us move to the test IdP terminal, keep this application running.

Starting the test IdP

We will download the SP metadata first.

$ curl http://localhost:5000/metadata/ -o sp.xml

Update the idp_conf.py file with the metadata file to point to the new XML file.

    "metadata": {
        "local": [full_path("./sp.xml")],
    },

Regenerate the idp.xml and copy it over to SP, and then restart SP Flask application.

$ make_metadata.py idp_conf.py > idp.xml
[2022-07-21 13:37:20,260] [DEBUG] [saml2.assertion.__init__] policy restrictions: None
[2022-07-21 13:37:20,260] [DEBUG] [saml2.assertion.__init__] policy restrictions: None
[2022-07-21 13:37:20,260] [DEBUG] [saml2.assertion.__init__] policy restrictions: {'default': {'lifetime': {'minutes': 15}, 'attribute_restrictions': None, 'name_form': 'urn:oasis:names:tc:SAML:2.0:attrname-format:uri', 'entity_categories': None}}
$ cp idp.xml ../samp-examples/sp/

Now, let us start the IdP.

$ python idp.py idp_conf
[2022-07-21 10:42:25,773] [DEBUG] [saml2.assertion.__init__] policy restrictions: None
[2022-07-21 10:42:25,773] [DEBUG] [saml2.assertion.__init__] policy restrictions: None
[2022-07-21 10:42:25,773] [DEBUG] [saml2.assertion.__init__] policy restrictions: {'default': {'lifetime': {'minutes': 15}, 'attribute_restrictions': None, 'name_form': 'urn:oasis:names:tc:SAML:2.0:attrname-format:uri', 'entity_categories': None}}
[2022-07-21 10:42:25,843] [INFO] [saml2.idp.<module>] Server starting
IDP listening on localhost:8088

On the browser

We will use Firefox as the browser for this step, you can use any browser of your choice. But, we will install saml-tracer plugin so that we can see the SAML request and response objects nicely in our browser.

After starting the tracer plugin, if we open the SP URL http://localhost:5000/ we will see the following.

Firefox & tracer

Next, we can click and login. The username/password combinations are mentioned in the idp.py file. We are using testuser/qwerty combination.

Firefox & tracer, full flow

We can all the claims released by the IdP at the last screen.

If we click on the SAML request in the tracer we can see the actual request.

SAML request

Same about SAML response.

SAML response

This where the tracer becomes so useful :) I personally found it super useful to see the actual SAML request/response and go through the various values inside of those.