28. April 2020
Migrating to Python 3 on Google App Engine - Part 4 - Setting up simple authentication and authorization
In today's post I continue my App Engine Python 3 migration series with adding authentication and authorization features using the tools provided by the Pyramid Web Framework and the Google Sign-In service.
Implementing entry level authentication and authorization for my blog application was the migration step, that encompassed the most changes so far.
The subtler details of securing your application are inherently specific to the use case and the architecture of the software in question. Hence, as my primary focus is to provide a good overview of the development steps needed, I have simplified each stage of the implementation and removed finer details, that are too specific to be useful to most.
As such, this article can by no means considered a definitive guide for securing your Pyramid web application, it is solely a mid-depth overview of what I have done to secure mine.
Why do I need this?
Securing my application is required, because, as mentioned in the first post of this series, Google has sunsetted the authentication & authorization service built into the App Engine runtime, it is no longer available in the Python 3 Standard Environment and as I wanted to migrate to this environment, I had to secure the administrative pages of my blog application myself.
Overview
There are various concepts and components in the Pyramid web framework that support the access control security of an application.
I will go through the components required to set up a basic authentication and authorization system for my blog engine.
First, I will cover the configuration of the access-control list (ACL), that defines different levels of permissions. These can be linked to one or more authenticated users using a concept called a principal.
Configuring the means of authenticating users will be the second step, I will configure a policy, that issues authentication cookies to clients once they have provided their credentials and use the issued cookie to verify the identity of those users later on.
Then, I move on with linking the views to be secured with their respective permissions in the ACL. This will ensure, that only the users, that are allowed for that permission will be granted access to the view.
Finally, the code logging the user in and out is implemented. The user is logged on on the client side using the JavaScript library provided by Google and the token acquired is verified on the server side.
Let's get things started with setting up the access-control list (ACL).
Configuring the ACL
Resources are objects that represent components or locations within your application. You can build a tree structure from these objects when your application is bootstrapped. Their main usage domain is routing using traversal. Resource object have other functions as well however.
One important control that can be defined on a resource object is a permission model called an access-control list or ACL for short that determines what action needs to be taken for a matching principal id for a given named rule (permission).
Since my blog engine uses URL dispatch, not traversal for routing purposes, and my authorization needs are pretty basic, I only need one resource object to define my ACL. This can be done by creating an __acl__
attribute in the resource class:
# module: resources
from pyramid.security import Allow, Authenticated
class Root:
__acl__ = [(Allow, Authenticated, 'view'),
(Allow, 'blogadmin@tamaskalman.com', 'admin')]
def __init__(self, request):
pass
__acl__
is a list of one or more tuples, these are called access-control entries (ACE), and each of these should have three items:
- An action;
pyramid.security.Allow
is used to allow access for the user(s) designated by the second item of the tuple. - A principal; which is either a string, describing a concrete user or a group of users, or a special class. Class
pyramid.security.Authenticated
is a built-in special group, that includes all authenticated users. - The name of the permission. This can be used to refer to the ACE from views as you will see later on.
I will touch on the subject of view permission specifications later, but let's see how the ACL will determine how a request is handled on a high level now:
- The request comes in and the appropriate view is matched by the routing engine.
- On the view, we have the permission required in the view decorator. This will match the appropriate ACE in the ACL.
- Once the ACE is matched, the authenticated user name will be matched against the principal.
- If the user matches the principal definition, the action specified in the ACE is performed. The request will either be allowed (
pyramid.security.Allow
) or a denial response (pyramid.security.Deny
) will be sent to the client.
Loading the resource holding the ACL
Once the ACL configuration of my Root
resource was ready, I have passed the import path of the class to the Configurator
constructor using the root_factory
keyword argument:
# module: main
from pyramid.config import Configurator
...
config = Configurator(settings=settings, root_factory='resources.Root')
The ACL on the resource will be automatically looked up for each request once my authorization policy is set up. Hence the next steps will be configuring the authentication and authorization policy.
Creating the authentication policy
An authentication policy in Pyramid is a chunk of code, that associates requests with a user (principal). The API required for this policy code is defined by the interface pyramid.interfaces.IAuthenticationPolicy
. Pyramid also offers a set of default policy implementations, that you can use almost out of the box.
For my application, I have picked pyramid.authentication.AuthTktAuthenticationPolicy
, this policy uses an authentication ticket cookie to link a request with a user. The cookie is issued when the user logs in and it can be looked up for authentication from there on. To give you an overview of the process, this is how the high level flow looks like using this policy for me:
- User initiates the login process, it sends her credentials.
- The credentials are verified.
- If the credentials are correct, the auth ticket cookie is sent to the client.
- Subsequent requests are sent with the auth ticket cookie set in the request header.
- The user's identity is extracted from the cookie and cross checked with a data source containing existing, previously authenticated users.
- If the user's identity is valid, it is matched with the principal of the ACE in effect for the resolved view, which will decide if the request should be allowed.
No. 3 and 5 are the steps where the authentication policy plays a key role, so, after presenting my policy implementation I will go into the greater details of how those steps are being handled.
# module: security
from pyramid.authentication import AuthTktAuthenticationPolicy
from models import User
class GoogleSigninPolicy(AuthTktAuthenticationPolicy):
def authenticated_userid(self, request):
useremail = request.unauthenticated_userid
if User.query().filter(User.email == useremail).get():
return useremail
Two methods implemented by AuthTktAuthenticationPolicy
play a key role in step 5:
unauthenticated_userid
uses the cookie issued in step 3 to obtain the user's id, it doesn't do any further verifications apart from extracting the id from the cookie payload.authenticated_userid
is a method, that can be overridden in a subclass to invoke further verification steps on the extracted id. If it isn't overloaded, the default implementation just returns the user id returned byunauthenticated_userid
.
Ultimately, authenticated_userid
is the method called by Pyramid to obtain the user id authorization decisions will be made upon.
Also, once a policy is configured for the application, these two methods are also added to each request object as members and thus, they can easily be interacted with from views if needed.
I have overridden authenticated_userid
in my code to add an additional step, that checks the list of existing users in the Datastore and only returns the id, if the user exists in the database.
Loading the authentication and authorization policy
When configuring Pyramid imperatively, meaning using Python statements, the framework has a nice facility to include certain aspects of the configuration, eg. authentication and authorization, that fit nicely together, from a separate module.
For instance, if you have a module file security.py
, you could include the configuration done in that module, like this:
# module: main
from pyramid.config import Configurator
...
config = Configurator(settings=settings, root_factory='resources.Root')
# include the config module
config.include('security')
...
In the module, you have to adhere to the appropriate API, which is pretty simple: a function called includeme
, that has a single argument, let's call it config
should be defined. Pyramid will pass the application's Configurator
object when it automatically calls includeme
as the module is included.
In my application, this is how includeme
looked like in the security
module, where my authentication policy class GoogleSigninPolicy
is also defined:
# module: security
from pyramid.authentication import AuthTktAuthenticationPolicy
from pyramid.authorization import ACLAuthorizationPolicy
from models import User
class GoogleSigninPolicy(AuthTktAuthenticationPolicy):
# as shown earlier
...
def includeme(config):
authn_policy = GoogleSigninPolicy('mysecret')
authz_policy = ACLAuthorizationPolicy()
config.set_authentication_policy(authn_policy)
config.set_authorization_policy(authz_policy)
config.set_default_permission('admin')
When includeme
is called, the following steps are carried out by the code above:
- My authentication policy class
GoogleSigningPolicy
is instantiated, the first argument of the constructor is the secret used to sign the cookie payload,'mysecret'
in the example above, make sure, that this is a sufficiently long, randomly generated secret. Reading Pyramid's succinct guideline section for secret sharing is highly recommended. - The authorization policy class
ACLAuthorizationPolicy
is instantiated. This authorization policy implementation is provided by Pyramid and supports the ACL based authorization model, that I have used out of the box. - Both policy objects are set in the configuration.
- The default permission will be mapped to the
admin
ACE from the ACL configuration I specified earlier. This is to avoid security holes, in case I forget to set a permission on a view. This way the view will be secured by default until I explicitly configure it otherwise.
Configuring the views
In order to grant or deny access to certain parts of the application, you have to assign the appropriate permissions (ACEs) to each view. The way, that I have done this is using the permission
argument of the view_defaults
or view_config
decorators we also touched on in the previous blog post of this series.
For example, my views related to writing posts use the admin
permission:
module: views_admin
from pyramid.view import view_config, view_defaults
...
@view_defaults(renderer='blog/admin/templates/postadmin.jinja2', permission='admin')
class PostAuthoring:
def __init__(self, request):
self.request = request
@view_config(route_name='post_authoring', request_method='GET')
def list_posts(self):
# handle request
...
As the default view has been set to the admin
permission however, it is not mandatory to define this permission on each view that requires this permission, but I would recommend it to make this explicit on the views, as it can avoid confusions later on.
On the other hand, it is a must to define the correct permission on the views, that need a different permission from the default, for instance, the majority of my blog is public facing, that should be accessible to anybody. You can set views up this way using the built-in configuration value pyramid.security.NO_PERMISSION_REQUIRED
:
# module: views
from pyramid.view import view_config, view_defaults
...
@view_defaults(renderer='blog/templates/blogpost.jinja2',
permission=security.NO_PERMISSION_REQUIRED)
class BlogPost:
def __init__(self, request):
self.request = request
@view_config(request_method='GET', route_name='blog_post')
def show_post(self):
# handle request
...
Since my authorization model is very simple, I only have these three levels of permissions used, including NO_PERMISSION_REQUIRED
. If you need more, you just have to add further entries to your ACL and refer to them in your views.
Logging the user in
The authentication and authorization policy I have set up is able to deny or grant access to a user based on her identity stored in the auth cookie, but the user has to obtain this cookie, which is why a login workflow is needed.
As mentioned earlier, I am using the Google Sign-in service mainly as it marries convenience with best-in-class security.
Google Sign-in can be integrated in an application multiple ways, I have used the simplest integration scenario, where the authentication workflow is roughly the following:
- I have created a sign-in page, where I load the JavaScript API library to access the Google Sign-in service.
- As the user logs into her Google account, if she is not logged in already, an authentication token is obtained from Google using the JS API.
- The authentication token is submitted to my server.
- In the view handling the token submission, a request is made using the Python API library of the Google Sign-in service. The API makes a call to verify the token with Google and returns a mapping with the user account information.
- The information returned can be used to verify the properties of the user account, including the user's id, and linked email address.
- If the user matches the criteria required, an authentication ticket cookie is issued to the client.
Let's go ahead and build the plumbing needed for this. First I will create a template for the login page, which will load and call Google's JS API to obtain the user's token, then I will create the view that verifies the token and sets the cookie if applicable. The final step is the creation of the view that will render the login page, and another one, that will log the user out when needed.
The login page template
Once I had my concept for the login workflow in place, I have created a new blank template for the login page and added the components needed step by step. Let's see the details.
First of all, I have written the skeleton of the template:
{% extends 'blog/admin/templates/blogadminlayout.jinja2' %}
{# template: auth.jinja2 #}
{% block login %}
<h1>Login</h1>
<p id="auth-message">{{ message }}</p>
<p>Logged in as:
{% if login %}
<strong>{{ login }}</strong>
<a href="{{ request.route_url('logout') }}" id="logout">(Log out)</a>
{% else %}
No user logged in.
{% endif %}
</p>
{% if login %}
<a href="{{ request.route_url('blogadmin_home') }}" class="btn btn-primary">Admin home</a>
{% endif %}
<div class="g-signin2" data-onsuccess="onSignIn"></div>
{% endblock login %}
Let's go through the basic items we will get started with on our login page template from top to bottom:
- I am extending a layout template used for my admin interface, so only the core of the page has to be implemented. You might need to add additional HTML elements depending on your layout usage.
- There is a heading showing Login.
- There is a
message
template variable embedded in a paragraph. I will use this to display system messages related to the authentication. - If the
login
variable is defined, meaning, that the user is logged in, the template displays the user's login and a hyperlink, that can be used for logging out of the system. Otherwise, it just shows the message No user logged in. - If the
login
variable is defined, meaning, that the user is logged in, the template will display a hyperlink to the administration dashboard. - Finally, I have added the
<div>
element required for the Google JavaScript API to show the sign-in button.
Loading the Google Sign-In JS library
Before I could use the API library of the Google Identity Platform, I had to obtain my Google API client ID.
You have to add all the URLs that are used to access your application, including localhost if you want to test locally as well, to the list of Authorized JavaScript origins of your Client ID configuration.
Once I had my client ID configured, I have added the following to the end of my template, before the endblock
directive; this loads the API library:
{% extends 'blog/admin/templates/blogadminlayout.jinja2' %}
{# template: auth.jinja2 #}
...
<script src="https://apis.google.com/js/platform.js?onload=init" async defer></script>
<meta name="google-signin-client_id" content="yourgooglesigningclientid.apps.googleusercontent.com">
{% endblock login %}
Where yourgooglesigningclientid.apps.googleusercontent.com
is the client ID obtained in the previous step.
When the login page (template) is rendered, it shows a Google sign-in button that the user can use to log in with her Google account. If the user is already signed into Google on another tab, this will happen automatically. The next step is to process the token received from Google when the sign-in is triggered. I have written a simple JavaScript function to take care of the token submission:
// auth.js
// Handler for Google sign in
function onSignIn(googleUser) {
gapi.load('auth2', function() {
var auth_control = gapi.auth2.init({client_id: 'yourgooglesigningclientid.apps.googleusercontent.com'});
// Sign user out from Google
function signOut() {
auth_control.signOut();
return true;
};
// Sign in user.
var id_token = googleUser.getAuthResponse().id_token;
document.getElementById('token-in').value = id_token
document.getElementById('auth-form').submit()
});
};
The function onSignIn
above will be called by the Google JS library automatically as the user is signing into her Google account, passing the user object into the function. The first call in the function body gapi.load
loads the auth2
library. Its second argument is a function, that is called once the library is loaded, hence you can safely interact with the library within this function.
I only need this to gain access to the authentication control object, that provides the API for signing out the user, let's skip the sign out portion for now and concentrate on the second part where the user is signed in, I will get back to the sign out functionality later on.
This is what I do to sign in the user step-by-step:
- Obtain the token from Google and store it in the
id_token
variable. - Store the token in the value property of the DOM input element holding id
token-in
. - Find the form element with id
auth-form
encompassing the input element set in the previous step and submit that form.
Alternatively, you can also submit the token to the server from JavaScript directly (AJAX), but it was a lot simpler for me to just use a form to submit the token.
The next section will show you, how I have created this form.
Creating the sign-in form
The form, that I have created has a single (hidden) input field:
{% extends 'blog/admin/templates/blogadminlayout.jinja2' %}
{# template: auth.jinja2 #}
...
<form id="auth-form" action="{{ request.route_url('login') }}" method="post">
<input id="token-in" name="idtoken" type="hidden">
</form>
...
As explained above, the token value is stored on this input field, then the form is submitted from the onSignIn
JS function.
In the next step, I create the view that will handle the login
route this form is submitted to and set the authentication cookie if the user is eligible.
Handling the login on the server side
I need a view to handle the token submission. In a nutshell, this what this view will do:
- Read the token from the request object.
- Send it to Google using the Python client library for verification.
- Do additional verification locally.
- If everything checks out, check the Datastore and add the user if it is not there already.
- Redirect to the login page.
This is how this looks like in practice:
# module: auth
from pyramid import security
from pyramid.view import view_config
from pyramid.httpexceptions import HTTPFound
from google.oauth2 import id_token
from google.auth.transport import requests
from models import User
@view_config(route_name='login', request_method='POST',
permission=security.NO_PERMISSION_REQUIRED)
def login(request):
# Read the token data submitted by the form
token = request.params['idtoken']
# Verify Google token
try:
idinfo = id_token.verify_oauth2_token(
token,
requests.Request(),
'yourgooglesigningclientid'
'.apps.googleusercontent.com')
if idinfo['iss'] not in ['accounts.google.com', 'https://accounts.google.com']:
raise ValueError('Wrong issuer.')
if 'hd' not in idinfo or idinfo['hd'] != 'tamaskalman.com':
raise ValueError('Wrong hosted domain.')
if not idinfo['email_verified']:
raise ValueError('Email address is not verified.')
userid = idinfo['sub']
useremail = idinfo['email']
except ValueError as ex:
# Invalid token
return HTTPFound(location=request.route_url('login',
_query=[('message', str(ex))]))
# Check if user is in the datastore,
# if not, add it.
persisted_user = User.query().filter(User.id == userid).get()
if not persisted_user:
persisted_user = User()
persisted_user.id = userid
persisted_user.email = useremail
persisted_user.put()
headers = security.remember(request, useremail)
return HTTPFound(location=request.route_url('login'),
headers=headers)
The heavy lifting is done by the google.oauth2.id_token.verify_oauth2_token
function, that performs the verification of the user's token and returns a mapping containing basic user id information.
You can use this user information to do further offline verifications depending on your authentication criteria. The documentation gives a concise summary of the contents of the mapping, but here are the items, that I have used to filter undesired users:
iss
holds the issuer of the response, this will always behttps://accounts.google.com
oraccounts.google.com
for Google ID tokens. It is a good idea to verify this to be 100% certain, that the originator of the account information is Google.hd
is optional and is only set if the user belongs to a G Suite hosted domain. This is a good method to exclude any users outside of your domain.email
the email address associated with the account. This might not be unique to this user and as such is not suitable as a primary key.
NOTE: The security of using the email address in your ACL permissions might be plausible if you are not limiting the accounts accepted to your hosted domain. Checking if the email address has been verified gives a good layer of extra security, but if a user manages to set your email address, having admin privileges in your app, and verify it with Google, she might be able elevate her permissions in your application.email_verified
this flag isTrue
if the email address associated with the account is verified, otherwise it isFalse
.sub
is the unique identifier of the Google user's account. "A Google account can have multiple email addresses at different points in time, but the sub value is never changed. Use sub within your application as the unique-identifier key for the user. Maximum length of 255 case-sensitive ASCII characters."
If any of the criteria set is not met, the associated error message is shown on the login page and the authentication is aborted.
If everything checks out, the view tries to look up the user in the datastore, if it isn't found, then the user is automatically registered there. This workflow might or might not work for you, it is also possible that the user has to be present in your database already, eg. by manually adding the user, if that suits your application and security model better.
After the previous step is completed, the code sets the authentication cookie for the client, this is done by using the pyramid.security.remember
function, which generates the HTTP header to be sent. Actually, this invokes the identically named method of the policy object itself.
Once the header is prepared, an HTTP redirect response is returned with the cookie header set on it. From this point on, the user will be identified using this auth cookie and can be matched against the permissions set in the ACL for subsequent requests.
Creating the login page's view
I have already shown the authentication page's template, however there is still no view defined to render it, so that will be the next thing to create.
Since this view is extremely simple, I show you the code first:
# module: auth
from pyramid.view import view_config
...
@view_config(route_name='login', request_method='GET', renderer='blog/admin/templates/auth.jinja2',
permission=security.NO_PERMISSION_REQUIRED)
def login_page(request):
return {'message': request.GET.get('message', ''),
'login': request.authenticated_userid}
You can probably figure out by now, what this view does, but let me sum it up briefly:
- The view uses the template I have shown earlier to render the response.
- It looks for the
message
field in the URL query string and it either passes its value to the template, or, if the field is absent, an empty string. I use this to send authentication error messages to the user, if thelogin
view encounters an error and redirects to this view. - It sets the
login
name of the user logged in, orNone
if the user has not yet logged in.
You might have noticed, that I am using the same route for the token submission and for rendering the login page. This is fine, as they are distinguished by their HTTP method, so the routing engine will know which view to engage for which request.
Handling errors and preventing loops in the client code
Now that we have implemented both the login_page
view, that renders the authentication page, and the login
view, that verifies with Google that the user's token is a valid one and issues an authentication cookie to the client, we are only a few steps away from being able to log in our first user.
However, there are still some issues that need to be taken care of in my workflow. Since as soon as the Google API JavaScript library is loaded the user will be automatically signed in, if she is already logged into her Google account, we need to avoid a looping situation, where:
- The user is already authenticated, but since I redirect to the login page after the user is logged into our application, the sign-in workflow will be triggered repeatedly.
- The authentication fails and the error message is shown, but since the user is already logged into the Google account, again, another sign-in attempt might take place immediately failing over and over again.
To handle these two scenarios, I have to update my authentication code, so that it detects if the user is already logged in, or encountered an error while logging in.
In the onSignIn
function in auth.js
, the following statements trigger the token submission to the server as shown earlier:
...
// Sign in user.
var id_token = googleUser.getAuthResponse().id_token;
document.getElementById('token-in').value = id_token
document.getElementById('auth-form').submit()
...
I have decided to add different branches to handle the scenario where the user is already authenticated or the login has failed:
// auth.js
...
// end of the onSignIn function
logoutEl = document.getElementById('logout')
if (logoutEl) {
// Also sign out from Google when logging off.
logoutEl.addEventListener("click", signOut);
} else if (document.getElementById('auth-message').innerText) {
// There is a problem, get rid of the Google session.
signOut();
} else {
// Sign in user.
var id_token = googleUser.getAuthResponse().id_token;
document.getElementById('token-in').value = id_token;
document.getElementById('auth-form').submit();
};
...
Two additional branches have been added to the sign in logic:
- If the user is already logged into the application, the
login
template variable will be set, and as a result, the logout link will be rendered. So I can look for the logout link element and if it exists, there is no need to send the token to the server. Additionally I add thesignOut
function as an event listener of the logout link, to also sign the user out from its Google account if Sign out is clicked. - If an error message is shown, the JS code doesn't send the token to the server again, instead we log the user out from her Google account.
Of course the solutions presented above are, deliberately, rudimentary in nature and further refinement is necessary depending on the particular requirements, but they are a good starting point to demonstrate how the process can be refined further if needed.
Logging the user out
The necessary building blocks for logging users off are already built into both the template and the JavaScript code, however, I haven't set up the view handling the logout for our application yet!
The view is very simple:
# module: auth
from pyramid import security
from pyramid.view import view_config
from pyramid.httpexceptions import HTTPFound
...
@view_config(route_name='logout', request_method='GET',
permission='view')
def logout(request):
headers = security.forget(request)
return HTTPFound(request.route_url('login'), headers=headers)
All this view does, is generating the header necessary to unset the authentication cookie using pyramid.security.forget
, then redirecting to the login
route with this header set.
Adding the route configuration
I have everything set up now, so I only have to add my authentication view file and routes to the configuration.
Adding the module holding the authentication views:
# module: main
from pyramid.config import Configurator
...
config = Configurator(settings=settings, root_factory='resources.Root')
...
# new routes
config.add_route('login', '/blog/admin/login')
config.add_route('logout', '/blog/admin/logout')
...
# new view module
config.scan('auth')
Disabling App Engine authentication and test
Now it's time to test if the authentication and authorization works correctly. The whole purpose of setting this system up for me is to replace the built in App Engine authentication service configure via the login
element in app.yaml
, so this element should be removed:
# app.yaml
...
- url: /blog/admin/?(.*)
script: main.app
# this can be removed now
login: admin
...
The testing can begin now, I fired up dev_appserver.py
and started testing using my Google account.
Wrapping up
I have all the plumbing to make authentication and authorization possible in place now.
As a quick recap, here are the steps I covered in this post:
- Setting the ACL up on my root resource.
- Creating the authentication policy.
- Including the authentication and authorization policy into Pyramid's app configuration.
- Configuring the views with the appropriate permissions.
- Creating the authentication page including the client JS code to handle the sign in process.
- Implementing the server side view to handle the token submitted by the client upon sign in.
- Adding additional code to handle errors, special cases and logging the user off.
- Adding the route configuration.
These steps encompass the essentials needed to have an authentication and authorization workflow working using the tools provided by Pyramid and the Google Sign-In service. Like I have mentioned at the beginning of this post, all the steps included provide the skeleton to have the respective components functioning, but all production implementations will require additional fine tuning and extension of the presented solutions. Here is a non-exhaustive list of what else is possibly needed to enhance access security:
- Creating a forbidden view that redirects unauthenticated users from secured pages to the login page would improve the user experience.
- Creating multiple permission levels as needed, that can be assigned to authenticated users by a user administrator. Users should have no, or very basic permissions by default.
- In my example, I have added new users to the database automatically, alternatively, users can be blocked from authentication altogether, until the new user is added to the user database manually.
- You can create resource object trees to implement complex RBAC models.
- The detection of different error and authentication states can be improved in the client code.
- The validity of auth ticket cookies can be controlled for automatic logout timeouts.
That's all for today, this post got a bit long, so if you got this far, thanks for reading and I hope you got something useful out of this.
Next up
In the next post of this series, I will wrap things up with making things ready for the App Engine Python 3 Standard Environment by updating app.yaml
and creating some additional configuration files. Once everything is set up, I will test the application locally and finally, deploy to Google Cloud.