Implementing User Authentication

Earlier in the blog, I showed how to use a 3rd party OpenId service to allow people to login to your site using their Google account, their Facebook account, etc.

That isn't appropriate for every type of site, so now I'll show how to allow people to register at your site and create a login and password specific for your site.

I'd planned on using Spring Security for this, so that the authentication would be robust. But, Spring Security has some dependencies on the Spring framework. Supposedly getting it to work without Spring is possible, but not knowing either product it probably won't happen in the limited amount of time I have available (sorry for those of you searching for how to do just that).

I do, by the way, highly recommend Spring for industrial strength applications built on Java servlets. It just isn't appropriate for the class that I'm teaching. So, the authentication we'll roll here won't be industrial grade. But it'll be a start.

Let's talk for a moment about the characteristics of a good authentication mechanism.

First, it must be impossible to fool. For example, an authentication mechanism that simply avoided displaying members only links to non-members would not prevent non-members from typing those links in directly. The server must authenticate on any restricted page load.

Second, it must be easy to use. The more places we have to touch to implement authentication for a page, the higher the chance that we'll get it wrong. In particular, if we have to write Java code for each page we want protected, the chances are good that we'll get it wrong sooner or later.

We'll use Java Servlet filters to accomplish this. A filter is basically a piece of Java code that is run before a servlet is run. The filter can allow the servlet to run, or it can display a page itself and block the servlet. This happens without adding code to the servlet itself.

First step is to set up our web.xml file to use the filter (okay, so we haven't written the filter yet, but editing web.xml is the easy part so we'll get it out of the way). Add something like the following to your web.xml:


<filter>
<filter-name>AuthenticationFilter</filter-name>
<filter-class>omega.server.AuthenticationFilter</filter-class>
</filter>

<filter-mapping>
<filter-name>AuthenticationFilter</filter-name>
<url-pattern>/members/*</url-pattern>
</filter-mapping>


This is like registering a servlet, in that you specify a URL pattern that triggers the filter. In my case, I used /members/*. This means that any URL under the members directory will trigger the filter. This will require changing my servlet URL mappings around a bit, so that any page that should be restricted to logged in users is mapped under /members, and any page that should be visible to anyone is not. So far for me, that's basically just the ProfileServlet, but I'll make a mental note to put future members only pages under /members.

Now we have to write the AuthenticationFilter class. This is similar to writing a servlet. Create a new class in your server package. There's no superclass for this one, but you will implement an interface. So in the new class dialog, click on the Add button to the right of the (empty) interface list, and type in Filter. You want the javax.servlet.Filter interface.

You'll get some Java code that leaves stubs for three methods: init, destroy, and doFilter. The primary purpose of the init is to save off the FilterConfig passed in, so create a private data member for doing that:


private FilterConfig filterConfig = null;

public void init(FilterConfig arg0) throws ServletException {
filterConfig = arg0;

}


You may not need that FilterConfig instance, but it can be used to get the ServletContext of the servlet that you're filtering.

The actual work of authentication will be done in the doFilter method. Here's the basic idea (not working code, but algorithm):


if (user authenticates correctly)
  chain.doFilter(request, response);
else
{
  User the response argument to generate error output, or redirect to a login form
}


So calling chain.doFilter passes the request on to the next filter in the chain, meaning that the AuthenticationFilter is happy with the request. If authentication fails, do something other than calling chain.doFilter.

The details of pulling a user's password from the datastore and comparing it to one passed into the HTTP request we've already covered. You should also be able to write the registration form and servlet to allow users to create their user account in the first place (and you can do verification of their email, too). You should also know what you need to know to have a "I forgot my password" link that will email a new password to them.

The one bit that we do still need to cover is storing passwords in an encrypted form. Storing user's passwords in plain text is bad form, so we won't do it.

Next post, encrypting passwords.

Signing A User Out

I realized that I haven't shown how to sign a user out yet. I have a link to /signout in my navigation template, but nothing mapped to that URL. This is where having a project plan (or at least a to-do list) would help!

So, I'll create a Signout servlet mapped to /signout. In that servlet, I need to get the session object and call the invalidate method on it. Afterward, I'll redirect the user to the front page.

It'll look something like this:


HttpSession session = req.getSession(false);

if (session != null)
session.invalidate();

resp.sendRedirect("/front");


and that's it. Not a big deal, but something we needed to implement.

Validating User Emails, part 2

The last two steps of validating user emails involved changing the profile servlet to display the link to resend the verification email when they already have an unverified email, and to send the verification email when they change their email.

Since both of those require sending the verification email, I'll put that in a utility class. I already have MailUtil, and this is a special sort of email, so I'll stick it there.

The main problem so solve before I can send the verification email is generating the unique verification code. I mentioned that MD5 is fine to use here, so we'll use that. The MessageDigest class in Java is what we use to generate the MD5 value, like this:


MessageDigest md5 = null;

try
{
md5 = MessageDigest.getInstance("md5");
} catch (NoSuchAlgorithmException e)
{
// Should never happen, but just in case
return "Unable to access md5 digest algorithm";
}

byte [] digest = md5.digest(user.getEmail ().getBytes());


Note the exception handling...that exception should never happen, because MD5 is a standard digest algorithm. But since the exception can be thrown, Java requires we handle it. This assumes the method returns a String that's an error message if something went wrong.

The byte array we get from the MessageDigest is not suitable for putting into an email yet. We need to convert it into hexadecimal first, like this:


String code = "";

for (byte element : digest)
code += Integer.toHexString(element & 0xFF);


If you're wondering about the use of "& 0xFF", it's because of the way that integral values are stored in Java. Think in terms of twos-complement sign extending from 1 byte to 4 bytes, and getting a bunch of extra 1s. We strip off anything extra by anding it with a single byte of 1s.

And if none of that makes sense, just believe me that you need it (or don't believe me, and remove it, and you'll see the difference in the hexadecimal generated).

So what I end up with is a hexadecimal string suitable for use as a verification code.

I also want to store the email I'll send as a Freemarker template, rather than have it embedded in Java code. This means a couple of things: I'll need to pass the Template object into the MailUtil method, and when I process the template I'll need to use a StringWriter so I can get the results as a String. The StringWriter serves in place of how we've been using PrintWriter for the templates.

I'll leave that, and finishing off the send verification email method as an exercise for the reader.

The last bit I'll mention here is that we'll need to include a link to the servlet we wrote last post that handles the verification clicks in the email. In the template we'll use something like this in the href arg of an a tag:


http://localhost:8080//validate-email?code=${code}


We'll use localhost for testing, but will want to change that when we're deploying the application. For those keeping track, we now have two places where we've embedded localhost and made notes to change it when we deploy.

Might be a very good idea to include a programmatic way to switch this depending on whether the application is running on the server or on the local machine. You can check that like this:


if (SystemProperty.environment.get().contains("Development"))


That if statement is true when you're running on the local host, and false otherwise. Use that if statement at appropriate points to change the host name used in URLs. The SystemProperty class used there is an App Engine provided one.

(Update: I just realized that the SystemProperty class doesn't exist on the App Engine version installed in the computer lab. So we'll likely update to the most recent version at some point, to keep things in sync with what students would have at home.)

Okay, so if you've done the exercises left for the reader, you have a MailUtil method that will send a verification email. We need to invoke that method in the profile servlet when the user changes their email, using code like this:


private String sendVerificationEmail (User user) throws IOException
{
Configuration config = ConfigFactory.get(getServletContext ());
Template template = config.getTemplate("emailVerify.ftl");

try
{
MailUtil.sendVerificationEmail(user, template);
}
catch (Exception e)
{
return e.getMessage ();
}

return "";
}


This is in the profile servlet, and will be called from the code that updates the email in the User. I'm not showing those parts, because they're basically the same as we did for updating the name.

The last bit we need is to display the link to resend the verification email when the user is editing their profile and their email isn't verified.

I'm going to add a query parameter to the ProfileServlet that will tell it to resend the verification email. It'll be something like this:


if (req.getParameter("resend") != null)
{
// Only resend if the email isn't verified already
if (! user.getEmailVerified())
{
message = sendVerificationEmail (user);
}
}


Now that the profile servlet will handle that query parameter, put the link in when the email isn't verified. I'll add the link to the ProfileLoggedIn template, just as a variable, e.g. ${resendLink}. Then in the profile servlet, when the email isn't empty but isn't verified, I'll set that variable to the resend link in the data model. When the email is empty, or verified, I'll set that variable to an empty string.

Since that uses only techniques we've already seen, I'll leave that as an exercise for the reader. Note also that I haven't tested the above, so it might very well blow up. When I get a chance to test it I'll update this post to reflect any fixes needed.

Validating User Emails

First step in adding user email addresses is pretty basic.

We need to add an email field to the User object, and add it to the ProfileServlet form and processing, so that the user can enter the email address. We'll also need a field that tells us if the email has been validated or not. That field won't be user editable via the profile form.

I'll leave the first step as an exercise for the reader (it uses only techniques we've already seen), and move on to email verification.

The basic strategy with email verification is threefold:

1) The user enters their email address in their profile

2) The web site sends an email to that email address, with a link to click

3) The user clicks that link, taking them back to the web site

That way, we know that the user entered a valid email address they control.

Step 1 is pretty basic and uses techniques we've already covered. Step 2 requires sending email, which we'll look at. Step 3 requires having a page the link in the email goes to, where we can mark the email as verified.

Tying together step 2 and step 3 means we'll have to generate a unique code for this particular email verification. This unique code needs to be impossible to predict, so users entering false email addresses cannot simply guess the verification code. It also needs to be something that can be used as a query parameter in a URL. An MD5 hash is a good candidate, and we can check the database to make sure it's unique before sending the email.

Okay, lots to do!

1) Create a class to send the activation email

2) Create a servlet to receive the click from the activation email. Either mark the email as verified, or tell the user the verification email cannot be found

3) Change ProfileServlet to automatically send the activation email when a new email is entered, and to set the verified status to false

4) Change ProfileServlet to provide a link to resend the activation email (only display when the email isn't verified)

Also, the GAE Mail API allows an application to receive email. The email gets transformed into an HTTP request to a servlet, which can then save the email text in the datastore or take some other action based on the email. So you can do some pretty sophisticated email driven functionality (such as allowing people to send an email to your app to get a listing of contests in their area). For now, we'll use this feature just to handle when the verification email bounces.

Sending Emails

Since this is something we might have to do often, I'll write a MailUtil class that will hide a little of the complexity. Error handling will still be exposed, since we'll need to display an error message on the web page. Here's what the only method so far in the MailUtil class will look like:


public static void sendOne (String subject, String body, String name, String address, String fromName, String fromAddress) throws Exception
{
Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);

Message msg = new MimeMessage(session);

msg.setFrom(new InternetAddress(fromAddress, fromName));
msg.addRecipient(Message.RecipientType.TO,
new InternetAddress(address, name));
msg.setSubject(subject);
msg.setText(body);
Transport.send(msg);
}


To put multiple lines in your message body, use the standard \n escape in the body text.

Note that running this in Eclipse doesn't seem to actually send email. Before this will work, you'll have to get an actual application created on Google App Engine for Java, and deploy the application. But you can safely call the method to send email when running it in Eclipse...it just won't send the email.

Verification Servlet

It might seem odd to create the servlet to respond to the user clicking the link in the email when we haven't written the code that generates that link yet. But it all has to be done in some order, so I'll do the processing of the click first.

I'll start by copying FrontPage.java and calling it ValidateEmail. I'll also add in web.xml mappings so that /validate-email calls the ValidateEmail servlet.

To recap the changes to User to support all this, I added in an email field (a String), an emailVerified field (a boolean that starts out as false), and a verificationCode field (String).

In the verification servlet, I want to look at the verification code passed in as a query parameter, and use it to look up the matching User in the datastore. The verification codes will be unique, so I'll only get zero or one User matching it.

If it's one User, I'll mark the email as verified. Otherwise, I display an error message. Here's a portion of that code.


Map root = new HashMap();

String code = req.getParameter("code");

query = pm.newQuery("select from omega.server.User where verificationCode == codeParam");
query.declareParameters("String codeParam");

try
{
results = (List<User>) query.execute(code);

if (results.size () == 0)
root.put ("errorMessage", "Verification code not found. Try resending the verification email.");
else
{
user = results.get(0);
user.setEmailVerified(true);
pm.makePersistent(user);
root.put ("errorMessage", "Email verified!");
}
}
finally
{
query.closeAll();
pm.close();
}

root.put("heading", "Email Verification Results");
root.put("loggedIn", loggedIn);
root.put("name", name);
root.put("userLevel", userLevel);
root.put("title", "GamesByTeens Email Verification");

Template temp = config.getTemplate("error.ftl");

All of that is added toward the bottom of the copy of the FrontPage servlet (the root.put statements show where the overlap is.

To test this I first load the page with a bogus verification code, to make sure it displays the error. Then I would need to force a valid verification code into the datastore. I can do this up at the start of the verification servlet itself, to make sure that the code I wrote works. Just don't forget to remove your testing code after you're done (or at least comment it out in case you need to use it again later).

This highlights an important part of any development, but web development especially. Make sure your code works on its own before you try to hook it to other parts of your site.

That's enough for one post. In the next I'll come back to the last two steps, both of which are modifications to the ProfileServlet.

Unique Display Names, Part 2

Okay, now we finally get to implementing unique display names.

In RPXResults.java, we have a section where we see if the user is already in the system. There are three possibilities now that we want unique display names:

1) The user is in the system, and has a unique display name.

2) The user is in the system, but hasn't updated their profile with a unique display name

3) The user is not in the system yet

For option 1, we can just do what we're already doing, redirect to the appropriate destination URL.

For option 2 and 3, we need to instead redirect to the profile page so they can enter their unique display name. Once they have entered a unique display name, we want to continue to the original destination URL.

That requires a change to RPXResults.java to redirect to the profile page in appropriate circumstances, and a change to ProfileServlet.java to redirect to the destination URL.

Note that we have a slight conflict here, because ProfileServlet.java wants to set a destination URL cookie, too, which will overwrite the one set by whatever page they were on when they clicked Sign In. Rather than complicate things by having multiple cookies for destination URLs, we'll just make ProfileServlet hide the complexity. It'll set its own cookie only when it is displaying its error template (which is the only time the Sign In link will show).

So we have these changes to make:

1) RPXResults must redirect to the profile page when appropriate

2) ProfileServlet must redirect to itself only when displaying the error template

3) ProfileServlet must require the user name be unique

4) ProfileServlet must, on a successful save of a unique user name, redirect to the destination URL

I won't show the code for 2, since that's just moving the two Cookie related lines into the if statement for showing the error template.

We'll start with the RPXResults changes. Right now we have logic like this:


if (results.size() == 0)
{
// create new user and store it
}
else
{
// Existing user, update needed values
}


We're going to change to something like this:


if (results.size() == 0)
{
// create new user and store it
// Redirect to /profile
}
else
{
if (user name is not set)
// Existing user no name, redirect to /profile
else
// Existing user name okay
}


For the purposes of making the code I post as simple as possible, I'm going to strip out the else portion where the existing user name is okay. The only reason to put code there is to update user records from previous versions during development, and now that we're trending more toward having the user enter their profile information rather than getting it from RPX, we won't need that as much.

I'll move the declaration of String destination = null up above the try block in RPXServlet, so that I can set an appropriate destination page manually if needed.

Here's the changed code for the case where the user doesn't exist in the system yet:


if (results.size() == 0)
{
user = new User ();
user.setIdentifier(identifier);
user.setName("");
user.setUserLevel (0);
pm.makePersistent(user);
destination = "/profile";
}


Note that I don't set the user name from the RPX data now. I'll let the user type in whatever they want in their profile.

The else part of that if is for when they're already in the system, and we just need to make sure that they already have a unique user name set:


else
{
user = results.get(0);

if (user.getName().equals(""))
destination = "/profile";
}


This assumes, of course, that ProfileServlet won't let them set a user name unless it's unique. So let's make that change now.

In ProfileServlet, we have code like this:


if (req.getParameter("submit") != null)
{
if (req.getParameter("name").trim().length () == 0)
message = "You must enter a name!";
else
{
user.setName(req.getParameter("name").trim());
pm.makePersistent(user);
}
}


We're going to add some more error checking, so that pm.makePersistent is only called when the user name is both non-empty and unique. To see if the name is unique, we need to build a new query based on the name.

Inside the else from above, I'll put:


Query nameQuery = pm.newQuery("select from omega.server.User where name == nameParam");
nameQuery.declareParameters("String nameParam");
List<User> nameResults = (List<User>) nameQuery.execute(req.getParameter("name").trim());

if (nameResults.size () != 0)


If the nameResults has some records, that still doesn't mean that the name is not unique. After all, maybe they pressed Submit without changing their name...then we'd get exactly one result, and it would be them. So we need to make sure the result we're getting isn't them.


if (nameResults.size () != 0)
{
if (nameResults.get(0).getId() != user.getId())
message = "That name is already taken, please try another";
}

if (message.equals(""))
{
user.setName(req.getParameter("name").trim());
pm.makePersistent(user);
}


And finally save the user name is it turns out that none of our error checking found problems.

That still leaves us with needing ProfileServlet to redirect appropriately. There's actually a bit of complexity here. We have a few cases to deal with:

1) They clicked on the Profile link, so our destination URL cookie is set to /profile

2) They clicked on Sign In and got here via RPXResults, so our destination URL cookie is set to something else, but their name wasn't unique so we need to stay here for now

3) They clicked on Sign In and got here via RPXResults, so our destination URL cookie is set to something else, and their name is okay so we can redirect

I'll focus my code on the else, just after the call to pm.makePersistent. It won't be pretty, but it'll work (this, by the way, is how unmanageable code evolves, with ongoing changes to existing code...it's a good idea to stop and refactor the code now and then to clean it up).

Just after the call to pm.makePersistent, I'll get the destination URL from the cookie and see if it's /profile. If it isn't, I'll redirect to it. It'll look something like this:


if (message.equals(""))
{
user.setName(req.getParameter("name").trim());
pm.makePersistent(user);

String destination = null;

Cookie [] cookies = req.getCookies();

if (destination == null && cookies != null)
{
for(int i=0; i<cookies.length; i++)
{
Cookie cookie = cookies[i];

if ("destinationURL".equals(cookie.getName()))
destination = cookie.getValue();
}
}

if (destination != null && ! destination.equals ("/profile"))
{
resp.sendRedirect (destination);
return;
}
}


Be prepared for there to be bugs with this, since we have enough cases that you should be worried about the stability of the code. Before going much farther, you really should redesign ProfileServlet to be more straightforward.

But, it should mostly work and shows the techniques to create the same sort of experience you're used to on well done membership sites.

Next up, I think, will be adding an email address to the user's profile, and requiring that their email address be verified before it's used by the system. This will introduce us to the Mail API.

Unique Display Names

I'm going to focus on new techniques in the coming posts, because the semester is starting.

I've decided to make display names unique in the system. I'm not especially happy with this, since it leads to funky display names like Dave5386. But, if I'm going to provide some sort of forum interaction eventually, I also need a way to differentiate between people who might decide to use the same display name.

I could simply require and display some context to the display name, such as "Jay, Muskingum University". But, providing information online for kids that might be used to locate them is not a good idea.

So, I'll go with unique display names. This means I need to change what I already have a bit.

RPXResults.java is what first puts the user into the datastore. Right now it's getting the user name from the RPX records. That isn't guaranteed to be unique. I could auto-generate a unique name, but then they might not go back and edit it.

So instead, RPXResults.java will, when a user logs in for the first time, redirect to the page for editing their profile. They will be required to update their profile with a unique name before doing using the rest of the site as a logged in user.

While I'm at it, I also want to modify RPXResults.java so that instead of redirecting to /front when a user logs in and they do have their profile info set, it redirects to the page they were trying to access. So if they save a bookmark on My Contests, and try to go to that after the session has expired, it'll take them to the login page and then to My Contests after they log in. That's more user-friendly, and best to do now before I have too many of those sorts of pages to edit.

Redirect to Target Page

We'll do the redirect to the target page first. This requires me to pass into /rpxresults the target page. Here's a page at RPXNow about passing query parameters through the token URL.

Unfortunately, that didn't work for me. The extra query parameter simply never came through to RPXResults.java.

Luckily we do have another option. We can store the destination URL as a cookie on the local machine and fetch it from RPXResults.java.

We'll do FrontPage.java first as an example of how to modify the servlets. Here's the code to save the destination URL as a cookie:


Cookie cookie = new Cookie ("destinationURL", "/front");
resp.addCookie(cookie);


I added this to just before the template is displayed. Now we need to modify RPXResults.java to use the value of this cookie as the destination URL. Here's the code that would replace the resp.redirect ("/front") line:


String destination = null;
Cookie [] cookies = req.getCookies();

if (destination == null && cookies != null)
{
for(int i=0; i<cookies.length; i++)
{
Cookie cookie = cookies[i];

if ("destinationURL".equals(cookie.getName()))
destination = cookie.getValue();
}
}

if (destination == null)
destination = "/front";

resp.sendRedirect(destination);


Now that RPXResults.java is set up, I can go back and modify the ProfileServlet and AdminServlet classes to add the cookie in the same way that the FrontPage servlet does.

Now if I remember to do that for every servlet I add after this, the user will always get to the page they wanted to get to, even if they have to log in first.

That's probably long enough for one post, so I'll do unique user names in the next.

Template Updates

A quick post about some template updates.

Following the pattern I used for navigation.ftl, I created header.ftl and moved everything up to and including the opening body tag into it. I replaced the page title with a Freemarker variable, so that each servlet could specify the page title (I then modified each servlet to set that page title). Every other template then included header.ftl at the top.

I did the same with footer.ftl, putting into it just the closing body and html tags for now. Later I'll have more to put there, and will have a central place to put it.

As an example of the end result, here's what my error.ftl looks like now:


<#include "header.ftl">

<h1>${heading}</h1>

<#include "navigation.ftl">

<p>${errorMessage}</p>

<#include "footer.ftl">


Nothing complicated here, just taking a moment to set the templates up for easier modification later on.