Shambling Towards Perfection

The goal for sprint 4 is to improve the database access code and the JSPs, as well as move Objects from the Session to the Request. The sprint backlog items are: Priority Description Points 200 Move properties out of the session so that users can do things like have multiple browser windows open and bookmark pages 6 190 Improve database access code so that pages load faster 6 180 Refactor JSPs to use JSTL 8 170 Make all JavaScript and HTML fully standards compliant 8 I essentially split what I had been thinking of for the old "Move properties out of the session so that users can do things like have multiple browser windows open and bookmark pages" item into 2, with the other half being "Improve database access code so that pages load faster". This makes things more clear as to what exactly I intend to do. This is 28 ideal man hours; the next two weeks look less busy for me so I think I've got a good chance at making it this time. And so it goes.

Today I met with my advisor and we discussed the last sprint, including the issues I had. He decided I should try using the CS email system without authentication, and if that didn't work, the university's email system. I was able to do this during lunch and while the CS system still wouldn't work, I was able to send through the university's system, so that's what we'll be using. This makes the item Fix the email notification system so that it works with the CS department email settings finished, with around 2 hours of work. We also decided to backburner the attempts to get SSL redirection working from the root of the webapp, seeing as I can't figure out the problem and nobody on StackOverflow has even been able to help. As a last resort I can do the redirect programatically, but I would much prefer to do it via Tomcat's configuration if possible. My professor did seem generally pleased with my progress, and when I described the issues I had with PMD...

Sprint 3 is now complete. It went fairly well, although again I had trouble finding time to work on the project. There were some other frustrations which I'll detail below. I finished 3 backlog items: Run automated tools e.g. FindBugs to ensure code is correct and clean (8 hours), Write a shell script or similar to start the database server and tomcat (1 hour), and Fix the build.xml to only copy the correct jars to the web app's lib directory (15 minutes). Then there was the stuff I worked on a lot and didn't get done... I started on both Restrict the login page to only allow access via SSL and Fix the email notification system so that it works with the CS department email settings. For the former, I got SSL redirection working if the user navigates directly to the login page (e.g. http://localhost:8080/ProjectManagementSystem/login.html) but despite my best efforts I couldn't figure out how to get it to redirect if the user goes to the root of the web app (http://loc...

One of the issues for this sprint was improving my code's correctness. I installed and used the Eclipse plugins FindBugs , PMD , and CheckStyle . While these did point out some very valid issues, including not properly closing database resources, doing really bad things in an override of equals, etc., they also pointed out some things that completely flummoxed me. The biggest example is that apparently in Java one can declare virtually anything as final, including method parameters. I had known that methods and classes could be declared final though I had never used such functionality (nor had it explained or recommended either during my Bachelors degree or during my Masters degree, despite MANY Java-centric classes), but parameters was a new one. Some claim parameters should be declared final, while others say it clutters code for very little benefit. Additionally if you do, in fact, reassign a value to a parameter, the very same tools will mark it as an error. So there's qu...

Just for fun, and because I don't have a whole lot to report right now, here's my current home office set-up. These photos were taken after I'd gotten most of the product backlog completed, but before I started on any sprints. Whiteboard (made from shower board from Home Depot), bookcase containing all my computer science and math books, electric kettle for a steady supply of caffeine, and computer tower. Monitors, printer/scanner, plasma ball, headset, speakers, printer, and desk. Corkboard, small magnetic whiteboard, and plastic drawers for computer parts. Product backlog, sprint backlog, burndown chart, sprint progress board. Microwave, fridge, footlocker for non-perishable snacks and tea, and a second shower board whiteboard. Large calendar with which to plan sprints. So if the zombie apocalypse comes, I'll be able to survive for some time in my home office, and keep coding all the while. Nothing, dead or undead, shall keep me from fini...

Yesterday I met with my advisor to demo what I accomplished in sprint 2. Of course most of it was behind-the-scenes so there wasn't a lot to show, but he seemed pleased with my progress thus far. We discussed the security issues I've been researching and the insanity of trying to secure a web app in general, and he said security would be a good topic to discuss in the project summary paper thing I'll have to write. So I'm already getting a general idea of what will have to go in the paper, which is good. We also discussed what I hope to accomplish in sprint 3 and some plans for sprint 4. Right now there's a ton of steps involved in setting up the project for the first time (configuration files to edit and put various places, editing of tomcat's xml files, creating the database, etc. etc.) and so he'd like to simplify all that so that he can deploy my project on his own machine more easily. So we agreed that that will go in as an item for sprint 4. I'll c...

The goal for sprint 3 will be to make the project usable for my adviser and start making it more user friendly and start making the code more correct. There are several small tweaks I need to do so that everything will work correctly for my prof, so I need to do that, plus the FindBugs item from the last sprint. Code correctness and user-friendliness are very long-term goals, but this will be their beginning. The sprint backlog items are: Priority Description Points 300 Run automated tools e.g. FindBugs to ensure code is correct and clean 6 253 Write a shell script or similar to start the database server and tomcat 3 252 Fix the build.xml to only copy the correct jars to the web app's lib directory 2 251 Restrict the login page to only allow access via SSL 2 250 Fix the email notification system so that it works with the CS department email settings 4 200 Move properties out of the session so that users can do things like have multiple browser windows open and bookmark...

So I finished sprint 2 today. Yet again my biggest problem is finding enough time to put in as much work as I'd like. I suspect this will be an ongoing problem. I am mostly pleased with how much I accomplished though. I finished 3 backlog items: Fix fake data insertion code (1 hour), Add additional sanitization of input and other security measures (6 hours), and Add logging statements throughout the code (3). The only somewhat troubling aspect is that I'm still not 100% sure I've considered all the possible security issues. I've done a LOT of googling, but this information is extremely scattered. There's no comprehensive "Java web app security guide" that I've been able to find (though the OWASP stuff is very good and somewhat close). Mostly I've gone searching for one thing, and a random link or post will lead me to other things. It seems like you have to know what all the issues are in advance and then start searching for solutions to those iss...

As it turns out, security is actually harder than the internets originally led me to believe. Preventing SQL injection is as simple as making sure to use PreparedStatements (and not doing anything stupid like concatenating user supplied values with safe strings in the PreparedStatements), but preventing XSS is a whole 'nother kettle of fish. So I'm following the advice given on the linked site, and using their library to take care of escaping. I was using Apache Commons StringEscapeUtils for this but it only has one type of escaping for HTML, when apparently multiple different types are needed depending on where in the HTML the user-generated content is going. The Apache documentation doesn't really specify what it's doing, so I don't feel comfortable using it for either encoding inside regular HTML elements or inside attributes. I'm still using Java's own URLEncoder for encoding URL parameter values, as it seems to do the same thing as OWASP's UR...

I've been researching what's required to make my application less vulnerable to things like SQL injection or other malicious attacks. Imagine my surprise when I discovered that I've apparently already done the two biggest things, which are using PreparedStatements and escaping special characters . I've googled quite a bit and these are the main things I've found so far. According to the latter article I do need to do a few more small things, such as specifying the character encoding in my headers, but that seems to be about it... I'll be double-checking all my SQL to make sure there are no inappropriate uses of Statements, but apparently this issue is going to be a lot faster and easier than I expected. Rather shocking, really... Hopefully google hasn't lead me astray. In other news I'm leaning towards revising the issue "Add error checking, unit testing, and integration testing using JUnit, DBUnit, FindBugs, etc." to just focusing on Find...