A long, long time ago, I took on running Aussieveedubbers, a forum based around the love of Volkswagens. We were on EzBoard, where the adverts and performance sucked so bad, that free was no longer acceptable. Over many iterations, I now run UltimaBB, a derivative of XMB. I had various titles – including lead programmer – at both projects for a while, but my main claim to fame was to get XMB 1.9.1 out the door with Tularis, the lead dev at the time, and the primary early force behind UltimaBB along with John Briggs. John implemented the hacks, and I did the architecture changes required to make them suck a lot less and hopefully make them more secure. In a way, I have – UltimaBB does not suffer from ANY of the XMB Bugtraq failures. Therefore, I am a coding god and UltimaBB is secure, right?
I got going in forum development primarily through security fixes. By the time the UltimaBB effort died and XMB imploded in 2008, I thought I had gotten most of the security bugs out of the code. UltimaBB was a shining beacon of the bazaar approach. Or so I thought.
But like most in open source, there’s a huge difference between what I think is secure, and what is actually secure. Just because I think it is secure doesn’t make it so. I wish more folks in open source projects would get this very simple message. You suck, and as long as you know that, everything (in time) will be okay.
So, like all good scientists, I do not let my good opinion of myself get in the way of a solid slap session. I gave UltimaBB a thorough code review using OWASP’s forthcoming Application Security Verification Standard at Level 2B, which encompasses a manual code review for 83 controls with an optional use of automated static code review tools. I used Fortify’s awesome Secure Code Analyzer, which is almost certainly the best in the business today. It didn’t find everything, but it highlighted a number of deficiencies I already knew about quite reliably.
I can’t claim I did this review because I suddenly had a huge chunk of time on my hands, but because I am implementing a new service at Pure Hacking, and we’re using ASVS as our verification baseline. I’ve added a few more things in as I think they’re important, but the main thing is that I’ve now validated that the ASVS produces a reasonable outcome for an average program, one that should have been reasonably secure – because *I* wrote a great deal of it, and certainly the security bits.
The good news is that ASVS can be used as a security consultancy’s baseline for a partially automated, but primarily manual code secure code review. I think with a bit of tweaking, we can develop automated searches in most languages for about 50-60% of its requirements, which will dramatically improve the quality and reliability of secure code reviews.
The bad news is that my forum sucks. I have a bunch of extreme, high, medium and low risks to fix. Until I’ve fixed them (as there’s precisely one forum running this code out there – mine!), I can’t go into them too much, but let’s just say that I was disappointed with myself, and I’ve taken myself out the back and slapped me silly.
Luckily, I have a good client for this review (me), who will actually fix all the defects and lacks found so far.