I have a bit of a code review job at the moment. It’s a large code base, and you all know what that means. LOTS OF RAM! So I got me a 16 GB upgrade. Then I found that I could only allocate 8 GB to a VM in VMWare Fusion. So here’s how to scan a big chunk of code with minimal pain:
The default VM disk size for a Easy Installed Ubuntu is 20 GB, with 8 GB of swap. WTF. So don’t use Easy Install as you’ll run out of disk space doing a scan of a moderate sized application. I expanded mine to 80 GB after it was all installed, but if you are smart, unlike me, do it when you first build the system.
To add more than 8GB to a VM in VMWare Fusion, allocate 8192 MB (the maximum amount) in the GUI whilst the VM is shutdown, open the package contents of the VM by right clicking the VM (I’m on a Mac, so if you rename a folder foobar.vmwarevm, it becomes a package automagically). Find the VMX file. Open it carefully in a decent editor (vi or TextWrangler or TextMate) – there is magic here and if you edit it wrong, your VM will not boot. Change memsize = “8192” to say memsize = “12384” and save it out. I wouldn’t go too close to your total memory size as you’ll start paging on the Mac, and that’s just pain. Boot the VM. Confirm you have enough memory!
First off, do not even try to do it within Audit Workbench. It will just fail.
Secondly, it seems that HP do not test the latest version of SCA on OpenSuse 12.2, which is a shame as I really liked OpenSuse. There’s no way to fix up the dependencies without using an unsafe (older) version of Java, so I gave it up.
Ubuntu, despite not being listed as a valid platform (CentOS, Red Hat, and OpenSuse are all listed as qualified), Ubuntu had a graphical installer compared to OpenSuse’s text only install. Alrighty, then.
Install Oracle Java 1.7 latest using the 64 bit JDK for Linux. I did it to /usr/local/java/ Weep for you now have a massive security hole installed.
Force Ubuntu to use that JVM using update alternatives:
sudo update-alternatives --install "/usr/bin/java" "java" "/usr/local/java/jdk1.7.0_15/bin/java" 1
sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/local/java/jdk1.7.0_15/bin/javac" 1
sudo update-alternatives --set java /usr/local/java/jdk1.7.0_15/bin/java
sudo update-alternatives --set javac /usr/local/java/jdk1.7.0_15/bin/javac
I created the following in /etc/profile.d/java.sh
#!/bin/sh
JAVA_HOME=/usr/local/java/jdk1.7.0_15
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin
export JAVA_HOME
export PATH
Note that I did not tell Ubuntu about Java Web Start. If you want to keep your Ubuntu box yours, you will not let JWS anywhere near a browser. If you did this step, it’s best to delete javaws completely from your system to avoid any potential for drive by download trojans.
Install SCA as per HP’s instructions.
Now, you need to go hacking as HP for some reason still insist that 32 bit JVMs are somehow adequate. Not surprisingly, Audit Workbench pops up an exception as soon as you start it if you take no further action to make it work. So let’s fix that up.
I went and hacked JAVA_CMD in /opt/HP_Fortify/HP_Fortify_SCA_and_Apps_3.80/Core/private-bin/awb/productlaunch to be the following instead of the JRE provided by HP:
JAVA_CMD="/usr/local/java/jdk1.7.0_15/bin/java"
After that, Audit Workbench will run.
Now, let’s work on ScanWizard. ScanWizard the only way really to produce repeatable scans that work without running out of memory. So run a ScanWizard. It’ll create a shell file for you to edit. You need to make the following changes:
MEMORY="-Xmx6000M -Xms1200M -Xss96M "
LAUNCHERSWITCHES="-64 "
There’s a space after -64. Without that it fails.
Then there’s bugs in the generated scan script that mean it would never work when using a 64 bit scan. It’s almost like HP never tested 64 bit scans on large code bases (> 4 GB to complete a scan). I struggle to believe that, especially as their on demand service is almost certainly using something very akin to this setup.
Change this bit of the scan shell script:
FILENUMBER=`$SOURCEANALYZER -b $BUILDID -show-files | wc -l`
if [ ! -f $OLDFILENUMBER ]; then
echo It appears to be the first time running this script, setting $OLDFILENUMBER to $FILENUMBER
echo $FILENUMBER > $OLDFILENUMBER
else
OLDFILENO=`cat $OLDFILENUMBER`
DIFF=`expr $OLDFILENO "*" $FILENOMAXDIFF`
DIFF=`expr $DIFF / 100`
MAX=`expr $OLDFILENO + $DIFF`
MIN=`expr $OLDFILENO - $DIFF`
if [ $FILENUMBER -lt $MIN ] ; then SHOWWARNING=true; fi
if [ $FILENUMBER -gt $MAX ] ; then SHOWWARNING=true; fi
if [ $SHOWWARNING == true ] ; then
To this:
FILENUMBER=`$SOURCEANALYZER $MEMORY $LAUNCHERSWITCHES -b $BUILDID -show-files | wc -l`
if [ ! -f $OLDFILENUMBER ]; then
echo It appears to be the first time running this script, setting $OLDFILENUMBER to $FILENUMBER
echo $FILENUMBER > $OLDFILENUMBER
else
OLDFILENO=`cat $OLDFILENUMBER`
DIFF=`expr $OLDFILENO "*" $FILENOMAXDIFF`
DIFF=`expr $DIFF / 100`
MAX=`expr $OLDFILENO + $DIFF`
MIN=`expr $OLDFILENO - $DIFF`
SHOWWARNING=false
if [ $FILENUMBER -lt $MIN ] ; then SHOWWARNING=true; fi
if [ $FILENUMBER -gt $MAX ] ; then SHOWWARNING=true; fi
if [ $SHOWWARNING = true ] ; then
Yes, there’s an uninitialized variable AND a syntax error in a few lines of code. Quality. Two equals signs (==) are not valid sh/bash/dash syntax, so obviously that was well tested before release! Change it to = or -eq and you should be golden.
After that, just keep an eye out for out of memory errors and any times you notice it saying “Java command not found”. To open a large FPR file may require bumping up Audit Workbench’s memory. I had to with a 141 MB FPR file. YMMV.
You’re welcome.
