cat slave diary

Category: olpc

My blog posts for the GaiaBB One Laptop Per Child project

  • Installing Fedora 18 (RTM) to VMWare Fusion 5 or VMWare Workstation 9

    I always live in hope that just one day, the folks over at Fedora will actually have a pain free VMWare installation. Not to be. Here’s how to do it with the minimal gnashing of teeth.

    Bugs that get you before anything else

    On VMWare Fusion 5, currently Fedora 18 x86_64 Live DVD’s graphical installer will boot and then gets stuck at a blue GUI screen if you have 3D acceleration turned on (which is the default if you choose Linux / Fedora 64 bit).

    • Virtual Machine -> Settings -> Display -> disable 3D acceleration.

    We’ll come back to this after the installation of VMWare Tools

    Installing Fedora 18 in VMWare Fusion / VMWare Workstation 8

    The installation is pretty straight forward … as long as you can see it.

    The only non-default choice I’d like you to change is to set your standard user up to be in the administrators group (it’s a checkbox during installation). Being in the administrators group allows sudo to run. If you don’t want to do this, drop sudo from the beginning of all of the commands below, and use “su -” to get a root shell instead. 

    The new graphical installer still has a few bugs:

    • Non-fatal – On the text error message screen (Control-Alt-F2) there’s an error message from grub2 (still!) about grub2 file not found /boot/grub2/locale/en.mo.gz. This will not prevent installation, so just ignore it for now (which the Fedora folks have for a couple of releases!). Go back to the live desktop screen by using Control-Alt-F1
    • PITA – Try not to move the installer window offscreen as it’s difficult to finish the installation if even a little off screen. If you get stuck, press tab until you hit the “Next” button – or just reboot and start again
    Update Fedora 18

    Once you have Fedora installed, login and open a terminal window (Activities -> type in “Terminal”)

    sudo yum update
sudo reboot
sudo yum install kernel-devel kernel-headers gcc make
sudo reboot

    Fix missing kernel headers

    At least for now, VMware Tools 9.2.2 build-893683 will moan about a path not found error for the kernel headers. Let’s go ahead and fix that for you:

    sudo cp /usr/include/linux/version.h /lib/modules/`uname -r`/build/include/linux/

    NB: The backtick (`) executes the command “uname -r” to make the above work no matter what your kernel version is.

    NB: Some highly ranked and well meaning instructions want you to install the x86_64 or PAE versions of kernel devel or kernel headers when trying to locate the correct header files. This is not necessary for the x86_64 kernel on Fedora 18, which I am assuming you’re using as nearly everything released by AMD or Intel for the last six years is 64 bit capable. Those instructions might be relevant to your interests if you are using the 32 bit i686 version or PAE version of Fedora 18.

    Mount VMWare Tools

    Make sure you have the latest updates installed in VMWare before proceeding!

    • Virtual Machine -> Install VMWare Tools

    Fedora 18 mounts removable media in a per-user specific location (/run/media/<username>/<volume name>), so you need to know your username and the volume name

    Build VMWare Tools

    Click on Activities, and type Terminal

    tar zxf /run/media/`whoami`/VMware\ Tools/VMw*.tar.gz
cd vmware-tools-distrib
sudo ./vmware-install.pl

    Make sure everything compiled okay, and if so, restart:

    sudo reboot

    NB: The backtick (`) executes the command “whoami” to make the above work no matter what your username is.

    No 3D Acceleration oh noes!1!! Install Cinnamon or Mate

    Now, all the normal VMWare Tools will work. Unfortunately, after all the faffing about, I didn’t manage working 3D acceleration. I ended up installing something a bit lighter than Gnome 3.6, which requires hardware 3D acceleration.

    • Activities -> Software -> Packages -> Cinnamon for a more modern desktop appearance or 
    • Activities -> Software -> Packages -> MATE for old school Gnome 2 desktop appearance
    • Apply 
    • Logout 
    • From the session pull down, change across to Cinnamon or Mate and log back in
    When VMWare updates support Tools to support Fedora 18 or vice versa, I’d still suggest Cinnamon over Gnome 3.6. Gnome 3.6 sucks way less than earlier Gnome 3.x releases, but that’s no great compliment. YMMV and you may really like Gnome 3.6, but without 3D support, it’s going to be painful. 

  • Fedora 17 install on VMWare Fusion 4 / Workstation 8

    I am moving over to using Fedora from Ubuntu as I am helping out with the OLPC XS (School Server) on XO laptop effort, which is Fedora based. Fedora 17, codename The Beefy Miracle (seriously), has just been released, so it’s time to update my Linux development workstations.

    Installing Fedora 17 in VMWare Fusion / VMWare Workstation 8

    There are two problems with Fedora 17 and VMWare at the moment:

    • Fedora 17’s graphical installer on VMWare (and I’ve heard on Oracle VirtualBox too) does not show the Back / Next buttons. They are there and they work if you tab to them, but they are offscreen due to the video mode.
    • The next problem is that “Linux Easy Install” doesn’t work in VMWare Tools build 8.8.3 in the Fedora 17 guest. At all. Don’t use it, or you’ll end up in dracut debug shell purgatory after a “dracut Warning: Unable to process initqueue” message. The rest of the instructions here gets you to where Easy install would have managed to get you anyway.

    So to get through it, you can either run a text install that produces a very minimal install of Fedora (great for hardened servers like the XS!)

    • Boot from the ISO
    • Enter the troubleshooting menu
    • Press tab to bring up the linuz boot arguments
    • Delete the vesa arguments
    • Type in “text” on the end of the command line

    or just use the graphical install…

    • If there’s only one control on screen, just press return and you can go to the next screen
    • If there’s many controls on screen, press tab until the focus disappears, and then press tab one more time (moves from the hidden Back button to the hidden Next button) and then press return.

    Updating Fedora to allow VMWare Tools compilation

    Once you have Fedora installed, login and open a terminal window

    sudo yum update
sudo reboot
sudo yum install kernel-devel kernel-headers gcc make

    Installing VMWare Tools

    Now you’re ready to install the VMWare Tools

    • Virtual Machine -> Install VMWare Tools. Unfortunately, F17 now mounts CD ROMs in a user specific location (/run/media/<username>/<volume name>), so you need to know your username if the instructions below don’t work
    • Open a terminal
    tar zxf /run/media/`whoami`/VMware\ Tools/VMw*.tar.gz
cd vmware-tools-distrib
sudo ./vmware-install.pl

    NB: With tools build 8.8.3, there will be compilation errors until an update for the tools is released by VMware, but enough modules will compile to allow you to use shared folders, have auto-resizing monitors, working cut-n-paste, and a few of the other things that make running the tools worthwhile. Drag and drop doesn’t work yet.

    Logout and then Restart the system to enable the tools. If you’re text only,

    sudo reboot

    will do the trick.

    You are welcome!

  • Upgrade to Ubuntu 10.04 LTS in VMWare Fusion – Keyboard issues

    I upgraded my VMware Fusion image to Ubuntu 10.04 LTS over the weekend, and everything went well except for the keyboard. It wouldn’t work.

    So here’s how I found out how to fix it:

    • Go to the Accessibility Preferences at the bottom of the screen, and tick on screen keyboard.
    • You have to reboot because for some unknown reason, it doesn’t start unless you do.
    • You can now type in your password on screen. Once logged in, your keyboard works
    • Go to terminal, and reconfigure your console:
    sudo dpkg-reconfigure console-setup
    • Reboot, and you’re done.

  • How to migrate to PDO without it hurting… much

    As we saw in the previous article, conversion to MySQLi is an awful lot of work. So let’s move to PDO.

    Step 0. Get PDO working with your database server

    Somewhere along the line, the PHP and MySQL folks decided to not be friends, so even though 99.99% of all PHP scripts require MySQL, in most cases, PDO doesn’t have MySQL support enabled.

    Check that PHP doesn’t already have PDO for MySQL ready to go:

    % php -i | grep -i pdo
Configure Command =>  '[...snip...]'
PDO
PDO support => enabled
PDO drivers => mysql, sqlite, sqlite2
pdo_mysql
PDO Driver for MySQL => enabled
pdo_mysql.cache_size => 2000 => 2000
pdo_mysql.default_socket => /tmp/mysql.sock => /tmp/mysql.sock
pdo_sqlite
PDO Driver for SQLite 3.x => enabled
    The default socket location is fine for a development workstation, but not so good for a production host. Change php.ini and my.cnf to make it safer, such as /var/run/mysql/mysql.sock
    If your PHP installation doesn’t show the above, whip out the compiler – there’s plenty of articles on the Interweb on how to get PDO and MySQL going. For now, let’s assume PDO and MySQL are good to go.
    Step 1. Conversion
    As in our previous article, it’s very tempting to just go through each file and make a 1:1 change from MySQL to PDO. That is actually a losing scenario.
    1. My non-trivial app has 1853 SQL queries littered through the code. At about 15 minutes to 30 minutes per query, with no test case, that’s about a year’s work with no change to the overall complexity of the app nor any improvements to the overall data architecture. Changing from one to the other is sure to introduce bugs.
    2. GaiaBB only has 13 tables. Using a DAO approach, with a list and search helper method for each (i.e forum list, search forum), that’s 6 * 13 = 78 properly written, tested DAO methods that need to be written from scratch. This is a saving of over 10 months work compared to just getting in there and getting my hands dirty.
    3. I can add security business requirements once to the DAO, such as fine grained access control, input validation, audit, and dirty output sanitization, thus fixing all my access control issues and dirty data issues in the same small piece of code. So each file that is converted to DAO gets more secure with no additional coding

    You’re probably wondering about “dirty output sanitization”. Sanitization is for the weak minded! No seriously, my database is 7 years old. There’s crud in there – I can’t trust it to be safe or good. There’s some nice tools out there like Arshan’s Scrubbr to scan and clean a database, but Scrubbr is not going to be able to decode BBCode and check to see if there’s a nasty [color] based XSS or CSRF attack in there. Additionally, some versions of my software and some buggy versions of MySQL == binary blob crappiness. Blobs coming out sometimes (but not always) need to be stripslashed(). Additionally, some versions of XMB used client side checks to prevent files of the wrong type to be uploaded. I’ve found a JavaScript malicious file in my production database. So it’s worth adding checks so that I don’t serve bad things to folks. You can’t do this if you have 100+ attachment related SELECT statements alone, not without lots of bugs and lots of code. Going the DAO way, means I can do it once, for every single use of the attachment sub-system.

    There’s gotta be a downside.

    It’s not all sunshine and roses. You should not open both a PDO and old style connection to the database for busy servers like mine – it more than doubles the time to serve the average script, the poor little database server gets whacked, and performance will drop.

    In GaiaBB, there’s a significant 7240 lines of shared code read in every single time a script does anything – header.php, functions.php, db.php, validation.php, constants.php, config.php, cache.php, english.lang.php, mail.class.php. So to only open ONE database connection requires these files to be ported to PDO first. But if I convert these files, every remaining single file totalling about 80,000 lines of PHP will also need to be converted.

    So my solution for GaiaBB is to create a side-by-side approach, using kernel.php for converted files instead of header.php. kernel.php includes PDO ready files rather than the old files. This temporarily makes GaiaBB approach the 90,000 line mark, but in the long run, it will allow me to make many of the scripts smaller and more object orientated. Once converted to DAO, I can simply eliminate many security checks within the code of each page, as the DAO will simply throw an exception if you’re not privileged to do something with the data you’re trying to work with.

    So my main piece of advice for you contemplating converting to PDO is to consider your data architecture. The old MySQL way is awful, buggy and insecure. Let’s exterminate it, but instead of standing still for months at a time, add in freebies like access control, audit, input validation and output sanity checking.

  • The OLPCs are here

    Wow –  that was quick.

    I have three OLPCs in my office. I need to go sort out some US power plugs for them so I can charge them, but they’re here! 🙂

  • Proto GaiaBB runs fine in Browse

    Phew.

    Screen shot 2009-10-21 at 11.33.00 PM

    Obviously, the default theme fails a bit when you see only two rows of topic activity. Will really need to make that work a whole lot better. However, the actual rendering is perfect. Yay! One less task to be done.

  • GaiaBB and OLPC

    Peter Quodling. an old friend, e-mailed out of the blue last week. I have a lot of time for Peter as he’s one of the few Australian IT architects that really knows his stuff, plus he’s a really nice guy. He is involved in OLPC in the PNG region. Last Christmas, I nearly bought an XO under the Buy One, Give One program so Mackenzie could have a cool first laptop … and somewhat more honestly, so I could play with the OLPC until she’s old enough to type let alone talk. However, circumstances prevented toy purchases of that magnitude, and I forgot all about the XO until this week.

    I thought through my various abandoned projects (for I have many), trying to work out which would help the OLPC project and kids all over the world. I’ve had an itch for a while to do something for the One Laptop Per Child project (OLPC), but never really had a substantial idea that would help transform kids’ lives rather than my own. But now I think I have just the project.

    So I put in a proposal for two laptops to help develop GaiaBB (which is UltimaBB++ (sic gloria transit), which is XMB++, which is awful) into a OLPC specific product.

    My current plans are:

    • Get GaiaBB.com back up and make it OLPC centric. Revitalize the Sourceforge project.
    • Finish OWASP ESAPI for PHP. I need it for this project.
    • Port GaiaBB to the OLPC, porting the database to sql lite, and probably using LightHTTPd. I could use Apache + MySQL as per now, but these are huge compared to SQL Lite and LightHTTPd, and on a device with limited NAND memory, every byte counts.
    • Ensure GaiaBB works properly with Browse, the XO browser. I might need to turn the nested tables into CSS templates a bit sooner than I intended
    • Beg, borrow or steal a graphic designer to come up with a GaiaBB theme that works well with the dual color display (it’s both black and white and color and more wide than tall), and possibly work out how to detect the XO’s current screen state from the web page so I can dynamically choose a grey scale or a color theme.
    • Simplify the product so that it’s more manageable by young ‘uns without dumbing down too much or removing some of the depth and surprise features of the product
    • Write an installer that makes it easy to install on the OLPC. I want kids to be able to create their own social communities and for them to easily share their forums with their friends.
    • And this is where it gets fancy… write a web service that allows authorized folks to replicate their version of the forum with other versions of the forum … without causing major security issues. That way when you’re at home and have no Internet service, you can still read the forum and write new posts and then sync when you’re at school. This is where I will have to significantly change the way GaiaBB works as right now, it’s a single database deal and assumes that there are trusted administrators.
    • Go through the code with a fine tooth comb, replacing all the crappy security bits with ESAPI for PHP. Some parts are truly ancient (circa 2000) and need refactoring. As part of this, I will ensure the code is easily modifiable. I learnt how to code by changing other folks’ code and then starting to write my own, and I want kids to modify the heck out of this forum so as to create a new generation of coders excited about programming and IT in general.
    • Lastly, possibly write a OLPC School Server centralized GaiaBB hub for schools to run “their” forum which students can sync with in a safe fashion.

    The proposal has been approved, and the laptops are on their way. They are sending me three XO’s! Awesome. Better get cracking!