Learning about BuildBot
Introduction
I'm looking at using BuildBot for regression testing for Arduino. BuildBot is my current choice due to being Python-based. This page documents the process.
Project Log
( 11 April 2009 )
- Realised I could start testing out the system on a virtual machine rather than waiting to get access to a internet facing server. Hopefully I won't have to re-do too much work in order to deploy.
- Started working through the BuildBot installation notes.
- Using an Ubuntu 8.04 desktop install.
- Installed python-twisted 2.5.0-2build2 and dependencies.
- Discovered buildbot 0.7.6-1 package is available from some repository I've already got set up on the system, so will start with that. (Even though it's an older version.)
- After installation buildbot --version returns:
Buildbot version: 0.7.6 Twisted version: 2.5.0
- Noted that buildbot can be install in a standard user's home directory.
- Executed the test suite with:
PYTHONPATH=. trial buildbot.test
- Got one error ( buildbot.test.test_vc.Patch.testPatch ) and one failure ( buildbot.test.test_scheduler.Scheduling.testNightly ). The latter is apparently a known daylight saving related bug. Also, one exception ignored.
- Installed subversion after getting annoyed that it doesn't have svn in the package name.
- Re-ran the test suite after install subversion and got a huge barf. Suspect the previous error and some of this one were related to a lack of patch on the system so installed that and re-ran.
- Installing patch solved all the barfage except the testNightly failure and the ignored exception.
- Created buildmaster user account.
- Created directory layout as suggested:
mkdir -p ~buildmaster/masters/arduino buildbot create-master ~buildmaster/masters/arduino
- The buildbot checkconfig command does not exist in 0.7.6
- Started modifying the configuration:
cp masters/arduino/master.cfg.sample masters/arduino/master.cfg
# Modify: c['projectName'] = "Arduino" c['projectURL'] = "http://arduino.cc/"
- @@ TODO: Look into using this or similar eventually but initialy just use PBChange source:
svnpoller.SVNPoller (polling the SVN repository)
- Downloaded Arduino source from SVN repository:
svn checkout http://arduino.googlecode.com/svn/trunk/ arduino
- Note that arduino/build/macosx/dist and the corresponding Windows directory contain large (up to ~63MB) binary files so if you're not building on a Mac or Windows you probably want to avoid downloading the directories.
- Install sun-java5-jdk , gcc-avr and avr-libc packages
- Simply executing the following works, even without further configuration as specified at the top of the script (and apparently the errors can be ignored):
./make.sh # then... /run.sh
( 14 April 2009 )
- Modified master configuration file:
c['schedulers'].append(Scheduler(name="all", branch=None,
treeStableTimer=2*60,
builderNames=["arduino-checkout"]))
- and
# CVS, SVN, and others.
from buildbot.process import factory
from buildbot.steps.source import SVN
from buildbot.steps.shell import Compile
from buildbot.steps.python_twisted import Trial
f1 = factory.BuildFactory()
f1.addStep(SVN(svnurl='http://arduino.googlecode.com/svn/trunk/app/',
mode="copy"))
#f1.addStep(Compile(command=["python", "./setup.py", "build"]))
#f1.addStep(Trial(testpath="."))
b1 = {'name': "arduino-checkout",
'slavename': "bot1name",
'builddir': "full",
'factory': f1,
}
c['builders'] = [b1]
- Then run this command:
buildbot start masters/arduino/
- If there is an error in the configuration file then note that the buildbot will still stay running, so you either need to do this:
buildbot stop masters/arduino/ buildbot start masters/arduino/
- or (presumably):
buildbot reconfig masters/arduino/
- It should then display something along the lines of:
The buildmaster appears to have (re)started correctly.
- Visiting http://localhost:8010 in a web browser on the machine should now show something like:
( 5 May 2009 )
- Installed Xvfb to try for initial test methodology
- Run Xvfb with (but note -ac is probably a security issue, possible solution with -auth):
export DISPLAY=:1; Xvfb -ac :1 &
- Then run arduino (should use run.sh ?):
cd ~/Code/arduino/build/linux/work ./arduino
- Verify display with:
xwd -display :1 -root | xwud
- The following error message from xwd means you probably ran Xvfb without specifying -ac or -auth:
No protocol specified xwd: unable to open display ':1'
- Install xautomation in order to send events to Xvfb in order to simulate keyboard/mouse.
- Successfully compiled a sketch with:
xte -x :1 'keydown Control_L' 'key r' 'keyup Control_L'
- Verified by messages in console and scaled screen:
xwd -display :1 -root | xwud -scale
- Quit using:
xte -x :1 'keydown Control_L' 'key q' 'keyup Control_L'
- Verified size of output:
$ avr-size /tmp/build8282.tmp/Blink.hex
text data bss dec hex filename
0 982 0 982 3d6 /tmp/build8282.tmp/Blink.hex
( 14 May 2009 )
- Attempt to install non-root Xvfb:
mkdir -p altroot/var/lib/dpkg/ mkdir -p altroot/var/cache/apt/archives/partial #apt-get -o=Debug::NoLocking=true -o=RootDir=altroot -s install xvfb ##apt-get -o=Debug::NoLocking=true -o=Dir::Cache=/<homedir>/altroot/var/cache/apt/ install xvfb ## doesn't yet work due to different xvfb & libpng versions.
( 18 May 2009 )
- Have given up on trying to install non-root Xvfb on server it seems installation requires chroot or something and won't let you install anything non-root even if you set-up non-root directories. Maybe revisit with source compilation but that sounds like a pita.
- Started to setup buildslave in VM.
- Created buildslave user account, unprivileged.
- Create directory in buildslave user account.
mkdir -p ~/Buildslave/arduino
- Configure buildslave:
buildbot create-slave ~/Buildslave/arduino/ localhost:9989 bot1name <password>
- Edit info files:
# Buildslave/arduino/info/admin # Buildslave/arduino/info/host
- Start the slave:
buildbot start Buildslave/arduino/
- It seems like the "force build" button mentioned in the docs isn't in the version installed.
- Finally got a build slave to complete a successful (partial) SVN checkout. Ran into a number of issues along the way—using the most recent buildbot version might have helped.
- I couldn't work out how to use the buildbot sendchange command correctly to force a rebuild. AFAICT this doesn't do anything:
#buildbot sendchange --master localhost:9989 --username change
- Thus, I decided to use debugclient instead, first you need to edit the master.cfg file with:
c['debugPassword'] = "<password>"
- Then you can run the application (as the buildmaster user) with:
buildbot debugclient --master localhost:9989 --passwd <password>
- If you get a segmentation fault on start-up it may be because you're trying to run the (graphical) app from a non-X terminal (e.g. via su possibly) which doesn't seem to be handled gracefully. I logged into the buildmaster user account and ran it from there.
- Don't forget to press the Connect button first and Reload .cfg if you need to do so.
- It wasn't clear to me what was supposed to entered in the Builder section, at first I thought it was supposed to be the buildslave name I wanted to force to build. But when I tried that I got the unhelpful error in the terminal of:
received error
- Further poking around led to this change set which suggests a patch to buildbot/clients/debug.py of:
Index: buildbot/clients/debug.py
===================================================================
--- a/buildbot/clients/debug.py
+++ b/buildbot/clients/debug.py
@@ -166,7 +166,5 @@
d.addErrback(self.err)
def err(self, failure):
- print "received error"
- failure.printTraceback()
-
+ print "received error:", failure
def run(self):
- The above patch (or similar) gave a slightly more useful error message which lead me to realise I was supposed to supply the builder name, not a specific bot name. Thus, once I supplied arduino-checkout as the value it no longer gave the error.
( 19 May 2009 )
- The build steps to produce a Linux arduino build with the latest version from SVN:
from buildbot.process import factory from buildbot.steps.source import SVN from buildbot.steps.shell import Compile, ShellCommand from buildbot.steps.python_twisted import Trial f1 = factory.BuildFactory() f1.addStep(SVN(svnurl='http://arduino.googlecode.com/svn/trunk/', mode="copy")) f1.addStep(ShellCommand(command=["/bin/sh","-c","./make.sh"], workdir="build/build/linux/"))
- Note that this line is because each step starts afresh so cd won't stick and (I guess) there's no auto execution of the shell file correctly:
f1.addStep(ShellCommand(command=["/bin/sh","-c","./make.sh"], workdir="build/build/linux/"))
( 23 May 2009 )
- The other day I sent a suggested patch to the Arduino developers list:
Namely, changing the file 'build/linux/dist/arduino' slightly to
enable a sketch file path to be supplied on the command line and
opened automatically. The functionality should be unchanged if no
sketch file path is supplied.
--- arduino~ 2009-04-12 01:09:11.000000000 +1200
+++ arduino 2009-05-20 00:46:31.000000000 +1200
@@ -13,4 +13,4 @@
LD_LIBRARY_PATH=`pwd`/lib:${LD_LIBRARY_PATH}
export LD_LIBRARY_PATH
-java processing.app.Base
+java processing.app.Base $1
A patch to 'run.sh' would also be required if this functionality was
to be available to it--but that patch is a little more complex due to
the file path manipulation required (if a relative path is supplied)
by the change of the directory path.
- Applying the above patch is necessary to get following screentest.sh first cut at a test script working:
#!/bin/sh # # screentest <path to arduino dir> <sketch path relative to arduino dir> # export DISPLAY=:1 # TODO fix -ac use Xvfb -ac :1 & XVFB_PID=$! sleep 5 cd $1 ./arduino $2 & ARDUINO_PID=$! sleep 5 xte -x :1 'keydown Control_L' 'key u' 'keyup Control_L' sleep 15 avr-size `dirname $2`/applet/*.hex xte -x :1 'keydown Control_L' 'key q' 'keyup Control_L' sleep 5 kill $ARDUINO_PID kill $XVFB_PID