For a last month or so I used Barnes & Noble Nook Color as Android tablet. This post will try to explain what is wrong with Android tablets and now to fix this.

For a start, original Android distribution on Nook doesn't make it really Android tablet (no market). But there is easy solution since this device is supported by CynanogenMod 7 for Nook Color to get proper Android distribution. So, let's get started with hints for easier life with your new tablet...

adb doesn't see Nook Color

For a start, you won't see Nook in adb devices listing. This is easily fixed with proper device id in ~/.android/adb_usb.ini like this:

dpavlin@t61p:~/.android$ echo 0x2080 > ~/.android/adb_usb.ini && adb kill-server && adb devices
* daemon not running. starting it now on port 5037 *
* daemon started successfully *
List of devices attached 
2011120012345678        device

Nook Color specific application

There are lot of application which will look quite broken on your Nook when running Android 2.3. Some of them require audio input, GPS or camera and they generally won't work. There is also problem with application which just don't know about wired 1024*600 screen size and produce strange small screen centered on big display.

However, this is just a list of few extremely useful apps which you should install on Nook with CM7:

• Nook Color Tweaks - change speed of CPU, turn USB host more and more
• Nook Screen Recalibrate - having problems with screen sensitivity?
• Nook Key Editor allows you to map back and menu keys which are very useful on Android 2.3 to volume up/down which makes tablet much more useful.

Ice Cream Sandwich

To get most out of Android on tablet, you have to hop over to CM9/ICS Nightly Builds and install unstable builds on ICS on Nook Color. After initial install reboot into recovery, flash Google Applications from gapps-ics-4.0.3-sam-noinit.zip, turn off signature verification and install fix-bootanimation.zip and telephony-permission-fix.zip. Unfortunately browser application doesn't work for me well, and usually segfaults with:

F/libc    ( 2878): Fatal signal 11 (SIGSEGV) at 0x0000001f (code=1)
I/DEBUG   ( 1072): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
I/DEBUG   ( 1072): Build fingerprint: 'bn/bn_encore/encore:2.3.4/IML74K/228551:user/release-keys'
I/DEBUG   ( 1072): pid: 2878, tid: 2895  >>> com.android.browser <<<

ICS on Nook currently doesn't have hardware acceleration (but older Androids also doesn't have it) and it freezes from time to time (so you will get quite used to holding power button for 8 seconds or more to turn it off), but ICS is really so much better experience on tablet that I would really recommend it.

Finally a note about hardware performance: Nook Color has 512Mb of RAM and Ti OMAP 3621 @ 800 MHz which makes it probably a bit slow for ICS. As o consequence I wouldn't really recommend any Android laptops which are slower than this for ICS.

Each year, we get a treat right before end of the year in form of 28C3. This year, I can recommend following lectures as a must-see:

• r0ket++ video
  Very nice ARM Cortex M3 embedded device with Nokia LCD, battery, RF interface and extension headers.
• 802.11 Packets in Packets video
  Drop wifi packet on unsuspecting listeners without access to network segment!
• Defending mobile phones video
  Visit GSMmap.org to see if your GSM networks are protecting you or contribute data!
• Reverse Engineering USB Devices video
  Very similar to my approach to USB examinations.
• How governments have tried to block Tor video
  It's arms race all over again, this time on Internet.
• Reverse-engineering a Qualcomm baseband video
  I can't wait to see release of this tools on code.google.com Available at http://code.google.com/p/qcombbdbg/
• KinectFusion video
  I would love to see something similar in open source code.
• Cellular protocol stacks for Internet video
  Very good overview of cellular stack as it is today
• Implementation of MITM Attack on HDCP-Secured Links video
  NeTv is very interesting ARM/FPGA/Linux device which overlays webkit content over HDMI signal on your TV using chroma key.
• Introducing Osmo-GMR video
  Osmocom got some support for satellite phones
• bup: Git for backups video
  Nice backup system with de-duplication implemented on top of git's pack file

We had another gathering called Nothing will happen in Moravice, and this time we wanted to be prepared for it. So we got 2 WRT and 2 TP-Link devices with good faith that we will have stable Internet there. However, our telecom provider decided to screw up our order to increase bandwidth from 4Mbit/s to 10Mbit/s for event and instead decided to downgrade our access to 512Kbit/s. So we opted to create following simple network architecture which involved multiple hops to alternative 4Mbit/s upstream and batman-adv mesh setup.

batman-adv is quite cool layer-2 mash network which operates in ad-hoc mode and allows adaptive routing over mash and multiple upstream providers (at DHCP request or renew time, so we made our DHCP lease time down to 5 minutes). Joining network is quite simple:

iwconfig wlan0 mode ad-hoc essid nsnd-batman
ifconfig wlan0 mtu 1528
modprobe batman_adv
batctl if add wlan0

ifconfig wlan0 up
ifconfig bat0 up
dhclient bat0

My own experience with this experiment is very positive, mostly because we had around 5 clients in mash at any time, compared with overloaded WRTs which handled rest of 40 wifi enabled devices. But, I hope that during next event more people will upgrade to 2.6.33 or newer kernels, so we can have even better mash connectivity.

As you might know by now, I'm very happy user of Kindle 3 (keyboard) and Kindle DX graphite. One of reasons why I choose those devices was community around Kindle Developer's Corner at mobileread forum. With new generation of Amazon's Kindle device on the way, let me stress that for me older Kindle 3 devices are more interesting since we still don't have the way to run home-brew software on newer Kindle 4.

One of the good things about next generation of Kindles is that older models get sold second-hand at reasonable price, and let me re-iterate: for around 100€ older Kindle 3 with keyboard are great devices. But what would you do with one when you get your hands on? Start a k3libre project to develop and document free libre command-line tools for Kindle development. It's very much work in progress, with useful example scripts which describe framebuffer format, but also a handy step-by-step instruction on how to liberate your Kindle by installing jailbreak and usbnetwork to get root ssh access over usb cable. Next step is launchpad which listens on /dev/input/input? and allows you to bind execution of programs on key-presses. We are doing all this to install to nice full-screen Kindle terminal. You can do most of this work in under an hour, so there is no excuse not to read man pages on your Kindle!

So now that you can run your own software on Kindle, with which would you like to start? I would suggest kindlepdfviewer - a PDF viewer made for e-ink framebuffer devices, using muPDF. It's implemented in lua, so adding new features are rather easy, and latest development include serialization of state into SQLite database and drawing characters on screen! OK, it's in early stage of developments, but already useful on real Kindle, and if you don't like something you can always just edit lua code a bit :-)

Sometimes, you need to connect two networks in some way. My usual motivation is ability to access machines behind multiple NATs for easy system administration. So far, I used combination of OpenVPN and DynamicForward in ssh with clever use of ProxyCommand and nc with a sprinkle of proxy.pac for Firefox to make everything seemingly work. However, I never successfully managed to tunnel various JavaWebStart based remote consoles which want to connect directly from your machine to remote IP using sockets (for which you have to disable all proxy settings using jcontrol and selecting direct connection).

So this got me thinking. I could configure another OpenVPN for this, but it has many steps and I was lazy. Wouldn't it be great if there is some kind of P2P network like Skype or Hamachi for Linux? Something like this:

n2n: a Layer Two Peer-to-Peer VPN is exactly what I was looking for. It allows you to construct IP network over nodes behind NAT. But is it really easier to configure for specific example of accessing private network on another LAN behind NAT? Let's find out.

Steps are simple:

• Install n2n (you will have to do this on supernode and two nodes)

  all$ sudo apt-get install n2n
  

• Start super node on public address with DNS name super.example.com

  internet$ supernode -l 1234
  

• Start first client

  local# edge -c community -d community  -k secret \
    -l super.example.com:1234 -a 10.1.2.1
  

• Start remote end-point somewhere within LAN

  remote# edge -c community -d community  -k secret \
    -l super.example.com:1234 -a 10.1.2.2 -r
  

  Note changed IP address and -r flag which will allow us to route over this node.

  remote# sysctl -w net.ipv4.ip_forward=1
  remote# iptables -t nat -A POSTROUTING -s 10.1.2.1 -o tun0 -j MASQUERADE
  

  This will turn forwarding and NAT for our packets coming from community tap interface and going out through tun0 to LAN. We also need to setup route on local side for remote LAN network:

  local# ip route add 172.18.0.0/16 via 10.1.2.2
  

Last few weeks, I was configuring huge ZFS pool of 50 disks over three machines. Aside from benchmarking, I wanted to setup monitoring of this disk pool. smartctl comes as natural candidate for getting smart data, but where should I keep it? I recently learned of git log -p output format which shows nicely changes in your source files, so natural question was can I use git to track smart disk statistics?

As it turns out, getting overview of disk layout is really easy under Linux if you know where to look. /proc/partitions first comes to mind, but it lacks one really important peace of information: disk serial number. It's only peace of information which won't change between reboots when you have to spin up 30+ disks, so you really want to use it as identification for disks, instead of device name for example (which I tried on first try and learned that disks move around).

Good naming of dump files is as important as always. In the end, I opted to use smart.id where id part is from /dev/disk/by-id/scsi-something. Paths in /dev/disk/by-id/ are essential useful when creating storage pools because they also don't change between reboots.

Now that we know where to look for disk identification and serial number, we are ready to start collecting smart data. However, this data is much more useful if coupled with info from controllers, so final version of smart-dump.sh script also supports dumping of controller status for LSI Logic / Symbios Logic and 3ware controllers. Have in mind that collecting smart info from disks does interrupt data transfers, so if you have huge pool you might want to spread those requests (or even issue them in parallel if you want one huge interruption as opposed to several smaller ones).

So was all this worth an effort? In fact, it was! In our sample of 50 3T disks, one disk reported errors after just 192 hours of lifetime. It would probably report it earlier, but this was second time that I run smartctl -t long on it. On the other side, it passed long check on first test which was 8 hours of LifeTime. Even if you read Failure Trends in a Large Disk Drive Population paper from Google, and concluded that smart is lying to you and you could ignore it, please monitor your drives!

I have been using ZFS on Linux for some time to provide backup appliance using zfs-fuse. Since then, we got native ZFS implementation on Linux, so I decided to move by backup pool from zfs-fuse to in-kernel ZFS.

Additional reason to move pool over to new machine was to change pool's RAID level. In current ZFS implementation(s) you can't change mirror to RAIDZ1 without re-creating pool and then transfering data over using zfs send and zfs receive. However, when you are creating snapshots for years, and expiring them using script you will have hundreds of snapshots which you need to transfer.

This is where zfs-pool-replicate.pl script comes handy. It uses Net::OpenSSH to connect to two machines (source and destination), list all snapshots on source and transfer them to destination. If you have filesystem without snapshots it will create one @send snapshot which will be transferred. It will also optionally use compression for transfer of snapshot over the network. I am using LZO which is fast compression which nicely transfers 150Mb/s or more over normal 1Gbit/s network without much CPU overheard (and we all have multi-core machines anyway, right?). Current implementation allows you to re-run replication script to transfer only new snapshots creating handy disaster recovery solution.

Current implementation is designed to run from third (management) machine, so I can envision central storage administration tool which will also allow you to transfer LVM snapshots into ZFS snapshots. For now, I'm using shell script for that, but rewriting it in perl would improve error recovery and reporting.

By default, MySQL installation on Debian comes without innodb_file_per_table option which spread tables in individual InnoDB files. Based on your usage patterns or backup strategies this might be better filesystem organization than one big /var/lib/mysql/ibdata1 file. I first heard about it in OurSQL Episode 36: It's Not Our (De)fault!. It's great podcast, but to be honest with each new episode I wish to have only PostgreSQL servers to maintain...

To enable this option you will need to create configuration file and restart MySQL server:

koha:/etc/mysql/conf.d# cat > file-per-table.cnf 
[mysqld]
innodb_file_per_table
CTRL+D
koha:/etc/mysql/conf.d# /etc/init.d/mysql restart

This won't change anything, because only new tables will be created in separate files. But, we can use ALTER TABLE table ENGINE=InnoDB on each table to force InnoDB to re-read tables and create separate files:

mysqlshow koha --status | grep InnoDB | cut -d'|' -f2 | sed -e 's/^/alter table/' -e 's/$/ engine=InnoDB;/' | mysql -v koha

If you replace grep InnoDB with grep MyISAM you might use same snippet to convert MyISAM tables into InnoDB (if you still have them or don't use fulltext search).

I think that system administration is like gardening. I don't know anything about gardening, but it seems to involve a lot of care here and there, seemingly without much pattern. In that sense, it's similar to wiki editing, you start somewhere and you really don't know where it lead you to.

You have to start reading by singing lady Ga-Ga with words: S, s, s, ss... SAML, SMAL2! It will help, really.

SAML 2 is latest in long line of different SSO implementation you will have to do sooner or later if you want to be part of larger web. Google and others seems to be using it, so it must be good, right?

It has two end-points: identity provider (IdP) which has user accounts and Service Provider (SP) which is usually your application. But of course, it's more complicated than that. For a start, you will need https on your host. I will assume that you already have domain, and you can get free SSL certificates at StartSSL so hop over there if you need one.

First, install SimpleSAMLphp. It's pimpliest possible way to get working SAML2 implementation of IdP and SP. You will want to follow first simpleSAMLphp Installation and Configuration and then SimpleSAMLphp Identity Provider QuickStart to configure simple IdP with static accounts so you can test your application against it. You will need both IdP and SP under your control to do development. It will also help if your remote IdP (identity provider which you intend to use) is also simpleSAMLphp (as AAI@EduHr is).

Installation is rather easy:

dpavlin@lib:/srv$ sudo apt-get install install memcached php5-memcache

dpavlin@lib:/srv$ wget http://simplesamlphp.googlecode.com/files/simplesamlphp-1.8.0.tar.gz

dpavlin@lib:/srv$ tar xf simplesamlphp-1.8.0.tar.gz
dpavlin@lib:/srv$ cd simplesamlphp-1.8.0/
dpavlin@lib:/srv/simplesamlphp-1.8.0$ cp config-templates/* config/
dpavlin@lib:/srv/simplesamlphp-1.8.0$ vi config/config.php

• auth.adminpassword
• secretsalt
• enable.authmemcookie

dpavlin@lib:/srv/simplesamlphp-1.8.0$ php5 -l config/config.php 
No syntax errors detected in config/config.php

To configure Apache side, you need Auth MemCookie but it isn't available in Debian package, so I opted for Apache::Auth::AuthMemCookie so I can flexibly modify IdP response before passing it on as environment variables.

dpavlin@lib:~$ cat /etc/apache2/conf.d/authmemcookie.conf 
Alias /simplesaml /srv/simplesamlphp-1.8.0/www
perlModule Apache::Auth::AuthMemCookie
<Location /cgi-bin>
        # get redirected here when not authorised
        ErrorDocument 401 "/simplesaml/authmemcookie.php"
        PerlAuthenHandler Apache::Auth::AuthMemCookie::authen_handler
        PerlSetVar AuthMemCookie "AuthMemCookie"
        PerlSetVar AuthMemServers "127.0.0.1:11211"
        PerlSetVar AuthMemDebug 1
        PerlSetVar AuthMemAttrsInHeaders 0
        AuthType Cookie
        AuthName "Koha SAML"
        Require valid-user
</Location>

To test it, easiest method is to create account in Feide OpenIdP and test against it. After all, it easiest to start with same implementation of SAML2 on both sides just to prevent following scenario:

On perl side I first tried Net::SAML2 and found out that it doesn't handle IdP without adding HTTP-Artifact support to the IdP. However, even after that I wasn't managed to make it work with simpleSAMLphp IdP implementation mostly because of my inpatience with SSL configuration of it.

On the bright side, for first test I didn't need to modify Koha (Library management software which I'm configuration SAML2 for) at all because it already has support for HTTP authorization.

As some of you already know, almost a moth ago we placed order for our first Kindle (3G with wifi). Week after it arrived, we also ordered bigger DXG to complement our reading habits. So why do we have two Kindles and do we have regrets with them? Well, no! This post will try to summarize what I learned since...

For a start, both devices have ARMv6, but K3G has 256 Mb of RAM while DXG has only 128 Mb. Both have 4Gb of flash of which about 3.5Gb is available for user content. Both have working 3G connectivity in Croatia. wpa_supplicant on Kindle won't allow connections to "enterprise" networks (like EduRoam). Smaller K3G is just too small for reading pdf files. Bigger DXG has e-paper in A5 size, so it's much more useful, but comes with older 2.5.8 firmware which has terrible browser (Netfront as opposed to WebKit in 3.1) and pdf reading without contrast settings (which is very, very useful).

But, I was fortunate enough to find mobileread forum post on migration of 3.1 software from K3G to DXG by Yifan Lu to whom I'm eternally grateful for much needed software upgrade.

However, even if you don't want to upgrade Amazon's own firmware, there is Duokan alternative reader software for Kindle. It has superb support for two column pdf files which allows you to just press next page key and read column on the left (from top to bottom) and then on the right (from top to bottom). It has also various other layouts for reading, and it makes smaller Kindle almost useful for pdf reading (and reading on bigger one a joy).

But, sooner or later, you will come across pdf which would really benefit from manual cropping. Take a look at briss which is cross-platform application for cropping PDF files.

While we are at software, you should really look at mobileforum's Kindle hacks thread which documents latest jailbreak (which will give you root permissions on Kindle), usbnetwork (to connect over USB cable) and various fonts and screensaver hacks.

Development for kindle can be done using Java2ME (remember that from cell phones?). However, Amazon doesn't actually seem to give out it's Kindle development kit but don't despair: Andrew de Quincey figured out a way to develop kindlets without the KDK. With KDK API from Amazon and JSR217 specification for Java2ME you can be in your marry Java development in no time.

I would like to see something like OpenInkport running on Kindle. This would allow free software developers to take full power from this nice e-paper device. If only I didn't have so much books to read...

We just got our conference booklet, and we need to publish it on the web. But, it has 152 Mb. This seemed excessive, so I googled a bit and found following:

$ gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=/screen -dNOPAUSE -dQUIET -dBATCH -sOutputFile=output.pdf input.pdf

$ ls -al input.pdf output.pdf 
-r-xr-xr-x 1 dpavlin dpavlin 158511430 May 11 16:23 input.pdf
-rw-r--r-- 1 dpavlin dpavlin   1646309 May 11 16:24 output.pdf

-dPDFSETTINGS=configuration to presets the "distiller parameters" to one of four predefined settings:

• /screen selects low-resolution output similar to the Acrobat Distiller "Screen Optimized" setting.
• /ebook selects medium-resolution output similar to the Acrobat Distiller "eBook" setting.
• /printer selects output similar to the Acrobat Distiller "Print Optimized" setting.
• /prepress selects output similar to Acrobat Distiller "Prepress Optimized" setting.
• /default selects output intended to be useful across a wide variety of uses, possibly at the expense of a larger output file.

While ago, I started writing MojoFacets, fast web-based faceted browser which keeps data in-memory. Since I last wrote blog post about it I added various features to it turning it into powerful spreadsheet-like application within browser in which you can mangle your data using perl code.

Let me start with a list of new features:

• run perl snippet over filtered subset of data, modifying columns (using $update) or creating aggregated result (using $out)
• format on-screen filter html with hidden ;, so that copy/paste into spreadsheet produce correct values and counts
• export dataset as tab separated values for easy migration into other applications
• use tab separated export and optional time format string with gnuplot to produce png graphs from data (this works well for huge datasets)
• export filtered values from facets in simple one-value per line format
• run perl snippets over filter's facet values to easily select ($checked) or calculate something with $value or $count
• import of CSV files (with optional encoding specified in filename)
• import from CouchDB databases or view
• import SQL query results from RDBMS using any DBI perl module (tested with PostgreSQL, mysql and SQLite)
• switch between loaded data-sets easily (filters are already shared between them, allowing poor man's join)
• implement lookup into different data-set with descriptive statistics on values

Adding perl code evaluation over dataset was logical extension since I already had web interface written in perl which had all data in memory. To make it fast, I had to implement indexes (and invalidation). But small things, like automatic generation of meaningful names for code snippets in form of dependent_col1,dep_col2.result_col1,res_col2 turned read-only web interface into powerful tool for application of reusable code snippets on data.

Latest feature is lookup to other datasets with ability to create multiple columns from lookup values. MojoFacets is now sufficiently advanced to replace relational database for quick join-like problems, but this time by writing a little snippet of perl looking like this:

lookup($row->{issn}, 'nabava', => 'issn', sub {
  my $stat = shift;
  push @{ $update->{listprice} }, $on->{listprice};
  $stat->add_data( $on->{listprice} );
},sub {
 my $stat = shift;
 $update->{price_min} = $stat->min;
 $update->{price_max} = $stat->max;
});

If you found Google Refine and wanted something similar, but in perl (with perl as query/mangling language) MojoFacets might be good choice.

I had a interesting problem with conference name tags this week. I wanted to use free software stack to produce them, and it turned out that this required patching bash to properly parse CSV files and learning new pdf tricks to print multiple pdf pages on single sheet of paper, so I will try to document them here, so I won't have to discover it again in two years for next one...

For design, we decided to use Inkscape, great vector drawing program. And even better, it already included Inkscape Generator extension just for that purpose.

For design, we decided to use ISO 7810 ID-1 card size of 85.60 × 53.98 mm and included template variables required by extension as you can see on picture.

Data for conference participants ended up in Gnumeric, and where exported in CSV file. And that's where we hit the first road-block. Current version of ink-generate extension doesn't support more than one comma inside quoted field. However, Inkscape Generator extension home page included useful pointer to correct bash code to parse CSV files by Chris F.A. Johnson so I decided to import it into ink-generator git repository and replace CSV parser. Few patches later and I had working extension which produces 600+ pdf files on disk.

In the process, I learned that you can invoke Inkscape extensions from command line, which is nice for generating previews while you edit code:

./generator.sh --var-type=name --data-file=test.csv --format=pdf --dpi=90 --output='$HOME/generator-output/%VAR_id%.pdf' --extra-vars=" " --preview=true akred1-var.svg

Now we had to print them. While some printer drivers have option to print multiple pages, the one we where using decided to center each name tag requiring too much cutting each side of each name tag manually. This was obviously non-efficient, and I knew about psnup utility for PostScript, but I didn't know that there is pdfnup which is part of PDFjam (and in Debian package). However, getting layout just right involved reading pdfpages package documentation, but this is what I ended up with:

pdfnup --suffix nup --nup '2x5' --paper a4paper --no-landscape --noautoscale true --frame true --outfile nup.pdf -- ~/generator-output/*.pdf

In the end, it didn't really worked out. Parsing CSV correctly (and supporting quotes inside quoted values) is a hard task in bash, and I had to admit that I don't really know how to fix it. With only a day to start of conference and no time to waste, I took my favorite language of choice, perl and wrote 60-line script which does same thing but uses Text::CSV perl module to parse data files.

There is something to be learned here: for a start language and good supporting libraries does matter. Second, sometimes you are better off starting from scratch. But that decision should be made only when you exhorted other options since fixing original extension would have benefit for wider community. There is a balance between scratching my own itch and common good which is tricky.

Browsing through subscribed videos this week on YouTube I stumbled upon video SImulating Markers with Tile Layers which described how to create custom tiles for Google maps using perl and PostgreSQL. John Coryat did great job at describing challenges, but also provided useful source code snippets and documentation how to create custom tiles. So, this weekend I decided to try it out using publisher field (260$a) from our Koha to show where are our books coming from?

I had several challenges to overcome, including migrating data from MySQL Koha database to PostgreSQL (so I can use great point data-type), geolocating publisher location using Yahoo's PlaceFinder (I tried to use Google's v3 geolocation API, but it had limit at 10 requests which wasn't really useful). I also added support for different icons (with arbitrary size) depending on the zoom level. In the process I also replaced cgi based tile server with mod_rewrite configuration which does same function but inside Apache itself.

Source code is available at github.com/dpavlin/google-map-tiles and it's really easy to overlay huge amount of data-points over Google maps using custom tiles, so try it out!

If you talked with me in last years or so, you probably heard me mention queues as new paradigm in application development. If your background is web-development, you probably wondered why are they important. This blog will try to explain why they are useful and important, and how you can make your app scale, even on same box.

Problem was rather simple: I needed to make monitoring which will pull data from ~9000 devices using telnet protocol and store it in PostgreSQL. Normal way to solve this would be to write module which first checks if devices are available using something like fping and then telnet to each device and collect data. However, that would involve careful writing of puller, taking care of child processes and so on. This seemed like doable job, but it also seemed a bit complicated for task at hand.

So, I opted to implement system using Gearman as queue server, and leave all scaling to it. I decided to push all functionality in gearman workers. For that, I opted to use Gearman::Driver which allows me to easily change number of workers to test different configurations. Requirement was to pull each machine in 20-minute intervals.

Converting existing perl scripts which collect data into gearman workers was a joy. At first run (with 25 workers) it took 15 minutes to collect all data. Just by increasing number of workers to 100 we managed to cut down this time just over 1 minute. And that's on single core virtual machine (which makes sense, since most of the time we are waiting on network).

For web interface, I decided to use Mojolicious. But, to make it work with Gearman, I write MojoX::Gearman which allows me to invoke gearman functions directly from Mojolicious. In fact, all functionality of web interface is implemented as Gearman workers, even querying database :-)