Avian’s Blog

In my previous post I've mentioned that Kindle's AR6002 wireless chip set seems to work fine with the configuration tools that come with Debian.

Such setup however has an annoying problem. The wireless interface doesn't see packets addressed to the MAC broadcast address (even in promiscuous mode with tcpdump). This means that the kernel on Kindle won't see things like ARP requests (but will see ARP replies) and pings of the broadcast IP. The former means that IP connections initiated from other machines on the network mostly won't work, unless these machines already have Kindle's MAC address in their ARP cache. You can work around that by adding a manual entry for Kindle in the ARP cache or making Kindle establish a connection first, but these are ugly workarounds.

It turns out that the AR6002 driver that was shipped by Kindle kind of implements standard Linux wireless extensions, but not really. Kindle OS uses a modified wpa_supplicant that uses some kind of a proprietary ioctl interface to talk to the kernel module and using that the broadcasts work correctly.

Since WPA supplicant comes under the BSD license, Amazon didn't feel the need to provide the source to the modified version in their GPL-compliance package (aren't permissive licenses great?). Hence I've again lifted the binaries from the Kindle OS image and created another Debian package, similar to the firmware-ar6002 I mentioned in the last post.

The wpasupplicant-ar6002 package is available here and of course will only install on the armel architecture. It uses a bit of dpkg-divert and shell scripting magic to transparently add a ar6000 driver to wpa_supplicant. This means that you can simply use the wpa-driver directive in the interfaces and the system will use Amazon's patched binary instead of the one shipped with Debian.

The full /etc/network/interfaces snippet for Kindle's wireless network now looks like this:

auto wlan0
iface wlan0 inet static
        pre-up wmiconfig -i wlan0 --power maxperf

        address xxx.xxx.xxx.xxx
        netmask xxx.xxx.xxx.xxx
        gateway xxx.xxx.xxx.xxx

        wpa-driver ar6000
        wpa-ssid xxx
        wpa-psk xxx

I spent the last week in Brussels. The CREW project I'm involved in at IJS has organized a couple of events there as well as a plenary meeting, so the past few days have been quite exhausting, not to mention the week leading to it spent in worries and preparations.

I haven't been to Belgium in a few years and this was the first time I actually flew in. The prices for flights from Ljubljana have always been unreasonably high, with kind of an urban legend going on that it's an unofficial way of the government subsidizing our national air line by paying high prices for frequent flights of various government officials.

In any case, my flight landed late at night and the Brussels airport was more or less dark and deserted. Dark, except for big, brightly lit LED boards with advertisements. These were giving optimistic visions of a bright and sustainable future all along the long path you have to walk from the gate to the train station. And the first of these was telling us in large, friendly letters that European Parliament protects our rights. I can tell you that coming from our small, unfashionable airport to this scene reminded me of a kind of certain not-so-optimistic science fiction stories. The fact that my hotel reservation came with a legal disclaimer about assistance with authorities did not help the issue either.

The first order of business in Brussels was a workshop on TV white-spaces for members of the European Commission. As you might know, there is a lot of discussion going on about how to re-use the frequencies that were freed by the transition to digital broadcasts. As a project that also works in that field we presented our view on that topic to people working on spectrum regulation.

The visit to the European Commission offices was actually quite different from what I expected. I was anticipating a dusty, gray place with laser-printed passive-aggressive notices hanging around the hallways that I usually associate with government buildings here. Instead, the part of the Beaulieu 33 I saw could probably compete with Hekovnik on the number of colorful and inspiring messages stuck to the walls. Not to mention various, kind of silly posters regarding network security (in the fashion of "you don't share your toothbrush, you shouldn't share passwords either"). The security procedures as well, while visible, were pretty unobtrusive and mostly involved displaying a kind of a self-destructing badge (it got crossed-out with "expired" all by itself after a day through some kind of a chemical process I guess).

The other public event we held were CREW training days at the Vrije Universiteit Brussel. I gave a tutorial there on how to use Jožef Stefan Institute's cognitive radio testbed in Logatec and the hardware I developed for various experiments. I'm happy that I received some positive responses to that. At least for me it was a big confirmation that the tools what we are developing at the Institute are actually useful to this research community and that we are contributing in a positive way.

While trying to enter the university building on Thursday we found ourselves in front of a crowd of protesters (there was a general strike in Brussels that day), so perhaps not everyone agrees with that. I'm not quite sure though whether the university itself or people employed there were the target of their protest or we just happened to be in the wrong place at the wrong time. I would say the latter, but on the list of offices on that particular address I couldn't find any kind of institution that would be worth protesting against in my opinion. Anyway, I was not able to understand any of their complaints through loud fireworks while we were entering the building under the watch of the local riot control police.

Setting up wireless LAN under Debian was a bit tricky. Kindle 3 has a Atheros AR6002-series chipset and Amazon ships an ar6000 kernel module for it. However simply loading this module doesn't work. User space needs to do a complicated firmware setup procedure before a standard wlan0 network device appears.

On a stock Kindle OS, firmware and the kernel module is loaded by /opt/ar6k/host/support/loadAR6000.sh that is invoked from /etc/init.d/wifi. This script in turn depends on two binary executables (bmiloader, eeprom.AR6002) and quite a few binary firmware files. For my new Debian system, I've shuffled things around a bit - I've made a new, LSB-compliant init script and moved files from /opt/ar6k to /usr/share/firmware-ar6002. The scripts themselves I left mostly unmodified, except for commenting out parts that were communicating with other parts of the (now absent) Kindle framework.

There's also a wmiconfig utility that allows you to set a lot of hardware-specific settings once the driver and firmware has been loaded.

For the rest of the wireless network setup, the stock Debian ifupdown framework seems to work just fine. This is what I have in /etc/network/interfaces at the moment:

auto wlan0
iface wlan0 inet static
	pre-up wmiconfig -i wlan0 --power maxperf

	address xxx.xxx.xxx.xxx
	netmask xxx.xxx.xxx.xxx
	gateway xxx.xxx.xxx.xxx

	wpa-ssid xxx
	wpa-psk xxx

The wmiconfig part comes from the Openmoko Wiki where it says that it makes networking more reliable (looks like at the expense of power consumption). Although I didn't see it make any noticeable difference.

I have seen a weird problem though where sometimes the Kindle won't see ARP broadcasts when another machine on the network tries to establish a connection to it. If that happens, tcpdump on Kindle doesn't show ARP request packets even when a sniffer on another machine on the same wireless network does see them. Other connections work normally however and if I manually add an ARP table entry on the connecting machine, it can connect to the Kindle without problems. So this doesn't look like a power management problem.

For anyone interested in running Debian on their Kindle like this, I've made the firmware-ar6002 package available here. It will install on armel architecture only and assumes that modprobe can load the ar6000 kernel module (module itself is not included).

Finally, a note about licensing might be appropriate here. This package contains scripts and binaries that were lifted from the read-only root partition that Amazon ships with Kindles. All of them seem to come from Atheros' SDK (see Openmoko SVN for example) and that code appears to be under GPL 2. However, Amazon doesn't include sources for these utilities on their source code page. It might be that Amazon got this code under some proprietary license from Atheros. There are certainly no apparent notices regarding that.

I'm going to assume nobody will be too angry if I publish this Debian package, but if you want to redistribute it, know that it's at best a legal gray area.

Playing with my Debian-running Kindle over the serial console (because I don't have networking running natively yet under Debian) exposed one interesting feature. The serial port would stop responding if it was left idle (no traffic in or out) for a few tens of seconds. By accident I found out that sliding the power button restores it back to normal operation, however the reason for this was a mystery until I looked into the Linux kernel source.

Kindle uses a NXP MX35-series CPU. The serial port /dev/ttymxc0 is provided by the mxc_uart driver, which is compiled into the kernel image shipped by Amazon. Inspecting the source (drivers/serial/mxc_uart.c in the Linux kernel tree) shows that the driver implements a power-saving feature that powers down the peripheral and disables its clock if there is no activity for 30 seconds.

This clock gating feature is enabled by default, but luckily you can disable it through the sysfs on a per-UART basis. I now have the following line in the /etc/rc.local file and the serial console remains available even if left idle for a while:

echo 0 > /sys/devices/platform/mxcintuart.0/uart_clk_state

It seems I'm bound to hacking various ARM-based devices these days. I've been using a kindly donated Kindle 3 with a broken screen for a few months now as kind of sandbox machine. It's always nice to have a few systems on-line one can log into from various unfashionable corners of the Internet for bouncing connections or for running a service on a whim. A Debian-running Kindle is especially nice for an always-on machine, since it consumes very little power.

So far I've been running the Kindle with a stock system, only jailbroken and with the usbnet hack installed. I installed a basic, 1.5 GB Debian chroot into the FAT partition on Kindle's flash and had a ssh daemon running on the wireless LAN interface. While it's still quite useful, that's only a part of the 4 GB of built-in storage and the daemons running in the stock system consume a lot of RAM. Plus, they sometimes mess with the hardware in ways I don't like. For instance, the wireless LAN connection usually drops when USB power goes away. Obviously the way to go was to run Debian natively.

Before I could install Debian onto the flash drive directly, I had to find out how the Kindle actually boots and what facilities are there to recover in case something goes wrong.

As seems to be the standard on Linux-running ARMs, the first visible thing that happens is U-boot. It's stored in boot flash memory that is mapped into CPU's address space (this flash is different from the built-in MMC drive that the operating system uses).

If you press a key on the serial console at the Hit any key to stop autoboot prompt, you get a simple command line. It allows you to inspect memory, load code or data via the serial port into RAM or flash and boot a kernel that is present somewhere in the address space. There seem to also be some facility for tftp network booting, but without a wired Ethernet adapter on board it appears useless. Although U-boot can also include functionality for accessing the MMC drive, version on my Kindle has that disabled.

If you don't want to mess with the boot flash (I don't want to for now), the only thing that you can do from U-boot is to boot the Linux kernel. Kernel and an initrd image are also stored in the boot flash and hence directly accessible in the ARM address space.

The initrd image contains some recovery utilities that can be accessed by pressing Enter at the Press [ENTER] for recovery menu... prompt. Since these run directly from the initrd, they work even if the MMC drive is completely corrupted. The most useful is option 3 from the recovery menu. It exports the MMC drive as an USB mass storage device, which can then be used to setup the root filesystem from another computer (with friendly still alive... messages periodically being sent to the serial line).

Menu
====
3. Load MMC0 over USB storage
4. Erase MMC0
I. Initialize Partition Table (fdisk) and format FAT
O. Format and overwrite FAT partition
E. Export FAT partition
U. Update using update*.bin file on FAT partition
M. Update using update*.bin file on FAT partition of second MMC port
D. dmesg / kernel printk ring buffer.
Q. quit 

Oh, and option 3 requires a password, which is the well-known fiona + four (not three as for the root password) hex digits of the MD5 digest of the serial number.

If you don't enter the recovery menu, the code in initrd mounts the root filesystem and executes /sbin/init. From there on things proceed as usual in a normal Linux system. Kernel parameters stored in U-boot set the root filesystem as the first partition on the built-in MMC drive.

This boot process seems to be completely compatible with an armel Debian Squeeze install. One snag I did encounter is that the root partition needs to have at least /dev/console and /dev/null special files. Other device files are created at boot time by udev, but without these two, I get a tried-to-kill-init kernel panic at boot. Also, it helps to double-check /etc/fstab and /etc/inittab, since debootstrap doesn't set these up for you. Root filesystem is /dev/mmcblk0p1. If you want to be able to log-in from the serial console, a line like this works in the inittab:

mxc0:2345:respawn:/sbin/getty -L 115200 ttymxc0

The original Kindle comes with a MBR-style partition table on the MMC drive (here it's called /dev/sdd since I was inspecting it from another computer - on Kindle it's /dev/mmcblk0):

Disk /dev/sdd: 4001 MB, 4001366016 bytes
4 heads, 16 sectors/track, 122112 cylinders, total 7815168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1   *        7688     1338943      665628   83  Linux
/dev/sdd2         1338944     1388095       24576   83  Linux
/dev/sdd3         1388096     1404479        8192   83  Linux
/dev/sdd4         1404480     7815167     3205344    b  W95 FAT32

sdd1 is the root partition for the stock Kindle OS with an ext3 filesystem. sdd2 contains another ext3 filesystem with some kind of configuration and log storage. sdd3 is empty and was all zeros on this Kindle. sdd4 contains the storage that is visible when you attach an un-modified Kindle to a computer. It contains another MBR partition table with a single, FAT32-formatted partition.

I removed all of these and just created a single ext3 partition over the complete drive. At boot this causes some scary messages from the initrd code which seems to attempt some sanity checks on the partition table and checks for possible system updates in the FAT32 partition. Apparently it tries to correct the situation, but luckily gives up without modifying the partition table before starting Debian's init process.

While I now have a basic Debian installation running directly on the Kindle, there are still things missing. For instance, I still have to get wireless LAN to work again and there are some oddities with power management which I hope to cover in another one of these posts. All in all, Kindle turned out to be much more hacker-friendly than the Chromebook, which is surprising considering that in contrast to the evil Kindle the former has been advertised as being developer friendly.