Avian’s Blog

Two weeks ago I got the idea that maybe it's time to move away from Nanoblogger and start using some "real" blog software.

Well, after experimenting with Wordpress and MovableType for last two weeks I must say that Nanoblogger still beats them all. Have you ever tried writing a post on EeePC with those Javascript WYSIWYG editors? I'll describe my experiences in some other post, for now I just want to say that all blogging software sucks, Nanoblogger just sucks the least.

So yes, I'm staying with Nanoblogger. I've modified it a bit, added support for comments and I'm hoping to rewrite some parts of it in Perl to make it faster.

Oh, and there's also jsMath:

Anyway, if something's not working for you now, please leave a comment or drop me a mail.

Today I started getting errors like this on Eee:

ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
ata2.00: cmd ef/05:fe:00:00:00/00:00:00:00:00/40 tag 0 cdb 0x0 data 0 
         res 51/04:fe:00:00:00/00:00:00:00:00/e0 Emask 0x1 (device error)
	 ata2.00: configured for UDMA/33
	 ata2: EH complete
	 sd 1:0:0:0: [sda] 7815024 512-byte hardware sectors (4001 MB)
	 sd 1:0:0:0: [sda] Write Protect is off
	 sd 1:0:0:0: [sda] Mode Sense: 00 3a 00 00
	 sd 1:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA

That's with a stock Debian 2.6.23 kernel which I've been running since I first installed Debian.

I just hope this doesn't mean the solid state drive is already reaching its maximum number of write cycles. So far these errors don't seem to have any effect short of annoying me when I'm using the console.

Update: Mystery solved. Some browsing through kernel source revealed that "ef" is ATA command ATA_CMD_SET_FEATURES. This sounded like something hdparm would do and indeed hdparm was being run from some misconfigured ACPI scripts I forgot to clean up yesterday.

I was wondering yesterday why when I press the LCD brightness hotkeys on my EeePC (Fn-F3 and Fn-F4) I don't see the nice GNOME-styled OSD (like the one that pops up when I change the speaker volume).

Well, it turned out that quite a few things needed fixing. Among other things HAL configuration and GNOME Power manager. However even before I could start fixing things there was a non-trivial matter of finding out all the steps that happen after you press the hotkey and before the volume or LCD brightness get changed. As far as I could see this isn't documented anywhere so in case someone else will do something similar, here's what I found out.

Note that this is most likely Debian and EeePC specific and will probably change soon.

Step 1: Hardware

When you press a Fn-F combination, laptop's hardware generates an ACPI event. This event is gets picked up by acpid using the Linux kernel ACPI driver.

In EeePC's case Fn-F4 and Fn-F3 keypresses also directly modify the brightness setting without any software intervention. Not so with audio volume hotkeys.

Step 2: ACPI

acpid consults its configuration in /etc/acpi/events and runs a shell script that corresponds to the ACPI event.

This shell scripts runs acpi_fakekey that inserts a keycode into kernel's keyboard event FIFO. This in effect simulates a keypress on the keyboard.

This is done like so because different laptops have different means of reporting hotkeys (different ACPI events, scancodes, etc.). After this step all these events are translated into a standardized keycode in the event FIFO (like 224 for KEY_BRIGHTNESSDOWN - see /usr/share/acpi-support/key-constants)

Step 3: GNOME

Some GNOME component gets the keypress via an X event and reacts to it. In case of LCD brightness hotkeys the component is GNOME Power Manager and in case of audio volume hotkeys it's the GNOME Settings Daemon.

It might be that HAL also plays a role here. Maybe these applications get the events through dbus from HAL. lshal -m suggests that HAL also reacts to these keypresses, but I haven't dig deep enough into the code to see if GNOME listens to it.

There's also some weird business of how keycodes get translated into X events via xkb settings. A document on Ubuntu Wiki tries to clear this up, but I failed to see the exact connection between codes in X configuration and codes in acpi_fakekey.

Step 4: HAL

GNOME component displays a box with the current status and sends a message to HAL to change the LCD brightness.

HAL has some configuration in /usr/share/hal/fdi/policy that tells it how to do that. For example in case of LCD brightness setting it calls /usr/lib/hal/scripts/hal-system-lcd-set-brightness and /usr/lib/hal/scripts/linux/hal-system-lcd-set-brightness-linux

In case of audio volume GNOME Settings Daemon looks like it changes the setting itself, without resorting to HAL.

Also, since EeePC's hardware changes the brightness itself, this last step is unnecessary - all GNOME needs to do is display the box (and that's one of the things that needed fixing).

Hruške has some well founded arguments against my last Python performance rant.

I stand corrected. I must have messed something up with my measurements. I have now repeated them on two different machines and I got the same results as Hruške. So the in operator in Python is indeed fast when the right hand operand is a dictionary.

I can only say in my defense that I thought that this part of Python documentation confirms that the in operator works in a general way on all iterable objects:

__iter__()
Return the iterator object itself. This is required to allow both containers and iterators to be used with the for and in statements. This method corresponds to the tp_iter slot of the type structure for Python objects in the Python/C API.

As for his other comment about Wikiprep code size:

avian@toybox:~/dev/html2latex-1.1$ sloccount .

... snip ...

Totals grouped by language (dominant language first):
perl:           797 (100.00%)

Total Physical Source Lines of Code (SLOC)                = 797

avian@toybox:~/dev/wikiprep$ sloccount .

... snip ...

Totals grouped by language (dominant language first):
perl:          2060 (85.12%)
cpp:            220 (9.09%)
python:          50 (2.07%)
ansic:           49 (2.02%)
sh:              41 (1.69%)

Total Physical Source Lines of Code (SLOC)                = 2,420

So yeah, Wikiprep Perl code which I maintain for Zemanta is still bigger.

That of course doesn't mean that there aren't any bigger Perl monstrosities in the wild. After a quick survey of free Perl software I have on the disk I found gtablix which has 6283 Perl SLOC and GCfilms with an unbelievable 25758 Perl SLOC (neither of which seems to be maintained BTW).

Today I wanted to burn a CD which I could listen in my car. Since the CD player there supports MP3 format this shouldn't be that hard, right? I'll just copy the music I already have on my hard disk (and which I ripped from my CD collection) to a data CD and that's it.

Well, not really. On my Debian machine, all the music got ripped into Ogg Vorbis (since that was the default setting in Sound Juicer). On my PowerBook, iTunes ripped everything into AAC (again, the default setting). I'm sure that if I had a machine running Windows, everything would be in some Microsoft proprietary format.

I know, I know. I should have checked the settings before feeding my computers all my CDs. Still, it's frustrating that I have to go through all that again.

And of course, I didn't have any such problems with music that I downloaded from the net (like free tracks from Machinae Supremacy, don't get me wrong).

A laptop on the back seat running a wireless LAN sniffer can yield some interesting results. Here's some statistics I gathered with Kismet while driving on the route Logatec - Ljubljana - Maribor - Graz:

Driving route

Percentage of networks using encryption

Channels used

Kismet has seen a total of 368 different networks.

One thing that I didn't expect to see is the channel distribution. Why are most networks on channel 11? Maybe it's a default setting for some popular equipment.

Also surprising were a couple of networks that are encrypted, but have names that include the owner's GSM number (like "041xxxxxx Call for service"). I haven't seen this before, but it seems like a nice gesture.

Unfortunately I didn't have a GPS attached to the computer, so I can't draw an exact distribution of networks on the map.

Quote Python documentation:

latex2html: Probably the longest Perl script anyone ever attempted to maintain.

Hm...

avian@toybox:~/dev/html2latex-1.1$ wc -l html2latex HTML/Latex.pm 
  276 html2latex
 1458 HTML/Latex.pm
 1734 total

avian@toybox:~/dev/wikiprep$ wc -l *.pl *.pm
 2466 wikiprep.pl
  129 images.pm
  359 languages.pm
   80 nowiki.pm
   76 revision.pm
  183 templates.pm
 3293 total

(cue manical laughter)

Update: Please ignore this rant. I don't know what I've been doing wrong here, but my later measurements confirm Hruške's findings.

Python provides a convenient in operator that returns true if the left hand value is an element of the right hand list or dictionary. That's nice and all, but the performance makes it pretty useless in all but trivial cases. Take a look at this graph:

On the vertical axis is time needed for a loop to finish while test_size value is on the horizontal axis.

I ran these three loops that are exactly equivalent as far as the effect of the if statement is concerned:

"in list": the obvious way

list = range(test_size)

for m in xrange(2 * test_size):
	if m in list:
		a += 1
	else:
		b += 1

"in dict": converting to hash doesn't help

list = range(test_size)
dict = dict.fromkeys(list, 1)

for m in xrange(2 * test_size):
	if m in dict:
		a += 1
	else:
		b += 1

"dict.get": the right way

list = range(test_size)
dict = dict.fromkeys(list, 1)

for m in xrange(2 * test_size):
	r = dict.get(m, None)
	if r:
		a += 1
	else:
		b += 1

If you haven't noticed, this blog is now a bit more web-two-ohish. I've added a Foxmarks widget to the sidebar (the "Shared bookmarks" thing) that shows the last few bookmarks I made (well, those that I let you see).

Foxmarks is a Firefox plug-in and a service that allows you to synchronize Firefox bookmarks between different computers. I've been using it for some time on my desktop and laptop computers and have been very satisfied with it - it quietly lives in the background and does its thing without annoying you with any questions. I prefer it to services like delicious because it's integrated into the browser and because it allows you to use your own sync server instead of theirs if you're concerned about privacy.

They introduced the bookmark sharing service a couple of weeks ago and I thought I'll give it a try.

This simple Python one liner will not do what you expect:

>>> import re
>>> re.sub("the", "", "The Last Lecture", re.I)

Before someone again accuses me of being illiterate: Yes, this behavior is documented. It's still one of those things that go undetected for a long time and bite you in the most inappropriate time. It's right up there with C's "if(x=0) ..." on the annoyance scale.