Avian’s Blog

I just found out that you have to load the ip_nat_irc kernel module on the firewall in order for outgoing IRC DCC transfers to work. The ip_conntrack_irc module, as inviting as it looks, doesn't work by itself.

Is it just me or netfilter kernel modules really don't have any official documentation? My Google searches for ip_conntrack_irc resulted only in numerous more or less informative and more or less up to date mailing list and forum posts. The best source of information I have found so far are the comments in the code itself.

Yes, it is actually possible to run Tablix on Windows XP!

The binary in the screenshot was compiled with GCC under Cygwin, but once it is built does not require a Cygwin installation.

You have to make some moderate changes to the Tablix source before it will compile cleanly on Windows. (Some of the changes are documented in the Wiki). Autotools (automake, autoconf, libtool, ...) fail spectacularly on Cygwin so also a new set of Makefiles is required.

PVM is horribly broken on Microsoft's systems, so Windows machines can't be a part of the cluster. After some testing it turned out that Tablix can solve problems of moderate size quite efficiently even in debug mode (i. e. genetic algorithm with a single population without using PVM), but you have to do some tweaking of the genetic algorithm parameters.

Someone is trying to post spam to mailing lists on my server. They are sending spam to the correct mailing list address with my old email address in the From: field.

Since I'm subscribed to the list with my new address, the Minimalist mailing list manager bounces spam back to me, with the attached warning saying that I'm not subscribed.

I got 50 such emails in just one day. This would wrack havoc on the mailing list if they would succeed to spoof my proper email address.

Is this only a coincidence or is this work of some bot that is smart enough to find the email addresses of people that are subscribed to the list (from the list's archives for example) and spoof them?

I don't currently see any simple way of preventing people from providing spoofed email addresses to the Minimalist. This spam is coming from a DSL IP range in Morocco so if this will get any worse, I'll simply route this IP range to /dev/null.

VI Improved 7.0 packages got into the Debian Unstable repository today (just 10 days after the upstream release)! Now it's just an apt-get install away from my machine...

I've fixed my problem with Mail.app today.

The problem was that when Mail said The certificate for this server has expired it actually wanted to say The root CA certificate that signed the certificate for this server expired. Another fine example of misleading error messages.

The fact that the keychain that holds root CA certificates isn't accessible by default on Mac OS X just further complicated things. You can only add certificates to it by selecting a mysterious X509 Anchors keychain on the Add Certificate dialog but there is no obvious (or documented as far as I know) way of deleting them. It took quite a lot of experimenting (for example with the quite limited security command line tool that claims to be able to do just about anything the Security framework is capable). In the end, I found out that you can access the X509 Anchors keychain by simply opening the Keychain application, choosing Add Keychain... from the menu and selecting /System/Library/Keychains/X509Anchors.

Mac OS X Mail application on my Powerbook earlier today suddenly realized that we are way past February 2008, which means that the SSL certificate for my mail server expired and I can't read my mail.

To make things a bit more interesting, the Keychain thingie keeps telling me that the certificate is still valid. The system date and time are also correct as far as I can see and Thunderbird on another computer and another operating system is perfectly happy with this certificate.

This has no effect.

As always, things like this happen at most unfortunate times.

I really like the way art.gnome.org displays dates. Instead of giving you a boring "Mon Mar 11 02:25:47", that leaves you wondering what the current date and time is, it will for example simply say "Two days ago".

I would like to display document modification dates in this same way in the next version of my Nanobeagle search engine. Of course, I could write a function that gives a date in this form myself. It would probably be just a matter of some simple arithmetic and comparisons, but why should I reinvent the wheel?

So I set out to find the original function behind this date display. A quick search through the FAQ and Forums on the site revealed no information on what software art.gnome.org is running. When searching for "art.gnome.org software" on Google all hits were about the graphical front-end software "Gnome Art" not the CGI scripts that are running the site itself.

Ok, perhaps GNOME people store their web scripts in their CVS repository. My first guesses were "web" and "web-devel-1" projects, but they seem to only contain static pages. Finally it turned out that "art-web" was the right choice. A quick cvs checkout and grep for the word "yesterday" later, I found the FormatRelativeDate() function I was looking for.

Put this script in your $HOME/.gnome2/nautilus-scripts directory. Make sure it has execute permission set.

You can then open a terminal window from the Nautilus context menu. If you have any directories selected in Nautilus one terminal window will be opened for each of them. Working directories will be automatically set to the directories selected in the browser. If no directories are selected only one terminal will be opened with the working directory set to the directory currently opened in Nautilus.

Referring to my previous post, the right way to get serial devices back under udev is to load kernel module 8250.

And yes, the serial devices are still named /dev/ttyS0, /dev/ttyS1, etc. in spite of Serial HOWTO claiming otherwise.

I'm easily annoyed by software that doesn't work or somehow mangles Slovenian characters č, ž and š. So I really want Tablix to work correctly with all languages.

Currently Tablix already handles different languages quite well, but according to LibXML documentation this is a mere coincidence, since a lot of my code is not written in The Right Way. I want to correct that.

However more I look into this subject more it seems to me that this task is really complicated. The main problem in internationalization is that for each string stored in the memory you must know exactly what its encoding is. This is not as trivial as it sounds, since there are a lot of different sources of strings in Tablix:

• Strings obtained from the LibXML are the least problematic, since all are encoded in UTF-8. These are also nicely contained in xmlChar type.
• Constant strings, internationalized with GetText (_() macro), are encoded in the encoding of the current locale which can be anything.
• Simple constant strings in C. I guess these are in whatever encoding the C source file was, but I'm far from sure after some googling.
• Strings returned by sprintf(). I think these should be in the encoding of the current locale, but again not sure.
• Command line arguments obtained from argv. Current locale I guess.

A similar problem appears when these strings are written somewhere. But here at least the situation is a bit less foggy.

• All strings passed to LibXML must be UTF-8.
• Strings written to the console must be in the current locale encoding.
• Strings written to HTML files must be in the encoding of that file.
• File names must probably be in the current locale encoding. Not sure.

I'll buy a beer for anyone who can clarify the points I'm not sure about.

I just managed to clean up and somewhat document the Templator script I made for the OpenOffice.org conference web site last September.

This is the first of a series of tools I developed in a hurry during those three days and that also proved useful for other projects. Other tools and scripts will probably follow in the more or less near future, depending on my free time and sleep patterns.

It seems these damn GNOME xkb errors are spreading. Now my home machine has contracted them too after upgrading from GNOME 2.10 to 2.12. And unfortunately the solution I mentioned previously no longer works.

Deleting all dot files in my home directory also doesn't help.

Making /var/lib/xkb world-writable - no effect.

/etc/X11/xkb/rules/xorg - check.

Asking for help on IRC in #gnome - no answer, since this problem has already been discussed to death on all sorts of message boards, mailing lists, wikis, bugzillas (earliest posts I could find regarding this were from 2002!). Of course, it could be that the error or my machine is caused by something completely different than in all those other hundred case, but how can I tell? The fscking error message is exactly the same and offers no clues on what caused it.

If you are getting annoying error messages like these every time you start GNOME or try to change the keyboard layout with gnome-keyboard-preferences and if setxkbmap seems to be working fine, check if you have this file: /etc/X11/xkb/xkbcomp. If you don't have it, try to find it somewhere (I found it in /usr/X11R6/bin) and make a symlink like this:

$ ln -s /usr/X11R6/bin/xkmcomp /etc/X11/xkm/xkbcomp

Why exactly one needs to spend a couple of hours stracing various Gnome binaries to get to this conclusion is left as an exercise for the reader (hint: try running a rogue gnome-settings-daemon from a terminal).

This is another fine example of how not to code your web application. The following window opened when I clicked on a link saying something like click here for details.

For english speakers: the text in red says "System error report" and the smaller text says "System error: Error".

So as I understand this, the detailed description of this error is that the system error report had a system error reporting an error which was caused by an error. Just great!

Wow. I just found out that Linux kernel on my machine wasn't using all of the available memory.

Thank you Borland, for making your antique Turbo C 2.01 compiler available on the internet as a free (as in beer) download.

I was stuck with my keylogger project. For some reason as soon as the size of the code for the 89C4051 microcontroller grew above 2048 bytes it stopped working. The program ran fine in the simulator but when I burned it to the actual chip, it remained completely dead.

I first suspected the old Keil compiler I'm using. But why would the code then run fine in the simulator? The programmer was also unlikely to cause this kind of a problem because the flash memory on the controller is programmed via a serial bus. So it is impossible for example for a single address line to loose connection.

At the end it turned out that the programmer software, a small DOS program, was only capable of dealing with 2 kilobytes of data at once. It took me so long to figure that out because the verify function also only compared the first 2 kb of the file on the disk with the first 2 kb of the flash memory. It looks like someone at Atmel really did not want to waste to much RAM.

After some searching on the Atmel website I found the C source code for this software, but without any instructions on which C compiler should I use. This string inside the compiled program gave me the first hint where to look:

MS Run-Time Library - Copyright (c) 1988, Microsoft Corp

So obviously this was compiled with a Microsoft C compiler. Now Microsoft C compilers for DOS vintage 1988 aren't exactly easy to find so I installed the best alternative I could find on my FreeDOS machine - Borland Turbo C. Of course the compiler then gave me a ton of errors. For example, Microsoft C obviously known C++ style comments back then, but Borland didn't. Also Microsoft C used some ugly-named functions for dealing with character graphics. Compare for example this from MS:

_clearscreen( _GCLEARSCREEN );

With the equivalent from Borland:

clrscr();

Luckily some search & replaces with the DOS version of VIM fixed that. It turned out however that a part of the program was written in assembler in a syntax that Borland C didn't recognize.

Some tweaking later I managed to successfully link the object file from Turbo C with the object file of the assembler part I got from Atmel that was made with Microsoft Assembler. I still can't believe that this worked. From my experience linking two object files even from different versions of the same compiler is asking for trouble.

After I got this working the solution to my problem was simply a matter of increasing a single constant from 2048 to 4096. So in case anybody else has the same problems, I'm putting the fixed compiled software on the net.

Reflections on Trusting Trust by Ken Thompson, one of the original authors of Unix, shows in a very nice way how you can't trust any binary if you can't trust the compiler that was used to compile it.

I think the part about the concept of a "learning" program is worth reading alone. I knew compilers are usually compiled by themselves (take GCC for example - it seems that the basic sign that a compiler is actually usable is that it can compile itself), but I never looked at this practice from the view point presented in the article.

I was planning to add a search function to tablix.org for some time. Specially the mailing list archive got so large that it is very time consuming to find posts about a specific subject.

All free search engines I found on the net resembled Google and other general internet search engines, which is not what I want. These search engines use a web crawler to index data on the web page which isn't very efficient if you want to index a trusted and a well known web site. Also simple web crawlers are bad because 1) They index all text (including menus, etc.) on web pages, not only content 2) They don't have any idea what exactly are they indexing. If you get a search hit, you have to check it manually if it is a news article, a mailing list post or a page in the documentation.

On the other hand, desktop searches, like Beagle are much more advanced - they use plugins to recognize and properly index each type of file on the filesystem. So I tried to make a web search engine that would be as flexible as that. It must use plugins to properly index mailing list archives, wiki pages, Nanoblogger posts, articles, Doxygen reference, etc. I used this Beagle mockup from Beagle UI Hackfest as a guide.

The result: Nanobeagle.

It's written in Perl, uses Swish-e indexing engine and it's not yet very stable. It currently has three indexing plugins: Hypermail archives, Nanoblogger news and Nanoblogger static articles. Icons are from the gnome-icon-theme part of the GNOME CVS repository.

I've been getting some strange spam lately. While Thunderbird doesn't seem to have problems, it crashes my Mac OS X Mail. Each time I open the Junk mail box, the application crashes with Sorting messages displayed on the status line. It doesn't respond to the Quit command and I have to use the Force quit from the Apple menu.

The solution I found at this time is to ctrl-click on Junk mailbox and select Delete junk mail. Mail will crash anyway, but next time I open it, the Junk mailbox is empty and the nasty mails are gone.

I'll see if I can find the specific message that causes this.

Do you want your code to be fast? Take my advice and don't even try hand optimizing it for these fancy SIMD instruction sets like MMX, SSE and similar. In fact, don't even google for "mmx optimization" or you'll be deceived by pretty numbers and benchmarks and you'll forget what you've read here. You will spend hours writing optimized memcpy() functions, parallelizing loops in your code and putting those damned PREFETCHes everywhere. And in the end (if you're anything like me), you'll rm -rf everything in frustration because no matter how you look at things, your cool assembly MMX optimized code will always run slower than plain dumb unoptimized C.

Or perhaps I'm doing something wrong here? Perhaps not. (No, I'm not mixing floating point and MMX!) Even AMD's optimization guide says that in most cases you should use scalar instructions instead of vector ones. Instead of parallelization it recommends some (in my opinion very ugly) hacks that enable you to take for example some control over what is stored in processor cache, how instructions are pipelined and how successful branch predictions will be. Those video software guys must be practicing black magic or something if they really get that 200% boost from MMX.

The problem I see here is that today's CPUs have too much internal logic that thinks it is smarter than you and reorders instructions, renames registers, controlls cache, ... I agree that's good for those 99.9% of the code that you don't want to bother optimizing, but for the few loops that need to be hand optimized, this is a nightmare. Instead of simply instructing the CPU to store this and that in its cache, I have to perform some weird sequence of reads from memory (take a look at some of the examples in that optimization guide if you don't believe me) so that I'll trick CPU's internal logic into thinking that perhaps it is a good idea after all to store that in the cache. And if I don't do it just right, the wrong things will end up in cache and that latest CPU will start emulating a 486. It would be nice if CPUs would have a mechanism to turn all this mess off and let me be in control for a while.

I miss the days when I could count the exact number of cycles the CPU needed to execute a function instead of having to measure it with 10% uncertainty.

I have to design, implement and document a simple electronic device that uses some kind of a microcontroller as a term project at the Faculty. Since I didn't find any of the proposed projects particularly interesting (why should I need another temperature controller or another digital thermometer?), I decided I will try to copy ThinkGeek's KeyKatcher.

It is critical for this device to be small (it should be handy, since you never know when you'll have to troubleshoot some software :) so I'm going to use a microcontroller that requires a minimum number of external components and a single high-density Flash or EEPROM chip.

I'm currently thinking of a combination of Atmel's AT89C2051 microcontroller and 24C1024 serial EEPROM. 2051 is a 8051-compatible chip that only needs an external quartz crystal for a clock generator. It has 2kb of internal read-only FlashROM for code, 128 bytes of RAM and runs at maximum clock frequency of 24 MHz, which should be more than enough for simple keylogging. 24C1024 would give my device the same memory capacity as the KeyKatcher, is both physically small (DIP8 or smaller package), has a large capacity and is simple to use.

FlashROM is cheaper then EEPROM and comes in even higher densities, but has two big problems: it mostly requires 3.3V or lower supply voltage (it's designed for cell-phones, etc. where power consumption is critical - keyboard supply voltage is 5V) and requires that a block of 128 or more bytes is written at once. Since 2051 only has 128 bytes of RAM, I don't have enough space there for a buffer that large. That means that I would have to either add some external RAM to 2051 or switch to a more powerfull CPU. First option means a bigger circuit and the second one means I would have to work with a CPU I don't know as well as 2051.

I guess 2051 won't be able to perform all of the functions KeyKatcher does (like searching through the stored data, recognizing URLs, etc.), but I don't think that should be a problem. You can always play with the captured keystrokes later once you download them to a PC.

Yesterday I spent five hours moving files from my old 40GB IBM DeathStar hard disk. After 4 years of service in my desktop it developed random bad sectors on the root and home partitions.

I found out that traditional Unix tools I used aren't really convenient for this task. I've read somewhere that the safest way to move a lot of data from one disk to the other is with tar like this:

$ cd /olddir
$ tar -c . | tar -C /newdir -x

This turned out to be a really bad idea. First, tar didn't restore hard links. I used hard links a lot in some hacks a while a ago and the copy of my home directory used far more disk space than the original. Second, tar has a weird idea of when to stop if it encounters a file it can't read. It will kind of continue after the error but won't copy everything it should. It will skip files that are perfectly OK.

The ordinary cp -a turned out to preserve hard links, but had the annoying property to stop at each corrupted file.

In the end the best solution I found was to use mv on each top level directory on the partition. It copied all files, skipped those that could not be read, but didn't delete anything from the source drive.

Today I've stumbled upon this sequence of commands that will allow you to unpack a Debian package even when dpkg isn't functional:

# cd /
# ar x xxx.deb
# tar xzf data.tar.gz

If I only knew this was so simple a few months ago. It would have saved me quite a few hours of unpacking packages on a functional machine and transferring files via a floppy to a crashed one.

Sometimes when I'm bored I apt-get random packages from the Debian archive on my computer. More than once I found some great piece of software this way that I didn't even know they existed (last two packages I remember finding were games Supertux and Viruskiller).

Now it looks like some people at Debian also realized that because of the sheer number of packages many nice packages remain unnoticed. A few days ago debaday.debian.org went on-line, which promises that it will present one Debian package a day. They even offer a RSS feed.

Anyone care to organize a Slovenian Gnome 2.12 release party tomorrow? Somehow I don't feel like traveling all the way to Madrid for this occasion.

OpenUsability is a project that brings Open Source Developers and Usability Experts together.

Quite a lot of free software projects seem to have registered on the site already. I was randomly browsing the site and the first forum post I looked at contained a link this interesting article about prototyping graphical user interfaces on paper. I'll definitely try this approach next time I will be writing a program with a GUI.

Just spent 30 minutes solving another problem that I could fix in 30 seconds if I would only get a proper error message. Whoever wrote OS X printing system obviously thought that error handling is for the weak. I just found out that OS X will simply say "Print job stopped" (in exactly the same way as if someone stopped the job manually) instead of reporting an error if lpd daemon on a remote printer refuses the connection.

After checking all sorts of configuration options in OS X, adding and removing printers I finally run tcpdump to see what exactly is happening on the network. This is what I got:

17:36:01.864288 IP orion.printer > sen.1012: P 1:57(56) ack 5 win 1448 <nop,nop,timestamp 20706385 3602102236>
        0x0000:  4500 006c d2e0 4000 4006 e450 c0a8 0102  E..l..@.@..P....
        0x0010:  c0a8 0108 0203 03f4 983c 26f2 d079 9481  .........<&..y..
        0x0020:  8018 05a8 7bbb 0000 0101 080a 013b f451  ....{........;.Q
        0x0030:  d6b3 b7dc 6f72 696f 6e3a 206c 7064 3a20  ....orion:.lpd:.
        0x0040:  596f 7572 2068 6f73 7420 646f 6573 206e  Your.host.does.n
        0x0050:  6f74 2068 6176 6520 6c69 6e65 2070 7269  ot.have.line.pri
        0x0060:  6e74 6572 2061 6363 6573 730a            nter.access.

Duh. One vi session of the /etc/hosts.lpd later, I could print again from my Powerbook, but how did OS X understand this as "Print job stopped" is beyond my comprehension.

The following piece of code has one simple syntax error on the first line:

287 for(x=0;x<XMAX/2+!;x++) {
288        for(y=0;y<YMAX/2+1;y++) {
289                for(z=ZMAX/2;z<ZMAX+1;z++) {
290                        space[x][y][z]=0;
291                }
292        }
293 }

When you try to compile it with GCC 4.0, you get this torrent of very informative error messages that point you to all other lines in the source file except the one that has the error. GCC obviously wants to make sure that the line with the error is the very last line of code that you will check.

lines.c:291: warning: control reaches end of non-void function
lines.c: At top level:
lines.c:324: warning: type defaults to ‘int’ in declaration of ‘id’
lines.c:324: error: initializer element is not constant
lines.c:324: warning: data definition has no type or storage class
lines.c:329: error: syntax error before ‘for’
lines.c:338: warning: type defaults to ‘int’ in declaration of ‘black’
lines.c:338: error: initializer element is not constant
lines.c:338: warning: data definition has no type or storage class
lines.c:340: error: syntax error before numeric constant
lines.c:340: warning: type defaults to ‘int’ in declaration of ‘g2_set_line_widt h’
lines.c:340: error: conflicting types for ‘g2_set_line_width’
/usr/include/g2.h:121: error: previous declaration of ‘g2_set_line_width’ was he re
lines.c:340: warning: data definition has no type or storage class
lines.c:342: warning: type defaults to ‘int’ in declaration of ‘s_draw’
lines.c:342: warning: parameter names (without types) in function declaration
lines.c:342: error: conflicting types for ‘s_draw’
lines.c:206: error: previous definition of ‘s_draw’ was here
lines.c:342: warning: data definition has no type or storage class
lines.c:344: warning: type defaults to ‘int’ in declaration of ‘g2_close’
lines.c:344: warning: parameter names (without types) in function declaration
lines.c:344: error: conflicting types for ‘g2_close’
/usr/include/g2.h:107: error: previous declaration of ‘g2_close’ was here
lines.c:344: warning: data definition has no type or storage class
lines.c:345: error: syntax error before string constant
lines.c:345: warning: type defaults to ‘int’ in declaration of ‘printf’
lines.c:345: error: conflicting types for ‘printf’
lines.c:345: note: a parameter list with an ellipsis can’t match an empty parame ter name list declaration
lines.c:345: warning: data definition has no type or storage class

Today I've found this review of cluster-oriented Linux distributions by a magazine called PC Quest.

They compared ClusterKnoppix, ParallelKnoppix and Tablix on Morphix. Unfortunately it seems that the authors of the article confused Tablix (software for solving timetabling problems) and Tablix on Morphix (slightly modified Morphix GNU/Linux distribution).

You can download the source code for ATC Advanced here.

If you want to try it, please keep in mind that most of this was written between 1am and 3am, so expect lots of weird code and strange error messages.

To compile, first edit common/pathnames.h and correct the hardcoded paths there (they should point to the location where you extracted the tar file). Then do a make clean && make. I haven't tried make install yet and I don't know what it does. Just run the binaries from the source tree for now.

To play you first have to start the server. Run it like this: server/atca_server -g OHare. You can replace "OHare" with any other game that is available in the games/ directory. After the server is running, start one or more clients like this: client/atca_client. Each client will ask you for your nickname and the IP of the server. After a few moments you should see the radar display.

All clients see the same radar screen. The only difference are the colors of the airplanes. Unassigned new airplanes are red. Green airplanes are assigned to you. White airplanes are assigned to other clients or are ignored (you can ignore an airplane by using the "i" command). You can assign an unassigned airplane by using the "p" command. Once the airplane has been assigned to someone, it can not be reassigned.

Please do not send bug reports unless you have a patch.

Here you can see the server running in the upper left window. I've removed all functions that were drawing the radar screen from the server code. Maybe I'll implement some kind of a command line in the future. Currently the server only outputs some status messages to the terminal.

Other three windows on the above screen shot are clients connected to the server. The lower left client just ordered plane "a" to change heading to 0 degrees. The upper right client ordered the same plane to turn to 90 degrees and the last client ordered airplane "b" to climb to 9000 feet.

I'm using the Simple Network Layer library for networking. It is very simple to use (contains only 4 functions) and as far as I can see it works perfectly. The only problem I've found is that I can't use telnet to directly talk to the server (the library returns "buffer allocation error") even though the network protocol I use is plain ASCII.

This is a screenshot of the first kind-of-working version of my multiplayer ATC hack.

ATC (Air Traffic Controller) is a simple game by Ed James which lets you try your hand at the nerve wracking duties of the air traffic controller without endangering the lives of millions of travelers each year. The original only supports one player and uses simple ncurses-based monochrome ASCII graphics.

Loopus and I got the idea at What The Hack to make a multiplayer version that could be played over the internet and perhaps use SDL for better graphics.

In the screenshot above, the top window is running the server. Any number of clients (one can be seen running in the bottom window) can then connect to the server and they can all send commands to all airplanes on the radar screen.

I'll release the source code as soon as I'll get this even remotely playable. I still have problems with sending commands over the network connection (the game uses a quite interesting state machine for entering commands for airplanes).

I'll post some more details tommorow.

If your tcpdump isn't showing any packets when you know it should be (like, you can ping another machine, but tcpdump doesn't show any packets on the wire), then the problem can be that it can't get a reverse DNS look up for the IPs in the packets it is receiving. You can use the -n option to turn off reverse lookups and solve this problem (or better, fix your DNS configuration).