Wi-Fi monitoring

15.04.2014 17:25

Ever since we setup the wireless testbed in Logatec I wanted to use VESNA spectrum sensors to monitor Wi-Fi and other unlicensed radio usage in the 2.4 GHz band.

Unfortunately, it turned out that the low-power mesh network is way too slow to do any kind of real-time transfers. It was designed for reading battery-powered temperature sensors every once in a while, not streaming radio spectrum data. The 2.4 GHz CC2500-based radio generates around 2 kB per second. That maybe doesn't sound like much these days, but in ideal circumstances a raw data stream from one radio alone more than saturates the mesh, much less 50 of them.

There is a wired Ethernet interface in the works that should take care of the limited bandwidth problem. Meanwhile, motivated by rants about unusable Wi-Fi in some parts of Ljubljana, I improvised and made a pair of stand-alone devices that simply record spectrum data to a SD card.

Stand-alone VESNA RF spectrum sensors.

Here are two particularly colorful visualizations of measurements I took from a balcony in a residential district. They show changes in a two-dimensional histogram of RF power samples over time. This display is similar to the persistence mode on expensive spectrum analyzers (it only becomes feasible on this hardware though when you have relatively long-term measurements).

Each individual sample recorded the total received power, averaged over 1 ms, in a 400 kHz wide channel. Samples were taken at 255 different central frequencies, continuously covering the band between 2.4 GHz and 2.5 GHz. Color on the picture shows how often a sample with that frequency (X axis) and power (Y axis) was encountered.

Following is a recording of a little more than one day, compressed down to 40 seconds:

A day in Wi-Fi

(Click to watch A day in Wi-Fi video)

You can see here several IEEE 802.11 networks. The nice thing about this kind of monitoring is that you can detect access points as well as devices that connect to them. Devices using 802.11b standard with direct sequence spread spectrum modulation have a slightly rounded footprint. Newer and faster 802.11g/n standards with OFDM leave a more sharply square shape.

Some narrow-band transmissions correspond to wireless keyboards and mice. Others I haven't identified yet. Regularly spaced bumps in the noise floor are due to internal interference from the receiver.

Similarly here is the complete week. Each video frame comprises of 5070 scans of the spectrum.

A week in Wi-Fi

(Click to watch A week in Wi-Fi video)

Posted by Tomaž | Categories: Life | Comments »

Thoughts on Bullet Journal

12.04.2014 20:04

Somewhere around November last year Jure introduced me to Bullet journal. Despite its chilling name, it's simply a set of recommendations on how to organize day-to-day notes and to-do lists in a plain old notebook. I gave it a try and experimented a bit with the way I put things down on paper. Here are some thoughts on the topic after a few months.

I've always kept a notebook with me. Before I started using the Bullet journal, I usually went through one 80 page notebook per year. Now I use up around 20 pages per month.

I don't think I write that much more than before though. I now leave much more space around notes for later additions and I typically start a new topic on a new page. This means that notes are less crowded and more readable after a while, but also sometimes a page will remain three-quarters empty if I don't return to some particular thought.

Previously I did a lot of random notes on various scraps of paper and printouts which invariably accumulated on my desk until they got lost or thrown away. I now tend to do all notes directly into the notebook. Sometimes with pencil if I predict a lot of corrections, but usually with ink. Annotated printouts get taped between the pages of the notebook so they don't get lost.

Every once in a while I tend to do a brain-dump page with lots of assorted tasks that sometimes pile up. But normally I keep notes and tasks organized under a common heading that spans one, two or three pages.

Monthly index page from my Bullet journal.

The biggest improvement that came from Bullet journal are the indexes. I number the pages and I write out an index page once per month. I group individual topics in the index by projects. I still keep dates in the margin of pages. I don't keep per-project indexes, but I plan to make a yearly index of projects. Page numbers don't restart when starting a new physical notebook.

I think indexes are really what made the notebook read-write and not only write-only. It used to be that in my old notebooks I only kept going back to a few pages with the most important notes or recipes that I had to keep looking up. Now I find myself daily browsing back to read a thought I have written down a month ago.

Using square boxes for marking to-do items is also a very good idea. I used to mark to-do items with arrows, but boxes are much more visually distinct and allow for quickly scanning the page for un-checked items. I don't use all of the marks described in the Bullet journal tutorial: I either check the box to mark it done, cross over the line to mark it not relevant or draw an arrow over the box to mark that the item has been moved to another page. This last one is fairly rare, since it's easy to look up past unfinished tasks.

Contrary to the Bullet journal proper, most of my notes are not organized by bullets. They tend to be a mix of sketches, bits of text, diagrams, and calculations. Often my notes start in the paper notebook and continue into a IPython Notebook or some other digital file.

For now I usually write down which digital file is connected to the paper notes and vice versa, add a comment in the file pointing to a paper page number. I try to keep duplication to a minimum, but between the notebook, digital files and project documentation which has to be shared or filed separately, there is necessarily some overlap.

Posted by Tomaž | Categories: Life | Comments »

CREW-TV trials

31.03.2014 21:09

The European FP7 CREW project I'm working on at Jožef Stefan Institute had a few open calls in the past two years. Anyone with an interest in one of our testbeds could apply and, if it passed a review, was given funds and support from the CREW consortium for running their experiment on our hardware.

One of the chosen experiments in the second open call was CREW-TV by Instituto de Telecomunicações from Portugal. They proposed to use our spectrum sensing network to dynamically update a database of occupied radio channels. They would then test this database by setting up an experimental video broadcasting system with dynamic spectrum access (they have an interactive demo you can click around).

The CREW-TV experiment ended this past week when experimenters from Portugal, a colleague from the Institute and myself performed an actual trial of the system in the field and took some spectrum measurements. You could see us around Logatec in a van full of laptops, instruments and a few antennas sticking out.

Van with measurement equipment and an antenna for CREW-TV trial.

Considering the amount of effort that went into this experiment from my part, the conclusions were disappointing to say the least.

The trial itself has been delayed for a month and missed an European Commission deadline because our testbed was damaged by ice. With local services still busy repairing more critical infrastructure, it was sometimes difficult to get the necessary support. Running an out-door testbed on public infrastructure has a hidden cost that shows itself in such situations. Even though people in Logatec have been most supportive of our activities, a lot of time and effort was necessary to coordinate everything across different organizations.

Discone antenna for CREW-TV trial base station.

I also learned a few hard lessons in organizing an event like this. I overestimated the amount of effort that was put into this experiment by our partners from Portugal. I should have insisted that we double check the list of required field equipment after they arrived. I should have insisted that they test their system first in the lab before attempting to set it up in the field, regardless of how confident they were. I should have insisted that we stick to reasonable working hours.

If my colleague and I would have spent two days waiting for them to debug their system at a warm desk, the trial would be a much more pleasant experience for everyone involved. Instead we were all freezing in the rain, attempted to download hundreds of megabytes of software through slow and costly mobile connections and had to mess with equipment at dangerous heights during the night.

I understand there are sometimes cultural differences and a language barrier, but at one point the hospitality must end.

Measuring DVB-T spectrum and bit error rate.

A year and a half ago our group organized a meeting for the CREW project and hosted some experiments at the then brand-new testbed. Not everything was working yet and a lot of things had to be hastily prepared in the last minute. I felt bad then because I thought we didn't show ourselves in a good light to our visitors. I guess I now find myself on a different side of a similar situation.

In the end, out of the mess of notes and spectrum measurements I have, there might be some useful data that could be used to evaluate the performance of VESNA spectrum sensors. So far some of it points at some shortcomings in detection of packet based transmissions. That would be interesting to investigate further in the lab. Other measurements are too contradictory to be of much use.

Posted by Tomaž | Categories: Life | Comments »

Concentrated Python cuteness

20.03.2014 19:45

Yesterday, I wrote the following piece of Python code to answer a question asked on IRC by Jure. It was more of a joke than a serious suggestion.

def list_pages(page_num, cur_page, n):
	k = sorted([	0, 2*n+1, 
			cur_page-(n+1), cur_page+n, 
			page_num-(2*n+1), page_num])
	return range(1, page_num+1)[k[1]:k[-2]]

What it does is take a list of page numbers from a total of page_num pages. If possible, the list is centered on cur_page and includes n pages in front and back. At the edges, it still returns the list of the same length. For example:

list_pages(10, 1, 2)  = [1, 2, 3, 4, 5]
list_pages(10, 2, 2)  = [1, 2, 3, 4, 5]
list_pages(10, 3, 2)  = [1, 2, 3, 4, 5]
list_pages(10, 4, 2)  = [2, 3, 4, 5, 6]
list_pages(10, 5, 2)  = [3, 4, 5, 6, 7]
list_pages(10, 6, 2)  = [4, 5, 6, 7, 8]
list_pages(10, 7, 2)  = [5, 6, 7, 8, 9]
list_pages(10, 8, 2)  = [6, 7, 8, 9, 10]
list_pages(10, 9, 2)  = [6, 7, 8, 9, 10]
list_pages(10, 10, 2) = [6, 7, 8, 9, 10]

This is likely one of the most incomprehensible pieces of Python code I have ever written. The idea came from Python's cutest clamp function. It's a kind of cleverness I never want to see in a serious product.

The version Jure ended up using took around 30 lines of code to accomplish the same thing. However even with descriptive variable names it's not immediately obvious to me what it does or how it does it. If I would come across it without any additional comments, it would probably take some time and a few tests to see what is its purpose.

I failed to come up with a version that would be self-explanatory.

Perhaps comprehensive code documentation is dead these days, but I think this is one example where an English sentence can be much more expressive than code.

Update: In case anyone is seriously considering using Python code above, please be warned that it does not work for a general case.

Posted by Tomaž | Categories: Code | Comments »

RGB LED versus RGB sensor

15.03.2014 12:43

Recently two somewhat related things met on my desk:

  • A self-cycling RGB LED extracted from a nose of a plastic deer, a kitschy New Year ornament and
  • a VESNA sensor board with a TCS3772 RGB sensor.

Naturally, I had to see how they work together.

Self-cycling LED from a New Year ornament.

I have no idea what the LED part is. It's simply connected in series with a resistor to a lithium button cell. When voltage is applied it slowly cycles through a range of colors in an apparently random fashion. SparkFun sells something that looks similar.

TCS3772 sensor on a VESNA expansion board.

TCS3772 is a surprisingly small (around 2 mm) component. It is the 6 pin package in the center of the photo above. It connects through I2C and reports values for red, green and blue light. It also has a sensor without a filter for over-all brightness and can do proximity detection when used together with an IR LED.

I used the default VESNA driver for it and it does not specify how these raw values translate to a physical unit. I haven't bothered to dig into the datasheet to find out.

Self-cycling LED measurements using TCS3772 sensor.

This is what I got when I touched the sensor to the LED in a dark room and recorded its readings in a infinite loop.

The LED sequence repeats in around 30 seconds, which surprised me a bit. When I was just watching it, it looked random or at least with a much longer sequence.

I'm not sure why there is such a big difference in the amplitude of different colors. All three colors look equally bright by eye. It might be because the wavelengths of the LEDs are not aligned well with the pass bands of the color filters on the sensor.

It's also obvious that this sensor (at least with the settings used by VESNA's driver) isn't really made for recording fast changes. The values that can be read out through the I2C bus only change every once in 600 ms.

Posted by Tomaž | Categories: Life | Comments »

Running TDA18219HN from external clock

06.03.2014 16:38

I'm finishing up the new design for VESNA's UHF receiver and one feature that sunk the most time was the ability to run two receivers synchronously from the same clock source.

Some of the silicon tuners from NXP, like the TDA18271HD, have a dual-tuner feature. The tuner runs a 16 MHz crystal oscillator that is used in a PLL to generate the local oscillator signal. If the chip is programmed as a master, it outputs a buffered differential sine signal on two XTOUT pins. Another tuner can be programmed as a slave and attached to that clock in place of the crystal resonator:

TDA18271HD dual tuner reception

Image by NXP

Unfortunately the TDA18219HN that I'm using doesn't have that feature. You cannot disable the oscillator and set the pins XTALP and XTALN as passive clock buffer inputs. On the other hand, it still has the clock output, because that is useful to run the demodulator. This made me think it should still be possible to run two tuners synchronously, albeit with some additional circuitry.

Datasheet reveals very little about what is behind the XTAL pins:

TDA18219HN crystal oscillator and PLL

Image by NXP

Note that the capacitors are wired in series with the quartz and not to ground. So one thing is immediately obvious: this chip does not use the common Pierce oscillator topology. That makes sense, since the chip mostly uses balanced signals. The single-ended digital clock signal generated by the common oscillator variant would be of little use. It also means that running this chip from an external clock is not a simple matter of finding which crystal pin is the input to the integrated CMOS inverter.

After some browsing of the literature I didn't find any quartz oscillator topologies that would require capacitors wired like that. In a number of experiments with a signal generator, oscilloscope and a pile of test circuits I came up with the following model:

Model of circuit behind TDA18219HN XTAL pins.

XTAL pins are inputs to a differential buffer B. You can produce the correct differential clock even by driving just one pin, although analog performance is reduced because twice the required input level causes saturation.

Pins are also connected to circuits C and D that provide power to a ringing quartz crystal. Circuits on both pins are independent: driving one pin does not cause any change on the other one. The circuit is very prone to oscillations even without a quartz attached and I wasn't able to measure its input impedance directly. From some indirect measurements and calculations it seems that it must lay on this circular path in the complex plane:

Possible values for input impedance of XTAL pins.

Since the circuit compensates for the losses in the oscillator, the real part of the impedance must be negative. The imaginary part is positive, so it exhibits an inductive reactance (at 16 MHz it's equivalent to around 6.6 μH). This is a bit weird, since quartz resonators typically also behave like inductors in a circuit and the inputs should compensate for that.

The final verdict is that it should be possible to drive XTAL inputs from respective XTOUT outputs, provided there's an AC-coupled voltage follower between them. The follower must be powerful enough to drive the relative low impedance of the XTAL inputs. The power required is non-negligible - my worst case estimate from the impedance plot is 10 mW, which seems a bit high considering this is far above the microwatt levels quartz crystals typically endure.

In the end, I can't really test this until I have a prototype circuit on my desk because any stray capacitance from cables ruins the result. I also don't know how much of a problem phase shift between the receivers will be. If it turns out that it won't work at all, scraping this feature won't be too big a loss anyway.

Posted by Tomaž | Categories: Analog | Comments »

Universe in a cup

01.03.2014 16:42

This topic came up in an IRC channel the other day. How does the universe compare to a cup of tea? How long after the big bang did it take the universe to cool through the temperatures, characteristic of the popular hot beverage?

Fortunately both tea and the early universe are fairly well understood these days.

In the case of tea, the cup cools down approximately exponentially. Time constant depends on the thermal resistance between the drink and the ambient, its mass and specific heat. Fortunately, a dedicated research unit provides empirical data on the subject.

As far as the universe is concerned, that part of its cooling is described by the Friedmann equation for the matter-dominated era.

Cooling of a cup of tea and the universe.

As you can see from the graph above, the universe took slightly longer than your five o'clock tea to cool down.

However, unlike the cup of tea, the universe at that age wasn't terribly interesting. All the fun sub-atomic business had finished long ago and darkness was filled more or less evenly with neutral hydrogen and helium.

It will take more than 100 million years for the first stars to light up and another 10-something billion years before an odd biped gets the idea of throwing a fistful of leaves into boiling water.

Posted by Tomaž | Categories: Ideas | Comments »

Elektro Ljubljana power outages

22.02.2014 17:50

Today is Open Data Day. Unfortunately our little local group didn't have an organized event this year, so I thought I would contribute by releasing a little project of my own.

During the glaze ice disaster this month I've been collecting data from Elektro Ljubljana to share with family and friends living on the affected areas and keep them up-to-date with the situation.

Elektro Ljubljana is one of the larger distributors of electrical energy in Slovenia. It maintains the distribution infrastructure in the central and southern regions of the country, including our capital city Ljubljana. They cover 36% of the population according to their website. From the beginning of the crisis they have been publishing reports on the state of their infrastructure several times per day. They continue to do so, since emergency crews still haven't been able to reach some damaged parts of their network.

Distribution network of Elektro Ljubljana

Image by Elektro Ljubljana d.d.

I've downloaded these hand-written reports and converted them into machine-readable JSON format through a hastily written Python scraper (not proud of parsing HTML with regular expressions, but I've been in a hurry). The data starts at the first report on January 31 and continues to the latest report published today.

You can get the extracted data on GitHub as well as scripts that have been used to extract it:

$ git clone https://github.com/avian2/elektro-ljubljana-outages.git

The following graph shows the number of clients without electrical power (red), affected transformer stations in the whole network (green) and affected transformer stations in the Logatec region (where, in addition to my family, also the Log-a-tec network resides):

Elektro Ljubljana outages due to glaze ice

The animation below shows the development of the outages over time. The color of each region shows the number of transformer stations without power. It would be better if the percentage of affected stations would be shown, but unfortunately I do not have data on the total number of stations in each region.

Posted by Tomaž | Categories: Life | Comments »

Story of ice and more ice

20.02.2014 21:36

As you might have heard, the beginning of February has been kind of crazy regarding weather.

I spent the last week of January with my parents in Austrian alps, enjoying a vacation away from the grid. During the days when we were planning to travel back home more than 2 meters of snow fell in the south of the country. This record snow-fall was too much even for the industrious Austrian road services and all connections with Slovenia were closed. We were lucky to be able to wait it out a warm place.

When the Austrian roads cleared on Saturday we headed home. Unfortunately, the worst was still waiting for us on the other side of the border. When we arrived at my parents', the country was covered in perhaps half-meter snow and encased in an inch or so of ice. Trees were already falling due to heavy glaze ice and we narrowly escaped a trunk falling on the road. It took picks and dirt shovels and heavy work until nightfall to dig ourselves through the snow and ice on the driveway to the house.

That evening first the cable TV went dark and soon after that the power grid. The night was something straight out of a nightmare. It wasn't a storm. Just a persistent slow drizzle freezing on everything it touched. There were constant flashes of incredibly bright blue-green light reflected from the overcast sky. Some close, some far away, from shorted transmission lines. The neighbor's house alarm was triggered by the loss of power and then died down. After that the only thing that could be heard from the dark were loud cracks from trees splintering under the weight of ice.

The next morning there was hardly a tree still standing on my parents' yard. The ice was still thickening and wouldn't clear for a week. Most of the country was in state of emergency due to closed roads and destroyed electrical transmission lines. Logatec, city my parents live in, was among the ones that were hit the hardest by ice.

Glaze ice in Logatec

It's a scary feeling to be in a situation like that. The local supermarket opened running on an emergency generator. We were let in through the staff entrance in the back. It was dark, with only the cash registers powered up. The refrigerators with frozen goods were locked. A customer was yelling loudly at a lady behind the counter how outraged he was that he couldn't get fresh bread. I kept thinking whether I should buy more food than usual in case this situation will last more than a few days and whether that would look weird.

It's amazing to what degree we take electric power for granted today. Water and heat run out surprisingly fast without it. Checking for announcements on local government web site becomes highly nontrivial when the battery in your mobile phone runs down and later when the generator at the GSM base station runs out of fuel. Surprisingly, the only service that was working almost without interruptions, at least at my parents' house, was the telephone land-line.

In the age of electric stoves and kilowatt hair dryers mere 2 kW from a generator is preciously little. Even when you think you have everything prepared for an emergency there are still surprises. My father bought a generator exactly for a contingency like this. He serviced it yearly and kept it in good order, but it still took a few overhauls until it was running smoothly under the load of the central heating system. Many people found out that their generators were out of oil or fuel after a few years of gathering dust.

The server handling this website, my mail and a few other services was also knocked off-line. I had an automated duplicity backup, which I regularly used to restore an odd file or two. I was confident that I could rely on it. When I tried restoring the full backup however I hit two bugs that made moving my things to other servers far less than trivial.

According to reports at the height of the crisis one-fifth of the country was without power and the distribution infrastructure in the Notranjska region was almost completely destroyed. While a temporary line was established to my parents' neighborhood after a week and a half, there are still frequent day-long outages.

Even though my place in Ljubljana never lost power, I now keep a few spare batteries and a battery powered FM radio in the top drawer. And I have even more respect for people I know that volunteered in local emergency services and did long shifts to help people in need.

Posted by Tomaž | Categories: Life | Comments »

CubieTruck Perl performance

23.01.2014 22:57

Two months ago I bought a CubieTruck, one of the many cheap, bare-bone ARM-based computers that keep popping-up everywhere these days. My idea was to replace the aging x86 server that is running this website with something more power-efficient. So I was looking for a reasonably powerful board with a proper SATA interface and a decent amount of RAM. Raspberry Pi was out of the question, but the latest incarnation of CubieBoard with a dual-core 1 GHz ARM Cortex-A7, 2 GB of RAM, SATA 2.0 and Gigabit Ethernet seemed to fit the bill.

Unfortunately I could not find any reliable benchmarks I could use to estimate how ARM SoCs perform in comparison with my existing setup. So before I decided to migrate I took a while to do some performance tests and get to know this hardware.


The software setup I'm interested in benchmarking is somewhat archaic in these days of Node.js and NoSQL. I'm using Perl 5 with HTML::Template doing most of the heavy lifting (at least according to Devel::NYTProf profiler). Most parts are statically generated and some are dynamic using a handful of Speedy CGI Perl 5 scripts. These are combined into a consistent website you see here with a somewhat convoluted Apache configuration using the threaded worker.

In the following benchmarks I'm comparing:

  • An AMD Duron at 700 MHz, 1.2 GB RAM running stock x86 Debian Squeeze. Root filesystem is mounted from an IDE hard drive.
  • A CubieTruck A20 running armhf Debian Wheezy and the kernel supplied for the CubieTruck Ubuntu Server installation. Root filesystem is mounted from an SD card.

Both machines were connected through a 100 Mb/s Ethernet switch to a laptop which was running the remote end of the benchmarks.

First, to see how fast the static part of the web site is generated, I ran the full (single threaded) HTML rebuild. I measured the required user space CPU time with the time utility. This is the fastest run of three on each machine:

AMD DuronCubieTruck
CPU time to rebuild static pages45.3 s61.8 s

Then, to check if network was operating at the bit rate I thought it was, I ran iperf to measure TCP throughput between the server and the laptop:

AMD DuronCubieTruck
iperf throughput test94.0 Mb/s94.5 Mb/s

Finally, I ran a suite of tests using the Apache benchmarking tool. I measured how many requests per minute a server can handle for different types of content and different number of concurrent requests. Numbers in parentheses show size of HTTP body (without headers).

CubieTruck requests per second for a static HTML page.

CubieTruck requests per second for a dynamic HTML page.

CubieTruck requests per second for an image.

CubieTruck requests per second for API call.

The site rebuild is somewhat disappointingly almost one-third slower than on a 10 year old PC. However the single threaded Apache performance is on par with it. In the case of more concurrent users the CubieTruck of course has an advantage because of an additional CPU core. Actually in both cases with static content CubieTruck managed to saturate the line when there was more than one concurrent request.

I tried to make these tests in a way that the slow SD card in the CubieTruck would minimally affect their outcome. All of data should fit into the buffer cache, which is why in the first test I only took into account the fastest run and only user space CPU time. However I now suspect that the SD card still affected the numbers somehow (the rebuild operation is the heaviest of the tests regarding filesystem I/O). I don't know for sure how kernel computes the time returned by the time utility.

These results are good enough that I can't dismiss CubieTruck based on performance. If a proper SATA drive wouldn't speed it up, I could probably parallelize the build process with not much work. That should cut down on time if it's really Perl performance on ARM that is slowing it down. On the other hand I'm having some other concerns about using CubieTruck as a personal server so I'm not completely decided yet about putting it on my rack.

Posted by Tomaž | Categories: Digital | Comments »