EuroPython 2014 wrap-up

29.07.2014 14:08

I spent the last week at the EuroPython conference. Since my last visit three years ago, the largest European meeting of the Python community moved from Florence in Italy to the familiar Berlin's Congress Center in Alexanderplatz. Last 7 days were chock-full of talks about everything related to Python, friendly conversations with fellow developers and hacking on this and that odd part of the free software ecosystem. So much so actually that it was a welcome change to spend a day outside, winding down in one of Berlin's numerous parks.

EuroPython 2014 schedule poster in the basement of the BCC.

It was an odd feeling to be back in the BCC. So far I knew it only by the Chaos Communication Congresses it hosted before they moved back to Hamburg. I was half-expecting to see the signature Fairy dust rocket and black Datenpirat flags flying in the front. Instead, part of the front yard was converted to a pleasant outdoor lounge area with an ice cream stand while a bunch of sponsor booths took place of the hackcenter.

Traces of the German hacker scene and the community around the CCC could still be easily seen though. The ubiquitous Club Mate was served on each floor. Blinking All Colors Are Beautiful and Mate Light installations were placed on two floors of the conference center. A few speakers reminisced about the congresses and the CCC Video Operations Center took care of the video recordings with the efficiency we are used to from the Congresses. You could even hear a public request to use more bandwidth now and then.

Constanze Kurz talking about surveillance industry.

There were several talks with a surprisingly political topic for a conference centered around a programming language. I felt odd listening to the opening keynote from Constanze Kurz about one year of Snowden revelations and the involvement of big Internet companies while those same companies were hunting for new employees just outside the lecture hall. Still, I don't think it hurts to remind people that even purely technical work can have consequences in the society.

From the keynotes, it's also worth mentioning Emily Bache and her views on test driven development and the history and future of NumPy development by Travis Oliphant.

Even though I'm not doing as much Python development as I used to in my previous job, there were still plenty of interesting presentations. In the context of scientific computing in Python, I found the talks about the GR plotting framework, simulations with SimPy and probabilistic programming most interesting. Some interesting scientific projects relevant to my work were presented in the poster session as well, although I had problems later finding them on the web to learn more about them. Especially, the work on propagating measurement errors in numerical calculations done by Robert Froeschl at CERN caught my eye.

Park in front of the BCC for EuroPython attendants.

I also attended the matplotlib training, although I was a bit disappointed with it. I was hoping to learn more about advanced usage of this Python library I use daily in my work. I enjoyed the historical examples of data visualization and general advice on how to present graphs in a readable way. However most of the technical details behind plotting and data manipulation weren't new to me and we ran out of time before digging into more advanced topics. I guess it was my fault though, since I now see that the training has a novice tag.

Going away from the topics strictly connected to my work, the talks about automated testing with a simple robot and using Kinect in Python left me with several crazy ideas for potential future projects.

Open Contracting table at EuroPython 2014 sprints.

During the weekend I joined the Open Contracting coding sprint. Jure and I represented our small Open Data group in helping to develop tools for governments to publish information about public tenders and contracts in a standard format. My result from these two days is the jsonmerge library, which deserves a blog post of its own.

In conclusion, I enjoyed EuroPython 2014 immensely. In contrast for instance with 30C3 last year, it was again one of the those events where I got to talk with an unusually large number of people during coffee breaks and social events.

On the other hand, attending it made me realize how much my interests changed after I left Zemanta. I visited Florence during my last days at that job and I was intimately familiar in all nuances of the Python interpreter. There I joined the spring hacking on the CPython distribution. This year however I was more interested in what Python can offer to me as a tool and not that much in developing Python itself.

Posted by Tomaž | Categories: Life | Comments »

Down time

21.07.2014 21:14

You might have noticed that for the past two days or so this website was off-line. The reason for it is a bit curious.

On Friday, 18 July at 17:45, two of my Ethernet switches failed - one in Logatec and one in Ljubljana, separated by around 35 km. They crashed simultaneously at the exactly the same second.

Here are the relevant log entries of two machines connected to them. Both had clocks synchronized via NTP, so the time stamps should be fairly accurate. The machines logged these messages when their Ethernet adapters reported the loss of carrier signal on the cable from their respective switches:

Jul 18 17:45:21 gildedale kernel: [1658547.286187] PHY: sunxi_gmac-0:00 - Link is Down
Jul 18 17:45:21 chandra kernel: [512418.004319] e100 0000:00:0c.0: eth3: NIC Link is Down

These two pieces of equipment were geographically separated and had nothing in common. As far fetched as it seems that they would fail because of a common reason, this appears to be the case here.

Newspapers reported on Friday afternoon that a small explosion occurred at a transformer station in Ljubljana. The distribution company confirms the event, but doesn't share many details. There is also no official source of the exact time, but the first tweet about it appeared at 17:45.

I guess that whatever occurred at the transformer station caused a transient on the power grid that crashed my switches. It must have been fairly short because two other computers that were connected to the same outlet did not reboot or report any problems.

It's curious that this effect threw both of the switches in a state where they didn't reboot, but didn't function either. One of these is a fairly old, low-cost affair that I have seen behave in this way a few times before. The other one is integrated into a Linksys WRT54GL access point that has been fairly reliable so far.

This is the second time this year that something completely unexpected happened with my servers. It's not that I try to run some kind of high-reliability shop, but it is annoying and makes me appreciate the work real system administrators do each day. I guess that's the cost of trying to stay away from the cloud these days.

Posted by Tomaž | Categories: Life | Comments »

Remembrance

08.07.2014 22:06

Back in my childhood I used to spend a lot of time at my grandparent's cottage. It was a wooden house amid a small orchard, just under the top of a hill outside of Ljubljana. My parents would often drive me there in a red Zastava 101 on weekends and holidays. I played with surplus plumbing parts and old tools that were lying around, slept in the attic and had nightmares from having to go to the outhouse after dark.

The cottage was designed and built by my grandfather as a weekend retreat. He was a mechanical engineer who moved to Ljubljana with my grandmother to one of the first apartment complexes in the city, proudly displaying his professional title above the door bell. He was designing turbines for hydroelectric power plants at the heavy machinery manufacturer Litostroj.

When I wasn't wasting potable water or setting things on fire, one of my favorite ways of spending time there was browsing through a book Elektrotehnika v slikah. I could always find it stashed between two planks above my grandfather's bed. It was bound into a hard crimson cover with gold lettering and even back then already had the smell of an old paperback. The title translates to English as Electrical engineering in pictures and it's actually a Slovenian translation of the German original Elektrotechnik in Bildern by Gustav Büscher.

Elektrotehnika v slikah, cover and page 56

I don't know how that book got there. I doubt that my grandfather learned anything new from it, considering it was printed in 1968 and he finished his 4-book, 5-inch thick thesis on designing Francis turbines 6 years earlier.

In any case, I loved to turn its pages. I looked at the pictures and read the short paragraphs until I knew most of it by heart. First few chapters at least, where the book explains the basics like current, voltage and resistance with simple water analogies. There, almost each paragraph had an illustration that was just serious enough not to appear overly childish and funny enough to keep it interesting. The book was, after all, targeted at adults.

Later chapters went on to describe electric machinery and vacuum tubes with more elaborate mechanical analogies. Judging by the tear-and-wear of the pages, I skipped most of those parts back then. Equations were definitely beyond me at that point, and I'm not sure I even bothered my parents asking about them.

Illustration of HE Završnica from Elektrotehnika v slikah.

In the chapter about hydroelectricity, the Završnica power plant is featured on several occasions. Maybe it was there in the original version of the book. Or, more likely, the illustrations were adapted when the book was translated. Regardless, Završnica was the first public power plant in the region, so it definitely deserved the attention. By the time this book was published, it was already more than 50 years old.

For a long time, if someone would ask me, that plant would be the only one I could name. I didn't know where it was, but the name has definitely stuck in my head.

Surge tank of HE Završnica

Even a month ago I wouldn't know where to put it exactly on the map. I was therefore a bit surprised when I found out that the plant is now being converted into a museum and that I will see it on my visit of HE Moste.

I now know that the dam on river Završnica is a short hike from the town of Žirovnica and is surrounded by a pleasant recreational park. I learned that the pressure pipeline goes under the town and discharges into river Sava below. And I also learned that the prominent building on the top of the hill you see when driving on the Gorenjska highway is its surge tank. Even though machinery in the old turbine hall is being cut open for display, the surge tank above still serves its original purpose. From up close it is an impressive monument to early 20th century hydro engineering.

Mechanical frequency meter in HE Završnica

A walk through the museum revealed many sights that were familiar to me from the book, like the mechanical frequency meter above. I don't think I ever saw a real one before.

HE Završnica museum.

What I also learned on the tour is that HE Moste next door was my grandfather's first big design project at Litostroj.

The original bronze Francis turbines dimensioned by him and cast by Litostroj have long been replaced, damaged by cavitation and sand particles carried from the mountains by river Sava. However one of them is now proudly displayed on a plinth in the park in front of the plant.

After HE Moste, my grandfather went on to design power plants throughout the former Yugoslavia and abroad. His design bureau was involved in projects all over the world. He kept a globe in his office marked with flight paths of his intercontinental flights that circled the Earth.

Bronze Francis turbine on display at HE Moste

The visit to HE Moste and HE Završnica last month has been as much a technical interest for me as it was a way to remember my grandfather. Without doubt he helped spark my interest for engineering. Either by letting me browse his books or watch him spend his days drawing carefully calculated lines on translucent paper.

He died in 2011 and was designing turbines in his home office for as long as he could hold a pen against a drafting table. His cottage is gone as well and Elektrotehnika v slikah is now standing on a bookshelf in my living room. It was nice to see in person one of more than a hundred power plants he helped design during his career that are still producing electricity.

Posted by Tomaž | Categories: Life | Comments »

Visiting HE Moste

22.06.2014 15:09

When you're driving from central Slovenia towards the mountains, the Gorenjska highway crosses the valley of the Sava river. From the viaduct you can see a high dam wedged between sides of a narrow gorge. Behind it is the water reservoir for the Hidroelektrarna Moste.

Viadukt Moste and HE Moste dam.

I've been watching this dam every time I was traveling on that road. For years I wanted to see it up close and the power plant below it. My wish came true when my mum got a contact at the plant and arranged for a guided tour. So my parents and I drove up to Žirovnica last week and visited the facility.

Hidroelektrarna Moste

To my surprise, turbines and the generator hall of the power plant are actually located quite a bit down river from the dam, not directly underneath it as I expected. Water from river Sava is routed through a tunnel under the nearby town Moste, from which the facility also got its name.

What you see above is the steep railway with which heavy machinery has been lowered into the valley during construction.

Safety valve on the high-pressure pipeline for HE Moste

This safety valve on the high-pressure pipe in the tunnel was the first impression of the scale of engineering of this place.

The valve is designed to operate in a fail-safe manner: in case of an emergency, it uses stored potential energy without the need for any external power. The huge red weight shuts off the water supply if electric power is lost or if sensors detect flooding in the facility. Later on in the generator hall we saw emergency buttons similar to fire alarms that would manually trigger it.

Behind the valve is a huge vertical surge tank embedded in rock. Its purpose is to absorb the kinetic energy of the water in the pipeline which could cause damage if the valve would suddenly close.

Even though around 20 m3 of water were flowing per second through the large pipe, there was surprisingly little sound to be heard.

Flotation device near the turbine of HE Moste

The site is actually home to two power plants:

In 1915 Hidroelektrarna Završnica was built here to become the first public power plant in the area. It used water from the river Završnica which was routed from a reservoir higher up in the mountains and discharged into the river Sava.

Later on, river Sava was dammed in 1952 and Hidroelektrarna Moste built alongside the old power plant. Now HE Završnica is no longer in use and is being slowly converted to a museum. Its water supply however has been routed to one of the turbines in HE Moste and is still used intermittently to generate electricity.

Francis turbine in HE Završnica.

This is one of the three 10 MW Francis turbines currently in operation. In contrast to the quiet flow of water in the pipeline, turbine wells were noisy enough that you had to shout to talk to someone.

While we were visiting, one of the generators had to be stopped because of a problem with one of the oil pumps. Our tour was put on hold while repairs were made. After the pump was fixed, the turbine was spun up, generator synchronized with the grid and control of the power plant handed back to the remote operations center in Ljubljana.

Modern hydraulic turbine speed governor.

This is one of the modern hydraulic turbine speed governors that are currently in use in the power plant.

All aspects of the power plant are remotely monitored using a SCADA system. Status of every temperature sensor and pump can be accessed from a central control room. Apart from electrical parameters they also measure and record structural status of the dam and its surroundings. They have automatic strain gauges and a grid of reference points where geometers regularly check for any unexpected earth movements.

In theory the plant could be operated entirely by remote control, but we were told that the company will continue to keep the site staffed round-the-clock. Problems can be detected and solved faster and cheaper if experienced engineers are around. Otherwise a maintenance team would have to be dispatched from somewhere else, which could lead to more downtime or a small problem growing into a more expensive one.

HE Moste, generator 4

In conclusion, I would like to thank the friendly staff of HE Moste for taking the time for us in a busy day and Mr. Pirjevec for guiding us on this tour. We were shown every nook and cranny of the power plant and he gave us a thorough description of the machinery and answered every question we had.

It was a pleasant and informative way to spend a morning and a welcome step away from all the low-power electronics I deal with each day.

Posted by Tomaž | Categories: Life | Comments »

Seminar on spectrum sensing methods

07.05.2014 17:34

Today I gave a short seminar on spectrum sensing at the Jožef Stefan International Postgraduate School. It contained a quick overview of the topic: what spectrum sensing is, what are the most common methods that are used in the context of dynamic spectrum access and how they can be implemented in practice.

Spectrum sensing methods and implementations title slide.

I couldn't go into much detail in the short time provided, but I've written a paper that goes with the talk and covers a bit more. The version on the School's website has reasonably unreadable formatting, so I'm publishing my original copy here. Of course, you can also download the slides.

When I first attended one of these seminars, I was somewhat surprised by the format. I haven't seen anywhere else that questions from the audience are written on slips of paper that are then passed over to the speaker to read and answer.

I sometimes attend seminars on interesting topics at the Faculty of Mathematics and Physics across the street from IJS. For example, there the Q&A is a normal human discussion between peers. The presentations are around 45 minutes while here I've heard a comment that they should be shortened to only 15 minutes.

I'm not sure about the reasoning behind these choices at IPS, but I can say that I like the common version better. If not for anything else, because I would not be hesitant to invite people that don't need mandatory attendance to join us.

Posted by Tomaž | Categories: Life | Comments »

WebCamp Ljubljana

27.04.2014 20:18

Yesterday I attended WebCamp Ljubljana held at the Faculty of Computer and Information Science. It was expertly organized by Jure and his team of volunteers as the conclusion to the OCWC global conference on open education.

Kiberpipa table at WebCamp Ljubljana 2014

Image by jskrablin

I managed to attend two talks, but otherwise I was mostly hanging around the Kiberpipa table where I was invited to show off some of my hardware projects. VESNA spectrum sensor, an interactive animated pony on an OLED display and a few other tidbits attracted a surprising amount of attention at the otherwise web-development focused event.

I recorded a trace of the 2.4 GHz band during the event to compare it with my earlier measurements at home. As you can see on the visualization below, it looks a bit more lively than my previous recordings that were made in a quiet residential area.

At the conclusion of the conference I made a hurriedly prepared lightning talk about these measurements and how they can help you choose the best channel for a Wi-Fi access point. I proposed to setup a crowd-funding campaign for a cheap spectrum sensor for that sole purpose if there is enough interest. From the feedback I got though I think I failed to present the topic in an understandable way for this audience.

Incidentally, the wireless network at the event was not particularly reliable. Although as I understand, the physical layer was not a problem in this case.

2.4 GHz use at WebCamp Ljubljana 2014

(Click to watch 2.4 GHz use at WebCamp Ljubljana 2014 video)

In general, the event went by in a very positive atmosphere. I think 20 minutes per talk gave it just the right pace for a gathering like this. I enjoyed the presentations about IPv6 and Apache Solr. I talked with a lot of nice people and I was surprised by the gender balance among the speakers and audience. Unfortunately I was too tired from getting up early on a Saturday and the hectic week before to check out the social event in Kiberpipa that followed in the evening.

Posted by Tomaž | Categories: Life | Comments »

Wi-Fi monitoring

15.04.2014 17:25

Ever since we setup the wireless testbed in Logatec I wanted to use VESNA spectrum sensors to monitor Wi-Fi and other unlicensed radio usage in the 2.4 GHz band.

Unfortunately, it turned out that the low-power mesh network is way too slow to do any kind of real-time transfers. It was designed for reading battery-powered temperature sensors every once in a while, not streaming radio spectrum data. The 2.4 GHz CC2500-based radio generates around 2 kB per second. That maybe doesn't sound like much these days, but in ideal circumstances a raw data stream from one radio alone more than saturates the mesh, much less 50 of them.

There is a wired Ethernet interface in the works that should take care of the limited bandwidth problem. Meanwhile, motivated by rants about unusable Wi-Fi in some parts of Ljubljana, I improvised and made a pair of stand-alone devices that simply record spectrum data to a SD card.

Stand-alone VESNA RF spectrum sensors.

Here are two particularly colorful visualizations of measurements I took from a balcony in a residential district. They show changes in a two-dimensional histogram of RF power samples over time. This display is similar to the persistence mode on expensive spectrum analyzers (it only becomes feasible on this hardware though when you have relatively long-term measurements).

Each individual sample recorded the total received power, averaged over 1 ms, in a 400 kHz wide channel. Samples were taken at 255 different central frequencies, continuously covering the band between 2.4 GHz and 2.5 GHz. Color on the picture shows how often a sample with that frequency (X axis) and power (Y axis) was encountered.

Following is a recording of a little more than one day, compressed down to 40 seconds:

A day in Wi-Fi

(Click to watch A day in Wi-Fi video)

You can see here several IEEE 802.11 networks. The nice thing about this kind of monitoring is that you can detect access points as well as devices that connect to them. Devices using 802.11b standard with direct sequence spread spectrum modulation have a slightly rounded footprint. Newer and faster 802.11g/n standards with OFDM leave a more sharply square shape.

Some narrow-band transmissions correspond to wireless keyboards and mice. Others I haven't identified yet. Regularly spaced bumps in the noise floor are due to internal interference from the receiver.

Similarly here is the complete week. Each video frame comprises of 5070 scans of the spectrum.

A week in Wi-Fi

(Click to watch A week in Wi-Fi video)

Posted by Tomaž | Categories: Life | Comments »

Thoughts on Bullet Journal

12.04.2014 20:04

Somewhere around November last year Jure introduced me to Bullet journal. Despite its chilling name, it's simply a set of recommendations on how to organize day-to-day notes and to-do lists in a plain old notebook. I gave it a try and experimented a bit with the way I put things down on paper. Here are some thoughts on the topic after a few months.


I've always kept a notebook with me. Before I started using the Bullet journal, I usually went through one 80 page notebook per year. Now I use up around 20 pages per month.

I don't think I write that much more than before though. I now leave much more space around notes for later additions and I typically start a new topic on a new page. This means that notes are less crowded and more readable after a while, but also sometimes a page will remain three-quarters empty if I don't return to some particular thought.

Previously I did a lot of random notes on various scraps of paper and printouts which invariably accumulated on my desk until they got lost or thrown away. I now tend to do all notes directly into the notebook. Sometimes with pencil if I predict a lot of corrections, but usually with ink. Annotated printouts get taped between the pages of the notebook so they don't get lost.

Every once in a while I tend to do a brain-dump page with lots of assorted tasks that sometimes pile up. But normally I keep notes and tasks organized under a common heading that spans one, two or three pages.

Monthly index page from my Bullet journal.

The biggest improvement that came from Bullet journal are the indexes. I number the pages and I write out an index page once per month. I group individual topics in the index by projects. I still keep dates in the margin of pages. I don't keep per-project indexes, but I plan to make a yearly index of projects. Page numbers don't restart when starting a new physical notebook.

I think indexes are really what made the notebook read-write and not only write-only. It used to be that in my old notebooks I only kept going back to a few pages with the most important notes or recipes that I had to keep looking up. Now I find myself daily browsing back to read a thought I have written down a month ago.


Using square boxes for marking to-do items is also a very good idea. I used to mark to-do items with arrows, but boxes are much more visually distinct and allow for quickly scanning the page for un-checked items. I don't use all of the marks described in the Bullet journal tutorial: I either check the box to mark it done, cross over the line to mark it not relevant or draw an arrow over the box to mark that the item has been moved to another page. This last one is fairly rare, since it's easy to look up past unfinished tasks.

Contrary to the Bullet journal proper, most of my notes are not organized by bullets. They tend to be a mix of sketches, bits of text, diagrams, and calculations. Often my notes start in the paper notebook and continue into a IPython Notebook or some other digital file.

For now I usually write down which digital file is connected to the paper notes and vice versa, add a comment in the file pointing to a paper page number. I try to keep duplication to a minimum, but between the notebook, digital files and project documentation which has to be shared or filed separately, there is necessarily some overlap.

Posted by Tomaž | Categories: Life | Comments »

CREW-TV trials

31.03.2014 21:09

The European FP7 CREW project I'm working on at Jožef Stefan Institute had a few open calls in the past two years. Anyone with an interest in one of our testbeds could apply and, if it passed a review, was given funds and support from the CREW consortium for running their experiment on our hardware.

One of the chosen experiments in the second open call was CREW-TV by Instituto de Telecomunicações from Portugal. They proposed to use our spectrum sensing network to dynamically update a database of occupied radio channels. They would then test this database by setting up an experimental video broadcasting system with dynamic spectrum access (they have an interactive demo you can click around).

The CREW-TV experiment ended this past week when experimenters from Portugal, a colleague from the Institute and myself performed an actual trial of the system in the field and took some spectrum measurements. You could see us around Logatec in a van full of laptops, instruments and a few antennas sticking out.

Van with measurement equipment and an antenna for CREW-TV trial.

Considering the amount of effort that went into this experiment from my part, the conclusions were disappointing to say the least.

The trial itself has been delayed for a month and missed an European Commission deadline because our testbed was damaged by ice. With local services still busy repairing more critical infrastructure, it was sometimes difficult to get the necessary support. Running an out-door testbed on public infrastructure has a hidden cost that shows itself in such situations. Even though people in Logatec have been most supportive of our activities, a lot of time and effort was necessary to coordinate everything across different organizations.

Discone antenna for CREW-TV trial base station.

I also learned a few hard lessons in organizing an event like this. I overestimated the amount of effort that was put into this experiment by our partners from Portugal. I should have insisted that we double check the list of required field equipment after they arrived. I should have insisted that they test their system first in the lab before attempting to set it up in the field, regardless of how confident they were. I should have insisted that we stick to reasonable working hours.

If my colleague and I would have spent two days waiting for them to debug their system at a warm desk, the trial would be a much more pleasant experience for everyone involved. Instead we were all freezing in the rain, attempted to download hundreds of megabytes of software through slow and costly mobile connections and had to mess with equipment at dangerous heights during the night.

I understand there are sometimes cultural differences and a language barrier, but at one point the hospitality must end.

Measuring DVB-T spectrum and bit error rate.

A year and a half ago our group organized a meeting for the CREW project and hosted some experiments at the then brand-new testbed. Not everything was working yet and a lot of things had to be hastily prepared in the last minute. I felt bad then because I thought we didn't show ourselves in a good light to our visitors. I guess I now find myself on a different side of a similar situation.

In the end, out of the mess of notes and spectrum measurements I have, there might be some useful data that could be used to evaluate the performance of VESNA spectrum sensors. So far some of it points at some shortcomings in detection of packet based transmissions. That would be interesting to investigate further in the lab. Other measurements are too contradictory to be of much use.

Posted by Tomaž | Categories: Life | Comments »

RGB LED versus RGB sensor

15.03.2014 12:43

Recently two somewhat related things met on my desk:

  • A self-cycling RGB LED extracted from a nose of a plastic deer, a kitschy New Year ornament and
  • a VESNA sensor board with a TCS3772 RGB sensor.

Naturally, I had to see how they work together.

Self-cycling LED from a New Year ornament.

I have no idea what the LED part is. It's simply connected in series with a resistor to a lithium button cell. When voltage is applied it slowly cycles through a range of colors in an apparently random fashion. SparkFun sells something that looks similar.

TCS3772 sensor on a VESNA expansion board.

TCS3772 is a surprisingly small (around 2 mm) component. It is the 6 pin package in the center of the photo above. It connects through I2C and reports values for red, green and blue light. It also has a sensor without a filter for over-all brightness and can do proximity detection when used together with an IR LED.

I used the default VESNA driver for it and it does not specify how these raw values translate to a physical unit. I haven't bothered to dig into the datasheet to find out.

Self-cycling LED measurements using TCS3772 sensor.

This is what I got when I touched the sensor to the LED in a dark room and recorded its readings in a infinite loop.

The LED sequence repeats in around 30 seconds, which surprised me a bit. When I was just watching it, it looked random or at least with a much longer sequence.

I'm not sure why there is such a big difference in the amplitude of different colors. All three colors look equally bright by eye. It might be because the wavelengths of the LEDs are not aligned well with the pass bands of the color filters on the sensor.

It's also obvious that this sensor (at least with the settings used by VESNA's driver) isn't really made for recording fast changes. The values that can be read out through the I2C bus only change every once in 600 ms.

Posted by Tomaž | Categories: Life | Comments »

Elektro Ljubljana power outages

22.02.2014 17:50

Today is Open Data Day. Unfortunately our little local group didn't have an organized event this year, so I thought I would contribute by releasing a little project of my own.

During the glaze ice disaster this month I've been collecting data from Elektro Ljubljana to share with family and friends living on the affected areas and keep them up-to-date with the situation.

Elektro Ljubljana is one of the larger distributors of electrical energy in Slovenia. It maintains the distribution infrastructure in the central and southern regions of the country, including our capital city Ljubljana. They cover 36% of the population according to their website. From the beginning of the crisis they have been publishing reports on the state of their infrastructure several times per day. They continue to do so, since emergency crews still haven't been able to reach some damaged parts of their network.

Distribution network of Elektro Ljubljana

Image by Elektro Ljubljana d.d.

I've downloaded these hand-written reports and converted them into machine-readable JSON format through a hastily written Python scraper (not proud of parsing HTML with regular expressions, but I've been in a hurry). The data starts at the first report on January 31 and continues to the latest report published today.

You can get the extracted data on GitHub as well as scripts that have been used to extract it:

$ git clone https://github.com/avian2/elektro-ljubljana-outages.git

The following graph shows the number of clients without electrical power (red), affected transformer stations in the whole network (green) and affected transformer stations in the Logatec region (where, in addition to my family, also the Log-a-tec network resides):

Elektro Ljubljana outages due to glaze ice

The animation below shows the development of the outages over time. The color of each region shows the number of transformer stations without power. It would be better if the percentage of affected stations would be shown, but unfortunately I do not have data on the total number of stations in each region.

Posted by Tomaž | Categories: Life | Comments »

Story of ice and more ice

20.02.2014 21:36

As you might have heard, the beginning of February has been kind of crazy regarding weather.

I spent the last week of January with my parents in Austrian alps, enjoying a vacation away from the grid. During the days when we were planning to travel back home more than 2 meters of snow fell in the south of the country. This record snow-fall was too much even for the industrious Austrian road services and all connections with Slovenia were closed. We were lucky to be able to wait it out a warm place.

When the Austrian roads cleared on Saturday we headed home. Unfortunately, the worst was still waiting for us on the other side of the border. When we arrived at my parents', the country was covered in perhaps half-meter snow and encased in an inch or so of ice. Trees were already falling due to heavy glaze ice and we narrowly escaped a trunk falling on the road. It took picks and dirt shovels and heavy work until nightfall to dig ourselves through the snow and ice on the driveway to the house.

That evening first the cable TV went dark and soon after that the power grid. The night was something straight out of a nightmare. It wasn't a storm. Just a persistent slow drizzle freezing on everything it touched. There were constant flashes of incredibly bright blue-green light reflected from the overcast sky. Some close, some far away, from shorted transmission lines. The neighbor's house alarm was triggered by the loss of power and then died down. After that the only thing that could be heard from the dark were loud cracks from trees splintering under the weight of ice.

The next morning there was hardly a tree still standing on my parents' yard. The ice was still thickening and wouldn't clear for a week. Most of the country was in state of emergency due to closed roads and destroyed electrical transmission lines. Logatec, city my parents live in, was among the ones that were hit the hardest by ice.

Glaze ice in Logatec

It's a scary feeling to be in a situation like that. The local supermarket opened running on an emergency generator. We were let in through the staff entrance in the back. It was dark, with only the cash registers powered up. The refrigerators with frozen goods were locked. A customer was yelling loudly at a lady behind the counter how outraged he was that he couldn't get fresh bread. I kept thinking whether I should buy more food than usual in case this situation will last more than a few days and whether that would look weird.

It's amazing to what degree we take electric power for granted today. Water and heat run out surprisingly fast without it. Checking for announcements on local government web site becomes highly nontrivial when the battery in your mobile phone runs down and later when the generator at the GSM base station runs out of fuel. Surprisingly, the only service that was working almost without interruptions, at least at my parents' house, was the telephone land-line.

In the age of electric stoves and kilowatt hair dryers mere 2 kW from a generator is preciously little. Even when you think you have everything prepared for an emergency there are still surprises. My father bought a generator exactly for a contingency like this. He serviced it yearly and kept it in good order, but it still took a few overhauls until it was running smoothly under the load of the central heating system. Many people found out that their generators were out of oil or fuel after a few years of gathering dust.

The server handling this website, my mail and a few other services was also knocked off-line. I had an automated duplicity backup, which I regularly used to restore an odd file or two. I was confident that I could rely on it. When I tried restoring the full backup however I hit two bugs that made moving my things to other servers far less than trivial.

According to reports at the height of the crisis one-fifth of the country was without power and the distribution infrastructure in the Notranjska region was almost completely destroyed. While a temporary line was established to my parents' neighborhood after a week and a half, there are still frequent day-long outages.

Even though my place in Ljubljana never lost power, I now keep a few spare batteries and a battery powered FM radio in the top drawer. And I have even more respect for people I know that volunteered in local emergency services and did long shifts to help people in need.

Posted by Tomaž | Categories: Life | Comments »

On forgetting passphrases

01.01.2014 19:57

If you're using encryption to protect your mail and personal files you might find this lesson useful. This goes double if you are (like me) trying religiously to avoid any holes or bit rot in your personal project archive. The gist of it is that you will sooner or later forget a password or a passphrase that you are not actively using.

Consider the case of changing your expired GPG key. If you forget the passphrase for the expired GPG key, you will lose access to your old encrypted mail. It seems obvious in hindsight, but I only realized that after finding out that after a few months of disuse I am unable to recall the old passphrase. I had to say goodbye to an (admittedly small) part of my mail archive.

On a similar note, an encrypted home directory on an old computer will soon be a bag of random bits after you switch to a new machine and change your user password. If forgetting old passwords used to be easy to circumvent (with init=/bin/bash and other venerable tricks), it's impossible now unless you can recall the keyboard sequence from muscle memory. I thoroughly clean out old hardware that leaves my hands. However if a laptop just ends up sitting in a drawer somewhere I'm usually sloppy enough that I often need to lift some old project files from a disused disk drive.

It's easy to avoid this problem once you know it exists. You can write the old GPG passphrase down somewhere or even remove it from the old secret key, depending how concerned you are about the content of old mail. Or you can keep the passphrases for old keys in sync with the new key. And move files you want to retain from old hardware. That saves it from stuck disk bearings as well.

Posted by Tomaž | Categories: Life | Comments »

Jahresrückblick

21.12.2013 18:31

It is that season again that makes power-hungry notebooks double as lap warmers and is conductive to large hacker Congresses in the north. Before I go catch this year's last few deadlines, attend festivities with my family and then lose myself among the blinkenlights in Hamburg, I thought I might join the custom of writing a personal year review.

Well, it's been an unusual year.

If anything has marked it for me it has been travel around Europe. I am pretty sure that I did more kilometers by plane, car or train than in any other single year in my life. If I count only longer trips, I've been to Athens, Brussels, Cologne, Ghent, Ilmenau, Ludwigsburg, Munich, Paris and more places in northern Great Britain than I can remember. I visited most of these places because of my job at the Institute, others for less formal meetups or simply running away from it all for a while.

The list would even be one city longer if I weren't rushed to surgery at one point which left me grounded and limited to the neighborhood of my doctor's office for a month or so.

Maybe because of travel or other things, I found it hard to concentrate on any really important thing this year. I've noticed that my context switches are getting longer. I can hardly work on two serious projects on two consecutive days. Since this is hardly compatible with looming deadlines and overflowing lists of tasks it has led to a lot of frustration and burn out on my part. If it's an effect of trying to focus on harder problems now, it sure feels often like I'm just wasting time on unimportant details.

The Slovenian Open Data group has been a most welcome source of motivation when all other things seemed to move in a wrong way. It's incredibly inspiring to talk with people honestly doing their best to improve the world.

It's been the year when Kiberpipa was shut down and when I more or less lost contact with the Computer Museum.

A new Debian was released which in one way or another broke many work flows I have been using for years. It left a strong impression that desktop software is slowly going the way of the dodo. Together with the continuing confirmations of surveillance on the Internet it has contributed significantly to the feeling of impending doom and doubts regarding where technology is headed today.

Perhaps because of that I also spent more time than I want to admit on pastel colored cartoon ponies. Even though I occasionally fear that this subculture is all a massive viral marketing campaign it's been at times weirdly fascinating to explore. It was a fun way to forget more serious things for a while. It got me to experiment with drawing and writing fiction which was an interesting new experience, even if most of what I made is laughably unoriginal or has been described as too depressing.

"You're discussing cartoons while your country is falling apart" was a comment I once heard that probably contains more truth than I would wish and maybe sums it all up pretty well. 2013 for me has been mostly about vastly more ideas than time and energy to properly implement them and no good way to select that one idea out of nine that would be worth focusing on.

Posted by Tomaž | Categories: Life | Comments »

UHF receiver redesign

03.12.2013 16:10

I'm currently in the middle of designing a new version of the UHF receiver for VESNA sensor nodes. It has been over a year and a half since I designed the first version. In this time I learned a lot about spectrum sensing and so I now have a much better understanding of the state of the art in this field. It has been obvious for quite a while that the design needs to be upgraded in order to keep up with the latest research.

In this time also came to know the capabilities of the VESNA platform much better and can now work on a design that will have a much better match between what is done in hardware and what can be left to digital signal processing in software. The whole idea is to keep it as flexible as possible while staying compatible with low-powered systems like VESNA (and not go for a full-blown software-defined radio approach).

Unfortunately, this project of mine has been taking way too long, with work-related and personal things interfering at the most inconvenient times. Right now I'm optimistic that I will have the hardware design finalized by the end of the year.

Last Friday I should have had a presentation on the topic of the UHF receiver redesign for our research group at the Institute. Sadly I had to cancel, but I am now publishing the slides I prepared for it here.

SNE-ESHTER block diagram slide.

Get the slides in the PDF format.

Posted by Tomaž | Categories: Life | Comments »