Analyzing PIN numbers

13.10.2018 12:17

Since I already had a dump from haveibeenpwned.com on my drive from my earlier password check, I thought I could use this opportunity to do some more analysis on it. Six years ago DataGenetics blog posted a detailed analysis of 4-digit numbers that were found in password lists from various data breaches. I thought it would be interesting to try to reproduce some of their work and see if their findings still hold after a few years and with a significantly larger dataset.

DataGenetics didn't specify the source of their data, except that it contained 3.4 million four-digit combinations. Guessing from the URL, their analysis was published in September 2012. I've done my analysis on the pwned-passwords-ordered-by-hash.txt file downloaded from haveibeenpwned.com on 6 October (inside the 7-Zip archive the file had a timestamp of 11 July 2018, 02:37:47). The file contains 517.238.891 SHA1 hashes with associated frequencies. By searching for SHA1 hashes that correspond to 4-digit numbers from 0000 to 9999, I found that all of them were present in the file. Total sum of their frequencies was 14.479.676 (see my previous post for the method I used to search the file). Hence my dataset was roughly 4 times the size of DataGenetics'.

Here are the top 20 most common numbers appearing in the dump, compared to the rank on the top 20 list from DataGenetics:

nnew nold PIN frequency
1 1 1234 8.6%
2 2 1111 1.7%
3 1342 1.1%
4 3 0000 1.0%
5 4 1212 0.5%
6 8 4444 0.4%
7 1986 0.4%
8 5 7777 0.4%
9 10 6969 0.4%
10 1989 0.4%
11 9 2222 0.3%
12 13 5555 0.3%
13 2004 0.3%
14 1984 0.2%
15 1987 0.2%
16 1985 0.2%
17 16 1313 0.2%
18 11 9999 0.2%
19 17 8888 0.2%
20 14 6666 0.2%

This list looks similar to the results published DataGenetics. The first two PINs are the same, but the distribution is a bit less skewed. In their results, first four most popular PINs accounted for 20% of all PINs, while here they only make up 12%. It seems also that numbers that look like years (1986, 1989, 2004, ...) have become more popular. In their list the only two in the top 20 list were 2000 and 2001.

Cumulative frequency of PINs

DataGenetics found that number 2580 ranked highly in position 22. They concluded that this is an indicator that a lot of these PINs were originally devised on devices with numerical keyboards such as ATMs and phones (on those keyboards, 2580 is straight down the middle column of keys), even though the source of their data were compromised websites where users would more commonly use a 104-key keyboard. In the haveibeenpwned.com dataset, 2580 ranks at position 65, so slightly lower. It is still in top quarter by cumulative frequency.

Here are 20 least common numbers appearing in the dump, again compared to their rank on the bottom 20 list from DataGenetics:

nnew nold PIN frequency
9981 0743 0.00150%
9982 0847 0.00148%
9983 0894 0.00147%
9984 0756 0.00146%
9986 0934 0.00146%
9985 0638 0.00146%
9987 0967 0.00145%
9988 0761 0.00144%
9989 0840 0.00142%
9991 0835 0.00141%
9990 0736 0.00141%
9993 0742 0.00139%
9992 0639 0.00139%
9994 0939 0.00132%
9995 0739 0.00129%
9996 0849 0.00126%
9997 0938 0.00125%
9998 0837 0.00119%
9999 9995 0738 0.00108%
10000 0839 0.00077%

Not surprisingly, most numbers don't appear in both lists. Since these have the lowest frequencies it also means that the smallest changes will significantly alter the ordering. The least common number 8068 in DataGenetics' dump is here in place 9302, so still pretty much at the bottom. I guess not many people choose their PINs after the iconic Intel CPU.

Here is a grid plot of the distribution, drawn in the same way as in the DataGenetics' post. Vertical axis depicts the right two digits while the horizontal axis depicts the left two digits. The color shows the relative frequency in log scale (blue - least frequent, yellow - most frequent).

Grid plot of the distribution of PINs

Many of the same patterns discussed in the DataGenetics' post are also visible here:

  • The diagonal line shows popularity of PINs where left two and right two digits repeat (pattern like ABAB), with further symmetries superimposed on it (e.g. AAAA).

  • The area in lower left corner shows numbers that can be interpreted as dates (vertically MMDD and horizontally DDMM). The resolution is good enough that you can actually see which months have 28, 30 or 31 days.

  • The strong vertical line at 19 and 20 shows numbers that can be interpreted as years. The 2000s are more common in this dump. Not surprising, since we're further into the 21st century than when DataGenetics' analysis was done.

  • Interestingly, there is a significant shortage of numbers that begin with 0, which can be seen as a dark vertical stripe on the left. A similar pattern can be seen in DataGenetics' dump although they don't comment on it. One possible explanation would be if some proportion of the dump had gone through a step that stripped leading zeros (such as a conversion from string to integer and back, maybe even an Excel table?).

In conclusion, the findings from DataGenetics' post still mostly seem to hold. Don't use 1234 for your PIN. Don't choose numbers that have symmetries in them or have years or dates in them. These all significantly increase the chances that someone will be able to guess them. And of course, don't re-use your SIM card or ATM PIN as a password on websites.

Another thing to note is that DataGenetics concluded that their analysis was possible because of leaks of clear-text passwords. However, PINs provide a very small search space of only 10000 possible combinations. It was trivial for me to perform this analysis even though haveibeenpwned.com dump only provides SHA-1 hashes, and not clear-text. With a warmed-up disk cache, the binary search only took around 30 seconds for all 10000 combinations.

Posted by Tomaž | Categories: Life | Comments »

Checking passwords against haveibeenpwned.com

07.10.2018 11:34

haveibeenpwned.com is a useful website that aggregates database leaks with user names and passwords. They have an on-line form where you can check whether your email has been seen in publicly available database dumps. The form also tells you in which dumps they have seen your email. This gives you a clue which of your passwords has been leaked and should be changed immediately.

For a while now haveibeenpwned.com reported that my email appears in a number of combined password and spam lists. I didn't find that surprising. My email address is publicly displayed on this website and is routinely scraped by spammers. If there was any password associated with it, I thought it has come from the 2012 LinkedIn breach. I knew my old LinkedIn password has been leaked, since scam mails I get are commonly mentioning it.

Screenshot of a scam email.

However, it came to my attention that some email addresses are in the LinkedIn leak, but not in these combo lists I appear in. This seemed to suggest that my appearance in those lists might not only be due to the old LinkedIn breach and that some of my other passwords could have been compromised. I thought it might be wise to double-check.

haveibeenpwned.com also provides an on-line form where they can directly check your password against their database. This seems a really bad practice to encourage, regardless of their assurances of security. I was not going to start sending off my passwords to a third-party. Luckily the same page also provides a dump of SHA-1 hashed passwords you can download and check locally.

I used Transmission to download the dump over BitTorrent. After uncompressing the 7-Zip file I ended up with a 22 GB text file with one SHA-1 hash per line:

$ head -n5 pwned-passwords-ordered-by-hash.txt
000000005AD76BD555C1D6D771DE417A4B87E4B4:4
00000000A8DAE4228F821FB418F59826079BF368:2
00000000DD7F2A1C68A35673713783CA390C9E93:630
00000001E225B908BAC31C56DB04D892E47536E0:5
00000006BAB7FC3113AA73DE3589630FC08218E7:2

I chose the ordered by hash version of the file, so hashes are alphabetically ordered. The number after the colon seems to be the number of occurrences of this hash in their database. More popular passwords will have a higher number.

The alphabetical order of the file makes it convenient to do an efficient binary search on it as-is. I found the hibp-pwlookup tool tool for searching the password database, but that requires you to import the data into PostgreSQL, so it seems that the author was not aware of this convenience. In fact, there's already a BSD command-line tool that knows how to do binary search on such text files: look (it's in the bsdmainutils package on Debian).

Unfortunately the current look binary in Debian is somewhat broken and bails out with File too large error on files larger than 2 GB. It needs recompiling with a simple patch to its Makefile to work on this huge password dump. After I fixed this, I was able to quickly look up the hashes:

$ ./look -b 5BAA61E4 pwned-passwords-ordered-by-hash.txt
5BAA61E4C9B93F3F0682250B6CF8331B7EE68FD8:3533661

By the way, Stephane on the Debian bug tracker also mentions a method where you can search the file without uncompressing it first. Since I already had it uncompressed on the disk I didn't bother. Anyway, now I had to automate the process. I used a Python script similar to the following. The check() function returns True if the password in its argument is present in the database:

import subprocess
import hashlib

def check(passwd):
	s = passwd.encode('ascii')
	h = hashlib.sha1(s).hexdigest().upper()
	p = "pwned-passwords-ordered-by-hash.txt"

	try:
		o = subprocess.check_output([
			"./look",
			"-b",
			"%s:" % (h,),
			p])
	except subprocess.CalledProcessError as exc:
		if exc.returncode == 1:
			return False
		else:
			raise

	l = o.split(b':')[1].strip()

	print("%s: %d" % (
		passwd,
		int(l.decode('ascii'))))

	return True

def main():
	check('password')

main()

Before I was able to check those passwords I keep stored in Firefox, I stumbled upon another hurdle. Recent versions do not provide any facility for exporting passwords in the password manager. There are some third-party tools for that, but I found it hard to trust them. I also had my doubts on how complete they are: Firefox has switched many mechanisms for storing passwords over time and re-implementing the retrieval for all of them seems to be a non-trivial task.

In the end, I opted to get them from Browser Console using the following Javascript, written line-by-line into the console (I adapted this code from a post by cor-el on Mozilla forums):

var tokendb = Cc["@mozilla.org/security/pk11tokendb;1"].createInstance(Ci.nsIPK11TokenDB);
var token = tokendb.getInternalKeyToken();
token.login(true);

var passwordmanager = Cc["@mozilla.org/login-manager;1"].getService(Ci.nsILoginManager);
var signons = passwordmanager.getAllLogins({});
var json = JSON.stringify(signons, null, 1);

I simply copy-pasted the value of the json variable here into a text editor and saved it to a file. I used an encrypted volume, so passwords didn't end up in clear-text on the drive.

I hope this will help anyone else to check their passwords against the database of leaks without exposing them to a third-party in the process. Fortunately for me, this check didn't reveal any really nasty surprises. I did find more hits in the database than I'm happy with, however all of them were due to certain poor password policies about which I can do very little.

Posted by Tomaž | Categories: Life | Comments »

Extending GIMP with Python talk

22.09.2018 18:47

This Thursday I presented a talk on writing GIMP plug-ins in Python at the monthly Python Meetup in Ljubljana. I gave a brief guide on how to get started and shared some tips that I personally would find useful when I first started playing with GIMP. I didn't go into the details of the various functions but tried to keep it high-level, showing most of the things live in the Python REPL window and demoing some of the plug-ins I've written.

I wanted to give this talk for two reasons: first was that I struggled to find such a high-level and up-to-date introduction into writing plug-ins. I though it would be useful for anyone else diving into GIMP's Python interface. I tried to prepare slides so that they can serve as a useful document on their own (you can find the slides here, and the example plug-in on GitHub). The slides also include some useful links for further reading.

Annotated "Hello, World!" GIMP plug-in.

The second reason was that I was growing unhappy with topics presented at these meetups. It seemed to me that recently most were revolving about web and Python at the workplace. I can understand why this is so. The event needs sponsors and they want to show potential new hires how awesome it is to work with them. The web is eating the world, Python meetups or otherwise, and it all leads to a kind of a monotonic series of events. So it's been a while since I've heard a fun subject discussed and I thought I might stir things up a bit.

To be honest, initially I was quite unsure whether this was good venue for the talk. I discussed the idea with a few friends that are also regular meetup visitors and their comments encouraged me to go on with it. Afterwards I was immensely surprised at the positive response. I had prepared a short tour of GIMP for the start of the talk, thinking that most people would not be familiar with it. It turned out that almost everyone in the audience used GIMP, so I skipped it entirely. I was really happy to hear that people found the topic interesting and it reminded me what a pleasure it is to give a well received talk.

Posted by Tomaž | Categories: Life | Comments »

Investigating the timer on a Hoover washing machine

20.07.2018 17:47

One day around last Christmas I stepped into the bathroom and found myself standing in water. The 20-odd years old Candy Alise washing machine finally broke a seal. For some time already I was suspecting that it wasn't rinsing the detergent well enough from the clothes and it was an easy decision to just scrap it. So I spent the holidays shopping for a new machine as well as taking apart and reassembling bathroom furniture that was in the way, and fixing various collateral damage on plumbing. A couple of weeks later and thanks to copious amounts of help from my family I was finally able to do laundry again without flooding the apartment in the process.

I bought a Hoover WDXOC4 465 AC combined washer-dryer. My choice was mostly based on the fact that a local shop had it on stock and its smaller depth compared to the old Candy meant a bit more floor space in the small bathroom. Even though I was trying to avoid it, I ended up with a machine with capacitive touch buttons, a NFC interface and a smartphone app.

Front panel on a Hoover WDXOC4 465 AC washer.

After half a year the machine works reasonably well. I gave up on Hoover's Android app the moment it requested to read my phone's address book, but thankfully all the features I care about can be used through the front panel. Comments I saw on the web that the dryer doesn't work have so far turned out to be unfounded. My only complaint is that 3 or 4 times it happened that the washer didn't do a spin cycle when it should. I suspect that the quality of embedded software is less then stellar. Sometimes I hear the inlet valve or the drum motor momentarily stop and engage again and I wonder if that's due to a bug or controller reset or if they are meant to work that way.

Anyway, as most similar machines, this one displays the remaining time until the end of the currently running program. There's a 7-segment display on the front panel that counts down hours and minutes. One of the weirder things I noticed is that the countdown seems to use Microsoft minutes. I would look at the display to see how many minutes are left, and then when I looked again later it would be seem to be showing more time remaining, not less. I was curious how that works and whether it was just my bad memory or whether the machine was really showing bogus values, so I decided to do an experiment.

I took a similar approach to my investigation of the water heater years ago. I loaded up the machine and set it up for a wash-only cycle (6 kg cottons program, 60°C, 1400 RPM spin cycle, stain level 2) and recorded the display on the front panel with a laptop camera. I then read out the timer values from the video with a simple Python program and compared them to the frame timestamps. The results are below: The real elapsed time is on the horizontal axis and the display readout is on the vertical axis. The dashed gray line shows what the timer should ideally display if the initial estimate would be perfect:

Countdown display on a Hoover washer versus real time.

The timer first shows a reasonable estimate of 152 minutes at the start of the program. The filled-in area on the graph around 5 minutes in is when the "Kg Mode" is active. At that time the display shows a little animation as the washer supposedly weighs the laundry. My OCR program got confused by that, so the time values there are invalid. However, after the display returns to the countdown, the time displayed is significantly lower, at just below 2 hours.

"Kg Mode" description from the Hoover manual.

Image by Candy Hoover Group S.r.l.

That seems fine at first - I didn't use a full 6 kg load, so it's reasonable to believe that the machine adjusted the wash time accordingly. However the minutes on the display then count down slower than real time. So much slower in fact that they more than make up for the difference and the machine ends the cycle after 156 minutes, 4 minutes later than the initial estimate.

You can also see that the display jumps up and down during the program, with the largest jump around 100 minutes in. Even when it seems to be running consistently, it will often count a minute down, then up and down again. You can see evidence of that on the graph below that shows differences on the timer versus time. I've verified this visually on the video and it's not an artifact of my OCR.

Timer display differences on a Hoover washer.

Why is it doing that? Is it just buggy software, an oscillator running slow or maybe someone figured that people would be happier when a machine shows a lower number and then slows down the passing minutes? Hard to say. Even if you don't tend to take videos of your washing machine, it's hard not to notice that the timer stands still at the last 1 minute for around 7 minutes.

Incidentally, does anyone know how the weighing function works? I find it hard to believe that they actually measure the weight with a load cell. They must have some cheaper way, perhaps by measuring the time it takes for water level rise up in the drum or something like that. The old Candy machine also advertised this feature and considering that was an electro-mechanical affair without sophisticated electronics it must be a relatively simple trick.

Pixel sampling locations for the 7-segment display OCR.

In conclusion, a few words on how I did the display OCR. I started off with the seven-segment-ocr project by Suyash Kumar I found on GitHub. Unfortunately it didn't work for me. It uses a very clever trick to do the OCR - it just looks at intensity profiles of two horizontal and one vertical line per character per frame. However because my video had low resolution and was recorded at a slight angle no amount of tweaking worked. In the end I just hacked up a script that sampled 24 hand-picked single pixels from the video. It then compared them to a hand-picked threshold and decoded the display values from there. Still, Kumar's project came in useful since I could reuse all of his OpenCV boilerplate.

Recording of a digital clock synchronized to the DCF77 signal.

Since I like to be sure, I also double-checked that my method of getting real time from video timestamps is accurate enough. Using the same setup I recorded a DCF77-synchronized digital clock for two hours. I estimated the relative error between video timestamps and the clock to be -0.15%, which means that after two hours, my timings were at most 11 seconds off. The Microsoft-minute effects I found in Hoover are much larger than that.

Posted by Tomaž | Categories: Life | Comments »

Faking adjustment layers with GIMP layer modes

12.03.2018 19:19

Drawing programs usually use a concept of layers. Each layer is like a glass plate you can draw on. The artwork appears in the program's window as if you were looking down through the stack of plates, with ink on upper layers obscuring that on lower layers.

Example layer stack in GIMP.

Adobe Photoshop extends this idea with adjustment layers. These do not add any content, but instead apply a color correction filter to lower layers. Adjustment layers work in real-time, the color correction gets applied seamlessly as you draw on layers below.

Support for adjustment layers in GIMP is a common question. GIMP does have a Color Levels tool for color correction. However, it can only be applied on one layer at a time. Color Levels operation is also destructive. If you apply it to a layer and want to continue drawing on it, you have to also adjust the colors of your brushes if you want to be consistent. This is often hard to do. It mostly means that you need to leave the color correction operation for the end, when no further changes will be made to the drawing layers and you can apply the correction to a flattened version of the image.

Adjustment layers are currently on the roadmap for GIMP 3.2. However, considering that Debian still ships with GIMP 2.8, this seems like a long way to go. Is it possible to have something like that today? I found some tips on the web on how to fake the adjustment layers using various layer modes. But these are very hand-wavy. Ideally what I would like to do is perfectly replicate the Color Levels dialog, not follow some vague instructions on what to do if I want the picture a bit lighter. That post did give me an idea though.

Layer modes allow you to perform some simple mathematical operations between pixel values on layers, like addition, multiplication and a handful of others. If you fill a layer with a constant color and set it to one of these layer modes, you are in fact transforming pixel values from layers below using a simple function with one constant parameter (the color on the layer). Color Levels operation similarly transforms pixel values by applying a function. So I wondered, would it be possible to combine constant color layers and layer modes in such a way as to approximate arbitrary settings in the color levels dialog?

Screenshot of the Color Levels dialog in GIMP.

If we look at the color levels dialog, there are three important settings: black point (b), white point (w) and gamma (g). These can be adjusted for red, green and blue channels individually, but since the operations are identical and independent, it suffices to focus a single channel. Also note that GIMP performs all operations in the range of 0 to 255. However, the calculations work out as if it was operating in the range from 0 to 1 (effectively a fixed point arithmetic is used with a scaling factor of 255). Since it's somewhat simpler to demonstrate, I'll use the range from 0 to 1.

Let's first ignore gamma (leave it at 1) and look at b and w. Mathematically, the function applied by the Color Levels operation is:

y = \frac{x - b}{w - b}

where x is the input pixel value and y is the output. On a graph it looks like this (you can also get this graph from GIMP with the Edit this Settings as Curves button):

Plot of the Color Levels function without gamma correction.

Here, the input pixel values are on the horizontal axis and output pixel values are on the vertical. This function can be trivially split into two nested functions:

y = f_2(f_1(x))
where
f_1(x) = x - b
f_2(x) = \frac{x}{w-b}

f1 shifts the origin by b and f2 increases the slope. This can be replicated using two layers on the top of the layers stack. GIMP documentation calls these masking layers:

  • Layer mode Subtract, filled with pixel value b,
  • Layer mode Divide, filled with pixel value w - b

Note that values outside of the range 0 - 1 are clipped. This happens on each layer, which makes things a bit tricky for more complicated layer stacks.

The above took care of the black and white point settings. What about gamma adjustment? This is a non-linear operation that gives more emphasis to darker or lighter colors. Mathematically, it's a power function with a real exponent. It is applied on top of the previous linear scaling.

y = x^g

GIMP allows for values of g between 0.1 and 10 in the Color Levels tool. Unfortunately, no layer mode includes an exponential function. However, the Soft light mode applies the following equation:

R = 1 - (1 - M)\cdot(1-x)
y = ((1-x)\cdot M + R)\cdot x

Here, M is the pixel value of the masking layer. If M is 0, this simplifies to:

y = x^2

So by stacking multiple such masking layers with Soft light mode, we can get any exponent that is a multiple of 2. This is still not really useful though. We want to be able to approximate any real gamma value. Luckily, the layer opacity setting opens up some more options. Layer opacity p (again in the range 0 - 1) basically does a linear combination of the original pixel value and the masked value. So taking this into account we get:

y = (1-p)x + px^2

By stacking multiple masking layers with opacity, we can get a polynomial function:

y = a_1 x + a_2 x^2 + x_3 x^3 + \dots

By carefully choosing the opacities of masking layers, we can manipulate the polynomial coefficients an. Polynomials of a sufficient degree can be a very good approximation for a power function with g > 1. For example, here is an approximation using the above method for g = 3.33, using 4 masking layers:

Polynomial approximation for the gamma adjustment function.

What about the g < 1 case? Unfortunately, polynomials don't give us a good approximation and there is no channel mode that involves square roots or any other usable function like that. However, we can apply the same principle of linear combination with opacity settings to multiple saturated divide operations. This effectively makes it possible to piecewise linearly approximate the exponential function. It's not as good as the polynomial, but with enough linear segments it can get very close. Here is one such approximation, for g = 0.33, again using 4 masking layers:

Piecewise linear approximation for the gamma adjustment function.

To test this all together in practice I've made a proof-of-concept GIMP plug-in that implements this idea for gray-scale images. You can get it on GitHub. Note that it needs a relatively recent Scipy and Numpy versions, so if it doesn't work at first, try upgrading them from PyPi. This is how the adjustment layers look like in the Layers window:

Fake color adjustment layer stack in GIMP.

Visually, results are reasonably close to what you get through the ordinary Color Layers tool, although not perfectly identical. I believe some of the discrepancy is caused by rounding errors. The following comparisons show a black-to-white linear gradient with a gamma correction applied. First stripe is the original gradient, second has the gamma correction applied using the built-in Color Levels tool and the third one has the same correction applied using the fake adjustment layer created by my plug-in:

Fake adjustment layer compared to Color Levels for gamma = 0.33.

Fake adjustment layer compared to Color Levels for gamma = 3.33.

How useful is this in practice? It certainly has the non-destructive, real-time property of the Photoshop's adjustment layer. The adjustment is immediately visible on any change in the drawing layers. Changing the adjustment settings, like the gamma value, is somewhat awkward, of course. You need to delete the layers created by the plug-in and create a new set (although an improved plug-in could assist in that). There is no live preview, like in the Color Levels dialog, but you can use Color Levels for preview and then copy-paste values into the plug-in. The multiple layers also clutter the layer list. Unfortunately it's impossible to put them into a layer group, since layer mode operations don't work on layers outside of a group.

My current code only works on gray-scale. The polynomial approximation uses layer opacity setting, and layer opacity can't be set separately for red, green and blue channels. This means that way of applying gamma adjustment can't be adapted for colors. However support for colors should be possible for the piecewise linear method, since in that case you can control the divider separately for each channel (since it's defined by the color fill on the layer). The opacity still stays the same, but I think it should be possible to make it work. I haven't done the math for that though.

Posted by Tomaž | Categories: Life | Comments »

FOSDEM 2018

08.02.2018 11:28

Gašper and I attended FOSDEM last weekend. It is an event about open source and free software development that happens each year in Brussels. I've known about it for some time, mostly because it was quite a popular destination among a group of Kiberpipa alumni a few years ago. I've never managed to join them. This year though I was determined to go. Partly because I couldn't get a ticket for 34C3 and was starting to miss this kind of a crowd, and partly because several topics on the schedule were relevant to one of my current projects.

Marietje Schaake on Next Generation Internet at FOSDEM 2018

FOSDEM turned out to be quite unlike any other event I've attended. I was surprised to hear that there are no tickets or registration. When we arrived at the event it was immediately apparent why that is the case. Basically, for two days the Université libre de Bruxelles simply surrenders their lecture rooms and hallways to software developers. Each room is dedicated to talks on a particular topic, such as software defined radio or Internet of things. Hallways are filled with booths, from companies offering jobs to volunteer projects gathering donations and selling t-shirts. I counted more than 50 tracks, distributed over 5 campus buildings. More than once I found myself lost in the unfamiliar passageways.

Opinions whether the event is too crowded or not seem to differ. I often found the halls unbearably packed and had to step out to get a breath of air, but I do have a low threshold for these kind of things. Mostly cold, wet and windy weather didn't help with crowds indoors either. The lecture rooms themselves had volunteers taking care they were not filled over capacity. This meant that once you got in a room, following the talks was a pleasant experience. However, most rooms had long queues in front. The only way to get in was to show up early in the morning and stay there. I didn't manage to get back into any smaller room after leaving for lunch, so I usually ended up in the biggest auditorium with the keynote talks.

Queues at the food court at FOSDEM 2018.

Speaking about keynotes. I enjoyed Simon Phipps's and Italo Vignoli's talk about the past 20 years of open source. They gave a thorough overview of the topic with some predictions for the future. I found myself thinking whether open source movement really was such a phenomenal success. Indeed it is everywhere behind pervasive web services of today, however the freedom for users to run, study and improve software at their convenience is mostly missing these days where software is hidden behind an opaque web server on someone else's computer. Liam Proven in Alternate histories of computing explored how every generation reinvents solutions and mistakes of their predecessors, with a focus on Lisp machines. It seems that one defining property of computer industry is that implementing a thing from scratch is invariably favored over understanding and building upon existing work. I also recommend Steven Goodwin's talk about how hard it is to get behavior right in simplest things like smart light switches. He gave nice examples why a smart appliance that has not been well thought out will cause more grief than a dumb old one.

From the more technical talks I don't have many recommendations. I haven't managed to get into most of the rooms I had planned for. From hanging around the Internet of things discussions one general sentiment I captured was that MQTT has become a de facto standard for any Internet-connected sensor or actuator. ESP8266 remains as popular as ever for home-brew projects. Moritz Fischer gave a fascinating, but very dense intro into a multitasking scheduler for ARM Cortex microcontrollers that was apparently custom developed by Google for use on their Chromebook laptops. Even though I'm fairly familiar with ARM, he lost me after the first 10 minutes. However if I ever need to look into multitasking on ARM microcontrollers, his slides seem to be a very good reference.

One of the rare moments of dry weather at FOSDEM 2018

I don't regret going to FOSDEM. It has been an interesting experience and the trip has been worth it just for the several ideas Gašper and I got there. I can't complain about the quality of the talks I've seen and although the whole thing seemed chaotic at times it must have been a gargantuan effort on the part of the volunteer team to organize. I will most likely not be going again next year though. I feel like this is more of an event for getting together with a team you've been closely involved with in developing a large open source project. Apart from drive-by patches, I've not collaborated in such projects for years now, so I often felt as an outsider that was more adding to the crowds than contributing to the discussion.

Posted by Tomaž | Categories: Life | Comments »

Tracking notebooks

27.01.2018 20:33

Two months ago, there was a post on Hacker News about keeping a lab notebook. If you like to keep written notes, the training slides from NIH might be interesting to flip through, even if you're not working in a scientific laboratory. In the ensuing discussion, user cgore mentioned that they recommend attaching a Bluetooth tracker on notebooks. In recent years I've developed a quite heavy note taking habit and these notebooks have become increasingly important to me. I carry one almost everywhere I go. I can be absentminded at times and had my share of nightmares about losing all my notes for the last 80 pages. So the idea of a Bluetooth tracker giving me a peace of mind seemed worth exploring further.

cgore mentioned the Tile tracker in their comment. The Tile Slim variant seemed to be most convenient for sticking to a cover of a notebook. After some research I found that the Slovenian company Chipolo was developing a similar credit card-sized device, but in November last year I couldn't find a way to buy one. The Wirecutter's review of trackers was quite useful. In the end, I got one Tile Slim for my notebook.

Tile Slim Bluetooth tracker.

This is the tracker itself. The logo on the top side is a button and there is a piezo beeper on the other side. It is indeed surprisingly thin (about 2.5 mm). Of course if you cover it with a piece of paper (or ten), the bump it makes under the pen is still hard to ignore when writing over it. I mostly use school notebooks with soft covers, so the first challenge was where to put it on the notebook for it to be minimally annoying. I also wanted it to be removable so I could reuse the Tile on a new notebook.

Tile Slim in a paper envelope on a notebook cover.

What I found to work best for me is to put the Tile in a paper pocket. I glue the pocket to the back cover of the notebook, near the spine. I still feel the edges when writing, but it's not too terrible. So far I was making the pockets from ordinary office paper (printable pattern) and they seem to hold up fine for the time the notebook is in active use. They do show noticeable wear though, so maybe using heavier paper (or duct tape) wouldn't be a bad idea. The covers of notebooks I use are thick enough that I haven't noticed that I would be pressing the button on the tile with my pen.

There is no simple way to disassemble the Tile and the battery is not replaceable nor rechargeable. Supposedly it lasts a year and after that you can get a new Tile at a discount. Unless, that is, you live in a country where they don't offer that. Do-it-yourself battery replacement doesn't look like an option either, although I will probably take mine apart when it runs out. I don't particularly like the idea of encouraging more throw-away electronics

Screenshots of the Tile app on Android.

I found the Android app that comes with the tracker quite confusing at first. Half of the screen space is dedicated to "Tips" and it has some kind of a walk-through mode when you first use it. It was not clear to me what was part of the walk-through and what was for real. I ended up with the app in a state where it insisted that my Tile is lost and stubbornly tried to push their crowd-sourced search feature. Some web browsing later I found that I needed to restart my phone, and after that I didn't notice the app erroneously losing contact with the Tile again. In any case, hopefully I won't be opening the app too often.

Once I figured them out, the tracker and the app did seem to do their job reasonably well. For example, if I left the notebook at the office and went home the app would tell me the street address of the office where I left it. I simulated a lost notebook several times and it always correctly identified the last seen address, so that seems encouraging. I'm really only interested in this kind of rough tracking. I'm guessing the phone records the GPS location when it doesn't hear the Bluetooth beacons from the tracker anymore. The Tile also has a bunch of features I don't care about: you can make your phone ring by pressing the button on the Tile or you can make the Tile play a melody. There's also a RSSI ranging kind of thing in the app where it will tell you how far from the Tile you are in the room, but it's as useless as all other such attempts I've seen.

I have lots of problems in a notebook.

With apologies to xkcd

The battery drain on the phone after setting up the app is quite noticeable. I haven't done any thorough tests, but my Motorola Moto G5 Plus went from around 80% charge at the end of the day to around 50%. While before I could usually go two days without a charger, now the battery will be in the red after 24 hours. I'm not sure whether that is the app itself or the fact that now Bluetooth on the phone must be turned on all the times. Android's battery usage screen lists neither the Tile app nor Bluetooth (nor anything else for that matter) as causing significant battery drain.

Last but not least, carrying the Tile on you is quite bad for privacy. The Tile website simply brushes such concerns aside, saying we are all being constantly tracked anyway. But the fact is that using this makes (at least) one more entity aware of your every move. The app does call home with your location, since you can look up current Tile location on the web (although for me, it says I don't own any Tiles, so maybe the call-home feature doesn't actually work). It's another example of an application that would work perfectly fine without any access to the Internet at all but for some reason needs to talk to the Cloud. The Tiles are also trivially trackable by anyone with a Bluetooth dongle and I wonder what kind of authentication is necessary for triggering that battery-draining play-the-melody thing.

I will probably keep using this Tile until it runs out. It does seem to be useful, although I have not yet lost a notebook. I'm not certain I will get another though. The throw-away nature of it is off-putting and I don't like being a walking Bluetooth beacon. However it does make me wonder how much thinner you could make one without that useless button and beeper, and perhaps adding a wireless charger. And also if some kind of a rolling code could be used to prevent non-authorized people from tracking your stuff.

Posted by Tomaž | Categories: Life | Comments »

Luggage lock troubles

05.10.2017 18:32

Back in August an annoying thing happened while I was traveling in Germany. My luggage lock wouldn't open. I was pretty sure I had the right combination, but the damn thing would not let me get to my things. I tried some other numbers that came to mind and a few futile attempts at picking it. I then sat down and went through all 1000 combinations. It would still not open. At that point it was around 1 am, I had a long bus ride that day and desperately needed a shower and sleep. I ran out of patience, took the first hard object that came to hand and broke off the lock. Thankfully, that only took a few moments.

Broken luggage combination lock.

This is the culprit, reassembled back to its original shape after the fact. It's a type of a 3-digit combination lock that locks the two zipper handles on the main compartment of the suitcase. It came off an "Easy Trip" suitcase of a Chinese origin. It's obviously just a slight inconvenience for anyone wanting to get in, but I didn't want to ruin my suitcase and dragging my clothes through a narrow crack made by separating the zipper with a ballpoint pen felt uncivilized.

Inside luggage combination lock in locked position.

This is how inside of the lock looks like in the locked position. There are three discs with notches, one for each digit. Placed over them is a black plastic latch with teeth that align with the notches. In the locked position, the latch is pushed right by the discs against the horizontal spring in the middle right. In this position, the two protrusions on the top of the latch prevent the two metal bolts from rotating in. The bolts go through the loops in the zipper handles. They also connect to the button on the other side of the lock, so in this position the button can't be moved.

Inside luggage combination lock in unlocked position.

When the right combination is entered, the teeth on the latch match the notches on the discs. The spring extends, snaps the latch left and moves it out of the way of the bolts. The button can be pressed and the bolts rotate to free the zipper.

So, what went wrong with my lock? When I removed the lock from the suitcase, the spring that moves the latch was freely rattling inside the case. Without it, the latch will not move on its own, even when the right combination is set. I think this is the likely cause of my troubles. The spring does not have a proper seat on the fixed end, so it seems likely that a strong enough knock on the lock could displace it. I guess that is not beyond the usual abuse luggage experiences in air travel. After replacing the spring the lock works just fine. I'm not using it again though.

Would it be possible to get my toiletries in a less destructive way? If I knew that the problem was the displaced spring, the solution would be simply to set the right combination, make sure the button is not pressed to remove friction on the latch and then rotate the suitcase until gravity moved the latch out of the way. Another possibility would be to pen the zipper and then unscrew the lock from the inside. Unscrewed lock is easily disassembled and removed from the zipper handles, but finding the screws and space for a screwdriver might be impractical with a full suitcase.

Posted by Tomaž | Categories: Life | Comments »

Galaksija retrospective and some news

29.04.2017 8:59

This year my Galaksija replica is a decade old. Back in January 2007 I first brought to life enough of the circuit in my father's workshop to display some faint, mirrored characters on the TV screen. It has been the culmination of 6 intensive months of reverse engineering and documenting the original design, ironing out the odd corners and designing a compatible computer around it. In September that year I've successfully defended my diploma thesis on Galaksija at the Faculty of Electrical Engineering.

In the following five years, I've mostly focused on the software. With the help of the working replica, I've been able to document a large portion of its ROM. I found convincing clues that Galaksija's BASIC was derived from Tandy TRS-80, not Microsoft BASIC as it is commonly believed. I explored old software and wrote some new demos. Along the way I confirmed the myth that high-resolution graphics were possible on that hardware. Finally, I've summed up most of my work in the Ultimate Galaksija talk in front of fellow hackers at the 29C3.

Applebloom simulation on Galaksija at 29C3 retro-gaming assembly.

I can't believe it's been ten years already. It feels like I've written all those articles two life times ago. In the mean time, I've been employed in vastly different fields and I guess I shouldn't be surprised that a lot has faded from my memory. The Hamburg talk was certainly the crowning moment and after the pretty exhausting preparations for it, I hardly returned to anything Galaksija-related. My replica still works, but while I once knew by heart minute timing details of every signal, the circuit diagram now only stirs up general ideas behind its operation. Still, the &2BA8 register address for horizontal sync adjustment immediately popped up from muscle memory upon touching the keyboard.

Anyway, I promised some news. There will be a Mini Maker Faire in Ljubljana in two weeks. I've always been intrigued by reports from Maker Faires and on a couple of occasions I've been very close to going to the one in Brighton. However, each time something else intervened. Hence I immediately jumped to the opportunity to participate in the first Maker Faire in our country. Also, a few people encouraged me to put my Galaksija on display once again. So, if you would like to play some Inspektor Spiridon on real hardware, or puzzle over why your BASIC program results in the WHAT? response, find my table at the Poligon Creative Centre on 13 May.

Galaksija circuit board from Mr. Ivetić.

The other news is that I was recently contacted by Mr. Vojislav Ivetić. As a fan of old computers, he acquired an old, original Galaksija from one of his colleagues and was interested in the possibility of renovating it and bringing it back to life. The exact state of the board is unknown, but supposedly it is not in a working condition. From the first glance it appears to be a mostly complete article, built from one of the original kits, albeit with some crude home brew modifications. Inspired by the recent series on Xerox Alto restoration by Ken Shirriff, I offered to have a look and at least do a quick checkup and a list of things that would need to be done to get it working.

I'll post my first findings from a more detailed look at this Galaksija in another blog post here in the near future. I would love to restore this computer back to its former glory and document the process. But that really depends on how much time I'll be able to dedicate to it and how other things in my life turn out in the following months.

Posted by Tomaž | Categories: Life | Comments »

Jahresrückblick

29.01.2017 18:56

So this has been 2016. Another year that went by all too fast. It has made the world a bit more broken and a whole lot more selfish and incomprehensible.

I haven't traveled much. Except for one project meeting I've hardly left the country. In part because other colleagues took the burden of attending meetings where the Institute had to have a presence. The other part must have been the effect of attacks around Europe and the newly erected borders and barbed wires. I haven't been to any foreign conference, I've skipped 33C3 and I had to cancel my visit to GalaCon at the last moment.

On the other hand, it feels like the local tech meetups have mostly devolved into TED-style talks. I miss the old grassroots events where people would discuss technologies they found interesting or fascinating projects that grew in their basements. It seems the only valid reason to be excited about something these days is if you want to be hired or want to hire someone.

Ljubljana

Last year more than before I found myself in situations where people were willing to talk collaboration up to the point where they got what they wanted - and then walked away without as much as a thank you. The charge more mantra is now being taught to every STEM student as a solution to this, and I find it increasingly depressing. Must you really state an hourly rate before even saying hello?

Social events now have codes of conduct, but seem more hostile than ever. Then again, maybe that tells more about myself than those events. Sometimes I feel like I don't understand people and these communities anymore.

All this obviously affected my motivation to work on projects in my free time. Existing Free software I maintain has mostly been on life support last year. Some things I would usually took the effort to clean and publish as open source were left on my disk. I've done very little electronics work outside of my duties at the Institute. In general I stepped away a lot from the usual technical stuff. I rather spent evenings reading, and at a drawing class in the National Gallery.

All we are

I'm now in my 6th year employed as a research assistant at the Institute. My project work last year has been mostly related to ultra-narrowband technology. I have 26 publications listed in the national bibliographic database and political games in academia still manage to catch me completely by surprise. With the start of the new academic year I've made a decision to finish my PhD and set a deadline to wrap things up by October.

Most of my time after that have been dedicated to concluding my research work, and writing journal papers. My doctoral topic application has been accepted, but I'm still short of the publication quota. A paper I've been trying to publish since April 2014 has since then accumulated more than 200 commits to the LaTeX source and is currently under yet another peer review.

I don't want to make any big plans for 2017. I have plenty of ideas for interesting projects and a backlog of (non-academic) things I would like to write about. However, given how much work I still have ahead of me before my viva voce, I doubt much of that will happen this year.

Posted by Tomaž | Categories: Life | Comments »

AFG-2005 noise generation bug

09.10.2016 13:49

The noise output function in the GW Instek AFG-2005 arbitrary function generator appears to have a bug. If you set up the generator to output a sine wave at 10 kHz, and then switch to noise output using the FUNC key, you get output like this:

Noise output in frequency domain.

Noise output in time domain.

The red trace shows the spectrum of the signal from the signal generator using the FFT function of the oscilloscope. The yellow trace shows the signal in the time domain.

The noise spectrum looks flat and starts to roll off somewhere beyond 10 MHz. This is what you would expect for an instrument that is supposed to have a 20 MHz DAC. However, if you set the output to a sine wave at 1 kHz before switching to noise output, the signal looks significantly different:

Noise output in frequency domain, low sampling rate.

Noise output in time domain, low sampling rate.

These two later oscilloscope screenshots have been made using the same settings as the pair above.

This is obviously an error on the part of the signal generator. The setting for the sine wave output shouldn't affect the noise output. It looks like the DAC is now only set to around 4 MHz sampling rate. Probably it has been switched to a lower sampling rate for the low-frequency sine wave output and the code forgot to switch it back for the noise function.

This behavior is perfectly reproducible. If you switch back to sine wave output, increase the frequency to 10 kHz or more and switch to noise output, the DAC sampling rate is increased again. Similarly, if you set a 100 Hz sine wave, the DAC sampling rate is set to 400 kHz. As far as I can see there is no mention of this in the manual and you cannot set the sampling rate manually. The FREQ button is disabled in Noise mode and there is no indication on the front panel about which sampling rate is currently used.

I've been writing about the AFG-2005 before. It's an useful instrument, but things like this make it absolutely necessary to always have an oscilloscope at hand to verify that the generator is actually doing what you think it should be doing.

Posted by Tomaž | Categories: Life | Comments »

A naive approach to rain prediction

02.10.2016 19:49

Ever since the Slovenian environment agency started publishing Doppler weather radar imagery I've been a regular visitor on their web site. For the last few years I try to use my bicycle for my daily commute as much as possible. However, I'm not such a hard-core fan of biking that I would do it in any weather. The animated map with the recent rainfall estimate history helps very much with that: it's quite easy to judge by eye whether there will be rain in the next 30 minutes or so and hence whether to seek alternative modes of transportation.

Some time ago I had a more serious (or should I say scientific) use for weather data, related to one of the projects at the Institute. Gašper helpfully provided me with some historical archives then and I also started collecting images myself straight from ARSO website. That project came and went, but the amount of data on my disk kept growing. I've been continuously tempted to do something interesting with it. I've previously written what the data seems to reveal about the radar itself.

Most of all, I've been playing with the idea of doing that should-I-take-the-bus prediction automatically. Obviously I'm not a weather expert, so my experiments were very naive from that perspective. For instance, a while ago I tried estimating an optical flow field from the apparent movement of regions with rain and then using that to predict their movement in the near future. That didn't really work. Another idea I had was to simply dump the data into a naive Bayesian model. While that also didn't work to any useful degree as far as prediction is concerned, it did produce some interesting results worth sharing.

What I did was model rain at any point in the radar image (x, y) and time t as a random event:

R_{x,y,t}

To determine whether the event happened or not from the historical data, I simply checked whether the corresponding pixel was clear or not - I ignored the various rates of rainfall reported. To calculate the prior probability of rain on any point on the map, I did a maximum-likelihood estimation:

P(R_{x,y}) = \frac{n_{x,y}}{N}

Here, nx,y is number of images where the point shows rain and N is the total number of images in the dataset.

I was interested in predicting rain at one specific target point (x0, y0) based on recent history of images. Hence I estimated the following conditional probability:

P(R_{x_0,y_0,t+\Delta t}|R_{x,y,t})

This can be estimated from historical data in a similar way as the prior probability. In other words, I was interested in how strongly is rain at some arbitrary point on the map x,y at time t related to the fact that it will rain at the target point at some future time tt. Are there any points on the map that, for instance, always show rain 30 minutes before it rains in Ljubljana?

The video below shows this conditional probability for a few chosen target points around Slovenia (marked by small white X). Brighter colors show higher conditional probability and hence stronger relation. The prior probability is shown in lower-right corner. In the video, the time difference runs in reverse, from 4 hours to 0 in 10 minute steps. For each point, the animation is repeated 4 times before moving to the next point.

The estimates shown are calculated from 62573 radar images recorded between July 2015 and September 2016. Since the format of the images has been changing over time it's hard to integrate older data.

As you might expect, when looking several hours into the future, there is very little relation between points on the map. All points are dull red. If it rains now in the east, it might rain later in Ljubljana. Similarly, if it rains in the west. There's no real difference - the only information you get is that it's generally rainy weather in the whole region.

When you decrease the time difference, you can see that nearby points are starting to matter more than those far away. Brighter colors concentrate around the target. Obviously, if it rains somewhere around Ljubljana, there's a higher chance it will shortly rain in Ljubljana as well. If you note the color scale though, it's not a particularly strong relation unless you're well below one hour.

What's interesting is that for some cities you can see that rain more often comes from a certain direction. Around the coast and in Notranjska region, the rain clouds seem to mostly move in from the Adriatic sea (lower-left corner of the map). This seems to fit the local saying you can sometimes hear, that the "weather comes in from the west". In the mountains (top-left), it seems to come from the north. All this is just based on one year of historical data though, so it might not be generally true over longer periods.


Of course, such simple Bayesian models are horribly out-of-fashion these days. A deep learning convolutional neural network might work better (or not), but alas, I'm more or less just playing with data on a rainy weekend and trying to remember machine learning topics I used to know. There's also the fact that ARSO themselves now provide a short-term rain prediction through an application on their website. It's not the easiest thing to find (Parameter Selection - Precipitation in the menu and then Forecast on the slider below). I'm sure their models are way better than anything an amateur like me can come up with, so I doubt I'll spend any more time on this. I might try to add the forecast to ARSO API at one point though.

Posted by Tomaž | Categories: Life | Comments »

A thousand pages of bullet journal

12.08.2016 17:27

A few weeks ago I filled the 1000th page in my Bullet Journal. Actually, I don't think I can call it that. It's not in fact that much oriented around bullet points. It's just a series of notebooks with consistently numbered, dated and titled pages for easy referencing, monthly indexes and easy-to-see square bullet points for denoting tasks. Most of the things I said two years ago still hold, so I'll try not to repeat myself too much here.

A thousand pages worth of notebooks.

Almost everything I do these days goes into this journal. Lab notes, sketches of ideas, random thoughts, doodles of happy ponies, to-do lists, pages worth of crossed-out mathematical derivations, interesting quotes, meeting notes and so on. Writing things down in a notebook often significantly clears them up. Once I have a concise and articulated definition of a problem, the solution usually isn't far away. Pen and paper helps me keep focus at talks and meetings, much like an open laptop does the opposite.

Going back through past notes gives a good perspective on how much new ideas depend on the context and mindset that created them. An idea for some random side-project that seems interesting and fun at first invariably looks much less shiny and worthy of attention after reading through the written version a few days or weeks later. I can't decide though whether it's better to leave such a thing on paper or hack together some half-baked prototype before the initial enthusiasm fades away.

The number of pages I write per month appears to be increasing. That might be because I settled on using cheap school notebooks. I find that I'm much more relaxed scribbling into a 50 cent notebook than ruining a 20€ Moleskine. Leaving lots of whitespace is wasteful, but helps a lot with readability and later corrections. Yes, whipping out a colorful children's notebook at an important meeting doesn't look very professional. Then again, most people at such meetings are too busy typing emails into their laptops to notice.

Number of pages written per month.

As much as it might look like a waste of time, I grew to like the monthly ritual of making an index page. I like the sense of achievement it gives me when I look back at what I've accomplished the previous month. It's also an opportunity for reflection. If the index gets hard to put all on one page, that's a good sign that the previous month was all too fragmented and that too many things wanted to happen at once.

The physical nature of the journal means that I can't carry the whole history with me at all times. That is also sometimes a problem. It is an unfortunate feature of my line of work that it is not uncommon for people to want to have unannounced meetings about a topic that was last discussed half a year ago. On the other hand, saying that I don't have my notes on me at that moment does present an honest excuse.

Indexes help, but finding things can be problematic. Then again, digital content (that's not publicly on the web) often isn't much better. I commonly find myself frustratingly searching for some piece of code or a document I know exists somewhere on my hard disk but can't remember any exact keyword that would help me find it. I considered making a digital version of monthly indexes at one point. I don't think it would be worth the effort and it would destroy some of the off-line quality of the notebook.

As I mentioned previously, gratuitous cross-referencing between notebook pages, IPython notebooks and other things does help. I tend not to copy tasks between pages, like in the original Bullet Journal idea. For projects that are primarily electronics related though, I'm used to keeping a separate folder with all the calculations and schematics, a habit I picked up long ago. There are not many such projects these days, but I did on one occasion photocopy pages from the notebook. I admit that made me feel absolutely archaic.

Posted by Tomaž | Categories: Life | Comments »

Oxidized diodes

08.08.2016 20:41

Out of curiosity, I salvaged these three LEDs from an old bicycle light that stopped working. Rain must have gotten inside it at one point, because the copper on the PCB was pretty much eaten away by the electrolysis. The chip that blinked the light was dead, but LEDs themselves still work.

Three LEDs damaged by water compared to a new one.

It's interesting how far inside the bulb the steel leads managed to oxidize. You can see that the right hand lead (anode) on all LEDs is brown all the way to the top, where a bond wire connects it to the die. I would have thought that the epoxy makes a better seal. For comparison, there's a similar new LED on the right. It's also interesting that the positive terminal was more damaged than the negative. On the right-most LED the terminal was actually eaten completely through just outside the bulb (although poking the remains with a needle still lights up the LED, so obviously the die itself was not damaged).

Posted by Tomaž | Categories: Life | Comments »

Newsletters and other beasts

23.07.2016 10:23

It used to be that whenever someone wrote something particularly witty on Slashdot, it was followed by a reply along the lines of "I find your ideas intriguing and would like to subscribe to your newsletter". Ten-odd years later, it seems like the web took that old Simpsons joke seriously. These days you can hardly follow a link on Hacker News without having a pop-up thrown in your face. Most articles now end with a plea for an e-mail address, and I've even been to real-life talks where the speakers continuously advertised their newsletter to the audience.

Recently I've been asked several times why I didn't support subscriptions by e-mail, like every other normal website. The short answer is that I keep this blog in a state that I wish other websites I visit would adopt. This means no annoying advertisements, respecting your privacy by not loading third-party Javascript or tracking cookies, HTTPS and IPv6 support, valid XHTML... and good support for the Atom standard. Following the death of Google Reader, the world turned against RSS and Atom feeds. However, I still find them vastly more usable than any alternative. It annoys me that I can't follow interesting people and projects on modern sites like Medium and Hackaday.io through this channel.

Twitter printer at 32C3

That said, you now can subscribe by e-mail to my blog, should you wish to do so (see also sidebar top-right). The thing that finally convinced me to implement this was hearing that some of you use RSS-to-email services that add their own advertisements to my blog posts. I did not make this decision lightly though. I used to host mailing lists and know what an incredible time sink they can be, fighting spam, addressing complaints and so on. I don't have that kind of time anymore, so using an external mass-mailing service was the only option. Running my own mail server in this era is lunacy enough.

Mailchimp seems to be friendly, so I'm using that at the moment. If turns to the dark side and depending how popular the newsletter gets, I might move to some other service - or remove it altogether. For the time being I consider this an experiment. It's also worth mentioning that while there are no ads, Mailchimp does add mandatory tracking features (link redirects and tracking pixels). Of course, it also collects your e-mail address somewhere.

Since I'm on the topic of subscriptions and I don't like writing meta posts like this, I would also like to mention here two ways of following my posts that are not particularly well known: if you are only interested in one particular topic I write about, you can search for it. Search results page has an attached Atom feed you can follow that only contains posts related to the query. If you on the other hand believe that Twitter is the new RSS, feel free to follow @aviansblog (at least until Twitter breaks the API again).

Posted by Tomaž | Categories: Life | Comments »