Dropping the "publicsuffix" Python package

02.12.2019 10:50

I have just released version 1.1.1 of the publicsuffix Python package. Baring any major bugs that would affect some popular software package using it, this will be the last release. I've released v1.1.1 because I received a report that a bug in publicsuffix package is preventing installation of GNU Mailman.

In the grand scheme of things, it's not a big deal. It's a small library with a modest number of users. I haven't done any work, short of answering mail about it, since 2015. Drop-in alternatives exist. People that care strongly about the issues I cover below have most likely already switched to one of the forks and rewrites that popped up over the years. For those that don't care, nothing will change. The code still works and the library is still normally installable from PyPi. Debian package continues to exist. The purpose of this post is more to give some closure and to sum up a few mail threads that started back in 2015 and never reached a conclusion.

Screenshot of the publicsuffix package page on PyPi.

I've first released the publicsuffix library back in 2011, two employers and a life ago. Back then there was no easily accessible Python implementation of Mozilla's Public Suffix List. Since I needed one for my work, I've picked up a source file from an abandoned open source project on Google Code (which was just being abandoned by Google around that time). I did some minor work on it to make it usable as a standalone library and published it on PyPi.

I've not used publicsuffix myself for years. Looking back, most of my open source projects that I still maintain seem to be like that. Even though I don't use them, I feel some obligation to do basic maintenance on them and answer support mail. If not for other reasons, then out of a sense that I should give back to the body of free software that I depend so much on in my professional career. Some technical problems are also simply fun to work on and most of the time there's not much pressure.

However one thing that was a source of long discussions about publicsuffix is the way the PSL data is distributed. I've written previously about the issue. In summary, you either distribute stale data with the code or fetch an up-to-date copy via the network, which is a privacy problem. These two are the only options possible and going with one or the other or both was always going to be a problem for someone. I hate software that phones home (well, phones Mozilla in this case) as much as anyone, but it's a problem that me as a mere maintainer of a Python library had no hope of solving, even if I got CC'd in all the threads discussing it.

The Public Suffix List is a funny thing. Ideally, software either should not care about the semantic meaning of domain names or this meaning should be embedded in the basic infrastructure of the Internet (e.g. DNS or something). But alas we don't live in either of those worlds and hence we have a magic text file that lives on a HTTP server somewhere and some software needs to have access to it if it wants to do its thing. No amount of worrying on my part was going to change that.

Screenshot of publicsuffix forks on GitHub.

The other issue that sparked at least one fork of publicsuffix was the fact that I refused to publish the source on GitHub. Even tough there are usually several copies of the publicsuffix code on the GitHub at any time, none of them are mine. I was instead hosting my own git repo and was accepting bug reports and other comments only over email.

Some time ago already GitHub became synonymous with open source. People simply expect a PyPi package to have a GitHub (or GitLab, or BitBucket) point-and-click interface somewhere on the web. The practical problem I have with that is that it hugely increases the amount of effort I have to spend on a project (subjectively speaking - keep in mind this is something I do in my free time). Yes, it makes it trivial for someone to contribute a patch. However in practice I find that it does not result in greater quantity of meaningful patches or bug reports. What it does do is create more work for me dealing with low-effort contributions I must reject.

I'm talking about a daunting asymmetry in communication. Writing two sentences in a hurry in a GitHub issue or pushing a bunch of untested code my way in a pull request can take all of a minute for the submitter. On the other hand, I don't want to discourage people from contributing to free software and I know how frustrating it can be to contribute to open source projects (see my post about drive by contributions). So I try to take some time to study the pull request and write an intelligible and useful answer. However this is simply not sustainable. Looking back I also seem to often fail at not letting my frustration show through in my answer. Hence I feel like requiring contributors to at least know how to use git format-patch and write an email forms a useful barrier to entry. It prevents frustration at both ends and I believe that for a well thought-out contribution, the overhead of opening a mail client should be negligible.

Of course, if the project is not officially present on GitHub you get the current situation, where multiple public copies of the project still exist on GitHub, made by random people for their own use. These copies often keep in my contact details and don't obviously state that the code has been modified and/or is not related to the PyPi releases. This causes confusion, since code on GitHub is not the same as the one on PyPi. People also sometimes reuse version numbers for their own private use that conflict with version numbers on PyPi and so on and so on. It is kind of a damned if you do and damned if you don't situation really.


How can I sum this up? I've maintained this software for around 8 years, well after I left the company for which it was originally developed. During that time people have forked and rewrote it for various, largely non-technical reasons. That's fine. It's how free software is supposed to work and my own package was based on another one that got abandoned. I might still be happy to work on technical issues, but the part that turned out much more exhausting than working on the code was dealing with the social and ideological issues people had with it. It's probably my failing that I've spent so much thought on those. In the end, my own interests have changed as well during that time and finally letting it go does also feel like a stone off my shoulders.

Posted by Tomaž | Categories: Code | Comments »

On reliability of pogo pins

26.11.2019 20:42

A bit over a year ago I designed and built a device for testing assembled printed circuit boards as they come off the assembly line. While I'm not new to electronic test fixtures, this was the first time I've used the bed-of-nails approach: the test jig has a number of spring-loaded pogo pins that make contact with various test pads on the device-under-test (DUT). This setup has now made thousands of cycles and the device proved itself to be capable of detecting a large variety of defects, without doubt preventing many expensive debugging sessions.

However one problem that has been constantly troubling this setup since the beginning is its unreliability. Even after a lot of fussing around with various adjustments, the procedure still has an abysmal false error rate compared to the actual rate of manufacturing defects. In many cases, the operator must remove, re-seat the DUT and restart the test several times before the test will signal a pass. Such test repetitions obviously cause a lot of frustration, decrease the confidence in the testing procedure and significantly lengthen a test that would otherwise take only a few moments. All evidence, like the fact that detected defect types appear completely random and that most test failures disappear when re-seating the DUT, firmly points towards the pogo pins as the cause.

I was surprised at this outcome, since I've never heard about bad contacts being such a problem with pogo pins. There are quite a few blog posts and basic tutorials around about the pogo pin test jigs. Hacker Noon mentions that getting the fine mechanical details correct can be tricky. The Big Mess o' Wires blog says that their test board only worked reliably after three iterations of the design. Thom wrote that they didn't have many issues with contacts on their test jig. It seems that reliability is not a common problem people have with pogo pins, once initial mechanical problems have been ironed out.

Pogo pins mounted on a test fixture.

My bed of nails setup is shown above. It uses P75-type pogo pins - a widely available, cheap variant of uncertain origin. For example, they are sold by Adafruit. The whole bed has 21 pins and uses a combination of needle heads (P75-B1) and cupped heads (P75-A1). There was not enough PCB space on the DUT for all the required test pads so I used cupped head pins to mate with the underside of THT connector pins. P75 pogo pins seem to use exposed steel for the head and plunger (they are slightly magnetic) and only have the gold plating on the bottom body part. I'm not using the mounting sleeves. The pin bodies are directly soldered to the test jig PCB.

The mechanical parts have been removed in the photograph above, but you can get an idea of how they look from the CAD render below. During the test the DUT is securely fixed onto the pins using a clamp, centering pins and a frame. This setup is similar to the one described by Hacker Noon. The difference is that I'm using two parallel PCB boards to position the pins instead of 3D printed parts. The setup was designed so that the pogo pins only compress to approximately half of their 100 mil travel. The mechanical frame carries most of the clamping force.

The boards I'm testing have a lead-free HASL finish and there is no solder paste applied to the test pads. This means that test pads might be sensitive to oxidation. However that shouldn't be a problem since the test is applied shortly after production. It's also worth mentioning that I'm testing an analog circuit. Compared to purely digital tests these are more sensitive to the resistance between the test fixture and the DUT.

CAD drawing of the test device with the bed-of-nails.

Since I have a lot of data collected from the test device I thought statistical analysis might shed some light on the reliability problem. If not directly showing a way to improve the existing device, perhaps it would at least give me some idea what can be expected from pogo pins when designing future test fixtures.

The first thing I was interested in was the resistance between a pogo pin on the test fixture and its corresponding test pad on the DUT. The test procedure was not designed to directly measure this. Fortunately however I found a way to estimate test point resistance for two specific pogo pins (out of 21). I calculated their resistances from certain other measurements I took during the test procedure. Of course, this was not as good as a direct measurement and the estimate is still affected somewhat by variations in some components on the DUT, the test device and resistances of other test points. A Monte Carlo simulation showed an error in the resistance estimate of less than 10 mΩ due to these effects.

As luck would have it, one of the pogo pins I was able to estimate was using the needle head while the second one was using the cupped head. This resulted in the following two histograms of resistances to two test points. They show how commonly each of the two test points exhibited a certain resistance over thousands of matings with the DUT:

Histogram of resistances through a needle-head pogo pin.

Histogram of resistances through a cupped-head pogo pin.

Different colors show data from different DUT production batches. Overall, you can see that most commonly the connection resulted in a resistance of around 0.1 Ω and majority of connections were below 0.5 Ω. This is pretty good, even if somewhat above the 50 mΩ rated contact resistance for this type of pins. The cupped head pin showed less variance than the needle head. Still, the values show much higher variance than the estimated 10 mΩ error, which gives some confidence that this is actually due to changing contact resistances of the pogo pins.

However, one thing that is not visible on these plots is the fact that some connections resulted in estimates well over 1 Ω (approximately 10% for the needle head and 6% for the cupped head). I could also only produce this estimate when the test progressed to the point where some voltage measurements have been made (which depend on a reasonably good contact over 4 pogo pins for needle head pin and 2 pogo pins for cupped head pin). Hence test runs where these measurements were not taken are not included in the histograms above.

So what about these failed attempts? One way to show them is the number of test repetitions that a DUT had to undergo before a test first passed. Using records of thousands of tests, the following histogram emerged:

Number of test repetitions required before the first pass.

Again, the colors show data from different production batches. Overall, approximately 60% of DUTs passed on the first test attempt. A bit above 20% passed on the second and around 10% on the third attempt. You can also see some differences in batches. For example, the batch shown in red was particularly bad and more DUTs required a second repetition than passed the first test. Number of DUTs that failed the test 10 times or more is very small - mostly these are the DUTs that actually had a manufacturing defect and didn't fail due to a false reading on the test fixture.

The histogram shows a nicely exponential characteristic - exactly what you would expect if each test repetition was a random event with a Ppass probability of succeeding. From the data I can estimate that:

P_{pass} = 59.4\%

If I further assume that a test will succeed if all pogo pins contact successfully, and that each of the 21 pogo pin contacts is an independent random event by itself, we can calculate the a probability Pfail-pin that a pogo pin will fail to make a good contact:

P_{fail-pin} = 1 - \sqrt[21]{P_{pass}} \approx 2.4\%

Using this model, I can back predict the probability that a DUT will pass the test after N test repetitions:

P_{pass-after-N-repetitions} = (1 - P_{pass})^{N-1} \cdot P_{pass}

This model fits almost perfectly with the measured histogram, as you can see on the picture below. The predicted number of test repetitions before first pass (red) is laid over the histogram of measurements (gray).

Comparing the model for test repetitions to measurements.

The model also fits reasonably well with number of cases where I've estimated test point resistances above 1 Ω. This might be a bit handwavy since it's hard to see how different failures would affect the results. For the needle-head test point I've seen approximately 10% of cases where resistance was above 1 Ω. This fits well with the fact that 4 points needed to be well connected for the measurement to be accurate and 2.4% failure rate for the connections:

P_{fail} = 1 - (1 - P_{fail-pin})^{N_{pins}} = 1 - (1 - 2.4\%)^4 = 9.3\%

Similarly for the cupped pin measurement, where I've seen 6% of measurements above 1 Ω and required 2 points to be well connected:

P_{fail} = 1 - (1 - 2.4\%)^2 = 4.7\%

In conclusion, my data shows that individual pogo pins seem to have approximately 2.4% chance of not mating correctly with their test points. When they do contact correctly, they usually show a reasonably low resistance of approximately 100 mΩ between the pin and the test pad, with worst cases being less than 500 mΩ. It's not clear from the data what is causing such a high rate of unsuccessful connections. Since the failure rate varies from batch to batch, this suggests that at least part of it is related in some way to the production process (for example, oxide or flux residue on the test pads). On the other hand, it's also possible that the pins themselves are responsible for these failures. The bad contact might in fact be between the plunger and pin body, not between the head and the test pad. In that case it might be worth experimenting with the more expensive pogo pins that have gold plated heads and plungers.

Posted by Tomaž | Categories: Analog | Comments »

ZX81 LPRINT bug and software archaeology

04.11.2019 19:07

By some coincidence I happened to stumble upon a week-old, unanswered question posted to Hacker News regarding a bug in Sinclair BASIC on a Timex Sinclair 1000 microcomputer. While I never owned a TS1000, the post attracted my interest. I've studied ZX81, an almost identical microcomputer, extensively when I was doing my research on Galaksija. It also reminded me of a now almost forgotten idea to write a post on some obscure BASIC bugs in Galaksija's ROM that I found mentioned in contemporary literature.

ZX81 exhibited at the Frisk festival.

The question on Hacker News is about the cause of a bug where the computer, when attached to a printer, would print out certain floating point numbers incorrectly. The most famous example, mentioned in the Wikipedia article on Timex Sinclair 1000, is the printout of 0.00001. The BASIC statement:

LPRINT 0.00001

unexpectedly types out the following on paper:

0.0XYZ1

This bug occurs both on Timex Sinclair 1000 as well as on Sinclair ZX81, since both computers share the same ROM code. Only the first zero after the decimal point is printed correctly while the subsequent zeros seem to be replaced with random alphanumeric characters. The non-zero digit at the end is again printed correctly. Interestingly, this only happens when using the LPRINT (line-printer print) statement that makes a hard-copy of the output on paper using a printer. The similar PRINT statement that displays the output on the TV screen works correctly (you can try it out on JtyOne's Online Emulator).

The cause of the bug lies in the code that takes a numerical value in the internal format of the BASIC's floating point calculator and prints out individual characters. One particular part of the code determines the number of zeros after the decimal point and uses a loop to print them out:

;; PF-ZEROS
L16B2:  NEG                     ; Prepare number of zeros
        LD      B,A             ; to print in B.

        LD      A,$1B           ; Print out character '.'
        RST     10H             ; 

        LD      A,$1C           ; Prepare character '0' 
				; to print out in A.

;; PF-ZRO-LP
L16BA:  RST     10H             ; Call "print character" routine
        DJNZ    L16BA           ; and loop back B times.

(This assembly listing is taken from Geoff Wearmouth's disassembly. Comments are mine.)

The restart 10h takes a character code in register A and either prints it out on the screen or sends it to the printer. Restarts are a bit like simple system calls - they are an efficient way to call an often-used routine on the Z80 CPU. The problem lies in the fact that this restart doesn't preserve the contents of the A register. It does preserve the contents of register B and other main registers through the use of the EXX instruction and the shadow registers, however the original contents of A is lost after the call returns.

Since the code above doesn't reset the contents of the A register after each iteration, only the first zero after the decimal point is printed correctly. Subsequent zeros are replaced with whatever was junk left in the A register by the 10h restart code. Solution is to simply adjust the DJNZ instruction to loop back two bytes earlier, to the LD instruction, so that the character code is stored to A in each iteration. You can see this fix in Geoff's customized ZX81 ROM, or in Timex Sinclair 1500 ROM (see line 3835 in this diff between TS1500 and TS1000).

This exact same code is also used when displaying numbers on the TV screen, however in that case it works correctly. The reason is that when set to print to screen, printing character 0 via the 10h restart actually preserves the contents of register A. Looking at the disassembly I suspect that was simply a lucky coincidence and not a conscious decision by the programmer. Any code calling 10h doesn't know whether the printer or the screen is used, and hence must assume that A isn't preserved anyway.


Of course, I'm far from being the first person to write about this particular Sinclair bug. Why then does the post on Hacker News say that there's little information to be found about it? The Wikipedia article doesn't cite a reference for this bug either.

It turns out that during my search for the answer, the three most useful pages were no longer on-line. Paul Farrow's ZX resource centre, S. C. Agate's ZX81 ROMs page and Geoff Wearmouth's Sinclair ROM disassemblies are wonderful historical resources that must have taken a lot of love and effort to put together. Sadly, they are now only accessible through the snapshots on the Internet Archive's Wayback Machine. If I wouldn't know about them beforehand, I probably wouldn't find them now. For the last one you even need to know what particular time range to look at on Archive.org, since the domain was taken over by squatters and recent snapshots only show ads (incidentally, this is also the reason why I'm re-hosting some of its former content).

I feel like we can still learn a lot from these early home computers and I'm happy that questions about them still pop-up in various forums. This LPRINT bug seems to be a case of a faulty generalization. It's a well known type of a mistake where the programmer wrongly generalizes an assumption (10h preserves A) that is in fact only true in a special case (displaying character on screen). History tends to repeat itself and I believe that many of the blunders in modern software wouldn't happen if software developers would be more aware of the history of their trade.

It's sad that these old devices are disappearing and that primary literature sources about them are hard to find, but I find it even more concerning that now it seems also these secondary sources are slowly fading out from general accessibility on the web.

Posted by Tomaž | Categories: Code | Comments »

Experiences in product certification

24.10.2019 9:45

Yesterday I was invited to give a presentation at a seminar on procedures for product certification. My former colleagues at the Department of Communication Systems at the Jožef Stefan Institute invited three companies to share their experiences in getting electronics products to European market. We discussed compliance with the CE mark, testing for safety and electromagnetic compatibility standards and various other approvals that are required before you can put mass-produced electronics on the shelves.

"Lessons learned" slide from the certification seminar.

In my presentation (slides are here) I've discussed my view of the two certification cycles I've participated in at Klevio during roughly the last year and a half. I did a short intro about the company and products and then listed what we chose to certify and how. I didn't do any general intro into the certification procedures and individual measurements. This in the end turned out just fine, because others did it better than I could. Most of the time I spent talking about purely practical lessons we learned on how to best prepare for the task. Looking back at it now, I feel like most of my advice boils down to having a good understanding of what is involved and not basing your expectations purely on certification lab's sales pitches.

I wasn't sure how much time I would have and what the interest of the audience was. Because of that I've put the details on debugging specific compliance problems into a separate part of the presentation. I've ended up doing that part of the talk as well. I discussed a problem we had with an ESD test temporarily disabling an audio codec IC and a problem with radiated emissions that took 3 months to debug and led to some pretty significant changes to one of the DC-DC converters.

I wanted to talk more about some interesting home-brew methods of estimating radiated emissions. I spent a lot of time researching and experimenting with them when I was debugging Klevio's compliance problems. In the end I realized that I could spend a whole talk just on that topic. It also turned out that none of those methods were actually useful in finding a solution, so it didn't make much sense to do more than just list them out. For more info on that, here's an article I found useful on making magnetic loop probes. The idea for the common-mode current probe based on a ferrite ring is from the Application Note AN045 by Richtek. The SDR-based method was my own idea based on my previous research on spectrum sensing and I might eventually do a longer write up on that in the future.

In the end, it was interesting to compare notes and what others have learned solving similar problems. I found that even though our EMI problem took longer to solve than others presented at the seminar, Klevio's experience didn't differ much in regard to timelines or certification lab practicalities. It seems almost everyone stumbles upon some problems during certifications, and while specific issues are unique, the biggest obstacle to finding a solution seems pretty universal: reproducing the problem outside of the certification lab.

Posted by Tomaž | Categories: Life | Comments »

Some more notes on Denon RCD-M41DAB

19.10.2019 16:13

Back in January I bought a new micro Hi-Fi after the transformer in my old one failed and I couldn't repair it. Since then I've been pretty happy using Denon RCD-M41DAB for listening to music and radio. I previously did some measurements on its output amplifier and found it meets its ratings in regards to power and distortion. In any case, that was more to satisfy my electrical engineering curiosity and I'm not particularly sensitive to reproduction quality. However after extended use two problems did pop up. Since I see this model is still being sold, I thought I might do a short follow up.

Denon RCD-M41DAB

The first thing is that when streaming audio to the Hi-Fi over Bluetooth there are about 2 seconds of lag. It looks as if there's some kind of an audio processing buffer inside the RCD-M41DAB, or it might simply be the choice of the codec used for wireless transfer. When listening to music the delay isn't that annoying. For example, when clicking "next track" or changing volume on the music player on the phone it takes about 2 seconds before you can actually hear the change. However this lag does make a Bluetooth-connected RCD-M41DAB completely useless as an audio output when playing videos or anything interactive. The lag happens on iOS and Android and I'm pretty sure this is not something related to my smartphone, since when I'm using Bluetooth-connected headphones, there is no discernible lag there.

The other annoyance is that the CD player has problems reading some CDs in my collection. Apparently it reads the table of contents fine, because the number of tracks is displayed correctly. However it has problems seeking to tracks. For example, one disc won't play any tracks at all and I can just hear the head continuously seeking. Another disc won't start playing the first track, but if I skip it manually, player will find and play the second one just fine. All such discs play normally in the CD drive in my desktop computer and other CD players I've tried. It still might be that the discs have some kind of an error in them or have bad reflectivity (all of the problematic ones are kind of niche releases I bought from Bandcamp), but other players apparently are able to work around it.

Posted by Tomaž | Categories: Life | Comments »

Specializing C++ templates on AVR pins

09.10.2019 19:56

AVR is a popular processor core for microcontrollers, probably most well known from the ATmega chips on the classic Arduino boards. There's an excellent, free software C and C++ compiler toolchain available for them, which also usually sits behind the well-known Arduino IDE. AVR is a Harvard architecture, which means that code and data memory (i.e. flash and RAM) have separate address spaces. On these small, 8-bit chips there is very little RAM space (can be as little as 128 bytes on ATtiny devices). On the other hand, the amount of flash is usually much less restrictive.

Sometimes you want the microcontroller to drive multiple instances of the same external peripheral. Let's say for example this peripheral uses one GPIO pin and two instances differ only in the pin they are connected to. This means that the code in the microcontroller for peripheral 1 differs from code for peripheral 2 only by the address of the special function register (SFR) it uses and a bit mask. A naive way of implementing this in clean C++ would be something like the following:

#include <avr/io.h>

class DIO {
	public:
		PORT_t * const port;
		const int pin_bm;

		DIO(PORT_t* port_, int pin_bm_) 
			: port(port_), pin_bm(pin_bm_) {}

		void setup(void) const
		{
			port->DIR |= pin_bm;
		}

		// other methods would go here ...
};

DIO dio1(&PORTB, PIN4_bm);
DIO dio2(&PORTB, PIN5_bm);

int main(void)
{
	dio1.setup();
	dio2.setup();

	// other code would go here ...

	while (1);
}

Note that I'm using the nice PORT_t struct and the bit mask constants provided by avr-gcc headers instead of messing around with register addresses and such.

Unfortunately, this isn't optimal. Even though the class members are declared as const, compiler stores them in RAM. This means that each instance of DIO class costs 4 precious bytes of RAM that, baring a stray cosmic particle, will never ever change during the course of the program.

Compiling the code above for the ATtiny416 with avr-gcc 5.4.0 confirms this:

Program Memory Usage  : 204 bytes   5,0 % Full
Data Memory Usage     : 8 bytes   3,1 % Full

We could store member variables as data in program space. This is possible by using some special instructions and variable attributes (PROGMEM, pgm_read_byte() etc.), but it's kind of ugly and doesn't play nicely with the C language. Another way is to actually have copies of the binary code for each peripheral, but only change the constants in each copy.

Obviously we don't want to have multiple copies in the source as well. In C the usual approach is to do something rude with C preprocessor and macros. Luckily, C++ offers a nicer way via templates. We can make a template class that takes non-type parameters and specialize that class on the register and bitmap it should use. Ideally, we would use something simple like this:

template <PORT_t* port, int pin_bm> class DIO {
	public:
		void setup(void) {
			port->DIR |= pin_bm;
		}
};

DIO<(&PORTB), PIN4_bm> dio1;
DIO<(&PORTB), PIN5_bm> dio1;

Sadly, this does not work. Behind the scenes, PORTB is a macro that casts a constant integer to the PORT_t struct via a pointer:

#define PORTB    (*(PORT_t *) 0x0620)

&PORTB evaluates back to an integral constant, however the compiler isn't smart enough to realize that and balks at seeing a structure and a pointer dereference:

main.cpp(8,7): error: a cast to a type other than an integral or enumeration type cannot appear in a constant-expression
DIO<(&PORTB), PIN4_bm> dio1;
main.cpp(8,7): error: '*' cannot appear in a constant-expression
main.cpp(8,7): error: '&' cannot appear in a constant-expression

I found a thread on the AVR freaks forum which talks about this exact problem. The discussion meanders around the usefulness of software abstractions and the ability of hardware engineers to follow orders, but fails to find a solution that does not involve a soup of C preprocessor macros.

Fortunately, there does seem to exist an elegant C++ solution. The compiler's restriction can be worked around without resorting to preprocessor magic and while still using the convenient constants defined in the avr-gcc headers. We only need to evaluate the macro in another expression and do the cast to PORT_t inside the template:

template <int port_addr, int pin_bm> class DIO {
	public:
		void setup(void) {
			port()->DIR |= pin_bm;

		}

		static PORT_t* port()
		{
			return (PORT_t*) port_addr;
		}
};

static const int PORTB_ADDR = (int) &PORTB; 
DIO<PORTB_ADDR, PIN4_bm> dio1;
DIO<PORTB_ADDR, PIN5_bm> dio2;

This compiles and works fine with avr-gcc. As intended, it uses zero bytes of RAM. Interestingly enough, it also significantly reduces the code size, something I did not expect:

Program Memory Usage  : 108 bytes   2,6 % Full
Data Memory Usage     : 0 bytes   0,0 % Full

I'm not sure why the first version using member variables produces so much more code. A brief glance at the disassembly suggests that the extra code comes from the constructor. I would guess that using a code-free constructor with initializer lists should generate minimal extra instructions. Apparently that is not so here. In any case, I'm not complaining. I'm also sure that in a non-toy example, the template version will use more flash space. After all, It has to duplicate the code for all methods.

You might also wonder how Arduino does this with the digitalWrite() function and friends. They store a number of data tables in program memory and then have preprocessor macros to read them out using pgm_read_byte(). As I mentioned above, this works, but isn't nicely structured and it's pretty far from clean, object-oriented C++ programming.

Posted by Tomaž | Categories: Code | Comments »

Voltage divider cheat sheet

16.09.2019 19:46

The two most common electrical measurements I do these days is measuring output impedance of a source and input impedance of a load. I don't have any special equipment for that. I'm not doing it so often that it would warrant buying or making a specialized tool. Since it's all either DC or around audio frequencies and I'm not interested in very precise values it's a really simple measurement. I usually just use an oscilloscope and whatever else is available around the shop to use as reference loads and sources.

Deriving unknown impedances or resistances from the voltage measurements and reference values is straightforward. It only takes a minute and I must have done it a hundred times at this point by inverting the voltage divider formula. I still don't know the resulting equations by heart though. So to avoid doing it the hundred and first time and the odd mistake, I've made myself a nice cheat sheet to stick above my desk. It contains the formulas for the three most common measurements I make.

Voltage divider measurements cheat sheet

PDF version

The notes about the best accuracy refer to the selection of the reference impedance such that the result is least affected by errors in voltage measurements (which, when measuring amplitudes with an oscilloscope, is the largest source of error in my case). The selection is quite obvious, but I guess I added it there for the sake of rigor.

The cheat sheet is made in LaTeX using the circuitikz package for schematics. I'm mostly putting it here for my own future reference, but maybe someone else will find it useful.

Posted by Tomaž | Categories: Analog | Comments »

z80dasm 1.1.6

09.09.2019 20:32

z80dasm is a command-line disassembler for the Z80 CPU. I initially released it way back in 2007 when I was first exploring Galaksija's ROM and other disassemblers I could find didn't do a good enough job for me. Since then it accumulated a few more improvements and bug fixes as I received feedback and patches from various contributors.

Version 1.1.6 is a release that has been way overdue. Most importantly, it fixes a segmentation fault bug that several people have reported. The patch for the bug has actually been committed to the git repository since June last year, but somehow I forgot to bump the version and roll up a new release.

The problem appeared when feeding a symbol file that was generated by z80dasm back as input to z80dasm (possibly after editing some symbol names). This is something that the man page explicitly mentions is supported. However, when this was done together with block definitions, it caused z80dasm to segfault with a NULL pointer dereference. Some code didn't expect that the symbol automatically generated to mark a block start could already be defined via the input symbol file. Thanks to J. B. Langston for first sending me the report and analysis of the crash.

I took this opportunity to review the rest of the symbol handling code and do some further clean ups. It has also led me to implement a feature that I have been asked for in the past. z80dasm now has the ability to sort the symbol table before writing it out to the symbol file.

More specifically, there is now a --sym-order command-line option that either takes a default or frequency argument. Default leaves the ordering as it was in the previous versions - ordered by symbol value. Frequency sorts the symbol table by how frequently a symbol is used in the disassembly. The most commonly used symbols are written at the start of the symbol file. When first approaching an unknown binary, this might help you to identify the most commonly used subroutines.

Anyway, the new release is available from the usual place. See the included README file for build and installation instructions. z80dasm is also included in Debian, however the new release is not in yet (if you're a Debian developer and would like to sponsor an upload, please get in touch).

Posted by Tomaž | Categories: Code | Comments »

What is a good USB cable?

31.08.2019 20:36

In conclusion of my recent series on the simple matter of USB cable resistance, I would like to attempt the question of what is a good USB cable. After having a reliable measurement of a cable's resistance, the next obvious question is of course whether that resistance complies with the USB standard and whether such a cable is suitable for powering single-board computers like the Raspberry Pi. Claims that most cables don't comply with the standard are quite common whenever this topic is discussed. I'm by no means an expert on USB, but luckily USB Implementers Forum publishes the standards documents in their on-line library. I went in and studied some of the documents on the Cable and Connector Specification topic which, among other things, specify cable resistance.

I've started my reading with USB 2.0, because micro- and mini-USB cables I tested in my previous post are unlikely to be older than USB 2.0. The standard is now nearly 20 years old and over the years it seems to have received many revisions and updates. Hence it's hard to pick up a random cable from the pile and say with certainty with which document it should comply. In addition, I find that the text of the standard itself often isn't particularly clear. For example, the following text implicitly defines the maximum allowable cable resistance in the Universal Serial Bus Cables and Connectors Class Document, revision 2.0 from August 2007:

Cable Assembly Voltage Drop requirement for USB 2.0

Image by USB Implementers Forum

Initially I thought this means voltage drop over the pair of wires. As in, total voltage drop over VBUS and GND wires should be less than 125 mV at 500 mA (effectively 250 mΩ round-trip resistance). However the fact that most cables seem to be around 500 mΩ suggests that manufacturers read this as 250 mΩ per wire (500 mΩ round-trip).

A later document amends this definition somewhat and makes it clearer that the voltage drops are for each wire separately and that this voltage drop includes contact resistance. The following is from Universal Serial Bus 3.0 Connectors and Cable Assemblies Compliance Document, revision 1.0 draft from October 2010. Also note that both the measurement current and the allowable voltage drop were increased. The measurement must now be done at 900 mA, however maximum effective single-wire resistance is still 250 mΩ, same as in USB 2.0:

Cable Assembly Voltage Drop requirement for USB 3.0

Image by USB Implementers Forum

An even later document addresses cable compliance with older revisions of the standard. USB 3.1 Legacy Cable and Connector Revision 1.0 from 2017 contains this calculation:

IR drop at device calculation from a USB 3.1 document.

Image by USB 3.0 Promoter Group

This equation clearly shows that the 250 mΩ figure from the other documents is supposed to be combined from two 30 mΩ contact resistances and a 190 mΩ wire resistance. It also multiplies the voltage drop by two due to the round trip through both VBUS and GND wires.

USB type C specification tries to make this even clearer and comes with schematics that explicitly show where voltage drop must be measured. Since in USB type C you can have different types of cables that are rated for different currents, that standard only specifies maximum voltage drop. Note also that in type C the requirements for the VBUS line were relaxed compared to previous standards. Previously, for a cable delivering 1 A of current, the VBUS line must have had a maximum resistance of 250 mΩ while in type C up to 500 mΩ is allowed.

Figure showing cable IR drop from USB Type-C specification.

Image by USB 3.0 Promoter Group

7 out of 12 micro USB and 5 out of 6 mini USB cables I found at my home have less than 500 mΩ round-trip resistance. So according to my understanding of the standard for pre-type C cables, roughly 70% of my cables comply with it. Here are my resistance measurements plotted versus cable length. I've also included measurements published by Balaur on EEVblog and martinm on their blog. Points in the shaded area represent cables that comply with the standard.

Plot of cable resistance measurements versus length.

So strictly according the USB standards, the situation out there isn't perfect, but it doesn't look like the majority of cable are completely out of spec either. This seems a bit at odds with the general opinion that finding a good cable for running Raspberry Pi is hard. However, things start getting a bit clearer when you look at what exactly Raspberry Pi boards demand from these cables.

In the following table I've extracted maximum required power for all Raspberry Pi model Bs from the Wikipedia article. These boards will display the infamous under-voltage warning when their power supply voltage falls under approximately 4.63V. Assuming a perfect 5 V power supply, this is enough data to calculate the maximum allowable cable resistance for powering these boards:

R_{max} = \frac{U_{supply} - U_{min}}{I_{max}} = \frac{5.00\mathrm{V} - 4.63\mathrm{V}}{I_{max}}
Model Max. supply
current [mA]
Max. cable
resistance [mΩ]
RPi 1 Model B 700 529
RPi 1 Model B+ 350 1057
RPi 2 Model B 820 451
RPi 3 Model B 1340 276
RPi 3 Model B+ 1130 327
RPi 4 Model B 1250 296

Raspberry Pi model Bs after version 2 appear to require cables with resistance well below 500 mΩ that the standard requires for micro USB cables. Only 3 cables from my collection would be able to power a Raspberry Pi 3 model B. Raspberry Pi 4 gets a pass because the type C standard is flexible enough and doesn't directly specify cable resistance (although its type C implementation has other power-related issues). Yet, since type C cables have 750 mV maximum voltage drop at rated current, it requires a cable rated for 3 A or more according to this estimate (I'm not sure if Raspberry Pi 4 uses the same APX803 voltage monitor as earlier versions).

Also note that this calculation is for a perfect 5V power supply, which is optimistic. Power supplies don't have perfect regulation and the calculations in the USB standard assume worst case 4.75 V at the source. Such a worst case power supply, even if it provides sufficient current, would require practically zero ohm cables to power a Raspberry Pi without under-voltage warnings and associated CPU throttling.

To sum up, yes there are USB cables out there that are out of spec. However based on this limited sample, most micro and mini USB cables do seem to actually comply with the standard. Also worth noting is that shorter ones tend to have a better chance of being compliant. On the other hand, at least part of the blame for the grief surrounding USB cables appears to fall onto the Raspberry Pi itself since they designed their boards with an requirement for better-than-the-standard cables and power supplies.

Posted by Tomaž | Categories: Analog | Comments »

Resistance measurements of various USB cables

23.08.2019 10:23

After I made my USB cable resistance tester I naturally wanted to measure some cables. I searched my apartment and ended up with a big jumble of 18 micro and mini USB cables of various lengths and origins. I didn't imagine I would find that many, but I guess today just about everything comes with one and I apparently never throw away anything. In fact some cables were in a very bad shape and already had insulation flaking off from old age.

USB kabelsalat.

I measured the resistance of each cable at 1 A using the voltage ratio method I described in my previous post. The following table lists the results. For a lot of cables I don't know their origin and they must have came bundled with various devices. I've listed the brand if it was printed on or if I knew for certain which device the cable came with. I'm afraid this comparison isn't very useful as a guide which cable brand to buy, but it does give an interesting statistic of what kind of cables can be found out there in the wild.

N Brand Color Type Length [cm] R [mΩ]
1 Wacom Black A / micro B 28 199
2 CellularLine Gray A / micro B 207 212
3 White A / micro B 105 224
4 White A / micro B 51 294
5 Wacom Black A / micro B 98 334
6 Samsung Black A / micro B 82 408
7 Nokia Black / gray A / micro B 115 490
8 CubeSensors White A / micro B 103 522
9 Black A / micro B 103 569
10 HTC Black A / micro B 128 597
11 Google Black A / micro B 153 613
12 Amazon White A / micro B 182 739
13 Silver A / mini B 30 177
14 Black A / mini B 146 323
15 Black A / mini B 125 396
16 Silver A / mini B 56 412
17 Canon White A / mini B 125 435
18 Silver A / mini B 180 804

Unsurprisingly, two short 30 cm cables came out as best in terms of resistance, measuring below 200 mΩ. A bit more unexpected was finding out that the 2 m CellularLine isn't far behind. This is a fancy and laughably overpriced cable I bought in a physical store not so long ago, the only one on this list that I'm sure didn't come bundled with any device. It appears in this case the price was at least somewhat justified.

I was also a bit surprised that some cables that came bundled with devices measured pretty high. The white Amazon was for charging a Kindle 3 and it had the highest resistance among the micro B cables I tested. On the other hand, it was also in pretty bad shape, so it might be that it was damaged somehow. Cables bundled with an HTC phone and Google Chromecast also measured more than 500 mΩ.

Other measurements I could find on the web seem to roughly agree with mine. martinm lists measured values between 289 and 1429 mΩ. Balaur on EEVblog forum measured between 276 and 947 mΩ on his cables. The only report that was really off was this forum post by d_t_a where most of the cables listed are lower than 200 mΩ.

Another thing I was interested in was how repeatable these measurements were. I mentioned several times in my previous posts that contact resistance can play a big role. Since each time you plug in a cable the contacts sit differently and have a slightly different resistance, contact resistance behaves like a random variable in the measurement results. When I was doing the measurements above this was quite obvious. Minimal movements of the cable caused the voltage displayed on the voltmeter to dance around.

Histogram of 10 measurements of cable 16.

I repeated the measurement of cable 16 from the table above 10 times. Before each repetition I unplugged and re-plugged both ends of the cable. Above you can see the histogram of those measurements. The results only vary for approximately ±1%, which is much less than I thought they would. This is about the same as the expected error of the measurement itself due to the accuracy of the reference resistor. Of course, this was all done over a short period of time. I'm guessing the resistance would change more over longer periods of time and more cycles as contacts deteriorate or gather dirt.

I also checked how the measurement is affected if I plug something between the tester and the cable. Gino mentioned in a comment they used an adapter and an extension cable in their measurement. So I repeated the measurement of cable 1 from the table with a short type A-to-type A extension in series. Just for fun, I also tested how much resistance a cheap USB multimeter adds:

Assembly R [mΩ]
Cable 1 202
Cable 1 + 45 cm extension 522
Cable 1 + Muker V21 multimeter 442

As you can see from the results above, both of these added quite a lot. With the excellent 200 mΩ cable, both more than doubled the total resistance. Even with an average 500 mΩ cable, this multimeter would add around 240 mΩ or approximately 50% on top. Battery-powered devices like smartphones adjust their charging current according to the voltage drop they see on their end. Hence they might charge significantly slower when the multimeter is in series with the cable compared to just using a cable. This puts some doubt on the usability of these USB multimeters for evaluating USB cables and power supplies.

Posted by Tomaž | Categories: Analog | Comments »