RAID and hard disk read errors

27.02.2021 19:07

I have been preoccupied with data storage issues lately. What I though would be a simple installation of a solid state drive into my desktop PC turned into a month-long project. I found out I need to design and make new drive rails, decided to do some overdue restructuring of the RAID array and then had to replace two failing drives. In any case, a thing that caught my eye recently was a warning about RAID 5 I saw repeated on a few places on the web:

Note: RAID 5 is a common choice due to its combination of speed and data redundancy. The caveat is that if one drive were to fail and another drive failed before that drive was replaced, all data will be lost. Furthermore, with modern disk sizes and expected unrecoverable read error (URE) rates on consumer disks, the rebuild of a 4TiB array is expected (i.e. higher than 50% chance) to have at least one URE. Because of this, RAID 5 is no longer advised by the storage industry.

This is how I understand the second part of the warning: it talks about a RAID 5 array with total usable capacity of 4 TiB. Such an array would typically consist of three 2-terabyte disks. In the described scenario one of the disks has failed and, after adding in a replacement drive, the array is restored by reading the contents of the remaining two disks. This means we need to read out 2 times 2 terabytes of data without errors to successfully restore the array.

I was surprised by the stated higher-than-50% chance of a read error during this rebuild procedure. It seemed too high given my experience. Hence I've looked up the reliability section of the datasheet for the new P300-series, 2 TB Toshiba desktop-class hard drive I just bought:

Reliability section of the datasheet for Toshiba P300 hard drive.

I'm a bit suspicious of the way probability is specified here. Strictly reading the exponential notation, 10E14 means that the probability of an unrecoverable error (URE) is one error per 10⋅1014 bits. Expressed as probability of an error when reading a single bit:

P_{URE} = \frac{1}{10\cdot10^{14}} = 10^{-15}

In another datasheet for a different series of drives (however this time for data center instead of consumer use) the error rate is given as 10 errors per 1016 bits. This again gives the same error probability of 10-15.

Consider this probability for a second. It's such a fantastically low number. I don't remember ever encountering an actual technical specification that would involve a probability that has a one preceded by fifteen zeros - or in other words - fifteen nines of reliability.

The number is just on the edge of what you can represent with the common 64-bit double-precision floating-point format. If using a tool like numpy that only uses double-precision, any calculations with such values need to be done extra carefully to ensure that loss of numerical precision doesn't lead to nonsensical results.

Hard drives tend to use SI prefixes instead of binary, so I'll do the calculation for 4 terabytes instead of 4 tebibytes like it says in the quote:

n = 4 \cdot 10^{12} \cdot 8 \mathrm{bits}

For this calculation it doesn't matter whether we're reading this number of bites from one or two drives since the URE probabilities are assumed independent. The probability of getting at least one error during the rebuild is:

P_{rebuild-error} = 1 - (1 - P_{URE})^n \approx 3.1\%

Note that if I read 10E14 in the original reliability specification as 1014, the probability of a rebuild error goes up to 27%.

This comes out a bit more optimistic than the higher-than-50% figure given in the warning, at least for this specific series of hard drives. I guess whether 3.1% is still too high depends on how much you value your data. However consider that in the original scenario this is the probability of an error given that another hard drive in the array has already catastrophically failed. So the actual probability of data loss is this multiplied with the (unknown) probability of a complete drive failure.

Then again, consider that this is a desktop drive. It is not meant to be put into a RAID array and is typically used without any redundancy. Some people will even scream at you if you use desktop drives in a RAID due to timeout issues. Without any redundancy this probability directly becomes the probability of data loss. And that seems exceedingly high - especially considering that drives up to 8 TB seem to be sold with this same error rate specification. Even with that amazing reliability of reading a single bit, modern drives are simply so large that the vanishingly tiny error probabilities add up.

Posted by Tomaž | Categories: Life | Comments »

Reading RAID stride and stripe_width with dumpe2fs

20.02.2021 20:08

Just a quick note, because I found this confusing today. stride and stripe_width are extended options for ext filesystems that can be used to tune their performance on RAID devices. Many sources on the Internet claim that the values for these settings on existing filesystems can be read out using tune2fs or dumpe2fs.

However it is possible that the output of these commands will simply contain no information that looks related to RAID settings. For example:

$ tune2fs -l /dev/... | grep -i 'raid\|stripe\|stride'
$ dumpe2fs -h /dev/... | grep -i 'raid\|stripe\|stride'
dumpe2fs 1.44.5 (15-Dec-2018)

It turns out that the absence of any lines relating to RAID means that these extended options are simply not defined for the filesystem in question. It means that the filesystem is not tuned to any specific RAID layout and was probably created without the -E stripe=...,stripe_width=... option to mke2fs.

However I've also seen some filesystems that were created without this option still display a default value of 1. I'm guessing this depends on the version of mke2fs that was used to create the filesystem:

$ dumpe2fs -h /dev/... |grep -i 'raid\|stripe\|stride'
dumpe2fs 1.44.5 (15-Dec-2018)
RAID stride:              1

For comparison, here is how the output looks like when these settings have actually been defined:

$ dumpe2fs -h /dev/md/orion\:home |grep -i 'raid\|stripe\|stride'
dumpe2fs 1.44.5 (15-Dec-2018)
RAID stride:              16
RAID stripe width:        32
Posted by Tomaž | Categories: Code | Comments »

Showing printf calls in AtmelStudio debugger window

11.02.2021 16:26

Writing debugging information to a serial port is common practice in embedded development. One problem however is that sometimes you can't connect to the serial port. Either the design lacks a spare GPIO pin or you can't physically access it. In those cases it can be useful to emulate such a character-based output stream over the in-circuit debugger connection.

A few years back I've written how to monitor the serial console on the ARM-based VESNA system over JTAG. Back then I used a small GNU debugger script to intercept strings that were intended for the system's UART and copy them to the gdb console. This time I found myself with a similar problem on an AVR-based system and using AtmelStudio 7 IDE for development. I wanted the debugger window to display the output of various printf statements strewn around the code. I only had the single wire UPDI connection to the AVR microcontroller using an mEDBG debugger. Following is the recipe I came up with. Note that, in contrast to my earlier instructions for ARM, these steps require preparing the source code in advance and making a debug build.

Define a function that wraps around the printf function that is built into avr-libc. It should render the format string and any arguments into a temporary memory buffer and then discard it. Something similar to the following should work. Adjust buf_size depending on the length of lines you need to print out and the amount of spare RAM you have available.

int tp_printf_P(const char *__fmt, ...)
	const int buf_size = 32;
	char buf[buf_size];

	va_list args;

	va_start(args, __fmt);
	vsnprintf_P(buf, buf_size, __fmt, args);

	// <-- put a tracepoint here
	return 0;

We will now define a tracepoint in the IDE that will be called whenever tp_printf_P is called. The tracepoint will read out the contents of the temporary memory buffer and display it in the debugger window. The wrapper is necessary because the built-in printf function in avr-libc outputs strings character-by-character. As far as I know there's is no existing buffer where we could find the entire rendered string like this.

The tracepoint is set up by right-clicking on the marked source line, selecting Breakpoint and Insert Tracepoint in the context menu. This should open Breakpoint settings in the source code view. You should set it up like in the following screenshot and click Close:

Setting up a tracepoint to print out the temporary buffer.

The ,s after the variable name is important. It makes the debugger print out the contents of the buffer as a string instead of just giving you a useless pointer value. This took me a while to figure out. AtmelStudio is just a customized and rebranded version of Microsoft Visual Studio. The section of the manual about tracepoints doesn't mention it, but it turns out that the same format specifiers that can be used in the watch list can also be used in tracepoint messages.

Another thing worth noting is that compiler optimizations may make it impossible to set the tracepoint at this specific point. I haven't seen this happen with the exact code I shown above. It seems my compiler will not optimize out the code even though the temporary buffer isn't used anywhere. However I've encountered this problem elsewhere. If the tracepoint icon on the left of the source code line is an outlined diamond instead of the filled diamond, and you get The breakpoint will not currently be hit message when you hover the mouse over it, this will not work. You will either have to disable some optimization options or modify the code somehow.

Example of a tracepoint that will not work.

To integrate tp_printf_P function into the rest of the code, I suggest defining a macro like the one below. My kprintf can be switched at build time between the true serial output (or whatever else is hooked to the avr-libc to act as stdout), the tracepoint output or it can be turned off for non-debug builds:

#  define kprintf(fmt, ...) printf_P(PSTR(fmt), ##__VA_ARGS__);
#    define kprintf(fmt, ...) tp_printf_P(PSTR(fmt), ##__VA_ARGS__);
#  else
#    define kprintf(fmt, ...)
#  endif

With DEBUG_TRACEPOINT preprocessor macro defined during the build and the tracepoint set up as described above, a print statement like the following:

kprintf("Hello, world!\n");

...will result in the string appearing in the Output window of the debugger like this:

"Hello, world!" string appearing in the debug output window.

Unfortunately the extra double quotes and a newline seem to be mandatory. The Visual Studio documentation suggests that using a ,sb format specifier should print out just the bare string. However this doesn't seem to work in my version of AtmelStudio.

It's certainly better than nothing, but if possible I would still recommend using a true serial port instead of this solution. Apart from the extra RAM required for the string buffer, the tracepoints are quite slow. Each print stops the execution for a few 100s of milliseconds in my case. I find that I can usually get away with prints over a 9600 baud UART in most code that is not particularly time sensitive. However with prints over tracepoints I have to be much more careful not to trigger various timeouts or watchdogs.

I also found this StackExchange question about the same topic. The answer suggests just replacing prints with tracepoints. Indeed "print debugging" has kind of a bad reputation and certainly using tracepoints to monitor specific variables has its place when debugging an issue. However I find that with a well instrumented code that has print statements in strategic places it is hard to beat when you need to understand the big picture of what the code is doing. Prints can often point out problems in places where you wouldn't otherwise think of putting a tracepoint. They also have a benefit of being stored with the code and are not just an ephemeral setting in the IDE.

Posted by Tomaž | Categories: Code | Comments »

Making replacement Chieftec drive rails, 2

08.02.2021 19:47

Two weeks ago I was writing about 3D-printable replacements for 3.5" drive rails used in an old Chieftec PC enclosure I have. The original plastic rails became brittle with time. They often broke when I was replacing hard drives and I've eventually ran out of spares. I've drawn up a copy in FreeCAD and modified the design so that it was easily printable on a FDM printer. At the time of my last post I was waiting to get a sample pair printed. This is a quick update on that. I've tried the new rails and found that they fit well, but need some minor improvements.

3.5" hard drive with the 3D printed rails.

This is how the new rails look when mounted on a 3.5" hard drive. Unfortunately I've specified a wrong diameter for the holes in the STL file, so the imperial-sized screws that go into these drives don't fit. I've manually drilled the holes with a larger diameter on the photo above, but obviously it's better if the rails come correct from the printer in the first place.

These pieces were kindly printed for me by Matjaž on a Prusa i3 MK3S in PETG filament. They feel sturdy and more than strong enough for their purpose. The only thing I was slightly worried about is them getting softer when mounted on warm disk drives. According to the Prusa website the material should be good enough up to 68°C. My monitoring says that no drive got hotter than 50°C in the last 12 months so even in summer they should be fine.

Drive on the new rails being inserted into the Chieftec case.

I was happy to see that the drive with the new rails fits into the case perfectly. Much better in fact than with the original rails. Original rails were very difficult to handle, even when new. These slide in and out with just enough force. The plastic arcs on the top and bottom of the rails engage nicely with the guides and provide enough friction so that the drive doesn't rattle in the case. The strengthened tabs on the side also fit nicely. I saw no need to fix any of the basic dimensions.

Apart from the holes the only other part that needed fixing is the flexible latch. This latch has to be printed in a separate piece. My original idea was that I will glue it to the base part of the rail. However the latches all broke off when I was testing them. Part of the problem was that the cyanoacrylate (superglue) I was using doesn't seem to have good adhesion to PETG plastic. I'll probably find a glue that works better, but I still wanted to change the design so that it depends less on the strength of the bond between the two plastic pieces.

The picture below shows how I've slightly modified the latch. The red part is the latch and purple is the base part of the rail. This picture shows a cross-section in FreeCAD. See my last post for the renders of the complete rail.

The new and the old design for the latch on the drive rail.

The new latch design (top) has a tab that inserts into a slot in the rail. When the latch bends, the slot itself should take most of the torque and hold it in place. The glue should only hold it so that the latch doesn't fall out of the slot when not under tension. In the old latch design (bottom) the glue itself was taking all of the torque when the latch was bending.

I would be even happier with a design that wouldn't require glue at all and where the latch would click into place somehow. However I felt that coming up with the right tolerances for that would require several more round trips between FreeCAD and the printer and, since I'm already at version 4 of the design, I didn't want to waste any more time on this.

Anyway, I've put the updated STL files at the same place as last time. Again I'm waiting for the printouts. Hopefully when I get the chance to test them no further changes will be necessary and I can finally install a new stack of hard drives into my case.

Posted by Tomaž | Categories: Life | Comments »

Notes on the APC Back-UPS 700VA

31.01.2021 10:59

I recently had to replace an old Belkin UPS that had developed some weird problems. After some searching around for a suitable replacement I bought an APC Back-UPS 700VA (model BX700UI) from a local web shop. Even though it's not listed under the Network UPS Tools hardware compatibility list, getting it to run under Debian was relatively simple. Still, here are some random notes about it in case anyone else finds them useful.

APC Back-UPS 700VA

Image by Schneider Electric

As this blog post reports, the UPS is fully supported by the usbhid-ups driver in NUT. I'm currently using version 2.7.4 as packaged in Debian Stretch. Apart from setting the driver in /etc/nut/ups.conf as described there is no other special NUT configuration necessary.

I also had to add an udev rule to change the owner and permissions of the USB device so that NUT could access the UPS. The correct rule can be found in this answer. Note that the "could not detach kernel driver from interface" part of the error message you get if this rule is missing is somewhat misleading.

I've been testing this UPS for around a week on a test setup and I didn't find any problems with the basic functionality. Switch over to battery works fine. The system shuts down cleanly when the UPS reports battery low and correctly turns off the UPS output afterwards. It also comes back on when the power returns.

My old Belkin UPS seemed to impose a hard 30 minute battery run time limit. It would not run the load for more than 30 minutes and would report that the battery is empty regardless of its state. This UPS doesn't have that limitation and will happily run on battery for hours if the load is low enough.

I did increase the battery.charge.low and battery.runtime.low to 20% and 5 minutes respectively. These settings determine when the system will consider the battery depleted and shut itself down. While I didn't find any problems with the defaults (10% and 2 minutes), they seemed to leave very little reserve. This reserve battery charge comes handy if the line power comes back for a minute and goes away again. In this case the UPS must have enough run time left to boot the system back up and shut it down again. Since the battery is large enough for my use case I thought increasing this safety margin made sense.

Note that setting the variables using upsrw requires that a upsd user is both defined in /etc/nut/upsd.users with the permission for the SET action and also has a permission to connect to upsd in /etc/hosts.allow. I overlooked that latter part and spent some time banging my head over the ERR ACCESS-DENIED error.

I'm not sure how persistent the settings done with upsrw are. They are supposed to write the values to the non-volatile memory in the UPS so they should only need setting once. They should survive across reboots and power losses, but so far I'm not totally convinced they will not revert back at some random time in the future.

As far as diagnostic information goes, this UPS isn't very generous. It doesn't measure line frequency, temperature or output voltage. It does measure input voltage and it returns reasonable values for it when the line power is present, but I noticed that it reports 256 V when the input voltage is in fact 0 V.

I had a quick look at the raw USB HID traffic from the UPS and I didn't find anything particularly interesting. At least it doesn't look like the relative lack of diagnostics is due to missing support in NUT.

The battery takes approximately 3 hours to fully recharge. I noticed some very faint crackling noise from it when it's charging. To me it hears like small bubbles popping when I press my ear against the case. This noise is distinct from the mains hum and doesn't hear like a DC-DC converter running. I don't have much experience with lead-acid batteries, so this is probably normal behavior. I just never noticed it before. Apparently even sealed gel batteries form some hydrogen gas that is then recombined inside with a catalyst.

Posted by Tomaž | Categories: Life | Comments »

Making replacement Chieftec drive rails

22.01.2021 19:32

I use an old Chieftec tower case for my home desktop computer. Instead of the usual screws the case uses purple plastic retainers for mounting 3.5" disk drives. These parts were quite fragile and even though the case came with plenty of spares I broke all of them over the years. I failed to find an off-the-shelf replacement so I went and designed a 3D printable part to replace them. I'm currently waiting to get the prototype printed.

I don't know the model name of the case or whether Chieftec had a specific name for this method of mounting drives. The only identifiable mark on the case is the name "Chieftec" embossed in the front panel. I wrote about it in this blog post so it must be at least 16 years old at this point. After quite a lot of web searching, this 2004 review of the BX-03BL-BL-BL model was the only other reference to this rail design I could find. It has a photo of the rails that look identical to the ones I have although the design of the rest of the case looks different.

3.5" drive bay in the Chieftec case.

I think these plastic rails were badly designed from the start. I remember even when the case was new it was very hard to insert or remove a drive. The rails are formed in the shape of a slight arc. When inserting a drive into the slot they straighten out which grips the drive in place. However the rails are fixed to the drive at two points so they can't flex freely. The other problem is that the triangular catch that latches into the metal case has no spring. You need to bend a relatively short and thick length of plastic to unlatch.

A set of broken and discolored Chieftec drive rails.

The plastic didn't age well. It got quite brittle and also changed color from a bright pastel purple to this purplish-gray. Very soon changing a hard drive in the case meant breaking one or both of the rails that held it in place. These days it begins to crack if I put even a slight pressure on it. An unfortunate flaw since otherwise I like this case and it has served me well for all those years.

The rails have a complex shape that isn't printable on a common FDM printer in one piece. To get a single, flat face that starts on the printer's build plate I removed the outward arc in the base shape. It was problematic anyway. Since holes for the sunken screw heads require an overhang in this orientation, I left a thin sacrificial layer that separates the two hole diameters. This needs to be drilled through before the screws can be inserted. I plan to use normal screws instead of the knurled pins the original rails used.

Replacement Chieftec disk rail, top view.

I moved the catch to a separate part that must be glued to the base part. It's mounted on a thin leaf spring that should provide the flexibility I was missing in the original. This part can be printed with layers following the spring shape which should make it more resistant to bending. I also extended the tabs on the sides to make them stronger. After the handle these were the second most common thing to break when unlatching the rail.

Replacement Chieftec disk rail, bottom view.

This was the first more complicated mechanical part I designed using the FreeCAD's Sketcher Workbench. I like the visual way of defining lengths and constraint on shapes mapped to object's faces. It's much clearer than the approach I took with my previous 3D printing projects - using cubes and other basic primitives from the Part Workbench, placing them using Python expressions and using unions and subtractions to get the final shape.

The problem I encountered however was that Sketcher isn't very convenient for experimenting. Because of the topological naming problem you can't easily go back and change things on a model. So far I've already ended up redrawing this shape 3 times from bottom up. Changing something in the sketch for the base shape tends to invalidate all other sketches for pockets and pads that are built on top of it. I don't know how to fix the missing edge and face references without starting all over again.

I'm now waiting to get these new rails 3D printed and I'm yet to see how they turn out in practice. This is the third iteration and I think I got all the measurements right. I'm mostly concerned if the new rails will fit snugly enough into the case to prevent the drive rattling and at the same time not be too hard to insert and remove. If they are too loose I may just add some rubber padding like the 2004 review I linked above suggests.

I've made the current version of my 3D printable design available for download here. I'll write another post when I have the chance to test these out and if I make any more iterations to the design.

Posted by Tomaž | Categories: Life | Comments »

On fake 50 ohm coaxial cables

03.01.2021 11:50

I guess by now everyone is already aware that cheap no-name items often aren't what they claim to be. I've recently found out that BNC coaxial cables sold as having a 50 Ω characteristic impedance often are in fact 75 Ω. Since this is something that is not trivial for the customer to measure and a 75 Ω cable will sort of work even in a 50 Ω system I guess it's easy for sellers to get away with this kind of scam.

This article goes into the details of how to measure characteristic impedance of a transmission line. I used the Smith chart method. I connected each cable to the (50 Ω) NanoVNA CH0 on one end and a Zref = 50 Ω calibration load on the other end. I then measured S11 over the frequency range from 10 to 200 MHz. NanoVNA was calibrated using the calibration kit that came with it.

I found the point where the S11 plot on the Smith chart crosses the real axis. This is when the cable acts as a quarter-wave transformer. The point on the chart marks the impedance Zt. It is marked in orange on the plots below. Characteristic impedance Z0 of the cable can then be calculated using the following formula:

Z_0 = \sqrt{Z_t Z_{ref}}

The Seafront 100 cm BNC cable sold as "50 Ω Oscilloscope Test Probe Cable Lead" measured 71.1 Ω using this method. The second cable I got in the pair had a bad ground connection and couldn't be measured at all. The reviews on the Amazon product page mention bad mechanical construction of the connectors. The cable has a readable SYV 75--3--2 marking that already raises suspicions that this is indeed a 75 Ω cable.

Label on the Seafront 100 cm cable.

Characteristic impedance measurement of Seafront 100 cm cable.

The Goobay 50 cm BNC cable sold as 50 Ω RG-58 measured 79.6 Ω. This cable in fact has RG58 50 OHM COAXIAL CABLE printed on it. It was either marked incorrectly from the factory or was relabeled later. There is a review on the Amazon product page from R. Reuter that also reports that the cable they received was 75 Ω based on a NanoVNA measurement.

Label on the Goobay 50 cm cable.

Characteristic impedance measurement of Goobay 50 cm cable.

The 60 cm LMR-200 SMA cable that I got with the NanoVNA measures 50.4 Ω. I'm showing it here for comparison with the two results above. The cable construction is not that great - one of the two such cables I got in the kit wasn't crimped properly and the connector fell off the cable after a few uses. However the cable actually has the correct characteristic impedance within a reasonable tolerance.

Characteristic impedance measurement of LMR200 60 cm cable.

I'm curious why this is happening. I'm guessing because 75 Ω cables are cheaper for some reason. Maybe there is a bigger demand for 75 Ω cables for video applications and 50 Ω used in RF systems is more of a niche application? Also the cables I measured didn't have good matching even for use in 75 Ω systems. Maybe they're in fact quality control rejects from a failed factory run? Anyway, it's good to be aware of this practice and double check if using a cable in an application that is sensitive to this sort of thing.

Posted by Tomaž | Categories: Analog | Comments »


01.01.2021 21:47

What can I say about the last year that hasn't already been said a hundred times over? It's been a year of restrictions, lockdowns and changing habits. I'm immensely thankful that everyone close to me has managed to avoid the pandemic so far. It has not been easy for anyone. The quickly changing information on the virus was a constant source of doubts in what was a good balance between keeping safe and retaining some resemblance of normal life.

I don't feel I have much right to complain though. Compared to other personal stories I've heard I'm lucky. I don't have many responsibilities that would require a lot of personal contacts. I work for a company that has recognized the seriousness of the situation very early on and switched largely to remote work where possible.

Curve tracer on the desk with other instruments.

It's been a year of sitting alone at my home desk. I've never been a particularly social person and I usually enjoy working alone, but the isolation this year sometimes felt painfully strong. I feel like the social distancing didn't just apply to physical contacts and we've all grown further apart even with all the video, voice and text communication within hand's reach. It was easy to get unreasonably angry over people holding different opinions on what risks they were prepared to take with their own health and the health of others.

Reduced work and the general chaos interrupting other plans meant that I worked a lot on personal projects. I've expanded the instrumentation in my home electronics lab: I've finished my home-brew curve tracer for non-linear device measurements. I didn't write much about that on the blog. I still have some unpublished notes on an interesting detail of bipolar junction transistors, which was the original motivation for making the instrument in the first place. Maybe I'll find the time to write it up next year.

Time multiplex circuit board assembled and powered on.

On the other hand I did write a series of posts on RF vector measurements. I'm very happy with how my SDR-based system turned into a small, versatile instrument. With a NanoVNA available at perhaps a tenth of what I spent in total it might not look like the project made much economic sense. However I found that my instrument complements it nicely, with better characteristics in some places, worse in others and much more flexibility. It seems to fit nicely between the extremely low cost NanoVNA and the astronomical price of a professional instrument. Not to mention that I learned a lot while making it and refreshed my knowledge from the under-grad microwave engineering course I took long ago.

I've again drawn a lot, more than any other year despite the arts classes being interrupted. It was my almost daily activity in the evenings after I got fed up with electronics. In May, my comic Social distancing won the second place in its category in the Life in the time of coronavirus competition organized by the National Museum of Contemporary History. With the bird flu epidemic hitting local bird populations later in the year, my two quarreling crows might have been an even better allegory for the times than I originally intended.

"Life in the time of coronavirus" exhibition panel.

In general I wanted to give some more meaning to the illustrations and characters I've drawn this year. This led me to write a few short stories. Looking back, these were probably more things I needed to do for myself to deal with various frustrations and anxieties. I've only published one so far here on this blog and even that was quite an internal struggle to do. I don't know about the others. I still like the pictures that go with them, but the text would need significant editing before I would be comfortable with putting them out in public.

Urbane vrane: Luč

The web was full of people cheering for 2021 to come as quickly as possible, as if viruses and other such things pay any attention to the numbers on the calendar. I have some ideas what I want to do next year: I want to keep drawing and, considering I'll be using my home desk as the company hardware lab for some time yet, I have some further ideas on new instruments I want to make. However I can't shake the feeling that the freedom to pursue my personal projects in relative idleness during lockdowns was on borrowed time and that I'll have that much less time for such things in the future. We'll see. If we all remain healthy it will be a good year.

Posted by Tomaž | Categories: Life | Comments »

Cleaning the DVD player remote

20.12.2020 14:47

The batteries in the remote control for my DVD player leaked. I thought it would be best to clean it out thoroughly as soon as possible. The electrolyte from the alkaline batteries attacks the phenolic paper substrate that is used for the cheap circuit boards. If left uncleaned the circuit board delaminates and basically falls apart after a few years. This is what happened to the remote for my old Hi-Fi and the circuit is more or less irreparable after that happens.

This is the remote model RC-5340 that came with the Philips DVP3360 DVD player.

Philips remote model RC-5340

Cleaning all the mess left by the batteries required opening the plastic enclosure. I couldn't do a good job through the battery compartment alone. Unfortunately the enclosure is really tightly held together with plastic tabs. I found it impossible to pry open without damage using my normal tools. There is almost no gap between the top plate and the lower part. I finally managed to open it with the iFixit Jimmy metal spudger tool kindly borrowed from Matjaž.

The best strategy was to find a weak spot along the longer sides and then work the gap from there. You can see on my photos where the plastic tabs are located. The tabs in the top and bottom sides are so tight that I couldn't open the enclosure without breaking some of them.

Disassembled plastic enclosure of the RC-5340 remote control.

The chip is marked "ET3013 0907".

Circuit board and rubber membrane of the RC-5340.

I scrubbed the whole thing with soap and water, dried it and also cleaned the rubber contacts with ethanol. There are already some dark spots on the PCB where the electrolyte has started to attack the board. I hope those won't spread too much.

Posted by Tomaž | Categories: Life | Comments »

The engineer and the librarian

27.11.2020 23:01

An engineer walks into a library and steps to the reference desk. "Good morning, sir. How may I help you today?" asks the friendly librarian. "I'm an engineer. I'm looking for some references on the types of bridge constructions. Could you find something for me please?" he says. "Of course sir, I know exactly what you are looking for. It will be just a moment."

True to his word, the librarian quickly returns with a stack of books. "Will you be taking these to the study room or checking them out sir?" The engineer picks the top book from the stack in surprise and shows it to the librarian. "This is a children's picture book. Don't you think I'm a bit too old for this?" The librarian isn't thrown off by the question. "It has large letters and is very easy to read. Even many middle-aged readers find large print books more pleasant to read" he answers. "That's very considerate of you, but thankfully my vision is fine so far." says the engineer. "I'm not sure you understand what I'm looking for. How does this even relate to bridges?" The librarian takes the book and opens it. There's a drawing of a family of ducks crossing a bridge over a stream. "See, it's right here on the first page? The topic you were looking for. Is there anything else you need?"

The engineer and the librarian.

The engineer is confused for a moment, but decides to drop the matter. He puts the picture book aside and takes the next one from the stack. It's a pocket dictionary. "I'm not sure how this one will help me either." asks the engineer. The helpful librarian has an answer at hand "Compared to the other books I found this one easily fits in your pocket. You can carry it around and read it on the go. I find many people like to read on the bus for instance." He opens the dictionary and shows it to the engineer. "You will find the definition of bridge under "B" right here."

"Ah, I think I see where the problem is. I'm more interested in books that have more content specifically about bridges. Format isn't that important." says the engineer. "I understand, sir. I'm sure the next book I found for you is the right one." answers the librarian and hands him another book from his stack. The glossy hardback cover promises a suspenseful crime novel. "This looks like fiction". "Indeed it is sir. This is one of our most popular books and from a best selling author. I'm sure you'll enjoy reading it." the ever friendly librarian replies. "Well, yes. But I want to learn about bridges you see, not read murder mysteries." says the engineer. The librarian looks at him in confusion "I'm not seeing the problem here sir. Most of the story revolves around a murder on the Brooklyn bridge. That is the topic you asked me for, isn't it? Most people that check out this book are happy with it".

The engineer rubs his forehead in frustration. "Maybe you're right. I wasn't clear enough that I was looking for non-fiction" he says to the librarian. "Is that a newspaper you are holding now?" "Indeed it is sir. It's from our selection of periodicals. Compared to these other texts that were all published at least several months ago this one is very fresh, just printed and delivered to us this morning. There's a news article on a local bridge renovation project on the third page I'm sure you'll find informative. It's always good to keep up-to-date on the topic of interest, don't you think?"

"Not really what I was looking for." says the engineer. "It's so unfortunate that you removed the dedicated engineering section you had here" he adds and points to the now empty desk down the hall way. The librarian looks at him and explains. "Not at all sir. We find that most people who visit our library find it less confusing if all kinds of literature are available from one desk." The engineer thinks about it for a second. "Did you bring any books from the engineering section in that stack?". The librarian smiles back. "Sorry sir, we are a modern library and no longer shelve books based on classification." He hands him another book. "However I believe this one is indeed on an engineering topic". The engineer sighs after taking one look and hands it back "This is a book on ship building. I'm interested in bridges." The librarian remains undeterred "Ah yes, but I'm sure you'll find this book useful if you take a second look. If you want to cross a body of water, a ship will do it as well as a bridge."

They go through the rest of the stack of books without any success. The engineer stresses that he is only interested in bridges and asks the librarian to fetch him another stack of books. And another. Ever more random books get picked up by the librarian and they get no closer to the topic the engineer was interested in. At last, the engineer gives up. He thanks the librarian for his help, but silently vows not go visit this library again. He tries his luck in the other library in town. The second librarian is a bit less friendly and less forthcoming with his explanations of why he thinks the books he found were something the engineer would like. He also turns out to be no more capable of finding a useful book on the topic than his colleague. Running out of options, the engineer returns home, disappointed that he has not found any literature to help him work on his problem.

Later that week he borrows a book from a friend of a friend. The book has a worn out cover and the paper is yellow at the edges. It has been published 50 years ago. The print is small and sometimes uneven. Back-and-white figures look like they have been drawn by hand. The text is dry and sometimes hard to understand. But on page 218 it has a chapter that helps him solve the problem he has been working on.

Posted by Tomaž | Categories: Ideas | Comments »