ESP8266 humidity monitoring

27.02.2017 20:11

Last year my flat developed a bit of a mold problem, or maybe I just found out about it then. It's possible the fungus already lived a long, fulfilling life before being discovered. It wouldn't be surprising for a building from an era when thermal- and hydro-isolation were pretty far down on the priority list. In any case, it made me want to monitor the relative air humidity and dew point levels a bit more closely. I had the apartment pretty well covered with sensors already, but the room with the mold in particular lacked a hygrometer.

Wireless humidity monitor based on the ESP8266 module.

Not to re-invent too much hot water, I more or less replicated the nicely documented temperature and humidity web server project from Adafruit. It was doubly appealing because I still had a full bag of old ESP8266 modules that I bought for pennies back when they were the exciting new thing. The only problem was the fact that the humidity sensors supported out-of-the-box by that project were only available from their shop, which is relatively expensive for small items with oversea shipping. Since I was in a hurry, I bought a few DHT11 modules anyway (which turned out to be a mistake).

There's not much to say about the hardware. It's the minimalistic Adafruit's circuit soldered on a perforated board. For the power supply I used a small 3.3V switch-mode converter module I had left over from another project. I was nicely surprised by how easy ESP8266 support was to install into the Arduino IDE. Another pleasant discovery was that ESP8266 with the Arduino-based firmware seems to consume much less power than with the stock AT-command firmware.

The Arduino IDE got updated since Adafruit's tutorial was written, so I had to experiment a bit with the firmware upload settings. Following values seemed to work with my particular modules. Another thing I discovered was that the RST line on ESP8266 has to be left floating for the firmware upload to work reliably. On my previous ESP8266 project I tied it to VCC.

Arduino firmware upload settings for ESP8266 modules.

Unfortunately, the DHT11 modules are pretty bad as far as accuracy is concerned. I only discovered Robert's wonderfully in-depth comparison of hygrometer modules after the fact. I played a bit with power supply filtering, but that doesn't seem to be the source of the noise in the data. I ended up modifying Adafruit's firmware so that it reads the sensor every 5 seconds and returns the average of last 8 readings. This alleviates somewhat the problem, but I definitely recommend using some other sensor to anyone wanting to build this.

For comparison, here is the daily humidity graph recorded using DHT11 with averaging:

DHT11 example daily humidity record.

And here is humidity recorded at the same time (albeit in a different room) by a TEMPerHUM USB dongle with no extra averaging applied:

TEMPerHUM example daily humidity record.

After several months of running, the Arduino-based ESP8266 turned out to be pretty reliable. I haven't seen any big outages in the log of sensor readings. This is a nice improvement over the stock firmware that I used in my Munin display, which still regularly gets lost to the point that it requires a power cycle.

Posted by Tomaž | Categories: Digital | Comments »

GIMP onion layers plug-in

21.02.2017 20:48

Some time ago I was playing with animation making applications on a (non-pro) iPad. I found the whole ecosystem very closed and I had to jump through some hoops to get my drawings back onto a Linux computer. However the fact that you can draw directly on the screen does make some things easier compared to a standalone Wacom tablet, even if the accuracy is significantly worse.

One other thing in particular stood out compared to my old GIMP setup. These applications make it very easy to jump frame by frame through the animation. In one touch you can display the next frame and do some quick edits and then move back with another touch. You can browse up and down the stack as a quick way to preview the animation. They also do something they call onion layering which simply means that they overlay the next and previous frames with reduced opacity so that it's easier to see how things are moving around.

This is all obviously useful. I was doing similar things in GIMP, except that changing frames there took some more effort. GIMP as such doesn't have a concept of frames. Instead you use image layers (or layer groups) as frames. You have to click to select a layer and then a few more clicks to adjust the visibility and opacity for neighboring layers if you want to have the onion layer effect. This quickly amounts to a lot of clicking around if you work on more than a handful of frames.

GIMP does offer a Python plug-in interface however, so automating quick frame jumps is relatively simple. Relatively, because GIMP Python Documentation turns out to be somewhat rudimentary if you're not already familiar with GIMP internals. I found it best to learn from the Python-Fu samples and explore the interface using the built-in interactive console.

Screenshot of the GIMP onion layers plug-in

The end result of this exercise was the GIMP onion layers plug-in, which you can now find on GitHub together with installation and usage instructions. The plug-in doesn't have much in terms of an user interface - it merely registers a handful of python-fu-onion- actions for stepping to previous or next frame, with or without the onion layer effect. The idea is that you then assign keyboard (or tablet button) shortcuts to these actions. You will have to define the shortcuts yourself though, since the plug-in can't define them for you. I like to use dot and comma keys since they don't conflict with other GIMP shortcuts and match the typical frame step buttons on video players.

If you follow the layer structure suggested by the Export layers plug-in, this all works quite nicely, including handling of background layers. The only real problem I encountered was the fact that the layer visibility and opacity operations clutter the undo history. Unfortunately, that seems to be the limitation of the plug-in API. Other plug-ins work around this by doing operations on a duplicate of the image, but obviously I can't do that here.

I should note that I was using GIMP 2.8.14 from Debian Jessie, so the code might be somewhat outdated compared to latest GIMP 2.8.20. Feedback in that regard is welcome, as always.

Posted by Tomaž | Categories: Code | Comments »

Python applications in a .zip

08.02.2017 10:20

Poking around youtube-dl I found this interesting recipe on how to package a self-contained Python application. Youtube-dl ships as a single executable file you can run immediately, or put somewhere into your PATH. This makes it very convenient to use even when you don't want to do the usual pip install dance. Of course, it comes at the cost of not resolving any dependencies for you.

I was expecting the file to be a huge, monolithic Python source file, but in fact it's a ZIP with a prepended hash-bang and nicely structured Python package inside. Simplified a bit, here is the gist of the Makefile part that builds it:

hello:
	cd src && zip ../hello hello/*.py __main__.py
	echo '#!/usr/bin/python' > hello
	cat hello.zip >> hello
	rm hello.zip
	chmod a+x hello

Now, if src/__main__.py contains:

import hello
hello.greet()

And src/hello/__init__.py contains:

def greet():
	print("Hello, World!")

Building the executable and running hello from the command-line should result in the standard greeting:

$ make
cd src && zip ../hello hello/*.py __main__.py
  adding: hello/__init__.py (stored 0%)
  adding: __main__.py (stored 0%)
echo '#!/usr/bin/python' > hello
cat hello.zip >> hello
rm hello.zip
chmod a+x hello
$ ./hello
Hello, World!

How does this work? Apparently it's quite an old trick with some added refinement. Already since version 2.3 Python knows how to import modules directly from ZIP files in the same way as from the usual directories. Python also allows executing modules from the command-line.

It sounds very much like Java JARs, doesn't it? The only missing part is the #!... line that makes the Linux kernel use the Python interpreter when executing the file. Since ZIP format ignores any junk that precedes the compressed data, the line can simply be prepended as if the whole file was a simple Bash script.

Posted by Tomaž | Categories: Code | Comments »