MagPi interview

Issue #91 of the official Raspberry Pi magazine, The MagPi, has just been published and it features an interview with me, about my time at the Foundation (and a bit before), letting the community know what I’m moving on to at the BBC.

You can buy or download the issue here: Issue #91

Here’s what the spread looks like, pages 82-83:

Here’s a gallery of pictures from my time at Raspberry Pi:

Meme Overflow

A few months ago I was looking through questions posted to the Raspberry Pi Stack Exchange site. If you’re not familiar, Stack Exchange provides Q&A sites like Stack Overflow for specific areas, such as particular programming languages, technologies or other topics. And while observing the erratic nature of the way people post questions to the site, I thought “wouldn’t it be funny to take questions from Stack Exchange and put them on meme templates?” and it seemed like a trivial enough task to automate, so I started googling things, and typing stuff into a Python shell, and within an hour or so I had it working. The results were random and hilarious, as I expected.

Sometimes ideas like this seem simple in theory, but in practice there’s some complexity you didn’t think of that gets in the way. But with this idea, everything was smooth sailing. I found the Stack Exchange API easy to use, and came across the imgflip site, which provides an API. All I needed to be able to do was retrieve question titles from a particular Stack Exchange site, and to add text to a random meme template. Both of these things were trivial, and as I’ve done a lot of things using web APIs in Python lately, all the skills were well practised, which helps a lot.

Once the concept was proven, I decided to set up a Twitter account for the memes. I added code to tweet the images. Again, this is something I’ve done a lot of, so no hassle. I forget how long it took to get the app approved and the bot working, but before long, @pi_stack was tweeting out memes autonomously.

Hit and miss

Meme templates are chosen randomly, so obviously not every question / meme combination will work. Some will never be funny, some won’t be a good match, some will end up on a template that doesn’t really work. Over time I got to see which meme templates would never work, and blacklisted them. I also made sure that text was placed in the correct position for most effect: some templates work better with the text in the second or third position than the first – and as I only have one piece of text to add, this makes all the difference.

Later I decided to add some logic to choose the right meme template based on simple rules like “starts with X”, or to use a different template if it “doesn’t end in Y”. This seemed to make a big difference, though it’s still very hit-and-miss.

The test suite

I decided to create a test suite so I could more confidently make changes to the library without risking breaking things. Even though this is a silly side project, it’s been another way for me to exercise good practice which makes it easier when you need to do this stuff for real.

In gpiozero we have a mock pin interface which allows us to test the workings of the library without running on real Raspberry Pi hardware. And in the test suite we use patch to mock behaviour. I had to learn some new techniques for mocking things like web requests, image uploads and even tweets. I’m sure my methods are far from perfect but it’s a decent attempt with good coverage. Check out the tests and coverage.

Instances

I started with the Raspberry Pi Stack Exchange, and later added other instances:

If you’re not familiar with the concept of “world building”, it’s for people who are designing worlds for books, stories, games and such – and have questions like “What is keeping my Terror Bird from being the size of a T. Rex?” which are perfect for this project.

I’ve tried to set up more (there are so many great Stack Exchange sites) but Twitter is not letting me authenticate more accounts with the same mobile number.

Best in show

Here are some examples that worked well:

One of the best ones ever was extremely unlikely. It was a template that almost never works, because it really requires two pieces of text to make sense. But in this one case, it was perfect. In the RPi.GPIO library, Python warnings are switched on by default, and you have to turn them off. Most people find them annoying and include the line that disables them. The problem with Python warnings is that they look like errors/exceptions – people see red text and think there’s something wrong with their program – when really it’s working fine but it’s just letting them know something unimportant like they’ve already configured that pin. So someone on Raspberry Pi asked about the warning message, posting just the warning text (verbatim) as the question title, where the desired outcome is that no warning is output. And – by chance – this subtle genius comes out of Meme Overflow:

Another recent one was from World Building. Again, completely random:

Is anyone following?

The Stack Overflow bot is the most popular, with nearly 500 followers at the time of writing. Raspberry Pi and Ask Ubuntu have over 100 each.

Run your own instance

If you want to run your own bot following a particular Stack Exchange site, you can pip install memeoverflow and use the example script, following the simple guide.

If you want to run a modified version, you can either fork the project or subclass the provided class.

Mr President

Goodbye Raspberry Pi, Hello BBC

Just a quick note to say I’m leaving the Raspberry Pi Foundation to start a new role at the BBC. I’ve been at Raspberry Pi for over six years, and it’s been great. I’m excited to join an innovation team called BBC News Labs. They find new ways to solve problems with technology, making lives easier for journalists, and they drive innovation within BBC News.

https://twitter.com/ben_nuttall/status/1214912375918383104

New Labs are based at New Broadcasting House in London, so I’ll be partly based there:

W1A

News Labs write up their projects at bbcnewslabs.co.uk – so read up there to find out more.

So goodbye Raspberry Pi!

https://twitter.com/ben_nuttall/status/1220792172993302534

And hello BBC!

https://twitter.com/ben_nuttall/status/1219941583040188416

What’s new in GPIO Zero v1.5?

It’s become customary for me to summarise what each new GPIO Zero release brings. This one’s been a long time coming. It’s been a quite while since our last release (a whole year since the last point release and 18 months since v1.4). I mostly attribute the lack of development to the launch of my other project, piwheels. I think that was time well spent, but I’m sorry to say that GPIO Zero hasn’t come along as far as I’d hoped by now. But hopefully you’ll find the contents of this release to be a satisfying step in the right direction.

Upgrade now:

sudo apt update
sudo apt install python3-gpiozero python-gpiozero

You can see the summary of what GPIO Zero v1.5.0 brings in the changelog. I’m pleased to say it’s the longest changelog entry to date – lots of fairly small changes, plus some pretty big (under-the-hood) ones thrown in for good measure.

So, what’s new?

New device classes

  • PiHutXmasTree
    • The Pi Hut’s 3D Christmas tree board Rachel designed is now natively supported, so you don’t have to create your own LEDBoard with all the pins.
  • PumpkinPi
    • A neat new Halloween themed LED board from ModMyPi
  • JamHat
    • Another new board from ModMyPi, the Jam HAT comprises two sets of traffic light LEDs, two buttons and a tonal buzzer
  • TonalBuzzer
    • As featured on the Jam HAT, this component allows you to make audible tones using PWM
  • LoadAverage
    • A new “internal” device you can use to read your Pi’s load average, or even feed its values into an output device, e.g. an LED or LED bar graph
  • DiskUsage
    • Similarly, another new internal device for showing the disk usage, say of your SD card or other media

Tones

Claire Pollard wrote some RPi.GPIO code to play tones on the tonal buzzer on the Jam HAT, and then converted it to GPIO Zero. I took what she’d done, and wrapped it in a TonalBuzzer class, a composite device taking inspiration from Servo, where the value is not directly tied to the duty cycle, like a standard PWMOutputDevice or a PWMLED, but instead proportional to the mid tone. I also replaced the dictionary look-up for MIDI notes and frequencies with the maths to calculate it. Then Dave moved the note/tone/frequency conversion into its own class, proving all that functionality in one place. On their own, tones could be used elsewhere, similar to the colour manipulation that was previously embedded within picamera, but became colorzero. Here’s a set of examples for creating tones:

>>> from gpiozero.tones import Tone
>>> Tone(frequency=440)
>>> Tone(midi=69)
>>> Tone(note='A4')

And used with TonalBuzzer:

>>> from gpiozero import TonalBuzzer
>>> from gpiozero.tones import Tone
>>> buzzer = TonalBuzzer(20)
>>> buzzer.play(Tone(note='A4'))

You can even set the source of a TonalBuzzer to play a series of tones, a source of artificial values or directly from another device. A siren effect can be achieved by setting the source to a sine wave:

from gpiozero import TonalBuzzer
from gpiozero.tools import sin_values

buzzer = TonalBuzzer()
buzzer.source = sin_values()

Since TonalBuzzer has a value range from -1->1, where -1 is an octave below its mid tone, and 1 is an octave above (by default), that means if you set its source to come from a device whose values are -1->1 you can control its full tonal range, and a device with range 0->1 can control from its mid tone to an octave above, but either way can be effective. Since sin_values outputs values from -1 to 1, it’s a perfect example (as is cos_values). You can use source_delay to speed up or slow down the rate it iterates over the values.

Another good example is DistanceSensor which has values between 0 (no distance to object) and 1 (max distance, configurable). So if you send the DistanceSensor‘s values into a TonalBuzzer you effectively have a theremin:

from gpiozero import TonalBuzzer
from gpiozero.tools import sin_values

buzzer = TonalBuzzer()
buzzer.source = sin_values()

Distance Sensor

Speaking of DistanceSensor, we had identified issues with the distance it reports. Our implementation is complex and uses threading. The inherent complexity is essential for the device to work with other GPIO Zero devices, and without requiring blocking other code from running or using up 100% of the CPU. However, where there is complexity there is scope for bugs. Thanks to some clever work from Dave, we now use timing information provided by underlying drivers, so that timing events from pins will be considerably more accurate (see #655). Also, Dave found that the default pin library, RPi.GPIO, would often miss edges during callbacks, which threw off the timing, so we now drop missed edges and get better accuracy as a result (see #719). We recommend using the pigpio pin factory for the best results, particularly with Pi 1 or Pi Zero.

Source/values

My friend Robie has built his own z-wave home automation project using a GPIO Zero-like API which includes a similar source/values interface, allowing him to easily connect devices together, like a light switch controlling a light but also a humidity sensor (depending on the humidity sensor reading) – and has completely custom rules for all the devices in his house. He suggested to me one day that sometimes you need access to the device object itself when setting source, not just the device’s values, and I realised that this would make the API even simpler as a result. So now you can use:

led.source = btn

Instead of:

led.source = btn.values

Although both methods still work. This also simplifies more complex device rules, as you can replace every instance of device.values with just device, and so:

led.source = all_values(btn1.values, btn2.values, btn3.values, btn4.values)

becomes:

led.source = all_values(btn1, btn2, btn3, btn4)

We also added graphviz diagrams for all the examples in the Source/Values docs page which I think helps visualise the examples:

Other improvements

  • Import time is massively reduced – we used to do an expensive pin factory computation at import time, but we removed that due to pkg_resources being slow as hell.
  • It’s now possible to import gpiozero without a valid pin factory, or any environment variables set.
  • RGBLED now supports colorzero, which is an awesome module previously buried in picamera, but I persuaded Dave it would be a useful as a separate library. It’s also used by Dave’s Sense HAT library pisense. Anyway, now you can now set the colour of an RGBLED using e.g. Color('purple') and do lots of really useful colour manipulation such as gradients.

Finally, a new flag allows you to type:

pinout -x

And it opens up the website pinout.xyz in a web browser.

Now, back to piwheels for a brief spell – we have some awesome stuff in the works – all to be revealed soon – not least the expansion to Python 3.7 for Raspbian Buster – but more on that soon. Then once that’s out the door, back to GPIO Zero. I am planning on upping the release cycle back to one per quarter, like in the good old days. You can count on it!

EuroPython, PyCon UK and a John Pinner Award

I’ve been lucky enough to have been able to attend (and speak at) two great Python conferences – EuroPython and PyCon UK – (almost) every year since 2014. I’ve been to EuroPython in Berlin, Bilbao and Rimini, and this year it came to the UK – specifically, the beautiful city of Edinburgh.

I presented a new iteration of my Python and Raspberry Pi poster at both conferences this year (click to download):

The talk I developed for this year’s conference season was entitled Programming paradigms for physical computing and IoT. It introduces four programming concepts I feel are necessary for writing effective and meaningful code for describing how devices behave.

Here’s the video from me giving the talk at EuroPython:

https://www.youtube.com/watch?v=pRtpXzS2Weo

I evolved the talk a bit (and cut some stuff out for a shorter talk slot) for PyCon UK:

https://www.youtube.com/watch?v=T2R1JAnL7-I

I also gave a lightning talk about piwheels:

https://www.youtube.com/watch?v=7w_qgGZM4ao&feature=youtu.be&t=303

And part way through a very busy week of PyCon UK, in between running the education summit, presenting my poster, giving a lightning talk and my scheduled talk, I was announced as one of the recipients of the John Pinner Award, which was a really nice way to be recognised by the community that’s given me so much. Thanks to Daniele, Peter, all the other organisers and congratulations to all other recipients.