MagPi interview

Issue #91 of the official Raspberry Pi magazine, The MagPi, has just been published and it features an interview with me, about my time at the Foundation (and a bit before), letting the community know what I’m moving on to at the BBC.

You can buy or download the issue here: Issue #91

Here’s what the spread looks like, pages 82-83:

Here’s a gallery of pictures from my time at Raspberry Pi:

Meme Overflow

A few months ago I was looking through questions posted to the Raspberry Pi Stack Exchange site. If you’re not familiar, Stack Exchange provides Q&A sites like Stack Overflow for specific areas, such as particular programming languages, technologies or other topics. And while observing the erratic nature of the way people post questions to the site, I thought “wouldn’t it be funny to take questions from Stack Exchange and put them on meme templates?” and it seemed like a trivial enough task to automate, so I started googling things, and typing stuff into a Python shell, and within an hour or so I had it working. The results were random and hilarious, as I expected.

Sometimes ideas like this seem simple in theory, but in practice there’s some complexity you didn’t think of that gets in the way. But with this idea, everything was smooth sailing. I found the Stack Exchange API easy to use, and came across the imgflip site, which provides an API. All I needed to be able to do was retrieve question titles from a particular Stack Exchange site, and to add text to a random meme template. Both of these things were trivial, and as I’ve done a lot of things using web APIs in Python lately, all the skills were well practised, which helps a lot.

Once the concept was proven, I decided to set up a Twitter account for the memes. I added code to tweet the images. Again, this is something I’ve done a lot of, so no hassle. I forget how long it took to get the app approved and the bot working, but before long, @pi_stack was tweeting out memes autonomously.

Hit and miss

Meme templates are chosen randomly, so obviously not every question / meme combination will work. Some will never be funny, some won’t be a good match, some will end up on a template that doesn’t really work. Over time I got to see which meme templates would never work, and blacklisted them. I also made sure that text was placed in the correct position for most effect: some templates work better with the text in the second or third position than the first – and as I only have one piece of text to add, this makes all the difference.

Later I decided to add some logic to choose the right meme template based on simple rules like “starts with X”, or to use a different template if it “doesn’t end in Y”. This seemed to make a big difference, though it’s still very hit-and-miss.

The test suite

I decided to create a test suite so I could more confidently make changes to the library without risking breaking things. Even though this is a silly side project, it’s been another way for me to exercise good practice which makes it easier when you need to do this stuff for real.

In gpiozero we have a mock pin interface which allows us to test the workings of the library without running on real Raspberry Pi hardware. And in the test suite we use patch to mock behaviour. I had to learn some new techniques for mocking things like web requests, image uploads and even tweets. I’m sure my methods are far from perfect but it’s a decent attempt with good coverage. Check out the tests and coverage.

Instances

I started with the Raspberry Pi Stack Exchange, and later added other instances:

If you’re not familiar with the concept of “world building”, it’s for people who are designing worlds for books, stories, games and such – and have questions like “What is keeping my Terror Bird from being the size of a T. Rex?” which are perfect for this project.

I’ve tried to set up more (there are so many great Stack Exchange sites) but Twitter is not letting me authenticate more accounts with the same mobile number.

Best in show

Here are some examples that worked well:

One of the best ones ever was extremely unlikely. It was a template that almost never works, because it really requires two pieces of text to make sense. But in this one case, it was perfect. In the RPi.GPIO library, Python warnings are switched on by default, and you have to turn them off. Most people find them annoying and include the line that disables them. The problem with Python warnings is that they look like errors/exceptions – people see red text and think there’s something wrong with their program – when really it’s working fine but it’s just letting them know something unimportant like they’ve already configured that pin. So someone on Raspberry Pi asked about the warning message, posting just the warning text (verbatim) as the question title, where the desired outcome is that no warning is output. And – by chance – this subtle genius comes out of Meme Overflow:

Another recent one was from World Building. Again, completely random:

Is anyone following?

The Stack Overflow bot is the most popular, with nearly 500 followers at the time of writing. Raspberry Pi and Ask Ubuntu have over 100 each.

Run your own instance

If you want to run your own bot following a particular Stack Exchange site, you can pip install memeoverflow and use the example script, following the simple guide.

If you want to run a modified version, you can either fork the project or subclass the provided class.

Mr President

Goodbye Raspberry Pi, Hello BBC

Just a quick note to say I’m leaving the Raspberry Pi Foundation to start a new role at the BBC. I’ve been at Raspberry Pi for over six years, and it’s been great. I’m excited to join an innovation team called BBC News Labs. They find new ways to solve problems with technology, making lives easier for journalists, and they drive innovation within BBC News.

https://twitter.com/ben_nuttall/status/1214912375918383104

New Labs are based at New Broadcasting House in London, so I’ll be partly based there:

W1A

News Labs write up their projects at bbcnewslabs.co.uk – so read up there to find out more.

So goodbye Raspberry Pi!

https://twitter.com/ben_nuttall/status/1220792172993302534

And hello BBC!

https://twitter.com/ben_nuttall/status/1219941583040188416

What’s new in GPIO Zero v1.5?

It’s become customary for me to summarise what each new GPIO Zero release brings. This one’s been a long time coming. It’s been a quite while since our last release (a whole year since the last point release and 18 months since v1.4). I mostly attribute the lack of development to the launch of my other project, piwheels. I think that was time well spent, but I’m sorry to say that GPIO Zero hasn’t come along as far as I’d hoped by now. But hopefully you’ll find the contents of this release to be a satisfying step in the right direction.

Upgrade now:

sudo apt update
sudo apt install python3-gpiozero python-gpiozero

You can see the summary of what GPIO Zero v1.5.0 brings in the changelog. I’m pleased to say it’s the longest changelog entry to date – lots of fairly small changes, plus some pretty big (under-the-hood) ones thrown in for good measure.

So, what’s new?

New device classes

  • PiHutXmasTree
    • The Pi Hut’s 3D Christmas tree board Rachel designed is now natively supported, so you don’t have to create your own LEDBoard with all the pins.
  • PumpkinPi
    • A neat new Halloween themed LED board from ModMyPi
  • JamHat
    • Another new board from ModMyPi, the Jam HAT comprises two sets of traffic light LEDs, two buttons and a tonal buzzer
  • TonalBuzzer
    • As featured on the Jam HAT, this component allows you to make audible tones using PWM
  • LoadAverage
    • A new “internal” device you can use to read your Pi’s load average, or even feed its values into an output device, e.g. an LED or LED bar graph
  • DiskUsage
    • Similarly, another new internal device for showing the disk usage, say of your SD card or other media

Tones

Claire Pollard wrote some RPi.GPIO code to play tones on the tonal buzzer on the Jam HAT, and then converted it to GPIO Zero. I took what she’d done, and wrapped it in a TonalBuzzer class, a composite device taking inspiration from Servo, where the value is not directly tied to the duty cycle, like a standard PWMOutputDevice or a PWMLED, but instead proportional to the mid tone. I also replaced the dictionary look-up for MIDI notes and frequencies with the maths to calculate it. Then Dave moved the note/tone/frequency conversion into its own class, proving all that functionality in one place. On their own, tones could be used elsewhere, similar to the colour manipulation that was previously embedded within picamera, but became colorzero. Here’s a set of examples for creating tones:

>>> from gpiozero.tones import Tone
>>> Tone(frequency=440)
>>> Tone(midi=69)
>>> Tone(note='A4')

And used with TonalBuzzer:

>>> from gpiozero import TonalBuzzer
>>> from gpiozero.tones import Tone
>>> buzzer = TonalBuzzer(20)
>>> buzzer.play(Tone(note='A4'))

You can even set the source of a TonalBuzzer to play a series of tones, a source of artificial values or directly from another device. A siren effect can be achieved by setting the source to a sine wave:

from gpiozero import TonalBuzzer
from gpiozero.tools import sin_values

buzzer = TonalBuzzer()
buzzer.source = sin_values()

Since TonalBuzzer has a value range from -1->1, where -1 is an octave below its mid tone, and 1 is an octave above (by default), that means if you set its source to come from a device whose values are -1->1 you can control its full tonal range, and a device with range 0->1 can control from its mid tone to an octave above, but either way can be effective. Since sin_values outputs values from -1 to 1, it’s a perfect example (as is cos_values). You can use source_delay to speed up or slow down the rate it iterates over the values.

Another good example is DistanceSensor which has values between 0 (no distance to object) and 1 (max distance, configurable). So if you send the DistanceSensor‘s values into a TonalBuzzer you effectively have a theremin:

from gpiozero import TonalBuzzer
from gpiozero.tools import sin_values

buzzer = TonalBuzzer()
buzzer.source = sin_values()

Distance Sensor

Speaking of DistanceSensor, we had identified issues with the distance it reports. Our implementation is complex and uses threading. The inherent complexity is essential for the device to work with other GPIO Zero devices, and without requiring blocking other code from running or using up 100% of the CPU. However, where there is complexity there is scope for bugs. Thanks to some clever work from Dave, we now use timing information provided by underlying drivers, so that timing events from pins will be considerably more accurate (see #655). Also, Dave found that the default pin library, RPi.GPIO, would often miss edges during callbacks, which threw off the timing, so we now drop missed edges and get better accuracy as a result (see #719). We recommend using the pigpio pin factory for the best results, particularly with Pi 1 or Pi Zero.

Source/values

My friend Robie has built his own z-wave home automation project using a GPIO Zero-like API which includes a similar source/values interface, allowing him to easily connect devices together, like a light switch controlling a light but also a humidity sensor (depending on the humidity sensor reading) – and has completely custom rules for all the devices in his house. He suggested to me one day that sometimes you need access to the device object itself when setting source, not just the device’s values, and I realised that this would make the API even simpler as a result. So now you can use:

led.source = btn

Instead of:

led.source = btn.values

Although both methods still work. This also simplifies more complex device rules, as you can replace every instance of device.values with just device, and so:

led.source = all_values(btn1.values, btn2.values, btn3.values, btn4.values)

becomes:

led.source = all_values(btn1, btn2, btn3, btn4)

We also added graphviz diagrams for all the examples in the Source/Values docs page which I think helps visualise the examples:

Other improvements

  • Import time is massively reduced – we used to do an expensive pin factory computation at import time, but we removed that due to pkg_resources being slow as hell.
  • It’s now possible to import gpiozero without a valid pin factory, or any environment variables set.
  • RGBLED now supports colorzero, which is an awesome module previously buried in picamera, but I persuaded Dave it would be a useful as a separate library. It’s also used by Dave’s Sense HAT library pisense. Anyway, now you can now set the colour of an RGBLED using e.g. Color('purple') and do lots of really useful colour manipulation such as gradients.

Finally, a new flag allows you to type:

pinout -x

And it opens up the website pinout.xyz in a web browser.

Now, back to piwheels for a brief spell – we have some awesome stuff in the works – all to be revealed soon – not least the expansion to Python 3.7 for Raspbian Buster – but more on that soon. Then once that’s out the door, back to GPIO Zero. I am planning on upping the release cycle back to one per quarter, like in the good old days. You can count on it!

Raspberry Pi Zero GPIO expander

The recent announcement of the latest release of the Raspberry Pi Desktop x86 image alongside Raspbian Stretch for Raspberry Pi included mention of a GPIO expander tool, which was followed up by another blog post explaining how it works and how to use it. Since it uses pigpio to control the GPIO pins, that means you can use my GPIO Zero Python library to use it. And as of yesterday, you can easily set this up on an Ubuntu PC, not just Raspberry Pi Desktop.

What is it?

The GPIO expander tool means you can connect a Pi Zero or Pi Zero W to a regular PC with just a USB micro cable, no SD card required, and control the GPIO pins. It currently works on Linux only, but support for Windows and Mac should be expected … at some point.

For now, if you don’t use Linux, you can live boot the Raspberry Pi Desktop x86 OS from a DVD or USB stick, and do it from a Windows PC or a Mac, but eventually it should be possible to do natively from those operating systems.

How does it work?

The Pi Zero features a USB OTG port, allowing you to boot over USB from a PC. Your PC sends the required boot firmware to the Pi over the USB cable, launching a mini version of Raspbian and booting it in RAM. The OS then starts the pigpio daemon, allowing “remote” access over the USB cable.

How to use on Raspberry Pi Desktop x86

You can download an ISO of the Raspberry Pi Desktop OS from raspberrypi.org which you can write to a USB stick or burn to a DVD. This must be the Stretch release, not the older Jessie image.

You can use these to live boot your PC or Mac into the OS (select “Run with persistence” and your computer will be back to normal afterwards). The OS comes with everything you need, ready to go.

Plug in your Pi Zero, and you’ll be prompted to select a role for the device. Select GPIO expansion board and continue. It will take 30 seconds or so to flash it, then the dialogue will disappear.

How to use on a Raspberry Pi

Using on Raspbian on a Pi is almost as easy as the x86 image, except that the usbbootgui tool is not pre-installed. You can install it on Raspbian Stretch:

$ sudo apt update
$ sudo apt install usbbootgui

Now when you plug in your Pi Zero, you get the dialogue as above.

How to use on Ubuntu

We have created a PPA of the packages you need to make this work. Start by adding the PPA:

$ sudo add-apt-repository ppa:rpi-distro/ppa
$ sudo apt update

If you have previously installed gpiozero or pigpio with pip, uninstall these first:

$ sudo pip3 uninstall gpiozero pigpio

Then install the required packages from the PPA:

$ sudo apt install usbbootgui python3-gpiozero python3-pigpio

Now when you plug in your Pi Zero, you get the dialogue as above.

Access the GPIOs

Looking at the output of ifconfig, and you’ll see a new ethernet connection with an IPv6 address. On Raspbian and Raspberry Pi Desktop, this will be usb0. On Ubuntu it’s some silly string (mine is enp0s29u1u7i2).

You can ping it (be sure to use ping6 as it’s IPv6 only) using the address fe80::1% followed by the connection string, e.g:

$ ping6 fe80::1%usb0

If you set the PIGPIO_ADDR environment variable, all calls to pigpio therafter will use that as the address to connect to:

$ export PIGPIO_ADDR=fe80::1%usb0

For a simple test, you can use the command line interface provided by pigpio, named pigs:

$ pigs w 25 1 # turn pin 25 on
$ pigs w 25 0 # turn pin 25 off

Enter GPIO Zero

For a while now, GPIO Zero has supported using pigpio as the back-end. This meant you could connect to a Pi on the network from a PC or another Pi, and even control multiple Pis from the same script, but that required the remote Pi to have an SD card and Raspbian running, and the pigpio daemon running, and allowing remote connections. Similarly, Pi Zero OTG mode was possible, but that also required an SD card, and lots of configuration.

Since the GPIO expander tool does exactly the same thing, naturally GPIO Zero just works out of the box! Like connecting to a remote Pi, you just specify that you’re using pigpio as the pin factory, and specify the address (as done earlier). You can do this with another environment variable:

$ export GPIOZERO_PIN_FACTORY=pigpio

Now when you use GPIO Zero within Python, connections will be to the Pi Zero:

Set your environment variables as above, and use python/python3/ipython at the command line, or an IDE. You probably want to set the environment variables globally – just export them in your .profile file like so:

export GPIOZERO_PIN_FACTORY=pigpio
export PIGPIO_ADDR=fe80::1%usb0

Scratch too

Scratch 2 is now included in Raspbian and Raspberry Pi Desktop x86. This includes a GPIO extension which uses pigpio. The x86 version has been adapted to allow access to Pi Zeros over USB:

Multiple Pi Zeros?

In theory, this should work for multiple Pi Zeros. However, it seems there’s an issue with the USB boot tool, knocking the first one out when you add a second.

On Raspberry Pi Desktop each Pi comes up with an enumerated name (usb0, usb1, etc). On Ubuntu mine were enp0s29u1u7i2 and enp0s29u1u8i2, but they’re easy to look up in ifconfig.

In GPIO Zero, if this worked, you’d be able to create devices on multiple devices using the pin_factory parameter:

from gpiozero import TrafficHat
from gpiozero.pins.pigpio import PiGPIOFactory
from signal import pause

usb0 = PiGPIOFactory('fe80::1%usb0')
usb1 = PiGPIOFactory('fe80::1%usb1')

hat0 = TrafficHat(pin_factory=usb0)
hat1 = TrafficHat(pin_factory=usb1)

hat0.button.when_pressed = hat1.lights.on
hat0.button.when_released = hat1.lights.off
hat1.button.when_pressed = hat0.lights.on
hat1.button.when_released = hat0.lights.off

pause()

(this example makes each Traffic HAT’s button control the lights of the other one)

Scratch 2 contains a dropdown for usb0 to usb3:

But as I say, it doesn’t work yet.

Limitations

The major limitations are that you cannot flash code to the Pi Zero, and that not all software is supported.

Since the Pi Zero has booted from your PC, as soon as you disconnect it, any running programs will terminate. There’s no way of keeping a battery attached, or anything like that, so you can only prototype your ideas with it, not deploy any finished projects to it without transitioning to the regular SD card setup.

Also, since GPIO access relies on the pigpio library, only software compatible with pigpio will work. Since GPIO Zero features multiple back-ends (you can use a number of different low-level pin libraries to do the GPIO stuff but the API for the code you write remains the same), and supports pigpio, then anything in GPIO Zero will work just fine.

However, things like Pimoroni’s extensive collection of HATs, which are written using libraries which do not support remote GPIO, just direct GPIO/SPI/I2C, will not be compatible. If you have GPIO software which is built on top of RPi.GPIO, wiringpi, spidev and others, you will need to re-implement to use pigpio in order to gain support for the GPIO expander. This includes the Sense HAT.

Also, the Scratch support is currently limited to Scratch 2. While Scratch 2 is a better and more modern interface, some people prefer Scratch 1 as it’s much faster (hugely optimised Smalltalk vs. late Adobe Flash port). In theory, Scratch 1 should be able to access Pi Zeros using this method, as it also uses pigpio for GPIO. However, it seems to be hard-coded to use localhost, and the code doesn’t seem accessible. Perhaps this will be addressed in a future release.

Workshops

This is a real game changer for Raspberry Jams, Code Clubs, CoderDojos and use in schools. You can live boot the Raspberry Pi Desktop OS from a USB stick, use Linux PCs or even install Raspberry Pi Desktop on old computers. Then you have really simple access to physical computing without full Raspberry Pi setups, and no SD cards to configure.