<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>Ben Nuttall's blog</title>
  <link href="https://bennuttall.com/blog/" />
  <link href="https://bennuttall.com/blog/atom.xml" rel="self" />
  <id>https://bennuttall.com/blog/</id>
  <author>
    <name>Ben Nuttall</name>
  </author>
  <updated>2026-05-07T12:14:59+00:00</updated>

  <entry>
    <title><![CDATA[Getting Claude to extract data from a 1997 football manager game]]></title>
    <link href="https://bennuttall.com/blog/2026/04/fsm97/" />
    <id>https://bennuttall.com/blog/2026/04/fsm97</id>
    <published>2026-04-29T19:27:19+00:00</published>
    <updated>2026-04-29T19:27:19+00:00</updated>
    <category term="python" />
    <category term="claude" />
    <category term="ai" />
    <category term="web-development" />
    <category term="github" />
    <category term="open-source" />
    <category term="football" />
    <summary><![CDATA[There's a football manager game I got for my birthday when I was nine years old: FIFA Soccer Manager 97. I spent countless hours playing it as a child, trying to win the league, the cup or just avoid getting the sack! Two years later I got a newer manager game with better graphics and more...]]></summary>
    <content type="html"><![CDATA[<p>There's a football manager game I got for my birthday when I was nine years old: <a href="https://www.mobygames.com/game/6314/fifa-soccer-manager/">FIFA Soccer
Manager</a>
97. I spent countless hours playing it as a child, trying to win the league, the cup or just avoid
getting the sack! Two years later I got a newer manager game with better graphics and more complex
gameplay but it ran incredibly slow and I didn't enjoy playing it at all, so I stuck with the 1997
one.</p>
<p>Every couple of years I get it out again and spend an evening basking in nostalgia and reliving the
experience. Originally it was made for Windows 95, and it was still working on Windows until (I
think) Windows 10. But it works perfectly well in Wine on Linux, so I can still play it today. It
just has a teeny tiny resolution and looks silly on my 42" monitor.</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2026/04/fsm97/images/fsm-screenshot-huge-monitor.webp" />
</figure>

<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2026/04/fsm97/images/fsm-swfc.webp" />
</figure>

<p>I wondered if I could get <a href="https://claude.ai">Claude</a> to extract data from the game files. I pointed
it at the directory my Wine installation installed to. I told it I was only interested in players,
teams, stadiums and that kind of thing - the stuff that maps to the real world of football rather
than internal game data. It found everything in the file <code>SM97.DAT</code> and was very quickly able to
answer simple questions like which is the biggest stadium or who is the best rated player.</p>
<p>I asked it to make an HTML page summarising all the data. It produced this:
<a href="https://files.bennuttall.com/fsm97/">https://files.bennuttall.com/fsm97/</a></p>
<p>I asked it to create CSV files of all the data it found, so I could make sense of it and make sure
it looked right. There were a few issues with it messing up the data but it was easily able to fix
it. It didn't know what all the column names for the player stats were, so I launched the game and
wrote them all down along with David Seaman's stats so it could calibrate. I expanded the scope to
all players and clubs, not just the English leagues.</p>
<p>Once I was happy with the CSV files, all future queries would be scripted against these files so I
could be sure it was coming from extracted data and not extracting from the game data again or even
hallucinating. I then asked for a more comprehensive website of all the data, with lots of
interlinking. I was really impressed with what it put together, and I spent some time diving deeper
into the data and making tweaks to the website.</p>
<p>I wanted to produce a set of Python code to make this process reproducible, so if anyone else wanted
to do it they could do so without needing an AI tool of their own. It's now on
<a href="https://github.com/bennuttall/fsm-97-data">GitHub</a>, and I've published the website too:
<a href="https://fsm.bennuttall.com/">fsm.bennuttall.com</a></p>
<figure class="wp-block-image">
<a href="https://fsm.bennuttall.com/"><img src="https://bennuttall.com/blog/2026/04/fsm97/images/fsm-website.webp" /></a>
<figcaption>fsm.bennuttall.com</figcaption>
</figure>

<p>I've worked with Claude to fine-tune the way the website works, trying to interlink everything and
make it easy to find what you're looking for, or explore the data looking for interesting things.</p>
<h2>Stats, trivia and EA All Stars</h2>
<p>I always found it annoying that it only ever used short versions of team names like "Sheffield W"
instead of "<a href="https://fsm.bennuttall.com/clubs/sheffield-wednesday/">Sheffield Wednesday</a>",
purely to keep the strings short enough to use everywhere. I had Claude correct them to their proper
titles. Stadium names are not used in the game, but they all exist in the game data, albeit with odd
mistakes sometimes, like "<a href="https://fsm.bennuttall.com/stadiums/bramall-lane/">Bramall Lane
Ground</a>". I corrected those too. Some more
obscure club or stadium names were just wrong so I fixed those too. I didn't want to mess with too
much of the game data, but felt these changes were reasonable.</p>
<p>One thing I found interesting was that there's data in there that isn't used in the game at all.
Players go by first initial and Surname, like "D. Beckham", but the data has "<a href="https://fsm.bennuttall.com/clubs/manchester-united/#david-beckham">David
Beckham</a>". It has all the
manager names of all English leagues and a few others, but they're never used in game. Same for club
nicknames and stadium names (and sometimes town/city and even first line of address!).</p>
<p>This revealed oddities like P. Shilton in the game actually being former England goalkeeper <a href="https://fsm.bennuttall.com/clubs/leyton-orient/#peter-shilton">Peter
Shilton</a>, who really did play for
<a href="https://fsm.bennuttall.com/clubs/leyton-orient/">Leyton Orient</a> until 1997, at the age of 47;
and oddly, Olympic decathlon champion <a href="https://fsm.bennuttall.com/clubs/mansfield-town/#daley-thompson">Daley
Thompson</a> at <a href="https://fsm.bennuttall.com/clubs/mansfield-town/">Mansfield
Town</a> (an easter egg).</p>
<p>I didn't realise there were some clubs that shared stadiums, which is reflected in the game. I
wonder if you managed <a href="https://fsm.bennuttall.com/clubs/wimbledon/">Wimbledon</a> and expanded the
stadium (<a href="https://fsm.bennuttall.com/stadiums/selhurst-park/">Selhurst Park</a>), if you played against
<a href="https://fsm.bennuttall.com/clubs/crystal-palace/">Crystal Palace</a> away, would you see the ground at
its default appearance or would you in fact see your own upgraded home stadium? (The Reddit
community confirms they do appear as different stadiums!)</p>
<p>I made sure the extracted data was aware of the concept of shared stadiums, and it also picked up on
the fact that some of the clubs' managers were also listed as players for the same club - a once
popular "player-manager" role.</p>
<p>I put together a few special pages showing some interesting stats, such as <a href="https://fsm.bennuttall.com/stats/top-players/">top rated
players</a>, <a href="https://fsm.bennuttall.com/stats/age-groups/">top 10 best players in each age
group</a>, <a href="https://fsm.bennuttall.com/stats/player-managers/">top rated
player-managers</a> and <a href="https://fsm.bennuttall.com/stats/stadiums/">stadiums by
capacity</a>.</p>
<figure class="wp-block-image">
<a href="https://fsm.bennuttall.com/stats/"><img src="https://bennuttall.com/blog/2026/04/fsm97/images/fsm-stats.webp" /></a>
</figure>

<p>One thing that stands out in top player stats is a set of highly rated players from a club called
"<a href="https://fsm.bennuttall.com/clubs/ea-all-stars/">EA All Stars</a>". None of these are real players —
they're actually the game developers and other staff who worked on the game. The Assistant Producer
<a href="https://fsm.bennuttall.com/clubs/ea-all-stars/#mark-bergan">Mark Bergan</a> made himself one of the
best rated players in the whole game, rated as highly as <a href="https://fsm.bennuttall.com/clubs/arsenal/#david-seaman">David
Seaman</a>, <a href="https://fsm.bennuttall.com/clubs/ac-milan/#george-weah">George
Weah</a> and
<a href="https://fsm.bennuttall.com/clubs/spare/#unknown-romario">Romario</a>.</p>
<figure class="wp-block-image">
<a href="https://fsm.bennuttall.com/stats/"><img src="https://bennuttall.com/blog/2026/04/fsm97/images/fsm-ea-all-stars.webp" /></a>
<figcaption>EA All Stars</figcaption>
</figure>

<h2>Data extraction method</h2>
<p>The <a href="https://github.com/bennuttall/fsm-97-data/blob/main/data-extraction.md">data extraction method</a>
is explained in detail on GitHub. Here's an example:</p>
<p>David Seaman is the first player record in Arsenal's block. His raw 87 bytes:</p>
<div class="codehilite"><pre><span></span><code> 0:  44 61 76 69 64 00 00 00 00 00 00 00 00 00 00 00   David...........
16:  00 00 00 00 00 00 00 00 53 65 61 6d 61 6e 00 00   ........Seaman..
32:  00 00 00 00 00 00 00 00 00 00 00 1a 00 00 01 48   ...............H
48:  5e 46 47 4a 48 10 48 19 2f 1a 57 5d 2c 2a 18 57   ^FGJH.H./.W],*.W
64:  57 5f 1d 50 58 40 23 00 00 50 b6 52 01 e9 5a 04   W_.PX@#..P.R..Z.
80:  04 01 ff 8a 05 00 00                               .......
</code></pre></div>

<h3>Names</h3>
<table>
<thead>
<tr>
<th>Bytes</th>
<th>Hex</th>
<th>Value</th>
</tr>
</thead>
<tbody>
<tr>
<td>[0:6]</td>
<td><code>44 61 76 69 64 00</code></td>
<td><code>David</code></td>
</tr>
<tr>
<td>[24:30]</td>
<td><code>53 65 61 6d 61 6e 00</code></td>
<td><code>Seaman</code></td>
</tr>
</tbody>
</table>
<h3>Stats block (starting at byte 42)</h3>
<div class="codehilite"><pre><span></span><code>Byte 42+1  = 0x1a = 26   → nationality index 26 (England)
Byte 42+2  = 0x00 =  0   → position GK
Byte 42+4  = 0x01 =  1   → shirt number 1
</code></pre></div>

<h3>Skills (stats[5:28] = bytes 47–69)</h3>
<div class="codehilite"><pre><span></span><code><span class="mf">48</span><span class="w"> </span><span class="mf">5</span><span class="n">e</span><span class="w"> </span><span class="mf">46</span><span class="w"> </span><span class="mf">47</span><span class="w"> </span><span class="mf">4</span><span class="n">a</span><span class="w"> </span><span class="mf">48</span><span class="w"> </span><span class="mf">10</span><span class="w"> </span><span class="mf">48</span><span class="w"> </span><span class="mf">19</span><span class="w"> </span><span class="mf">2</span><span class="n">f</span><span class="w"> </span><span class="mf">1</span><span class="n">a</span><span class="w"> </span><span class="mf">57</span><span class="w"> </span><span class="mf">5</span><span class="n">d</span><span class="w"> </span><span class="mf">2</span><span class="n">c</span><span class="w"> </span><span class="mf">2</span><span class="n">a</span><span class="w"> </span><span class="mf">18</span><span class="w"> </span><span class="mf">57</span><span class="w"> </span><span class="mf">57</span><span class="w"> </span><span class="mf">5</span><span class="n">f</span><span class="w"> </span><span class="mf">1</span><span class="n">d</span><span class="w"> </span><span class="mf">50</span><span class="w"> </span><span class="mf">58</span><span class="w"> </span><span class="mf">40</span>
</code></pre></div>

<table>
<thead>
<tr>
<th>Attribute</th>
<th>Hex</th>
<th>Value</th>
</tr>
</thead>
<tbody>
<tr>
<td>Speed</td>
<td><code>48</code></td>
<td>72</td>
</tr>
<tr>
<td>Agility</td>
<td><code>5e</code></td>
<td>94</td>
</tr>
<tr>
<td>Acceleration</td>
<td><code>46</code></td>
<td>70</td>
</tr>
<tr>
<td>Stamina</td>
<td><code>47</code></td>
<td>71</td>
</tr>
<tr>
<td>Strength</td>
<td><code>4a</code></td>
<td>74</td>
</tr>
<tr>
<td>Fitness</td>
<td><code>48</code></td>
<td>72</td>
</tr>
<tr>
<td>Shooting</td>
<td><code>10</code></td>
<td>16</td>
</tr>
<tr>
<td>Passing</td>
<td><code>48</code></td>
<td>72</td>
</tr>
<tr>
<td>Heading</td>
<td><code>19</code></td>
<td>25</td>
</tr>
<tr>
<td>Control</td>
<td><code>2f</code></td>
<td>47</td>
</tr>
<tr>
<td>Dribbling</td>
<td><code>1a</code></td>
<td>26</td>
</tr>
<tr>
<td>Coolness</td>
<td><code>57</code></td>
<td>87</td>
</tr>
<tr>
<td>Awareness</td>
<td><code>5d</code></td>
<td>93</td>
</tr>
<tr>
<td>Tackling Det.</td>
<td><code>2c</code></td>
<td>44</td>
</tr>
<tr>
<td>Tackling Skill</td>
<td><code>2a</code></td>
<td>42</td>
</tr>
<tr>
<td>Flair</td>
<td><code>18</code></td>
<td>24</td>
</tr>
<tr>
<td>GK Kick</td>
<td><code>57</code></td>
<td>87</td>
</tr>
<tr>
<td>GK Throw</td>
<td><code>57</code></td>
<td>87</td>
</tr>
<tr>
<td>GK Handling</td>
<td><code>5f</code></td>
<td>95</td>
</tr>
<tr>
<td>Throw-in</td>
<td><code>1d</code></td>
<td>29</td>
</tr>
<tr>
<td>Leadership</td>
<td><code>50</code></td>
<td>80</td>
</tr>
<tr>
<td>Consistency</td>
<td><code>58</code></td>
<td>88</td>
</tr>
<tr>
<td>Determination</td>
<td><code>40</code></td>
<td>64</td>
</tr>
</tbody>
</table>
<h3>Remaining stats</h3>
<div class="codehilite"><pre><span></span><code>stats[28] = 0x23 = 35   → Greed
stats[30] = 0x00 =  0   → Form
stats[31] = 0x50 = 80   → Energy
stats[35] = 0xe9 = 233  ⎤
stats[36] = 0x5a =  90  ⎦ → 233 + 90×256 = 23273 days → 1963-09-19 (DOB)
</code></pre></div>

<p>Age on 29 July 1996: <strong>32</strong>.</p>
<h2>The 2079 Bug</h2>
<p>I'd read on the <a href="https://www.reddit.com/r/fisom/">thriving Reddit community</a> for this game about the
"2079 bug". Apparently, if you play the game through to 2079, at some point the age of players hits
an overflow and they all think it's time to retire.</p>
<p>This is because each player's date of birth is stored as a 16-bit little-endian unsigned integer:
the number of days elapsed since 30 December 1899 (the same base date used by Microsoft Excel and
the game's internal date system). The two bytes are combined as:</p>
<div class="codehilite"><pre><span></span><code>days = stats[35] + stats[36] * 256
dob  = date(1899, 12, 30) + timedelta(days=days)
</code></pre></div>

<p>This is a WORD (16-bit), so the maximum representable date is day 65,535 — 5 October 2079. After
that point the game overflows and all player ages are calculated incorrectly. This is known as the
2079 Bug.</p>
<p>I've never played the game that far. I have occasionally taken it to the current day (as recently as
the 2020s), but it's known that the game has limitations in endurance.</p>
<h2>Full motion video cutscenes</h2>
<p>The game features a number of full motion video cutscenes. These include specially-filmed
live-action football footage, heavily tinted blue/purple, with overlaid title text in a
typewriter-style font. They were shown when you won or lost a cup final, won the league, got
promoted, relegated, or sacked. There was also an intro and a closing credits video.</p>
<p>I knew these videos would be in the data too, though I wouldn't have known what format or how they'd
be stored. Claude managed to find and identify them:</p>
<blockquote>
<p>The game's video files use EA's proprietary TGQ format (<code>.TGQ</code> extension), also known as EA TGQ or
EABT. The container uses EA's chunk format, identifiable by the <code>SCHl</code> magic bytes at the start of
each file.</p>
</blockquote>
<p>I hoped it would be able to maybe extract the frames, but it had no problem decoding the video
files.</p>
<p><a href="https://www.ffmpeg.org/">ffmpeg</a> can decode these files natively using the <code>ea</code> demuxer:</p>
<div class="codehilite"><pre><span></span><code>ffmpeg -i INPUT.TGQ -c:v libx264 -c:a aac OUTPUT.mp4
</code></pre></div>

<p>Docs and explanation of the video files can be found here:
<a href="https://github.com/bennuttall/fsm-97-data/blob/main/docs/videos.md">github.com/bennuttall/fsm-97-data/blob/main/docs/videos.md</a></p>
<figure class="wp-block-image">
<iframe width="560" height="315" src="https://www.youtube.com/embed/AgQDXbmpIKA?si=KASZvIvw091DlPaq" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
</figure>

<h2>Credits</h2>
<p>The end credits are something of a masterpiece. The game developers and all the staff who worked on
the game are credited, along with childhood photos and aspects of what they worked on, including
example lines of C/C++ code, stadium sketches, colours and coordinates, specs and more:</p>
<figure class="wp-block-image">
<iframe width="560" height="315" src="https://www.youtube.com/embed/S8Ir0qe_7p8?si=z0i5-yXJKGEoDjnz" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
</figure>

<p>Since many of these staffers are also included in the game, I was able to create a <a href="https://fsm.bennuttall.com/credits/">credits
page</a> listing them and link to their player listings on the <a href="https://fsm.bennuttall.com/clubs/ea-all-stars/">EA
All Stars</a>.</p>
<h2>Join the project</h2>
<p>If you have a copy of the game, head over to the <a href="https://github.com/bennuttall/fsm-97-data">GitHub</a>
page and see if you can extract the data yourself, and have a play with it, maybe find things I've
not found yet.</p>
<p>There are a few other things I'd like to investigate. I wasted a bunch of time (and Claude usage)
trying to extract the stadium graphics, to no avail. I was hoping to extract one of these for each
club:</p>
<figure class="wp-block-image">
<a href="https://fsm.bennuttall.com/stadiums/highbury/"><img src="https://bennuttall.com/blog/2026/04/fsm97/images/fsm-highbury.webp" /></a>
<figcaption>Highbury</figcaption>
</figure>

<figure class="wp-block-image">
<a href="https://www.youtube.com/watch?v=S8Ir0qe_7p8"><img src="https://bennuttall.com/blog/2026/04/fsm97/images/fsm-the-end.webp" /></a>
<figcaption>The end?</figcaption>
</figure>]]></content>
  </entry>
  <entry>
    <title><![CDATA[Glen Etive]]></title>
    <link href="https://bennuttall.com/blog/2026/04/glen-etive/" />
    <id>https://bennuttall.com/blog/2026/04/glen-etive</id>
    <published>2026-04-19T21:17:29+00:00</published>
    <updated>2026-04-19T21:17:29+00:00</updated>
    <category term="kayaking" />
    <category term="travels" />
    <category term="scotland" />
    <summary><![CDATA[I realised I hadn't blogged anything outdoorsy for an awfully long time. I used to occasionally post about travels and adventures, or my kayaking in general, but I guess I haven't done anything blogworthy in a while. I've been kayaking (on-and-off) for over twenty years, mostly in the Lake District...]]></summary>
    <content type="html"><![CDATA[<p>I realised I hadn't blogged anything outdoorsy for an awfully long time. I used to occasionally post
about <a href="/blog/tags/travel">travels</a> and <a href="/blog/tags/adventure">adventures</a>, or my
<a href="/blog/tags/kayaking">kayaking</a> in general, but I guess I haven't done anything blogworthy in a
while.</p>
<p>I've been kayaking (on-and-off) for over twenty years, mostly in the <a href="/blog/tags/lake-district">Lake
District</a> and <a href="/blog/tags/wales">North Wales</a>, but also
<a href="/blog/tags/france">France</a>, <a href="/blog/tags/austria">Austria</a>, <a href="/blog/tags/switzerland">Switzerland</a>
and the <a href="/blog/tags/usa">USA</a>.</p>
<p>I haven't paddled a lot in <a href="/blog/tags/scotland">Scotland</a>. Just the
<a href="https://www.ukriversguidebook.co.uk/rivers/scotland/southern-uplands/river-nith-glen-airlie-picnic-site-to-drumlanrig-bridge/">Nith</a>
(a long time ago) and more recently the
<a href="https://www.ukriversguidebook.co.uk/rivers/scotland/southern-uplands/river-esk-langholm-canonbie/">Esk</a>,
both just over the Scottish border. I managed to drop my phone in the river on the Esk last year -
luckily and bizarrely it was found two weeks later by a fly fisherman who managed to access my
emergency contacts on the lock screen. I <em>just happened</em> to be in Scotland at the time, and arranged
to pick up my phone from him in Langholm on the way home, before I'd got around to replacing it.</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2026/04/glen-etive/images/found-lost-phone.webp" />
<figcaption>The found lost phone, with the last photo taken</figcaption>
</figure>

<p>Big shout out to the maker of the <a href="https://www.amazon.co.uk/dp/B06Y21DLWB?th=1">waterproof case</a>!</p>
<p>A bunch of us went for a weekend trip to the Scottish Highlands last month. We were especially
excited about doing the <a href="https://www.ukriversguidebook.co.uk/rivers/scotland/west-highlands/river-etive-triple-falls-to-the-allt-achaorainn/">River
Etive</a>
in the stunningly beautiful <a href="https://en.wikipedia.org/wiki/Glen_Etive">Glen Etive</a> near <a href="https://en.wikipedia.org/wiki/Glen_Coe">Glen
Coe</a> in the Scottish Highlands. I've been wanting to do it
for many years but never had the chance. It's a river that's full of exciting drops with names like
"letterbox", "ski jump", "twist and shout", "crack of doom" and "crack of dawn". I've always been
envious of the amazing photos I've seen of people running what I now know is called "<a href="https://www.ukriversguidebook.co.uk/rivers/scotland/west-highlands/photos/river-etive-right-angle-falls">right angle
falls</a>".</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2026/04/glen-etive/images/maps.webp" />
<figcaption>It's a <em>really</em> long drive</figcaption>
</figure>

<p>We arrived at the Kingshouse Hotel, where we were staying (in the bunkhouse next to the hotel), to
find a stag gracefully meandering the hotel car park:</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2026/04/glen-etive/images/kingshouse-stag.webp" />
</figure>

<p>In the morning we woke to this amazing view:</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2026/04/glen-etive/images/kingshouse-view.webp" />
<figcaption>The morning view from the Kingshouse Hotel</figcaption>
</figure>

<p>We drove a few minutes to the river in the stunning Glen Etive:</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2026/04/glen-etive/images/glen-etive.webp" />
<figcaption>Glen Etive</figcaption>
</figure>

<p>The river starts with three consecutive drops called "triple falls". A great way to warm up! Only
one of us in the group had paddled the Etive before, but Connah had read up on all the features, so
we stopped to scout everything to make sure we were prepared. Here's a video of me on triple
falls: <a href="https://www.youtube.com/shorts/-pzJEMfXEcw">https://www.youtube.com/shorts/-pzJEMfXEcw</a></p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2026/04/glen-etive/images/etive-kayakers.webp" />
</figure>

<figure class="wp-block-image">
<iframe width="560" height="315" src="https://www.youtube.com/embed/9kbu1WdKYSE?si=MvJe6YGK9GWPQFen" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
</figure>

<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2026/04/glen-etive/images/etive-connah.webp" />
<figcaption>Connah</figcaption>
</figure>

<p>Finally we arrived at "right angle falls". It's a 20-foot waterfall with a right angle immediately
before the drop, so you've got to make sure you don't capsize right above it!</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2026/04/glen-etive/images/etive-right-angle-falls.webp" />
<figcaption>Right angle falls</figcaption>
</figure>

<p>It looked extremely daunting — but the right angle is what worries you. You're expecting it to catch
you and send you down the fall backwards or upside-down, which is a little terrifying. Anyway, we
all successfully made our way down it without issue!</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2026/04/glen-etive/images/etive-after-ra-falls.webp" />
<figcaption>Right after the falls</figcaption>
</figure>

<p>Here's the video of me doing it:</p>
<figure class="wp-block-image">
<iframe width="560" height="315" src="https://www.youtube.com/embed/dJTzqlFzBX0?si=Fci9diFSkeszhVt9" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
</figure>

<p>Another angle:</p>
<figure class="wp-block-image">
<iframe width="560" height="315" src="https://www.youtube.com/embed/AO8FcjMtexI?si=bzsFUqN2YyQtbVFf" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
</figure>

<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2026/04/glen-etive/images/etive-matt-after-ra-falls.webp" />
<figcaption>Matt after we did right angle falls</figcaption>
</figure>

<p>So far the trip had gone without incident. We went back to the start and did another run of triple
falls, and Kelvin managed to send himself down the middle drop backwards:
<a href="https://www.youtube.com/shorts/JtmX0p4xmfc">https://www.youtube.com/shorts/JtmX0p4xmfc</a></p>
<p>A great trip! I can't wait to go back.</p>]]></content>
  </entry>
  <entry>
    <title><![CDATA[Beemo Analytics]]></title>
    <link href="https://bennuttall.com/blog/2026/04/beemo-analytics/" />
    <id>https://bennuttall.com/blog/2026/04/beemo-analytics</id>
    <published>2026-04-11T22:54:39+00:00</published>
    <updated>2026-05-01T01:19:45+00:00</updated>
    <category term="python" />
    <category term="blogging" />
    <category term="claude" />
    <category term="ai" />
    <category term="web-development" />
    <category term="meta" />
    <category term="github" />
    <category term="open-source" />
    <category term="dave-jones" />
    <category term="beemo" />
    <summary><![CDATA[Last Summer I wrote about building my own static site generator called Beemo. I went on to use it as the generator for a few other websites I manage, and made a few enhancements along the way.   Beemo  Recently I had the idea to use lars (a web server log analyser written by Dave Jones) as the...]]></summary>
    <content type="html"><![CDATA[<p>Last Summer I wrote about <a href="/blog/2025/08/another-website-revamp/">building my own static site
generator</a> called <a href="https://pypi.org/project/beemo/">Beemo</a>.
I went on to use it as the generator for a few other websites I manage, and made a few enhancements
along the way.</p>
<figure class="wp-block-image">
<a href="https://github.com/bennuttall/beemo"><img src="https://bennuttall.com/blog/2026/04/beemo-analytics/images/beemo.webp" /></a>
<figcaption>Beemo</figcaption>
</figure>

<p>Recently I had the idea to use <a href="/blog/2020/06/lars/">lars</a> (a web server log analyser written by
Dave Jones) as the basis of an analytics site builder. We use lars in the
<a href="https://www.piwheels.org/">piwheels</a> project to count <a href="https://blog.piwheels.org/2024/07/half-a-billion-downloads/">wheel
downloads</a>, and also web page hits for
the few pages we have on the site.</p>
<p>I've been using the <a href="https://claude.ai/">Claude</a> CLI to help me with tasks recently so I wanted to
guide it through building out what I had in mind.</p>
<p>I took the log processing lars code from piwheels, stripped out some piwheels-specific stuff and
asked Claude to make it generate a CSV file of just requests to HTML index pages with an HTTP 200
status code, given a gzipped Apache log file. We iterated on it, adding user agent parsing and bot
detection, and then I asked it to turn it into a library instead of just a script.</p>
<p>My process with Claude is to start by giving it some background context, pointing it at some
libraries and things I want it to observe, then iterate by guiding it to do one task at a time with
a clear goal, reviewing the approach — the same way I would build something myself. I wish I'd been
committing along the way — and it would probably make sense to store the prompts along with the
commit message, otherwise you end up with very large diffs you can't easily review, wind back or
investigate regressions. I need to get better at that.</p>
<p>If I want to explore an idea but don't know exactly what I want, that's when I'm less prescriptive.
Once we'd built the log parsing how I wanted, I wondered if Claude could turn it into an analytics
site by itself — I let it have a go:</p>
<blockquote>
<p>Make an HTML stats page based on the one CSV</p>
</blockquote>
<p>I had no opinions about how it should lay out the HTML, CSS and JavaScript, or which graphing
library to use — or even what I wanted it to include. But what it did produce looked ideal — simple
and modern design reporting on numbers, graphs, and lists of pages.</p>
<p>I then wanted to see if we could integrate this with Beemo:</p>
<blockquote>
<p>The website is generated by my static site builder (beemo) and the content is in a git repo. I've
checked out both repos here. Since we have access to the generator, the exact structure of the
output and the page metadata (titles etc), it might be a good idea to add the lars analytics to
beemo, and be able to extract the page title and such in reporting. Inspect both beemo and
web-content and make a plan for how we can integrate lars log analysis and reporting for a site
like this.</p>
</blockquote>
<p>Now we had page titles, and it added labels for page types (page, post, tag, etc).</p>
<p>I gave it a month's worth of logs, and initially it made an HTML page per log file, as an
extrapolation of the initial task of doing it for a single log file, which isn't really what I
wanted. I assumed we'd need to combine them all to summarise everything, so I asked:</p>
<blockquote>
<p>If I want a summary combining all data from all CSVs should we combine them in an sqlite db or
something?</p>
</blockquote>
<p>Interestingly, it pushed back rather than assuming I was right or that's what I wanted. We pressed
on just using the combined CSVs, and continued a few more iterations making tweaks. We then moved
the log processing and analytics site generation code into the Beemo library, giving <code>beemo</code> three
subcommands:</p>
<div class="codehilite"><pre><span></span><code>build       Build the site.
logs        Process Apache log gz files into CSV.
analytics   Generate HTML analytics site from log CSVs.
</code></pre></div>

<p>We iterated on some finer details — what to exclude, separating page hits with blog posts and things
like tag and archive pages, date and number formatting, and colouring the bot user agent bars in
grey.</p>
<p>I finished things off, added a <a href="https://beemo.readthedocs.io/">readthedocs</a> site and prepared a new
release for PyPI, and configured it to automatically build the analytics site on my web server.</p>
<p>I now have a content-aware bot-aware self-hosted analytics site for my website:</p>
<figure class="wp-block-image">
<a href="https://files.bennuttall.com/beemo-analytics-demo"><img src="https://bennuttall.com/blog/2026/04/beemo-analytics/images/beemo-analytics.webp" /></a>
<figcaption>Beemo analytics</figcaption>
</figure>

<figure class="wp-block-image">
<a href="https://files.bennuttall.com/beemo-analytics-demo"><img src="https://bennuttall.com/blog/2026/04/beemo-analytics/images/beemo-analytics-2.webp" /></a>
<figcaption>Top posts</figcaption>
</figure>

<figure class="wp-block-image">
<a href="https://files.bennuttall.com/beemo-analytics-demo"><img src="https://bennuttall.com/blog/2026/04/beemo-analytics/images/beemo-analytics-3.webp" /></a>
<figcaption>Best of the rest</figcaption>
</figure>

<p>I've archived a snapshot of the current analytics by means of a demo:
<a href="https://files.bennuttall.com/beemo-analytics-demo/">https://files.bennuttall.com/beemo-analytics-demo</a></p>
<p>I've configured analytics for the four websites I'm currently generating with Beemo on my server,
thanks to logrotate, Apache vhosts, certbot and a cron job, it's all working smoothly.</p>
<p>It's early days, I'm bound to want to explore this more, and will probably find that it doesn't work
effectively for a large dataset, but I'll see how it goes and update it as needed.</p>
<p><em>Please note that despite the enthusiasm for Claude, and the use of em dashes throughout, I can
confirm I have written this post myself.</em></p>
<p>What are you waiting for? Try <s>Claude</s> Beemo today!</p>]]></content>
  </entry>
  <entry>
    <title><![CDATA[PyCon UK 2025]]></title>
    <link href="https://bennuttall.com/blog/2025/10/pyconuk-2025/" />
    <id>https://bennuttall.com/blog/2025/10/pyconuk-2025</id>
    <published>2025-10-03T01:01:55+00:00</published>
    <updated>2025-10-03T01:01:55+00:00</updated>
    <category term="python" />
    <category term="pyconuk" />
    <category term="manchester" />
    <category term="pydantic" />
    <category term="pycon" />
    <category term="talks" />
    <summary><![CDATA[Last week was PyCon UK, which took place in Manchester for the first time. I've attended the conference every year it's run since 2014, and each year I've given a talk. This year I decided not to submit a talk, instead trying my hand at running a workshop on "Data modelling with Pydantic". Luckily,...]]></summary>
    <content type="html"><![CDATA[<p>Last week was <a href="https://2025.pyconuk.org/">PyCon UK</a>, which took place in Manchester for the first
time. I've attended the conference every year it's run since 2014, and each year I've given a talk.
This year I decided not to submit a talk, instead trying my hand at running a workshop on "Data
modelling with Pydantic". Luckily, my proposal was accepted and over the last few weeks I've been
preparing for it. It ended up being a <em>lot</em> more effort preparing a 90 minute workshop than a 25
minute talk!</p>
<p>PyCon UK is a unique conference in many ways. One of the <em>many</em> inclusive initiatives it runs is a
"young coders" track on the Saturday, giving local kids and often the children of attendees the
chance to have a go at some programming activities. Workshops are operated on a proposal basis like
the main conference, so they're run by attendees.</p>
<p>The day before the conference started, Claire Wicher asked me to see if I could step in to run a
workshop on the kids track. I'm not very good at saying no - so I said I'd find something. I asked
around and my good friend Nicholas Tollervey was able to provide me with a brilliant worksheet using
Dan Pope's <a href="https://pygame-zero.readthedocs.io/en/stable/">pygame-zero</a> and the
<a href="https://github.com/mu-editor/mu">Mu editor</a> (unfortunately, now a retired project).</p>
<p>The evening before the first day, as attendees started arriving in town, a bunch of people ended up
in the <a href="https://share.google/Ma3XYO9ncyKDHdosk">Temple of Convenience</a>, an old public toilet on
Oxford Street, converted into a bar. We proceeded to move to
<a href="https://www.societymanchester.com/">Society</a> to get food, and I remembered that <a href="https://en.wikipedia.org/wiki/Eric_Idle">Eric
Idle</a> of <a href="https://en.wikipedia.org/wiki/Monty_Python">Monty
Python</a> was playing a show over the road from there at
<a href="https://www.bridgewater-hall.co.uk/">Bridgewater Hall</a>. I asked one of my new friends if he fancied
seeing if there were walk-in tickets available, and we ended up seeing the show, which was great
fun.</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2025/10/pyconuk-2025/images/bridgewater-hall.webp" />
</figure>

<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2025/10/pyconuk-2025/images/eric-idle.webp" />
</figure>

<p>The conference opened with the usual introduction, which included Becky Smith unveiling the new
PyCon UK logo, based on the <a href="https://en.wikipedia.org/wiki/Symbols_of_Manchester#Worker_bee">Manchester worker
bee</a> representing the hard work done
by volunteers to make the conference happen:</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2025/10/pyconuk-2025/images/pyconuk-manchester-logo.webp" />
</figure>

<p>The opening keynote was a brilliant commentary on the history, growth and development of Python's
rich and complex ecosystem since the launch of Python 3:</p>
<figure>
<iframe width="560" height="315" src="https://www.youtube.com/embed/gDvwRpl9erE?si=_wnWPcL4D5soxxG-" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
</figure>

<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2025/10/pyconuk-2025/images/hynek-python-is-fine.webp" />
</figure>

<p>The second day included the young coders day in the workshop space. Here's a photo of me running the
workshop:</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2025/10/pyconuk-2025/images/pgzero-workshop.webp" />
<figcaption><a href="https://www.flickr.com/photos/203482025@N06/54799967120/in/album-72177720329149392">Photo</a> by Mark Hawkins</figcaption>
</figure>

<p>The pygame-zero workshop went really well. It was an ideal activity as there wasn't too much typing,
people could go at their own pace as it didn't rely on following what was on screen, and it was fun
to make a silly splat-a-cat game.</p>
<p>After the day's talks are over, there's a slot before the lightning talks where the kids get to
present on the main stage, such as projects they've worked on, and anything they want to share.
It's amazing how confident they can be on stage in front of so many people - showing the game
they've made with Python or telling everyone what they enjoyed or found difficult. They always
manage to make the audience smile, laugh and applaud like mad.</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2025/10/pyconuk-2025/images/young-coders-day.webp" />
<figcaption><a href="https://www.flickr.com/photos/203482025@N06/54800924813/in/album-72177720329149392">Photo</a> by Mark Hawkins</figcaption>
</figure>

<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2025/10/pyconuk-2025/images/big-hands.webp" />
<figcaption>In Big Hands with Eloisa and Ben</figcaption>
</figure>

<p>Day three, Sunday, was the day of my workshop. I've run countless workshops before, but mostly for
kids, teachers or complete beginners. This one was more targeted at peers. It was intended to be
suitable for less experienced Python users but relied on some basics, and dove deeper than a
standard beginner workshop would.</p>
<p>The workshop room was packed out, with all desk spaces filled and a couple of people joining from
the floor! The wifi provided by the venue was poor so I'd asked people intending to attend to clone
the repo ahead of time, and turn up early if they needed help getting set up. Some people didn't see
the message but we managed to get everyone sorted and started just about on time. There were a few
people with Windows laptops so I had to look up the virtualenv activation instructions for them. One
guy even turned up with an Android tablet. He seemed to know what he was doing, and had vscode and
Python running on there somehow, although it was a bit awkward tabbing between the editor and
instructions.</p>
<p>Rather than use slides, I'd written up the steps and code examples in a series of markdown documents
on a GitHub repo, so people could follow along, copy-pasting code with me as I talked through it.
This also meant that they could take it home to continue if we didn't get through it all. David
Seddon had to head home early so he missed the workshop, but told me he completed it on the train
home!</p>
<p>The workshop went well, and people seemed to get a lot out of it. I probably tried to squeeze in too
much, but that's not so bad as they were able to take it home and carry on. I covered Pydantic,
built a little FastAPI app and touched on Typer for making CLIs. The workshop materials are
available on GitHub if you want to have a go yourself:
<a href="https://github.com/bennuttall/pydantic-workshop">https://github.com/bennuttall/pydantic-workshop</a></p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2025/10/pyconuk-2025/images/pydantic-workshop.webp" />
<figcaption><a href="https://www.flickr.com/photos/203482025@N06/54802069465/in/album-72177720329168007">Photo</a> by Mark Hawkins</figcaption>
</figure>

<p>This was the first PyCon UK since we lost our dear friend <a href="https://ntoll.org/article/my-friend-michael/">Michael
Foord</a>. We raised a glass to him in the pub on the
last day of the conference and shared fond memories of him.</p>
<p>The keynotes and talks are all recorded and promptly uploaded to YouTube and can be found here:
<a href="https://www.youtube.com/@PyconUKSoc/videos">https://www.youtube.com/@PyconUKSoc/videos</a></p>
<p>The official photos taken by the photographer Mark Hawkins are available on Flickr:
<a href="https://www.flickr.com/photos/203482025@N06/albums">https://www.flickr.com/photos/203482025@N06/albums</a></p>
<p>See you there next year, I hope!</p>]]></content>
  </entry>
  <entry>
    <title><![CDATA[Another website revamp]]></title>
    <link href="https://bennuttall.com/blog/2025/08/another-website-revamp/" />
    <id>https://bennuttall.com/blog/2025/08/another-website-revamp</id>
    <published>2025-08-23T23:54:17+00:00</published>
    <updated>2026-05-07T11:33:38+00:00</updated>
    <category term="python" />
    <category term="blogging" />
    <category term="web-development" />
    <category term="meta" />
    <category term="bbc" />
    <category term="bbc-news" />
    <category term="media-city" />
    <category term="manchester" />
    <category term="github" />
    <category term="open-source" />
    <category term="pydantic" />
    <category term="beemo" />
    <summary><![CDATA[I have a habit of revamping my website every few years. Since starting on Blogger some ... 19 years ago I've moved between platforms and redesigned the layout several times over. This time I decided to go for a static site build, which means not requiring managing WordPress plugins and updates, or...]]></summary>
    <content type="html"><![CDATA[<p>I have a habit of revamping my website every few years. Since starting on Blogger some ... 19 years
ago I've moved between platforms and redesigned the layout several times over.</p>
<p>This time I decided to go for a static site build, which means not requiring managing WordPress
plugins and updates, or running PHP on my server. It'll also perform a lot better if I get another
big hit, like <a href="/blog/2021/04/the-surreal-experience-of-my-first-developer-job/">The surreal experience of my first developer
job</a> which sat at the top of
<a href="https://news.ycombinator.com/item?id=28058816">Hacker News</a> for a whole day.</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2025/08/another-website-revamp/images/website.webp" />
<figcaption>bennuttall.com 2025</figcaption>
</figure>

<p>My regular site revamping has been documented over the years...</p>
<blockquote>
<p>I decided I needed a new website template so I made one</p>
</blockquote>
<p>— <a href="/blog/2009/04/version-3-and-problems-with-ie/">Version 3 and Problems with IE</a> (2009)</p>
<blockquote>
<p>I have to make a series of decisions - where do I take the blog, and how do I get it there? Do I
migrate to another blogging service - and if so, temporarily or permanently? Do I write my own
temporary fix? Do I write my own blogging platform - and if so, would it work in the same way
Blogger did?</p>
</blockquote>
<p>— <a href="/blog/2010/02/goodbye-blogger/">Goodbye Blogger?</a> (2010)</p>
<blockquote>
<p>I wonder how long before I get bored of this template and create or find another!</p>
</blockquote>
<p>— <a href="/blog/2020/05/blogging-about-blogging/">Blogging about blogging</a> (2020)</p>
<p>The <a href="/blog/2010/02/goodbye-blogger/">2010 post</a> also included a prediction which seems to have come
true:</p>
<blockquote>
<p>Ideally, in time, I will write my own Content Management System which will manage my entire site,
not just the blog. It will work exactly how I want it to, and it will be perfectly suited my own
personal requirements - bespoke for my own purposes. PHP may be the language I write this in - it
may not. By the time I get round to doing it properly I might be writing in a completely different
language.</p>
</blockquote>
<p>About that...</p>
<h2>The project</h2>
<p>For now I've kept the layout the same, but I built a markdown based static site generator. I
absolutely could have used one of the many frameworks that do exactly this, but I just
<em>really fancied</em> having a go at writing my own in Python. It has a <a href="/blog/">blog index</a>, an
<a href="/blog/archive/">archive</a>, archives by <a href="/blog/2021/">year</a> and <a href="/blog/2021/04/">month</a>,
<a href="/blog/tags/">tags</a>, an <a href="/sitemap.xml">XML sitemap</a> and an <a href="/atom.xml">Atom feed</a>. It also supports
Twitter cards and general <a href="https://ogp.me/">OpenGraph</a> tags for other social media sites such as
<a href="https://mastodon.social/">Mastodon</a>:</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2025/08/another-website-revamp/images/og.webp" />
</figure>

<p>I didn't spend very long building the new site, and I enjoyed working on it. It's quite satisfying
building something of your own, and it's much easier to get done quickly with ChatGPT and a wealth
of open source tools available.</p>
<p>I used <a href="https://chameleon.readthedocs.io/en/latest/">Chameleon</a>,
<a href="https://docs.pydantic.dev/latest/">Pydantic</a>, <a href="https://pypi.org/project/Markdown/">markdown</a>,
<a href="https://pypi.org/project/PyYAML/">pyyaml</a> and
<a href="https://pypi.org/project/beautifulsoup4/">BeautifulSoup</a>.</p>
<p>The source code for the site generator is available on my GitHub. I called it
<a href="https://github.com/bennuttall/beemo">Beemo</a> and I've released it to
<a href="https://pypi.org/project/beemo/">PyPI</a>. I'll try to make it more generalised to be useful to
others.</p>
<figure class="wp-block-image">
<a href="https://github.com/bennuttall/beemo"><img src="https://bennuttall.com/blog/2025/08/another-website-revamp/images/beemo.webp" /></a>
<figcaption>Beemo</figcaption>
</figure>

<h2>Content</h2>
<p>I wrote a <a href="https://gist.github.com/bennuttall/5505efe6f52b875e4c4a547374af5bfb">script</a> to pull all
the posts from my WordPress site via its JSON API, keeping their titles, slugs and tags along with
the content. I used <a href="https://pandoc.org/">pandoc</a> to mangle the HTML content;
<a href="https://pypi.org/project/beautifulsoup4/">BeautifulSoup</a> to extract the images; and
<a href="https://imagemagick.org/">imagemagick</a> to resize them. I ran the same script on my abandoned
<em>Tooling Tuesday</em> blog, a separate WordPress instance, to bring those posts in too.</p>
<p>I also copied the source of some of the article I wrote for
<a href="https://opensource.com/">opensource.com</a> which stopped running a couple of years ago. I had another
look through my old Blogger account, and found a few really old posts I wanted to preserve, like
<a href="/blog/2008/09/athletics/">Athletics?</a> and <a href="/blog/2007/01/my-received-files/">My Received Files</a> so
I've ported those over too.</p>
<h2>BBC</h2>
<p>One thing that struck me while looking back at the content is the number of old posts tagged with
<a href="/blog/tags/bbc/">bbc</a> or <a href="/blog/tags/bbc-news/">bbc news</a>, either written about or referencing
BBC News articles, or about BBC TV shows.</p>
<p>One was about an <a href="/blog/2012/04/hack-to-the-future/">event</a> I was involved with which <a href="https://www.bbc.co.uk/blogs/researchanddevelopment/2012/04/teaching-coding-to-kids-at-hac.shtml">BBC R&amp;D wrote
up</a>
on their blog (and looking now I recognise some of the names on that site); one was about a time in
2007 when my parkour group was <a href="/blog/2007/08/trace-gathering-2007/">filmed for a regional TV show</a>
which was <a href="/blog/2008/us-on-bbc-inside-out/">broadcast in 2008</a>; another in 2009 where we were
<a href="/blog/2009/04/lots-of-training-and-a-session-with-the-bbc/">auditioned for a TV programme</a> about
intelligence stereotypes. One time I had an <a href="/blog/2012/10/hero-kayakers/">eventful kayaking trip</a>
which was <a href="https://www.bbc.co.uk/news/uk-england-manchester-19735175">covered by BBC News</a>. I even
posted about <a href="/blog/2016/03/fork-handles/">the death of Ronnie Corbett</a> in 2016, and related this to
<a href="https://github.com/bennuttall/handles">GitHub</a>.</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2025/08/another-website-revamp/images/media-city.webp" />
<figcaption>BBC at Media City</figcaption>
</figure>

<p>I was a student in Manchester when Media City was being built, and the BBC were beginning to set up
a new base there. I loved the idea of getting a job there once I finished university. I didn't see
anything relevant at the time and for some reason I hadn't been looking at graduate schemes. It
wasn't until I happened to spot a vacancy in <a href="https://www.bbc.co.uk/rdnewslabs">BBC News Labs</a> in
2019 that I finally put in an application, and started in January 2020, initially in London. Later I
relocated back to Manchester and ended up with Media City as my base. So I got there eventually!</p>]]></content>
  </entry>
  <entry>
    <title><![CDATA[hostedpi]]></title>
    <link href="https://bennuttall.com/blog/2025/07/hostedpi/" />
    <id>https://bennuttall.com/blog/2025/07/hostedpi</id>
    <published>2025-07-14T16:50:51+00:00</published>
    <updated>2025-07-14T16:58:15+00:00</updated>
    <category term="hostedpi" />
    <category term="mythic-beasts" />
    <category term="python" />
    <category term="raspberry-pi" />
    <category term="pydantic" />
    <summary><![CDATA[Cambridge-based hosting company Mythic Beasts — long-time friends of Raspberry Pi people and products — launched a unique service back in 2016: Raspberry Pi hosting, what they call their "Pi Cloud".    When I developed the first version of piwheels, it made sense to make use of this service to...]]></summary>
    <content type="html"><![CDATA[<p>Cambridge-based hosting company <a href="https://www.mythic-beasts.com/">Mythic Beasts</a> — long-time friends
of Raspberry Pi people and products — launched a unique service back in 2016: Raspberry Pi hosting,
what they call their "<a href="https://www.mythic-beasts.com/order/rpi">Pi Cloud</a>".</p>
<figure class="wp-block-image">
<a href="https://www.mythic-beasts.com/order/rpi"><img src="https://bennuttall.com/blog/2025/07/hostedpi/images/picloud.webp" /></a>
</figure>

<p>When I developed the first version of <a href="https://www.piwheels.org/">piwheels</a>, it made sense to make
use of this service to natively compile Python packages on Raspberry Pi hardware. When I'd proved
that the idea worked in concept, I knew I needed several Pis to complete the backlog for completion.
Mythic were more than happy to provide this, and have effectively sponsored the running of the
project ever since. Piwheels has proven how capable their Pi Cloud is – delivering <a href="https://blog.piwheels.org/half-a-billion-downloads/">hundreds of
millions of downloads</a> from a single Pi. It's
been a great selling point for them.</p>
<p>They later introduced a <a href="https://www.mythic-beasts.com/support/api/raspberry-pi">web API</a> for
provisioning and managing Pis in their cloud. I initially used a simple script to help provision Pis
as I needed them. I later developed a Python library and CLI providing the full range of features,
and gave a <a href="https://www.youtube.com/watch?v=9wNYRF5DGdk">talk</a> on it at a Raspberry Pi meetup.</p>
<p>I just finished off a revamp of the library, including using
<a href="https://docs.pydantic.dev/latest/">Pydantic</a>, and a new CLI built with
<a href="https://typer.tiangolo.com/">Typer</a>.</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2025/07/hostedpi/images/hostedpi.webp" />
<figcaption>The new piwheels CLI in all its glory</figcaption>
</figure>

<ul>
<li>The package can be found on <a href="https://pypi.org/project/hostedpi/">PyPI</a> (and
  <a href="https://piwheels.org/project/hostedpi/">piwheels</a>, of course)</li>
<li>Documentation is on <a href="https://hostedpi.readthedocs.io/en/stable/">readthedocs</a></li>
<li>The source is on <a href="https://github.com/piwheels/hostedpi">GitHub</a></li>
</ul>
<p>It's been a pleasure to use Pydantic to manage data models for the API requests and responses, and
to provide an easy way for users to create validated Pi specs. Typer made it easy to create a
practical CLI with all the bells and whistles, and no boilerplate. I'll be giving a workshop at
<a href="https://2025.pyconuk.org/">PyCon UK 2025</a> on data modelling with Pydantic, covering
<a href="https://fastapi.tiangolo.com/">FastAPI</a> and Typer too.</p>]]></content>
  </entry>
  <entry>
    <title><![CDATA[PyCon UK 2023 talks]]></title>
    <link href="https://bennuttall.com/blog/2023/10/pycon-uk-2023-talks/" />
    <id>https://bennuttall.com/blog/2023/10/pycon-uk-2023-talks</id>
    <published>2023-10-25T11:10:29+00:00</published>
    <updated>2026-04-30T00:51:30+00:00</updated>
    <category term="bbc" />
    <category term="bbc-news" />
    <category term="bbc-news-labs" />
    <category term="bbcrd" />
    <category term="python" />
    <category term="pycon" />
    <category term="talks" />
    <summary><![CDATA[I was back in Cardiff for PyCon UK recently and gave a talk on a BBC project I've been working on, called Live Highlights. I'm not doing as many conference talks as I used to – this is my only one of the year!    I spoke about how we prototyped an idea and built it up to be used in audience-facing...]]></summary>
    <content type="html"><![CDATA[<p>I was back in Cardiff for PyCon UK recently and gave a talk on a BBC project I've been working on,
called <em>Live Highlights</em>. I'm not doing as many conference <a href="/talks/">talks</a> as
I used to – this is my only one of the year!</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2023/10/pycon-uk-2023-talks/images/ben-pycon-uk.webp" />
</figure>

<p>I spoke about how we prototyped an idea and built it up to be used in audience-facing trials:</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2023/10/pycon-uk-2023-talks/images/bbc-live-highlights.webp" />
</figure>

<figure>
<iframe width="560" height="315" src="https://www.youtube.com/embed/UEGMR0oCOqI?si=3zpwO9NhSK5F6jSm" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
</figure>

<p>I also gave a lightning talk about numeronyms – those silly shorthand words with numbers like <code>k8s</code>
and <code>a11y</code> – and how I find them obnoxious (starts at <a href="https://www.youtube.com/watch?v=chxsC2WYNTE&amp;t=3677s">this
timecode</a>):</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2023/10/pycon-uk-2023-talks/images/ben-pycon-uk-numeronyms.webp" />
</figure>

<figure>
<iframe width="560" height="315" src="https://www.youtube.com/embed/chxsC2WYNTE?si=FbSOy8rVbYMYsW1U&amp;start=3677" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
</figure>

<p>Photos by <a href="https://www.flickr.com/photos/184390836@N04/albums">Mark Hawkins</a></p>]]></content>
  </entry>
  <entry>
    <title><![CDATA[BBC News Labs 2021]]></title>
    <link href="https://bennuttall.com/blog/2022/01/bbc-news-labs-2021/" />
    <id>https://bennuttall.com/blog/2022/01/bbc-news-labs-2021</id>
    <published>2022-01-24T21:54:21+00:00</published>
    <updated>2022-01-24T22:27:45+00:00</updated>
    <category term="bbc" />
    <category term="bbc-news" />
    <category term="bbc-news-labs" />
    <summary><![CDATA[I've now completed my second full year working as software engineer in the News Labs team at the BBC. Another year working from home; the team officially moved into BBC R&D; and I was promoted to senior engineer. In 2021 I worked on:  mosromgr, a Python library for managing TV and   radio running...]]></summary>
    <content type="html"><![CDATA[<p>I've now completed my second full year working as software engineer in the <a href="https://bbcnewslabs.co.uk/">News
Labs</a> team at the BBC. Another year working from home; the team
officially moved into <a href="https://www.bbc.co.uk/rd">BBC R&amp;D</a>; and I was promoted to senior engineer.</p>
<p>In 2021 I worked on:</p>
<ul>
<li><a href="https://bbcnewslabs.co.uk/projects/mosromgr/"><strong>mosromgr</strong></a>, a Python library for managing TV and
  radio running orders, which this year we <a href="https://github.com/bbc/mosromgr">open sourced</a> and we're
  currently working on rolling out to teams around the BBC to make use of</li>
<li><strong><a href="https://bbcnewslabs.co.uk/news/2021/slicer-today-trial/">Semi-automated chapter points for the Radio 4 Today
  programme</a>,</strong> which went
  <a href="https://twitter.com/BBCr4today/status/1405910025961459715">live</a> as a month-long trial</li>
<li><a href="https://bbcnewslabs.co.uk/projects/modus/"><strong>MODUS</strong></a>, an experiment in using machine-summarised
  articles for new formats such as the bullet-point list in the public beta of the BBC News app,
  currently being trialled for red button services</li>
<li><strong>BBC Images</strong>, a new image search tool for journalists based on the Guardian's open source
  <a href="https://github.com/guardian/grid/">GRID</a> platform</li>
<li><a href="https://bbcnewslabs.co.uk/projects/oriel/"><strong>Oriel</strong></a>, an image collections curation tool based
  on BBC Images with a metadata enrichment pipeline</li>
<li><strong>IDX</strong>, a prototype tool for marking important segments of live TV and radio programmes for
  re-use to make them accessible to personalisation services and more</li>
</ul>
<p>Here's our 2021 highlights video:</p>
<figure>
<iframe width="560" height="315" src="https://www.youtube.com/embed/j8Ig20Nkfqw?si=eqU4xbJH4BJUYiiD" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
</figure>

<p>I also gave a talk about mosromgr and our work on R4 Today at EuroPython:</p>
<figure>
<iframe width="560" height="315" src="https://www.youtube.com/embed/JfrqKOO0Yoc?si=XjjDwQImh_-sArUS" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
</figure>

<p>Check out the <a href="https://bbcnewslabs.co.uk/">News Labs website</a> and you can follow us on Twitter:
<a href="https://twitter.com/BBC_News_Labs">@BBC_News_Labs</a>.</p>]]></content>
  </entry>
  <entry>
    <title><![CDATA[Client Horror Stories]]></title>
    <link href="https://bennuttall.com/blog/2021/09/client-horror-stories/" />
    <id>https://bennuttall.com/blog/2021/09/client-horror-stories</id>
    <published>2021-09-28T13:20:39+00:00</published>
    <updated>2021-09-28T13:20:39+00:00</updated>
    <category term="web-development" />
    <summary><![CDATA[Following the success of my article The surreal experience of my first developer job, which made it to the top of Hacker News for a whole day, I was invited to tell the story on the excellent podcast Client Horror Stories. The video is available on clienthorrorstories.com:    Thanks to Morgan for...]]></summary>
    <content type="html"><![CDATA[<p>Following the success of my article <a href="/blog/2021/04/the-surreal-experience-of-my-first-developer-job/"><em>The surreal experience of my first developer
job</em></a>, which made it to
the <a href="https://news.ycombinator.com/item?id=28058816">top</a> of <a href="https://news.ycombinator.com/">Hacker
News</a> for a whole day, I was invited to tell the story on the
excellent podcast <em><a href="https://clienthorrorstories.com/">Client Horror Stories</a></em>. The video is
available on clienthorrorstories.com:</p>
<figure class="wp-block-image">
<a href="https://clienthorrorstories.com/e/that-time-you-had-to-develop-two-apps-a-day-every-day-for-a-19-years-old-boss-with-ben-nuttall/"><img src="https://bennuttall.com/blog/2021/09/client-horror-stories/images/client-horror-stories.webp" /></a>
</figure>

<p>Thanks to <a href="https://westegg.com/">Morgan</a> for having me on!</p>]]></content>
  </entry>
  <entry>
    <title><![CDATA[BBC open sourcing mosromgr]]></title>
    <link href="https://bennuttall.com/blog/2021/04/bbc-open-sourcing-mosromgr/" />
    <id>https://bennuttall.com/blog/2021/04/bbc-open-sourcing-mosromgr</id>
    <published>2021-04-14T14:48:00+00:00</published>
    <updated>2025-07-11T15:20:23+00:00</updated>
    <category term="bbc" />
    <category term="bbc-news-labs" />
    <category term="open-source" />
    <category term="python" />
    <summary><![CDATA[For the last few months in BBC News Labs, I've been working on various projects involving extracting metadata from running orders for BBC programmes such as Newsnight and the R4 Today programme. Like many traditional broadcasters, the BBC uses the MOS Protocol for communications between newsroom...]]></summary>
    <content type="html"><![CDATA[<p>For the last few months in BBC News Labs, I've been working on various projects involving extracting
metadata from running orders for BBC programmes such as
<a href="https://www.bbc.co.uk/programmes/b006mk25">Newsnight</a> and the <a href="https://www.bbc.co.uk/programmes/b006qj9z">R4
Today</a> programme.</p>
<p>Like many traditional broadcasters, the BBC uses the <a href="http://mosprotocol.com/">MOS Protocol</a> for
communications between newsroom computers and servers where media files such as audio and video are
stored.</p>
<figure class="wp-block-image">
<img src="https://bennuttall.com/blog/2021/04/bbc-open-sourcing-mosromgr/images/mos.webp" />
</figure>

<p>Production teams create running orders using
<a href="https://www.cgi.com/en/solutions/openmedia">OpenMedia</a>, and every time they add details or make
changes, MOS XML messages are emitted by the newsroom computer systems. We've created a way to
process these messages to build up a machine-readable version of the running order.</p>
<p>We developed a general solution for dealing with MOS messages – a Python library called mosromgr.
It's based on open standards and could be useful to other broadcasting organisations, so we've
released it under an open source licence.</p>
<ul>
<li>The source code is available on <a href="https://github.com/bbc/mosromgr">GitHub</a></li>
<li>The documentation for the library is available on <a href="https://mosromgr.readthedocs.io/">readthedocs</a></li>
<li>The project is available from the <a href="https://pypi.org/project/mosromgr/">Python Package Index</a></li>
</ul>
<p>Read more at <a href="https://www.bbc.co.uk/opensource/projects/project/mosromgr">bbc.co.uk/opensource</a> and
<a href="https://www.bbc.co.uk/rdnewslabs/projects/mosromgr">bbc.co.uk/rdnewslabs</a></p>]]></content>
  </entry>
</feed>