Mind the gap

Most modern software solutions consist of multiple layers or tiers. Each of these has responsibility for processing inputs and outputs in different ways. For web applications you’ll find a user interface, one oe more APIs which serve data, and probably multiple tiers handling data on the server.

These tiers can be completely separate, as in the case of a web UI and its API. Sometimes they are very closely tied together, for example data access and repository pattern layers in a single component. In all cases these tiers have to – and I realise how much I’m stating the obvious – communicate with each other. So they have to know how to communicate with each other.

Nothing ground-breaking there. But I’ve found that this is exactly where software projects can fail. There’s lots of thought and information gone into how each of the components work, but not so much thought gone into how they will communicate. Here’s an analogy:

Towbar fitted to the back of a car

The humble towbar. Doesn’t look much, does it? A curved bit of metal, with just enough of a shape to allow something to be fixed loosely to the end of it.

Yet this simple bit of technology is responsible for joining two huge components – a car and a caravan or trailer. In many cases the two components it joins are hugely expensive and complicated pieces of machinery – but this simple hook of metal means they can work together.

It’s not glamorous or highly technical, but the specification of this hook had to be known to both of the components it was joining. Without a known and agreed specification there was little hope of successful communication between the components.

Let’s translate that to software. Imagine you’re on a team who need to deliver a web app. The web app must call an API to get some data crucial to the app. The API is being built by a different team, over who you have no control. An architect may put together a diagram explaining when the components should communicate:

Example sequence diagram showing a client app, API, and business rules serverBut without the how this is little use to the development team. The how is the actual specification of those request and response messages – what gets sent, what gets returned.

This detail is crucial and must be discussed and agreed early in the project. This detail is the system. Without known, agreed specifications for all of the communication points between the different components you’re in grave danger of building a bunch of cogs which don’t quite work together.

The specification of those messages allows a number of important things to be discussed and checked-off:

  • What data do I need to send?
    • What is optional? What is required?
    • What are the bounds of the data? Are there any value contraints?
  • What data do I get back?
    • Is everything there that I need?
    • What are the bounds of the data? Are there any value contraints?
    • Are there values which I need to translate in any way?
  • What about errors?
    • What possible error response could I get?
    • What if no response is returned?
    • Is there a timeout I need to cater for?
  • Is this even right?
    • Does this request even need to be made? Am I requesting data I already have?
    • Is there a more efficient or robus mechanism to do the same thing?

These things make the difference between a system which is deployed riddled with potential runtime bugs, and one which you have prepared for as many scenarios as possible.

OK, how do we agree on and document this specification so everyone is on the same page? Or at least, in the same book.

(Thanks to Craig Milner for that last line. He took it further: “I’ve known teams who were not just not on the same page, they weren’t even in the same library.”)

I think there’s a lack of what I’m going to call “3D software architecture documentation”, or at least I’m not aware of any. What I mean is documents which, like the sequence diagram above which shows two axes, also allow the viewer to go deeper into more detail. Imagine if you could click any of those request/response arrows and view the specification for that message. Then click “back” and go back to the more zoomed-out level.

I guess what I’m describing is a web page. Yes, I’ve just invented links. Go me!

And the specification for messages? That’s easy: for REST APIs (which a lot of the time is what we’re talking about) you should use OpenAPI – a standard for describing APIs.

This is what I used when defining the API for a large automotive data company. I wrote the specification for the API using OpenAPI, it could then be “navigated” using an OpenAPI viewer, and discussed by the team before we built anything. Once a part of the API was built we could then compare the output with the original spec.

Sometimes, pragmatic changes had to be made so the real API was slightly different to the specification – these changes were always discussed during development. But more often than not, because adequate thought had gone into a high-enough resolution specification, the developers knew exactly what to build.

This approach is “design-first API development” as you design what the API is going to look like up-front before you break ground writing any code. This same approach can be used for different types of components – GraphQL APIs, SOAP, even code-level interfaces.

So the takeaway here is to spend some time early in a project to talk about and document how components will communicate. That’s the detail which can make or break a solution.

This is starting to get silly

It’s been a while since I made any LEGO office equipment. I had to fix that.

I found my desk getting cluttered with a few bits I always had hanging round:

  • Pens and pencils
  • Headphones
  • Short USB cables (for charging my phone etc)
  • Couple of small backup drives
  • USB memory sticks
  • USB adapters
  • Capo (I play a bit of guitar)
  • Plectrums. Should that be plectra?

All necessary, all needed at my fingertips, but messy to leave lying around. So I built a desk tidy.

My LEGO desk tidy

It’s got a place for everything, and everything in its place. Somewhere to put my plectrums/plectra.

Plectrum holder in my LEGO desk tidy

A pot on top for USB memory sticks and adapters, and any other small knick-knacks.

Small box of bits in my LEGO desk tidy

Cable hooks on each side. I decided against a more complicated system to store cables as … well, it’s not needed.

Cable hooks on my LEGO desk tidy

Round the back is a bracket for my capo. There are actually two brackets, not that I have two capo (capi?) but I needed the unit to be symmetrical.

Capo bracket on my LEGO desk tidy

And there are two caddies for storing the backup drives. I’ve also got an eraser in there, too.

Backup drive storage in my LEGO desk tidy

This was a fun little project to build! But I need to stop, as my wife is starting to think I’m strange. Or rather, stranger than she already thought I was.

More LEGO office equipment

Oops, I did it again.

The arrival of another laptop meant I had a requirement for another laptop stand. Yes, of course I made it out of LEGO!

LEGO laptop stand number 2

I decided on a simpler design this time, with no attempts at making it adjustable.

Side view of my LEGO laptop stand

I also made the cable tidy smaller and more integrated.

A laptop stand made of LEGO

And finished it off by adding a few mini figs, including Spiderman.

Some LEGO minifigs guarding my LEGO laptop stand

I have to say I really enjoyed this, and am trying to think of other office equipment I can make out of LEGO. Any ideas?

LEGO office equipment

In May 2020, as every year, I attended the wonderful All Day Hey conference. This year, due to The Pandemic Situation, the conference couldn’t be held in the wonderful Yorkshire city of Leeds, instead it was online. The attendees were encouraged to post pictures of where they were attending the conference from, and I was embarrassed by my lunchbox laptop stand.

My desk with my laptop sitting on a lunchbox, which I used as a makeshift stand

I could have ordered a proper stand online, but thought I could make one instead. Out of Lego. Because why not.

Everyone here loves Lego, right? Of course we all do. It’s perfect for building office equipment. Here’s the finished product, just visible underneath my laptop:

My desk, featuring my laptop stand made out of Lego

I have to be honest, I’m a massive Lego fan. I don’t get much chance to dabble these days, but when I do I remember just how much I enjoy it – Technical Lego especially. Here you can see how I constructed the stand to ensure it provides a stable base:

My laptop stand made out of Lego

And if we look a little closer at the back, you can see the stand has springs to help sit the laptop better. I did plan on making the height adjustable, but this is good enough for now.

Detail of the rear of my laptop stand made out of Lego

Of course I needed minifigs to guard my laptop! I chose Batman and the Terminator, because nobody would mess with them, right?

Mini Batman guarding my laptop stand made out of Lego

And I incorporated a simple cable tidy into the stand:

The cable tidy for my laptop stand made out of Lego

Overall I’m pretty pleased. As a beta product I think it works well, but I would like to investigate how difficult it would be to make the height and tilt angle adjustable. That will have to wait for another day, though.

I’m learning React

Some people who know me well may not quite have heard me correctly, so I’ll repeat:

I’m learning React.

Does this mean I’m turning my back on the principle of Progressive Enhancement? Does it mean I’m going to start building websites which are 99% JavaScript and 1% everything else? Have I given in?

No, dear reader, my opinions – shaped by luminaries like Jeremy Keith, Alex Russell, Tim Kadlec and many others – have not changed. The web is neck-deep in JavaScript and sinking deeper all the time, and organisations drinking the SPA framework kool-aid think that browsers are waving not drowning.

There are two reasons I’m learning this framework – and I’ll also be looking at Vue as well (I already have some commercial experience with Angular).

Firstly, when talking with teams and encouraging them to reduce their reliance on JavaScript for core functionality I’ve repeatedly received a “but you don’t understand it!” response. I do understand it, my 20+ years building websites hasn’t been spent hiding under a rock. JavaScript is cool, I get it,

This “but we’re building a web app, not a static site” is a common fallacy, and in a large part fuelling the current untenable position. I’ve not yet found anyone who can explain the difference between an “app” and a “site”, and most grudgingly accept there’s a big grey area between the extremes of a rarely-updated content site and, say, GMail. Most projects involve a mixture of slow and fast-moving information.

If I learn React then I can counter the lack-of-understanding argument. I can speak in the language of die-hard Reactians (is that the right word) and – hopefully – put across some reasons why core functionality should be delivered using the simplest technology possible (generally server-side generated HTML).

Secondly, I expect to fail to convince many people to use less JavaScript. So I want to have some practical examples of apps that use React (or any JavaScript framework) in a less all-or-nothing way.

After all, I don’t see any reason why this should be delivered to a browser:

<!doctype html>
<html>
<head>
<title>MY 'app'</title>
<script src="my-huge-bundle.js"></script>
</head>
<body>
<div id="app"></div>
</body>
</html>

If I learn React maybe I can implement some new patterns that will incorporate a Progressive Enhancement mindset.

I may fail at both those aims, of course. In which case I will still have learnt a new technology.