Mind the gap

Most modern software solutions consist of multiple layers or tiers. Each of these has responsibility for processing inputs and outputs in different ways. For web applications you’ll find a user interface, one oe more APIs which serve data, and probably multiple tiers handling data on the server.

These tiers can be completely separate, as in the case of a web UI and its API. Sometimes they are very closely tied together, for example data access and repository pattern layers in a single component. In all cases these tiers have to – and I realise how much I’m stating the obvious – communicate with each other. So they have to know how to communicate with each other.

Nothing ground-breaking there. But I’ve found that this is exactly where software projects can fail. There’s lots of thought and information gone into how each of the components work, but not so much thought gone into how they will communicate. Here’s an analogy:

Towbar fitted to the back of a car

The humble towbar. Doesn’t look much, does it? A curved bit of metal, with just enough of a shape to allow something to be fixed loosely to the end of it.

Yet this simple bit of technology is responsible for joining two huge components – a car and a caravan or trailer. In many cases the two components it joins are hugely expensive and complicated pieces of machinery – but this simple hook of metal means they can work together.

It’s not glamorous or highly technical, but the specification of this hook had to be known to both of the components it was joining. Without a known and agreed specification there was little hope of successful communication between the components.

Let’s translate that to software. Imagine you’re on a team who need to deliver a web app. The web app must call an API to get some data crucial to the app. The API is being built by a different team, over who you have no control. An architect may put together a diagram explaining when the components should communicate:

Example sequence diagram showing a client app, API, and business rules serverBut without the how this is little use to the development team. The how is the actual specification of those request and response messages – what gets sent, what gets returned.

This detail is crucial and must be discussed and agreed early in the project. This detail is the system. Without known, agreed specifications for all of the communication points between the different components you’re in grave danger of building a bunch of cogs which don’t quite work together.

The specification of those messages allows a number of important things to be discussed and checked-off:

  • What data do I need to send?
    • What is optional? What is required?
    • What are the bounds of the data? Are there any value contraints?
  • What data do I get back?
    • Is everything there that I need?
    • What are the bounds of the data? Are there any value contraints?
    • Are there values which I need to translate in any way?
  • What about errors?
    • What possible error response could I get?
    • What if no response is returned?
    • Is there a timeout I need to cater for?
  • Is this even right?
    • Does this request even need to be made? Am I requesting data I already have?
    • Is there a more efficient or robus mechanism to do the same thing?

These things make the difference between a system which is deployed riddled with potential runtime bugs, and one which you have prepared for as many scenarios as possible.

OK, how do we agree on and document this specification so everyone is on the same page? Or at least, in the same book.

(Thanks to Craig Milner for that last line. He took it further: “I’ve known teams who were not just not on the same page, they weren’t even in the same library.”)

I think there’s a lack of what I’m going to call “3D software architecture documentation”, or at least I’m not aware of any. What I mean is documents which, like the sequence diagram above which shows two axes, also allow the viewer to go deeper into more detail. Imagine if you could click any of those request/response arrows and view the specification for that message. Then click “back” and go back to the more zoomed-out level.

I guess what I’m describing is a web page. Yes, I’ve just invented links. Go me!

And the specification for messages? That’s easy: for REST APIs (which a lot of the time is what we’re talking about) you should use OpenAPI – a standard for describing APIs.

This is what I used when defining the API for a large automotive data company. I wrote the specification for the API using OpenAPI, it could then be “navigated” using an OpenAPI viewer, and discussed by the team before we built anything. Once a part of the API was built we could then compare the output with the original spec.

Sometimes, pragmatic changes had to be made so the real API was slightly different to the specification – these changes were always discussed during development. But more often than not, because adequate thought had gone into a high-enough resolution specification, the developers knew exactly what to build.

This approach is “design-first API development” as you design what the API is going to look like up-front before you break ground writing any code. This same approach can be used for different types of components – GraphQL APIs, SOAP, even code-level interfaces.

So the takeaway here is to spend some time early in a project to talk about and document how components will communicate. That’s the detail which can make or break a solution.

Clara’s Den, Filey – holiday apartment

I’ve been building websites for a long time. Websites of all shapes and sizes, and sometimes for bricks and mortar businesses. My latest website is for a physical place, but this time – it’s MY place! Clara’s Den, a holiday apartment on an award-winning holiday village just south of Filey on the beautiful Yorkshire coast.

If you’ve not stayed on the Yorkshire coast, sampling the delights of Whitby and Scarborough, Filey and Bridlington, Staithes and Sandsend, then you’re missing out. There are loads of details on the website about what to find in The Bay village, along Filey Bay from the Brigg to Bempton Cliffs, and further afield along the east coast.

If you’re looking for a self-catering holiday apartment in walking distance of the beach then we’d love to welcome you to Clara’s Den.

Book review: Digital Adaptation by Paul Boag

Almighty God, our heavenly Father,
we have sinned against you and against our fellow men,
in thought and word and deed,
through negligence, through weakness,
through our own deliberate fault.

It’s no secret to web design and development professionals that many organisations just don’t “get” the web. This shouldn’t be a surprise: I expect that chefs would say that many organisations don’t “get” cuisine, or newspaper editors would say that many organisations don’t “get” the media industry. That’s not necessarily a problem – we can’t expect everyone to know HTML and CSS.

Chefs are expected to know the nuances of many ingredients, but a restaurant manager only needs to know whether customers like the food and are willing to pay for it. Newspaper editors must understand the process of finding, investigating and writing stories, printing and distributing their publications, but a paper shop only needs to know what newspapers they need to stock. In the same way most organisations don’t need to know the technical aspects of building websites, but they do need to know how they can adapt themselves to get the best from the digital world we now inhabit.

At the risk of being accused of hyperbole I’m going to stress that “adapt” is the right word here. It’s about a constant evolution to enable growth and long-term survival.

For many years Paul Boag has been one of the leading voices in the web industry (my cheque is in the post, right, Paul?) through his website and podcast, not to mention the great work he and his colleagues at Headscape have done for a wide range of clients. So it’s great news that those years of insight have been poured into his latest book: Digital Adaptation.

It’s a great read; full of nuggets of wisdom gleaned from countless meetings with organisations of all shapes and sizes. When it comes to clients who want websites, Paul has seen it all. Which brings me onto the quote above. Yes, there is a point to it.

You see, people – and that who we have to deal with; real, human people – are a complicated bunch. We have a huge range of attitudes and thoughts, all shaped by the vast array of experiences we each travel through. We’re not always perfect, and we certainly don’t always make the right decisions. We fail in a number of ways: in thought, in word, in deed.

Sometimes we think something is true when it’s not – for example thinking carousels are a good idea. Sometimes we think something is false when the reality is different.

More often that we care to admit we say things that aren’t right, either. We speak without thinking; we tear down rather than build up.

And, if we’re honest, our actions rarely live up to what our best selves would do. We react badly, act thoughtlessly, ignore opportunities to do good.

And the reasons why are as varied as our failures. We fail through negligence, through weakness, and sometimes through our own deliberate fault.

But I’m not here to be a tree-hugging motivational speaker, I’m reviewing Paul’s book! Which is about … people. People who misunderstand the web, who don’t know what is possible, who sometimes (it has to be said) want their own way or the high way.

Organisations are just groups of people. And whatever the reasons why organisations don’t “get” the web, and irrespective of how those reasons take form, those people can be challenged and educated to start to “get” it. That’s what Paul’s book is about at it’s heart; kick-starting the changes in attitudes that are required for organisations to adapt to the digital landscape.

I’ve already said it’s a great book, but I do have a central problem with it. Paul repeatedly uses the word “digital” as a noun, but most people are used to it being used as an adjective. This might be a barrier to some people. Here’s an example:

Similar trends are occurring in the newspaper industry and among cable TV companies, as digital changes how consumers read the news and watch TV.

Wrapped up in that word “digital” is a whole set of other more definite nouns; the Internet and web, digital downloading, email and SMS, smartphones, tablets and much more. But while we in the digital industry understand that “digital” as a noun encompasses these things, and much more yet to be invented, my hunch is that those outside the industry (shall we call them “Muggles“?) may not get this.

Paul also uses digital in its more traditional form as an adjective; ‘digital team’, ‘digital strategy’ etc. This, I feel, will be more understandable to those we’re trying to educate. But on the whole this is a small nit-pick.

At 176 pages the book is brief enough to read in a couple of sittings, and it feels great. Printed on thick paper, with Veerle Pieters gorgeous illustrations. Even the main title on the front cover has a subtle texture to it. It’s a quality product.

Paul starts with an honest appraisal of the problem we face as digital professionals; that organisations can be ignorant, scared or indifferent to the changing world. His insight into the thought processes of the management in these organisations is enlightening and thought-provoking.

One thing I must particularly highlight is Paul stressing that the management attitudes digital professionals often struggle with may not be all deliberate (remember that “through ignorance, through weakness, through our own deliberate fault” thing?). This is timely and often needed; understanding and good communication must work both ways.

The bulk of the book consists of practical and well-reasoned arguments why organisations should not just embrace the digital revolution but fully engage with it – to the extent that it informs their entire strategy. This is “digital by default”, a motif which runs throughout the book and the premise of which I fully agree with.

This, however, is my second bug-bear with Digital Adaptation. I know from experience that design teams are sometimes perceived by outsiders as unapproachable, uninterested in the wider aims of the organisation, and unwilling to be pragmatic about design. Of course that is rarely the truth. But we have to face facts: the offices of professionals in most other industries don’t contain beer fridges, games consoles or loud music. Perhaps those things are necessary for a creative environment. I’m no designer, so I can’t really comment.

Digital Adaptation nods towards these attitudes and tries to cut through the frippery to the real heart of the matter; that designers are serious professionals who are crucial to the ongoing success of an organisation in the changing culture we inhabit. The problem for me is that I don’t think enough is said to distance the design industry from the perceptions I’ve encountered in more traditional industries.

There are excellent and strong examples of a modern digital strategy doing wonders for organisations, notably in the discussion of the gov.uk work done by Martha Lane Fox, Mike Bracken and the gov.uk team. And there are plenty of stories from a range of modern businesses – Twitter, Google, Mailchimp and others – to further strengthen Paul’s central message that digital (yes, as a noun) matters.

All in all this is a great book which I believe will further and enhance the conversations happening in boardrooms, design studios and development offices in organisations all over the world.

Welcome to stillbreathing.co.uk 2009

Well, here it is. This new theme for stillbreathing.co.uk has been mainly designed by the great Paul Cook who worked with me for several years in a previous job. I’ve said on a number of occasions he’s the best designer I know, and that’s true. After all, I don’t know any other designers :0)

Anyway, he’s pulled a corker out of the bag with this one. Basically all the bits that look really cool (the very top and very bottom of the page) were done by him, the rest (the rubbish bit in the middle) was done by me. Hopefully I’ll be able to pick Paul’s brains at some point and put some finishing touches to the design.

However, this isn’t just a new design. It’s also the beginnings of a new direction for stillbreathing.co.uk which has been my personal site for *checks archives* ooh, October 2004. Basically I’m keeping this site for more personal things (which of course will include code), but I’ll be creating a Brand New Site for my freelance business.

I’ve mentioned the name of that new site a few times, but as it’s not even nearly ready for previewing I won’t link to it just yet. Still, along with the new PerformerJS.org site I hope to have that finished in the next few weeks.

New BeatsBase mix widget

I’m currently working on a widget for BeatsBase.com, the social networking and mix hosting site I developed for my friend and client Robbie a couple of years ago. The widget will allow you to have a Radio BeatsBase player on your website, just like this:

That one shows the latest mixes added to the site, but if things go according to plan you’ll also be able to have a player with just one users mixes in as well. If you’re into dance, trance, techno and other styles of “young person” music take a look.