The Future of Software Engineering

“The future is here; it’s all around us. It’s not so clear that you actually can see it, but you will in the next few years — you’re going to see how things change and become more advanced.”

So said Mrs Mary Poppendieck in her keynote speech at the Agile Singapore Conference 2016, held at Hotel Fort Canning on 6-7 October 2016.

Together with her husband Tom, she authored the book Lean Mindset, and created the Lean Software Development series.

And by “future”, she meant the emerging digital trends and technologies that are impacting software engineering in a big way, such as cloud computing, Big Data, virtual reality, and the Internet of Things.

Software engineering in the next decade

First, there is the problem of companies attempting to scale up their software capabilities, when they really should be scaling out, she shared.

While the concept of scaling out has been around for a decade now, it is still every bit as relevant today, she said.

In the nineties, it was absolutely required that the main data for any company sit on a single server, since the distance between multiple servers meant that data could not be instantly available and at the same time completely consistent.

But along came Google around the year 2000, and thanks to its sheer size in growth, couldn’t possibly scale up anymore. Google was one of the first to move away from the single server system, Mrs Poppendieck said.

She quoted American computer scientist Grace Hopper, who once said, “In the pioneer days, they used oxen for heavy pulling, and when one ox couldn’t budge a log, they didn’t try to grow a larger ox. They got two oxen instead.”

Thus, when greater computing power is needed, Mrs Poppendieck remarked, the answer isn’t to get a bigger computer; but rather, companies should scale out and build a system of computers that are operated in parallel, much like in Google’s case.

Robustness of systems is another factor to consider.

Fault injection is a testing technique where software tools introduce defects into systems on purpose.

By deliberately taking out a whole server or launching various attacks, fault injection helps companies to develop anti-fragile systems, which actually thrive on being disrupted and hurt, and then ‘regenerate’ and come back all the more stronger for it.

“This is something you need to get good at, because it will give you really robust and rapidly recoverable code,” she noted.

Netflix is probably the most well-known for fault injection practices. The entertainment giant regularly deploys a suite of tools which they call their Simian Army, and its sole purpose is to deliberately launch attacks and disruptions on their systems.

The name was inspired by the idea of literally unleashing a wild monkey into your data center and defending against the chaos it brings.

“It’s this whole concept of creating ‘fault-tolerant’ systems, rather than ‘fault-free’ ones,” she added.

The Power of Disruptive Platforms

Platforms are another trend to watch.

Likening platforms to a marketplace in a square, she described how markets are where buyers and sellers get together, while somebody provides the space.

The city owns the square and sets up rules governing the marketplace. In this scenario, platforms are the people who own the city square.

They don’t bring in or buy the food, but they create the marketplace where commerce takes place.

Platforms are therefore a matchmaker of sorts, Mrs Poppendieck said, and software platform companies are businesses that connect members of one group with members of another.

“Spotify, AirBnB, Youtube; they’re not so much worried about making one customer happy but rather creating an environment in which that one transaction — sell a room, listen to a song, create a ride — is a really good transaction, and that there are a lot of such transactions,” she remarked.

“If your company is in an industry where a platform has just been introduced, it will be in trouble.”

The digitisation of objects is another hot area.

From self-driving cars to tennis rackets, we will be seeing consumer products of all kinds get increasingly digitised, at a much more detailed level, she said.

And even dessert won’t be spared.

“Someone said to me over dinner last night, ‘I’m going to name something and you tell me how to digitise it’, and he said ice cream,” she shared with a laugh.

“Well, ice cream comes in a little container. What if the container had a sensor in it which tracks temperature throughout the supply chain?” she mused.

“What if the distributor never lets that temperature get out of a set range, so by the time the consumer gets that ice cream, it’s been guaranteed to have been handled safely during the entire trip?”

These examples show where the future’s going, and that all of us — individuals and businesses alike — should sit up and take notice.

“You need to be paying attention to them because they’re going to be coming to a company near you; either yours or one of your competitors. And it will be better if you participated, rather than fight against these trends.”