At a recent event Keir Finlow-Bates took us on a journey of blockchain starting with looking at the fundamental principles behind blockchain, decentralised ownership and governance during incentivisation and attribution of value to tokens, to the different agencies involved in blockchain, both good and bad, and the risks that they bring and to examine some failures in the DeFi world, looking at how they happened and how the community has responded. This is Part 2 and focuses on the software industry and quality.
Software industry and Quality
There’s this saying in the software industry, that when you’re producing a software product you have three things to consider. You have quality, you have cost, and you have time.
Time, Cost or Quality?
The problem with the bind in software is that you can pick any two of those three and you can make a product that is cheap to produce and that you make quickly, but then the quality will suffer. Or you can make something as high quality and cheap, but then it’s going to take you a long time. you can make something that is high quality and is produced quickly, but then it’s going to cost you a lot more money. It actually gets a bit more complicated than that. For example, if you pay for a large team of developers, you see a problem in that there are more and more people that have to communicate in order to produce the product.
The quality costs time triangle is a neat simplification of some of the issues when working in software projects and of course a blockchain is fundamentally a software product when you are actually getting down to the nuts and bolts of developing it. One of the problems that we have in software development is that we’ve built up this culture of, as Facebook’s Mark Zuckerberg said, “move fast and break things.”
Back in the 70s, 80s and 90s when you were programming you made these software products and you put them on a cassette, or a floppy disc and you mail them out to customers. Now with the internet and with all the software libraries we have, you have things like web programming and mobile app programming where you can push a product out. Then as you discover bugs, you can fix them and automatically update them.
I’m sure you’re all familiar when your phone says it’s updating this program, or if you ever used something like Microsoft teams. For example, you’ll find that you miss your meeting because it spends five minutes updating to the latest version because there’s some fixing being pushed out there.
What’s more important?
One of the problems this web programming paradigm has brought is that the decision is made again and again, to go for reduced costs and decreased speed because you can catch up with quality later. Except that it’s not quite as simple as that. The architectural decisions that are made in a rush can have long-term ramifications and can actually be quite difficult to unpick and untangle.
That’s kind of a summary of where we are in the development world. If you go back to, for example, the 60s you can look at things like the Apollo’s Moon Shot, something Keir likes to call ‘initial programming.’ They actually wrote computer programs that ran on the spacecraft and they were faced with a problem that once they launched, if there was a software problem, unless it was a very, very simple one it couldn’t be fixed.
We used to also have this kind of problem with embedded software, with a device having a program running inside it like a watch or, some early IOT device. You flashed the software, and it was very hard to update it. There might be one or two upgrades in its lifecycle. The programmers had to be a lot more careful about the coding and it required a lot more testing in order to ensure that they weren’t shipping out something that then later turned out to be a dud.
Otherwise, you would have millions of these devices sitting out there that were dead. So, we do have some history that we can draw on in order to look into how programming should be done to assure that the quality is as high as possible. But the vast majority of programmers out there are now familiar with the, “smooth, fast and break things paradigm.”
The Issue with Blockchain
This leads onto blockchain. The issue that we have with blockchain is that it goes back to this more inflexible programming paradigm. That if you launch a blockchain and you have nodes that start running to maintain it and you’ve set all the parameters for your blockchain, it becomes very hard to make changes. For example, if you decide that you want to move from one consensus system, like Proof of Work to another one such as, Proof of Stake. You need to convince all the participants on the blockchain that it is indeed a good move. You really want to make sure that you get it right or as right as possible the first-time round, because in programming you can’t get it a hundred percent right immediately.
There will always be bugs, but you can make sure that the list of bugs is as small as possible when your code is put out there. So, if you’re looking at deploying a blockchain, you should be paying close attention to quality. Similarly, if you’re deploying smart contracts that are programs that run on a blockchain. Again, because of the nature of blockchains they can be very hard to update.
There are tricks that you can use in order to upgrade your smart contracts as time goes on. But they are tricks, and they have some issues associated with them. So again, before a smart contract is launched onto a blockchain, it really pays to make sure that the quality is as high as possible. So how do you go about that?
Testing 1, 2, 3
It does really boil down to testing. One thing that the blockchain does is that you’ll find that for any blockchain worth its salt, there is a parallel blockchain running known as a Testnet. In some cases, there can be four or five or even six Testnets. For example, Ethereum has at least four for people to check that everything is working before they spend real money to deploy a program.
Going back to quality and its sort of sibling security. How did we end up with it being nice to have? The answer there is that it was down to basically business practices when you’re looking to sell or gain traction for a software product you are primarily looking at making a profit.
You can ask yourself the question. If you go to buy a car and you’re talking to the salesperson, what kind of questions are you going to ask them? You’re going to ask them, “what’s the top speed. What’s its fuel efficiency. Can I have a sunroof with that?” Very few people ask a salesperson how much testing did this car get? It’s not something that’s at the top of your mind as someone purchasing a car, what’s the safety of this car going to be?
The reason for that is that we rely on regulation and we rely on the fact that the governments have passed laws that say that cars need to meet certain thresholds before they can be sold. You cannot go and produce a car and mount a big spike on the steering wheel anymore. Bad news is that there were actually models of cars like that in the fifties.
Experience has shown us that that’s not a good idea. Laws are passed that said, cars have to meet these certain standards before you can sell them. This is the problem when you look at something like blockchain programming, we don’t have the same kind of regulation in place. We don’t have that demand at the moment. Again, in traditional programming if you’re going as an enterprise to buy some software from another corporation, you may make demands.
For example, you may say that you want to meet ISO 9,000 standards or you may demand that there’s been a high-quality security audit on it. But even those things have problems. Keir worked as an ISO standards compliance officer before and says, “It sometimes felt a bit like rubber stamping to me, you know, you checked all the boxes, but there wasn’t really a passion on the part of the company to make sure that we met these standards. It was just, can you please make sure that we can tick the boxes and that when an audit comes in, they agree that those boxes had been ticked correctly.”
‘Quality and Security become business decisions’
This is the thing, quality and security become business decisions. They’re not anything in terms of kind of ‘develop a pride’. For example, certainly in the early stages of start-ups they go on the back burner because they’re expensive. You want to make sure that you’ve got market traction, that you’ve got a good customer base before you start putting money into those kinds of things.
Ironically, it makes them seem like a bit of a luxury, which as you can see in the long run, they’re definitely not.
Blockchain on fire?!
You can compare it to having fire insurance on your house. The odds of it burning down are very low, but you know that the impact is going to be very high on the life that you’re leading.
But the cost of fire insurance is relatively low. Imagine if it costs you 20% or 30% of your yearly salary to get insurance for your house, would you still buy it or would you take a gamble and say, “I’m just going to risk that it doesn’t burn down.” Another thing we see is that as a result of this, when you look at salaries, you’ll find that test engineers get paid less than developers. Although, they do get paid more than tech writers.
For example, a start-up or a fledgling blockchain project, it does make emotional sense to do as little testing as possible. It’s just one of those unfortunate things in the way that business works.
That’s covered testing and quality in software, in part 3 Keir looks at at a number of cases where blockchains have gone wrong and why they went wrong.