In a previous article, where I discussed the fractal nature of software, I touched upon one of the difficult dilemmas in software engineering: how much complexity is needed to implement a feature, and more broadly, a fully functional system? If you’re adding more complexity than strictly needed, you’re gold plating. But if you cut a couple of corners too much, sooner or later, your project will hit the wall, and you’ll have to face a potentially costly refactoring exercise to add the needed complexity so that you can make progress again.
In many cases, it will be obvious how much complexity your system needs to be able handle. Often, the scope for a new system is defined in terms of a set of requirements derived from user needs experienced and expressed by actual users. For smaller systems, these requirements can be very concrete, and therefore give a good overview of the complexity in the new system. Be aware though, there will be surprises, and things that seem easy and straightforward may turn out to be quite complex once you start investigating the problem a bit closer.
Grey Areas
For larger systems, there’s usually a grey area between clearly cutting too many corners on the one hand, and gold plating the solution on the other hand. The less concrete the description of the requirements for the new system, the larger this grey area will be. Scoping down a project isn’t just about removing features from the wish list that aren’t really needed, but also reducing this grey area by describing more precisely the requirements that have been prioritized into the new system.
But for as long as you’ve described the requirements in text and not in code that has been fully end user tested, there will continue to be some uncertainty as to how much complexity you really need. Adding a bit more code to handle a particular condition in the process flow may be not so expensive, and it gives you, as a developer, that satisfactory feeling that you’ll be able to handle most, if not all, corner cases you’re able to think of. If you’re an experienced developer, that’s where you can make the difference. If you’re good, your experience lies precisely in being able to imagine all the important corner cases that can blow up the system if they aren’t covered by your code, without imagining too many hypothetical situations that will never occur in reality and only increase the size of your code base.
Validating a Date with a Regular Expression
Now let’s investigate what happens when you cut one corner too many. Say we want to validate dates again (and for some reason can’t use a function from the standard library). By accident, we’ve just listened to a random podcast about regular expressions, so we conclude that that’s how we can keep things simple and easy. Sure, a couple of colleagues are trying to tell us that it’s not such a good idea, but that’s probably because they haven’t listened to the same podcast as we did, right? Anyway, we’re going for something like the following:
^d{4}(0[1–9]|1[0–2])([0][1–9]|[12][0–9]|3[01])$
This regular expression requires years to have four digits, and restricts months to be between 01 and 12, and days of the month to be between 01 and 31. It won’t, however, stop users from entering dates like 31 June or 30 February, and doesn’t handle leap years either. Note that even though it screams “Wrong!” to most people, the solution has an impressive accuracy of 99.93% for regular years, and 99.94% for leap years. (Out of 10,000 combinations for each year, there are only seven or six false positives, and no false negatives.)
Now, let’s increase the accuracy even further by adding another layer of complexity. Handling the shorter months in a regular expression is still a feasible job to do. But most developers will agree that the technical debt of the solution is starting to get rather large. And yes, it’s possible to extend the regular expression even further, so that it can handle leap years too. I’m sure your favorite AI coding assistant will be more than happy to help you with that. But the result won’t be very readable or maintainable.
When Refactoring Is Needed
But what happens when you suddenly find out you need to be able to validate dates across the Gregorian reform? That’s where you really hit the wall. I’m sure there’s a way to solve it with a regular expression (there always is…), and it would probably also be a lot of fun as a challenge on its own. But the only sensible thing to do now is to refactor the function, probably using a couple of easily testable if statements and/or subfunctions. However, this is the sort of expensive refactoring that frustrates managers and users so much, partly because it’s so hard to explain why it needs to be done, but also partly because it involves us admitting that we made a terrible misjudgement at the start of the implementation of this feature.
An experienced developer isn’t only able to imagine all important corner cases that a system needs to be able to handle in order to function well. He also knows which approach has a good chance of covering all the complexity that’s needed, without overcomplicating the design, in a testable, maintainable and extensible manner. Sometimes, a regular expression is the right approach for that. In many cases, it’s probably not.
When Are You Gold Plating, and When Are You Cutting Corners? was originally published in Compendium on Medium, where people are continuing the conversation by highlighting and responding to this story.
