by Vladimir Mandić
The software industry lives with a phenomenon that is conveniently named technical debt (TD) [1]. According to some estimates 25% of all effort spent in the industry is related to this phenomenon [2], eighter by having difficulties while implementing software changes or new features that turn to be too expensive for the value that they bring to the business; or by having development teams struggling with reengineering and refactoring software parts that are critical for future system operations. Interestingly, the TD phenomenon is prevalent in organizations of all sizes, from small to large companies [1].
On the other hand, nowadays the majority of software development teams adopt and follow one of Agile development methodologies. In 2000s, Agile methodologies (rooted in Agile manifesto) emerged as an answer to ever-increasing demand for delivering value to the businesses. This was even further accentuated, around the 2010, through ideas of amending agile development with Lean principles—Lean Software Development [3]. To simplify, this is yet another attempt to streamline value creation throughout the entire software development lifecycle with continuous (and fast) value delivery. Thus, Agile software development is often associated with speed, i.e. intention to rapidly deliver functionality and features that can be deployed to the users. Just to be clear, Agile methodologies have many other qualities besides the concept of short time-boxed iterations.
At about the same time, in the 1990s, Ward Cunningham proposed the technical debt (TD) concept or a metaphor. The metaphor of technical debt has been instrumental in communicating hidden—internal software quality—problems that result from suboptimal decisions made by all involved actors during the software creation process. As a result of that, investors became much more sensitive to technical debt when investing in startups or software products. Investing in a software product that is overloaded with technical debt will lead to higher costs for future software development and maintenance, and may even lead to technical bankruptcy [4], i.e., a situation where the cost of change is so high that it makes no sense to implement those changes.
So far, what we know about TD is that it can be injected into any artifact during software development. It is not just about source code, it can be injected into requirements, user stories, architecture, design, etc. [5, 6]. But probably the most elusive characteristic of technical debt is context. To explain this, let me use an analogy with DNK genome and genes. Software artifacts can carry TD issues, e.g. some parts of source code can be burdened with TD, however the exposure—effects of TD—depend on the context of how the system is used. Similarly, in a genome, some disease-carrying genes can be present, and whether they will be activated depends on the environment, i.e. context. This epigenetic-like characteristic makes managing TD challenging.
One of the results of InsighTD project, was identification of TD causes, effects, payment practices and payment avoidance reasons. [6]
Figure 1. gives an illustration of the most common causes that lead to the TD injection, followed by the most common effects, payment practices and avoidance reasons.
There is a multitude of factors that lead to favorable situations for TD injection and later to TD exposure, as well as many factors that impact a decision to remediate the injected debt or to accept it. Some factors are unique and very specific, while others can occur in different project contexts. Furthermore, comparing technical and non-technical factor occurrence gives us insight into different development role groups’ perspectives. Consequently, it is expected that better alignment leads to better communication with less friction between groups.
Figure 1 depicts the process of injecting and handling the injected TD. Prior to the injection of TD, potential causes can build-up a situation that results with newly-created TD. Ideally, causes can be recognized before debt injection. This would require active monitoring of causes, by all involved practitioners. From Figure 1, practitioners can be informed about the most significant causes, either from technical or from non-technical perspectives. Figure 1. presents the top five ranked causes that are more likely to contribute to TD injection.
Once TD is present in the project, a number of symptoms can be identified. Note that symptoms or effects may be considered as an indication of a present TD. Special attention is advised for top listed effects in Figure 1. According to TD experienced practitioners, listed effects indicate a higher probability of having TD in the project. This can be used by practitioners to assess the project situation before any specific TD issues are identified.
At some point, a decision is made to remediate TD issues. For example, in some organizations, a form of cost-benefit analysis will be used to argument the decision, while in others no decision will be made explicitly, i.e., implicitly TD is accepted without clear reasoning behind such a decision. TD practices and payment avoidance reasons (Figure 1) can help by suggesting the most effective TD payment practices and providing the most significant avoidance reasons.
Especially, the list of payment avoidance reasons, can be used as a powerful tool to objectively prepare and argument TD payment decisions. For example, if a consensus is made to deal with TD, then the practitioners should address all major payment avoidance reasons in advance by taking actions to neutralize them. This can include dedicating funding, time, resources or altering short term goals. By doing so, the success of dealing with TD will be increased.
In conclusion, just to be clear, debt-free software is an utopistic idea, we do not live in an ideal world with unlimited resources and time. Also, ever-present tension to deliver software under time constraints compels all involved actors to “borrow” some time from the future by implementing suboptimal solutions that need to be fixed in the future, after the release, i.e. when the delivery date is long past behind. We can all start building better software if all involved actors start asking a question: “How much do we need to borrow?” By answering that question, we will be aware of how much we need to pay off after the release date is past behind.
References
[1] Ramač, R., Mandić, V., Taušan, N., Rios, N., Freire, S., Pérez, B., Castellanos, C., Correal, D., Pacheco, A., Lopez, G. and Izurieta, C., 2022. Prevalence, common causes and effects of technical debt: Results from a family of surveys with the IT industry. Journal of Systems and Software, 184, p.111114. https://www.sciencedirect.com/science/article/abs/pii/S0164121221002119
[2] Besker, T., Martini, A. and Bosch, J., 2019. Software developer productivity loss due to technical debt—A replication and extension study examining developers’ development work. Journal of Systems and Software, 156, pp.41-61. https://www.sciencedirect.com/science/article/abs/pii/S0164121219301335
[3] Mandić, V., Oivo, M., Rodríguez, P., Kuvaja, P., Kaikkonen, H. and Turhan, B., 2010. What is flowing in lean software development?. In Lean Enterprise Software and Systems: First International Conference, LESS 2010, Helsinki, Finland, October 17-20, 2010. Proceedings (pp. 72-84). Springer Berlin Heidelberg. https://link.springer.com/chapter/10.1007/978-3-642-16416-3_12
[4] Beltan Tönük, 2021. Understanding Technical Debt: Your Guide to Navigating in the Age of Digital Disruption, Independently published
[5] Berenguer, C., Borges, A., Freire, S., Rios, N., Tausan, N., Ramac, R., Pérez, B., Castellanos, C., Correal, D., Pacheco, A. and López, G., 2021, November. Technical debt is not only about code and we need to be aware about it. In Proceedings of the XX Brazilian Symposium on Software Quality (pp. 1-12). https://dl.acm.org/doi/abs/10.1145/3493244.3493285
[6] Mandić, V., Taušan, N., Ramač, R., Freire, S., Rios, N., Pérez, B., Castellanos, C., Correal, D., Pacheco, A., Lopez, G. and Izurieta, C., 2021. Technical and nontechnical prioritization schema for technical debt: Voice of TD-experienced practitioners. IEEE Software, 38(6), pp.50-58. https://ieeexplore.ieee.org/abstract/document/9508982
