You may have heard the buzzword – technical debt.
While definitions may vary, the costs inherent in maintaining aging, highly-customized computer programs are clear - operational inefficiencies, strategic liabilities, higher total cost of ownership and lost opportunities to innovate. These systems can also negatively impact customer experience and data security. Building upon last week's blog that discussed why customized software development should be discouraged, today's blog talks about technical debt and how Arizona could begin to resolve it.
Technical debt is metaphor coined by Ward Cunningham, one of the original founders of Agile programming. Generally speaking, technical debt arises as software developers make more and more customizations to computer code to quickly fix system problems, despite it not being the best overall solution. The Agile Alliance explains that over time these hacks to source code can cause a number of issues “related to architecture, structure, duplication, test coverage, comments and documentation, potential bugs, complexity, code smells, coding practices and style.”
Like financial debt, if technical debt is not repaid (or fixed), it can accumulate “interest” making it more difficult and costly to implement changes in the future.
While all organizations have some form of technical debt, it seems to be more prevalent in government operations. A 2018 article estimated that over the next five years, government agencies will have more than $7 billion in technical debt—old hardware and software that needs to be replaced or updated.
Arizona is not unique.
Arizona maintains aging computer systems that have existed well beyond their expected lifecycles. But with more visible spending priorities, like education and healthcare, departments have been forced to put their legacy systems on life support for as long as they can to avoid new capital expenses, even though that doesn’t result in a better system.
In fact, just last week, the issue of technical debt came up at the Arizona IT Authorization Committee meeting. An agency spoke about its efforts to upgrade a statewide computer system. The system had accrued between 600-700 system customizations (that’s down from 1200) since its implementation in 2004, and now the software version will not be supported next year. With so many customizations, the current system is unmaintainable.
According to officials, upgrading to the next version of the existing software offered no improvement in user experience, didn't comply with the state Cloud First Policy, and still missed important functionality.
One astute Committee member asked about the benefits of replacing the system instead of spending more money on a software solution that ultimately doesn’t meet the state’s needs and only delays the inevitable. State officials pointed out that replacing the existing system with a best-in-breed solution would eliminate the time and effort of dealing with the customizations (they would simply become irrelevant) and would leapfrog the state into a modern business process with features aligned to our mobile workforce.
Faced with the decision of whether to spend money on upgrading an increasingly obsolete system or implementing a new one, state officials must confront technical debt.
So, what should Arizona officials do?
Make the Technical Debt Visible
To prevent technical debt from accumulating, we have to stop ignoring problems when they first present themselves. As time goes by, making changes to old computer code only gets more complicated as new business needs arise.
The Consortium for IT Software Quality has proposed an industry standard for defining and measuring technical debt by analyzing software code and identifying potential defects. Below are the main components:
- Security: Critical security violations in the source code drawn from the Top 25 security weaknesses in the Common Weakness Enumeration repository;
- Reliability: Critical violations of availability, fault tolerance, and recoverability of software;
- Performance Efficiency: Critical violations of response time, as well as processor, memory, and utilization of other resources by the software; and,
- Maintainability: Critical violations of modularity, architectural compliance, reusability, analyzability, and changeability in software.
When used properly, a framework like this can create a more systematic approach for managing, measuring and remediating technical debt.
Assign Economic Value
In an article by John Breeden in NextGov, he explains: “Like all debt, technical debt is bad because the process of satisfying it actually prevents efforts to eliminate it. In a sense, the government is paying off the interest on its technical debt by maintaining legacy and aging systems, with nothing left over to purchase new systems that would be much more efficient and less labor-intensive overall.”
Once the technical debt is categorized, government officials can begin assigning monetary values in terms of system upgrade costs, security patches and personnel costs of maintaining systems. Soft costs such as reduced productivity could also be discussed. Now, technical debt can become a powerful management tool.
Overcome the Political Barriers
Government has a lot of technical debt for a reason - unpredictable and long budget cycles, an unskilled workforce, slow procurement processes, agency leadership turnover, and risk adverse cultures. This isn’t exactly a lean start-up in Silicon Valley. Over time, these factors have created a situation where a disproportionate amount of agency funding goes to maintaining legacy systems. According to the federal GAO, the federal government spent over 75 percent of its FY15 IT budget on operations and maintenance, which has caused a decline development, modernization, and enhancement activities.
As government better understands technical debt and the true dollar costs, as well as the impacts on service delivery, the hope is that the political barriers will not be so pronounced.
Be Open to Financing
Eventually technical debt will reach a tipping point when government agencies must decide whether or not to honor its original debt or start new. The decision can be challenging especially if officials are afraid of failure.
However, with the advent of software-as-a-service and subscription-based pricing, government officials now have payment models to reduce upfront capital expenditures of building systems or customizing transfer solutions. Collaborating with the supplier community to think creatively about how to finance a new system is a worthwhile endeavor. And, taxpayers will save money in the long-term.
Why This Matters
If we can begin to explain the costs of technical debt and the long-term implications of spending good money to chase bad solutions, perhaps government leaders will better understand the trade-offs. Spending taxpayer dollars to support software that is insecure, unreliable and hard to maintain is not a wise investment.
Arizona government provides services that affect people's livelihood, and every hack or customization that gets coded into an old software system puts those people at risk. What is more, that technical debt limits the state's ability to innovate. If Arizona wants to be the most innovative state, it has to have the technology foundation to allow it.