This is the Part III in the blog series Technical Debt – Deal with it Now or it’ll Deal with you. Catch up with Part I: What is Technical Debt and Part II: Identifying Technical Debt

Hopefully by now it has become apparent that it is nearly impossible to avoid accumulating technical debt while building software and systems. The question is not so much whether or not you are going to take on technical debt but rather how quickly, what kind and what you are going to do about it in the long term. Just like consumer debt, technical debt will show the magic of compound interest but in reverse – just as small investments to a 401k over time will build up a nice nest egg for retirement, a bit of technical debt added to consistently over time will result in a nearly insurmountable pile of technical debt. Taking on a responsible amount of technical debt along with schedule payments will kept your leverage low and still enable you to make those large purchases that are necessary for large product development.

Preventing Unplanned Technical Debt

Developing software without a concerted long-term quality plan including accepted development standards, regular quality checks and a unified understanding of what quality code is will result in chaos and debt as quickly as any band of cowboy coders will. One of the most important social compacts amongst a team of application developers is a solid Definition of Done – a Definition of Done (DoD) is a checklist that enables a team to say “this feature is shippable”. Key elements of a DoD include an agreement on extent and type of tests, acceptable code style, degree of peer review, amount of documentation and acceptable frameworks and versions. A rigorous DoD will make the entire team aware of the level of quality expected of committed code – and that not meeting the DoD will likely result in incremental technical debt and thus requires more work.

Best Practices

Pair programming and peer reviews are both highly encouraged Agile methods of vetting the approaches, consistency and elegance of code. Both mechanisms enable more senior developers to mentor apprentice developers and for apprentice developers to make senior developers aware of new technologies. Having a second pair of eyes and hands on the code base tends to bring down the level of technical debt since inadvisable code is likely to jump out to someone new to the code – who, by the way, did not author it.

There is a practical limit to the amount of pair programming and peer reviews that can be done on an ongoing basis in most shops. For this reason, Mercury has found automated pipelines with quality checks to be a reasonable base level proxy and a timesaver to guard against common stylistic failings, reduce the amount of naïve technical debt introduced into a product’s code base and increase code consistency. Tools like Grunt, Gulp and SonarQuebe along with associated code linters will help point out sloppy code or approaches counter to the organization’s DoD – and since it is done by a machine upon every check-in there is essentially no overhead added to the development effort.

Keep Your Planned Debt Responsible

Just as you should avoid large levels of debt on high interest credit cards, developers should strive to avoid knowingly take on high interest technical debt. High interest technical debt is the kind of technical debt that is present in a module that is often modified or depended on by other modules and is either brittle/difficult to change or throws a lot of exceptions/performance problems. Rather than having expensive code routines that carry out single operations we have now spread the indebtedness throughout the application and this debt will cost the team money every day as we add features. Conversely, those parts of the app that aren’t frequently touched can be prioritized lower, even if their implementation is considered to be particularly poor.

When Your Credit Rating Gets Bad

With consumer debt a poor enough credit rating will render you unable to buy a new car or finance a home and remedial credit action is needed to get that new car or home. Such is life with technical debt in software also. The software team needs to stop new feature development and remove the technical debt so that not only stability will come back to the product but so that additional features can be added without immediate breaks to prior functionality. Consider making these kinds of lump sum payments against the software mortgage that is your technical debt so that the application’s credit rating returns to a reasonable level and typical spending habits can resume. Know, though, that many experts consider this a “condition critical” move only and not one that should be leaned on regularly.

Good vs. Bad Credit

Handling Technical Debt in Agile

The best method for handling technical debt I have ever heard of comes from “Uncle Bob” Martin and leans on the old Boy Scout rule: “Always leave the campground cleaner than you found it.” By intentionally improving the environment for the next group of campers (developers) the team can move into the next sprint knowing that they do not need to dedicate the first part of the sprint cleaning up a technical mess from the prior sprint.

Team members must be empowered and culturally encouraged to clean up a technical mess they identify while working on adding new features. This debt service should only take place to a reasonable threshold, though, rather than completely re-factoring and re-implementing a routine while simultaneously developing the committed new feature. While such refactoring is admirable it’s not often realistic – do enough to make the situation better and flag any extreme problems as a future user story in the product backlog.

Which brings us to the topic of “balloon payments”. In his book Essential Scrum, Kenneth Rubin warns against the practice of regularly carrying out an entire sprint for refactoring or technical debt reduction work (a balloon payment) on an application. In Rubin’s opinion the regular use of balloon payments subtly encourages the team to accumulate technical debt, avoid dealing with technical debt as they find it and rather postpone to a concerted balloon payment sprint (which may never come). This practice can also lead to regressions because many areas of the solution that were not recently developed are touched in a very short amount of time during the “bug hunt”.

Instead, Rubin recommends that the team handle technical debt while performing client-valuable work within each sprint. One example of doing this is address trouble tickets that come in from the field during a sprint. When technical debt is found to be at the root of the ticket it should be flagged as such and then the team should determine if can be worked into the current sprint. Another in-sprint debt handling approach is that when a feature is being added or extended to take note of any technical debt (particularly if it is impeding progress) and refactor or otherwise reduce debt while constructing the new feature. Not only is this likely to be a more efficient approach but should also improve the bandwidth committed to thinking about removing the technical debt in the most optimal fashion.

Finally, another ongoing method to actively reduce technical debt is to write stories and add them to the overall product backlog on a regular basis and assign a percent of the product budget to address this identified technical debt. In this way the Product Owner can occasionally work high priority technical debt stories into sprint backlogs.

I started my journey here at Mercury as an intern, fulfilling a credit requirement towards my computer science degree. The stress and uncertainty associated with an internship is hardly an uncommon sentiment among students and those just starting out in the tech world or any profession for that matter. I would like to offer those that are in the humble beginnings of their computer science careers my positive account of growing from an intern to an application developer. The impact a technologically diverse, tenacious and structured company like Mercury can have on one’s progress is invaluable.

My First Day

I hadn’t the slightest inkling of what to expect of this internship. My limited job experience consisted of bar tending, my perception of a genuine office environment consisted of what I had seen on television. I was vaguely aware that my education had not begun to scratch the surface when it came to the real application of programming and development, thus my nerves were getting the best of me.

My first day at the office, I was greeted with cheerful faces and shown to my desk. I had to make an effort not to reveal the unadulterated excitement having my own desk, computer and official Mercury email address triggered, it felt like the first big step towards being a fully operational grown up. Much to my dismay, the excitement quickly mutated to nervous fear once I had a few days to observe these mystical creatures in their element and began dipping my toes in meetings. Everyone might have well have been speaking Chinese (note: I cannot speak Chinese). The sheer number of acronym’s being nonchalantly thrown about made my head spin. I could barely hold on to the concepts discussed. It was clear that a dauntingly wide variety of intricate moving parts went into the production of these applications, all of which I had no comprehension of. All sorts of anxious, self-deprecating thoughts were peppering my brain, diminishing its capacity for reason. What am I doing here? Who in the world designed the learning plans for programming at my school? These people are on another level, how long will it be before I could possibly have a shot at doing anything they do? Will they discover how little I understand and wonder if I’m in the wrong profession? What the heck is a scrum, is there a Nuget package for it?

Stressed Business Woman

My first duty was to shadow a developer while she worked. I sat down, saw Visual Studio open to some C# and was relieved to see something familiar. The feeling didn’t last long, that comforting feeling was ripped away from me within moments of studying the screen. I had never seen a fully operational, packaged and shipped solution at play. Not only that, she’s maneuvering between various utilities and software, casually describing complex operations like builds and pipelines, she’s using frameworks I had never heard of (at this point if I had heard of it, there’s a 100% chance I had zero experience implementing it). Suddenly the knowledge I had of C# and .NET felt painfully dismal.

I watched and feverishly jotted things to Google and told myself I’d master the concepts this weekend before they notice I have absolutely no clue what is going on (bless my little heart). I continued to watch the developers while they worked and attended meetings so I could familiarize myself with the concepts of Scrum. The lists of things to Google filled entire notebooks by the second week. I continued to live in silent fear for a short while.

To get my feet wet, Pluralsight credentials were presented, I would be directed to watch some tutorials on content management systems and a number of other subjects that Mercury has foundation built on. Granted, these calmed my nerves a tad as they defined some of the technologies and subjects I had been hearing about, they also introduced an exponential number of other subjects that must be understood in order to even begin to dabble. I found myself feverishly chasing my tail and running in circles in an attempt to find a place to start.

Real Work Begins

The time had come that I receive some bona fide work to do, it was to be unit tests. I had heard of test driven development, but my school (which shall remain nameless) fell woefully short on this topic, among many others. I was assigned a mentor, Graham Morris, one of the principal application developers here at Mercury. Yet another idea that made me queasy. I’ll inevitably have to code in front of this guy and finally be outed as a fraud and probably publicly humiliated, they will throw things at me and consult with the tech gods to ensure I never get a job anywhere for as long as I live. The sky was undoubtedly about to fall. Graham was to sit at my desk and talk me through the process.

For the first time in two weeks I was able to relax. Graham had no intention of testing me, only teaching. He walked me through the process, step-by-step, and provided several examples to guide me. He left me to my own devices and graciously helped whenever I requested it. Mind you, it was apparent that I didn’t fully comprehend the complex classes or components I had been using to write the tests, nonetheless he joined me in celebrations of even the smallest of accomplishments.

Light bulb moment

After a couple months of unit tests and following endless rabbit holes through this mountainous labyrinth of code, out of nowhere, dots began to connect. I had accidentally realized why and how a piece of the puzzle worked. Once the epiphanies started rolling in, they would beget more epiphanies. Light-bulbs far and wide. This was an unbelievable feeling and a pivotal moment for my overall perception, despite the realization that I had an increasingly long road ahead of me, the professionals around me no longer evoked fear, but inspiration. Graham was consistently knowledgeable, patient and helpful with me. Everyone was willing to extend a helping hand. I was relieved to be at Mercury with access to such capable experts and really began to enjoy this endeavor.

The End was near or was it…

Once the end of the internship approached, I didn’t want to walk away from the constant flow of learning and apprenticeship, Mercury was more than happy to let me keep coming in and gaining hands on experience, giving me full access to its priceless resources. Watching these remarkable, talented people work respectively and effectively together with innovative technologies was and still is providing me with the ideal environment one needs to gain solid experience and constructive habits.

Eventually, Mercury gave me the honor of taking me on as a full time junior developer. They encourage creativity through exploration and frequently extend the opportunities to discover new technologies and frameworks.

Don Bickel, owner, partner and head of the technical division recently let me take a stab at Android development. (It took me all of my strength not to ask him if he was experiencing some momentary insanity for letting me touch one of his first native apps, luckily my beaming pride overshadowed my doubt). He took a risk and let a junior work on a project comprised of uncharted territories, he provided ample resources and worked together with the team to improvise plans for a new found concept of development. Don maintained unsullied enthusiasm throughout the learning process, he experienced the unknown along with us, fully immersed as we were.

I value Don’s penchant for fresh tech and ever-growing objectives. He avidly researches the constant influx of technologies, paying careful attention to those that break new ground in the tech community. I’ve heard rumors of burn outs and stagnant goals in this profession, I struggle to fathom such a scenario in Mercury’s future.

I never imagined I’d even touch Android after choosing .NET as my path, and now I’m gaining opportunity to experiment with a wealth of platforms. Did I mention I’ve only been doing this full time for 6 months? Thanks to Mercury, 6 months has taught me a number of languages and frameworks- and not passively- I’ve become increasingly comfortable with the AngularJS framework (so much that I’ve developed a fanatical emotional attachment to it), I’ve been heavily exposed to Azure, functional testing with various tools, the use of several IDE’s, the exploration of countless .NET and Microsoft Resources, my skills manipulating data with T-SQL never seize to expand.

Don has put a great deal of efforts into the evolution of our dev-ops procedures. Despite my short time here I have personally witnessed our continuous delivery pipelines ascend into godliness combining source control, testing and rigorous automated code analysis. Something tells me this isn’t a typical commencement to my chosen career path, but it certainly should be.

Final Thoughts

The Mercury team has molded me from a student with about as much real-world technological comprehension as a lamp, into something I’m very proud to be. Admittedly, my mentor still has to hold my hand when the going gets tough, but I’m fortunate to have such a knowledgeable coding virtuoso take the time and energy to give me guidance, in addition to support from each of the talented team members I’ve had the great pleasure to work among.

Most would likely agree this particular profession requires apprenticeship from within the trenches to achieve a well-rounded perspective paired with a bedrock built to carry abundant future prowess, Mercury is an excellent example of a company that nurtures the growth of its employees.

I would describe my internship with this company as the second most significant event in my young life thus far, only bested by becoming a mother. They have immense respect for the team and as a result we all take great pride in our work. I look forward to a long, rewarding career with Mercury as long as they’ll have me.