The Importance of Upstreaming Issues

In modern software development, all software builds upon other software – nothing truly starts from scratch. Even the most trivial programs rely on software libraries and tools that were built by someone else. All of those libraries and tools have bugs. In many of those libraries, there are going to be features that would be nice to have as well. So what do we do when a bug is discovered or a need for a new feature arises?

The Two Schools of Thought

There are two schools of thought when it comes to these bugs and feature requests in dependent software:

  1. Work around the problem or implement the enhancement locally.
  2. Report the bug or enhancement request to the dependent software’s maintainer or vendor (also known as “upstream”), then wait for them to make the fix or add the feature, working with them as necessary.

The “Easy” Way

Option 1 is the so called easy way – or at first, it seems easy. The basic idea is that if you encounter a problem, you work around it. Let’s say a developer is using a charting library to graph some data, and finds that when certain types of data are graphed, the graph doesn’t look right. For the sake of this article, let’s assume that infinities aren’t handled in a way that makes sense. One approach is, whenever an infinity is encountered, substitute a value which makes the graph look acceptable by replacing the infinite number with a really big number instead. In a matter of an hour or two, the bug has a workaround.

This approach seems great – the problem was solved quickly. However, perhaps there are multiple graphs in the application – this workaround must now be made to all the areas of code that perform graphing. And when a new graph is introduced, we have to remember to add this workaround.

Worse yet, now consider that some time has passed and the project is done and the developer has moved on. A new developer is now working on the project, and adds a new graph – and is baffled by why infinities don’t work right. Time is lost as the new developer has to re-discovered the original developer’s workaround, or worse, develop a different workaround creating an even more complicated code base.

And even worse, let’s say the original developer is now working on a completely new project that happens to need graphs. Having experience with this graphing library, they choose it for the new project. And once again, they get a bug report saying that infinities are not handled properly. And once again, they must re-discover and re-apply a workaround all over this new project as well.

The end result is that there is substantial time wasted at multiple levels with lots of unhappy people:

  • The developer loses time by applying the same fix in multiple places.
  • The testers/QA lose time by continuously finding the same bug in multiple places.
  • The project managers get annoyed as the schedule is less than predictable.
  • The client is unhappy because a bug that was “fixed” keeps reappearing.
  • On subsequent projects, all the same time loss and malaise occurs again and again.

The Right Way

Option 2 initially seems like more work but ends up being a significantly lower effort for everyone. Continuing the previous scenario, if instead of simply making a one off fix, the developer immediately reported the bug to the charting library’s development team. They spend a few minutes to come up with a test case, fills in the form on the charting library’s bug tracker, and then moves on to something else. There are two things that happen at this point: either someone will reply to the bug with a fix, or not.

If someone replies with a fix, and that fix gets included in the charting library, then great! The developer never needed to do anything more than report the bug – they simply upgrade the charting library to the latest, fixed, version. They immediately saved time and effort, and the problem is fixed for this particular chart, all other charts in the project, and all other projects using this library.

If no one replies, the developer eventually goes back and tries to fix it themselves. They figure out how to fix the problem, and submit a patch file with the code needed to fix the issue to the project bug tracker.  With luck, someone accepts the patch into the project, and as with the previous case, the bug is fixed for everyone who updates to the latest version of the library. In this scenario, the developer may have actually spent more time on the bug than in the “Easy” way scenario above (as he had to collaborate with the charting library project, get the patch to be acceptable, etc) – but this bug won’t be coming back. And, because the experts from the charting library project reviewed the work, we can be confident the work is good, maintainable, and doesn’t introduce other problems.

In this scenario, the project manager is happy – the same bug isn’t being reported over and over. The developer is happy  – they don’t have to fix the same thing multiple times. And the client never encounters that annoying scenario of having to question why the developers are not fixing things they claimed to have fixed. Plus, other managers and developers out there are unknowingly happy, as they will never encounter this problem.

In the Real World

In the real world, project teams tend to not think ahead. As deadlines loom, projects enter “fire fighting” mode – rather than taking the time to do something right, the response becomes “just hack it and worry about it another day.” All of these hacks add up to increased technical debt (a term meaning, “we’ll take care of it later if there’s time”), leading to a scenario where a project is in constant fire fighting mode and can never catch up, constantly fixing bugs caused by earlier hacks, laying hacks on hacks. This spiral can result in turnover of managers and developers which leads to unhappy clients.

The graphing example above came from an Android app that I once worked on.  Rather than writing our own charting library, which would have taken a great deal of extra time, we chose a 3rd party open source library.  While the library wasn’t perfect, there were some bugs (including the one highlighted above) and it lacked some nice to have features, it still ended up saving us a great deal of time in the long run.

The project deadline was tight.  And it would have been easy to just code some workarounds for bugs and missing features, but my experience has shown taking shortcuts up front leads to longer delays later on.  So the team and I were determined to do things “The Right Way.” We worked with the library developers to fix our most pressing issues and learned a great deal in the process. All this while creating a clean code base that was easier to maintain.

When the Right Way Isn’t Possible

While doing things “The Right Way” is great, it’s not always possible.  There are cases where upstream contributions are not welcome, slow to respond, or physically prohibitive.  Some examples include:  

  • The problem is in hardware, and replacing it isn’t feasible. In that case, reporting the issue is still important as the hardware’s creator could help design the best workaround, but the end result will still be a workaround and not a new, fixed version.
  • The bug is in proprietary software that doesn’t accept bug reports or bugs are addressed in order of demand, making it likely issues that affect a smaller number of users are never addressed.   
  • The problem is in software that cannot be replaced/upgraded/modified for any number of reasons (legal, compliance, lack of source code access, etc). However, reporting the issue, if possible, is still important so the author could potentially help in formulating a workaround.

In the above cases, or any other where you’re unable to influence a fix at the source, it’s important that any workarounds you do come up with are well documented with what the problem is and why the workaround exists.  And if you’ve tried to solve the problem by upstreaming the issue, why it failed or what state it was last in can also help future developers pick up where you left off.