2nd Aug 2019
In the previous v1.1.0 – Async Rendering, we split our components and successfully used setTimeout to move the expensive search off to a separate execution stack, allowing the browser to paint input updates instead of waiting for the search algorithm to complete:
Unfortunately, there was no getting around the search taking as long as 330ms per searchTerm and blocking the main thread causing UI lag.
I recently read about Progressive Web Apps and how the introduction of Service Workers had made them much more practical by offloading certain processes like caching to a separate thread and so I started reading up on Service Workers. It was a brand new api that I would need to invest time in learning.
But, before I was willing to invest this time I wanted a proof of concept that the addition of extra threads could be implemented to actually improve performance.
Server as a second thread
This particular implementation was more challenging than previous searches and auto suggests I worked on because it was all being done on the Front End. Usually, searches would ping an API to get the results and this made the Front End performance much easier to manage. One way to think about it is that APIs and the servers they run on are essentially extra-thread resources for the Front End application to utilize.
So, I came up with the idea of making a simple node server that I would ping with the searchTerm and use as a secondary thread to run the expensive search. This turned out to be especially easy because I already had a custom development server setup on this project, so all I needed to do was add a route:
and change the setTimeout inside of SearchResults to a fetch:
Moment of truth:
Rock on! Those input updates are almost perfect. The results updates on the other hand are still lagging pretty far behind, but that’s OK! Remember our first priority is to get that feedback to the user that what they are typing has registered as fast as possible. Let’s see how this looks in the profile:
Look at all that beautiful white space in the main thread. We’re no longer seeing that pile up of execution stacks from before. This will be a recurring theme in performance optimization going forward. The more empty space on your main thread the better!
You can see in the top how long those testSearch requests are taking. The last, longest one is actually over 800ms which seemed surprising until you realize that isn’t the search taking that long, but rather the fact that our Node server is still single threaded. It can’t start the next search until it finishes the first. Because the typing is so much faster than the search, the searches get queued up and delayed. The actual work of the search function is that final section after the previous search finished, around 315ms.
Essentially, by offloading the expensive work to the server thread, we’ve moved the pile up to the server. So while there’s room for improvement, that main thread looks awesome and the UI feels a lot more responsive!
We’ve proven the concept, let’s get to work!
After a bit of research, it turns out Service Workers aren’t an option for us because they are not compatible with Internet Explorer. Fortunately, they have a close cousin called Web Workers which are compatible with every major browser, have a much simpler API, and do exactly what we need!
Next up, Web Workers!