Disclaimer: The opinions expressed below are the particular point of view of one of our developers and do not necessarily represent the views of Gandi as a whole.
As the Web’s capabilities grow and ever increasingly becomes the platform of choice, websites have been growing too, in capabilities, in complexity, but first and foremost, websites have been growing in weight.
With the rise of Single Page Applications, built upon a vast eco-system and popular libraries such as React or Vue.js, a sizeable part of this phenomenon is enabled by and can be attributed to JavaScript, and from which Gandi is not exempt. Let’s take a further look at Gandi’s approach to building websites, some of the difficulties we face, and our take on solving them, and then reflect on our JavaScript footprint.
The increase in page weight & JavaScript
In the last six years, according to the HTTP Archive, the median webpage weight served to desktop users went from 1176kB to 2059kB, or a 75% increase. At the same time, the median amount of JavaScript went from 240kB, to 452kB: a 88.33% increase. While these numbers are on a steady, upward trend, they’re not particularily concerning in themselves, after all as the HTTP Archive put it: “JavaScript powers the modern web, enabling rich and interactive web applications.”
A website or a rich web application?
The Web has been evolving towards a new paradigm of rich web applications built with JavaScript. Traditionally this means that most navigation and interactions trigger subsequent rendering of the page that are handled by JavaScript, without requiring a full page reload. This is arguably a great fit for administration panels and complex user interfaces—rich web applications—as opposed to more classic websites, such as a company’s homepage or a blog.
However, the architecture of a website is oftentimes much more complicated than this purposely brief description and oversimplified examples, there are perfectly valid reasons to build a website in one way or another, however one must be aware that in any case, a website’s architecture is the result of the tradeoffs one is willing to make and comes with its own set of challenges, caveats, and costs, although it’s never a bad idea to refer to the “Rule of Least Power“.
What’s the cost of JavaScript for the user?
JavaScript’s impact on user experience—due to the complexities of how JavaScript can be loaded in a web page and how it may interact with it and Web APIs is tremendous, yet far from trivial to measure.
Various companies and tools are specialized in designing KPIs, analyzing websites, and providing insights, as well as actionnable feedback to developers and businesses so they can optimize their websites for performance, in which JavaScript plays a major part.
For the sake of brevity, and in the context of this article, we will take for granted that the following principles are beneficial to user experience:
- the less JavaScript the better
- the less parser-blocking scripts the better
- the more actions and content are available to the user without requiring JavaScript, the better.
With these three principles in mind, let’s jump into how Gandi builds websites, by focusing on two case studies.
Case study 1: www.gandi.net
Gandi’s insititutional website is a classic website, according to the categories we’ve defined earlier: all the pages are generated dynamically and served using Pyramid, a Python Web Framework. This website has been designed for performance and cacheability in mind.
At the time of writing, Gandi’s homepage in English scores a respectable 86/100 Lighthouse Performance rating and loads 211kB (79.44kB gzipped level 9) of JavaScript across two script files (note: with DNT enabled) that are required on every page of the website, and cached in the browser for subsequent page loads or visits.
These numbers would place it roughly within the 25th percentile of webpages analysed by the HTTP Archive for JavaScript bytes.
Because this website has been designed and is maintained with progressive enhancement and graceful degradation in mind, it is fully functional even with JavaScript disabled, which of course is understandably easier to do on an institutional website than other types of websites.
A fun fact about this website is that its development started around the end of 2014 and as of this day, it still relies a lot on the technical choices that were made at this time, including but not limited to page interactions handling using custom jQuery plugins, a tailored UI library built on Sass, a build system based on Browserify, Babel, Gulp…
If you spent a little time analyzing further the performance of these webpages, in particular related to JavaScript resources, there is no doubt you would find a number of improvements that could be made to further improve web performance and reduce the site’s JavaScript footprint, such as:
- using a slimmer build of jQuery—or removing it altogether 😉
- delaying loading of scripts that interact with elements that are below the fold
- using more modern build tools enabling various kind of optimizations such as dead code elimination or tree-shaking
- etc.
While a revamp of these tools and practices would without a doubt benefit both Gandi developers’ and user experience, we have not prioritized it thus far, and we consider the current situation satisfactory enough on all accounts.
Now let’s move on to a whole different kind of website for our next case study.
Seeing more “JavaScript footprint reduction” quick-wins for www.gandi.net we might have missed? Tell us in the comments section 😉
Case study 2: shop.gandi.net
Gandi’s shopping website is, I believe, the most interesting website to talk about in the context of this article on the concept of JavaScript footprint.
The first interesting part is that it is both a single page application, and a server-side rendered website; in fact, if we were playing keyword bingo, we could describe it as a React/Redux isomorphic SSR OAuth2 SPA ….
Then, out of ten apps sharing the same architecture, built on the same building blocks, and living in a single Git monorepo, app-shop is one of the most important, whether it be in terms of business criticality, sales, numbers of screens, diversity of use-cases, interactivity, etc…
Furthermore, it is an app that requires constant and parallel developments by all four feature teams – “squads” – within Gandi, while heavily relying on shared business logic or UI components libraries, which are, in most cases, each maintained by one of these teams.
Finally, it’s worth pointing out that the internal framework used for this project, as well as nine more, was initiated at the beginning of 2016, and at that time, Gandi’s Front-End engineers had little to no experience building such apps & frameworks, and that the React/Redux/npm ecosystem wasn’t nearly as mature as it is today.
But let’s get back to our main topics of interest, the web performance and JavaScript footprint of this app.
At the time of writing, the unauthenticated, English version of Gandi’s domain name search and suggestion engine requires a whopping 2.12MB (618.68kB gzipped level 9) of JavaScript across three script files—excluding runtime scripts of our self-hosted Piwik/Matomo analytics (for which one can opt-out via DNT) and self-hosted Sentry error reporting.
Still referring to the HTTP Archive reports, this web page ranges well within the top 10% of the heaviest in terms of JavaScript loaded resources.
Despite a testing taking place on an intermediate URL and from the US to our servers located in France, the previously mentionned URL scores a 42/100 Lighthouse rating for Performance.
There’s a saying in the web performance community that says something along the lines of “you only need to be faster than your competitors.” This is not what we aim for. Both as a Front-End engineer as well as a web user, I disagree—this is not good enough. We need to, and will, in time, do better.
However, taking a look at the associated sourcemaps for client.ES2017.[hash].bundle.js, you can notice that:
- this file weighs roughly 1.74MB (495.86kB gzipped level 9) out of the 2.12MB total mentioned earlier
- out of the file weight, about 610KB represents the code handling the rendering and all interactions within the whole app
- the actual app code handling the entirety of this screen represents about 82KB
What makes for such differences?
Why is so much JavaScript code required up front if only ~4% of that JavaScript truly represents the code for this single screen?
Well, this reasoning conveniently ignores the React & custom framework runtime, as well as common tooling & logic. Just including the weight of the two other JavaScript files brings the share of “truely necessary JavaScript code” from ~82KB / 4% to ~480kB / 22.64%, and this is still a very simplified and inexact estimate, but an interesting train of thought nevertheless.
But that still doesn’t explain what lies in the rest of the JavaScript code, that is loaded upfront, but not strictly necessary on this screen, let’s go over it quickly, one can see it boiled down to two things:
- Unnecessary code that covers use-cases that are not immediately or at all pertinent for this app or page
- Cross-browser compatibility code that’s actually unused or redundant on most browsers
Unnecessary code
In my opinion, this is at the same time a great strength and a great weakness when building a Single Page App, especially more so on a custom-made framework: most of the time, and by default, the full code of the application is loaded upfront.
On the plus side, once the JavaScript application code is loaded, all subsequent navigations bear no additional cost, and repeated visits benefit from it being cached in the browser, significantly reducing the impact of the app’s JavaScript footprint, even if high.
On the other hand, it easily leads to a similar situation to the one we just described: the initial cost of entry paid by the user can be unnecessarily high.
Although this behavior can be considered a sane default, there exist advanced techniques which allow a “best of both worlds” situation, in theory: “code-splitting“, where the idea is to load upfront only the strictly necessary runtime and application code that is required for the initial screen, while the rest is being loaded asynchronously, as soon as possible, or while the browser is idle, when the user is interacting with the current page, or when a subsequent navigation or other type of UI interaction occurs.
However, such techniques, often considered a “silver bullet” to this problem, require significant development effort and bring their own sets of risks and challenges, and thus have purposely not yet been implemented into Gandi’s app framework.
Instead, we have so far concentrated our efforts into reducing the amount of unnecessary code in our apps, the “hard way” by:
- reducing the number of and re-organizing dependencies in our apps, both first and third party
- replacing some dependencies with lighter alternatives
- rewriting unnecessarily heavy “legacy” code into lighter a form, enabled by the evolution of our tooling and core libraries such as React
- defining and enforcing best practices
There’s also another sub-problem to Single Page Applications loading the whole app code upfront that is worth mentioning here, which is that this code is usually and by default, loaded at once, from a single file, which prevents efficient caching by the browser when new versions of the app are rolled out in between visits of a given user. In 2019 we implemented the first iteration of a technique commonly called “bundle-splitting” which aims at improving scripts’ cache efficiency and consists in splitting the application’s scripts into multiple scripts that work together but can be cached independently. As each new version of the app rarely updates all scripts, previously generated scripts that have not changed since the last visit of a returning visitor can be used from their browser cache.
This can be achieved using Webpack’s SplitChunksPlugin, for which the relevant configuration may look like this (as is in our framework’s current configuration):
optimization: {
splitChunks: {
cacheGroups: {
[`react.ES${ES_TARGET}`]: {
test: /[\\/]node_modules[\\/](react|react-dom|scheduler|prop-types)[\\/]/,
name: `react.ES${ES_TARGET}`,
chunks: 'all',
enforce: true,
},
[`utils.ES${ES_TARGET}`]: {
test: /[\\/]node_modules[\\/](lodash|moment|intl)[\\/]/,
name: `utils.ES${ES_TARGET}`,
chunks: 'all',
enforce: true,
},
}
}
},
Which, along with the rest of our Webpack configuration results in the three JavaScript files we mentioned earlier:
- react.ES2017.[hash].bundle.js
- utils.ES2017.[hash].bundle.js
- client.ES2017.[hash].bundle.js
Note: this implementation remains the first iteration of this advanced technique in Gandi’s app framework, and while satisfactory to Gandi’s current needs and constraints, may not be suited to this exact configuration on other projects, it also brings some risks and drawbacks of its own, such as reduced compression efficiency, increase in number of HTTP requests, and more complex script loading mechanics.
Cross-browser compatibility
By nature, cross-browser compatibility is one of the core challenges of Web development.
While the websites implementing the latest of the Web Platform’s capabilities theorically benefit from those “for free”, without extra effort, users of older browsers are simply excluded from using such features, and sometimes, excluded from using the website itself, and, to prevent that, web developers have always vied for ingenuity.
Gandi’s app framework, as many web applications and JavaScript projects today, heavily relies on Babel, the JavaScript compiler in order to widely produce cross-browser compatible JavaScript code (ES5), from authored modern—but not necessarily standard—JavaScript code (ES2017++, JSX, Flow, ES Proposals, etc…).
See Gandi’s Babel preset on GitHub.
“Babel is a toolchain that is mainly used to convert ECMAScript 2015+ code into a backwards compatible version of JavaScript in current and older browsers or environments.
Here are the main things Babel can do for you:
- Transform syntax
- Polyfill features that are missing in your target environment (through @babel/polyfill)
…”
Now this is an exceptionally great tool that lifts a heavy burden off the back of Web developers, though it of course also brings its own set of challenges and requires some effort to configure and use inside a given project and even more so when it comes to performance, but these are already widely documented and not our primary topic of interest.
Except for one specific problem that could be expressed like this: the wider the gap between the “modernness” of a JavaScript project’s authored code and the “ancientness” of the oldest browser we want to support for the project, the bigger the amount of unnecessary or redundant polyfills and syntax transformations we are going to introduce to all our users, even if the majority of them use a modern browser, which supports most of the recent JavaScript features we use in our code. How do we prevent, or reduce, this “compatibility tax” from having to be paid by our users using modern browsers?
Inspired by research and articles by Philip Walton, Jake Archibald, and Andrea Giammarchi on this topic as well as the community’s resources and discussions, and leveraging Babel’s target.esmodules option, we were able to implement a working partial solution to this problem and use it in production in 2019.
Simply put, this solution consists of building two separate JavaScript bundles from our source code:
- one compiled down to ES5, including a large amount of polyfills and transformations, resulting in a widely compatible script for older browsers, though less efficient and heavier in size
- the other one compiled down to ES2017, including the least possible amount of polyfills & transformations, resulting in a script directly leveraging the newest JavaScript APIs available on modern browsers, as well as slightly more efficient and lighter in size
These two scripts are then declared into the HTML document, and using the appropriate script are loaded by the browser thanks to the <script type=”module” / nomodule > technique.
Depending on the app this upgrade of our app framework was applied to, it resulted in 5% to 20% lighter JavaScript bundles for modern browsers. The most significant reductions were obtained on apps that include the most app code compared to dependencies code, which, without getting into too much detail, is exactly why this solution is only currently implemented as a first iteration. In fact, we can expect to at least double these gains once we would have managed to apply this technique to our internal JavaScript libraries when used in Gandi’s apps, which will eventually happen but will require a bit more time.
The last reason this solution was mentionned as being “partial” is that today, there’s a significant part of our JavaScript bundles that consist of open source libraries distributed on npm, which are directly published in ES5 or earlier, so it can be consumed safely by older browsers and, while there are a lot of efforts, discussions, and experimentations going on around this topic in the JavaScript community, still lacks an effective and widely adopted solution as of today, though I have no doubt this time will eventually come.
Note: Again, this solution is an advanced technique that should not be implemented lightly since it comes with a variety of challenges, caveats, and risks, it also may not be suitable for all projects.
Conclusion
As the Web increasingly becomes the platform of choice, websites are getting bigger and JavaScript is eating the world, the websites, apps, and services we use in our daily lives are changing too, for various reasons, sometimes following trends, adopting popular technology, scaling up in complexity to catch up with users’ and businesses’ expectations. Also, with every new library, pattern, technology, architecture, and the like, aiming to solve a set of difficult problems comes a set of limitations, challenges, and risks that must be analyzed and taken into consideration to make the best trade-off for the project and for the users. In the context of websites and web applications, we’ve made efforts to introduce the concept of the JavaScript footprint through sharing a bit about the challenges we face and the approach we have building and maintaning websites for Gandi’s business.
From there, we invite you too to reflect on this concept in the comments section or on social media, whether you see it concretely as the amount of JavaScript required for a webpage to work, or maybe in a more abstract way as a whole, correlating benefits and drawbacks to user experience, engineering efforts, velocity, and overall complexity—that’s up to you!
Tim is a Lead Front-End Engineer who has worked on developing Gandi’s websites. You can follow him on Twitter @tpillard or his brand new tech blog.
