The Wrong Kind of Fast: Embracing Evidence-Based Architecture

Originally aired:

About the Session

"Move fast and break things" was Facebook's motto for years, valuing time-to-market over stability. "React is faster than the DOM" is one of the commonly repeated features of the framework, although this is demonstrably untrue and disavowed by Dan Abramov, top React committer and creator of Redux.

Of course we want software to be fast, but we should want it to be the right kind of fast. In this talk, Scott Davis (Web Architect and Developer Advocate, ThoughtWorks) debunks common software tropes with empirical, quantifiable, measurable facts -- what he calls "Evidence-Based Architecture". We'll look at making your web app fast by creating measurable performance budgets. We'll explore your development process through the prism of the book "Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations". And we'll create "objective integrity assessments" through measurable Fitness Functions, as discussed in the book "Building Evolutionary Architectures: Support Constant Change".


My name is Scott Davis. I'm going to be talking about the wrong kind of fast, embracing evidence-based architecture. I do want to point out that sometimes over video slides can be very small and I've got a lot of details here. So if you are interested, please feel free to head out to my website Thirstyhead.com. I'm pasting that URL in the chat window and you should see the wrong kind of fast, right there. If you click on a of links, you can pull up the slides that you're seeing on my screen right now. So why don't we begin?

As I said, my name is Scott Davis. I'm a principal engineer and a web architect with ThoughtWorks. There are lots of ThoughtWorkers here. You've probably been listening to Neal Ford already. A lot of my colleagues from Australia will be joining us as well. And we have, a booth here as well, a virtual booth. So I hope that you will enjoy all the talks from ThoughtWorkers here as we go. I'm used to actually standing on stage and giving this to you in Bangalore or Pune or Hyderabad or Chennai or places like that.

So, it is a treat doing this for you virtually. I'm sitting in slippers in my home in Denver, Colorado. So that is a bit of a change from what I am used to doing.

Now, it's a curious title, isn't it? The wrong kind of fast. How could there be a "wrong" kind of fast? Speed is good, right? Fast is good. Isn't it. We want to get up and running as quick as possible. We want to add new features as quickly as possible. We want to get to production as quickly as possible. So yes, fast is good. There's no question about that.

When we're dealing with a, a new web framework, like react, they recommend that we go in and type in something like create react app and this is the same in angular as well. If you, are familiar with angular, they encourage you to install the angular CLI the command line interface, and then type in ng new my app, and even in vue, they encourage you to install the vue CLI and then vue create myproject. So this is fantastic right? We get up and running. We typically have a sample web server. We have test frameworks in place. We have some kind of path to production, so what could possibly go wrong? Well, that's what we're here to talk about today. Spoiler alert, we'll discuss it over the next 60 minutes, but there are certainly benefits to these frameworks and I don't want to ignore the benefits, but unfortunately, these frameworks ignore the costs of using them.

So I am forced to, discuss what they don't discuss in their documentation and their promotional material. Because as Neal Ford often says, especially in his most recent book, Fundamentals of Software Architecture, he and his coauthor Mark Richards, who is also here today. They say that in Architecture, everything is a trade off. Everything is a trade off. And so what I want to do is discuss these trade offs without, saying you should never use frameworks again what I do want to, deal with is, is presenting you with evidence, because it's quite easy for us to fall into this idea of opinions that this is good, or this is bad, or this is better, or this is worse.

And so, in fact, what I want to do is present you with a number of facts, if you will. things like going up to the command line and looking at the file size on it, disc or running reports so we can see what the speed is and performance and security and accessibility. So this is what I mean by evidence-based architecture.

Efficiency Vs. Technical Debt

And at this point in the presentation, I've been lingering here for a while, because I do want to warn you. I've been giving this talk for years. You're getting a brand new version of it, but I've been giving this talk for years and many people come out at the end of the talk and feel like I've been picking on them personally. I can assure you I'm not picking on you personally. I can't see you. So I promise you, I'm not picking on you. but then people say, Oh, you're picking on my favorite framework. and that's a harder problem to solve right here. So perhaps you'll join me in looking at some of these facts. And if you do get angry, perhaps you can direct your anger in the proper direction. Duly noted? Are we ready to dive in? Okay.

We've been talking about this idea of create react app, and this is fantastic. It does set up an entire framework ready for you to be up and running. But here's some of my concerns. Here's some of the costs we have here. When I install it, I have 1,684 packages and then another 17 later on. So by typing create react app, I am left with 1700 packages from over 800 contributors. Now this isn't unique to react. If I go in and deal with the angular CLI what's amazing is once I install the CLI, it takes 260 packages from over 200 contributors just to implement the command line interface. And then after the fact, once I actually ng create new, I'm left with 2,500 packages on my system. And I find these packages by typing in npm list. If you haven't done this, we often type in npm install and npm update with little regard to what we've already installed or what we already have in our system. So if you type in npm list, it'll give you a large tree of everything that we have installed and you can see what I did is I pipe that through word count and I just asked for the number of lines since each dependency on its own line. This is how I get a rough count. Now certainly there'll be duplicate packages in here. This is what de duped means de duplicated. And so in this list, it means there is only one copy on your file system, even though it shows up in multiple places, but I can assure you, the order of magnitude is relatively unchanged by even focusing on de duped packages. I can assure you that there aren't two packages here that are repeated 2,580 times even when I use it.

This is the vue CLI. I am left with nearly 1,200 packages on my system from over 660 contributors. And I want to leave here. Not that I'm picking on vue. I told you I'm not picking on you or your framework of choice, but this is why I get concerned by 1200 packages, 1700 packages, 2,500 packages because so many of them end up being deprecated request has been deprecated, makedir p is no longer supported. corejs is no longer maintained and not recommended for usage due to the number of issues. Oh, and Oh, by the way, the author of this library is looking for a job as well. But this is the crux of the problem with this overwhelming number of packages. And we're going to give you more facts later on to let you know why this is so concerning, but all of these frameworks have been giving you stories about efficiency and efficiency is quite important. I'm not here to take a thing away from efficiency, but there is also real technical debt that comes along with these as well.

The technical debt that we have just begun to unpack, and we're going to discuss further throughout this presentation. So while efficiency is outstanding. Well, that is something we want to minimize. And in fact, we want to find a balance. Everything is a compromise as Neal Ford and Mark Richard say, we want to find that compromise that balance. And that balance is business value, where we have enough efficiency with as little technical debt as we can to feel like we've made the right decision.

So dependencies. And in fact, indirect dependencies because when you have a dependency and, that dependency has dependencies and that dependency has a dependency, has a dependency on top of that can be the source of these hidden vulnerabilities. Now in the effort of looking at facts, I want to point you to a firm called Snyk. Snyk releases a state of open source security, in 2020, and in this report, they found that 86% of JavaScript vulnerabilities occur in indirect dependencies. Now stop for a moment to think about that. If you have 1700 dependencies or 2,500 dependencies, and this is the source of nearly 90%, your security vulnerabilities, that's a concern. That's problematic. This is what I mean by these trade offs. We're so quick to get started, but we don't consider the tech debt or the drawbacks or the real concerns of getting up and running so quickly. Adding 90% of your bugs to your platform before you write your single line of code. And I know I keep coming back to these numbers, 1200 1700 2,500.

And the real reason I do is because in another report by Veracode, they've pointed out that the average JavaScript application is 377 dependencies. That's still an astonishing, large number to me. I don't want to manage nearly 400 dependencies. But all of those frameworks we discussed here are quite literally an order of magnitude larger than what we're dealing with here. The best case scenarios it has four times as many dependencies as the average program in the worst case scenario, as far worse than that. And if you're sitting back to yourself and saying, well, Scott, that's just the way things are. I know you haven't been in the industry long, but that's how no JS and web development works these days. Been in the industry for over 30 years. Thank you. And it's not the way things have to be. It might be the way things are in these given. Frameworks, but there are also other frameworks we can look at that don't have those same problems.

Happy with hapi

I've left, hapi up here for a while now, hapi as a web framework, as well as a web server and a web framework. And this is the view from npmjs.org. So you can go into npmjs.org and do this kind of analysis on any of the prep packages we're dealing with. But here's, what's interesting, hapi as 18 dependencies. And what do you notice about these dependencies? beyond the, the fact that they have one of their dependencies is joy. Because of course, if you have a package called hapi, you need to have a corresponding package called joy. That's just common sense. Now, what I want you to pay attention to here is that hapi's dependencies are all hapi dependencies hapi's dependencies are all based on their own coding. And if we go do hapi's homepage right now, we'll see that hapi isn't a toy project. In fact, it was developed by Walmart and it was released on black Friday, the most busy, popular, overwhelming shopping day on the internet. If you do a web search on Walmart, nodejs black Friday, you'll see a number of case studies around this. And while that might sound foolhardy, it was in fact, a bold statement saying, we trust this. We have built this to the scale where we know it can handle even black friday levels of traffic.

Now on their homepage, we see a praise quote from Brendan Eich, who is the creator of JavaScript that should give us some confidence in the enterprise grade solution we're talking about, but here's what sold me. When you npm install hapi, every single line of code you get every single line. Has been verified. You never have to worry about some deep dependency or some sketchy dependency. That's powerful. So we're going to talk about a number of different kinds of fast, the right kinds or the wrong kinds. This is our first wrong kind of fast that I'd like to propose to you. That unmanaged dependency bloat is a solvable problem. You can actively explore your dependencies. You can actively manage your dependencies. And I told you, I've been giving this talk for years.

Reduce Dependencies; Reduce Vulnerabilities

Many people said, wow, Scott, you're normally such a positive fella. Why are you only talking about the wrong kind of fast? And so in this new 2020 edition, I do want to talk about the right kind of fast as well. And as we've seen that, there's almost a one to one correlation that as you reduce your dependencies, you reduce your vulnerabilities. If 90% of your vulnerabilities come from someone else's code. It's common sense to produce the someone else's code that you depend on. All right. So that's one kind of wrong kind of fast.

Let's go back to our friends at Synk once again. because when we're talking about evidence based architecture, what I really like is their studies are based on actual analysis. We can see in this study, they analyze JavaScript code across 133,000 different websites. And that's the beauty of open source. If these websites are on github we're on a gitlab or any of these open source projects on these open source repository, they can go in and look at the source code that's being used, but they can also analyze the living, breathing, running websites for these kinds of security vulnerabilities. Now what's interesting about this report. Is, they said the medium site, they analyzed use the library that is over three years old. Over three years old. Now let me pause for a moment here and make sure it might've been a while since you've had your math classes back in school. And so I'm going to talk a lot about average and I'm going to talk a lot about median as well.

I think it's important that we really understand if I have one website that has one security vulnerability. And I have another website that has a hundred security vulnerabilities. If I average the two together, I get 50 security vulnerabilities. And unfortunately that dramatically overstates the number of vulnerabilities in one website and dramatically under reports, the number of vulnerabilities on the other so many times, we'll talk about median and median is just what it sounds like. The median in the middle of the road are truly the middle of the road. So median means that at exactly half of the websites in their study, exactly. Half of the 133,000 websites that they analyzed used a framework that's over three years old. That means that on the good side, half of a 133,000 of those websites use the library that's newer than three years old, but what's the downside to this? Yes. Half of the websites they looked at use libraries that are older than three years old. That's incredible. That's like me walking into the Apple store with a windows 95 computer saying I can't get Safari to run on this and ignoring the idea that Apple wouldn't support Microsoft operating systems to begin with that gives you a rough order of magnitude of how grossly out of date using three year old libraries are in modern web systems. From the same report, they say from a remediation perspective, the picture painted by our data is bleak. We observed that only a very small fraction of these potentially vulnerable websites, literally one and a half percent of the dot coms and their surveys could become free of vulnerabilities by applying patch level updates.

And remember semantic versioning means patch level are these last digits in here? This three to four, those are specifically for bug fixes. The middle dot is for minor upgrades and that's for adding new features in a non-breaking way and major updates from 1.0 to 2.0 of course signifies brand new breaking changes, so if only one and a half percent of the websites in their surveys could even do a patch upgrade to fix these vulnerabilities. We're in a bad state of affairs. We're in a bad state of affairs. I don't know if you remember the Equifax security breach from several years back at the time, it was one of the largest breaches of consumer private financial data at the time, over 143 million consumers.

Equifax was using Struts 2, oh, wait for you to get all the Struts 2 jokes out of your mind right now. but Struts 2, in fact, had recognized the security vulnerability and patched it. And Equifax had that patch. They knew about the upgrades and the folks in charge, the C level folks said, we knew about the patch. But we could not upgrade. We are so far behind that we couldn't apply the security patch needed to close this breach.

And I know we're talking about operating systems. I know we're talking about programming languages and web frameworks and yes, even Docker has this concern. Everyone thinks that Docker is such a wonderful way to get up and running and it is it's incredibly fast. Isn't it to reach out to a Docker container and start using it. When all of a sudden though, if the top 10 Docker containers each contain at least 30 vulnerabilities and notice that's a boon too. If we look at Redis or memcache D or Couchbase, that's nearly 50 vulnerabilities. If we look at my sequel or Mongo or httpd, that's nearly 70 vulnerabilities. If we look at Nginx, a web server and Postgres, that's close to 90 and there's nodejs. I love nodejs.

So I am not here criticizing nodejs, but the popular nodejs Docker container has nearly 600 security vulnerabilities. This is a very real problem and a hidden problem if we focus on how quickly we can add these to our code base, without considering the potential security vulnerabilities that come along with them, or even doing the simplest possible audit of these things.

So now we've discovered our second kind of wrong kind of fast. We've talked about this unmanaged dependency bloat. And now when we talk about grossly. Out of date dependencies. And I'm sorry, when we are multiple years behind, when we are potentially multiple major versions behind, there's no other way to describe that than grossly out of date. There is no other way to describe that. If you are committing to using a framework, you also have to commit to update it every single time. I've done this on a number of my projects and yes, we all grown. We say, Oh, but I want to add this new feature. I want to quickly add as many new features as I can, but it is professionally irresponsible to ship out of date software with potential security flaws in them. And as we learned with Equifax, it's not just professionally irresponsible. It's criminally negligent.

Wow, this is a fun presentation. Isn't it? this is exactly what you wanted to hear first thing in the morning, isn't it? It'll get better. I promise you, there are some solutions I'm going to offer you as well. I'm not just going to here and make you cry onto your laptop. As you watch this. But there'll be a little bit more crying.

And this is where people do feel triggered here as well. Oh no, here we go. Scott's calling out react. If you've heard this myth "React is faster than DOM". Someone no less than Dan Abramov, who is the creator of the Redux framework and a core contributor to the react framework says in the comments, no, we don't claim that in the react document react can't possibly be faster than the same DOM mutations written by hand because by definition react has more work to do. That's actually a very reasonable response. I was really pleased with that. Dan does seem to be like a reasonable fellow when, when, when it comes to these kinds of things. He goes on to say something else that I feel is very reasonable as well. We do think it helps to create maintainable applications and we feel that it is fast enough for practical use cases. Fast enough. That's something we all feel right. Is this fast enough? Yeah, it feels fast enough, of course. But when I'm talking about evidence-based architecture, we can't just say this feels fast enough or it doesn't feel fast enough or can you make it faster? That's not evidence that's aspirational. So what we want is to try to apply real engineering rigor to this.

So what does fast enough feel like. If we actually measure performance, and of course I'm not talking about having a stopwatch and watching someone use this, I'm talking about the built-in instrumentation that we're going to learn about that is baked into every Google Chrome browser and this set of tools that we can npm install and make a part of our cd pipeline and everything else.

But Netflix removed client side reacts now, not from their entire website, not from every page. This was specifically on their landing page, but when they removed react, it resulted in a 50% performance boost. Now you can see in their slide, this is a bit meta, isn't it. You're watching a presentation of someone giving a presentation of someone talking about someone else, giving a presentation, but in the presentation you see on the screen, not my presentation. You see, we saw our 50% reduction, our time to interactive or TTI. And this is something else that we're going to talk about quite a bit, because this is measurable and time to interactive is probably what you think it already is. When you go to a website, how long do you have to wait before it's actually usable? We have different metrics we can track first contentful paint means the first time you see some kind of pixel on your screen, but TTI is when is this website usable?

I didn't mention this, but every one of these slides, I'd say you see on my presentation is now a link back to the original source. So every one of these articles that I cite, every one of these articles that I present to you, you can click through and read on them. And I encourage you to watch this video because Netflix was very thoughtful as they're doing this. And they explain the engineering rigor behind this. And of course doing a web search on "Netflix removes reactjs"will give you a whole host of case studies as well, but this is a trend we're beginning to see over and over again. Github removed Jquery, Netflix removed react, and what's really interesting is github said instead of replacing this with another library or framework, we were simply able to achieve everything we needed using standards based browser native framework free code. That's exciting. That's exciting that this is the era we're living in. So when we talk about performance, I am very insistent that we don't talk about it in an aspirational terms, but instead we talk about it in measurable terms and we must have targets that we want to achieve. If you have a personal budget at home, you don't say, huh?

Performance Budget

I just hope I'll spend less next month. You say, no, this is what I spent on groceries this month. This is my target. And then every month I'll see how close I get to my target. Or if I go over my target, why that was. So before I walk into any client, before I know anything about their app, this is the performance budget I propose to them. Your page weight needs to be other one megabyte and your TTI needs to be under five seconds. And I talk about page weight and TTI, because these things are hard to teach. Because one megabyte takes five seconds to download on a typical 3G network. And a typical 3G network is what about 75% of the planet is on right now.

So we can see page weight and TTI are really deeply linked just by the physics of networking at this point. If this kind of conversation interests you and I hope it does because we got stuck on another 30 minutes. You're going to be dead dreadfully bored. If this isn't interesting to you, but this article here, the cost of JavaScript by Addy Osmani. He works for Google. This is an incredibly insightful article and here's the TLDR right here. Here's the executive summary bite for bite javaScript is still the most expensive resource that we send to mobile phones today. Now, I've got a couple of more things I want to present to you in this report, but Addy is someone else who says, Hey, you got to get interactive in five seconds. This is just table stakes. This is what we have to strive for. So as we begin hearing this five second TTI show up from Google and from a variety of different sources, we start saying, well, there. There might be something to this, but rather than just saying, you've got to get this, Addy goes one step further. He says, here's the actual cost of JavaScript when we begin looking at a website like cnn.com. Now there's a lot to unpack here right now. So let's start along the bottom. This is the time to interactive and it's measured in milliseconds. So this first, vertical bar here is the ten second mark. Next is 20 seconds. 30 seconds. Yeah, we do have to go up to 40 seconds in this case. That's brightening in and of itself. And if our goal is five second TTI, you'll notice these bottom two values here, a desktop and a think pad over here. These are the only two browsers in this chart that come close to hitting that five second TTI. Everything else above that is a mobile phone. And with the exception of the iPhone 8, which was the high end model iPhone. At the time he wrote this in 2018 with the exception of the iPhone 8, every other phone takes at least 10 seconds or more to render that same page. That's really something isn't it.

Would you still stick around for 22 seconds or 42 seconds to wait for a webpage to render? Doubtful doubtful at best, but here's what I think is most interesting about this, these yellow bars in here. This is the JavaScript. And again, this isn't a guess. This is based on the analytics that you're going to see through the rest of this presentation. So the yellow bars are JavaScript and think about CNN for a moment. It's a new site. CNN is text and images. And yes, video as well. But what's interesting is our phones here all have dedicated chips for dealing with video. This is how you get eight hours of battery life watching videos all day on YouTube or Netflix or anything else.

So ironically, what you might think would be a slow down a video in fact is a no up here, but JavaScript is measurably the long pole in the tent here. And the reason I say this is if we removed all the yellow in this chart, and we were left with HTML and CSS, every single one of these devices would come in at five seconds or just slightly more four seconds. Four and a half seconds, five and some change, eight seconds, five seconds, four seconds. Even the slowest phone in the list would come in at six seconds. This is evidence-based architecture. This is how we can measure the true cost of JavaScript. As we send it down to smartphones. And the reason why I think this is so important, as someone said in the chat eye opening here as well is because this is where we do all of our development.

So you're not a bad developer. I promise you. You're not a bad developer for using frameworks or for shipping these JavaScript. If you haven't come to my talk yet, if you do this after the fact, then you are a bad developer. All right, let's get that clear. Oh, that's a joke. Of course. the reason why you're not a bad developer is that this is your experience. And as you're testing it and developing this feels fast and you probably don't pull out your phone and test these things regular, you probably don't experience these kinds of things, which is where analytics and reports and best practices can come in to give you the education and give you the eye-opening idea that wow, there is a very real cost. Does shipping that much JavaScript across the wire. Because median webpage there's median once again, right, half of the web pages in this study, and this study, by the way, is based on the HTTP archive, you can go out to the httparchive.org. You can find these datasets, they have roughly 6 million mobile websites and 6 million desktop websites. and so the middle webpage, the 50th percentile ship's about 350 kilobytes of JavaScript. Now that's compressed in this very hard to read text down here. You can see that equates to greater than one megabyte, uncompressed, and remember what our performance budget is a mega JavaScript or a mega page weight, I should say, and five seconds.

So if we've got to make a JavaScript here, how come we've got a 15 seconds. We saw why didn't we? We saw why there's a 15 second TTI. And here's another way of looking at this. Now let's focus on JavaScript once again, let's look at the 50th percentile. And before you get concerned that here, we're looking at 420 kilobytes. And just before we said 350 kilobytes. Addys report was based on the July, 2018 HTTP archive, Katie's report here is based on the October 2018 data. So while these are off a little bit, they're the still same, the order of magnitude. They're still on the right neighborhood. So by 50% of the website shipping 420 kilobytes of JavaScript or less, and then we look at the 75th percentile and saying 75% of the websites ship 780 kilobytes or less. And even at the 90th percentile, when I am asking you to have a page weight of one megabyte or less, I'm really asking you just to be better than 50% of the website out there right now.

And you can't get to this if you are shipping 1.2 megabytes of JavaScript, but wait, Oh my goodness. Oh, my goodness. Facebook ships, seven megabytes of JavaScript, LinkedIn ships, seven megabytes, LinkedIn, even ships six megabytes to their mobile platforms. This is unbelievable. This is an unconscionable amount of JavaScript to ship. If you have even the vaguest concern about performance and the user experience. This is inexcusable, but when we're talking about percentages and medians and percentiles, it can be really hard to put these things together.

A Tweet on Margaret Hamilton

So I love this tweet when I first saw it. Now, most of this tweet is true. There's one important line. We'll get to that in a moment, but what is true is this is Margaret Hamilton. What is true is she was a software engineer. And as a matter of fact, if you call yourself a software engineer, it's because Margaret Hamilton was the first person in history to call herself that and call others that as well. What you're seeing here in fact is her source code that she wrote to put the first men on the moon to fly the first rocket from the earth to the moon. So this is the source code in fact, that flew men to the moon, not as the tweet implies next to a printout of her node modules directory for her first, hello world react app.

I think that's hilarious, but I don't get out enough, but this is what I think is even funnier. Gary went on to say, huh? What if we actually did print the dependencies of a hello world, create react app. How tall would that stack be? So he does a little simple, he's like, all right, well, I arbitrarily choose this font. And so that gives me roughly 50 lines of code per sheet. I type in Tokei node modules. We're going to talk about Tokei in just a minute here, but I type in Tokei node modules. And that gives me about 2.6, 2.7 million. Lines of code in my node modules directory. If a sheet of copy is about 0.1 millimeters thick multiplying sheet by thickness, we get roughly 18 feet. So congratulations. This is in fact, Margaret Hamilton standing next to her create react app hello world and fun fact. Margaret Hamilton is 18 feet tall. Okay. That's the lie. Margaret Hamilton in reality is about five feet tall. And later on in the comments on this Twitter thread, some helpful developer pointed out that, you know what, Gary, your calculations assume that all the source code was printed out single sided.

You know, if you print it out double-sided then that means Margaret Hamilton is only nine feet tall. Okay. Well, nice try. I guess. This is what I mean. This is an unconscionable amount of JavaScript for us to apply to a hello world app. It's unbelievable.

Tokei and Lighthouse

And if you want to prove this for yourself and you should, this is the whole crux of evidence-based architecture. You should npm install Tokei. You should run Tokei on your node modules just to make sure this wasn't a Twitter lie. And I wanted to make sure the facts bore out. When I did a create react app and I ran Tokei on that in my local system. Wow. I got 2.9 million lines of code. I got nearly 2.2 million lines of JavaScript. I get almost 127,000 lines of TypeScript. I get 228,000 lines of JSON.

These numbers are what I would expect from 1700 dependencies from over 600 contributors, but that doesn't make it right. That doesn't make it right. So, how do we solve if you haven't heard of Lighthouse? I'm so glad you're here. This is one of my absolute favorite tools. This is my crystal ball. This is what allows me to practice evidence based architecture, because if you have Google Chrome and chances are you do, it has about two thirds of the market right now. If you have Google Chrome, you have Lighthouse simply go into your Chrome dev tools. And click on the audit tab in the very recent version of Chrome that is now called Lighthouse, so it's even easier to find. So you can run these ad hoc reports against any URL, your URL, someone else's URL, your competitors, your home website, whatever you want. But here's, what's fantastic about it. Not only can you run ad hoc reports in your browser, you can also run Lighthouse from the command line, which means you can add it to your continuous delivery pipeline or even your build process so say you kick off by hand.

So, what does this look like? If you don't have Chrome installed, you got to install it. And once you have it installed, you go to the URL of your choice. This can even be local host. If you would like this can be your prod environment or your test environment or your integration environment or QA environment or any environment you want, but you plug in a URL, you'd go to dev tools, you click on the audit tab in about a minute. Probably. Yes. You end up with this lovely little report card, letting you know. On a scale from one to 100, how well you did in these various categories and what's even most, impressive about this is they give you very specific, actionable advice. So as we drill in, they say hey, you've got some unoptimized images, it could save you five kilobytes and roughly one third of a second. And that's fantastic. Thank you. I've been looking to shave a third of a second off my TTI. Thanks for your help Lighthouse. what I typically do is end up looking for, Oh my goodness. Okay. Three and a half seconds on one request. Okay.

Now I've got something to look into and I can open this up and do some more detailed analysis on these things. And if you like the idea of performance budgets, and I hope you do at this point, and I want you to add lighthouse to your CD pipeline. It all begins with an npm install Lighthouse. And then of course, this article that you can click through to gives you the following steps that you would take.

So What's the Wrong Kind of Fast? Outrageously large page weight and TTI. And you can see that I'm using words like bloat and grossly and outrageously, because if it takes an 18 foot stack of JavaScript for a hello world, that's what I call outrageously large. And we know that page weight and TTI are deeply related. So the more JavaScript you add, not only are you adding more out of date, dependencies and security flaws, you are directly impacting your TTI as well. So what is the right kind of fast look like? Performance budgets, as we said, and here's where you should start page weight under a megabyte. TTI under five seconds.

And then after you start running these Lighthouse reports and you see what your normal is standardized on that, give yourself goals. If your TTI right now is 15 seconds, I've got a great goal for you chip away at that. See what you can do to get to 10 seconds, see what you can do to eight seconds. See what you can get to five seconds, have measurable, specific goals that you want to achieve.

All right. We have two more wrong kind of fast to get into. But at this point, I think you get the feeling of what this presentation is all about. Many people say that SSR our Server Side Rendering, we've been having such performance problems with client side react. If we could lean to the server side, this will fix our problems.

Do you think this is going to fix your problems? No. But what I really like about this article is it's written by a fan it's written by someone says, I like react. So this isn't a hit job. This is someone who wants react to be successful in this scenario. But the author does recognize that there's a lot of overhead in just a single page view.

So what the author did is this, he created a simple, nodejs app, and he's providing you a link with a source right there. He says, all right, we're going to render a simple HTML table with a hundred rows of employee data from a JSON file. All right. And we're going to use ES6 templates. That's ECMAScript 6, that's just JavaScript, that is standards based browser native code. We're going to use nunjucks and pug, these are two popular server side rendering libraries. They don't have anything to do with JavaScript or react or anything else. And then he also brings in react and SSR. And this is why I love this article so much.

He says for good measure, I threw in a control. If we are computer scientists, you can't experiment without a control. And so his control was static HTML. If I have static HTML and then four different templating solutions, we should be able to get some really good insight out of this. This is one of those charts where the numbers are incredibly small, especially if you're watching this on an iPhone or anything else.

So let me show you that we're looking for throughput. So request per second are here along the left. This is a thousand requests per second, two thousand three thousand four thousand requests per second. Along the bottom we have one concurrent server or five concurrent servers or 50 or a hundred or 250. So throughput is good. High bars are good. So what are these blue high bars here that are coming consistently close to getting 4,000 requests per second on five concurrent, 50, 102 hundred 50 fairly consistent across those concurrency levels. That pale blue bar is the static HTML website. That's the number to beat.

Now let's do the opposite. Let's look all the way down to these green bars at the bottom that are very consistently reporting, about 300 requests per second. What is that green bar? That's react SSR. Orders of magnitude difference once again. Now to the author's credit he says, you know what? I realized that I was running react SSR in dev mode, and that has all the logging and all the debug output and everything. I should probably run this in prod mode and I am thrilled to see the bump we see moving from dev mode to prod mode. We now have pretty consistently about 1600 requests per second. When we run SSR in production mode. That's still less than half of the throughput we see from static HTML, and it is less than any other framework in this test suite.

Now if we look at this in a different way, if we look at response times instead of throughput requests per second, all of a sudden now we see that low bars are good and high bars are bad. Our response wait is two tenth of a second, four tenths of a second, six tenths, eight tenths of a second 800 milliseconds. And then we have the same one concurrent five 50, 102 hundred 50 concurrent units. What's this large green bar that's leaping out at us? That is react SSR in dev mode. What's the shortest bar on the chart that we're seeing static, HTML. And again, I love seeing this dramatic drop between react SSR in dev mode and react SSR in prod mode. It takes react SSR in prod mode down to a 150 milliseconds per request. That's respectable. It's not nearly as fast as the 32nd response. Excuse me. The 30 millisecond, the one third of a second response time that we see here, not even one third. Excuse me. Yeah. zero point. Zero three. Math is hard.

You get the idea right. Quite lower. Wow. That was a credibility hit. Wasn't there. I'm talking about math and simple arithmetic leaves me. I apologize. It'll only get worse when I give my next presentation at four in the morning, Denver time. So tune in for that one as well. It should be a treat, but we began seeing here, going back to what Dan Abramov said, I would grant him that this might very well be good enough.

This very well might be good enough. This seems in line with everything else we're seeing. It is still slower than every other framework we measure. But if this is good enough, then it's good enough. So these numbers are trade offs every time. And if you want do this kind of exercise, create a simple test.

The number one thing I encourage you to do is not do your experimentation on production code. That's crazy talk, isn't it create tiny little tests and it's so strange because developers, I talk to, they're afraid to do anything outside of their framework and outside of their code base. They're afraid to create tiny little prototypes that you can use to make sure you understand the problem you're trying to solve to make sure that solution you've come up with, or the prototype is performant or that it is secure enough or any of these kinds of things. We can't be afraid to do experiments. Like we just witnessed here. We can't be afraid to do experiments outside of the framework we're using in production, because how do we measure the framework we're using in production if we don't have a control, if we don't have other candidates to judge it against. So, if you want to set up a simple website in angular and vue and react, and then go to web.dev, you don't even need to get into your control panel. You don't even need to npm install anything. If you go to web.dev you type in a URL, you run the audit and you see the results we get.

Here is the audit I ran just today. Over lunch. cnn.com came in at an outstanding 48 second TTI. What's especially nice about this is web.dev runs this out on the cloud. It runs it somewhere else. So some people get concerned that am I gauging my whole network speeds or my company network speeds or my laptop speeds or anything else like this. So this is another way where you can go out in the cloud and just kind of gut check and see if these numbers make sense. So what's the wrong kind of fast, only experimenting with production code.

And what's the right kind of fast? Experimenting, like the computer scientist you are or who you claim you are.

Yeah. All right. We have one more wrong kind of fast to talk about here. And this is accessibility because once again, we're talking about evidence-based architecture. And so many times with accessibility, we say, ah, I don't know anything about accessibility. I can see. So I don't know what a low vision or blind experience is. I can hear, so I don't know what a low hearing or deaf experience is. We have evidence that we can bring to this party. This is not a replacement for having people with actual disabilities, evaluate your website. That is priceless, but this is a good first step. This is a good first step.

The WebAIM Million once again designed an accessibility analysis of the top 1 million homepages. And what did they find? Out of a million homepages, there were 60 million distinct accessibility errors, an average of 60 errors per page. That works out to be roughly one out of every dozen elements on the page has some kind of measurable accessibility error. And in this note they say it's probably worse because when we come across a div or a span in a website and remember divs and spans are semantically, no, they mean nothing. A paragraph is a paragraph in images, an image, a link is a link a table is a table, but a div is a nothing. A span is a nothing. These are the potato chips of web development. These are the crisps of web development. They are semantically, nutritionally void. And as you can see, we are dying of malnutrition here.

98% of the webpages surveyed had accessibility errors and these were not weird edge case esoteric errors, low color contrast on 85% of the homepages missing alt text on an image on two thirds of the web pages. Over half of the web pages had empty links. And forms without form labels on them. This is HTML one O one. This is not a hard problem to solve. And you might be thinking yourself, well, thank goodness I have my framework to help me out with this. You have, you know, good paying attention. Why would you say that at this point in the talk we are almost done and of course if you're using react on average, they detected 10% more accessibility errors on them pages not using react.

If you're using angular, they detected a 20% increase in accessibility errors. If you're using vue, they detected an almost 25% increase in accessibility errors. And they do go on to say this doesn't necessarily mean that the frameworks caused these errors. But what it does say is mathematically empirically we have evidence that webpages that use react on average have 25% more accessibility errors, and they did this across a number of libraries. Modernizr is a library I used to love. If you have Modernizr and 8% increase in accessibility errors. If you use Lodash, a framework, I love you have on average, 15% more accessibility errors.

If you're using Underscore 26 more errors, 26% more even jQuery on average is over 40% more accessibility errors. These libraries aren't causing the errors, but statistically there is a correlation between websites that use these frameworks and an increase in accessibility errors. If you use something like bootstrap, I used to love me some bootstrap. That's gonna buy you a 10% increase in accessibility errors. Homepages that used bootstrap had 1.3 million more accessibility errors than pages that didn't utilize bootstrap. And boy, if you have advertising on a web page, look out, look out. It is perhaps the number one source of accessibility errors we can find them third party code. How do we fix this? It's our friend Lighthouse. Once again, we've talked about performance. We've talked about TTI. Accessibility is number one of these numbers here.

So if I go to cnn.com and I run a Lighthouse report and I look at accessibility, look at the problems they have. Background and foreground color do not have sufficient contrast ratio, image elements do not have an alt tag form elements do not have a label, links don't have a discernible name, all the things that we called out in that WebAIM One million survey.

So what's the wrong kind of fast expressing, expecting frameworks to solve your accessibility bugs. Or for that matter, expecting frameworks to solve any of your problems, expecting frameworks to solve your performance problems or your security problems, or certainly your accessibility problems. You simply can't outsource these to frameworks.

We are living in an era right now of unearned trust in these frameworks, right? It doesn't mean these frameworks are bad. But what it does mean is you can't outsource your responsibility for security, for performance, for accessibility, for these frameworks, because they are not helping you. In fact, they are actively actively hurting you. So what is the right kind of fast? Understanding these elements, understanding privacy, understanding security, understanding performance, understanding accessibility, all the things we talked about right here that is the right kind of fast, understanding them and owning them.

Hopefully on a positive note, you now know what to look for. I call these things apps. Cause everyone says I'm not building web pages anymore. Scott I'm building web apps. Fantastic. If you're building web apps then what you need to be paying attention to its accessibility, its performance, its privacy, its security, not how quickly you can build a Twitter clone. Not how quickly you can build it to do app, not how quickly you can get up and running and how quickly you can add features and how quickly you can get into production. And all the things that I've shown you here at this point, they're called fitness functions. Because we all know what unit tests are. Unit tests are testing at the lowest possible level. They're testing at the micro level. They're testing the bricks of your building and fitness functions are in fact, testing the building.

Fitness Functions for APPS

If you want to read more about fitness functions, building evolutionary architectures is where I was first introduced to them. They're written by Rebecca Parsons, a peer of mine, a CTO at ThoughtWorks. Neal Ford, who you're listening to here and Patrick Kua, they wrote this and I want to leave you with this. This is one of the best definitions I've found a fitness functions. Once architects have chosen important characteristics, whether it's accessibility or performance or privacy or security. These architects want to guide those changes to protect those characteristics. So any tests we have to preserve these characteristics, to protect these characteristics, to ensure we're hitting our intended goals. These are simply called fitness functions.

So when you see a performance budget like this, when I see product pages must ship with less than 160 kilobytes of JavaScript on mobile that is specific and measurable and actionable. When they say our search page must include less than two megabytes of images on a desktop. That is something, yeah. We can track through Lighthouse to web.devs, and number of different tools.

If our homepage must load and get interactive TTI in less than five seconds on a slow 3G network, wherever you heard that before. If your blog must be greater than 80% of lighthouse for performance greater than 80 greater than 80% on accessibility, greater than 80% on any of these things that are important to us, this is a very real fitness function.

This is a very real performance budget from the Guardian's desktop homesite right here, but this is evidence based architecture in action. So when people say we've got to go faster, we're not going fast enough. How can we get install this framework more quickly? How can we get features in place more quickly? How can we get to production more quickly? I want you to say these are all important goals. Efficiency is nothing we should be ignoring in these, but if you focus only on efficiency, And you ignore the very real cost of that efficiency, the very real technical debt that we introduced in terms of, out of date dependencies in terms of sheer number of dependencies, in terms of page weight and TTI, and our lack of accessibility is optimizing efficiency, really giving you the business, this value that you hope it will?

That my friends is the wrong kind of fast.

I certainly hope you enjoyed yourself. It has been such a treat talking to you right here. My only regret that as I'm in Denver right now and not seeing you face to face, I hope you enjoy the rest of this conference.

I hope you are enjoying every speaker that you hear as you move forward. Many of these speakers, my friend, I hope you hear Brian Sletten I hope you hear Mark Richards. I hope you hear Neal Ford. I hope you listen and respect and pay attention to all the speakers here. I'll be talking to you again at the end of the conference about design thinking.

I hope to see you there. If I don't please enjoy the rest of the time of your conference. And thank you for sharing this time with me today. I really do appreciate it. Thank you very much. .

See Highlights of

Hear What Attendees Say

“Once again Wurreka has knocked it out of the park with interesting speakers, engaging content and challenging ideas. No jetlag fog at all, which counts for how interesting the whole thing was."

Cybersecurity Lead, PwC

“Very much looking forward to next year. I will be keeping my eye out for the date so I can make sure I lock it in my calendar"

Software Engineering Specialist, Intuit

“Best conference I have ever been to with lots of insights and information on next generation technologies and those that are the need of the hour."

Software Architect, GroupOn

Hear What Speakers & Sponsors Say

“Happy to meet everyone who came from near and far. Glad to know you've discovered some great lessons here, and glad you joined us for all the discoveries great and small."

Scott Davis, Web Architect & Principal Engineer, ThoughtWorks

“What a buzz! The events have been instrumental in bringing the whole software community together. There has been something for everyone from developers to architects to business to vendors. Thanks everyone!"

Voltaire Yap, Global Events Manager, Oracle Corp.

“Wonderful set of conferences, well organized, fantastic speakers, and an amazingly interactive set of audience. Thanks for having me at the events!"

Dr. Venkat Subramaniam, Founder - Agile Developer Inc.