+ - 0:00:00
Notes for current slide
Notes for next slide

Webpack deep dive

Johannes Ewald
Peerigon GmbH
@jhnnns

1 / 123

2 / 123

3 / 123

4 / 123

1. What is webpack?

5 / 123

1. What is webpack?

2. How does it work?

6 / 123

1. What is webpack?

2. How does it work?

3. How do you get the most out of it?

7 / 123

What is webpack?

8 / 123

Webpack is just a module bundler which
turns a dependency graph into
an optimized chunk graph.

;)

9 / 123

Webpack is just a module bundler which
turns a dependency graph into
an optimized chunk graph.

;)

10 / 123

Webpack is just a module bundler which
turns a dependency graph into
an optimized chunk graph.

;)

11 / 123

Webpack is just a module bundler which
turns a dependency graph into
an optimized chunk graph.

;)

12 / 123

Webpack is just a module bundler which
turns a dependency graph into
an optimized chunk graph.

;)

13 / 123

Webpack is just a module bundler

14 / 123

Without a module bundler...

// message.js
const message = "Hello World";
printMessage();
// printMessage.js
function printMessage() {
  console.log(message);
}
  • What if message.js is included before printMessage.js?
  • What if multiple files use the variable name message?
15 / 123

Without a module bundler...

  • the <script> order is important
  • top-level variables and functions are global

As your application grows, this becomes a problem.

16 / 123

And there's an additional problem:

All module systems are slow to initialize when a lot of modules need to be loaded.

17 / 123

This is due to the roundtrip problem:

18 / 123

Module bundlers to the rescue!

19 / 123

Module bundlers...

...analyze JavaScript modules by reading their source code...
...to generate compatible and efficient script JavaScript files (bundle).

20 / 123

Module bundlers leverage the fact that all
module systems share the same mental model:

  • A module has a unique id
  • Modules reference other modules
    as dependency via specifiers
  • A module specifier can be resolved to a module id
21 / 123

This model is called the dependency graph.

22 / 123

A dependency graph can be simple...

23 / 123

...or complex...

24 / 123

...or very complex.

25 / 123

Module bundlers...

...try to gather as much information as possible about the dependency graph in order to apply clever optimizations.

26 / 123

Module bundlers...

...try to do as much as possible on build time...
...so that there's less to do on runtime.

27 / 123

Comparison of JavaScript module systems

  AMD CommonJS ESM
Analyzable With limitations With limitations Yes
Resolving Asynchronous Synchronous Implementation specific
Linking Copy by value Copy by value Live binding
Scoping Function wrapper Function wrapper New language environment
28 / 123

Interoperability is a mess.

@bradleymeck implementing ESMs for Node.js

29 / 123

Webpack is just a module bundler which
turns a dependency graph into
an optimized chunk graph.

30 / 123

How does webpack work?

31 / 123

The best way to understand webpack is to follow the steps webpack takes when someone runs:

webpack
32 / 123

lib/webpack.js

Step 1: Validate the webpack config.
33 / 123

lib/webpack.js

Step 2: Create Compiler instance.
34 / 123

Compiler

  • Owns the webpack configuration
  • Owns one or multiple compilations
  • Kicks off the actual compilation process
  • Is responsible for writing (emitting) the output files
  • Stays alive in watch mode
35 / 123

lib/webpack.js

Step 3: Kick off a compilation by calling compiler.run
36 / 123

lib/Compiler.js

Step 4: Trigger compilation lifecycle hooks
37 / 123

Everything inside webpack is built of plugins which rely on a common interface, called Tapable:

npm install tapable
38 / 123

A tapable instance exposes hooks...

39 / 123

...and plugins can listen on these hooks.

40 / 123

lib/Compiler.js

The Compiler class exposes a lot of hooks.
41 / 123

lib/Compiler.js

The hook type specifies the control flow of the hook (sync/async, bail/waterfall/parallel).
42 / 123

lib/Compiler.js

Let's take a closer look at the shouldEmit hook.

The shouldEmit hook can be used by plugins to tell the compiler that no files should be written on disk.

43 / 123

lib/NoEmitOnErrorsPlugin.js

For instance: the hook is "tapped" by the NoEmitOnErrorsPlugin which returns false if an error is found in the compilation stats.
44 / 123

lib/Compiler.js

The Compiler triggers the shouldEmit hook after all compilation hooks have been triggered, asking "anyone" if the next emitAssets hook should be triggered?
45 / 123

You might wonder:

How does the NoEmitOnErrorsPlugin get activated?

46 / 123

lib/WebpackOptionsApply.js

The plugin is activated if you set optimization.noEmitOnErrors in your webpack.config.js.
47 / 123

lib/WebpackOptionsApply.js

WebpackOptionsApply is a good starting point if you want to know which webpack option triggers which plugin.
48 / 123

lib/Compiler.js

Step 5: Now the Compiler calls the make hook which tells the compilation to add the first module or modules to the dependency graph. These are called entry dependencies.
49 / 123

Compilation

  • Represents a single build process
  • Owns the dependency and the chunk graph
  • Provides module factories that know how to build and parse modules
  • Each change creates a new compilation in watch mode
50 / 123

lib/Compilation.js

Everytime a dependency is added to the graph, the compilation looks for a module factory for this kind of dependency. In a lot of cases, it's the NormalModuleFactory.
51 / 123

Module

  • Represents a node in the dependency graph
  • Has a parser to parse the source code
  • Has a generator to generate source code
  • Holds all the relevant information for a module
    like the source code, source map, imports and exports
52 / 123

Until now, no source code has been loaded from disk. We just have a path to a module, which can even be relative:

"./src/main.js"
"../../other.js"
"react"
53 / 123

lib/NormalModuleFactory.js

Step 6: The NormalModuleFactory calls a resolver to resolve the module specifier.
54 / 123

Resolver

  • Knows how to find modules
  • Resolves relative module specifiers to absolute file paths
  • Uses a bunch of lookup algorithms
55 / 123

Resolving is complicated nowadays...

56 / 123

Resolvers should at least mimic the Node.js resolving algorithm:

57 / 123

But for compatibility reasons, it should also look for:

  • a browser entry in package.json
  • a module entry in package.json
  • bower_components folders (not activated by default anymore)
  • bower.json (not activated by default anymore)
  • ...
58 / 123

Resolving in webpack is done by enhanced-resolve:

59 / 123

lib/NormalModuleFactory.js

Step 7: Once the absolute path of the module has been resolved, webpack tries to find out which processing rules have been configured for this module.
60 / 123

Typical processing rules include loader configurations:

{
    test: /\.jsx?$/,
    include: [resolve(projectRoot, "src")],
    use: [
        {
            loader: "babel-loader",
            options: {
                ...babelConfig(),
                cacheDirectory: true,
            },
        },
    ],
},
61 / 123

Heads up

All rules will be applied, not just the first one that matches.

62 / 123
{
  test: /app\.js$/,
  use: ["babel-loader"],
},
{
  test: /\.jsx?$/,
  use: ["babel-loader"],
},

Applies the babel-loader twice to app.js.

63 / 123

lib/Compilation.js

Step 9: With all the processing information about the module, the Compilation starts the module build.
64 / 123

lib/NormalModule.js

Step 10: The module calls the loader-runner, which reads the file contents and processes the loader pipeline.
65 / 123

Loader

A loader is a function that:

  • takes source code or binary data as input
  • and returns new source code as output.

Loaders work on a per-file basis.

66 / 123

The simplest loader:

module.exports = function (content) {
  return content;
};
67 / 123

A simplified file-loader:

module.exports = function (content) {
  const url = generateHashedUrl(content);
  this.emitFile(url, content);
  return "export default " + JSON.stringify(url) + ";";
};
  • this.emitFile() tells webpack to emit a file with
    the given contents and filename to the output folder
  • return the JavaScript representation of that file
68 / 123

The loader-runner executes the loaders from right to left:

use: ["css-loader", "sass-loader"],
// like: cssLoader(sassLoader(fileContent));
69 / 123

The sass-loader receives Sass source code and produces CSS source code:

use: ["css-loader", "sass-loader"],
// like: cssLoader(sassLoader(fileContent));
70 / 123

The css-loader receives CSS source code and produces JavaScript code:

use: ["css-loader", "sass-loader"],
// like: cssLoader(sassLoader(fileContent));
71 / 123

Before the actual execution, the loader-runner performs a pitching phase from left to right.

This allows loaders to intercept the execution.

72 / 123

For instance, the thread-loader uses the pitching phase to push the execution to worker threads:

use: ["thread-loader", "css-loader", "sass-loader"],
73 / 123

The result of the loader pipeline should either be:

  • a JavaScript module, preferably ESM
  • JSON source
  • a WebAssembly module
74 / 123

lib/NormalModule.js

Step 11: The module hands the loader result to its parser.
75 / 123

Parser

  • Parses source code to an abstract syntax tree (AST)
  • Exposes AST nodes to webpack plugins
76 / 123

Webpack uses acorn to parse JavaScript modules.

If you want to find out how an AST looks like: astexplorer.net

77 / 123

lib/dependencies/CommonJsRequireDependencyParserPlugin.js

Step 12: Turn AST nodes into new dependencies for the dependency graph
78 / 123

Once a module has been built and parsed, all the previous steps starting from resolve are repeated until all dependencies have been processed.

 

79 / 123

The dependency graph might now look like this:

80 / 123

lib/Compiler.js

Step 13: Now the compiler triggers the seal hook which tells the compilation to optimize the dependency graph.
81 / 123

lib/Compilation.js

Step 14: During seal, the optimizeDependencies hooks are triggered.
82 / 123

During optimizeDependencies, the FlagDependencyUsagePlugin starts to walk through the dependency graph while keeping track of exports and imports.
83 / 123

This works reliable with ESMs:

import {a, b} from "some-module";
export {a, b, c, d, e, f};

Only a and b from "some-module" are imported.

84 / 123

But doesn't work with CommonJS:

const someModule = require("some-module");
module.exports = {a, b, c, d, e, f};

What properties of someModule are used?

85 / 123

Also during optimizeDependencies, the SideEffectsFlagPlugin looks for modules that have no side-effects.

86 / 123

Consider the following situation:

import {a} from "some-library";
// some-library/index.js
import {b} from "./b.js";
export const a = "a";
export {b};
// some-library/b.js
export const b = "b";
loginUser();

Webpack can't remove the unused export b because b.js contains a side-effect that needs to be executed.

87 / 123

It's impossible to detect that a module contains no side-effects if there's code to be executed.

Thus, module authors need to tell webpack explicitly in their package.json that their module won't execute side-effects when being imported:

{
  "name": "some-library",
  "sideEffects": false
}
88 / 123

89 / 123

lib/Compilation.js

Step 15: The chunk graph is created and optimized.
90 / 123

Webpack is just a module bundler which
turns a dependency graph into
an optimized chunk graph.

;)

91 / 123

What is the chunk graph?

The chunk graph tells webpack which module to put in which file (chunk).

92 / 123
For example, this dependency graph would be transformed into...
93 / 123
...this chunk graph.
94 / 123

The chunk graph get's more interesting when you're using async import():

import("./a.js")
  .then(a => {
    console.log("module a has been loaded");
  });

This tells webpack that a.js and all its dependencies are only necessary when the import() function is called.

95 / 123
Dashed arrows indicate async dependencies.
96 / 123

These allow webpack to put these modules into different files.
97 / 123

  1. Notice the duplication of b.css
  2. Also notice the empty chunk caused
    by a.js importing b.js asynchronously
98 / 123

Calculating the optimal chunk graph is a trade-off between code duplication...

99 / 123

...and big initial bundle size.

100 / 123

There are also other important things to consider:

  • Big chunks tend to have a better compression rate
  • But big chunks do also change more often which is bad for HTTP caching
101 / 123

lib/optimize/ModuleConcatenationPlugin.js

Step 16: The ModuleConcatenationPlugin tries to merge as many modules as possible into one scope (function wrapper).
102 / 123

This feature is also known as scope hoisting.

It decreases the bundle size and drastically improves the app startup time.

103 / 123

Old benchmark when webpack did not implement scope hoisting.

https://nolanlawson.com/2016/08/15/the-cost-of-small-modules/
104 / 123

lib/MainTemplate.js

Step 17: Render the chunk source code using templates (like the MainTemplate).
105 / 123

Templates

  • Renders a module, chunk, ... to source code
  • May include the module runtime
  • Ensure that source maps are working properly
106 / 123

The webpack output is an IIFE that can be divided into two parts:

(function (modules) {
  // module runtime
})([
  // module function wrappers
  function () {
    // source code of module 1
  },
  function () {
    // source code of module 2, 3, 4
  }
])
107 / 123

The remaining boring steps

  • Step 18: Minimize JavaScript using UglifyJS
  • Step 19: Write assets into output directory (emit)
108 / 123

How do you get the most out of it?

109 / 123

1.

Write ECMAScript modules.

Write idiomatic JavaScript code (listen to Benedikt & Sigurd).

110 / 123

2.

Make sure that your babel- or typescript-loader generate ESMs:

// .babelrc.js
{
  presets: [
    // babel 7 does that by default
    // babel 6
    ["env", {
      // don't transpile modules
      modules: false
    }],
  ],
}
// tsconfig.json
{
  "compilerOptions": {
      "module": "esnext",
  },
}
111 / 123

3.

Don't do side-effects in the top-level module scope...

import {loginUser} from "./loginUser.js";
loginUser() // please don't do that
  .then(() => {
    console.log("User logged in");
  })

...unless it's the entry module.

112 / 123

4.

Tell webpack about your good intentions by setting sideEffects: false in your package.json

{
  "sideEffects": false
}
113 / 123

5.

Use the module field in your package.json to point webpack to ESM code instead of an unoptimizable dist.js.

{
  "main": "./dist.js"
  "module" "./esm.js"
}

esm.js should still be transpiled down to ES5, except imports and exports.

114 / 123

6.

Use import() where possible to defer less important code.

async function openModal() {
  const modal = await import("./modal.js");
  modal.open();
}
115 / 123

7.

Use the magic webpack comment webpackPrefetch: true.

import(/* webpackPrefetch: true */"./modal.js");

Webpack will use <link rel="prefetch"> to prefetch chunks in browser idle times.

116 / 123

8.

Enable persistent loader caches if supported to reduce the build time.

{
  loader: "babel-loader",
  options: {
    cacheDirectory: true
  }
}

(or use the cache-loader)

117 / 123

9.

Restrict the scope of your loader pipelines to your src folder.

{
  test: /\.jsx?$/,
  include: [resolve(projectRoot, "src")],
  use: [
    "babel-loader"
  ]
},
118 / 123

10.

Remember that each import of a .scss or .less file is a separate Sass or Less compilation.

import "./modal.less";  // <---- may contain duplicate css
import "./map.less";    // <---- may contain duplicate css
119 / 123

11.

Check the webpack-bundle-analyzer from time to time.

120 / 123

12.

Listen to webpack's performance hints:

121 / 123

Happy webpacking 🖖

122 / 123

Thank you

(P.S: we're hiring :))

123 / 123

2 / 123
Paused

Help

Keyboard shortcuts

↑, ←, Pg Up, k Go to previous slide
↓, →, Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
b Toggle blackout mode
f Toggle fullscreen mode
c Clone slideshow
p Toggle presenter mode
w Pause/Resume the presentation
t Restart the presentation timer
?, h Toggle this help
Esc Back to slideshow