layout: true class: theme-whiskey, slides-left --- name: cover # Webpack deep dive Johannes Ewald
Peerigon GmbH
[@jhnnns](https://twitter.com/jhnnns)
---
--- layout: true class: theme-whiskey, slides-centered ---
---
--- ### 1. What is webpack? -- ### 2. How does it work? -- ### 3. How do you get the most out of it? --- class: slides-chapter ## What is webpack? --- layout: true class: slides-centered --- ### Webpack is just a module bundler which
turns a dependency graph into
an optimized chunk graph. ###
;)
--- ### Webpack is just a module bundler which
turns a dependency graph into
an optimized chunk graph. ### ;) --- ### Webpack is just a
module bundler
which
turns a dependency graph into
an optimized chunk graph. ### ;) --- ### Webpack is just a
module bundler
which
turns a
dependency graph
into
an optimized chunk graph. ### ;) --- ### Webpack is just a
module bundler
which
turns a
dependency graph
into
an
optimized chunk graph
. ### ;) --- ###
Webpack is just a
module bundler
--- ### Without a module bundler... .slides-content[ .slides-columns[ ``` // message.js const message = "Hello World"; printMessage(); ``` ``` // printMessage.js function printMessage() { console.log(message); } ``` ] - What if `message.js` is included before `printMessage.js`? - What if multiple files use the variable name `message`? ] --- ### Without a module bundler... .slides-content[ - the `<script>` order is important - top-level variables and functions are global As your application grows, this becomes a problem. ] --- .slides-content[ And there's an additional problem: All module systems are slow to initialize when a lot of modules need to be loaded. ] --- .slides-content[ This is due to the **roundtrip problem**:
] --- ### Module bundlers to the rescue! .slides-columns[
] .slides-columns[
] --- Module bundlers... ...**analyze JavaScript modules** by reading their source code...
...to generate **compatible and efficient script JavaScript files** (bundle). --- Module bundlers leverage the fact that all
module systems share the same **mental model**: - A module has a **unique id** - Modules reference other modules
as dependency via **specifiers** - A module specifier can be resolved to a module id --- This model is called the **dependency graph**. --- A dependency graph can be simple...
--- ...or complex...
--- ...or **very** complex.
--- .slides-content[ Module bundlers... ...try to gather as much information as possible about the dependency graph in order to apply **clever optimizations**. ] --- .slides-content[ Module bundlers... ...try to do as much as possible **on build time**...
...so that there's less to do **on runtime**. ] --- ### Comparison of JavaScript module systems .slides-content[ | AMD | CommonJS | ESM :------|:----|:---------|:--- **Analyzable** | With limitations | With limitations | Yes **Resolving** | Asynchronous | Synchronous | Implementation specific **Linking** | Copy by value | Copy by value | Live binding **Scoping** | Function wrapper | Function wrapper | New language environment ] --- Interoperability is a mess.
@bradleymeck
implementing ESMs for Node.js
--- ###
Webpack is just a
module bundler
which
turns a
dependency graph
into
an optimized chunk graph.
--- class: slides-chapter, theme-whiskey ## How does webpack work? --- layout: true class: slides-centered --- .slides-content[ The best way to understand webpack is to follow the steps webpack takes when someone runs: ``` webpack ``` ] --- .slides-content[
`lib/webpack.js`
.slides-caption[ **Step 1:** Validate the webpack config. ] ] --- .slides-content[
`lib/webpack.js`
.slides-caption[ **Step 2:** Create `Compiler` instance. ] ] --- ### `Compiler` - Owns the webpack configuration - Owns one or multiple `compilations` - Kicks off the actual `compilation` process - Is responsible for writing (emitting) the output files - Stays alive in `watch` mode --- .slides-content[
`lib/webpack.js`
.slides-caption[ **Step 3:** Kick off a compilation by calling `compiler.run` ] ] --- .slides-content[
`lib/Compiler.js`
.slides-caption[ **Step 4:** Trigger compilation lifecycle hooks ] ] --- .slides-content[ Everything inside webpack is built of plugins which rely on a common interface, called [Tapable](https://github.com/webpack/tapable):
``` npm install tapable ``` ] --- .slides-content[ A **tapable instance** exposes hooks...
] --- .slides-content[ ...and **plugins** can listen on these hooks.
] --- .slides-content[
`lib/Compiler.js`
.slides-caption[ The `Compiler` class exposes a lot of hooks. ] ] --- .slides-content[
`lib/Compiler.js`
.slides-caption[ The hook type specifies the control flow of the hook (sync/async, bail/waterfall/parallel). ] ] --- .slides-content[
`lib/Compiler.js`
.slides-caption[ Let's take a closer look at the `shouldEmit` hook. The `shouldEmit` hook can be used by plugins to tell the compiler that no files should be written on disk. ] ] --- .slides-content[
`lib/NoEmitOnErrorsPlugin.js`
.slides-caption[ For instance: the hook is "tapped" by the `NoEmitOnErrorsPlugin` which returns false if an error is found in the compilation stats. ] ] --- .slides-content[
`lib/Compiler.js`
.slides-caption[ The `Compiler` triggers the `shouldEmit` hook after all compilation hooks have been triggered, asking "anyone" if the next `emitAssets` hook should be triggered? ] ] --- .slides-content[ You might wonder: ### How does the `NoEmitOnErrorsPlugin` get activated? ] --- .slides-content[
`lib/WebpackOptionsApply.js`
.slides-caption[ The plugin is activated if you set `optimization.noEmitOnErrors` in your `webpack.config.js`. ] ] --- .slides-content[
`lib/WebpackOptionsApply.js`
.slides-caption[ `WebpackOptionsApply` is a good starting point if you want to know which webpack option triggers which plugin. ] ] --- .slides-content[
`lib/Compiler.js`
.slides-caption[ **Step 5**: Now the `Compiler` calls the `make` hook which tells the compilation to add the first module or modules to the dependency graph. These are called entry dependencies. ] ] --- ### `Compilation` - Represents a single build process - Owns the dependency and the chunk graph - Provides module factories that know how to build and parse modules - Each change creates a new compilation in `watch` mode --- .slides-content[
`lib/Compilation.js`
.slides-caption[ Everytime a dependency is added to the graph, the compilation looks for a module factory for this kind of dependency. In a lot of cases, it's the [`NormalModuleFactory`](https://github.com/webpack/webpack/blob/master/lib/NormalModuleFactory.js). ] ] --- ### `Module` - Represents a node in the dependency graph - Has a `parser` to parse the source code - Has a `generator` to generate source code - Holds all the relevant information for a module
like the source code, source map, imports and exports --- .slides-content[ Until now, no source code has been loaded from disk. We just have a path to a module, which can even be relative: ``` "./src/main.js" "../../other.js" "react" ``` ] --- .slides-content[
`lib/NormalModuleFactory.js`
.slides-caption[ **Step 6**: The `NormalModuleFactory` calls a `resolver` to resolve the module specifier. ] ] --- ### `Resolver` - Knows how to find modules - Resolves relative module specifiers to absolute file paths - Uses a bunch of lookup algorithms ---
**Resolving is complicated nowadays...** --- .slides-content[ Resolvers should at least mimic the [Node.js resolving algorithm](https://nodejs.org/docs/latest/api/modules.html#modules_all_together):
] --- .slides-content[ But for compatibility reasons, it should also look for: - a `browser` entry in `package.json` - a `module` entry in `package.json` - `bower_components` folders (not activated by default anymore) - `bower.json` (not activated by default anymore) - ... ] --- .slides-content[ Resolving in webpack is done by [enhanced-resolve](https://github.com/webpack/enhanced-resolve):
] --- .slides-content[
`lib/NormalModuleFactory.js`
.slides-caption[ **Step 7**: Once the absolute path of the module has been resolved, webpack tries to find out which **processing rules** have been configured for this module. ] ] --- .slides-content[ Typical processing rules include loader configurations: ``` { test: /\.jsx?$/, include: [resolve(projectRoot, "src")], use: [ { loader: "babel-loader", options: { ...babelConfig(), cacheDirectory: true, }, }, ], }, ``` ] --- .slides-content[ .slides-label.slides-danger[Heads up] **All rules** will be applied, **not just the first one that matches.** ] --- ``` { test: /app\.js$/, use: ["babel-loader"], }, { test: /\.jsx?$/, use: ["babel-loader"], }, ``` Applies the `babel-loader` twice to `app.js`. --- .slides-content[
`lib/Compilation.js`
.slides-caption[ **Step 9:** With all the processing information about the module, the `Compilation` starts the module build. ] ] --- .slides-content[
`lib/NormalModule.js`
.slides-caption[ **Step 10:** The module calls the [`loader-runner`](https://github.com/webpack/loader-runner), which reads the file contents and processes the loader pipeline. ] ] --- .slides-content[ ### Loader A loader is a function that: - takes source code or binary data as input - and returns new source code as output. Loaders work on a per-file basis. ] --- .slides-content[ The simplest loader: ``` module.exports = function (content) { return content; }; ``` ] --- .slides-content[ A simplified [`file-loader`](https://github.com/webpack-contrib/file-loader): ``` module.exports = function (content) { const url = generateHashedUrl(content); this.emitFile(url, content); return "export default " + JSON.stringify(url) + ";"; }; ``` - `this.emitFile()` tells webpack to emit a file with
the given contents and filename to the output folder - `return` the JavaScript representation of that file ] --- .slides-content[ The `loader-runner` executes the loaders from right to left:
``` use: ["css-loader", "sass-loader"], // like: cssLoader(sassLoader(fileContent)); ``` ] --- .slides-content[ The `sass-loader` receives Sass source code and produces CSS source code:
``` use: ["css-loader", "sass-loader"], // like: cssLoader(sassLoader(fileContent)); ``` ] --- .slides-content[ The `css-loader` receives CSS source code and produces JavaScript code:
``` use: ["css-loader", "sass-loader"], // like: cssLoader(sassLoader(fileContent)); ``` ] --- .slides-content[ Before the actual execution, the `loader-runner` performs a **pitching phase** from left to right. This allows loaders to intercept the execution. ] --- .slides-content[ For instance, the [`thread-loader`](https://github.com/webpack-contrib/thread-loader) uses the pitching phase to push the execution to worker threads: ``` use: ["thread-loader", "css-loader", "sass-loader"], ``` ] --- .slides-content[ The result of the loader pipeline should either be: - a JavaScript module, preferably ESM - JSON source - a WebAssembly module ] --- .slides-content[
`lib/NormalModule.js`
.slides-caption[ **Step 11**: The module hands the loader result to its `parser`. ] ] --- .slides-content[ ### `Parser` - Parses source code to an abstract syntax tree (AST) - Exposes AST nodes to webpack plugins ] --- .slides-content[ Webpack uses [acorn](https://github.com/acornjs/acorn) to parse JavaScript modules. If you want to find out how an AST looks like: [astexplorer.net](https://astexplorer.net/) ] --- .slides-content[
`lib/dependencies/CommonJsRequireDependencyParserPlugin.js`
.slides-caption[ **Step 12**: Turn AST nodes into new dependencies for the dependency graph ] ] --- .slides-content[ Once a module has been built and parsed, all the previous steps starting from resolve are repeated **until all dependencies have been processed**. ] --- .slides-content[ The dependency graph might now look like this:
] --- .slides-content[
`lib/Compiler.js`
.slides-caption[ **Step 13**: Now the compiler triggers the seal hook which tells the compilation to optimize the dependency graph. ] ] --- .slides-content[
`lib/Compilation.js`
.slides-caption[ **Step 14**: During `seal`, the `optimizeDependencies` hooks are triggered. ] ] ---
.slides-caption[ During `optimizeDependencies`, the [`FlagDependencyUsagePlugin`](https://github.com/webpack/webpack/blob/df7da2cdd58e9534ae5e73a25e9c064b02c4d625/lib/FlagDependencyUsagePlugin.js#L103) starts to walk through the dependency graph while keeping track of exports and imports. ] --- .slides-content[ This works reliable with ESMs: .slides-columns[ ``` import {a, b} from "some-module"; ``` ``` export {a, b, c, d, e, f}; ``` ] Only a and b from `"some-module"` are imported. ] --- .slides-content[ But doesn't work with CommonJS: .slides-columns[ ``` const someModule = require("some-module"); ``` ``` module.exports = {a, b, c, d, e, f}; ``` ] What properties of `someModule` are used? ] --- Also during `optimizeDependencies`, the [SideEffectsFlagPlugin](https://github.com/webpack/webpack/blob/master/lib/optimize/SideEffectsFlagPlugin.js) looks for modules that have no side-effects. --- .slides-content[ Consider the following situation: .slides-columns[ ``` import {a} from "some-library"; ``` ``` // some-library/index.js import {b} from "./b.js"; export const a = "a"; export {b}; ``` ``` // some-library/b.js export const b = "b"; loginUser(); ``` ] Webpack can't remove the unused export `b` because `b.js` contains a side-effect that needs to be executed. ] --- .slides-content[ It's impossible to detect that a module contains no side-effects if there's code to be executed. Thus, module authors need to tell webpack explicitly in their `package.json` that their module won't execute side-effects when being imported: ``` { "name": "some-library", "sideEffects": false } ``` ] ---
--- .slides-content[
`lib/Compilation.js`
.slides-caption[ **Step 15**: The **chunk graph** is created and optimized. ] ] --- ###
Webpack is just a
module bundler
which
turns a
dependency graph
into
an
optimized chunk graph
. ### ;) --- ### What is the chunk graph? The chunk graph tells webpack which module to put in which file (chunk). --- .slides-columns[
] .slides-caption[ For example, this dependency graph would be transformed into... ] --- .slides-columns[
] .slides-caption[ ...this chunk graph. ] --- .slides-content[ The chunk graph get's more interesting when you're using async `import()`: ``` import("./a.js") .then(a => { console.log("module a has been loaded"); }); ``` This tells webpack that `a.js` and all its dependencies are only necessary when the `import()` function is called. ] ---
.slides-caption[ Dashed arrows indicate async dependencies. ] ---
.slides-caption[ These allow webpack to put these modules into different files. ] ---
.slides-caption[ 1. Notice the duplication of `b.css` 2. Also notice the empty chunk caused
by `a.js` importing `b.js` asynchronously ] --- Calculating the optimal chunk graph is a trade-off between code duplication...
--- ...and big initial bundle size.
--- There are also other important things to consider: - Big chunks tend to have a **better compression rate** - But big chunks do also change more often which is **bad for HTTP caching** --- .slides-content[
`lib/optimize/ModuleConcatenationPlugin.js`
.slides-caption[ **Step 16:** The `ModuleConcatenationPlugin` tries to merge as many modules as possible into one scope (function wrapper). ] ] --- This feature is also known as **scope hoisting**. It decreases the bundle size and drastically improves the app startup time. ---
Old benchmark when webpack did not implement scope hoisting.
.slides-footnote[ https://nolanlawson.com/2016/08/15/the-cost-of-small-modules/ ] --- .slides-content[
`lib/MainTemplate.js`
.slides-caption[ **Step 17:** Render the chunk source code using templates (like the `MainTemplate`). ] ] --- .slides-content[ ### Templates - Renders a `module`, `chunk`, ... to source code - May include the **module runtime** - Ensure that source maps are working properly ] --- The webpack output is an [IIFE](https://en.wikipedia.org/wiki/Immediately-invoked_function_expression) that can be divided into two parts: ``` (function (modules) { // module runtime })([ // module function wrappers function () { // source code of module 1 }, function () { // source code of module 2, 3, 4 } ]) ``` --- ### The remaining boring steps - **Step 18:** Minimize JavaScript using [UglifyJS](https://github.com/webpack-contrib/uglifyjs-webpack-plugin) - **Step 19:** Write assets into output directory (emit) --- class: theme-whiskey, slides-chapter ## How do you get the most out of it? --- layout: true class: slides-centered --- .slides-content[ ### 1. Write ECMAScript modules. Write idiomatic JavaScript code (listen to Benedikt & Sigurd). ] --- .slides-content[ ### 2. Make sure that your babel- or typescript-loader generate ESMs: .slides-columns[ ``` // .babelrc.js { presets: [ // babel 7 does that by default // babel 6 ["env", { // don't transpile modules modules: false }], ], } ``` ``` // tsconfig.json { "compilerOptions": { "module": "esnext", }, } ``` ] ] --- .slides-content[ ### 3. Don't do side-effects in the top-level module scope... ``` import {loginUser} from "./loginUser.js"; loginUser() // please don't do that .then(() => { console.log("User logged in"); }) ``` ...unless it's the entry module. ] --- .slides-content[ ### 4. Tell webpack about your good intentions by setting [`sideEffects: false`](https://webpack.js.org/guides/tree-shaking/#mark-the-file-as-side-effect-free) in your `package.json` ``` { "sideEffects": false } ``` ] --- .slides-content[ ### 5. Use the [`module` field](https://github.com/dherman/defense-of-dot-js/blob/master/proposal.md) in your `package.json` to point webpack to ESM code instead of an unoptimizable `dist.js`. ``` { "main": "./dist.js" "module" "./esm.js" } ``` `esm.js` should still be transpiled down to ES5, except imports and exports. ] --- .slides-content[ ### 6. Use `import()` where possible to defer less important code. ``` async function openModal() { const modal = await import("./modal.js"); modal.open(); } ``` ] --- .slides-content[ ### 7. Use the magic webpack comment [`webpackPrefetch: true`](https://medium.com/webpack/link-rel-prefetch-preload-in-webpack-51a52358f84c). ``` import(/* webpackPrefetch: true */"./modal.js"); ``` Webpack will use `<link rel="prefetch">` to prefetch chunks in browser idle times. ] --- .slides-content[ ### 8. Enable persistent loader caches if supported to reduce the build time. ``` { loader: "babel-loader", options: { cacheDirectory: true } } ``` (or use the [cache-loader](https://github.com/webpack-contrib/cache-loader)) ] --- .slides-content[ ### 9. Restrict the scope of your loader pipelines to your `src` folder. ``` { test: /\.jsx?$/, include: [resolve(projectRoot, "src")], use: [ "babel-loader" ] }, ``` ] --- .slides-content[ ### 10. Remember that each `import` of a `.scss` or `.less` file is a separate Sass or Less compilation. ``` import "./modal.less"; // <---- may contain duplicate css import "./map.less"; // <---- may contain duplicate css ``` ] --- .slides-content[ ### 11. Check the [`webpack-bundle-analyzer`](https://www.npmjs.com/package/webpack-bundle-analyzer) from time to time.
] --- .slides-content[ ### 12. Listen to webpack's performance hints:
] --- ## Happy webpacking 🖖 --- ## Thank you (P.S: [we're hiring](https://peerigon.com/en/jobs) :))