Skip to content

Commit

Permalink
chore: add illustration of partial bundling
Browse files Browse the repository at this point in the history
  • Loading branch information
wre232114 committed Sep 4, 2023
1 parent 11a8b51 commit 3401882
Show file tree
Hide file tree
Showing 4 changed files with 25 additions and 9 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
34 changes: 25 additions & 9 deletions rfcs/003-partial-bundling/rfc.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,18 +21,18 @@ For traditional bundlers, we may have a hard time to configure complex `splitChu

# Motivation
There are two main methods of handling modules in web build tools now: Bundling or native ESM. But they both have drawbacks:
* For bundling, bundlers aim to bundle everything together and then split them out for optimization, but splitting is often hard to configure and is hard to balance resources loading performance and cache hit rate by hand.
* For bundling, bundlers aim to bundle everything together and then split them out for optimization, but splitting is often hard to configure and is hard to balance resources loading performance and cache hit rate manually.
* For native esm, every module can be compiled, cached separately, but the load performance are heavily affected when there are hundreds of modules.

So I was always thinking that if there is a strategy to Avoid these two extremes - maybe we can do partial bundling? we can just bundle the project into several limited, size balanced resources directly and automatically. I named this thinking `Module Merging` - Find a balance between bundle and unbundled, only bundles a few related modules to improve loading performance instead of losing cache granularity.
So I was always thinking that if there is a strategy to avoid these two extremes - maybe we can do partial bundling? we can just bundle the project into several limited, size balanced resources directly and automatically. I named this thinking `Module Merging` - Find a balance between bundle and unbundled, only bundles a few related modules to improve loading performance instead of losing cache granularity.

> I renamed `Module Merging` to `Partial Bundling` later because I think `Partial Bundling` can expresses more accurately what I am thinking.
But some of my friends thought that is what `splitChunks` of webpack does, but I don't think so, they are similar but their basic ideas is different:
* Farm does not always bundle, only if concurrent module requests exceed Farm's threshold, Farm does partial bundling only when necessary.
* Farm does bundling only for performance reason, if one day hundreds concurrent requests are acceptable, then Farm will not bundle any more.

And this foundation difference also greatly affects the chooses when implement bundling. The detailed designs will be described in following sections.
And this fundamental difference also greatly affects the chooses when implement bundling. The detailed designs will be described in following sections.

# Design Philosophy
To achieve our two goals(reduce request numbers and increase cache hit rate), some general optimize principles should be respected:
Expand All @@ -55,16 +55,32 @@ This section explains the technical part of Partial Bundling.

## Related Terms That Farm Defines
* **Module**: Basic compilation Unit for Farm, it can be a file loaded for disk or a virtual module generated by plugins.
* **ModuleGraph**: Graph of modules, if a module `imports` another module, then a dependency edge would be created.
* **ModuleGroup**: All necessary modules for initial execution or dynamic request. For example,
* ModuleGroupGraph
* ModuleBucket
*
* **ModuleGraph**: Graph of modules, if a module `imports` another module, then a dependency edge would be created in the graph.
* **ModuleGroup**: All necessary modules for initial execution or dynamic request. For example, all statically imported modules start from `index.ts` or all statically imported modules start from a dynamically imported modules. We'll illustrate it in following sections.
* **ModuleGroupGraph**: Graph of module groups, if any module of module group dynamically imports another module group's entry modules, then a dependency edge would be created in the graph.
* **ModuleBucket**: Modules which are in the Module Groups. A module can be in many Module Groups, if all the ModuleGroups of two modules are the same, then these two modules are in the same ModuleBucket.
* **ModulePot**: Modules in the same Module Pot would always be in the same output resource. For example, modules in the same package will be in the same ModulePot if there are many packages.
* **ResourcePot**: A Resource Pot consist of one or more Module Pot, and a Resource Pot should produce a `output resource` and a optional `.map` resource

## Partial Bundling Process
Ge
![General Bundle Process](./resources/general-process.png)

The partial bundling process is split into 4 steps:
1. Create `ModuleGroupGraph` base on module graph that we created in `Build Stage`.
2. Create `ModuleBuckets` from `ModuleGroupGraph`
3. Create `ModulePots` for each `ModuleBucket` that satisfy our rules fined above.
4. Merge `ModulePots` into `ResourcePot`

we discuss the details of each step in following sections.

Assume that we already have following `ModuleGraph`:

<img src="./resources/ModuleGraph.png" width="200px" />

## Generate Module Groups
ModuleGroups need to be generated first like illustration below:
![ModuleGroup](./resources/ModuleGroup.png)


## Generate Module Buckets

Expand Down

0 comments on commit 3401882

Please sign in to comment.