Accelerated Package Installation via Async Exec Graphs #5

New Issue

sortedcord · 2026-06-25T00:01:05+05:30

sortedcord commented

2026-06-25 00:01:05 +05:30

Traditional package managers (apt, dnf, pacman) are highly synchronous. When bootstrapping a new machine or installing a large batch of tools, the user is forced to wait for sequential operations: resolving package A, downloading A, extracting A, installing A... before moving on to package B.

This synchronous nature wastes significant time waiting on Network I/O.

Because the proposed bootstrap architecture separates tools into binary, managed, and system strategies, it naturally unlocks the ability to orchestrate an accelerated, asynchronous installation pipeline.

By taking control of the installation pipeline, we can implement an asynchronous, non-blocking execution graph:

1. Concurrent Downloads (Network I/O)

The most time-consuming part of installing multiple CLI tools is waiting for sequential network requests. Since binary strategy tools are just raw file downloads from GitHub, these can be perfectly parallelized.

Using curl -Z (parallel transfers) or spawning background processes (download_file ... &), we can rapidly download multiple .tar.gz and .deb files concurrently into the temporary directory.

2. Dependency Sequencing (Blocking vs. Non-Blocking)

We can analyze the installation intent to build an execution graph:

Non-blocking tasks: Extracting a binary tool like lazygit has zero side-effects and depends on nothing. Ten binary tools can be extracted and copied into $BOOTSTRAP_DIR/bin/ simultaneously in background subshells.
Blocking tasks: system strategy tools (like Docker) require exclusive locks on the native package manager (dpkg/pacman) and must be executed synchronously. managed tools (like Node via NVM) might require their base manager to be installed first.

3. Implementation Approach

When a user runs b up or installs multiple tools at once (e.g. b install lazygit bat zoxide starship yazi), the CLI will execute the following pipeline:

Resolve Strategies: Map all requested tools to their corresponding installation strategies (binary, system, or managed).
Fetch Concurrently: Fire off concurrent downloads for all binary and managed assets in the background.
Queue System Packages: Hand off all system strategy packages to a single blocking pkg_install command (sudo apt install package-a package-b) so the OS package manager resolves them efficiently in one transaction.
Extract & Link Concurrently: As the background downloads finish, concurrently extract and link the binary tools into $BOOTSTRAP_DIR/bin/.
Wait & Register: Use wait in bash to block until all background jobs finish. Finally, sequentially register all successful installations into the registry.json using flock to prevent race conditions.

Outcome

This architecture will make bootstrapping a new machine or batch-installing tools feel incredibly fast, drastically outperforming standard synchronous package managers by saturating network bandwidth and CPU cores during extraction.

Traditional package managers (`apt`, `dnf`, `pacman`) are highly synchronous. When bootstrapping a new machine or installing a large batch of tools, the user is forced to wait for sequential operations: resolving package A, downloading A, extracting A, installing A... before moving on to package B. This synchronous nature wastes significant time waiting on Network I/O. Because the proposed `bootstrap` architecture separates tools into `binary`, `managed`, and `system` strategies, it naturally unlocks the ability to orchestrate an accelerated, asynchronous installation pipeline. By taking control of the installation pipeline, we can implement an **asynchronous, non-blocking execution graph**: ### 1. Concurrent Downloads (Network I/O) The most time-consuming part of installing multiple CLI tools is waiting for sequential network requests. Since `binary` strategy tools are just raw file downloads from GitHub, these can be perfectly parallelized. - Using `curl -Z` (parallel transfers) or spawning background processes `(download_file ... &)`, we can rapidly download multiple `.tar.gz` and `.deb` files concurrently into the temporary directory. ### 2. Dependency Sequencing (Blocking vs. Non-Blocking) We can analyze the installation intent to build an execution graph: - **Non-blocking tasks:** Extracting a `binary` tool like `lazygit` has zero side-effects and depends on nothing. Ten binary tools can be extracted and copied into `$BOOTSTRAP_DIR/bin/` simultaneously in background subshells. - **Blocking tasks:** `system` strategy tools (like Docker) require exclusive locks on the native package manager (`dpkg`/`pacman`) and must be executed synchronously. `managed` tools (like Node via NVM) might require their base manager to be installed first. ### 3. Implementation Approach When a user runs `b up` or installs multiple tools at once (e.g. `b install lazygit bat zoxide starship yazi`), the CLI will execute the following pipeline: 1. **Resolve Strategies:** Map all requested tools to their corresponding installation strategies (`binary`, `system`, or `managed`). 2. **Fetch Concurrently:** Fire off concurrent downloads for all `binary` and `managed` assets in the background. 3. **Queue System Packages:** Hand off all `system` strategy packages to a single blocking `pkg_install` command (`sudo apt install package-a package-b`) so the OS package manager resolves them efficiently in one transaction. 4. **Extract & Link Concurrently:** As the background downloads finish, concurrently extract and link the `binary` tools into `$BOOTSTRAP_DIR/bin/`. 5. **Wait & Register:** Use `wait` in bash to block until all background jobs finish. Finally, sequentially register all successful installations into the `registry.json` using `flock` to prevent race conditions. ## Outcome This architecture will make bootstrapping a new machine or batch-installing tools feel incredibly fast, drastically outperforming standard synchronous package managers by saturating network bandwidth and CPU cores during extraction.

sortedcord changed title from ~~improved package batching~~ to Accelerated Package Installation

2026-06-26 10:07:27 +05:30

sortedcord added a new dependency 2026-06-26 10:09:45 +05:30

#16 Installation Strategies Redesign

sortedcord changed title from ~~Accelerated Package Installation~~ to Accelerated Package Installation via Async Exec Graphs

2026-06-26 10:11:22 +05:30

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Depends on

#16 Installation Strategies Redesign

sortedcord/bootstrap

Reference: sortedcord/bootstrap#5