Issue Details

Number
30983
Title
RFC: Multiprocess binaries and packaging options
Description
Issue for discussion about ways multiprocess functionality could be packaged and released. Trying to figure out which of the 3 release options described below makes the most sense. ### Binaries One goal of the [multiprocess project](https://github.com/bitcoin/bitcoin/issues/28722) has been to provide minimal binaries that only include node, wallet, or gui code, which spawn or connect to other processes as needed to provide other functionality. The idea implemented in #10102 is to have 3 binaries: - **`bitcoin-node`** - contains node code and libraries (leveldb), and no wallet or gui code - **`bitcoin-wallet`** - contains wallet code and libraries (sqlite), and no node or gui code - **`bitcoin-gui`** - contains gui code and libraries (qt), and no node or wallet code Which are built differently than current binaries: - **`bitcoind`** - contains node code+libraries (leveldb) and wallet code+libraries (sqlite) - **`bitcoin-qt`** - contains node code+libraries (leveldb), wallet code+libraries (sqlite), and gui code+libraries (qt) ### Original release plan My original idea implementing this was for there to be a separate multiprocess bitcoin release with the multiprocess binaries. The main release would be unchanged, including existing `bitcoind` and `bitcoin-qt` binaries, but there would be a separate release containing `bitcoin-node`, `bitcoin-wallet` and `bitcoin-gui` binaries. The main release would be available in the normal place like *bitcoincore.org/bin/bitcoin-core-27.1/*, and the multiprocess release would be available alongside, at *bitcoincore.org/bin/bitcoin-core-multiprocess-27.1/* or someplace similar. Then, after a release cycle or two, the multiprocess release could replace the main release if it was working well. But this approach has some drawbacks, namely that it would be a burden for maintainers to create separate releases and make them accessible, and that it could be confusing for users to encounter two releases with different binaries. So it doesn't seem like this approach is optimal (at least not for general users, maybe it would be ok for miners using the IPC mining interface). ### Release options overview Taking a step back, it seems like there are 3 possible options for releasing multiprocess functionality: 1. **Side-release**: create a separate multiprocess release for multiprocess binaries, as described above. 2. **Side-binaries**: add multiprocess binaries (`bitcoin-node`, `bitcoin-wallet`, `bitcoin-gui`) to existing releases alongside existing binaries (`bitcoind` and `bitcoin-qt`). 3. **Combined binaries**: add multiprocess functionality to existing `bitcoind` and `bitcoin-qt` binaries that can be enabled/disabled with runtime arguments. 4. [**Minimally combined binaries**](https://github.com/bitcoin/bitcoin/issues/30983#issuecomment-2620155964): add limited multiprocess functionality to existing `bitcoind` and `bitcoin-qt` binaries that can be enabled/disabled with runtime arguments. Difference from option 3 is that IPC side-binaries would also exist and `bitcoind` and `bitcoin-qt` would not contain IPC code, but automatically `exec()` IPC binaries when `-ipcbind` or other IPC options were enabled. ### Release options comparison From discussion in recent PR's and offline, it seems like approach (3) might be favored, but it is worth considering pros and cons of the three approaches. 1. **Side-release** - Pros: - *Opt-in*: Keeps original binaries unchanged for users who don't want multiprocess functionality. - *Minimal binaries*: Node binary contains only node code not wallet code, gui binary contains no node or wallet code. - Cons: - *Extra work*: It's more work for maintainers and signers to create an extra set of packages and make them accessible. - *Complexity*: Posting a separate release could be confusing for users. 2. **Side-binaries**: - Pros: - *Opt-in* and *Minimal binaries*: same as previous - *Easy to implement*: Implemented in [#30975](https://github.com/bitcoin/bitcoin/issues/30975) and basically requires no changes other than flipping cmake options. - Cons: - *Confusing*: This would probably be the most confusing approach for users. They would see one release with `bitcoind`, `bitcoin-qt`, `bitcoin-node`, `bitcoin-wallet`, `bitcoin-gui` binaries and not be able to easily understand how they are intended to be used. - Potential mitigations: - We would probably put multiprocess binaries in another folder (like `multiprocess/bin/` instead of `bin/`) so they are clearly labeled and aren't confused with the default binaries. - We might want to rethink our approach to packaging binaries to begin with. We are already shipping a substantial collection of binaries: `bitcoin-cli`, `bitcoind`, `bitcoin-qt`, `bitcoin-tx`, `bitcoin-util`, `bitcoin-wallet`, and `test_bitcoin` which users are expected to call individually, and are probably already somewhat confusing. By contrast, git ships dozens of binaries and nobody is confused by them because they live in a libexec directory and are wrapped with single `git` command that finds and executes the right ones. Maybe taking `git` as inspiration we could have add a unified `bitcoin` command that looks like: ```bash bitcoin gui [OPTIONS] # Start a gui bitcoin daemon [OPTIONS] # Start a daemon bitcoin rpc [OPTIONS] COMMAND # Call an rpc bitcoin wallet [OPTIONS] COMMAND # Access a wallet bitcoin test [OPTIONS] # Run tests bitcoin help # ... and more for bitcoin-util, bitcoin-tx ... ```` **Update:** wrapper command is implemented in [#31375](https://github.com/bitcoin/bitcoin/pull/31375) 3. **Combined binaries**: - Pros: - *Compatibility*: Users could download and invoke `bitcoind` and `bitcoin-qt` binaries as before, not know they support IPC, not use IPC unless it's switched on by default, and there would be runtime options to turn it on or off. - Cons: - *Bigger binaries and more dependencies*: Binaries will both contain node, wallet, and IPC code. There will be no non-IPC binaries users can deploy and run separately if they don't use IPC, and no node binary users can deploy and run separately if they don't use the wallet, or want to run the wallet and node on different hosts. - *Requires code changes*: Will require some thinking about how to add runtime options to [#10102](https://github.com/bitcoin/bitcoin/pull/10102) to control whether `bitcoind` should spawn separate wallet process or run wallet code internally, and similarly whether `bitcoin-gui` should use a separate node process or run node code in the same process. Will also require updating the build system. 4. **Minimally combined binaries**: - Pros: - Combines pros listed in option 1 with pros listed in option 3. - Cons: - *Some implementation messiness*: A downside of this approach is options will need to be parsed twice. For example `bitcoind` will need to read arguments and config file to detect if any IPC features were requested, then it will need to `exec()` `bitcoin-node` which will read the same options and configuration again. In general this doesn't seem as nice and clean as having a single wrapper binary like #31375 that calls all the other binaries, and can show help displaying what subcommands are available. As mentioned, approach (3) seems favored so far, though personally I like approach (2), especially if we could have a unified `bitcoin` command. Creating this issue to get feedback and document any decisions we make. --- This issue is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
URL
https://github.com/bitcoin/bitcoin/issue/30983
Closed by
Back to List