v1.6.2-rc2

Overview

We're happy to announce the Boost v1.6.2-rc2 release.

Lotus v1.21.0 Support

This is a compatibility update for the Lotus v1.21.0-rcX release, and is backwards compatible with Lotus v1.20.x for users who don't yet wish to upgrade Lotus. See the Boost and Lotus compatibility matrix for up to date information, and to plan your upgrade.

Please make sure to upgrade to test the new features shipped in this release. See our upgrade guide for details.

What's Changed

• use forks of graphsync, go-data-transfer and go-fil-markets by @dirkmc in #1333
• chore: release v1.6.2-rc2 by @LexLuthr in #1340

Full Changelog: v1.6.2-rc1...v1.6.2-rc2

v1.6.2-rc1

Overview

We're happy to announce the Boost v1.6.2-rc1 release.

See the Boost and Lotus compatibility matrix for up to date information, and to plan your upgrade.

Please make sure to upgrade to test the new features shipped in this release. See our upgrade guide for details.

🌟 Highlights

This release candidate fixes a few of recently seen issues.

1. The traversal errors seen in graphsync retrievals should go away
   ERROR gs-queryexecutor queryexecutor/queryexecutor.go:190 traversal completion check failed, nBlocksRead=1, err=unregistered adl requested: "unixfs-preload"

2. Legacy deals UI should currently display if the deal is verified or not

3. Offline deal import should now work with relative path

What's Changed

• fix: legacy deal verified status by @LexLuthr in #1324
• fix: update go-unixfsnode enough to make sure unixfs-preload is available by @rvagg in #1323
• fix: expand file path by @LexLuthr in #1330

Full Changelog: v1.6.1...v1.6.2-rc1

v1.6.1

Overview

We're happy to announce the Boost v1.6.1 release. This release contains several fixes and improvements for Boost 1.6.0, users are highly encouraged to upgrade.

See the Boost and Lotus compatibility matrix for up to date information, and to plan your upgrade.

🌟 Highlights

🚀 Improved performance for retrievals over graphsync

We have simplified how unpaid retrievals over graphsync work. This results in significantly less calls to the Lotus API, and legacy code paths, which have improved download speeds and overall retrieval performance. All paid deals, or custom voucher types, will fallback to the old retrieval path.

A highlight of some of the key improvements:

• Resolved a potential memory leak that could occur from improperly terminated retrievals in the old code path.
• Reduced the frequency of timeout errors in popular retrieval clients, such as Lassie, by up to 30x. Improvements were more pronounced with higher parallelism.
• Reduced total memory utilization of retrieval by 40%
• While there is significant variance in retrieval performance due to latency, block size of the data being retrieved, size of retrieval, and number of parallel retrievals, we saw significant gains in performance over the existing code:
  • On a production SP with 30ms of latency we saw a minimum 20% improvement in retrieval speed
  • The benefit was less pronounced for higher latency, and larger retrievals, seeing on average an improvement of 5% with 120ms of latency
  • In a 0 latency test environment, retrieval speeds increased more than 35x
• Unpaid graphsync retrievals now use the same lightweight cache as booster-bitswap, which will improve performance for spikes in retrieval for the same content. This should help with performance when serving to retrieval clusters. Note, you don't need to be running booster-bitswap for this, as the cacheing layer is shared via boostd process.

💾 Online backup

Boost now supports online backups, so you no longer have to stop your node first.

boostd backup <backup directory>

See the backup and restore docs for more details.

What's Changed

• feat: make deal over json-rpc by @LexLuthr in #1262
• fix: set cancel log to debug level by @LexLuthr in #1275
• chore(deps): bump webpack from 5.72.1 to 5.76.1 in /react by @dependabot in #1278
• Simple graphsync retrieval server by @dirkmc in #1236
• async contract monitor by @nonsense in #1282
• feat: Online backup by @LexLuthr in #1265
• Dont log errors when the transfer event queue is full by @dirkmc in #1291
• fatally fail contract deals on publish errors by @nonsense in #1294
• better logs for deal id by @nonsense in #1298
• timer for eth_call by @nonsense in #1299
• print protos as part of boostx stats by @nonsense in #1300
• fixed "unknown actor code" constantly appearing in the boost logs via lotus upgrade @jacobheun in #1293
• Set user agent from boost build version instead of lotus by @dirkmc in #1305
• boostx stats: output agent version and retrieval protocols by @dirkmc in #1304
• fixed a potential panic in go-graphsync by @dirkmc in #1313
• fixed an issue where free storage space could be miscalculated after a config change by @LexLuthr in #1315

New Contributors

• @brossetti1 made their first contribution in #1302

Full Changelog: v1.6.0...v1.6.1

v1.6.1-rc1

Overview

We're happy to announce the Boost v1.6.1-rc1 release.

See the Boost and Lotus compatibility matrix for up to date information, and to plan your upgrade.

Please make sure to upgrade to test the new features shipped in this release. See our upgrade guide for details.

🌟 Highlights

• This release contains multiple fixes and improvements for contract deals
• SP's can now create deals over to the RPC api, which should help improve deal flows when acting on behalf of your clients, as you no longer need to use libp2p
• Fixed an issue causing excessive unknown actor code logs

Improved performance for retrievals over graphsync

We have simplified how unpaid retrievals over graphsync work. This results in significantly less calls to the Lotus API, and legacy code paths, which should result in increased download speeds and overall retrieval performance. All paid deals, or custom voucher types, will fallback to the old retrieval path. We'll be posting more information about our performance testing as part of the official release.

What's Changed

• feat: make deals over json-rpc by @LexLuthr in #1262
• fix: set cancel log to debug level by @LexLuthr in #1275
• Simple graphsync retrieval server by @dirkmc in #1236
• async contract monitor by @nonsense in #1282
• feat: Online backup by @LexLuthr in #1265
• Disregard unseal price for unpaid retrieval by @dirkmc in #1289
• Dont log errors when the transfer event queue is full by @dirkmc in #1291
• fatally fail contract deals on publish errors by @nonsense in #1294
• better logs for deal id by @nonsense in #1298
• timer for eth_call by @nonsense in #1299
• print protos as part of boostx stats by @nonsense in #1300

Full Changelog: v1.6.0...v1.6.1-rc1

v1.6.0

Overview

We're happy to announce the Boost v1.6.0 release.

This release is compatible with Lotus v1.20.0 release. You should only upgrade to this release if you are also upgrading Lotus to v1.20.0 or higher for NV18 network upgrade. See the Boost and Lotus compatibility matrix for up to date information, and to plan your upgrade.

Please make sure to upgrade to test the new features shipped in this release. See our upgrade guide for details.

Boost v1.5.x Adopters

If you have been running the v1.5.x and want to update to this release for lotus compatibility, you can do so as long as you are also upgrading Lotus to v1.20.0 or later. This PR contains the latest 1.5.3 codebase and we will continue to update it with any subsequent releases.

🌟 Highlights

Includes updates previously noted in RC builds of this release

FVM Contract Deal Support

• With the release of FVM, it is now possible for smart contracts to make deal proposals on-chain. This is made possible though the DealProposal FRC.
• DataDAOs, as well as other clients who want to store data on Filecoin, can now deploy a smart contract on the FVM which adheres to the DealProposal FRC, and make deal proposals that are visible to every storage provider running Boost who monitors the chain.
• In order to enable DealProposal FRC, you have to edit your config.toml and enable contract deal monitoring. By default it is disabled. Here is an example configuration:

[ContractDeals]
  Enabled = true
  AllowlistContracts = []
  From = "0x0000000000000000000000000000000000000000"

Cancelation of stalled retrievals

Boost now runs a background process that will cancel retrievals that have not been updated in the past 30minutes by default. You can configure this by setting the new DealMaking.StalledRetrievalTimeout property in the Boost config.

The following config will cancel all retrievals that have not changed state, or transferred bytes in the past 10minutes.

[Dealmaking]
  ...
  RetrievalLogDuration = "24h0m0s"
  StalledRetrievalTimeout = "10m0s"

Note: For efficiency of the job, Boost leverages the Retrieval Logs DB, so your RetrievalLogDuration should always exceed the StalledRetrievalTimeout.

Updated deal filter

Boost now exposes the following additional deal variables to the external deal filters like CIDgravity. This should allow more extensive filter logic for deal acceptance.

• IsOffline
• SkipIPNIAnnounce
• RemoveUnsealedCopy

🤩 Upcoming Features

These features will be part of the upcoming releases.

🆕 Graphsync Retrieval Server

• This is a simpler version of current graphsync server for retrievals
• It should allow much faster retrievals
• Only free retrievals will be handled by this code path. Paid retrievals will still go via the old code path

Online backup 💾

• Boost will support online backups along with the current offline version

What's Changed

• move markets code from lotus to boost by @dirkmc in #1186
• move some utilities to extern/boostd-data/shared by @dirkmc in #1195
• Port filecoin-project/lotus#9627 to Boost by @LexLuthr in #1202
• update go-bitswap dependency by @LexLuthr in #1204
• fix: BasicDealFilter logic by @LexLuthr in #1210
• feat: cache sealing pipeline status by @LexLuthr in #1209
• initial boost / fvm contract deals support by @nonsense in #1183
• port #1213 to main by @LexLuthr in #1214
• fix: bug bypassing basic filters by @LexLuthr in #1217
• add start-epoch-head-offset by @anjor in #1223
• Update types.go by @RobQuistNL in #1226
• fix homedir expansion by @nonsense in #1231
• feat: add job to cancel stalled retrievals by @jacobheun in #1233
• clarify cid type by @anjor in #1234
• test: devnet quality of life updates by @jacobheun in #1245
• Simplify docker build tasks by @airenas in #1230
• remove MessageConfidence from boost build params by @nonsense in #1246
• fix docker image tagging by @nonsense in #1247
• FRC DealProposal integration by @nonsense in #1216
• feat(ui): display end of peerid instead of front by @jacobheun in #1252
• remove defer statement for boost retrieve by @nonsense in #1254
• add recover to contract monitor by @nonsense in #1256

New Contributors

• @anjor made their first contribution in #1223
• @RobQuistNL made their first contribution in #1226

Full Changelog: v1.5.3...v1.6.0

v1.6.0-rc3 - Lotus v1.20.0 Compatibility

Overview

We're happy to announce the Boost v1.6.0-rc3 release.

This is a compatibility update for the Lotus v1.20.0 release. You should only upgrade to this release if you are also upgrading Lotus to v1.20.0. See the Boost and Lotus compatibility matrix for up to date information, and to plan your upgrade.

Please make sure to upgrade to test the new features shipped in this release. See our upgrade guide for details.

Boost v1.5.x Adopters

If you have been running the v1.5.x and want to update to this release for lotus compatibility, you can do so as long as you are also upgrading Lotus to v1.20.0. This PR contains the latest 1.5.3 codebase and we will continue to update it with any subsequent releases.

🌟 Highlights

Includes updates previously noted in RC builds of this release

FVM Contract Deal Support

• With the release of FVM, it is now possible for smart contracts to make deal proposals on-chain. This is made possible though the DealProposal FRC.
• DataDAOs, as well as other clients who want to store data on Filecoin, can now deploy a smart contract on the FVM which adheres to the DealProposal FRC, and make deal proposals that are visible to every storage provider running Boost who monitors the chain.
• In order to enable DealProposal FRC, you have to edit your config.toml and enable contract deal monitoring. By default it is disabled. Here is an example configuration:

[ContractDeals]
  Enabled = true
  AllowlistContracts = []
  From = "0x0000000000000000000000000000000000000000"

Cancelation of stalled retrievals

Boost now runs a background process that will cancel retrievals that have not been updated in the past 30minutes by default. You can configure this by setting the new DealMaking.StalledRetrievalTimeout property in the Boost config.

The following config will cancel all retrievals that have not changed state, or transferred bytes in the past 10minutes.

[Dealmaking]
  ...
  RetrievalLogDuration = "24h0m0s"
  StalledRetrievalTimeout = "10m0s"

Note: For efficiency of the job, Boost leverages the Retrieval Logs DB, so your RetrievalLogDuration should always exceed the StalledRetrievalTimeout.

Updated deal filter

Boost now exposes the following additional deal variables to the external deal filters like CIDgravity. This should allow more extensive filter logic for deal acceptance.

• IsOffline
• SkipIPNIAnnounce
• RemoveUnsealedCopy

What's Changed

• move markets code from lotus to boost by @dirkmc in #1186
• move some utilities to extern/boostd-data/shared by @dirkmc in #1195
• Port filecoin-project/lotus#9627 to Boost by @LexLuthr in #1202
• update go-bitswap dependency by @LexLuthr in #1204
• fix: BasicDealFilter logic by @LexLuthr in #1210
• feat: cache sealing pipeline status by @LexLuthr in #1209
• initial boost / fvm contract deals support by @nonsense in #1183
• port #1213 to main by @LexLuthr in #1214
• fix: bug bypassing basic filters by @LexLuthr in #1217
• add start-epoch-head-offset by @anjor in #1223
• Update types.go by @RobQuistNL in #1226
• fix homedir expansion by @nonsense in #1231
• feat: add job to cancel stalled retrievals by @jacobheun in #1233
• clarify cid type by @anjor in #1234
• test: devnet quality of life updates by @jacobheun in #1245
• Simplify docker build tasks by @airenas in #1230
• remove MessageConfidence from boost build params by @nonsense in #1246
• fix docker image tagging by @nonsense in #1247
• FRC DealProposal integration by @nonsense in #1216
• feat(ui): display end of peerid instead of front by @jacobheun in #1252
• remove defer statement for boost retrieve by @nonsense in #1254
• add recover to contract monitor by @nonsense in #1256

New Contributors

• @anjor made their first contribution in #1223
• @RobQuistNL made their first contribution in #1226

Full Changelog: v1.5.3...v1.6.0-rc3

v1.5.3

Overview

We're happy to announce Boost 1.5.3 which includes automatic cancelation of stalled retrievals.

Lotus 1.20.0 Users: If you have already upgraded Lotus to 1.20.0 you should be using Boost 1.6.0-rc1 or later. See the Boost and Lotus compatibility matrix for up to date information, and to plan your upgrade.

An official release of 1.6 is scheduled for March 8th.

Highlights

Cancelation of stalled retrievals

Boost now runs a background process that will cancel retrievals that have not been updated in the past 30minutes by default. You can configure this by setting the new DealMaking.StalledRetrievalTimeout property in the Boost config.

The following config will cancel all retrievals that have not changed state, or transferred bytes in the past 10minutes.

[Dealmaking]
  ...
  RetrievalLogDuration = "24h0m0s"
  StalledRetrievalTimeout = "10m0s"

Note: For efficiency of the job, Boost leverages the Retrieval Logs DB, so your RetrievalLogDuration should always exceed the StalledRetrievalTimeout.

What's Changed

• feat: back port add job to cancel stalled retrievals (#1233) by @jacobheun in #1239

Full Changelog: v1.5.2...v1.5.3

v1.6.0-rc2 - Lotus v1.20.0-rcX Compatibility

Overview

We're happy to announce the Boost v1.6.0-rc2 release.

This is a patch release to fix slow offline deal processing by Boost. You should upgrade to this release if you are currently running Boost v.1.6.0-rc1. See the Boost and Lotus compatibility matrix for up to date information, and to plan your upgrade.

Please make sure to upgrade to test the new features shipped in this release. See our upgrade guide for details.

What's Changed

• feat: cache sealing pipeline status by @LexLuthr in #1209
• fix: BasicDealFilter logic by @LexLuthr in #1210
• fix: send more deal params to deal filters by @LexLuthr in #1213
• fix: bug bypassing basic filters by @LexLuthr in #1217

Full Changelog: v1.6.0-rc1...v1.6.0-rc2

v1.5.2

Overview

We're happy to announce the Boost v1.5.2 release.

This is a patch release to fix slow offline deal processing by Boost. You should upgrade to this release if you are currently running Boost v.1.5.1. See the Boost and Lotus compatibility matrix for up to date information, and to plan your upgrade.

Please make sure to upgrade to test the new features shipped in this release. See our upgrade guide for details.

What's Changed

• fix: offline deal slow acceptance and ConsiderOnlineStorageDeals rejection by @LexLuthr in #1211
• fix: send more deal params to deal filters by @LexLuthr in #1213
• fix: basic deal filters no longer skipped #1218

Full Changelog: v1.5.1...v1.5.2

v1.6.0-rc1 - Lotus v1.20.0-rc1 Compatibility

Overview

We're happy to announce the Boost v1.6.0-rc1 release.

This is a compatibility update for the Lotus v1.20.0-rc1 release. You should only upgrade to this release if you are also upgrading Lotus to v1.20.0-rc1. See the Boost and Lotus compatibility matrix for up to date information, and to plan your upgrade.

Please make sure to upgrade to test the new features shipped in this release. See our upgrade guide for details.

Boost v1.5.1 RC Adopters

If you have been running the v1.5.1 RC and want to update to this release for lotus compatibility, you can do so as long as you are also upgrading Lotus to v1.20.0-rc1. This PR contains the latest 1.5.1 codebase and we will continue to update it with any subsequent releases.

What's Changed

• Update support for lotus v1.20.0-rc1 by @jacobheun in #1188
• feat: add flags for bitswap server internals by @jacobheun in #1182
• Introduce deal proposal protocol v1.2.1 by @dirkmc in #1185
• Devnet images using Lotus from DockerHub by @airenas in #1157

Full Changelog: v1.5.1-rc5...v1.6.0-rc1