Frame: Agile Coretime Broker pallet (RFC-1) (#14568)

* Add Broker pallet * Flesh out CorePart * Repotting and fleshing out * more drafting * process timeslice * Test Fungibles completed * Auctions * Price morphing * First tests * Tidying up config/status * Docs * Timeslice todying * More Timeslice tidying * Tests] * Repotting. * Tests * Tests * System InstaPool cores and payout * Better Relay Test framework * Tests and instapool fixes * Support NFT interface * Proper renewals * Better events, results * Test transfer * Renewal test * Repot some impls and make dispatchables. * Better weight * Test migration * Document events * Introduce durations * Core count * Allow reassignment * Better naming * Error docs * Docs * Formatting * Advance notice period is in RC blocks, not timeslices * Docs * Formatting * Docs * Missing file * Added some events * Events for all dispatchables * Remove benchmark * Fix * Adds benchmark for configure and some basic setup * Adds benchmark for reserve and unreserve * Adds a couple of more benchmarks * Docs * Event * Fix * Adds benchmark for purchase * Dedup * Add some weight breakdowns * Repotting * Adds more benchmarks * Renaming and one more event * Sale event * Better price API and docs * Avoid possibility of clobbering renewal record * Avoid possibility of clobbering renewal record * Fixes a few benchmarks * Another test * More tests * Drop history test * Rename and CORE_MASK_BITS constant * Update frame/broker/src/dispatchable_impls.rs Co-authored-by: Oliver Tale-Yazdi <[email protected]> * Update frame/broker/src/dispatchable_impls.rs Co-authored-by: Oliver Tale-Yazdi <[email protected]> * Update frame/broker/src/dispatchable_impls.rs Co-authored-by: Oliver Tale-Yazdi <[email protected]> * Update frame/broker/src/utility_impls.rs Co-authored-by: Oliver Tale-Yazdi <[email protected]> * Update frame/broker/src/dispatchable_impls.rs Co-authored-by: Oliver Tale-Yazdi <[email protected]> * Update frame/broker/src/mock.rs Co-authored-by: Oliver Tale-Yazdi <[email protected]> * Addresses few review comments * Addresses few review comments * Addresses few review comments * Merge * Merge * ".git/.scripts/commands/fmt/fmt.sh" * Integrates broker in kitchensink * Minor update * Fixes typo * Moves balance back to u64 * Fixes kitchensink build * Fixes worst case for assign * Adds benchmark for process_core_count * Adds a couple of more benchmarks * Adds an assert for partition * Uses max_timeslices as input in claim_revenue benchmark * Adds benchmark for drop_renewal * Adds benchmark for process_core_schedule * Adds benchmark for process_pool * Adds assertion for transfer * Fixes benchmark for broker in kitchensink * Adds todo for process_revenue benchmark * Minor update * Fix for pool revenue history * remove TODOs * Fix tests * Document CoretimeInterface * rename part to mask * Fixes * Grumble * ".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_broker * Adds benchmark for drop_history and fixes worst case for claim_revenue * Adds drop_history in WeightInfo * ".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_broker * Minor fix for Quick Benchmark CI * Fixes * Docs * Headers * Expose a couple of APIs for benchmarking (#14688) * Expose a couple of APIs for benchmarking * Adds doc * Minor fix in CoretimeInterface impl for kitchensik * Minor * Cap renewal price * Adds a few tests * Adds more tests * Minor updates * Adds a test for an edge case * Fixes feature propagation * Fixes feature propagation * Adds doc fix * Syntax nits Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Reuse Bit assign functions Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Bitwise tests Signed-off-by: Oliver Tale-Yazdi <[email protected]> * adapt_price: Edge case for sold == target Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Add sanity checking to ConfigRecord Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Add deny(missing_docs) where possible Signed-off-by: Oliver Tale-Yazdi <[email protected]> * partition: forbid pivot_offset == 0 Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Sort features zepter format features Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Import Zero from new location Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Clippy: remove redundant clone Signed-off-by: Oliver Tale-Yazdi <[email protected]> * try to fix build Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Fix CI Signed-off-by: Oliver Tale-Yazdi <[email protected]> --------- Signed-off-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: Nikhil Gupta <[email protected]> Co-authored-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: command-bot <>
paritytech · Aug 24, 2023 · 46c78d5 · 46c78d5
1 parent 2fe6bc9
commit 46c78d5
Show file tree

Hide file tree

Showing 29 changed files with 5,884 additions and 18 deletions.
diff --git a/Cargo.lock b/Cargo.lock
diff --git a/Cargo.toml b/Cargo.toml
@@ -131,6 +131,7 @@ members = [
 	"frame/benchmarking",
 	"frame/benchmarking/pov",
 	"frame/bounties",
+	"frame/broker",
 	"frame/child-bounties",
 	"frame/collective",
 	"frame/contracts",

diff --git a/bin/node/runtime/Cargo.toml b/bin/node/runtime/Cargo.toml
@@ -67,6 +67,7 @@ pallet-babe = { version = "4.0.0-dev", default-features = false, path = "../../.
 pallet-bags-list = { version = "4.0.0-dev", default-features = false, path = "../../../frame/bags-list" }
 pallet-balances = { version = "4.0.0-dev", default-features = false, path = "../../../frame/balances" }
 pallet-bounties = { version = "4.0.0-dev", default-features = false, path = "../../../frame/bounties" }
+pallet-broker = { version = "0.1.0", default-features = false, path = "../../../frame/broker" }
 pallet-child-bounties = { version = "4.0.0-dev", default-features = false, path = "../../../frame/child-bounties" }
 pallet-collective = { version = "4.0.0-dev", default-features = false, path = "../../../frame/collective" }
 pallet-contracts = { version = "4.0.0-dev", default-features = false, path = "../../../frame/contracts" }
@@ -160,6 +161,7 @@ std = [
 	"pallet-bags-list/std",
 	"pallet-balances/std",
 	"pallet-bounties/std",
+	"pallet-broker/std",
 	"pallet-child-bounties/std",
 	"pallet-collective/std",
 	"pallet-contracts-primitives/std",
@@ -254,6 +256,7 @@ runtime-benchmarks = [
 	"pallet-bags-list/runtime-benchmarks",
 	"pallet-balances/runtime-benchmarks",
 	"pallet-bounties/runtime-benchmarks",
+	"pallet-broker/runtime-benchmarks",
 	"pallet-child-bounties/runtime-benchmarks",
 	"pallet-collective/runtime-benchmarks",
 	"pallet-contracts/runtime-benchmarks",
@@ -324,6 +327,7 @@ try-runtime = [
 	"pallet-bags-list/try-runtime",
 	"pallet-balances/try-runtime",
 	"pallet-bounties/try-runtime",
+	"pallet-broker/try-runtime",
 	"pallet-child-bounties/try-runtime",
 	"pallet-collective/try-runtime",
 	"pallet-contracts/try-runtime",

diff --git a/bin/node/runtime/src/lib.rs b/bin/node/runtime/src/lib.rs
@@ -35,7 +35,7 @@ use frame_support::{
 	pallet_prelude::Get,
 	parameter_types,
 	traits::{
-		fungible::ItemOf,
+		fungible::{Balanced, Credit, ItemOf},
 		tokens::{nonfungibles_v2::Inspect, GetSalary, PayFromAccount},
 		AsEnsureOriginWithArg, ConstBool, ConstU128, ConstU16, ConstU32, Currency, EitherOfDiverse,
 		EqualPrivilegeOnly, Everything, Imbalance, InstanceFilter, KeyOwnerProofSystem,
@@ -56,6 +56,7 @@ use frame_system::{
 pub use node_primitives::{AccountId, Signature};
 use node_primitives::{AccountIndex, Balance, BlockNumber, Hash, Moment, Nonce};
 use pallet_asset_conversion::{NativeOrAssetId, NativeOrAssetIdConverter};
+use pallet_broker::{CoreAssignment, CoreIndex, CoretimeInterface, PartsOf57600};
 use pallet_election_provider_multi_phase::SolutionAccuracyOf;
 use pallet_im_online::sr25519::AuthorityId as ImOnlineId;
 use pallet_nfts::PalletFeatures;
@@ -1877,6 +1878,77 @@ impl pallet_statement::Config for Runtime {
 	type MaxAllowedBytes = MaxAllowedBytes;
 }
 
+parameter_types! {
+	pub const BrokerPalletId: PalletId = PalletId(*b"py/broke");
+}
+
+pub struct IntoAuthor;
+impl OnUnbalanced<Credit<AccountId, Balances>> for IntoAuthor {
+	fn on_nonzero_unbalanced(credit: Credit<AccountId, Balances>) {
+		if let Some(author) = Authorship::author() {
+			let _ = <Balances as Balanced<_>>::resolve(&author, credit);
+		}
+	}
+}
+
+parameter_types! {
+	pub storage CoreCount: Option<CoreIndex> = None;
+	pub storage CoretimeRevenue: Option<(BlockNumber, Balance)> = None;
+}
+
+pub struct CoretimeProvider;
+impl CoretimeInterface for CoretimeProvider {
+	type AccountId = AccountId;
+	type Balance = Balance;
+	type BlockNumber = BlockNumber;
+	fn latest() -> Self::BlockNumber {
+		System::block_number()
+	}
+	fn request_core_count(_count: CoreIndex) {}
+	fn request_revenue_info_at(_when: Self::BlockNumber) {}
+	fn credit_account(_who: Self::AccountId, _amount: Self::Balance) {}
+	fn assign_core(
+		_core: CoreIndex,
+		_begin: Self::BlockNumber,
+		_assignment: Vec<(CoreAssignment, PartsOf57600)>,
+		_end_hint: Option<Self::BlockNumber>,
+	) {
+	}
+	fn check_notify_core_count() -> Option<u16> {
+		let count = CoreCount::get();
+		CoreCount::set(&None);
+		count
+	}
+	fn check_notify_revenue_info() -> Option<(Self::BlockNumber, Self::Balance)> {
+		let revenue = CoretimeRevenue::get();
+		CoretimeRevenue::set(&None);
+		revenue
+	}
+	#[cfg(feature = "runtime-benchmarks")]
+	fn ensure_notify_core_count(count: u16) {
+		CoreCount::set(&Some(count));
+	}
+	#[cfg(feature = "runtime-benchmarks")]
+	fn ensure_notify_revenue_info(when: Self::BlockNumber, revenue: Self::Balance) {
+		CoretimeRevenue::set(&Some((when, revenue)));
+	}
+}
+
+impl pallet_broker::Config for Runtime {
+	type RuntimeEvent = RuntimeEvent;
+	type Currency = Balances;
+	type OnRevenue = IntoAuthor;
+	type TimeslicePeriod = ConstU32<2>;
+	type MaxLeasedCores = ConstU32<5>;
+	type MaxReservedCores = ConstU32<5>;
+	type Coretime = CoretimeProvider;
+	type ConvertBalance = traits::Identity;
+	type WeightInfo = ();
+	type PalletId = BrokerPalletId;
+	type AdminOrigin = EnsureRoot<AccountId>;
+	type PriceAdapter = pallet_broker::Linear;
+}
+
 construct_runtime!(
 	pub struct Runtime
 	{
@@ -1950,6 +2022,7 @@ construct_runtime!(
 		MessageQueue: pallet_message_queue,
 		Pov: frame_benchmarking_pallet_pov,
 		Statement: pallet_statement,
+		Broker: pallet_broker,
 	}
 );
 
@@ -2030,6 +2103,7 @@ mod benches {
 		[pallet_bags_list, VoterList]
 		[pallet_balances, Balances]
 		[pallet_bounties, Bounties]
+		[pallet_broker, Broker]
 		[pallet_child_bounties, ChildBounties]
 		[pallet_collective, Council]
 		[pallet_conviction_voting, ConvictionVoting]

diff --git a/client/consensus/grandpa/src/communication/gossip.rs b/client/consensus/grandpa/src/communication/gossip.rs
@@ -546,9 +546,7 @@ impl<N: Ord> Peers<N> {
 		who: &PeerId,
 		update: NeighborPacket<N>,
 	) -> Result<Option<&View<N>>, Misbehavior> {
-		let Some(peer) = self.inner.get_mut(who) else {
-			return Ok(None)
-		};
+		let Some(peer) = self.inner.get_mut(who) else { return Ok(None) };
 
 		let invalid_change = peer.view.set_id > update.set_id ||
 			peer.view.round > update.round && peer.view.set_id == update.set_id ||

diff --git a/frame/broker/Cargo.toml b/frame/broker/Cargo.toml
@@ -0,0 +1,56 @@
+[package]
+name = "pallet-broker"
+version = "0.1.0"
+description = "Brokerage tool for managing Polkadot Core scheduling"
+authors = ["Parity Technologies <[email protected]>"]
+homepage = "https://substrate.io"
+edition = "2021"
+license = "Apache-2.0"
+repository = "https://github.com/paritytech/substrate"
+
+[package.metadata.docs.rs]
+targets = ["x86_64-unknown-linux-gnu"]
+
+[dependencies]
+codec = { package = "parity-scale-codec", version = "3.0.0", default-features = false, features = [ "derive"] }
+scale-info = { version = "2.0.0", default-features = false, features = ["derive"] }
+bitvec = "1"
+sp-std = { version = "8.0.0", default-features = false, path = "../../primitives/std" }
+sp-arithmetic = { version = "16.0.0", default-features = false, path = "../../primitives/arithmetic" }
+sp-core = { version = "21.0.0", default-features = false, path = "../../primitives/core" }
+sp-runtime = { version = "24.0.0", default-features = false, path = "../../primitives/runtime" }
+frame-benchmarking = { version = "4.0.0-dev", default-features = false, optional = true, path = "../benchmarking" }
+frame-support = { version = "4.0.0-dev", default-features = false, path = "../support" }
+frame-system = { version = "4.0.0-dev", default-features = false, path = "../system" }
+
+[dev-dependencies]
+sp-io = { version = "23.0.0", path = "../../primitives/io" }
+
+[features]
+default = [ "std" ]
+
+std = [
+	"codec/std",
+	"frame-benchmarking?/std",
+	"frame-support/std",
+	"frame-system/std",
+	"scale-info/std",
+	"sp-arithmetic/std",
+	"sp-core/std",
+	"sp-io/std",
+	"sp-runtime/std",
+	"sp-std/std",
+]
+
+runtime-benchmarks = [
+	"frame-benchmarking/runtime-benchmarks",
+	"frame-support/runtime-benchmarks",
+	"frame-system/runtime-benchmarks",
+	"sp-runtime/runtime-benchmarks",
+]
+
+try-runtime = [
+	"frame-support/try-runtime",
+	"frame-system/try-runtime",
+	"sp-runtime/try-runtime",
+]
diff --git a/frame/broker/README.md b/frame/broker/README.md
@@ -0,0 +1,26 @@
+# Pallet Broker
+
+Brokerage tool for managing Polkadot Core scheduling.
+
+Properly described in RFC-0001 Agile Coretime.
+
+## Implemnentation Specifics
+
+### Core Mask Bits
+
+This is 1/80th of a Polkadot Core per timeslice. Assuming timeslices are 80 blocks, then this
+indicates usage of a single core one time over a timeslice.
+
+### The Sale
+
+```nocompile
+					1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2
+0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7
+--------------------------------------------------------
+< interlude  >
+			  <                   sale                 >
+							... of which ...
+			  <  descending-price   ><   fixed-price   >
+														| <-------\
+price fixed, unsold assigned to instapool, system cores reserved -/
+```
diff --git a/frame/broker/src/adapt_price.rs b/frame/broker/src/adapt_price.rs
@@ -0,0 +1,84 @@
+// This file is part of Substrate.
+
+// Copyright (C) Parity Technologies (UK) Ltd.
+// SPDX-License-Identifier: Apache-2.0
+
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// 	http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#![deny(missing_docs)]
+
+use crate::CoreIndex;
+use sp_arithmetic::{traits::One, FixedU64};
+
+/// Type for determining how to set price.
+pub trait AdaptPrice {
+	/// Return the factor by which the regular price must be multiplied during the leadin period.
+	///
+	/// - `when`: The amount through the leadin period; between zero and one.
+	fn leadin_factor_at(when: FixedU64) -> FixedU64;
+	/// Return the correction factor by which the regular price must be multiplied based on market
+	/// performance.
+	///
+	/// - `sold`: The number of cores sold.
+	/// - `target`: The target number of cores to be sold (must be larger than zero).
+	/// - `limit`: The maximum number of cores to be sold.
+	fn adapt_price(sold: CoreIndex, target: CoreIndex, limit: CoreIndex) -> FixedU64;
+}
+
+impl AdaptPrice for () {
+	fn leadin_factor_at(_: FixedU64) -> FixedU64 {
+		FixedU64::one()
+	}
+	fn adapt_price(_: CoreIndex, _: CoreIndex, _: CoreIndex) -> FixedU64 {
+		FixedU64::one()
+	}
+}
+
+/// Simple implementation of `AdaptPrice` giving a monotonic leadin and a linear price change based
+/// on cores sold.
+pub struct Linear;
+impl AdaptPrice for Linear {
+	fn leadin_factor_at(when: FixedU64) -> FixedU64 {
+		FixedU64::from(2) - when
+	}
+	fn adapt_price(sold: CoreIndex, target: CoreIndex, limit: CoreIndex) -> FixedU64 {
+		if sold <= target {
+			FixedU64::from_rational(sold.into(), target.into())
+		} else {
+			FixedU64::one() +
+				FixedU64::from_rational((sold - target).into(), (limit - target).into())
+		}
+	}
+}
+
+#[cfg(test)]
+mod tests {
+	use super::*;
+
+	#[test]
+	fn linear_no_panic() {
+		for limit in 0..10 {
+			for target in 1..10 {
+				for sold in 0..=limit {
+					let price = Linear::adapt_price(sold, target, limit);
+
+					if sold > target {
+						assert!(price > FixedU64::one());
+					} else {
+						assert!(price <= FixedU64::one());
+					}
+				}
+			}
+		}
+	}
+}