-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proxy: Figure out the plan for Hyper 1.0 migration #8733
Comments
Some of the things that are definitely worth keeping an eye on:
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions. |
this is a prepatory chore, laying more ground to facilitate upgrading our hyper dependency to the 1.0 major release. this commit adds the `deprecated` cargo feature to each of the hyper dependencies in the cargo workspace. to prevent this from introducing a large number of compiler warnings to the build, the `#[allow(deprecated)]` attribute is added to relevant expressions. these uses of deprecated interfaces will be updated in subsequent commits. broadly, these fall into a few common cases: * `hyper::client::conn::SendRequest` now has protocol specific `h1` and `h2` variants. * `hyper::server::conn::Builder` now has protocol specific `h1` and `h2` variants. * `hyper::server::conn::Http` is deprecated. * functions like `hyper::body::aggregate(..)` and `hyper::body::to_bytes(..)` are deprecated. see linkerd/linkerd2#8733 for more information on the hyper 1.0 upgrade process. Signed-off-by: katelyn martin <[email protected]>
i've been driving this work forward recently, and was kindly pointed at this issue by @olix0r. i thought i'd lay down some thoughts on the work done thus far: first, i drove a few small exploratory spikes into upgrading the proxy here, here, and here. most of the friction related to this upgrade will be found in the i've landed a few preparatory changes, in linkerd/linkerd2-proxy#3379, linkerd/linkerd2-proxy#3380, and linkerd/linkerd2-proxy#3382. these pulled assorted standalone bits of http/hyper infrastructure that define
this will let us bump 🥚 🔜 🐣 next stepsnext, we can use the linkerd/linkerd2-proxy#3405 adds the next, we'll address those deprecations, making use of the |
this is a prepatory chore, laying more ground to facilitate upgrading our hyper dependency to the 1.0 major release. this commit adds the `deprecated` cargo feature to each of the hyper dependencies in the cargo workspace. to prevent this from introducing a large number of compiler warnings to the build, the `#[allow(deprecated)]` attribute is added to relevant expressions. these uses of deprecated interfaces will be updated in subsequent commits. broadly, these fall into a few common cases: * `hyper::client::conn::SendRequest` now has protocol specific `h1` and `h2` variants. * `hyper::server::conn::Builder` now has protocol specific `h1` and `h2` variants. * `hyper::server::conn::Http` is deprecated. * functions like `hyper::body::aggregate(..)` and `hyper::body::to_bytes(..)` are deprecated. see linkerd/linkerd2#8733 for more information on the hyper 1.0 upgrade process. Signed-off-by: katelyn martin <[email protected]>
for the sake of closing the loop on the details above:
|
hyper 0.14.x provided a collection of interfaces related to collecting and aggregating request and response bodies, which were deprecated and removed in the 1.x major release. this commit updates calls to `hyper::body::to_bytes(..)` and `hyper::body::aggregate(..)`. for now, `http_body::Body` is used, but we can use `http_body_util::BodyExt` once we've bumped our hyper dependency to the 1.x major release. for more information, see: * linkerd/linkerd2#8733 * hyperium/hyper#2840 * hyperium/hyper#3020 Signed-off-by: katelyn martin <[email protected]>
* chore(app/admin): add `http-body` dependency before we address deprecated hyper interfaces related to `http_bodies`, we'll want to add this dependency so that we can call `Body::collect()`. Signed-off-by: katelyn martin <[email protected]> * refactor(app): update deprecated hyper body calls hyper 0.14.x provided a collection of interfaces related to collecting and aggregating request and response bodies, which were deprecated and removed in the 1.x major release. this commit updates calls to `hyper::body::to_bytes(..)` and `hyper::body::aggregate(..)`. for now, `http_body::Body` is used, but we can use `http_body_util::BodyExt` once we've bumped our hyper dependency to the 1.x major release. for more information, see: * linkerd/linkerd2#8733 * hyperium/hyper#2840 * hyperium/hyper#3020 Signed-off-by: katelyn martin <[email protected]> --------- Signed-off-by: katelyn martin <[email protected]>
with linkerd/linkerd2-proxy#3405 landed, we're now in a good position to begin addressing particular deprecations in the hyper 0.14.31 interface to prepare the proxy for the hyper 1.0 upgrade. linkerd/linkerd2-proxy#3411 is an example of one such change, handling calls to deprecated |
this `Server` type is not used by any tests. this commit removes it, to help facilitate the upgrade to the hyper 1.0 major release. see <linkerd/linkerd2#8733>. Signed-off-by: katelyn martin <[email protected]>
this `Server` type is not used by any tests. this commit removes it, to help facilitate the upgrade to the hyper 1.0 major release. see <linkerd/linkerd2#8733>. Signed-off-by: katelyn martin <[email protected]>
#3427) this branch contains a sequence of commits that focus on addressing deprecation warnings related to hyper::client::conn::Builder. this branch enables the backports feature, and then replaces some of these builders with the backported, http/2 specific, hyper::client::conn::http2::Builder type. relates to linkerd/linkerd2#8733. --- * chore(proxy/http): address `h2::Connection` deprecation this commit updates `Connection<B>` to use the backported, http/2 specific `SendRequest` type. this commit enables the `backports` feature flag in the `hyper` dependency. Signed-off-by: katelyn martin <[email protected]> * chore(proxy/http): address `server::tests` deprecations Signed-off-by: katelyn martin <[email protected]> * refactor(proxy/http): consolidate connect functions `connect()` is never called elsewhere. let's avoid the misdirection and move it into the body of `connect_h2()`. Signed-off-by: katelyn martin <[email protected]> --------- Signed-off-by: katelyn martin <[email protected]>
while handling the upgrade to hyper 1.0, i noticed some small changes that'd be nice to make. most importantly, with respect to linkerd/linkerd2#8733, this commit outlines `http_util::http_request()`. hyper 1.0 provides version specific `SendRequest` types for HTTP/1 and HTTP/2, so rather than try to be polymorphic across both senders, we send the request at the call site. two other commits remove `pub` attributes for functions that aren't called externally. we'll address `run_proxy()` and `connect_client()` in a subsequent PR, because the dependent tests use both HTTP/1 and HTTP/2. --- * refactor(app/test): remove `pub` from `http_util::connect_client()` this is not used elsewhere. it can be private. Signed-off-by: katelyn martin <[email protected]> * refactor(app/test): remove `pub` from `http_util::run_proxy()` Signed-off-by: katelyn martin <[email protected]> * refactor(app/test): remove `http_util::http_request()` this function abstracts over one statement, at the cost of needing to name the `SendRequest` type. because hyper's 1.0 interface provides multiple `SendRequest` types based on the HTTP version being used. to avoid needing to deal with polymorphism, and to implicitly address a function-level allowance of deprecated types, we remove this function and move this statement to the function's call sites. Signed-off-by: katelyn martin <[email protected]> * refactor(app/test): collect bodies with `String::from_utf8` this function previously called `std::str::from_utf8`, before calling `str::to_owned` on the result. because of this, there is a bit of extra gymnastics, passing a `&body[..]` slice along after collecting the body bytes. this is both more baroque and less performant. this commit updates the call, using `String::from_utf8` to collect the body, sparing a needless `to_owned()`/`clone()` call. Signed-off-by: katelyn martin <[email protected]> * docs(app/test): document `io` submodule Signed-off-by: katelyn martin <[email protected]> * refactor(app/test): remove duplicate `io` reëxport the same `pub use` bundle is found in the parent `lib.rs`. this removes it, and refers to the same `mod io {}` defined above. Signed-off-by: katelyn martin <[email protected]> * review(app/inbound): use `ServiceExt::oneshot(..)` #3428 (comment) we can simplify these statements sending requests, by using `ServiceExt::oneshot(..)`. Signed-off-by: katelyn martin <[email protected]> --------- Signed-off-by: katelyn martin <[email protected]>
this addresses hyper 1.0 deprecations in the server side of the inbound proxy's http unit test suite logic. see <linkerd/linkerd2#8733> for more information. the client end of this change ends up being slightly involved, due to changes that will need to be made in `linkerd_app_test::http_util`. accordingly, those deprecations will be addressed in a subsequent commit. Signed-off-by: katelyn martin <[email protected]>
note: this commit will not compile, code changes are intentionally elided from this commit. this commit upgrades hyper, http, tonic, prost, related dependencies, and their assorted cargo features. see <linkerd/linkerd2#8733>. --- squash: chore(deps): add hyper-util workspace dependency chore(deps): add http-body-util workspace dependency chore(deps): upgrade linkerd2-proxy-api this commit represents main as of linkerd/linkerd2-proxy-api#421. Signed-off-by: katelyn martin <[email protected]>
note: this commit will not compile, code changes are intentionally elided from this commit. this commit upgrades hyper, http, tonic, prost, related dependencies, and their assorted cargo features. see <linkerd/linkerd2#8733>. --- squash: chore(deps): add hyper-util workspace dependency chore(deps): add http-body-util workspace dependency chore(deps): upgrade linkerd2-proxy-api this commit represents main as of linkerd/linkerd2-proxy-api#421. Signed-off-by: katelyn martin <[email protected]>
this commit removes a redundant set of trait bounds from `linkerd_tls::Client<L, C>`'s `tower::Service<T>` implementation. this client type is generic over a `C`-typed `MakeConnection`. this trait is effectively an alias for particular services, and already by definition is prerequisite upon `Connection` responses that are an asynchronous reader/writer. see the definition of the trait, here: ```rust // linkerd/stack/src/connect.rs pub trait MakeConnection<T> { /// An I/O type that represents a connection to the remote endpoint. type Connection: AsyncRead + AsyncWrite; /// Metadata associated with the established connection. type Metadata; type Error: Into<Error>; type Future: Future<Output = Result<(Self::Connection, Self::Metadata), Self::Error>>; /// Determines whether the connector is ready to establish a connection. fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>>; /// Establishes a connection. fn connect(&mut self, t: T) -> Self::Future; // contd... } ``` thus, we can remove these bounds from the tls client. the connection is already, by virtue of `C: MakeConnection`, an `AsyncRead + AsyncWrite` type. see linkerd/linkerd2#8733. Signed-off-by: katelyn martin <[email protected]>
this commit removes a redundant set of trait bounds from `linkerd_tls::Client<L, C>`'s `tower::Service<T>` implementation. this client type is generic over a `C`-typed `MakeConnection`. this trait is effectively an alias for particular services, and already by definition is prerequisite upon `Connection` responses that are an asynchronous reader/writer. see the definition of the trait, here: ```rust // linkerd/stack/src/connect.rs pub trait MakeConnection<T> { /// An I/O type that represents a connection to the remote endpoint. type Connection: AsyncRead + AsyncWrite; /// Metadata associated with the established connection. type Metadata; type Error: Into<Error>; type Future: Future<Output = Result<(Self::Connection, Self::Metadata), Self::Error>>; /// Determines whether the connector is ready to establish a connection. fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>>; /// Establishes a connection. fn connect(&mut self, t: T) -> Self::Future; // contd... } ``` thus, we can remove these bounds from the tls client. the connection is already, by virtue of `C: MakeConnection`, an `AsyncRead + AsyncWrite` type. see linkerd/linkerd2#8733. Signed-off-by: katelyn martin <[email protected]>
note: this commit will not compile, code changes are intentionally elided from this commit. this commit upgrades hyper, http, tonic, prost, related dependencies, and their assorted cargo features. see <linkerd/linkerd2#8733>. --- squash: chore(deps): add hyper-util workspace dependency chore(deps): add http-body-util workspace dependency chore(deps): upgrade linkerd2-proxy-api this commit represents main as of linkerd/linkerd2-proxy-api#421. Signed-off-by: katelyn martin <[email protected]>
see linkerd/linkerd2#8733 for more information. in hyper 1.x, `Incoming` bodies do not provide a `Default` implementation. compare the trait implementations here: * https://docs.rs/hyper/0.14.31/hyper/body/struct.Body.html#impl-Default-for-Body * https://docs.rs/hyper/latest/hyper/body/struct.Incoming.html#trait-implementations this commit removes these bounds from the `linkerd_app::trace_collector::otel_collector::SpanExporter<T, S>` and other connected functions. Signed-off-by: katelyn martin <[email protected]>
see linkerd/linkerd2#8733 for more information. in hyper 1.x, `Incoming` bodies do not provide a `Default` implementation. compare the trait implementations here: * https://docs.rs/hyper/0.14.31/hyper/body/struct.Body.html#impl-Default-for-Body * https://docs.rs/hyper/latest/hyper/body/struct.Incoming.html#trait-implementations this commit removes these bounds from the `linkerd_app::trace_collector::otel_collector::SpanExporter<T, S>` and other connected functions. Signed-off-by: katelyn martin <[email protected]>
linkerd/linkerd2#8733 for more information. see also, #3651 for another related pull request. in hyper 1.x, `Incoming` bodies do not provide a `Default` implementation. compare the trait implementations here: * https://docs.rs/hyper/0.14.31/hyper/body/struct.Body.html#impl-Default-for-Body * https://docs.rs/hyper/latest/hyper/body/struct.Incoming.html#trait-implementations this commit removes these bounds from `linkerd_proxy_identity_client::Certify<C, S>`. Signed-off-by: katelyn martin <[email protected]>
see linkerd/linkerd2#8733 for more information. see also, #3651 and #3653 for some related pull requests. in hyper 1.x, `Incoming` bodies do not provide a `Default` implementation. compare the trait implementations here: * https://docs.rs/hyper/0.14.31/hyper/body/struct.Body.html#impl-Default-for-Body * https://docs.rs/hyper/latest/hyper/body/struct.Incoming.html#trait-implementations this commit removes `Default` bounds from policy lookup in the inbound proxy. Signed-off-by: katelyn martin <[email protected]>
see linkerd/linkerd2#8733 for more information. see also, #3651 #3653, and #3654 for some related pull requests. in hyper 1.x, `Incoming` bodies do not provide a `Default` implementation. compare the trait implementations here: * https://docs.rs/hyper/0.14.31/hyper/body/struct.Body.html#impl-Default-for-Body * https://docs.rs/hyper/latest/hyper/body/struct.Incoming.html#trait-implementations this commit removes `Default` bounds from policy lookup in the outbound proxy. this means that in `linkerd-app`, we can invoke `Outbound::build_policies()` when using hyper 1.x (_see #3504_) Signed-off-by: katelyn martin <[email protected]>
see linkerd/linkerd2#8733 for more information. see also, #3651 #3653, and #3654 for some related pull requests. in hyper 1.x, `Incoming` bodies do not provide a `Default` implementation. compare the trait implementations here: * https://docs.rs/hyper/0.14.31/hyper/body/struct.Body.html#impl-Default-for-Body * https://docs.rs/hyper/latest/hyper/body/struct.Incoming.html#trait-implementations this commit removes `Default` bounds from policy lookup in the outbound proxy. this means that in `linkerd-app`, we can invoke `Outbound::build_policies()` when using hyper 1.x (_see #3504_) Signed-off-by: katelyn martin <[email protected]>
see linkerd/linkerd2#8733 for more information. see also, #3651 #3653, and #3654 for some related pull requests. in hyper 1.x, `Incoming` bodies do not provide a `Default` implementation. compare the trait implementations here: * https://docs.rs/hyper/0.14.31/hyper/body/struct.Body.html#impl-Default-for-Body * https://docs.rs/hyper/latest/hyper/body/struct.Incoming.html#trait-implementations this commit removes `Default` bounds from `Resolve<S>`. this means that in `linkerd-app`, we can invoke `new_recover_default()` when using hyper 1.x (_see #3504_) Signed-off-by: katelyn martin <[email protected]>
see linkerd/linkerd2#8733 for more information. see also, #3651 #3653, #3654, and #3655 for some related pull requests. in hyper 1.x, `Incoming` bodies do not provide a `Default` implementation. compare the trait implementations here: * https://docs.rs/hyper/0.14.31/hyper/body/struct.Body.html#impl-Default-for-Body * https://docs.rs/hyper/latest/hyper/body/struct.Incoming.html#trait-implementations this commit removes `Default` bounds from `Client<R, S>` used to create watches on service profiles. Signed-off-by: katelyn martin <[email protected]>
see linkerd/linkerd2#8733 for more information. in hyper 1.x, `Incoming` bodies do not provide a `Default` implementation. compare the trait implementations here: * https://docs.rs/hyper/0.14.31/hyper/body/struct.Body.html#impl-Default-for-Body * https://docs.rs/hyper/latest/hyper/body/struct.Incoming.html#trait-implementations this commit removes these bounds from the `linkerd_app::trace_collector::otel_collector::SpanExporter<T, S>` and other connected functions. Signed-off-by: katelyn martin <[email protected]>
see linkerd/linkerd2#8733 for more information. see also, #3651 and #3653 for some related pull requests. in hyper 1.x, `Incoming` bodies do not provide a `Default` implementation. compare the trait implementations here: * https://docs.rs/hyper/0.14.31/hyper/body/struct.Body.html#impl-Default-for-Body * https://docs.rs/hyper/latest/hyper/body/struct.Incoming.html#trait-implementations this commit removes `Default` bounds from policy lookup in the inbound proxy. Signed-off-by: katelyn martin <[email protected]>
see linkerd/linkerd2#8733 for more information. see also, #3651 #3653, and #3654 for some related pull requests. in hyper 1.x, `Incoming` bodies do not provide a `Default` implementation. compare the trait implementations here: * https://docs.rs/hyper/0.14.31/hyper/body/struct.Body.html#impl-Default-for-Body * https://docs.rs/hyper/latest/hyper/body/struct.Incoming.html#trait-implementations this commit removes `Default` bounds from policy lookup in the outbound proxy. this means that in `linkerd-app`, we can invoke `Outbound::build_policies()` when using hyper 1.x (_see #3504_) Signed-off-by: katelyn martin <[email protected]>
see linkerd/linkerd2#8733 for more information. see also, #3651 #3653, and #3654 for some related pull requests. in hyper 1.x, `Incoming` bodies do not provide a `Default` implementation. compare the trait implementations here: * https://docs.rs/hyper/0.14.31/hyper/body/struct.Body.html#impl-Default-for-Body * https://docs.rs/hyper/latest/hyper/body/struct.Incoming.html#trait-implementations this commit removes `Default` bounds from `Resolve<S>`. this means that in `linkerd-app`, we can invoke `new_recover_default()` when using hyper 1.x (_see #3504_) Signed-off-by: katelyn martin <[email protected]>
see linkerd/linkerd2#8733 for more information. see also, #3651 #3653, #3654, and #3655 for some related pull requests. in hyper 1.x, `Incoming` bodies do not provide a `Default` implementation. compare the trait implementations here: * https://docs.rs/hyper/0.14.31/hyper/body/struct.Body.html#impl-Default-for-Body * https://docs.rs/hyper/latest/hyper/body/struct.Incoming.html#trait-implementations this commit removes `Default` bounds from `Client<R, S>` used to create watches on service profiles. Signed-off-by: katelyn martin <[email protected]>
linkerd/linkerd2#8733 for more information. see also, #3651 for another related pull request. in hyper 1.x, `Incoming` bodies do not provide a `Default` implementation. compare the trait implementations here: * https://docs.rs/hyper/0.14.31/hyper/body/struct.Body.html#impl-Default-for-Body * https://docs.rs/hyper/latest/hyper/body/struct.Incoming.html#trait-implementations this commit removes these bounds from `linkerd_proxy_identity_client::Certify<C, S>`. Signed-off-by: katelyn martin <[email protected]>
note: this commit will not compile, code changes are intentionally elided from this commit. this commit upgrades hyper, http, tonic, prost, related dependencies, and their assorted cargo features. see <linkerd/linkerd2#8733>. --- squash: chore(deps): add hyper-util workspace dependency chore(deps): add http-body-util workspace dependency chore(deps): upgrade linkerd2-proxy-api this commit represents main as of linkerd/linkerd2-proxy-api#421. Signed-off-by: katelyn martin <[email protected]>
see #3651 and linkerd/linkerd2#8733. #3651 missed this unused trait bound, which we want to loosen to account for changes in hyper's api. Signed-off-by: katelyn martin <[email protected]>
note: this commit will not compile, code changes are intentionally elided from this commit. this commit upgrades hyper, http, tonic, prost, related dependencies, and their assorted cargo features. see <linkerd/linkerd2#8733>. --- squash: chore(deps): add hyper-util workspace dependency chore(deps): add http-body-util workspace dependency chore(deps): upgrade linkerd2-proxy-api this commit represents main as of linkerd/linkerd2-proxy-api#421. Signed-off-by: katelyn martin <[email protected]>
Hyper is planning a major 1.0 milestone that will impact many of their public APIs and, therefore, the proxy. We should get a better understanding of the planned changes so that we can begin to scope and plan the required proxy changes (and so that we can provide meaningful feedback before the APIs are finalized).
"here's some links!" -kate 💐 🧢
issues and pull requests related to upgrading linkerd2 to hyper 1.0:
linkerd-http-version
crate linkerd2-proxy#3379linkerd-http-insert
crate linkerd2-proxy#3380Body
middleware types linkerd2-proxy#3382deprecated
feature flag linkerd2-proxy#3405max_pending_accept_reset_streams()
hyperium/hyper#3796Server
interfaces linkerd2-proxy#3421hyper::client::conn::Builder
deprecations linkerd2-proxy#3427linkerd-app-test
linkerd2-proxy#3428server::conn::Http
deprecations linkerd2-proxy#3432connect_and_accept_http1(..)
function linkerd2-proxy#3461ServeHttp<N>
linkerd2-proxy#3459SendRequest
links linkerd2-proxy#3465hyper::body::HttpBody
linkerd2-proxy#3467BoxBody::empty()
creates an empty body linkerd2-proxy#34680.14.28
to0.14.32
#13492Builder
keep-alive interfaces hyperium/hyper#3816hyper::Body
withBoxBody
linkerd2-proxy#3479Receiver::poll_recv(..)
method tokio-rs/tokio#7059PeekTrailersBody<B>
only peeks empty bodies linkerd2-proxy#3509hyper::Body
linkerd2-proxy#3515linkerd-http-upgrade
linkerd2-proxy#3531Http11Upgrade
isClone
linkerd2-proxy#3540PeekTrailersBody<B>
linkerd2-proxy#3556is_end_stream()
is true for empty bodies linkerd2-proxy#3558PeekTrailersBody<B>
withFrame<T>
linkerd2-proxy#3559ReplayBody
tests toFrame<T>
linkerd2-proxy#3564ReplayBody
tests toFrame<T>
linkerd2-proxy#3567Body::data()
calls linkerd2-proxy#3573Poll::{map_ok, map_err}
linkerd2-proxy#3586ReplayBody<B>
withFrame<T>
linkerd2-proxy#3598MockBody
test body linkerd2-proxy#3611ForwardCompatibleBody<B>
linkerd2-proxy#3614Default
bounds linkerd2-proxy#3651Default
bounds linkerd2-proxy#3653Default
bounds linkerd2-proxy#3654Default
bounds linkerd2-proxy#3655Default
bounds linkerd2-proxy#3656Default
bounds linkerd2-proxy#3657Default
bound linkerd2-proxy#3660in particular, this PR:
The text was updated successfully, but these errors were encountered: