Conversation
I added an extractor `RemoteAddr` in the `extractors.rs`. This extractor works by getting an extension from the HTTP headers which contains the remote address. For this extension to be set, axum has to be started with `handler.into_make_service_with_connect_info`. The extractor then provides the IP address. Furthermore, when a proxied connection was declared using one of the following headers, the proxied IP will be used: - CF-Connecting-IP - CloudFront-Viewer-Address - Fly-Client-IP - Forwarded - X-Forwarded-For - True-Client-IP - X-Envoy-External-Address - X-Real-Ip In order to implement proxied HTTP header detection the `client_ip` crate was used. In order to comply with the orphan rules, `cot-core` was given access to `tokio` via `tokio.workspace = true`.
I added support for configuring a proxy via a new trusted_proxy field in the configuration file. The field supports CIDR notated subnets and several headers. I cut the `client-ip` dependency. Currently supported headers are: - Forwarded - X-Forwarded-For - CF-Connecting-IP - X-Real-IP
a8b0157 to
c861d03
Compare
|
|
||
| #[cfg(test)] | ||
| mod tests { | ||
| use serde::Deserialize; |
| use crate::request::extractors::{FromRequest, Json, Path, UrlQuery}; | ||
|
|
||
| use serde::Deserialize; | ||
| use std::net::{Ipv4Addr, Ipv6Addr}; |
There was a problem hiding this comment.
We don't use this import anywhere, no?
| pub struct IpWithSubnet { | ||
| ip: IpAddr, | ||
| mask: Option<u8>, | ||
| } | ||
|
|
||
| impl Serialize for IpWithSubnet { | ||
| fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error> | ||
| where | ||
| S: serde::Serializer, | ||
| { | ||
| if let Some(mask) = self.mask { | ||
| serializer.serialize_str(&format!("{}/{}", self.ip, mask)) | ||
| } else { | ||
| serializer.serialize_str(&self.ip.to_string()) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| struct IpWithSubnetVistor; | ||
|
|
||
| impl Visitor<'_> for IpWithSubnetVistor { | ||
| type Value = String; | ||
|
|
||
| fn expecting(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { | ||
| formatter.write_str("a CIDR notation") | ||
| } |
There was a problem hiding this comment.
I'm not sure about this; doing this manually is a bit too finicky and error-prone. If the goal is to support Cidr, we should delegate this to a battle-tested dependency like ipnet
| /// An extractor that extracts the IP address of the remote. | ||
| /// This automatically checks for proxy IP headers and contains their IP if one such is specified. | ||
| /// | ||
| /// # Examples | ||
| /// ```rust | ||
| /// pub async fn example_handler(ip: RemoteAddr) -> cot::Result<Html> { | ||
| /// dbg!(ip.ip()); // Prints the IP as a debug statement | ||
| /// dbg!(ip.closest_ip()); // Prints the closest IP as a debug statement | ||
| /// ///... | ||
| /// } | ||
| /// ``` |
There was a problem hiding this comment.
The extractor should live in requests/extractor.rs
| pub struct ProxyConfig { | ||
| trusted_proxies: Vec<IpWithSubnet>, | ||
| trusted_headers: Vec<String>, | ||
| } | ||
|
|
||
| impl ProxyConfig { | ||
| #[must_use] | ||
| /// Get the proxies that have been configured to be trusted. | ||
| pub fn get_trusted_proxies(&self) -> Vec<IpWithSubnet> { | ||
| self.trusted_proxies.clone() | ||
| } | ||
|
|
||
| #[must_use] | ||
| /// Get the proxy headers that have been configured to be trusted. | ||
| pub fn get_trusted_headers(&self) -> Vec<HeaderName> { | ||
| self.trusted_headers | ||
| .iter() | ||
| .filter_map(|h| HeaderName::from_str(h).ok()) | ||
| .collect() | ||
| } | ||
|
|
||
| #[must_use] | ||
| /// Create a new [`ProxyConfig`] with trusted proxies and trusted headers. | ||
| pub fn new(proxies: Vec<IpWithSubnet>, headers: Vec<String>) -> Self { | ||
| Self { | ||
| trusted_proxies: proxies, |
There was a problem hiding this comment.
I think we should aim for some consistency in the API here. The config struct here should be a builder(we do that by using the derive_builder::Builder macro).
We should aim for something like this:
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize, derive_builder::Builder)]
#[builder(build_fn(name = "build_impl"))]
pub struct ClientIpConfig {
#[serde(default)]
#[builder(default)]
pub trusted_proxies: TrustedProxies,
#[serde(default = "default_client_ip_headers")]
#[builder(default = "default_client_ip_headers()")]
pub headers: Vec<ClientIpHeader>,
}
fn default_client_ip_headers() -> Vec<ClientIpHeader> {
vec![ClientIpHeader::XForwardedFor]
}
impl Default for ClientIpConfig {
fn default() -> Self {
Self {
trusted_proxies: TrustedProxies::default(),
headers: default_client_ip_headers(),
}
}
}While we're here, I think we need some kind of type-safety with deserializing proxy IPs and headers. For headers, we want to provide an enum(ClientIpHeader in the case of the example above), with supported headers (x-forwarded-for, true-client-ip, etc) as variant but also an escape hatch for custom headers. This should look somewhat like this:
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ClientIpHeader {
/// `X-Forwarded-For`, the de facto standard, a comma-separated list of
/// hops, closest-to-origin first.
XForwardedFor,
/// `X-Real-IP`, commonly set by nginx, a single address.
XRealIp,
/// `True-Client-IP`, used by Akamai and Cloudflare Enterprise.
TrueClientIp,
/// `CF-Connecting-IP` set by Cloudflare.
CfConnectingIp,
/// Any other header name.
Custom(HeaderName),
}I would also expect TrustedProxies (in the example above) to look something like this:
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct TrustedProxies(Vec<TrustedProxyEntry>);
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum TrustedProxyEntry {
/// Trusts every proxy address. (we should have a security note that states the risk of using this)
Any,
/// Trusts every address in this network (a single address is a network
/// with a full-length prefix).
Network(ipnet::IpNet),
}In terms of design, i would expect to abstract the header and trusted proxy structs into the cot-core crate (cot-core/src/request/client_ip.rs or equivalent) since they dont need axum or the project config
There was a problem hiding this comment.
Wouldn't TrustedProxyEntry::Any be equivalent to the user having 0.0.0.0/0 and ::/0?
There was a problem hiding this comment.
These belong to 2 different families (IPv4 and IPv6). 0.0.0.0/0 means allowing all IPv4 addresses and doesn't apply to IPv6, whereas ::/0 means allowing all IPv6 addresses and doesn't apply to IPv4. We should allow people to do that. In theory, TrustedProxyEntry::Any is equivalent to (or desugars to) specifying ["0.0.0.0/0", "::/0"]. So specifying 0.0.0.0/0 or ::/0 should not collapse into TrustedProxyEntry::Any.
| @@ -0,0 +1,586 @@ | |||
| use axum::{extract::connect_info::Connected, serve::IncomingStream}; | |||
There was a problem hiding this comment.
we should improve the abstraction design here. Core components, code, or data structures that do not integrate with third-party dependencies in cot (like axum) or ProjectConfig should move into cot-core. In other words, the contents of this file should move into the appropriate modules
| impl<'a> Connected<IncomingStream<'a, tokio::net::TcpListener>> for RemoteAddr { | ||
| fn connect_info(stream: IncomingStream<'a, tokio::net::TcpListener>) -> Self { | ||
| let closest_ip = stream.remote_addr().ip(); | ||
| RemoteAddr { | ||
| direct: closest_ip, | ||
| proxied: None, | ||
| } | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
why do we need this?
There was a problem hiding this comment.
Connected<IncomingStream<'a, TcpListener>> must be implemented for this to work:
axum::serve(
listener,
handler.into_make_service_with_connect_info::<RemoteAddr>(),
)There was a problem hiding this comment.
It is essentially so that axum can give me the IP address in the first place.
- I moved the code into different files and some to cot-core - I removed my own IpWithSubnet in favour of IpNet - The code now uses an enum for handling headers - I fixed some mistakes that made the CI/CD fail
bb0ba80 to
324d7c7
Compare
m4tx
left a comment
There was a problem hiding this comment.
Hey, thanks for this, and for your patience waiting for the review! I've added a few comments - when these will be addressed, I think we can merge this!
|
|
||
| #[derive(Debug, Clone, PartialEq, Eq)] | ||
| /// A header used for deriving the IP of the client. | ||
| pub enum ClientIpHeader { |
There was a problem hiding this comment.
I'm wondering if we need an enum for this. Perhaps just a regular String (or some newtype wrapping a String) would do? We could have the common values listed in the docs, and we wouldn't limit the users to a predefined set of values.
EDIT: ahh, I see the behavior is in fact different for different headers. I'm wondering if we should have a "custom" enum field, though, to specify a header name that would be used for when a single address arrives. This would allow us to remove XRealIp, TrueClientIp, and CfConnectingIp, which would be nice considering they are really focused on specific reverse proxies and are not really standard (and they seem to share the same behavior).
There was a problem hiding this comment.
Yeah this is reasonable. I changed it in an upcoming commit.
| //! the Cot framework and for building custom extensions. | ||
|
|
||
| mod body; | ||
| pub mod remote_addr; |
There was a problem hiding this comment.
Do you think there's a value in making this public? I'm not exactly sure if we want to expose this, as the usefulness of this for end-users is probably pretty limited, and we'd need to keep the public API promises if it's pub.
There was a problem hiding this comment.
If I understand correctly, this should only be public inside the workspace, no? So as long as it doesn't get exported from cot/src/lib.rs it shouldn't be public?
There was a problem hiding this comment.
Ah right, this is cot-core, not cot. Yeah, then it should be fine.
| tower-sessions = { workspace = true, features = ["memory-store"] } | ||
| tracing.workspace = true | ||
| url = { workspace = true, features = ["serde"] } | ||
| ipnet.workspace = true |
There was a problem hiding this comment.
It's generally discouraged to add more dependencies to the main crates: https://github.com/cot-rs/cot/blob/master/CONTRIBUTING.md#dependencies
However, I feel like it's okay to keep this one, as its functionality is not trivial. If you feel like it, though, feel free to reimplement its functionality in this PR, or another one!
There was a problem hiding this comment.
This one I used because it was previously suggested to me as a replacement for doing IP subnet calculations myself.
There was a problem hiding this comment.
Ah, I see; sorry for confusion then.
- I formatted the code - I removed an unused import - I changed the enum to have a Custom field instead of several non-standard
28b27a5 to
059fc2c
Compare
There was a problem hiding this comment.
Looks good to me, but seems like the clippy and tests are failing.
For clippy, seems like you'll need to use std::future::ready in your remote address extractor (or bypass the clippy lint as we do in other extractors now), and add docs to the fields inside ClientIpConfig so that the builder can pick them up.
For the tests, seems like you need to update Cargo.lock.
@ElijahAhianyo do you want to have another look?
I added an extractor
RemoteAddrin theextractors.rs. This extractor works by getting an extension from the HTTP headers which contains the remote address.For this extension to be set, axum has to be started with
handler.into_make_service_with_connect_info.The extractor then provides the IP address. Furthermore, when a proxied connection was declared using one of the following headers, the proxied IP will be used:
In order to implement proxied HTTP header detection the
client_ipcrate was used.In order to comply with the orphan rules,
cot-corewas given access totokioviatokio.workspace = true.Related issue or discussion
Description
Type of change
Checklist
just test-all)just clippy)cargo fmt)