yume/fedivet
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

init

Browse source 
Signed-off-by: eternal-flame-AD <yume@yumechi.jp>
This commit is contained in:
ゆめ 2024-10-14 23:13:16 -05:00
commit 7eed71fe68
No known key found for this signature in database
18 changed files with 3240 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
.gitignore vendored Normal file
View file

@ -0,0 +1 @@
/target

1997
Cargo.lock generated Normal file
View file

File diff suppressed because it is too large Load diff

19
Cargo.toml Normal file
View file

@ -0,0 +1,19 @@
[package]
name = "fedivet"
version = "0.1.0"
edition = "2021"
[dependencies]
async-trait = "0.1.83"
axum = "0.7.7"
axum-server = { version = "0.7.1", features = ["tokio-rustls", "rustls-pemfile", "tls-rustls"] }
clap = { version = "4.5.20", features = ["derive"] }
dashmap = "6.1.0"
env_logger = "0.11.5"
futures = "0.3.31"
log = "0.4.22"
lru = "0.12.5"
reqwest = { version = "0.12.8", features = ["stream"] }
serde = { version = "1.0.210", features = ["derive"] }
serde_json = "1.0.128"
tokio = { version = "1.40.0", features = ["rt", "rt-multi-thread", "macros", "net", "sync", "fs", "signal", "time"] }

73
LICENSE Normal file
View file

@ -0,0 +1,73 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files.
"Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions:
(a) You must give any other recipients of the Work or Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License.
You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives.
Copyright 2024 yume
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

115
README.md Normal file
View file

@ -0,0 +1,115 @@
# Fedivet
Software-defined Application Firewall for ActivityPub inboxes.
## Objective
The goal of this project is to write a dynamic trust evaluation framework for ActivityPub inboxes
that do not rely on the message content itself by default.
Currently I want to stay as unopinionated as possible and focus on the infrastructure and features, after I tested the system out I plan to roll out more opinionated and easy-to-use configurations. I also plan to allow zero-code or even zero-configuration deployments.
## What's wrong with current solutions?
### Content-based Filtering
While an effective and well-known method, particularly in the email world, they may not be as effective in the ActivityPub world, reasons include:
- Emails are usually formal and have a clear structure, while ActivityPub messages are more free-form and can contain arbitrary data, some of them even humans have trouble understanding.
- Short, confusing messages in ActivityPub are usually not spam, but they are in email.
- Content-based filtering is not effective against spam that is not in the content, such as follow requests, photos, links, etc.
- It is hard to set a hard threshold for things like "too many mentions" or "too many links" because they are not always spam. A heated debate can easily trigger these thresholds while being legitimate.
### Instance-level Blocking
Instance-level blocking is not effective against spam and abuse where the attacker has control over multiple instances (either by owning the domains themselves or by hijacking open registration instances).
Sometimes large instances are also used to send spam, and it is not always desirable to block them, especially if they are not malicious by themselves.
### Our Approach
Instead we will focus on machine-readable data that can be used to evaluate the trustworthiness of the incoming requests that considers context on each of global, instance, user and message level.
## Infrastructure
## Data Sources
In addition to the decoded inbox message itself,
we provide keyed LRU caches with TTL to retrieve additional supporting
information about the incoming requests,
such as domain history, user history, instance metadata, etc.
Multiple requests to the same key will be deduplicated and only one request will be made.
### Built-in Data Sources
- [ ] [Fediverse Observer](https://fediverse.observer)
- [ ] Domain WHOIS
- [ ] Advertised NodeInfo
- [ ] Federation Reports from Friend Instances
## Evaluators
Evaluator is an async function that takes an incoming request and either passes it through or return an response early. When all evaluators pass the request, the request will be forwarded to the backend.
Evaluators can be written as a free async closure or a struct implementing the `Evaluator` trait.
### Built-in Evaluators
- [ ] Dangling mentions: If most of the mentioned user does not appear in the parent message, it is likely spam.
- [ ] High-frequency messages with highly similar content (maybe Bloom Filter or a real Fuzzy Hash)
- [ ] Completely fresh instance sending PMs or large number of mentions
- [ ] Open Registration Instances with Abnormal User Growth
- [ ] WHOIS from known bad registrars
- [ ] Instances already blocked by Friend Instances
```rs
#[allow(clippy::unused_async)]
async fn build_state(args: &Args) -> AppState<MisskeyError> {
let mut state = AppState::new(args.backend.parse().expect("Invalid backend URL"));
let instance_history = Arc::new(LruData::sized(
&|host| async move { Ok::<_, ()>("Todo") },
512.try_into().unwrap(),
Some(Duration::from_secs(600)),
));
state.push_evaluator(Box::new(move |info: &APRequestInfo<'_>| {
let act = info.activity.as_ref().map_err(|_| ERROR_DENIED).cloned();
let instance_history = Arc::clone(&instance_history);
async move {
let act = act?;
let host = act
.actor
.as_ref()
.and_then(|s| Url::parse(s).ok())
.ok_or(ERROR_DENIED)?;
let instance = instance_history
.query(host.host_str().unwrap().to_owned())
.await;
match instance {
Ok(i) => {
log::info!("Instance history: {:?}", i);
Ok(())
}
Err(_) => Err::<(), _>(ERROR_DENIED),
}
}
}));
// let user_history = Arc::new(LruData::sized( ... ));
state.push_evaluator(Box::new(|info: &APRequestInfo<'_>| {
let act = info.activity.as_ref().map_err(|_| ERROR_DENIED).cloned();
async move {
let act = act?;
log::debug!("Activity: {:?}", act);
Ok(())
}
}));
state
}
```

45
src/client.rs Normal file
View file

@ -0,0 +1,45 @@
use std::{
net::{IpAddr, SocketAddr},
pin::Pin,
};
use dashmap::DashMap;
use tokio::time::Instant;
use crate::network::new_safe_client;
#[derive(Default, Clone)]
pub struct ClientCache {
clients: DashMap<IpAddr, (Instant, reqwest::Client)>,
}
impl ClientCache {
pub fn new() -> Self {
Self {
clients: DashMap::new(),
}
}
pub async fn with_client<'a, F, R>(&self, addr: &SocketAddr, f: F) -> R
where
F: FnOnce(reqwest::Client) -> Pin<Box<dyn std::future::Future<Output = R> + Send + 'a>>
+ 'a,
{
let client = self.clients.entry(addr.ip()).or_insert_with(|| {
(
Instant::now(),
#[allow(clippy::expect_used)]
new_safe_client(addr).expect("Failed to create client"),
)
});
f(client.1.clone()).await
}
pub fn gc(&self, dur: std::time::Duration) {
let now = Instant::now();
self.clients
.retain(|_, (created, _)| now.duration_since(*created) < dur);
}
}

43
src/evaluate/mod.rs Normal file
View file

@ -0,0 +1,43 @@
use axum::response::IntoResponse;
use reqwest::StatusCode;
use crate::{model::error::MisskeyError, APRequestInfo};
pub const ERROR_DENIED: MisskeyError = MisskeyError::new_const(
StatusCode::FORBIDDEN,
"ACTIVITY_REJECTED",
"408cffa1-273b-4b60-8cf5-ca5910856f55",
"This server cannot accept this activity.",
);
pub enum Disposition<E: IntoResponse> {
Next,
Intercept(E),
}
#[async_trait::async_trait]
pub trait Evaluator<E: IntoResponse> {
async fn evaluate<'r>(&self, info: &APRequestInfo<'r>) -> Disposition<E>;
}
impl<E: IntoResponse> Into<Disposition<E>> for Result<(), E> {
fn into(self) -> Disposition<E> {
match self {
Ok(()) => Disposition::Next,
Err(e) => Disposition::Intercept(e),
}
}
}
#[async_trait::async_trait]
impl<
E: IntoResponse,
R: Into<Disposition<E>>,
Fut: std::future::Future<Output = R> + Send,
F: Fn(&APRequestInfo<'_>) -> Fut + Send + Sync,
> Evaluator<E> for F
{
async fn evaluate<'r>(&self, info: &APRequestInfo<'r>) -> Disposition<E> {
(*self)(info).await.into()
}
}

272
src/lib.rs Normal file
View file

@ -0,0 +1,272 @@
#![feature(ip)]
#![feature(async_closure)]
#![warn(clippy::unwrap_used, clippy::expect_used)]
#![warn(unsafe_code)]
#![warn(missing_docs)]
use std::sync::Arc;
use std::{fmt::Display, net::SocketAddr};
use axum::{
body::Body,
extract::{ConnectInfo, OriginalUri, State},
http::{HeaderMap, Method, Uri},
response::{IntoResponse, Response},
};
use client::ClientCache;
use evaluate::{Disposition, Evaluator};
use futures::TryStreamExt;
use model::{ap, error::MisskeyError};
use network::stream::LimitedStream;
use network::Either;
use reqwest::{self, StatusCode};
pub(crate) mod client;
pub mod evaluate;
pub mod model;
pub(crate) mod network;
pub mod serve;
pub mod source;
#[derive(Debug)]
pub struct APRequestInfo<'r> {
pub method: &'r Method,
pub uri: &'r Uri,
pub header: &'r HeaderMap,
pub connect: &'r SocketAddr,
pub activity: &'r Result<ap::Activity, (serde_json::Value, serde_json::Error)>,
}
impl Display for APRequestInfo<'_> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{} {} from {}, ", self.method, self.uri, self.connect)?;
match self.activity {
Ok(activity) => write!(f, "activity: {:?}", activity),
Err((value, error)) => write!(f, "unparsable activity: {} ({})", value, error),
}
}
}
pub struct AppState<E: IntoResponse + 'static> {
backend: reqwest::Url,
clients: ClientCache,
inbox_stack: Vec<Box<dyn Evaluator<E> + Send + Sync>>,
}
impl<E: IntoResponse> AppState<E> {
pub fn new(backend: reqwest::Url) -> Self {
Self {
backend,
clients: ClientCache::new(),
inbox_stack: Vec::new(),
}
}
pub fn push_evaluator(&mut self, evaluator: Box<dyn Evaluator<E> + Send + Sync>) -> &mut Self {
self.inbox_stack.push(evaluator);
self
}
pub fn client_pool_ref(&self) -> &ClientCache {
&self.clients
}
}
const ERR_BAD_REQUEST: MisskeyError = MisskeyError::new_const(
StatusCode::BAD_REQUEST,
"UNPARSABLE_REQUEST",
"659c1254-1392-458e-aa77-557444031da8",
"The request is not HTTP compliant.",
);
const ERR_INTERNAL_SERVER_ERROR: MisskeyError = MisskeyError::new_const(
StatusCode::INTERNAL_SERVER_ERROR,
"INTERNAL_SERVER_ERROR",
"3a19659e-89e9-4c37-ada8-bbf620f2fc1a",
"An internal server error occurred.",
);
const ERR_SERVICE_TEMPORARILY_UNAVAILABLE: MisskeyError = MisskeyError::new_const(
StatusCode::SERVICE_UNAVAILABLE,
"SERVICE_TEMPORARILY_UNAVAILABLE",
"39992bed-f58f-484d-bb67-8c8db6f0b224",
"The service is temporarily unavailable.",
);
const ERR_PAYLOAD_TOO_LARGE: MisskeyError = MisskeyError::new_const(
StatusCode::PAYLOAD_TOO_LARGE,
"PAYLOAD_TOO_LARGE",
"8007d1b7-0eab-41b2-bb17-95f06926ba2b",
"The request payload is too large.",
);
pub struct ProxyApp<E>(std::marker::PhantomData<E>);
#[allow(clippy::too_many_arguments)]
impl<E: IntoResponse + 'static> ProxyApp<E> {
/// Pass through the request to the backend with basic error handling.
pub async fn pass_through(
method: Method,
State(app): State<Arc<AppState<E>>>,
ConnectInfo(addr): ConnectInfo<SocketAddr>,
OriginalUri(uri): OriginalUri,
header: HeaderMap,
body: Body,
) -> Result<impl IntoResponse, MisskeyError> {
let path_and_query = uri.path_and_query().ok_or(ERR_BAD_REQUEST)?;
app.clone()
.clients
.with_client(&addr, |client| {
Box::pin(async move {
let req = client
.request(
method.clone(),
app.backend
.join(&path_and_query.to_string())
.map_err(|_| ERR_INTERNAL_SERVER_ERROR)?,
)
.headers(header)
.body(reqwest::Body::wrap_stream(Box::pin(
body.into_data_stream(),
)))
.build()
.map_err(|_| ERR_INTERNAL_SERVER_ERROR)?;
let url_clone = req.url().clone();
let resp = client.execute(req).await.map_err(|e: reqwest::Error| {
log::error!(
"Failed to execute request: {} ({} {})",
e,
method,
url_clone
);
ERR_SERVICE_TEMPORARILY_UNAVAILABLE
})?;
let mut resp_builder = Response::builder().status(resp.status());
resp_builder
.headers_mut()
.ok_or(ERR_INTERNAL_SERVER_ERROR)?
.extend(resp.headers().clone());
resp_builder
.header("X-Forwarded-For", addr.to_string())
.body(Body::from_stream(resp.bytes_stream().inspect_err(|e| {
log::error!("Failed to read response: {}", e);
})))
.map_err(|e| {
log::error!("Failed to build response: {}", e);
ERR_SERVICE_TEMPORARILY_UNAVAILABLE
})
})
})
.await
}
pub async fn inbox_handler(
State(app): State<Arc<AppState<E>>>,
ConnectInfo(addr): ConnectInfo<SocketAddr>,
OriginalUri(uri): OriginalUri,
header: HeaderMap,
body: Body,
) -> Result<impl IntoResponse, Either<MisskeyError, E>> {
let path_and_query = uri.path_and_query().ok_or(Either::A(ERR_BAD_REQUEST))?;
let restricted_body_stream = LimitedStream::new(body.into_data_stream(), 32 << 20);
let mut body = Vec::new();
restricted_body_stream
.try_for_each(|chunk| {
body.extend_from_slice(&chunk);
futures::future::ready(Ok(()))
})
.await
.map_err(|e| match e {
Some(e) => {
log::error!("Failed to read body: {}", e);
Either::A(ERR_INTERNAL_SERVER_ERROR)
}
None => {
log::error!("Request payload too large");
Either::A(ERR_PAYLOAD_TOO_LARGE)
}
})?;
let decode = {
let activity_decode = serde_json::from_slice::<ap::Activity>(&body);
match activity_decode {
Ok(activity) => Ok(activity),
Err(e) => Err(match serde_json::from_slice::<serde_json::Value>(&body) {
Ok(value) => (value, e),
Err(e) => (serde_json::Value::Null, e),
}),
}
};
let info = APRequestInfo {
method: &Method::POST,
uri: &uri,
header: &header,
connect: &addr,
activity: &decode,
};
for evaluator in &app.inbox_stack {
match evaluator.evaluate(&info).await {
Disposition::Next => {}
Disposition::Intercept(e) => return Err(Either::B(e)),
}
}
app.clone()
.clients
.with_client(&addr, |client| {
Box::pin(async move {
let req = client
.request(
Method::POST,
app.backend
.join(&path_and_query.to_string())
.map_err(|_| Either::A(ERR_INTERNAL_SERVER_ERROR))?,
)
.headers(header)
.body(body)
.build()
.map_err(|_| Either::A(ERR_INTERNAL_SERVER_ERROR))?;
let url_clone = req.url().clone();
let resp = client.execute(req).await.map_err(|e: reqwest::Error| {
log::error!(
"Failed to execute request: {} ({} {})",
e,
Method::POST,
url_clone
);
Either::A(ERR_SERVICE_TEMPORARILY_UNAVAILABLE)
})?;
let mut resp_builder = Response::builder().status(resp.status());
resp_builder
.headers_mut()
.ok_or(Either::A(ERR_INTERNAL_SERVER_ERROR))?
.extend(resp.headers().clone());
Ok(resp_builder
.header("X-Forwarded-For", addr.to_string())
.body(Body::from_stream(resp.bytes_stream().inspect_err(|e| {
log::error!("Failed to read response: {}", e);
})))
.map_err(|e| {
log::error!("Failed to build response: {}", e);
Either::A(ERR_SERVICE_TEMPORARILY_UNAVAILABLE)
})?)
})
})
.await
}
}

118
src/main.rs Normal file
View file

@ -0,0 +1,118 @@
use std::sync::Arc;
use std::time::Duration;
use clap::Parser;
use fedivet::evaluate::ERROR_DENIED;
use fedivet::model::error::MisskeyError;
use fedivet::serve;
use fedivet::source::LruData;
use fedivet::APRequestInfo;
use fedivet::AppState;
use reqwest::Url;
#[derive(Parser)]
pub struct Args {
#[clap(short, long, default_value = "127.0.0.1:3001")]
pub listen: String,
#[clap(short, long, default_value = "http://web:3000")]
pub backend: String,
#[clap(long)]
pub tls_cert: Option<String>,
#[clap(long)]
pub tls_key: Option<String>,
}
#[allow(clippy::unused_async)]
async fn build_state(args: &Args) -> AppState<MisskeyError> {
let mut state = AppState::new(args.backend.parse().expect("Invalid backend URL"));
let instance_history = Arc::new(LruData::sized(
&|host| async move { Ok::<_, ()>("Todo") },
512.try_into().unwrap(),
Some(Duration::from_secs(600)),
));
state.push_evaluator(Box::new(move |info: &APRequestInfo<'_>| {
let act = info.activity.as_ref().map_err(|_| ERROR_DENIED).cloned();
let instance_history = Arc::clone(&instance_history);
async move {
let act = act?;
let host = act
.actor
.as_ref()
.and_then(|s| Url::parse(s).ok())
.ok_or(ERROR_DENIED)?;
let instance = instance_history
.query(host.host_str().unwrap().to_owned())
.await;
match instance {
Ok(i) => {
log::info!("Instance history: {:?}", i);
Ok(())
}
Err(_) => Err::<(), _>(ERROR_DENIED),
}
}
}));
// let user_history = Arc::new(LruData::sized( ... ));
state.push_evaluator(Box::new(|info: &APRequestInfo<'_>| {
let act = info.activity.as_ref().map_err(|_| ERROR_DENIED).cloned();
async move {
let act = act?;
log::debug!("Activity: {:?}", act);
Ok(())
}
}));
state
}
#[tokio::main]
async fn main() {
if std::env::var("RUST_LOG").is_err() {
std::env::set_var("RUST_LOG", "info");
}
env_logger::init();
let args = Args::parse();
let state = Arc::new(build_state(&args).await);
let (jh, handle) = serve::serve(
state.clone(),
&args.listen,
args.tls_cert.as_deref(),
args.tls_key.as_deref(),
)
.await;
let mut gc_ticker = tokio::time::interval(std::time::Duration::from_secs(300));
gc_ticker.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
let mut sigterm = tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate())
.expect("Failed to register SIGTERM handler");
tokio::select! {
_ = gc_ticker.tick() => {
state.client_pool_ref().gc(std::time::Duration::from_secs(120));
}
res = jh => {
if let Err(e) = res {
log::error!("Server error: {}", e);
}
}
_ = tokio::signal::ctrl_c() => {
log::info!("Gracefully shutting down...");
handle.graceful_shutdown(Some(std::time::Duration::from_secs(30)));
}
_ = sigterm.recv() => {
log::info!("Received SIGTERM, shutting down...");
handle.graceful_shutdown(Some(std::time::Duration::from_secs(30)));
}
}
}

47
src/model/ap.rs Normal file
View file

@ -0,0 +1,47 @@
// https://www.w3.org/TR/activitystreams-core/
use std::collections::HashMap;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(untagged)]
pub enum FlatArray<T> {
Single(T),
Multiple(Vec<T>),
}
impl IntoIterator for FlatArray<Object> {
type Item = Object;
type IntoIter = std::vec::IntoIter<Object>;
fn into_iter(self) -> Self::IntoIter {
match self {
FlatArray::Single(obj) => vec![obj].into_iter(),
FlatArray::Multiple(objs) => objs.into_iter(),
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Object {
#[serde(rename = "@context")]
pub context: String,
pub id: String,
#[serde(rename = "type")]
pub ty_: String,
pub name: Option<String>,
pub to: Option<Vec<String>>,
pub cc: Option<Vec<String>>,
pub bcc: Option<Vec<String>>,
#[serde(flatten)]
pub rest: HashMap<String, serde_json::Value>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Activity {
#[serde(flatten)]
pub meta_obj: Object,
pub actor: Option<String>,
pub object: Option<FlatArray<Object>>,
}

120
src/model/error.rs Normal file
View file

@ -0,0 +1,120 @@
use std::{
borrow::Cow,
fmt::{Debug, Display},
};
use axum::{
response::{IntoResponse, Response},
Json,
};
use reqwest::StatusCode;
use serde::Serialize;
pub trait APResponse: Debug + Display + Send + IntoResponse {}
#[derive(Debug, Clone, Serialize)]
pub struct Ignore {
reason: Cow<'static, str>,
}
impl Ignore {
pub fn new(reason: impl Into<Cow<'static, str>>) -> Self {
Self {
reason: reason.into(),
}
}
}
impl Display for Ignore {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "[IGNORE] {}", self.reason)
}
}
impl IntoResponse for Ignore {
fn into_response(self) -> Response {
(StatusCode::NO_CONTENT).into_response()
}
}
impl APResponse for Ignore {}
#[derive(Debug, Clone, Serialize)]
pub struct MisskeyError {
#[serde(skip)]
status: StatusCode,
code: &'static str,
id: Cow<'static, str>,
message: Cow<'static, str>,
}
impl MisskeyError {
pub const fn new_const(
status: StatusCode,
code: &'static str,
id: &'static str,
message: &'static str,
) -> Self {
Self {
status,
code,
id: Cow::Borrowed(id),
message: Cow::Borrowed(message),
}
}
pub fn new(
status: StatusCode,
code: &'static str,
id: impl Into<Cow<'static, str>>,
message: impl Into<Cow<'static, str>>,
) -> Self {
Self {
status,
code,
id: id.into(),
message: message.into(),
}
}
pub fn s_413(
code: &'static str,
id: impl Into<Cow<'static, str>>,
message: impl Into<Cow<'static, str>>,
) -> Self {
Self::new(StatusCode::PAYLOAD_TOO_LARGE, code, id, message)
}
pub fn s_500(
code: &'static str,
id: impl Into<Cow<'static, str>>,
message: impl Into<Cow<'static, str>>,
) -> Self {
Self::new(StatusCode::INTERNAL_SERVER_ERROR, code, id, message)
}
pub fn s_400(
code: &'static str,
id: impl Into<Cow<'static, str>>,
message: impl Into<Cow<'static, str>>,
) -> Self {
Self::new(StatusCode::BAD_REQUEST, code, id, message)
}
}
impl Display for MisskeyError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "[{}] {}: {}", self.status, self.code, self.message)
}
}
impl IntoResponse for MisskeyError {
fn into_response(self) -> Response {
#[derive(Serialize)]
struct Wrapper {
error: MisskeyError,
}
(self.status, Json(Wrapper { error: self })).into_response()
}
}
impl APResponse for MisskeyError {}

2
src/model/mod.rs Normal file
View file

@ -0,0 +1,2 @@
pub mod ap;
pub mod error;

63
src/network/mod.rs Normal file
View file

@ -0,0 +1,63 @@
use std::{
net::{IpAddr, SocketAddr},
sync::Arc,
};
use axum::response::IntoResponse;
use reqwest::{dns::Resolve, redirect::Policy};
pub enum Either<A, B> {
A(A),
B(B),
}
impl<A: IntoResponse, B: IntoResponse> IntoResponse for Either<A, B> {
fn into_response(self) -> axum::response::Response {
match self {
Either::A(a) => a.into_response(),
Either::B(b) => b.into_response(),
}
}
}
pub(crate) mod stream;
/// A Safe DNS resolver that only resolves to global addresses unless the requester itself is local.
pub struct SafeResolver(IpAddr);
impl Resolve for SafeResolver {
fn resolve(&self, name: reqwest::dns::Name) -> reqwest::dns::Resolving {
let self_is_loopback = self.0.is_loopback();
Box::pin(async move {
match tokio::net::lookup_host(format!("{}:80", name.as_str())).await {
Ok(lookup) => {
if self_is_loopback {
Ok(Box::new(lookup)
as Box<dyn Iterator<Item = std::net::SocketAddr> + Send>)
} else {
Ok(Box::new(lookup.filter(|addr| match addr {
std::net::SocketAddr::V4(a) => a.ip().is_global(),
std::net::SocketAddr::V6(a) => a.ip().is_global(),
}))
as Box<dyn Iterator<Item = std::net::SocketAddr> + Send>)
}
}
Err(e) => {
log::error!("Failed to resolve {}: {}", name.as_str(), e);
Err(e.into())
}
}
})
}
}
/// A reqwest client safe for parallel use.
pub(crate) fn new_safe_client(addr: &SocketAddr) -> reqwest::Result<reqwest::Client> {
reqwest::Client::builder()
.dns_resolver(Arc::new(SafeResolver(addr.ip())))
.connect_timeout(std::time::Duration::from_secs(10))
.timeout(std::time::Duration::from_secs(20))
.redirect(Policy::limited(4))
.tcp_keepalive(Some(std::time::Duration::from_secs(20)))
.build()
}

80
src/network/stream.rs Normal file
View file

@ -0,0 +1,80 @@
use std::{
pin::Pin,
sync::atomic::{AtomicUsize, Ordering},
task::Poll,
};
use axum::body::Bytes;
use futures::{Stream, StreamExt};
pub struct LimitedStream<T, E>
where
T: Stream<Item = Result<Bytes, E>> + Send + 'static,
{
ended: bool,
stream: Pin<Box<T>>,
limit: AtomicUsize,
}
impl<T, E> LimitedStream<T, E>
where
T: Stream<Item = Result<Bytes, E>> + Send + 'static,
{
pub fn new(stream: T, limit: usize) -> Self {
Self {
ended: false,
stream: Box::pin(stream),
limit: AtomicUsize::new(limit),
}
}
}
impl<T, E> Stream for LimitedStream<T, E>
where
T: Stream<Item = Result<Bytes, E>> + Send + 'static,
{
type Item = Result<Bytes, Option<E>>;
fn poll_next(
mut self: Pin<&mut Self>,
cx: &mut std::task::Context<'_>,
) -> Poll<Option<Self::Item>> {
match self.limit.load(Ordering::Relaxed) {
0 => {
if !self.ended {
self.ended = true;
Poll::Ready(Some(Err(None)))
} else {
Poll::Ready(None)
}
}
remaining_len => {
let p = self.stream.poll_next_unpin(cx);
match p {
Poll::Ready(Some(Ok(mut data))) => {
if data.len() > remaining_len {
self.limit.store(0, Ordering::Relaxed);
Poll::Ready(Some(Ok(data.split_to(remaining_len))))
} else {
self.limit.fetch_sub(data.len(), Ordering::Relaxed);
Poll::Ready(Some(Ok(data)))
}
}
_ => p.map_err(|e| Some(e)),
}
}
}
}
}
#[allow(unsafe_code)]
unsafe impl<T, E: Send> Send for LimitedStream<T, E> where
T: Stream<Item = Result<Bytes, E>> + Send + 'static
{
}
#[allow(unsafe_code)]
unsafe impl<T, E: Sync> Sync for LimitedStream<T, E> where
T: Stream<Item = Result<Bytes, E>> + Send + 'static
{
}

70
src/serve.rs Normal file
View file

@ -0,0 +1,70 @@
use std::{net::SocketAddr, sync::Arc};
use axum::{
response::IntoResponse,
routing::{any, post},
Router,
};
use axum_server::{tls_rustls::RustlsConfig, Handle};
use tokio::task::JoinHandle;
use crate::{APRequestInfo, AppState, ProxyApp};
#[allow(clippy::panic)]
#[allow(clippy::unwrap_used)]
#[allow(clippy::expect_used)]
/// Start the server
///
/// # Parameters
/// - `listen`: The address to listen on
/// - `tls_cert`: The path to the TLS certificate file
/// - `tls_key`: The path to the TLS key file
///
/// Use [`tokio::select`] to listen on multiple addresses
pub async fn serve<E: IntoResponse + 'static>(
state: Arc<AppState<E>>,
listen: &str,
tls_cert: Option<&str>,
tls_key: Option<&str>,
) -> (JoinHandle<Result<(), std::io::Error>>, Handle) {
let app = Router::new()
.route(
"/inbox",
post(ProxyApp::inbox_handler)
.put(ProxyApp::inbox_handler)
.patch(ProxyApp::inbox_handler),
)
.fallback(any(ProxyApp::pass_through));
let ms = app
.with_state(state)
.into_make_service_with_connect_info::<SocketAddr>();
let handle = Handle::new();
match (tls_cert, tls_key) {
(Some(cert), Some(key)) => {
let tls_config = RustlsConfig::from_pem_file(cert, key)
.await
.expect("Failed to load TLS certificate and key");
log::info!("Listening on https://{}", listen);
let server =
axum_server::bind_rustls(listen.parse().expect("invalid listen addr"), tls_config)
.handle(handle.clone())
.serve(ms);
let jh = tokio::spawn(server);
(jh, handle)
}
(None, None) => {
log::info!("Listening on http://{}", listen);
let server = axum_server::bind(listen.parse().expect("invalid listen addr"))
.handle(handle.clone())
.serve(ms);
let jh = tokio::spawn(server);
(jh, handle)
}
_ => {
panic!("TLS certificate and key must be provided together")
}
}
}

0
src/source/friend.rs Normal file
View file

174
src/source/mod.rs Normal file
View file

@ -0,0 +1,174 @@
use std::{
future::Future,
hash::Hash,
num::NonZeroUsize,
pin::Pin,
sync::{atomic::AtomicU32, Arc, Mutex, RwLock},
task::Poll,
time::{Duration, Instant},
};
use lru::LruCache;
pub mod friend;
pub mod nodeinfo;
pub struct LazyFuture<T: Clone, E, F: Future<Output = Result<T, E>>> {
future: Mutex<Pin<Box<F>>>,
ttl: Option<Duration>,
cache: RwLock<Option<(Instant, Result<T, E>)>>,
}
impl<T: Clone, E: Clone, F: Future<Output = Result<T, E>>> LazyFuture<T, E, F> {
pub fn new(future: F, ttl: Option<Duration>) -> Self {
Self {
future: Mutex::new(Box::pin(future)),
ttl,
cache: RwLock::new(None),
}
}
fn handle(self: &Arc<Self>) -> LazyFutureHandle<T, E, F> {
LazyFutureHandle {
inner: self.clone(),
}
}
fn expired(&self) -> bool {
if let Some((last, _)) = self.cache.read().unwrap().as_ref() {
if let Some(ttl) = self.ttl {
return last.elapsed() >= ttl;
}
}
false
}
fn shared_poll(&self, cx: &mut std::task::Context<'_>) -> std::task::Poll<Result<T, E>> {
if let Some((last, result)) = self.cache.read().unwrap().as_ref() {
if let Some(ttl) = self.ttl {
if last.elapsed() < ttl {
return Poll::Ready(result.clone());
}
}
}
let mut future = self.future.lock().unwrap();
match future.as_mut().poll(cx) {
Poll::Ready(result) => {
self.cache
.write()
.unwrap()
.replace((Instant::now(), result.clone()));
Poll::Ready(result.clone())
}
Poll::Pending => Poll::Pending,
}
}
}
struct LazyFutureHandle<T: Clone, E: Clone, F: Future<Output = Result<T, E>>> {
inner: Arc<LazyFuture<T, E, F>>,
}
impl<T: Clone, E: Clone, F: Future<Output = Result<T, E>>> LazyFutureHandle<T, E, F> {
pub fn inner_ref(&self) -> &LazyFuture<T, E, F> {
&self.inner
}
}
impl<T: Clone, E: Clone, F: Future<Output = Result<T, E>>> Clone for LazyFutureHandle<T, E, F> {
fn clone(&self) -> Self {
Self {
inner: self.inner.clone(),
}
}
}
impl<T: Clone, E: Clone, F: Future<Output = Result<T, E>>> Future for LazyFutureHandle<T, E, F> {
type Output = Result<T, E>;
fn poll(self: Pin<&mut Self>, cx: &mut std::task::Context<'_>) -> Poll<Self::Output> {
self.inner.shared_poll(cx)
}
}
pub struct LruData<
'a,
K: Hash,
T: Clone,
E: Clone,
F: Future<Output = Result<T, E>>,
FF: Fn(&K) -> F,
> {
factory: &'a FF,
gc_counter: AtomicU32,
ttl: Option<Duration>,
inner: Mutex<LruCache<K, Arc<LazyFuture<T, E, F>>>>,
}
impl<
'a,
K: Hash + Clone + Eq,
T: Clone,
E: Clone,
F: Future<Output = Result<T, E>>,
FF: Fn(&K) -> F,
> LruData<'a, K, T, E, F, FF>
{
pub fn sized(factory: &'a FF, size: NonZeroUsize, ttl: Option<Duration>) -> Self {
Self {
factory,
ttl,
gc_counter: AtomicU32::new(0),
inner: Mutex::new(LruCache::new(size)),
}
}
pub fn evict_expired(&self) {
let mut inner = self.inner.lock().unwrap();
let to_evict = inner
.iter()
.filter_map(|(k, f)| if f.expired() { Some(k) } else { None })
.cloned()
.collect::<Vec<_>>();
for key in to_evict {
inner.pop(&key);
}
}
fn maybe_evict(&self) {
if self
.gc_counter
.fetch_add(1, std::sync::atomic::Ordering::Relaxed)
== 512
{
self.evict_expired();
self.gc_counter
.store(0, std::sync::atomic::Ordering::Relaxed);
}
}
pub fn get_or_insert(&self, key: K) -> Arc<LazyFuture<T, E, F>> {
let factory = self.factory;
let entry = self.inner.lock().unwrap().get(&key).cloned();
match entry {
Some(future) => future,
None => {
let future = Arc::new(LazyFuture::new(factory(&key), self.ttl));
self.inner.lock().unwrap().put(key, future.clone());
future
}
}
}
pub fn query(&self, key: K) -> impl Future<Output = Result<T, E>> {
self.maybe_evict();
let future = self.get_or_insert(key);
future.handle()
}
}

1
src/source/nodeinfo.rs Normal file
View file

@ -0,0 +1 @@