replikey

Misskey logical replication tool for replication over insecure connections.

Current development status:

• Automated tested: Step 1,2,3,4,5
• Works on my machine: Step 6,7,8
• Docs: Later :)

Architecture

This is essentially a DB configuration tool and a Web PKI CA workflow and an mTLS proxy combined into one with feature flags to enable or disable each part.

Network architecture is as follows:

For postgres the tool has automation for setting up official logical replication, for redis it was just one command so didn't bother automating it.

Setup Workflow

Overview, we want to do these:

1. Create a root CA for authenticating the server and client
2. Create a server certificate for the Misskey instance
3. Create a client certificate for the replication client
4. Sign the server and client certificates
5. Set up Postgres for replication
6. Test the connection
7. Integrate the program into docker-compose
8. Start the replication

1. Create a root CA (you will be prompted for a password to encrypt the key)

replikey cert create-ca --valid-days 1825 --dn-common-name "MyInstance Replication Root Certificate Authority" -o ca-certs

2. Create a server CSR, SAN can be any number of combinations of DNS and IP addresses

If you use DNS name SAN, all SNIs you later use must match one of the DNS name or wildcard in the SAN If you use IP address SAN, all connections (supposedly) to your IP address will be considered from your server

replikey cert create-server --valid-days 365 --dn-common-name "MyInstance Production Server" -d '*.replication.myinstance.com' --ip-address 123.123.123.123 -o server-certs

3. Sign the server CSR

replikey cert sign-server-csr --valid-days 365 --ca-dir ca-certs --input-csr server-certs/server.csr --output server-certs-signed.pem

Enter password: 
CSR Params:
Serial number: 7b6a82c3d9171f7ba8fbd8973aac0146dac611dd
SAN: DNS=*.replication.myinstance.com
SAN: IP=123.123.123.123
Not before: 2024-11-02 22:43:56.751788095 +00:00:00
Not after: 2025-11-02 22:43:56.751783366 +00:00:00
Distinguished name: DistinguishedName { entries: {CommonName: Utf8String("MyInstance Production Server")}, order: [CommonName] }
Key usages: [DigitalSignature, DataEncipherment]
Extended key usages: [ServerAuth]
CRL distribution points: []
Do you want to sign this CSR? (YES/NO)
IMPORTANT: Keep this certificate or its serial number for revocation

4. Create a client CSR (for each client)

Ideally the workflow is the client should generate their own CSR and send it to you, you sign the certificate and send it back to them.

replikey cert create-client --valid-days 365 \
  --dn-common-name "MyInstance Replication Client" \
  -o client-certs

5. Sign the client CSR (for each client)

replikey cert sign-client-csr --valid-days 365 \
    --ca-dir ca-certs \
    --input-csr client-certs/client.csr \
    --output client-certs-signed.pem

BTW.0 Later if you want to revoke a certificate, generate a CRL with the following command, then pass a URL or path to the CRL(s) to any networking command via the --crl option

replikey cert generate-crl --ca-dir ca-certs --serial abcdef --serial 123456 --output revoked.crl

6. Check your certificates can communicate, this is just a zstd wrapper around rustls, so you should be able to use any TLS client or server

replikey network reverse-proxy --listen 0.0.0.0:8443 \
    --redis-sni localhost --redis-target 127.0.0.1:22 \
    --postgres-sni postgres --postgres-target 127.0.0.1:8441 \
    --cert server-signed.pem --key test-server/server.key \
    --ca test-ca/ca.pem &

# this SNI MUST match one of the dns name in the server certificate or the IP address is signed (not recommended)
replikey network forward-proxy --listen 0.0.0.0:8444 \
    --sni localhost --target localhost:8443 \
    --cert client-signed.pem --key test-client/client.key \
    --ca test-ca/ca.pem & 

ssh -p8444 localhost # this should work

7. Prepare the replication server for connection

Login to your master Misskey instance postgres and create a user for connection. You do not have to and should not grant any permissions to the replication user

CREATE ROLE replication WITH REPLICATION LOGIN ENCRYPTED PASSWORD 'password';

BTW.1 Table names for checking replication status

pg_catalog.pg_publication
pg_catalog.pg_subscription
pg_catalog.pg_stat_subscription

8. Create postgres publication on the master side

# DATABASE_URL should be _the_ connection string Misskey uses to connect to the database
replikey setup-postgres-master setup --must-not-exist --publication "my_name"
replikey setup-postgres-master drop-table --publication "my_name" -t auth_session -t password_reset_request -t access_token

9. Prepare postgres slave on the slave side

# DATABASE_URL should be any valid connection string to the master database, probably the user you created in step 7
replikey setup-postgres-slave setup --must-not-exist --subscription "my_subscription_name" --publication "my_name"

10. Set redis slave to replicate from the master

# replace REDIS_PROXY with the address of the redis TLS proxy listener
# replace PORT with the port of the redis TLS proxy listener
redis-cli 'REPLICAOF REDIS_PROXY PORT'

Integration into docker-compose:

WIP, but I have replikey service subcommand for running the proxies with environment variables or config files and optionally set up the replication on startup.