Malcolm

A transparent caching HTTP/HTTPS proxy optimized for package manager traffic.

Features

• HTTP/HTTPS forward proxy with CONNECT support
• Blake3-based content-addressable cache storage
• Prometheus metrics endpoint (/metrics)
• TLS termination (custom certs and Let's Encrypt auto-cert)
• Transparent proxy via TPROXY/REDIRECT
• Request coalescing via singleflight
• Bandwidth throttling for large downloads
• Built-in package manager detection:
  • APT (Debian/Ubuntu)
  • YUM/DNF (RHEL/Fedora/Rocky)
  • Pacman (Arch Linux)
  • APK (Alpine Linux)
• Configurable mirror fallback and health checking

Quick Start

git clone https://git.lan.thwap.org/thwap/malcolm.git
cd malcolm
make build
./malcolm

The proxy starts on :8080 with cache directory ./cache.

Point your browser or package manager at the proxy:

export http_proxy=http://127.0.0.1:8080
export https_proxy=http://127.0.0.1:8080

Configuration

All configuration is via environment variables:

Variable	Default	Description
LISTEN_HOST	0.0.0.0	Bind address
LISTEN_PORT	8080	Listen port
CACHE_DIR	./cache	Cache storage path
CACHE_MAX_SIZE	1073741824	Max cache size (bytes, 1 GiB)
CACHE_MAX_ENTRIES	10000	Max cached entries
CACHE_DEFAULT_TTL	3600	Default TTL in seconds
LOG_LEVEL	info	Log level (debug, info, warn, error)
LOG_FORMAT	text	Log format (text, json)
TLS_CERT_FILE	``	TLS cert path (custom cert mode)
TLS_KEY_FILE	``	TLS key path (custom cert mode)
AUTO_TLS	false	Enable Let's Encrypt auto-cert
TLS_DOMAIN	``	Domain for Let's Encrypt

TLS Modes

The server operates in one of three TLS modes:

1. Plain HTTP (default) — no TLS, set http_proxy accordingly
2. Custom certificates — set TLS_CERT_FILE and TLS_KEY_FILE
3. Let's Encrypt auto — set AUTO_TLS=true and TLS_DOMAIN=your.domain.com

Prometheus Metrics

Available at http://<proxy>:8080/metrics:

• malcolm_cache_hits_total
• malcolm_cache_misses_total
• malcolm_request_duration_seconds
• malcolm_requests_total
• malcolm_cache_size_bytes
• malcolm_active_connections

Installation

From source

make build
sudo make install

Cross-compilation

make build-linux-amd64   # amd64 binary
make build-linux-arm64   # ARM64 binary
make build-all           # both

Transparent Proxy

See docs/transparent-proxy.md for iptables and nftables rules.

Package Manager Configuration

See docs/package-managers.md for configuring APT, YUM/DNF, Pacman, and APK to use the proxy.

Performance

• Concurrent request pool with configurable worker count
• Request coalescing prevents duplicate origin fetches
• Blake3 hashing for fast content addressing
• Atomic file writes prevent partial cache entries
• LRU, size-based, and TTL-based cache eviction