pget — Parallel file downloader

pget downloads files over HTTP and FTP using multiple parallel connections, with resume support, rate limiting, checksum verification, and a real-time progress bar.

Installation

# From source
git clone https://git.lan.thwap.org/thwap/pget.git
cd pget
go build -o pget ./cmd/pget

Usage

pget [options] <url>

Options

Flag	Default	Description
-n	3	Number of concurrent connections
-o	(from URL)	Output file path
-c	false	Resume partial download
-t	30s	Request/connection timeout
-r	3	Max retries per chunk
--rate	(unlimited)	Download speed limit (e.g. 500k, 2M, 1G)
--checksum	(none)	Expected SHA256 (64 hex) or MD5 (32 hex) checksum
--header	(none)	Custom HTTP header, can be repeated
-A	pget/1.0	User-Agent header
-q	false	Suppress non-error output
-v	false	Verbose output (worker details, server caps)

Examples

Basic download:

pget https://example.com/file.iso

File gets saved as file.iso (derived from URL). If it already exists, saves as file-1.iso.

With output path:

pget -o myfile.iso https://example.com/file.iso

Parallel download (5 connections):

pget -n 5 https://example.com/large-file.iso

Resume partial download:

pget -c https://example.com/large-file.iso

Rate-limited download (500 KB/s):

pget --rate=500k https://example.com/file.iso

Verify checksum after download:

pget --checksum=abcd1234... https://example.com/file.iso

Supports both SHA256 (64 hex chars) and MD5 (32 hex chars).

Custom HTTP headers:

pget --header "Authorization: Bearer token" https://example.com/file.iso
pget --header "Cookie: session=abc" --header "X-Custom: value" https://example.com/file.iso

FTP download:

pget ftp://example.com/file.iso
pget ftp://user:pass@example.com/file.iso
pget ftps://example.com/file.iso

FTPS (TLS) is supported via ftps:// scheme. Credentials resolve from URL, ~/.netrc, or anonymous fallback.

Verbose output:

pget -v https://example.com/file.iso

Shows worker retries, completions, and server capabilities (file size, range support).

Configuration

Create ~/.pgetrc to set persistent defaults:

concurrency = 5
timeout = 60s
rate = 1M
user-agent = my-agent/1.0
verbose = true

Priority: built-in defaults < config file < CLI flags < environment variables.

Environment Variables

Variable	Overrides
PGET_CONCURRENCY	-n
PGET_RETRIES	-r
PGET_TIMEOUT	-t
PGET_USER_AGENT	-A
PGET_OUTPUT	-o
PGET_RESUME	-c
PGET_QUIET	-q
PGET_VERBOSE	-v
PGET_RATE	--rate
PGET_CHECKSUM	--checksum
PGET_HEADER	--header (semicolon-separated pairs)

Features

• Parallel chunked downloads (configurable concurrency)
• HTTP range request support with single-stream fallback
• FTP/FTPS with REST command support
• Real-time progress bar with speed and ETA
• Resume interrupted downloads (-c)
• Rate limiting (--rate)
• SHA256/MD5 checksum verification (--checksum)
• Custom HTTP headers (--header)
• .netrc credential support for FTP
• .pgetrc configuration file (~/.pgetrc)
• Signal handling (SIGINT/SIGTERM — clean shutdown, resume-state save)
• Verbose mode (-v) for worker-level detail