hoover

Preserve directory structure

#33 by fuzzy was closed 2026-04-18 08:17:13 +00:00 0 / 3

Handle index.html for directory-like URLs

#32 by fuzzy was closed 2026-04-18 08:17:13 +00:00

Create subdirectories as needed

#31 by fuzzy was closed 2026-04-18 08:17:13 +00:00

Map URL paths to local filesystem paths

#30 by fuzzy was closed 2026-04-18 08:17:13 +00:00

Download assets in parallel

#29 by fuzzy was closed 2026-04-18 08:17:13 +00:00 0 / 3

Retry on transient failures (3 attempts)

#28 by fuzzy was closed 2026-04-18 08:17:13 +00:00

#27 by fuzzy was closed 2026-04-18 08:17:13 +00:00

Use worker pool (configurable size)

#26 by fuzzy was closed 2026-04-18 08:17:13 +00:00

Build recursive URL traversal

#25 by fuzzy was closed 2026-04-18 08:17:13 +00:00 0 / 4

Respect -depth limit

#24 by fuzzy was closed 2026-04-18 08:17:13 +00:00

#23 by fuzzy was closed 2026-04-18 08:17:13 +00:00

Track visited URLs (set)

#22 by fuzzy was closed 2026-04-18 08:17:13 +00:00

Parse base URL and domain

#21 by fuzzy was closed 2026-04-18 08:17:13 +00:00

Implement HTTP client wrapper

#20 by fuzzy was closed 2026-04-18 08:17:13 +00:00 0 / 4

Handle cookies and sessions if needed

#19 by fuzzy was closed 2026-04-18 08:16:41 +00:00

Follow redirects with limit

#18 by fuzzy was closed 2026-04-18 08:16:41 +00:00

Set sensible User-Agent

#17 by fuzzy was closed 2026-04-18 08:16:41 +00:00

Respect robots.txt (optional, with flag)

#16 by fuzzy was closed 2026-04-18 01:51:34 +00:00

Phase 2: Core Fetching Engine