gobz2

Parallel bzip2 compression/decompression library for Go, leveraging libbz2 via CGo for maximum performance.

Features

• Parallel compression: Use multiple CPU cores for faster compression
• CGo bindings: Direct libbz2 integration for optimal speed/compatibility
• Thread-safe: Safe for concurrent use across goroutines
• Streaming API: Support for io.Reader/io.Writer interfaces
• Memory efficient: Configurable buffer pools and allocation strategies

Requirements

• Go 1.25 or later
• libbz2 development headers (libbz2-dev on Debian/Ubuntu, bzip2-devel on RHEL/Fedora)

Installation

go get git.lan.thwap.org/thwap/gobz2

Static Linking

To link statically against libbz2, use the static build tag:

go build -tags static ./...

This will produce a fully statically linked binary (including libc). Note: Static linking requires the static library libbz2.a (often provided by libbz2-dev or bzip2-devel packages).

Versioning

This project follows Semantic Versioning. Given a version number MAJOR.MINOR.PATCH:

• MAJOR version for incompatible API changes
• MINOR version for new functionality added in a backward compatible manner
• PATCH version for backward compatible bug fixes

The public API consists of the exported functions, types, and constants documented in the godoc.

Quick Start

package main

import (
    "bytes"
    "fmt"
    "git.lan.thwap.org/thwap/gobz2"
)

func main() {
    data := []byte("Hello, world! Parallel bzip2 compression test.")

    // Compress data
    compressed, err := gobz2.CompressBytes(data)
    if err != nil {
        panic(err)
    }

    fmt.Printf("Compressed %d bytes to %d bytes (%.1f%%)\n",
    len(data), len(compressed), 100*float64(len(compressed))/float64(len(data)))

    // Decompress data
    decompressed, err := gobz2.DecompressBytes(compressed)
    if err != nil {
        panic(err)
    }

    if bytes.Equal(data, decompressed) {
        fmt.Println("Data integrity verified!")
    }
}

Performance Benchmarks

Benchmarks run on AMD Ryzen 5 5600 (6 cores) with Go 1.25, Linux.

Throughput (MB/s)

Operation	1 KB	100 KB	1 MB	10 MB	100 MB	256 MB	512 MB
Compress (single)	3.6	13.1	14.9	70.9	77.2	79.7	78.0
Decompress (single)	11.8	20.3	24.1	53.9	110.5	158.4	170.1
Parallel compress	—	—	14.9	66.5	70.9	68.1	68.8
Parallel decompress	—	—	28.6	142.1	165.3	161.3	163.8

Times are median of 3 runs. Parallel decompression shows significant speedup (2.64× for 10 MB data) by utilizing multiple CPU cores. Parallel compression throughput is similar to single‑threaded for these sizes due to chunking overhead; benefits increase with larger datasets.

Comparison with system bzip2 (1 MB data)

Implementation	Compression Time	Compression Throughput
gobz2 (single)	68.6 ms	15.28 MB/s
system bzip2	77.1 ms	13.61 MB/s

gobz2 provides ~12% faster compression than system bzip2 single-threaded, with significantly faster decompression (2× faster than compress/bzip2).

Documentation

API docs: godoc.

License

BSD 3-Clause - see LICENSE for details.