Hacker News is a community-driven platform for sharing and discussing technology news, startups, and programming-related topics. Through user submissions and comments, Hacker News offers insights into emerging technology trends, industry developments, and entrepreneurial ventures. Readers can participate in discussions, share their insights, and stay informed about the latest advancements in technology and innovation.

Hacker News

Hardwood is a new open-source Apache Parquet parser for Java 21+, built from scratch to address the heavy Hadoop dependency and single-threaded limitations of parquet-java. It offers two APIs: a row-oriented RowReader for complex nested schemas and a columnar ColumnReader for peak throughput via typed primitive arrays. Performance is achieved through page-level parallelism, adaptive page prefetching, cross-file prefetching, and minimizing allocations. Benchmarks show parsing ~9.2 GB of NYC taxi data (650M rows) in ~1.2 seconds using the column API on an M3 Max. The project is at 1.0.0.Alpha1, supports all column types and key encodings, with predicate push-down and a parquet-java compatibility layer on the roadmap. The author also shares candid reflections on using Claude Code as an AI assistant during development, emphasizing that LLM-generated code requires human review and ownership.

Hardwood: A New Parser for Apache Parquet