A project exploring procedural Minecraft world generation using a two-stage pipeline: a VQ-VAE to tokenize 3D voxel space into a compressed codebook of structural signatures, followed by a GPT-style transformer to learn the spatial grammar of terrain. Key challenges addressed include 3D data scarcity (solved by extracting chunks from a self-run Minecraft instance), extreme class imbalance (solved with weighted cross-entropy loss), and dead codebook embeddings. The model successfully generates recognizable terrain features including caves, coastlines, snow-capped peaks, and tree structures across a grid of chunks.
Table of contents
The Challenge of 3D Generative ModelingData PreprocessingArchitecture OverviewRaw Voxel Problem and Tokenizing 3D SpaceFurther DetailsBrick by BrickResultsReflections and Future WorkCitations and LinksA Note on the DatasetSort: