Polars is emerging as a strong competitor to pandas for Python data analysis, boasting significant performance improvements due to its Rust backend optimized for parallel processing and vectorized operations. This post tests Polars against pandas with varying vCores, finding Polars generally faster, though it encounters some challenges with single vCore setups. While Polars shows great promise, considerations like cost, compatibility, and maturity remain important when evaluating a switch from pandas.
Table of contents
The setupThe dataThe premiseTestingA note on eager vs lazy evaluationGroup by + AggregateQuantile ComputationFilteringSortingSorting (with multithreading=False)Sort: