I wrote a thread pool library (Tpool) in C as a learning project and spent a couple of hours experimenting on it. I want to share the results of concurrently running the recursive calls of the quicksort algorithm using Tpool vs without multithreading. The result is obvious but I think the more important thing is about how to correctly apply concurrency in a recursive function in this particular experiment.
I came across an interesting deadlock bug while using Tpool in recursive function which made “consumers to act as producers” resulting in both consumers and producers (worker threads) to keep on waiting that caused deadlock. I have discussed further details in the repo’s readme. Check the link below for details on the experiment, deadlock bug, how I correctly applied concurrency in recursive function, and project code.
Next, I plan to implement a lock-free data structure specifically a work stealing deque to further improve performance.
Please provide your feedback on my analysis/benchmark, and the project itself. I appreciate technical discussions and feedback related to correctness, design, and the approach.
Overview: Tpool is a thread pool written in C based on single producer multi consumer queue and is built with pthread API, mutex, and condition variable. It uses a circular buffer data structure and follows FIFO scheduling policy.
Experiment: A ~4 GB integer array was allocated and initialized, then used as input to the quicksort algorithm. Total of 6 worker threads were created using Tpool. Two tests were performed - First, recursive calls of the quicksort algorithm were ran concurrently using Tpool workers. Second, quicksort was run in single thread.
Results: The sorting took 14504.181 ms to complete when running the recursive calls concurrently using the thread pool while it took 43424.730 ms to complete using single thread under identical condition. A significant performance gain of 67% was observed while running recursive calls in multiple threads.
Link - <a href="https://github.com/aatbip/tpool" target="_blank" rel="noopener nofollow">https://github.com/aatbip/tpool</a>
#systemsprogramming #lowlevel #Cprogramming #concurrency #multithreading #pthread #linux

This is the place to discuss everything about C/C++ and Assembly —from basic programming concepts to advanced topics like memory management, concurrency, and performance optimization. Share knowledge, ask questions, and collaborate with developers.

C/C++ Community

A developer shares Tpool, a thread pool library written in C using pthreads, mutex, and condition variables with a circular buffer and FIFO scheduling. The post covers a benchmark experiment sorting a ~4GB integer array with concurrent recursive quicksort calls using 6 worker threads vs single-threaded execution, achieving a 67% performance improvement (14.5s vs 43.4s). A notable deadlock bug is discussed where consumers acting as producers caused all threads to wait indefinitely. Future plans include implementing a lock-free work-stealing deque for further performance gains.

Tpool - A Thread pool library written in C

impressive project! i think you should compare your thread pool to other thread pool implementations instead of single threading. not only in performance but also in developer experience.

Great technical deep dive on the deadlock. Recursive quicksort is a classic “stress test” for thread pool implementations because of how it creates parent-child dependencies.
Here are a few high-assurance engineering points to consider for your next iteration:
<ul>
<li>The Single Queue Bottleneck: Your “consumer as producer” deadlock is a textbook case of resource exhaustion in a single-queue model. When a worker thread waits for a sub-task that is stuck behind it in the same FIFO queue, you get a circular dependency. In security-critical infrastructure, we avoid this by ensuring the task dependency graph is a DAG (Directed Acyclic Graph) and that the scheduler can’t be starved by its own children.</li>
<li>Work-Stealing is the Solution: Your plan to implement a lock-free work-stealing deque is exactly what’s needed. By giving each thread its own local deque, you minimize global mutex contention and allow “parent” tasks to remain active or for other workers to “steal” the child tasks from the bottom of the deque, which is much more efficient for recursive workloads.</li>
<li>Formal Verification of Invariants: In multithreaded C, “obvious” gains often hide subtle race conditions. To achieve real high-assurance, you should define formal invariants for your queue state and use model checking to prove that deadlocks are mathematically impossible under any scheduling interleaving.</li>
<li>Determinism &amp; Memory Safety: Since you are working in C, manual memory management is your biggest attack surface. Applying formal methods to prove memory safety and ensuring your pool produces deterministic results—regardless of thread timing—is what separates a “learning project” from production-grade systems programming.</li>
</ul>