ParallelQueryDecoder Part1

The Python Performance Trap: Bypassing the GIL for High-Throughput Data Clients

Recently, I’ve been focus on contributing performant Python code to open source. My latest contributions were for the Trino Python client. I picked this because the Trino python client requires code to handle xomplex, high-throughput tasks.

Optimizing the Query Data Decoder:

• I tackeld a critical performance issue within the query data decoder. When the Trino Server returns large, compressed query segments, our python client needed to perform to highly CPU-bound operations. They’re Zstandard/LZ4 decompression and subsequent JSON parsing. These are pure high computational tasks that will create a bottleneck within the client itself.

• To solve this potential bottleneck, I had to dig into Python’s infamous Global Interpereter Lock.

Bottleneck:

• Typically, for cuncurrency, threads are used via ThreadPoolExecutor. For CPU-intensive tasks like decompression, threads provide no speed advantage in CPython due to the GIL.
  • Decompression and JSON parsing require constant CPU cycles and minimal waiting for I/O
  • The GIL is a mechanism that ensures only one Python thread exexutes bytecode at any givenmoment. If a thread is busy calculating, it holds the GIL, forcing all other Python threads to wait even if the machine has multiple CPU cores available
  • If I used threads for the decoder, the decompression task and the main thread would simply take turns using CPU core. This allows us to achieve concurrency but not true parallelism. THe actual execution time would be basically identical to synchronous execution and thats not accounting for the overhead from context-switching

Solution:

• My solution centered around using the ProcessPoolExecutor to utilize multi-core hardware and scale the decoder’s performance
• Processes can bypass the GIL because each process runs its own independent Python interpreter and has its own independent memory space. THis allows the OS to schedule the heavy decompression and parsing work onto separate CPU cores. This allows us to achieve the trupe parallelism.

Refactoring:

• Refactored the CompressedQueryDataDecoder to act as the execution scheduler:

  • A global ProcessPoolExecutor is initialized based on the available CPU count
  • The executor is passed down to the compressed decoder instances via the factory

• Execution Flow

  1. The main thread calls self._cpu_executor.submit() with the _execute_full_decode_decode funtion
     • We do Fast delegation to the process queue
  2. A worker process picks up the task and begins the CPU-heavy work(decompression -> JSON parsing_
     • Bypass the GIL, minimizing computational time
  3. The main thread encounters future Result
     • The main thread must wait here, as the final list of rows is a dependency

• Core Principle

  • The perfomance gain is anot about keeping the main thread busy but mroeso about minimizing the duration that the main thread is blocked
  • By offloading the decompression and JSON parsing to a dedicated parallel process, we can sneusre that the main thread is waiting for the fastest possible completion time. This allows our Trino Client efficiently across machines with many CPU cores Conclusion

• The optimization exemplifies the core rule of concurrency in Python

  • I/O Bound: Use Threads( they’ll relase the GIL while waiting).
  • CPU Bound: Use Processes( they bypass the GIL for true prallelsim).

Thanks for reading and I hope to see you in part two of this series digging deeper into the Python GIL