Epilogue: Global Interpreter Lock (GIL)
The standard Python interpreter (called
CPython) does not support the use of threads well.
This is because it, and by association all C/C++/Fortran-based
CPython have all been written to assume that an individual Python
script is serial (i.e. it only has a single thread of execution). The
Python interpreter uses a “Global Interpreter Lock” to ensure
that only a single line of a Python script can be interpreted
at a time, thereby preventing memory corruption caused by multiple
threads trying to read, write or delete memory in parallel.
This means that, even if you use the Python
module, you will still only execute a single line of your script
at a time in
CPython. There are attempts and discussions
aimed at removing the GIL from
CPython, but it is a task that is
extremely difficult to achieve.
Because of the GIL, parallel Python is normally based on running multiple
forks of the Python interpreter, each with their own copy of the
script and their own GIL. Every time a script needs to run in
parallel, the Python interpreter is forked into multiple processes,
with each forked process performing their part of the shared work.
Once the parallel work is complete (e.g. the
is terminated), the forked processes are killed.
multiprocessing module solves the problem of the GIL, but
at the cost of very high overhead (multi-milliseconds)
for entering and leaving each parallel section of code,
and the higher cost of sharing data between workers as compared
to a true multi-threaded program.
However, this higher overhead is not a true problem for most
scripts. If the overhead it too high, and greater performance is desired, then the “slow”
parts of the script can be rewritten in a compiled language such as
C, C++ or Fortran, and linked in to Python as a
extension. If performance is still a problem, then these
CPython extensions can then be parallelised themselves
using OpenMP, MPI
etc. (there is nothing stopping a
extension from running its own code in parallel, as long
as it doesn’t try to back-call into the
If you want more information on how to write
extensions in C++, check out this tutorial.