Introduction

Parallel computing is a crucial aspect of modern-day computing that enables programs to perform computations faster and more efficiently. Python has several libraries that allow developers to write parallel programs, including Joblib, Ray, and Multiprocessing. In this blog post, we will explore the usage of these libraries and how they can help write efficient parallel programs.

Choice 1: Joblib

Joblib is a Python library that allows for easy parallelization of CPU-bound tasks. It provides a simple and lightweight interface to parallelize functions using threads or processes. Joblib is easy to use and requires no prior knowledge of multi-processing.

To use Joblib, you need to import the Parallel and delayed functions from the joblib module. The Parallel function is used to create a parallel object, and the delayed function is used to specify which function to parallelize.

Here is an example:

1
2
3
4
5
6
7
8
9
10
11
from joblib import Parallel, delayed
import time 

def compute_square(n):
    time.sleep(1)
    return n ** 2

if __name__ == '__main__':
    inputs = range(10)
    results = Parallel(n_jobs=2)(delayed(compute_square)(i) for i in inputs)
    print(results)

In the example above, we define a function compute_square that takes an input and returns its square. We then use Joblib’s Parallel function to create a parallel object with two parallel jobs and apply it to the compute_square function using the delayed function. Finally, we print the results.

Also, joblib supports several backends for parallel processing.

Here are some of the available backends for joblib Parallel:

  1. “loky” - The default backend that is recommended for most users. It uses the “Loky” process-based backend and allows for inter-process communication.
  2. “threading” - This backend uses Python’s built-in threading module to run tasks in parallel.
  3. “multiprocessing” - This backend uses Python’s built-in multiprocessing module, which spawns new processes to execute tasks in parallel.
  4. “dask” - This backend uses the Dask library to distribute tasks across multiple nodes in a cluster or on a single machine.
  5. “ray” - This backend uses the Ray library to parallelize functions across multiple cores or multiple machines.

Choice 2: Ray

Ray is a powerful Python library for building distributed applications. It makes it easy to parallelize Python code across multiple CPUs or GPUs. Ray provides a simple to use API for parallelizing embarrassingly parallel and data-intensive workloads.

To use Ray, you need to import the ray module and use the ray.remote decorator to decorate a function that you want to parallelize. When you call the function, Ray will automatically create a worker process to execute the function.

Here is an example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import ray
import time

ray.init()

@ray.remote
def compute_square(n):
    time.sleep(1)
    return n ** 2

if __name__ == '__main__':
    inputs = [1, 2, 3, 4, 5]
    results = ray.get([compute_square.remote(i) for i in inputs])
    print(results)

In the example above, we define a compute_square function that will sleep for one second and return the square of an input number. We use the @ray.remote decorator to specify that the function is going to be executed remotely. In the main block, we call the remote function using the ray.get method, and finally, we print the results.

Choice 3: Multiprocessing

Multiprocessing is a Python library that allows developers to write parallel programs using processes. It provides a simple and easy-to-use interface to spawn multiple processes to take advantage of multi-core CPUs.

To use Multiprocessing, you need to import the multiprocessing module, and then create a Process object for each process you want to spawn. In each Process object, you pass the function you want to execute, along with its arguments.

Here is an example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
from multiprocessing import Process
from multiprocessing import Value
import time


def compute_square(n, result: Value):
    time.sleep(1)
    result = n**2


if __name__ == '__main__':
    inputs = [1, 2, 3, 4, 5]
    results = [Value('f', 0.) for i in range(5)]
    processes = [
        Process(target=compute_square, args=(inputs[i], results[i]))
        for i in range(5)
    ]
    for p in processes:
        p.start()
    for p in processes:
        p.join()

    print([v.value for v in results])

In the example above, we define a compute_square function that will sleep for one second and return the square of an input number. In the main block, we create a Process object for each input number, and then we start each process using the start method. Finally, we wait for all processes to complete using the join method.

Conclusion

In this blog post, we explored three Python libraries, Joblib, Ray, and Multiprocessing that you can use to write parallel programs in Python. These libraries provide a simple and easy-to-use interface to parallelize computations, which can help you write programs that run faster and more efficiently. When deciding which library to use, it’s essential to consider the type of program you’re writing and the specific requirements of your application.