python - Why multiprocessing is slow -

python - Why multiprocessing is slow -

i started reading multiprocessing sake of speeding programs. hence, wrote 2 basic examples extract prime numbers list of random numbers.

example 1: using multiprocessing

from multiprocessing import process, queue random import randrange import time def randomlist(q, size, nmax): l = [] r = randrange(2, nmax) in range(size): while r in l: # avoid replicating numbers r = randrange(2, nmax) l.append(r) q.put(r) def checkprime(numbers, prime): if numbers.qsize(): n = numbers.get() count = 0 # divisors counter d = 2 # divisor while not count , d<=n/2: if n%d: d+=1 else: count+=1 if not count: prime.put(n) if __name__=="__main__": numbers = queue() prime = queue() randomlist(numbers, 50, 1000) # 50 number | 100 max value t1 = time.time() while numbers.qsize(): in range(10): # running 10 processes p=process(target=checkprime, args=(numbers, prime)) p.start() p.join() t2 = time.time() primes = [] in range(prime.qsize()): primes.append(prime.get()) print("[+] prime numbers:") print(primes) print("[+] time elapsed:"+str(t2-t1))

output:

[+] prime numbers: [17, 227, 389, 593, 953, 757] [+] time elapsed:9.41699981689

exemple 2: same illustration 1 without multiprocessing

[...] while numbers.qsize(): checkprime(numbers, prime) [...]

output:

[+] prime numbers: [193, 227, 241, 439, 499, 877, 479, 743, 929] [+] time elapsed:0.00999999046326

so, multiprocessing makes programme (specifically maybe) hugely slower without using it. explanation? using wrong way?

i'm thinking method of multiprocessing poor. rather splitting work 10 processes , starting them @ once, you're starting single process @ time , each 1 doing single unit of work exiting. implementation create (and destroy) 50 processes on lifetime, creates lot of overhead.

you're joining processes after starting them, create you're never running multiple processes. bring together makes wait kid process finish before continuing.

finally, there's gotta improve way homecoming results using queue , getting single value @ time. if can start each process @ 1 time set of work , homecoming results in list master thread, cut down overhead of using queue.

python performance python-2.7 multiprocessing primes