python - How are are NumPy's in-place operators implemented to explain the significant performance gain -

python - How are are NumPy's in-place operators implemented to explain the significant performance gain -

i know in python, in-place operators utilize __iadd__ method in-place operators. immutable types, __iadd__ workaround using __add__, e.g., tmp = + b; = tmp, mutable types (like lists) modified in-place, causes slight speed boost.

however, if have numpy array modify contained immutable types, e.g., integers or floats, there more important speed boost. how work? did illustration benchmarks below:

import numpy np def inplace(a, b): += b homecoming def assignment(a, b): = + b homecoming int1 = 1 int2 = 1 list1 = [1] list2 = [1] npary1 = np.ones((1000,1000)) npary2 = np.ones((1000,1000)) print('python integers') %timeit inplace(int1, 1) %timeit assignment(int2, 1) print('\npython lists') %timeit inplace(list1, [1]) %timeit assignment(list2, [1]) print('\nnumpy arrays') %timeit inplace(npary1, 1) %timeit assignment(npary2, 1)

what expect similar difference python integers when used in-place operators on numpy arrays, results different:

python integers 1000000 loops, best of 3: 265 ns per loop 1000000 loops, best of 3: 249 ns per loop python lists 1000000 loops, best of 3: 449 ns per loop 1000000 loops, best of 3: 638 ns per loop numpy arrays 100 loops, best of 3: 3.76 ms per loop 100 loops, best of 3: 6.6 ms per loop

each phone call assignment(npary2, 1) requires creating new 1 1000000 element array. consider how much time takes allocate (1000, 1000)-shaped array of ones:

in [21]: %timeit np.ones((1000, 1000)) 100 loops, best of 3: 3.84 ms per loop

this allocation of new temporary array requires on machine 3.84 ms, , on right order of magnitude explain entire difference between inplace(npary1, 1) , assignment(nparay2, 1):

in [12]: %timeit inplace(npary1, 1) 1000 loops, best of 3: 1.8 ms per loop in [13]: %timeit assignment(npary2, 1) 100 loops, best of 3: 4.04 ms per loop

so, given allocation relatively slow process, makes sense in-place add-on faster assignment new array.

numpy operations on numpy arrays may fast, creation of numpy arrays relatively slow. consider, example, how much more time takes create numpy array python list:

in [14]: %timeit list() 10000000 loops, best of 3: 106 ns per loop in [15]: %timeit np.array([]) 1000000 loops, best of 3: 563 ns per loop

this 1 reason why improve utilize 1 big numpy array (allocated once) rather thousands of little numpy arrays.

python arrays numpy