Tuesday, 15 September 2015

python - memory leak - gunicorn + django + mysqldb -



python - memory leak - gunicorn + django + mysqldb -

i have memory leak in gunicorn + django 1.5.1 + mysqldb. start explore code gc , objgraph

when gunicorn worker became on 300mb collected stats:

data['sum_leak'] = sum((getsizeof(o) o in objgraph.get_leaking_objects())) #2 mb data['total_objects_length'] = sum((getsizeof(o) o in gc.get_objects())) #6 mb

so 2+6=8 mb, while gunicorn worker on 300 mb.

so think problem not in python code, it's deeper.

i have: gunicorn==0.17.2 mysqldb==1.2.4

i update whem 19.0.0 , 1.2.5 via pip install pip frreze shows old versions while gunicorn -v , mysqldb.version_info show lastly updated.

so think how totaly reinstall gunicorn , mysqldb shure totaly remove old ones (may old rudiments create problem) ?

also pmap info:

pmap -x 805 805: /usr/bin/python /usr/local/bin/gunicorn engine.wsgi:application -b 127.0.0.1:9005 --workers=2 address kbytes rss dirty mode mapping 08048000 0 1444 0 r-x-- python2.7 0829e000 0 4 4 r---- python2.7 0829f000 0 204 120 rw--- python2.7 082f4000 0 44 44 rw--- [ anon ] 09947000 0 3360 3360 rw--- [ anon ] 09c91000 0 253204 253204 rw--- [ anon ] b5500000 0 4 4 rw--- [ anon ] b5521000 0 0 0 ----- [ anon ] b56d5000 0 0 0 ----- [ anon ] b56d6000 0 1552 1552 rw--- [ anon ] b6257000 0 12 0 r-x-- libpcre.so.3.12.1

it seem leak here - 09c91000 0 253204 253204 rw--- [ anon ]

but don't know this.

need help ways how prepare leak?

if think problem if caused gunicorn workers there easy way test hypothesis:

start workers parameter max_requests=*some positive number*

this create gunicorn restart every worker after had server number of requests.

in documentation say: this simple method help limit harm of memory leaks.

python django memory-leaks mysql-python gunicorn

No comments:

Post a Comment