Wednesday, 15 February 2012

Python C extension decorator -



Python C extension decorator -

i apologize in advance long-winded background info.

i've been playing around python/c-api (python 3.4) , have gotten stumped. goal have c extension can utilize function/method decorator. have reasonably working prototype lru cache in gist, https://gist.github.com/pbrady/916495198910e7d7c713. although cache works , fast, there 2 issues problems :

the type returned decorater not function: i.e.

>>> lrucache import lrucache >>> @lrucache() ... def f(a, b): ... homecoming a+b ... >>> type(f) >>> <class 'lrucache.cache'>

because of this, decorator not work on methods - seems self gets lost because python doesn't create instance method when sees class (which makes sense).

copying __doc__ decorated function cache class doesn't impact message displayed help.

my thought around these issues homecoming user function few modifications rather new custom class object. these modifications are

add __wrapped__ attribute function , point function.

overwrite __call__ attribute function calls directed custom c routine.

i able accomplish (1) overriding functions __call__ method doesn't since it's not used interpreter.

my question (finally): how create python function (i.e., pyfunctionobject) phone call c function?

or, if there improve way this, interested in well. largely learning/fun exercise i'm not interested in cython based solutions.

well (2) step fails because special attributes looked on class, not on instance:

in [1]: class test: ...: def __call__(self): ...: print('called class attribute!') ...: in [2]: t = test() in [3]: def new_call(): ...: print('called instance attribute!') ...: in [4]: t.__call__ = new_call in [5]: t() called class attribute!

and don't want modify function class __call__ method, since modify semantics of python code in substantial way.

as far know there one way instantiate pyfunctionobject, calling pyfunction_new. problem requires pycodeobject argument , create such object can call:

pycode_newempty: creates invalid code object. used only create frame objects don't care of code set there. pycode_new: creates bytecode object. beast 14 arguments, 1 of pyobject *code should readable buffer contains binary representation of bytecode. py_compilestringobject: seems reasonable solution. compile built-in.

once have created code object creation of function trivial. note wont gain any performance benefit doing since body of function object interpreted.

however there other solutions problem:

you create lrucache descriptor. python methods automatically pass self first parameter, can emulate behaviour.

you write very simple python module imports c extension , provides function such as:

from functools import wraps _lrucache import cache def lrucache(func): cached_func = cache(func) @wraps(func) def wrapper(*args, **kwargs): homecoming cached_func(*args, **kwargs) homecoming wrapper

in way side-step whole problem via python code.

you create subclass of function type , homecoming object instead. might not work if python checking exact type when building class, , implementation might not trivial since interpreter might assume properties function objects you'd have provide. no, can't subclass function...

python c decorator python-c-api

No comments:

Post a Comment