Python is great, it is easy to code, flexible, dynamic, etc. but it has a downside: it is very slow. If, like me, you sometimes need to speed up a few functions, there are several options. One is to use Cython, which is kind of a C extention for python… their goal is to allow you to right python-like code (i.e. easy to right and high level) and easily obtain some serious (factor of several hundreds possible) speed up, up to C speed in some cases. If the function you have to speed up looks like its going to be a pain to write in C, Cython is the way to go!
That is what I’ve been doing over the last week… until I realized it could be a bit of a pain. Cython is not magic (so I was told several times I asked for help, thanks) and there’s no way one command line is going to give you blast speed. Instead, although it is easy to transform a naive python function into a Cython version, and easy to obtain reasonable speedup, if you’re looking for a real optimization, you’ll probably need to dig into the behind the scenes of Cython… understanding why the hell Cython transformed this line of python into 10 lines of C whereas this apparently-exactly-same-kind-of-line of python into 250 lines of C… this kind of stuff. And in the end, making your code very fast may be a pain… as compared to writing it in C in the first place!
That is why, depending on how much you feel your function will be easy to write in C, I’d seriously consider to do it, rather that going through the pain of optimizing cython code. I’ll now explain how to call C functions to you python code, and even C functions that work on numpy arrays, because I’m sure that’s probably what you’re up to.
This is the C file that contains the C functions you will call from your python module. I’ve made three examples, one function that just prints the basic hello world to the scren, the second takes a numpy array and prints it, and the third one copies one array into the other. If you know C, this is all pretty simple.
#include <stdlib.h>
#include <stdio.h>
/* basic function that just print some stuff */
void helloworld(void)
{
printf("Hello World!!\n");
}
/* this will print a given array */
void displaynparray(double *array, size_t s)
{
unsigned int i;
for (i = 0; i < s; i++)
{
printf("array[%d] = %lf\n",i,array[i]);
}
}
/* this will copy the content of 'in' into 'out' */
void copyarray(double *in, size_t s, double *out)
{
unsigned int i;
for (i=0; i < s; i++)
{
out[i] = in[i];
}
}
then you compile your code so to produce a shared library :
gcc -fPIC -shared -o module.so module.c
Now here is the python module that will make use of the library we’ve just done :
from numpy.ctypeslib import ndpointer
import ctypes
import numpy as np
def displaynparray(lib):
array = np.arange(0,10,dtype=np.float64)
disp = lib.displaynparray
disp.restype=None
disp.argtypes = [ndpointer(ctypes.c_double), ctypes.c_size_t]
disp(array,array.size)
def helloworld(lib):
hw = lib.helloworld
hw.restype = None
hw()
def copyarray(lib):
arr_in = np.arange(0,10,dtype=np.float64)
arr_out = np.zeros(10, dtype=np.float64)
cpa = lib.copyarray
cpa.restype = None
cpa.argtypes = [ndpointer(ctypes.c_double),
ctypes.c_size_t,
ndpointer(ctypes.c_double)]
cpa(arr_in, arr_in.size, arr_out)
print arr_in
print arr_out
def main():
lib = ctypes.cdll.LoadLibrary('./module.so')
print 'function helloworld() : '
helloworld(lib)
print 'function displaynparray()'
displaynparray(lib)
print 'function copyarray'
copyarray(lib)
if __name__ == '__main__':
main()
here is what I get in my terminal when I execute the script :
python module.py
function helloworld() :
Hello World!!
function displaynparray()
array[0] = 0.000000
array[1] = 1.000000
array[2] = 2.000000
array[3] = 3.000000
array[4] = 4.000000
array[5] = 5.000000
array[6] = 6.000000
array[7] = 7.000000
array[8] = 8.000000
array[9] = 9.000000
function copyarray
[ 0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]
[ 0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]
The Ferret Touch 