Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The numpy style interface might be interesting. The problem is that it would probably kill performance to marshal into a Python process. Unless there's some way to embed numpy?

Supporting custom functions is definitely something we want to do.



Why would it kill performance? All you need to do is this:

    double* values = (double*)malloc(sizeof(double)*num_of_values);
(If you are unfamiliar with numpy, it's just a python wrapper around raw blocks of memory.)


You still have to get the bytes out of the server and into the Python process.


Assuming Python is not running in the same process as InfluxDB, amend my code snippet:

    double* data = shmget(key, sizeof(double)*num_data_pts, whatever_flag);
But directly embedding a Python interpreter is pretty easy in C, so I imagine it should also be fairly easy in Go as well.

(Of course, given that my code snippets are C, you can probably deduce I've never written any Go, so take my comments on how easy it is with a grain of salt.)


Go interop with C is very doable (although I've never tried it). Not sure about shared memory WRT garbage collection though.


A route to this would be providing an FFI for loadable modules that provide custom procedures or functions.

Folks that wanted to could then write Python/Numpy code for these and use Numba+LLVM to compile them.

It could be quite performant and avoid having to marshal data or do IPC, could possibly avoid even copying data in some cases (Numpy/Numba have pretty robust support for structures coming over an FFI, not sure about FFI in Go.)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: