Add support for record with callbacks? #177

hakonhagland · 2019-08-23T23:02:41Z

I have this C program using the GNU scientific library (the library can be installed on Ubuntu from package libgsl-dev):

#include <stdio.h>
#include <gsl/gsl_errno.h>
#include <gsl/gsl_math.h>
#include <gsl/gsl_roots.h>

double
my_f (double x, void *params)
{
    return x * x * x;
}

double
my_df (double x, void *params)
{
    return 3.0 * x * x;
}

void
my_fdf (double x, void *params, 
               double *y, double *dy)
{
    *y = x * x * x;
    *dy = 3.0 * x * x;
}

int main (void)
{
    gsl_root_fdfsolver *s = gsl_root_fdfsolver_alloc ( gsl_root_fdfsolver_newton );
    gsl_function_fdf FDF;
    FDF.f = &my_f;
    FDF.df = &my_df;
    FDF.fdf = &my_fdf;
    FDF.params = NULL;
    double initial_guess = 5.0;
    gsl_root_fdfsolver_set (s, &FDF, initial_guess);

    for (int i = 1; i <= 20; i++ ) {
        gsl_root_fdfsolver_iterate (s);
        double root = gsl_root_fdfsolver_root (s);
        printf( "%.8f\n", root);
    }
    gsl_root_fdfsolver_free (s);
    return 0;
}

The above can be compiled using gcc -Wall -o solve solve.c -lgsl -lgslcblas -lm and produces output:

I started implementing this in Perl using FFI::Platypus:

package GSL::RootSolver::GslFunctionFDF;
use FFI::Platypus::Record;
record_layout(
    qw [
  (double,opaque)->double               f 
  (double,opaque)->double               df 
  (double,opaque,opaque,opaque)->double fdf 
  opaque                                params
   ]
);

package main;
use feature qw(say);
use strict;
use warnings;

use FFI::Platypus;
use FFI::CheckLib;
my $ffi = FFI::Platypus->new();
$ffi->lib( find_lib_or_exit( lib => 'gsl' ) );

$ffi->type('opaque'  => 'gsl_root_fdfsolver_type_ptr');
$ffi->type('opaque => 'gsl_root_fdfsolver');
$ffi->attach(
    'gsl_root_fdfsolver_alloc',
    [ 'gsl_root_fdfsolver_type_ptr' ] => 'gsl_root_fdfsolver'
);
my $solver_type = $ffi->cast(
    'opaque' => 'opaque*', 
    $ffi->find_symbol('gsl_root_fdfsolver_newton')
);
my $solver = gsl_root_fdfsolver_alloc( $$solver_type );
my $fdf = GSL::RootSolver::GslFunctionFDF->new();
my $f_func = $ffi->closure(
    sub {
        my( $x, $params ) = @_;
        return $x*$x*$x;
    }
);
$fdf->f( $f_func );
my $df_func = $ffi->closure(
    sub {
        my( $x, $params ) = @_;
        return 3 * $x * $x;
    }
);
$fdf->df( $df_func );
my $fdf_func = $ffi->closure(
    sub {
        my( $x, $params, $y, $dy ) = @_;
        $$y = $x * $x * $x;
        $$dy = 3.0 * $x * $x;
    }
);
$fdf->fdf( $fdf_func );
$fdf->params( undef );

When I run this:

$ p.pl
type not supported ((double,opaque)->double f) at ./p.pl line 6.

So callbacks in records is not implemented yet? Would it be difficult to add support for callbacks passed in records like this?

The text was updated successfully, but these errors were encountered:

plicease · 2019-08-24T00:04:16Z

yeah, atm closures aren't supported as record members. However, you can use the opaque type instead and cast the closure value to opaque:

$fdf->f($ffi->cast( '(double,opaque)->double' => 'opaque', $f_func));

It would be nice to have broader support for these and other types for the record type though, so I'll keep this open so that it can be worked out when there is time.

plicease · 2019-08-24T00:05:38Z

Keep in mind that you need to make sure that $f_func needs to stay in scope (somewhere) as long as it could be called from C space.

hakonhagland · 2019-08-24T07:28:25Z

Thanks! The cast seems to work fine for $fdf->f but for $fdf->fdf I would need an opaque* (and not an opaque) as input argument (I think ?):

$fdf->fdf(
    $ffi->cast( '(double,opaque,opaque*,opaque*)->double' => 'opaque', $fdf_func)
);

But this gives error:

Only native types and strings are supported as closure argument types (68) at /home/hakon/perlbrew/perls/perl-5.28.1/lib/site_perl/5.28.1/x86_64-linux/FFI/Platypus/TypeParser/Version0.pm line 67.

plicease · 2019-08-24T08:31:18Z

But this gives error:

Only native types and strings are supported as closure argument types (68) at /home/hakon/perlbrew/perls/perl-5.28.1/lib/site_perl/5.28.1/x86_64-linux/FFI/Platypus/TypeParser/Version0.pm line 67.

right closures have a limited support for types. Some types like record couldn't be supported the way they are currently implemented. Others are tricky because of memory ownership (returning a string from a closure probably won't ever be supported). Pointer types probably could be supported in the future, but it hasn't been a priority so far.

Most of these thing can be dealt with using a cast inside the closure though.

# closure that takes an opaque* as an argument
$ffi->closure(sub {
  my($a, $b, $c, $d) = @_;
  $c = $ffi->cast('opaque' => 'opaque*', $c);
  ...
});

# closure that returns a string
$ffi->closure(sub {
  ...
  state $ret;  # cannot be my
  $ret = ...;
  return $ffi->cast('string' => 'opaque', $ret);
});

hakonhagland · 2019-08-24T09:31:26Z

Interesting! Ok so if I try:

my $fdf_func = $ffi->closure(
    sub {
        my( $x, $params, $y, $dy ) = @_;
        $y = $ffi->cast('opaque' => 'opaque*', $y);
        $dy = $ffi->cast('opaque' => 'opaque*', $dy);
        $$y = $x * $x * $x;
        $$dy = 3.0 * $x * $x;
    }
);

I now get error:

Modification of a read-only value attempted at ./p.pl line 58.

where line no 58 is this:

        $$y = $x * $x * $x;

plicease · 2019-08-29T21:13:53Z

What you want is to write a double to an arbitrary location of memory which Perl doesn't allow, but you can use the C memcpy function. Something like: (not tested)

 # (double, opaque, opaque, opaque)->void
my $fdf_func = $ffi->closure(
    sub {
        my( $x, $params, $y, $dy ) = @_;
        my $memcpy = $ffi->function( memcpy => [ 'opaque', 'double*', 'size_t']=> 'opaque');
        my $size = $ffi->sizeof('double');
        $memcpy->($y, \($x*$x*$x), $size);
        $memcpy->($dy, \(3.0*$x*$x), $size);
    }
);

(of course if you expect this to be called frequently, which presumably it will, you will want to attach memcpy and save the sizeof value.

hakonhagland · 2019-08-30T07:41:41Z

Wow! This actually works now:

package GSL::RootSolver::GslFunctionFDF;
use FFI::Platypus::Record;

record_layout(
    qw [
  opaque        f 
  opaque        df 
  opaque        fdf 
  opaque        params
   ]
);

package main;
use feature qw(say);
use strict;
use warnings;

use Data::Printer;
use FFI::Platypus;
use FFI::CheckLib;
my $ffi = FFI::Platypus->new();
$ffi->lib( find_lib_or_exit( lib => 'gsl' ) );

$ffi->type('opaque'       => 'gsl_root_fdfsolver_type_ptr');
$ffi->type('opaque'     => 'gsl_root_fdfsolver');
$ffi->attach(
    'gsl_root_fdfsolver_alloc',
    [ 'gsl_root_fdfsolver_type_ptr' ] => 'gsl_root_fdfsolver'
);
my $solver_type = $ffi->cast(
    'opaque' => 'opaque*', 
    $ffi->find_symbol('gsl_root_fdfsolver_newton')
);
my $solver = gsl_root_fdfsolver_alloc( $$solver_type );
my $fdf = GSL::RootSolver::GslFunctionFDF->new();
my $f_func = $ffi->closure(
    sub {
        my( $x, $params ) = @_;
        return $x*$x*$x;
    }
);
$fdf->f($ffi->cast( '(double,opaque)->double' => 'opaque', $f_func) );
my $df_func = $ffi->closure(
    sub {
        my( $x, $params ) = @_;
        return 3 * $x * $x;
    }
);
$fdf->df($ffi->cast( '(double,opaque)->double' => 'opaque', $df_func) );
my $fdf_func = $ffi->closure(
    sub {
        my( $x, $params, $y, $dy ) = @_;
        my $memcpy = $ffi->function(
            memcpy => [ 'opaque', 'double*', 'size_t']=> 'opaque'
        );
        my $size = $ffi->sizeof('double');
        $memcpy->($y, \($x*$x*$x), $size);
        $memcpy->($dy, \(3.0*$x*$x), $size);
    }
);
$fdf->fdf(
    $ffi->cast( '(double,opaque,opaque,opaque)->double' => 'opaque', $fdf_func)
);

$fdf->params( undef );
my $initial_guess = 5.0;
$ffi->type("record(GSL::RootSolver::GslFunctionFDF)" => 'gsl_function_fdf');
$ffi->attach(
    'gsl_root_fdfsolver_set',
    [ 'gsl_root_fdfsolver', 'gsl_function_fdf', 'double' ] => 'gsl_root_fdfsolver'
);
$ffi->attach(
    'gsl_root_fdfsolver_iterate', [ 'gsl_root_fdfsolver' ] => 'int'
);
$ffi->attach(
    'gsl_root_fdfsolver_root', [ 'gsl_root_fdfsolver' ] => 'double'
);
$ffi->attach(
    'gsl_root_fdfsolver_free', [ 'gsl_root_fdfsolver' ] => 'void'
);

gsl_root_fdfsolver_set ($solver, $fdf, $initial_guess);

for my $i (1..20) {
    gsl_root_fdfsolver_iterate ($solver);
    my $root = gsl_root_fdfsolver_root ($solver);
    printf "%.8f\n", $root;
}
gsl_root_fdfsolver_free ($solver);

Output

plicease · 2019-12-13T02:18:04Z

I am pretty sure we can support record pass-by-value into a closure, but not pointer to a record (at least based on the current record implementation).

plicease · 2021-03-03T00:46:08Z

I am pretty sure we can support record pass-by-value into a closure, but not pointer to a record (at least based on the current record implementation).

Revisiting some old tickets here and using structs from FFI::C you can pretty easily write a closure that takes a struct as an argument, but you have to cast it from an opaque pointer. It would be good to:

Document this
Extend the type plugin interface so that you can do this automatically

plicease · 2021-03-10T00:00:00Z

#312 adds support for passing a record into a closure. Passing a pointer into a closure already worked, but due to limitations in the record interface, that is actually a copy too, so records can only be passed into a closure by-value, and they are read-only.

For passing a pointer to a record into a closure it makes more sense for the closure to take a opaque and cast that into a FFI::C object, which you can then inspect and even modify. If #304 happens this will be possible to do without a cast. But that is down the road.

plicease added the 🐣Enhancement Things that make it work better label Aug 24, 2019

plicease mentioned this issue Mar 3, 2021

Interface for custom types + closures #304

Open

plicease closed this as completed Mar 10, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for record with callbacks? #177

Add support for record with callbacks? #177

hakonhagland commented Aug 23, 2019 •

edited by plicease

Loading

plicease commented Aug 24, 2019 •

edited

Loading

plicease commented Aug 24, 2019

hakonhagland commented Aug 24, 2019 •

edited by plicease

Loading

plicease commented Aug 24, 2019

hakonhagland commented Aug 24, 2019 •

edited by plicease

Loading

plicease commented Aug 29, 2019 •

edited

Loading

hakonhagland commented Aug 30, 2019 •

edited by plicease

Loading

plicease commented Dec 13, 2019

plicease commented Mar 3, 2021

plicease commented Mar 10, 2021

Add support for record with callbacks? #177

Add support for record with callbacks? #177

Comments

hakonhagland commented Aug 23, 2019 • edited by plicease Loading

plicease commented Aug 24, 2019 • edited Loading

plicease commented Aug 24, 2019

hakonhagland commented Aug 24, 2019 • edited by plicease Loading

plicease commented Aug 24, 2019

hakonhagland commented Aug 24, 2019 • edited by plicease Loading

plicease commented Aug 29, 2019 • edited Loading

hakonhagland commented Aug 30, 2019 • edited by plicease Loading

Output

plicease commented Dec 13, 2019

plicease commented Mar 3, 2021

plicease commented Mar 10, 2021

hakonhagland commented Aug 23, 2019 •

edited by plicease

Loading

plicease commented Aug 24, 2019 •

edited

Loading

hakonhagland commented Aug 24, 2019 •

edited by plicease

Loading

hakonhagland commented Aug 24, 2019 •

edited by plicease

Loading

plicease commented Aug 29, 2019 •

edited

Loading

hakonhagland commented Aug 30, 2019 •

edited by plicease

Loading