netfgpa10g: segfault rofl::crofsock::parse_message during OFTEST flow_stats.TwoFlowStats

[tytussielach]

xDPd gets Segmentation fault while running OFTEST flow_stats.TwoFlowStats

The logs:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff77d52db in rofl::crofsock::parse_message (this=0x200b9320, mem=0x202c0a20) at ../../../../src/rofl/common/crofsock.cc:390
390         env->recv_message(this, msg);
(gdb) bt
#0  0x00007ffff77d52db in rofl::crofsock::parse_message (this=0x200b9320, mem=0x202c0a20)
    at ../../../../src/rofl/common/crofsock.cc:390
#1  0x00007ffff77d48e1 in rofl::crofsock::handle_read (this=0x200b9320, socket=...) at ../../../../src/rofl/common/crofsock.cc:205
#2  0x00007ffff7786449 in rofl::csocket_impl::handle_read (this=0x200bca60) at ../../../../src/rofl/common/csocket_impl.h:411
#3  0x00007ffff777ff09 in rofl::csocket_impl::handle_revent (this=0x200bca60, fd=34)
    at ../../../../src/rofl/common/csocket_impl.cc:170
#4  0x00007ffff7773692 in rofl::ciosrv::__handle_revent (this=0x200bcbc8, fd=34) at ../../../../src/rofl/common/ciosrv.cc:158
#5  0x00007ffff777448f in rofl::cioloop::run_loop (this=0x200b6180) at ../../../../src/rofl/common/ciosrv.cc:388
#6  0x00007ffff777540b in rofl::cioloop::run () at /root/xdpd_all/rofl-core/build/../src/rofl/common/ciosrv.h:418
#7  0x00007ffff7773226 in rofl::ciosrv::run () at ../../../../src/rofl/common/ciosrv.cc:83
#8  0x000000000045ac26 in xdpd::system_manager::init (argc=5, argv=0x7fffffffe598)
    at ../../../../src/xdpd/management/system_manager.cc:235
#9  0x00000000004524a2 in main (argc=5, argv=0x7fffffffe598) at ../../../src/xdpd/xdpd.cc:25
(gdb) print env
$1 = (rofl::crofsock_env *) 0x200b91a0
(gdb) print }env
A syntax error in expression, near `}env'.
(gdb) 
A syntax error in expression, near `}env'.
(gdb) print *env
$2 = {_vptr.crofsock_env = 0x0}
(gdb) 

[akoepsel]

This is a very unlikely bug, as it would indicate that the embedding environment of crofsock is gone. The only reason for this is an abnormal termination of the entire management entity, a closing OpenFlow connection with a race condition during the connection shutdown or a memory corruption somewhere.

Which hardware platform and driver was used? Please confirm that this bug also exists on x86 platform with the GNU/Linux software driver. If it does, please re-run the test with valgrind to check for any weird conditions.

[msune]

@akoepsel this backtrace was obtained with the netfpga10g driver during a flowstats req/reply by oftest 1.0. Tytus, and afterwards me, were synchronizing with the latest devel-0.4 (make maintainer-clean), both in rofl and xdpd, and it continued to happen the same.

I believe it is a race condition as you say, but since this was compiled in Fedora13 (netfpga10g SDK), the bug may have popped up now, due to a different interlacing or generated asm code by the old gcc.

[msune]

I would recommend first fixing all the valgrind errors and warningson your driver, and retry as for #32.

Leaving the issue open for the moment