engine: Add timeline events for the breathe cycle #873
Conversation
Conflicts: src/program/packetblaster/packetblaster.lua
Provide a clear compilation error on platforms that don't have a 64bit integer size, for example i386. Include a link to porting information so that the user will understand that this is not an arbitrary restriction.
Rationale: people are smart and will figure.
This fix makes Snabb work much better on lab servers.
Using the method suggested by Matthew Emerson on Github snabbco#752.
…ra-next Have a clear compile-time error on non-64-bit platforms.
This will be used by the lwAFTR.
An initial contributing.md file.
…next Extended documentation guide.
Currently shm.map() will create the file and any containing directories if it does not exist. This is not so great, so I split it into separate open() and create() functions, adapt all callers, and remove map() from the public interface of the shm module.
This commit adds a histogram facility that can record the distribution of different sampled magnitudes. In particular, it is useful for recording distributions of times. Histograms can be mapped into /var/run/snabb, where they can be analyzed by a separate process. This commit also wires up the app.breathe() loop to record latencies for its breath cycles into a well-known file (/var/run/snabb/PID/engine/latency), and wires up "snabb top" to do some basic statistics on this data.
The idea is, there are lots of different kinds of analyses you might want to do on a histogram, and it's too early to canonicalize one of them. Better to build your own on top of iterate(). Also adds tests for iterate().
Note: This changes the behavior of the program.
This change assumes that the Github repository Igalia/snabbswitch will be renamed to Igalia/snabb. Note: Github will redirect the old name automatically so old links and git checkouts should not be disrupted.
This update has been done manually. There were a few instances where the name 'snabbswitch' should be preserved: links to external sites and handling of older repo versions in snabbdoc.
This file was missed in the previous renaming effort that was rooted in src/.
The events are currently always enabled. This needs to be optimized. Goal for now is simply to have interesting information in the log.
src/Makefile: Use correct paths.
This is worth including as a fix because the master version of the README.md file is currently serving as a landing page for the project.
|
Here is a quick screenshot from some tooling I am doing around timeline. This is a sample taken during the NFV benchmark to forward packets to a DPDK VM. I defined a metric Lag as the time between a log message and the previous one - treating this loosely as a duration.
This is based on the Agile Visualization software stack (Pharo) and I am experimenting with some visualizations. Here is a quick screenshot where each event is drawn as a rectangle, sized based on time "lag" (area = cycles), and with each message being assigned a distinct color. (This is the same data as in the table above.)
Early days but the timeline does seem able to provide interesting data to go digging in. |
|
Here is a summary of the diagram above that shows the sum total cycles for each kind of event in the log and draws edges between events that follow each other:
The big square is Virtio-net, the next is Intel10G, etc. On the top-left there are two different events that can precede the start of a breath: the end of the previous breath (top-left) or commit on the counter objects to shared memory (on the right). This visualization is a bit more like what you see with a traditional profiler i.e. overall proportion of time spent on each activity. |
Use shm.create() instead of shm.map(). Required after update in 9fe397e.
|
I really like the visualizations, Pharo seems like a kick-ass tool for this! |
lwAFTR app should not handle ethernet details
The events are currently always enabled. This needs to be optimized.
Goal for now is simply to have interesting information in the log.
also experimenting with visualization of the logs. Very preliminary; no obvious advantage over simple text (yet).