Logging nonstandard fields

[normelton]

This just a newbie question ...

Our IPFIX exporter is sending fields that are not included in the default output, such as postNATSourceIPv4Address (id 225). I think I understand the concept of mapping one field into another, but is there a recipe for adding this field to the regular output? Do I need to wait for the goflow2 to receive a template from the exporter to recognize the packet format?

Thanks!

[lspgn]

Hi @normelton,

Do I need to wait for the goflow2 to receive a template from the exporter to recognize the packet format?

Yes. Once GoFlow2 has the template, it will map the IPFIX data fields it into a protobuf field

Have you seen the example mapping.yaml?

Try with the following config and pass it to the goflow2 CLI using -mapping mapping.yaml

formatter:
  fields:
    - type
    - time_received_ns
    - postnat
  protobuf:
    - name: postnat
      index: 999
      type: bytes
  render:
    postnat: ip
ipfix:
  mapping:
    - field: 225
      destination: postnat

[normelton]

Yep I've been working through the mapping.yaml syntax. Using your sample, I get:

line 11: cannot unmarshall !!seq into map[string]protoproducer.RendererID

If I remove the 'render' section, I do get output that shows the three fields, with the postnat IP address in hex. That's progress!

[lspgn]

Are you using the latest version of GoFlow2 v2.1.1?
The sample above does not error for me. postnat: ip must be a map element, not a list item - postnat: ip

[normelton]

Ahh, I had a YAML-typo, converting to a map element fixed that. No more error, but the field is still displaying as hex.

Yes, version 2.1.1. I double confirmed that my producer/proto/render.go file has the IP renderer in it.

Just to be sure, here's my mapping.yaml file:

formatter:
  fields:
    - type
    - nat_event
    - time_received_ns
    - bytes
    - packets
    - src_addr
    - dst_addr
    - src_port
    - dst_port
    - postnat
  protobuf:
    - name: postnat
      index: 999
      type: bytes
    - name: nat_event
      index: 998
      type: varint
  render:
    postnat: ip
ipfix:
  mapping:
    - field: 225
      destination: postnat
    - field: 230
      destination: nat_event

[lspgn]

I'll try to reproduce. Just in case, would you be able to provide a pcap (just need 1 template and 1 data packet).

[normelton]

Sure thing, attached.

ipfix.pcap.zip

[lspgn]

Was able to reproduce.
It was actually a bug I solved a few days ago but hadn't merged the changes: #277
Can you try with this branch? Otherwise, I'll do a release over the weekend

[normelton]

Brilliant, that worked. This is enough for our proof-of-concept, we'll watch for the fix in an upcoming release.

Thanks so much!