Skip to content

.net implementation of the grok πŸ“

License

Notifications You must be signed in to change notification settings

yexcoffier/grok.net

This branch is 83 commits behind Marusyk/grok.net:main.

Folders and files

NameName
Last commit message
Last commit date

Latest commit

5c0d1ad Β· Sep 24, 2020

History

32 Commits
Sep 24, 2020
Sep 24, 2020
Oct 2, 2019
Jul 12, 2019
Oct 18, 2019
Jul 12, 2019
Sep 24, 2020
Jul 13, 2019
Jul 12, 2019
Sep 24, 2020
Oct 31, 2019
Oct 2, 2019
Oct 2, 2019

Repository files navigation

Grok

grok.net

Cross platform .NET grok implementation as a NuGet package

Windows Build Linux Build MacOs Build

GitHub release NuGet version Nuget License contributions welcome

How to Install

You can directly install this library from Nuget. There is package:

grok.net

PM> Install-Package Grok.Net

What is grok

Grok is a great way to parse unstructured log data into something structured and queryable. It sits on top of Regular Expression (regex) and uses text patterns to match lines in log files.

A great way to get started with building yours grok filters is this grok debug tool: https://grokdebug.herokuapp.com/

What can I use Grok for?

  • reporting errors and other patterns from logs and processes
  • parsing complex text output and converting it to json for external processing
  • apply 'write-once use-everywhere' to regular expressions
  • automatically providing patterns for unknown text inputs (logs you want patterns generated for future matching)

The syntax for a grok pattern is %{SYNTAX:SEMANTIC}

The SYNTAX is the name of the pattern that will match your text. SEMANTIC is the key.

For example, 3.44 will be matched by the NUMBER pattern and 55.3.244.1 will be matched by the IP pattern. 3.44 could be the duration of an event, so you could call it simply duration. Further, a string 55.3.244.1 might identify the client making a request. For the above example, your grok filter would look something like this:

%{NUMBER:duration} %{IP:client}

Examples: With that idea of a syntax and semantic, we can pull out useful fields from a sample log like this fictional http request log:

55.3.244.1 GET /index.html 15824 0.043

The pattern for this could be:

%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}

More about grok

How to use

Create a new instanse with grok pattern:

Grok grok = new Grok("%{MONTHDAY:month}-%{MONTHDAY:day}-%{MONTHDAY:year} %{TIME:timestamp};%{WORD:id};%{LOGLEVEL:loglevel};%{WORD:func};%{GREEDYDATA:msg}");

then prepare some logs to parse

string logs = @"06-21-19 21:00:13:589241;15;INFO;main;DECODED: 775233900043 DECODED BY: 18500738 DISTANCE: 1.5165
               06-21-19 21:00:13:589265;156;WARN;main;DECODED: 775233900043 EMPTY DISTANCE: --------";

You are ready to parse and print result

var grokResult = grok.Parse(logs);
foreach (var item in grokResult)
{
  Console.WriteLine($"{item.Key} : {item.Value}");
}

Build

On Windows:

build.ps1

On Linux/Mac:

build.sh

Contributing

Would you like to help make grok.net even better? We keep a list of issues that are approachable for newcomers under the good-first-issue label.

Also. please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

License

This project is licensed under the MIT License - see the LICENSE.md file for details

About

.net implementation of the grok πŸ“

Resources

License

Code of conduct

Stars

Watchers

Forks

Packages

No packages published

Languages

  • C# 54.2%
  • PowerShell 33.2%
  • Shell 12.6%