Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect multiline messages initial detection #48

Closed
piomar123 opened this issue Dec 28, 2018 · 4 comments
Closed

Incorrect multiline messages initial detection #48

piomar123 opened this issue Dec 28, 2018 · 4 comments
Assignees

Comments

@piomar123
Copy link

Supposedly an initial pattern detection in logs with multiline entries doesn't work properly. Currently if the first few messages include stack traces, the patterns are disabled because the rule matching more than 5/25 lines fails.
Example multiline message with a stack trace:

2018-12-28 13:50:42.137  [MAIN] ERROR com.example.myproject.Author: An error occurred
Exception in thread "main" java.lang.IllegalStateException: A book has a null property
        at com.example.myproject.Author.getBookIds(Author.java:38)
        at com.example.myproject.Bootstrap.main(Bootstrap.java:14)
Caused by: java.lang.NullPointerException
        at com.example.myproject.Book.getId(Book.java:22)
        at com.example.myproject.Author.getBookIds(Author.java:36)

Removing stack traces from the first few messages allows to detect patterns correctly for the whole file. The initial detection should ignore lines not matching "Message start pattern".

There is also a similar problem when there is fewer than 5 lines in the log.

Settings:
Message pattern: ^(\d{4}-\d\d-\d\d\s\d\d:\d\d:\d\d\.\d{3})\s+[^\[]*\[\s*([^\]]*)\]\s+(\w+)\s+(.*)$
Message start pattern: ^\d
Time: yyyy-MM-dd HH:mm:ss.SSS

@HarukaMa
Copy link

HarukaMa commented Feb 11, 2020

I have a very similar situation, and decided to play around with it for a while.

After many tries, It seems that the maximum multi-line part (the lines that are not the first line in a multi-line log) in the first 5 log lines must be less than 20 lines. Otherwise the log type detection will fail.

Here is an example of default Monolog format:
Pattern: ^\[(.*?)] (\w*)\.(\w*): (.+)$
Start pattern: ^\[\d
Time format: yyyy-MM-dd HH:mm:ss

[2020-02-11 23:38:00] local.INFO: Message 1
1
2
3
4
5
6
7
8
9
10
[2020-02-12 02:48:37] local.ERROR: Message 2
11
12
13
[2020-02-12 02:48:37] local.ERROR: Message 3
[2020-02-12 02:48:37] local.ERROR: Message 4
14
15
16
[2020-02-12 02:48:37] local.ERROR: Message 5
17
18
19
[2020-02-11 23:38:00] local.INFO: Message 6
[2020-02-11 23:38:00] local.INFO: Message 7
...

Any more multi-line part in the first 5 messages would cause the detection to fail, but adding new lines to all messages below would be fine.

This is actually quite annoying as there could be logs that begin with some long stack traces in the first few messages, and they would cause the whole file to become not recognized.

EDIT: I then realized that there must be more than 5 messages in the first 25 lines to make the plugin correctly recognize the format. Maybe making the limit more relaxed would be a good choice?

@gren236
Copy link

gren236 commented Mar 12, 2020

Got the same issue with Monolog. +1 for @HarukaMa 's proposal.

@wayheming
Copy link

same problem

@ArtemKar123
Copy link
Collaborator

Fixed in #165. Will be released in the next version. Thanks for your feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants