Modifications to the C lexer

[samuelmarquis]

I want to add a few types to the C lexer. Specifically--I think having size_t but no ssize_t is a bit of an oversight. I'd love to also have the Linux kernel's u8 u16 u32 u64 types, and their signed (? I think, gonna have to do a little more digging) s8 s16 s32 s64 equivalents. Obviously it's not ideal to include too much from specific C code bases... but hey, it's the Linux kernel.

[orbitalquark]

No ssize_t does sound like an oversight.

I think I'm fine with adding Linux kernel types as long as they are separate from normal types, such as lex:tag(lexer.TYPE .. '.linux', lex:word_match(lexer.TYPE .. '.linux')) and a corresponding lex:set_word_list(lexer.TYPE .. '.linux', {'u8', 'u16', ... }).

[samuelmarquis]

How are common structs best handled? I see that tm and a few others are listed in the types block.

Not gonna go overboard here, but I'd like to at least get everything in include/linux/types.h.

[samuelmarquis]

Especially, there's a few of these with names like list_head that might be fairly common, which I'll omit for the time being until there exists a better way to scope the linux stuff.

[orbitalquark]

I don't know enough about those common structs to comment on this. I might be able to comment on a PR that had this with more context.

[samuelmarquis]

How do you feel about lexing any ALL_CAPS_NAME as a constant? That tag is currently unused in this lexer.

[samuelmarquis]

Which begs the question, should ALL_CAPS_FUNCTIONS() be parsed separately from how they are now? (leaning towards "probably not" here)

[orbitalquark]

I'm not in favor because that's a convention, and not a rule. Furthermore, as you point out, macro functions commonly use all-caps names, so there's some ambiguity.

[samuelmarquis]

PR #125 sent for this.

Another thought; maybe there's some value in parsing dunder-prefixed names as builtin functions/constants, or something like this?

[samuelmarquis]

& now that I'm thinking about it... it might be prudent (and reduce a TON of complexity) to just tag anything with a _t suffix as a type.

[orbitalquark]

Possibly. Still, it's a convention and not a rule. I think explicit is better than implicit, but I could see universal _t highlighting with a tag like lexer.TYPE .. '.posix'. That's more explicit in my opinion.

[samuelmarquis]

Seems a solid compromise, I'll sketch something out for this.

[orbitalquark]

Actually after looking over your PR, I'm remembering that the whole point with word lists is that it allows clients to easily extend word lists for their domain. I prefer to keep domain-specific things like Linux types, constants, etc. out of an ANSI/ISO C lexer. Users that want to use the C lexer for highlighting Linux source code could do one of two things:

1. Write a new lexer that inherits from the C lexer and modifies type, constant, etc. rules to add Linux-specific stuff (using lex:modify_rule()).
2. Call a lexer's lex:set_word_list() with Linux-specific words and the append argument being true.

[samuelmarquis]

Super reasonable--you can just toss the second commit and keep the 1st & 3rd, or I can rebase w/o it if that's easier.

[orbitalquark]

We'll go through the current review process so I don't miss anything if that's okay with you :)

[orbitalquark]

Oh, you know why there is no ssize_t? It's because it's not standard C. It looks like a GNU extension, or Linux-defined thing (sys/types.h).

Sadly, it appears that only the float change will make it into your PR :(