Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add n and rank to corpus-level turn dynamics #68

Open
mdingemanse opened this issue Sep 4, 2024 · 0 comments
Open

Add n and rank to corpus-level turn dynamics #68

mdingemanse opened this issue Sep 4, 2024 · 0 comments

Comments

@mdingemanse
Copy link

mdingemanse commented Sep 4, 2024

Back when the turn dynamics PR was merged and closed, some issues fell by the wayside. One is this:

we have two more utterance-level column that are really quite useful, but that relate to the corpus (and therefore often the language) as a whole:

rank : the rank of this utterance in the frequency distribution of utterances in this language
n : the number of tokens of this utterance type in this language

E.g. below you can see that the n of î (third row) is 53: there are 53 utterances like this in the language. Its rank is 1: it is the most frequently attested turn format in the language.

Debatable whether this is truly an utterance feature, but we do use it for a lot of things, e.g. to pull up the most frequent utterance formats in a language, identify 'streaks' of similarly frequent utterances, etc.

Right now, neither scikit-talk nor talkr provide this functionality, meaning that most of the basic examples in our publications cannot be replicated yet with production code as it currently stands. I would argue these features are basic enough to be included somewhere. Open to discussion about best place to do it. As the issue notes, these are really best done at corpus level (we are usually not interested in the relative frequency of utterances in a single source).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant