Chart Review 1.0.0 Released!

[mikix]

This is a new project that we've been working on!

Chart Review is a new command line tool for calculating accuracy/agreement between two chart annotators. F1, sensitivity, confusion matrix, etc. for all the labels in your study.

Example:

$ chart-review accuracy jane john
accuracy-jane-john:
F1     Sens   Spec   PPV    NPV    TP  FN  TN  FP  Label           
0.929  0.958  0.908  0.901  0.961  91  4   99  10  *               
0.895  0.895  0.938  0.895  0.938  17  2   30  2   cough     
0.815  0.917  0.897  0.733  0.972  11  1   35  4   fever  
0.959  1.0    0.812  0.921  1.0    35  0   13  3   headache   
0.966  0.966  0.955  0.966  0.955  28  1   21  1   stuffy-nose

Why is this part of the Cumulus Project?

Well, it doesn't need to be used with Cumulus, but it gets a lot more useful when you do.

Not only can you compare two human reviewers, but you can compare external annotations from NLP or ICD10 codes that you can pull back out of Cumulus's Athena database. Which is particularly helpful during study design or when experimenting with NLP strategies.

Installing

pip install chart-review

Documentation

Read more on our documentation site