EvidenceNLP — Reddit Movie Discussion Classifier
A transparent NLP classifier that labels public Reddit movie-discussion comments by dominant communicative purpose: analysis, opinion, or reaction.

test accuracy
56.67%
macro F1
0.56999
gold examples
200
split
140 / 30 / 30
The task
EvidenceNLP classifies public Reddit movie-discussion comments by their primary purpose: analysis for reasoned or evidence-backed interpretation, opinion for personal judgment or prediction, and reaction for immediate emotional, humorous, or conversational responses.
What shipped
The repository includes a locked, stratified dataset split with no overlap between train, validation, and test IDs, annotation rules for borderline examples, reproducible scripts, evaluation reports, and a documented AI usage disclosure.
Model comparison
The classical TF-IDF plus balanced Logistic Regression pipeline is the production reference. It reached 56.67% accuracy and 0.56999 macro-F1 on the locked test set. The exploratory zero-shot Groq baseline scored 46.43% accuracy and 0.4323 macro-F1. A separate DistilBERT experiment scored lower, so it is retained as an honest negative result rather than presented as an improvement.
What the errors reveal
The hardest boundary is between short opinions and reactions, followed by analysis versus opinion when evidence is sparse. The model learns strong lexical cues for sentiment and explicit reasoning, but compressed fan-discussion language still makes dominant purpose ambiguous.