Back to projects
Web / ML2026Production reference

EvidenceNLP — Reddit Movie Discussion Classifier

A transparent NLP classifier that labels public Reddit movie-discussion comments by dominant communicative purpose: analysis, opinion, or reaction.

PythonNLPscikit-learnTF-IDFLogistic Regression
EvidenceNLP — Reddit Movie Discussion Classifier preview

test accuracy

56.67%

macro F1

0.56999

gold examples

200

split

140 / 30 / 30

The task

EvidenceNLP classifies public Reddit movie-discussion comments by their primary purpose: analysis for reasoned or evidence-backed interpretation, opinion for personal judgment or prediction, and reaction for immediate emotional, humorous, or conversational responses.

What shipped

The repository includes a locked, stratified dataset split with no overlap between train, validation, and test IDs, annotation rules for borderline examples, reproducible scripts, evaluation reports, and a documented AI usage disclosure.

Model comparison

The classical TF-IDF plus balanced Logistic Regression pipeline is the production reference. It reached 56.67% accuracy and 0.56999 macro-F1 on the locked test set. The exploratory zero-shot Groq baseline scored 46.43% accuracy and 0.4323 macro-F1. A separate DistilBERT experiment scored lower, so it is retained as an honest negative result rather than presented as an improvement.

What the errors reveal

The hardest boundary is between short opinions and reactions, followed by analysis versus opinion when evidence is sparse. The model learns strong lexical cues for sentiment and explicit reasoning, but compressed fan-discussion language still makes dominant purpose ambiguous.