NLP / ML

Telling real news from fabricated, at 97% F1

A linguistic-feature + TF-IDF + XGBoost pipeline that separates fabricated articles from real reporting, wrapped in a paste-an-article app for live inference.

97%
macro F1 score
14.4K
held-out test articles
TF-IDF
text representation
XGBoost
top model vs. baselines
01Problem

Fabricated news spreads because it's cheap to produce and hard to screen at volume — no newsroom can fact-check the entire feed by hand. A useful first line of defence isn't a verdict on truth (that needs human judgement) but a fast triage signal: a model that flags which articles read like fabrication and deserve a closer look, so reviewers spend their time where it counts.

02Data

Tens of thousands of labelled news articles across real and fabricated classes, cleaned and preprocessed with spaCy, then split into training and a held-out test set of about 14,400 articles the model never saw during training.

03Method

Articles were vectorised with TF-IDF over the cleaned text, alongside engineered linguistic features, and fed to several classifiers — logistic regression and tree ensembles among them. XGBoost came out ahead. The model sits behind a Streamlit interface where you paste an article and get a real-or-fake call with the model's confidence.

04Finding

The XGBoost model reaches a 97% macro F1 on roughly 14,400 held-out articles — strong precision and recall on both classes, not just the easy majority.

Macro-averaging is the point: it proves the model catches fabricated articles, not that it's riding the base rate of real ones.

05Recommendation

Deploy this as a triage layer that ranks incoming articles by fabrication probability and routes the high-risk tail to human reviewers — it doesn't replace editorial judgement, it focuses it. The logged next step is a DistilBERT baseline, to test whether contextual embeddings beat the TF-IDF features on the hardest, most adversarial examples.

The live classifier

Streamlit · ready to deploy
Paste-an-article classifier

The Streamlit app is built and runs locally. Deploy it to Streamlit Cloud and it embeds here as a live demo — paste any article, get a real-or-fake call.

View the app code ↗