Supervised Learning with scikit-learn
Hands-on 4-hour DataCamp course teaching classification, regression, tuning, and pipelines in scikit-learn with real datasets in Python.
Precision answers: of everything the model flagged as positive, how much really was positive? Recall answers: of all the real positives, how many did the model catch? High precision means few false alarms. High recall means few misses. On imbalanced problems like fraud, churn or disease detection, accuracy can look excellent while the model is useless, which is why precision and recall matter.
Classification metrics are where many new data scientists get caught out, in interviews and in real projects. A model that's "99% accurate" can miss nearly every case you actually care about. One worked example makes the difference clear.
A payment company checks 10,000 transactions. 100 of them are fraudulent (1%). A model flags 150 transactions as suspicious, and 80 of those really are fraud.
Every classification result falls into one of four boxes:
| Actually fraud | Actually legitimate | |
|---|---|---|
| Flagged as fraud | True positive (TP) = 80 | False positive (FP) = 70 |
| Not flagged | False negative (FN) = 20 | True negative (TN) = 9,830 |
Check: 80 + 70 = 150 flagged. 80 + 20 = 100 actual frauds. All four add up to 10,000.
Accuracy = (TP + TN) ÷ total = (80 + 9,830) ÷ 10,000 = 99.1%
That sounds excellent, until you notice that a "model" which flags nothing at all would score 99.0%, because 99% of transactions are legitimate. Accuracy barely separates a useful model from doing nothing.
Precision = TP ÷ (TP + FP) = 80 ÷ 150 = 53.3% About half of the flagged transactions are real fraud. The other half are legitimate customers whose payments get blocked or reviewed.
Recall = TP ÷ (TP + FN) = 80 ÷ 100 = 80% The model catches 80% of the fraud and misses 20 cases.
F1 score combines the two (their harmonic mean): 2 × P × R ÷ (P + R) = 2 × 0.533 × 0.80 ÷ 1.333 ≈ 0.64
Now you can have a real conversation: "we catch 80% of fraud, and about half our alerts are false alarms". That's something the business can act on, which "99.1% accurate" is not.
Most classifiers output a probability, and you choose the cut-off above which a case gets flagged. Changing it moves precision and recall in opposite directions.
| Threshold | Flagged | True fraud caught | Precision | Recall |
|---|---|---|---|---|
| Lower (flag more) | 150 | 80 | 53.3% | 80% |
| Higher (flag less) | 60 | 50 | 83.3% | 50% |
With a higher threshold, alerts become much more trustworthy (83% are real), but half the fraud gets through. There's no free lunch. Which point is right depends on what each kind of error costs.
| Situation | Costlier error | Prioritize | Example |
|---|---|---|---|
| Missing a positive is very costly | False negative | Recall | Fraud, disease screening, safety defects |
| A false alarm is very costly | False positive | Precision | Spam filters (losing a real email), automatic account bans |
| Both matter, and classes are imbalanced | Both | F1, or choose a threshold using business costs | Churn campaigns with limited budget |
The best approach is to put money on each box. If a missed fraud costs $500 on average and reviewing a false alarm costs $5 in staff time plus some customer annoyance, you can calculate the expected cost at each threshold and pick the cheapest, instead of arguing over abstract percentages.
Precision and recall are also a way of asking any decision process "what are we missing, and what are we wrongly flagging?" That includes marketing target lists, credit rules and even AI assistants that search documents. For a retrieval system, recall asks whether it found all the relevant documents, and precision asks how much of what it returned was actually relevant.
These metrics build on the basics of supervised learning. Supervised Learning with scikit-learn (DataCamp) has hands-on exercises on confusion matrices and thresholds. The Machine Learning Specialization explains precision, recall and skewed datasets very clearly (see our review). Browse all Machine Learning courses.
What's a good precision or recall? There's no universal number. It depends on base rates and on what each error costs. Compare against a simple baseline and against the business cost of each type of error.
Can I have both high precision and high recall? Only if the model separates the classes well. Better features and better models improve both. For a given model, moving the threshold trades one for the other.
When should I use F1? When you need one number to compare models and both error types matter roughly equally. If one error is clearly more expensive, optimize for that directly instead.
Is recall the same as sensitivity? Yes. They're two names for the same calculation, TP ÷ (TP + FN).
Accuracy asks "how often is the model right overall?", and on imbalanced data that question hides the failures that matter. Precision asks how trustworthy the alerts are, and recall asks how many real cases are caught. Build the confusion matrix, put a cost on each type of error, and choose the threshold that makes business sense.
Hands-on 4-hour DataCamp course teaching classification, regression, tuning, and pipelines in scikit-learn with real datasets in Python.
A three-course beginner program from DeepLearning.AI and Stanford Online covering supervised, unsupervised and neural network methods in Python.