Debugging: Runtime Errors Are Not the Only Errors
Learning goal: separate syntax/runtime errors from silent analytical errors.
"The code did not error" is often mistaken for "the analysis has no problem." But an analysis script can run completely cleanly while never filtering out draft or superseded rows, or while adding together values from different units (different metrics), or while a resubmitted record_id gets counted twice, producing a result that looks normal but is wrong. Python never flags this kind of error; only a person actively checking will catch it.
Runtime / Syntax Errors
- The code cannot run; Python raises an error directly
- Usually easy to notice, since execution stops
- Common causes: a mistyped column name, a type mismatch, division by zero
Silent Analytical Errors
- The code runs to completion with no error message at all
- Only noticed when the result itself looks wrong, or someone independently re-verifies it
- Common causes: not filtering by status, summing different unit metrics together, a duplicated record_id
import pandas as pd
df = pd.read_csv("DS202_Analysis_Package_Practice.csv", parse_dates=["period"])
# Never filtered to status == "validated", and never grouped by unit first,
# so values from different metrics (count, hours, usd...) get summed together
summary = df.groupby("group")["value"].sum()
print(summary)
# The code never errors, but this sum has no business meaning| Method | What it does |
|---|---|
| Row-count reconciliation | Do the row counts before/after filtering and before/after groupby match expectations? |
| Manual spot-check | Hand-calculate a small sample and compare it against the code's output |
| Edge-case inspection | Deliberately check how extreme values, missing values, or duplicate record_ids are handled |
| Sensitivity testing | Adjust filter conditions or grouping and see whether the conclusion is fragile |
If there were 480 rows before filtering, there should be fewer than 480 after; if the row count after filtering is identical to before, the filter never took effect at all. This kind of simple row-count check often catches the problem faster than reading the code over again.
- Do the row counts before and after filtering match what I expected?
- Have I picked a small sample and hand-calculated it myself to compare?
- How are extreme values, missing values, or a repeated record_id actually being handled?
- If I switch to a different, equally reasonable filter or grouping, does the conclusion change completely?
Warning
An analysis script runs to completion with no errors or warnings. Does that mean the analysis result must be correct?