Assessment
Rubrics
Grading rewards quality of reasoning and growth over the course, not speed or right answers alone.
Grading weights
| Component | Weight | What’s assessed |
|---|---|---|
| Weekly deliverables | 40% | Clarity of trade-offs, quality of reasoning |
| Code review scorecards | 15% | Flaws caught, false alarms, severity ranking |
| Calibration improvement | 15% | Gap between confidence and accuracy, week 1 vs week 8 |
| Capstone proposal and defense | 30% | Framing, options, failure modes, response to pushback |
Reasoning rubric
Each written deliverable and the capstone are scored 1–4 on five criteria. A right answer with weak reasoning scores lower than a wrong answer with sound reasoning.
- Options: Considered more than one real alternative.
- Trade-offs: Named what is being given up, specifically.
- Failure modes: Anticipated how it could go wrong.
- Reversibility: Knows which parts are hard to undo and treated them with more care.
- Confidence: Stated how sure they are and what would change their mind.
What each score looks like
| Criterion | 1 · Missing | 2 · Surface | 3 · Solid | 4 · Strong |
|---|---|---|---|---|
| Options | One option, presented as the only one | A second option exists only to be dismissed | Two or more options a reasonable team could pick | Options span different approaches, including “do less” or “do nothing yet” |
| Trade-offs | No cost named | Generic costs (“more complex”) | Specific costs tied to this system and team | Costs are quantified or bounded, and the author says why they’re acceptable |
| Failure modes | None mentioned | Lists generic risks | Names likely failures with triggers | Ranks failures by likelihood and impact, with a mitigation or detection for each |
| Reversibility | Treats every part the same | Mentions reversibility in passing | Separates one-way from two-way doors | Spends effort in proportion: more care on the one-way parts, faster on the rest |
| Confidence | No confidence stated | A number with no basis | A number plus what it rests on | A number plus the specific evidence that would change the decision |
Use the 1–4 scale for feedback, not ranking learners against each other.
Code review scorecard
Week 2 PR reviews are scored against each PR’s answer key.
| Measure | How it’s counted |
|---|---|
| Catch rate | Planted flaws you found ÷ planted flaws in the answer key |
| Severity-weighted catch | High = 3, medium = 2, low = 1; points found ÷ points available |
| False alarms | Comments that flag a problem that isn’t one. Style preferences don’t count either way. |
| Severity ranking | Did you rank the flaws you found in the same order as the key? One point per flaw within one level of the key. |
Keep your scorecards. The deliverable for week 2 includes a personal checklist built from the flaws you missed most often.