You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The grading scripts (modules/*/grading/grade.sh.nw) run per-repo/per-student checks but flatten them to P/F plus feedback text. For the vt-terminal/vt-git/vt-latex studies, each check failure is an observation of an undiscerned critical aspect — the grading rubric doubles as the coding scheme (full design: dbosk/vt-git#2).
To do:
Give every check a stable check-id, and map check-ids to the papers' critical aspects (the mapping lives in the papers; the ids live here).
After each grading run, append per-check outcomes to a structured log (CSV/JSONL): run timestamp, course, module, pseudonymous student id, check-id, outcome (pass/fail), attempt number.
Keep the log out of Canvas and out of git; store where the grading runs (server), pseudonymised.
Extraction tooling filters by the research-consent list at extraction time (grading itself runs for everyone); consent is collected by the papers' pre-quizzes ("my submissions in the course").
The existing results CSVs (e.g. modules/latex/grading/results-datintro20-LAB1.csv) already give attempt trajectories per student — keep them, they are the longitudinal backbone.
The grading scripts (modules/*/grading/grade.sh.nw) run per-repo/per-student checks but flatten them to P/F plus feedback text. For the vt-terminal/vt-git/vt-latex studies, each check failure is an observation of an undiscerned critical aspect — the grading rubric doubles as the coding scheme (full design: dbosk/vt-git#2).
To do:
Fits naturally into the ongoing review-all-modules grading-script work. Related: #127, #50; dbosk/vt-terminal#3, dbosk/vt-git#2, dbosk/vt-latex#2.