Skip to content

Commit 3cd619b

Browse files
committed
emphasise skill acquisition in conclusion
Signed-off-by: Nathaniel <[email protected]>
1 parent 2be76ef commit 3cd619b

File tree

2 files changed

+23
-27
lines changed

2 files changed

+23
-27
lines changed

examples/case_studies/bayesian_sem_workflow.ipynb

Lines changed: 13 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -4616,7 +4616,11 @@
46164616
"cell_type": "code",
46174617
"execution_count": 32,
46184618
"id": "43912a6c",
4619-
"metadata": {},
4619+
"metadata": {
4620+
"tags": [
4621+
"hide-input"
4622+
]
4623+
},
46204624
"outputs": [
46214625
{
46224626
"data": {
@@ -4653,7 +4657,7 @@
46534657
"id": "e2f5f307",
46544658
"metadata": {},
46554659
"source": [
4656-
"This kind of sensitivity analysis is one approach to model validation, we can be even more clinical in our assessment of these models. We can perform parameter recovery exercises to properly validate that our model specification can identify the structural parameters of our complex SEM architecture. "
4660+
"This kind of sensitivity analysis is one approach to model validation, but we can be even more clinical in our assessment of these models. We can perform parameter recovery exercises to properly validate that our model specification can identify the structural parameters of our complex SEM architecture. "
46574661
]
46584662
},
46594663
{
@@ -5814,7 +5818,7 @@
58145818
"source": [
58155819
"#### The Parameter Recovery Process\n",
58165820
"\n",
5817-
"But we can actually assess the degree of parameter recovery because we know the true values. This is a pivotal part of the model building process, akin to how writer's read aloud their own work to test it for assonance, cogency and flow. In simulation based probabilistic modelling we should be able generate data from models with known parameters, and recover the latent parameter values through the inferential workflow. "
5821+
"With this set up we can actually assess the degree of parameter recovery because we know the true values. This is a pivotal part of the model building process, akin to how writer's read aloud their own work to test it for assonance, cogency and flow. In simulation based probabilistic modelling we are able generate data from models with known parameters, and recover the latent parameter values through the inferential workflow. We see how our model performs, much a like how a writer must test if their audience truly grasps their intended message."
58185822
]
58195823
},
58205824
{
@@ -5880,7 +5884,7 @@
58805884
"id": "41e491be",
58815885
"metadata": {},
58825886
"source": [
5883-
"Here we see how the posterior distributions can “recover” the true values within uncertainty ensuring the model is faithful to the data generating process. Were the effort at parameter recover to fail, we would equally have learned something about our model. Parameter recovery exercises helps discover issues of mis-specification or unidentified parameters. Put another way, they tell us how informative our data is with respect to our data generating model. Verlyn Klinkenborg starts his justly famous book _Several short sentences about writing_ with the following advice: \n",
5887+
"The posterior estimates can “recover” the true values within uncertainty, ensuring the model is faithful to the data generating process. Were the effort at parameter recover to fail, we would equally have learned something about our model. Parameter recovery exercises helps discover issues of mis-specification or unidentified parameters. Put another way, they tell us how informative our data is with respect to our data generating model. Verlyn Klinkenborg starts his justly famous book _Several short sentences about writing_ with the following advice: \n",
58845888
"\n",
58855889
"> \"Here, in short, is what i want to tell you. Know what each sentence says, What it doesn't say, And what it implies. Of these, the hardest is know what each sentence actually says\" - V. Klinkenborg\n",
58865890
"\n",
@@ -5894,7 +5898,7 @@
58945898
"source": [
58955899
"### Hypothesis Evaluation: How does group membership change direct effects?\n",
58965900
"\n",
5897-
"In this case we've encoded a differnce in the regression effects for each of the two groups. These effects are easily recovered with quantifiable measures of uncertainty around the treatment effects."
5901+
"In this case we've encoded a difference in the regression effects for each of the two groups. These effects are easily recovered with quantifiable measures of uncertainty around the treatment effects."
58985902
]
58995903
},
59005904
{
@@ -6587,7 +6591,7 @@
65876591
"id": "49fe7be1",
65886592
"metadata": {},
65896593
"source": [
6590-
"We can see here how our model structure now has two likelihood terms that are both based on the latent constructs `eta`. To demonstrate parameter recovery we need to sample from both outcome simultaneously. "
6594+
"We can see here how our model structure now has two likelihood terms that are both based on the latent constructs `eta`. To demonstrate parameter recovery we need to sample from both outcomes simultaneously. "
65916595
]
65926596
},
65936597
{
@@ -6957,9 +6961,9 @@
69576961
"source": [
69586962
"We can recover the inverse relationship we encoded in the outcomes between job-satisfaction and the choice to stay. This is encouraging. \n",
69596963
"\n",
6960-
"The \"action\" in human decision making is often understood to be driven by these hard-to-quantify constructs that determine motivation. SEM with a discrete choice component offers us a way to model these processes, while allowing for measurement error between the observables and the latent drivers of choice. Secondly, we are triangulating the values of the system between two sources of observable data. On the one hand, we measure latent constructs in the SEM with a range of survey measures (`JW1`, `JW2`, ... ) but then calibrate the consequences of that measurement against revealed choice data. This is a powerful technique for abstracting over the expressed attitudes of rational agents, and deriving an interpretable representation of the latent attitude in their expressions. These representations are then further calibrated against the observed choices made by the agent. \n",
6964+
"The \"action\" in human decision making is often understood to be driven by these hard-to-quantify constructs that determine motivation. SEM with a discrete choice component offers us a way to model these processes, while allowing for measurement error between the observables and the latent drivers of choice. Secondly, we are triangulating the values of the system between two sources of observable data. On the one hand, we measure latent constructs in the SEM with a range of survey measures (`JW1`, `JW2`, ... ) but then calibrate the consequences of that measurement against revealed choice data. This is a powerful technique for abstracting over the expressed attitudes of rational agents, and deriving an interpretable representation of the latent attitude in their expressions. These representations are then further calibrated against the observed choices made by the agents. \n",
69616965
"\n",
6962-
"This two-step of information compression and prediction serves to concisely quantify and evaluate the idiosyncratic attitudes of a complex agent. As we iteratively layer-in these constructs in our model development, we come to understand their baseline and interactive effects. This perspective helps us gauge the coherence between attitudes and actions of the agents under study. "
6966+
"This two-step of information compression and prediction serves to concisely quantify and evaluate the idiosyncratic attitudes of a population of complex agents. As we iteratively layer-in these constructs in our model development, we come to understand their baseline and interactive effects. This perspective helps us gauge the coherence between attitudes and actions of the agents under study. "
69636967
]
69646968
},
69656969
{
@@ -6969,17 +6973,11 @@
69696973
"source": [
69706974
"## Conclusion: Workflow and Craft in Statistical Modelling\n",
69716975
"\n",
6972-
"We have now seen how to articulate Structural Equation models and their variants in PyMC. The SEM workflow is, at heart, Bayesian in temperament: construct then check. Check then refine. Refine then expand. Both disciplines reject the checklist mentality of “fit once, report, move on.” Instead, they cultivate a slow, deliberate practice. Each discipline forces an apprenticeship in how assumptions shape understanding and how the world resists impositions of false structure. Each iteration is a dialogue between theory and evidence. At each juncture we ask whether this model speaks true? Whether this structure reflects the facts to hand. \n",
6976+
"We have now seen how to articulate Structural Equation models and their variants in PyMC. The SEM workflow is, at heart, Bayesian in temperament: construct then check. Check then refine. Refine then expand. Both disciplines reject the checklist mentality of “fit once, report, move on.” Instead, they cultivate a focused, deliberate practice. Each discipline forces an apprenticeship where skill is developed. Skill to handle how assumptions shape understanding and how the world resists impositions of false structure. Skill to find the right structures. Each iteration is a dialogue between theory and evidence. At each juncture we ask whether this model speaks true? Whether this structure reflects the facts to hand. \n",
69736977
"\n",
69746978
"In the end, the value of craft in statistical modeling lies not in improving benchmark metrics, but in the depth of understanding we cultivate through careful communication and justification. The Bayesian workflow reminds us that modeling is not the automation of insight but its deliberate construction. Our workflow is process of listening, revising, and re-articulating until the model speaks clearly. Like any craft, its worth is measured not by throughput but by fidelity: how honestly our structure reflects the world it seeks to describe. Each diagnostic, each posterior check, each refinement of a latent path is a form of attention — a small act of resistance against the flattening logic of metrics and checklists. These are the constructive thought processes that drive job-satisfaction. __To practice modeling as craft is to reclaim pride in knowing what our models say, what they do not say, and what they imply.__ To find, in that attention, the quiet satisfaction of meaningful work.\n"
69756979
]
69766980
},
6977-
{
6978-
"cell_type": "markdown",
6979-
"id": "2d12b3af",
6980-
"metadata": {},
6981-
"source": []
6982-
},
69836981
{
69846982
"attachments": {},
69856983
"cell_type": "markdown",

0 commit comments

Comments
 (0)