Repository navigation
Import MPTinR model strings and EQN files (MPT 4/6, #255) - #487
Merged
Merged
Conversation
GidonFrischkorn
force-pushed
the
255-mpt-import
branch
from
October 4, 2026 08:06
df99a55 to
5a38893
Compare
mpt_from_string() and mpt_from_eqn() build an mpt() model from MPTinR-style text or EQN files, sanitizing names brms cannot use and reporting the renaming map.
GidonFrischkorn
force-pushed
the
255-mpt-import
branch
from
October 4, 2026 18:45
5a38893 to
da02087
Compare
…igration (#255) Review of PR #487 found that the <tree>_ prefix strip could merge two categories of a tree without an error, and that several MPTinR/TreeBUGS conventions failed or misled (readLines() input, # comments, the first line, numeric tree and category labels, trees with different category sets). mpt_from_eqn() now errors on prefix collisions, unknown categories names, restrictions and covariates not written as in the file, short or unparsable lines (by line number), and invalid tree or category labels; it skips only a lone-integer header and warns on a count mismatch. Both importers take impossible; mpt_from_eqn() takes tree_names; mpt_from_string() reads readLines() output, skips # comment lines and takes categories by tree. mpt_renaming lists every name with a renamed column, and the source is recorded. The help pages show by-name reshapes from MPTinR and TreeBUGS data.
…ce (#255) Guards the degrees of freedom of the identifiability check for trees with impossible categories, which have no branch and add no rows to the Jacobian.
… single-use variables (#255)
- A category label that is only its tree prefix (`old_`) is refused instead of shifting or merging categories. - A factor, NA or empty `categories` value, duplicated `tree_names` in mpt_from_string(), and a tree named twice in `impossible` get their own messages. - Sanitizing problems are reported together, with the remedy that fits parameters or categories; a parameter that sanitizes to an empty name is named. - The title hint leads for a header such as `6 # count` and is not shown for a one-line first tree whose expression is 1. - Docs: mapped category values are sanitized too; the first-line note on MPTinR/TreeBUGS is exact.
#255) 'Removing underscores and dots produces duplicated names: Do' named a string that appears nowhere in the file when D_o and D.o clash. The message now lists the spellings as the file writes them, before the '<tree>_' prefix strip for categories: 'D_o', 'D.o' become 'Do'. Reviewer C's message item from the #487 re-review.
Collaborator
Author
Update 2026-10-12: design merged (head eaf48c1)No change to the importers at this level since the last update. The level carries the current design (#486) forward: the two review fixes of the range check, the TreeBUGS recipe fix, the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #255.
Base branch:
255-mpt-design(stack rebased on develop 6f04c69, 2026-10-04; design 07494c7 merged in, which carries PRs 1 to 3 and develop 186250e; develop 222a7ef merged in 2026-10-08; head edba61b, 2026-10-09). PR 4 of 6 in the MPT stack. 7 files, +2818/-0.What this adds
mpt_from_string()reads MPTinR-style model text. Each line is one branch, blank lines separate trees, and the category comes from an inline# categorycomment or fromcategories. Thecategoriesargument can be a list by tree for models where trees use different response labels, or one shared character vector for trees that share labels. Comments (lines starting with#) are skipped without separating trees; only blank lines separate trees.mpt_from_eqn()reads EQN files (multiTree, MPTinR, TreeBUGS). Lines that end in the same category are summed. Thetree_namesargument maps file tree labels to bmm tree names, e.g.c("1" = "old", "2" = "new"). The first line of an EQN file is read as an equation count if it is a lone integer; otherwise it is parsed as an equation like any other line.Both functions take
impossible, a named list of response categories that cannot occur in certain trees, e.g.list(tree1 = c("a", "b")).Both pass
tree_id,covariates,simplex,restrictionsandlinkson tompt().User-visible changes and error messages
impossibleorcategoriesnames are unknown to the model (not in the trees or categories after import).mpt_from_eqn(): prefix-collision error when the<tree>_prefix strip creates duplicate category names (B1).mpt_from_eqn(): merge message when thecategoriesargument maps multiple categories of a tree onto one (Q1).mpt_from_eqn(): restrictions are checked to use only names from the file, before renaming.mpt_from_string(): now handlesreadLines()output correctly, preserving blank lines that separate trees.mpt_renamingattribute is a data frame with columnskind(parameter/tree/category),tree(NA for parameters),file_name,bmm_name, and a newrenamedcolumn (TRUE if a name changed). The import message prints only rows where a name changed. Tree rows track the tree-label mapping.mpt_from_eqn(): error when covariates are symbols that are not in the EQN file.mpt_from_eqn(): error when a category name does not start with a letter (points to thecategoriesargument).mpt_from_eqn()help page: states that bmm reads the first line unless it is the equation count, so users should delete a title line or start it with#for TreeBUGS/MPTinR files.Design decisions
File format and line skipping (decision 1). The first line of an EQN file is skipped only if it is a single integer. When skipped, the count is compared with the number of equation lines that follow and a warning is emitted on a mismatch. A first line that is not a lone integer is parsed as an equation. A line with fewer than three fields anywhere is an error naming the line number and its text.
Categories argument as list by tree (mpt_from_string). The
categoriesargument can be either a character vector shared by all trees, or a list with one vector per tree. In either form, entries are matched to lines by position. A shared vector suits trees that use the same responses in the same order; a list suits trees with different response sets.Impossible categories (decision 2). The
impossibleargument takes a named list of categories that cannot occur in each tree, using the category names after mapping/sanitizing. Unknown tree names or categories error by naming the trees or categories that are not in the model.Tree name mapping (reviewer default Q5). The
tree_namesargument maps file tree labels to bmm names, e.g.c("1" = "old", "2" = "new"). bmm tree names must start with a letter and contain only letters, digits, or underscores. Unmapped labels keep their name; unknown keys error.Category prefix strip and name sanitization (reviewer default Q9; existing design). Categories in EQN files often carry a
<tree>_prefix to distinguish trees (e.g.old_hit,new_fa). The prefix is stripped if the category is not mapped by thecategoriesargument. All remaining underscores and dots are then stripped (brms does not allow them in non-linear parameter names). Names that would clash after this are errors. Restrictions are written with the file's own names and are renamed together with the parameters.Renaming map (reviewer default Q10). The
mpt_renamingattribute is a data frame with columnskind(parameter/tree/category),tree(NA for parameters),file_name,bmm_name, andrenamed(TRUE if the name changed). It holds one row per parameter, tree label and category. The import message prints only the rows where a name changed, in a compact multi-line table with one line per parameter rename and categories grouped per tree.Provenance (reviewer default Q6).
mpt_from_eqn()storesattr(model, "mpt_source") <- list(file = normalizePath(file), md5 = unname(tools::md5sum(file)), lines = raw_lines).mpt_from_string()storeslist(text = text_as_one_string).Comments in mpt_from_string (decision 4). A line whose first non-blank character is
#is a comment and is skipped (not a tree separator; only blank lines separate trees). An inlineexpression # categorycomment labels that line. This differs from MPTinR, where everything after#is ignored and#lines count as blank.Validation
The Al Hadhrami, Bartsch & Oberauer (2025) binding-model validation ran on 2 October from the published EQN files and still needs a rerun on this head. The imported model objects (unitization and hybrid) are identical to that import. The validation checked 3 trees, 10 categories, 12 and 18 free parameters, probit,
(1 |p| id).The identifiability check of PR 1 on these imports, at this head: unrestricted, Jacobian rank 19 of 21 (unitization) and 21 of 27 (hybrid), which the parameter count does not catch; with the design constants, 12 of 12 and 18 of 18.
Test coverage
The #487 review round: 120 tests, 487 expectations in the import tests, 0 failed. Its code verifier found no silent wrong model in about 160 adversarial inputs and killed 79 of 79 mutants. Tests cover model construction, file format errors, category mapping, the merge message (Q1), the renaming map with its
renamedcolumn and tree rows, impossible categories, tree names, the prefix-collision error (B1), restrictions in file names, unknown and invalid category names, the covariate-symbol check,readLines()input and the header advice. One test added at the merge-up (d1dc7d2): the identifiability check counts only the categories a tree can produce. The re-review round took the import file from 37 tests / 110 expectations to 41 / 128; the 120 / 487 figures above are the wholemptfilter, not this file.Changes since the re-review of 2026-10-09 (e579ffb)
An internal review (2026-10-09) gave the verdict MERGE AFTER FIXES. One silent wrong model and four narrower gaps, fixed in one round plus a nit round, checked by three independent reviewers:
old_as a label in treeoldleft an empty name after the prefix strip, which the renaming map then dropped; the import went through with the wrong categories. This was the one silent-wrong-model finding.categoriesortree_namesvalue that is a factor,NAor the empty string, a duplicatedtree_namesentry inmpt_from_string(), and a tree named twice inimpossibleeach get their own message instead of a rawnzchar()error (98aaaac).mpt_from_eqn()now refuses a list of length-1 values forcategories— a behaviour change against a6a55ae, and the only one.6 # countand is not shown for a one-line first tree whose expression is1; the merge message inmpt_from_string()matchesmpt_from_eqn()'s (e9851ad, b4a444a).Verification: every one of the 13 reference imports identical to a6a55ae apart from the intended refusals, MPTinR parameter sets equal (21 and 27), author mutants 79 of 79 killed,
check()0/0/0 before and after. The committed tree's diff hash matched the independent reviewer's.Since 2026-10-08 (develop sync 2, open items)
D_oandD.ogave "produces duplicated names: 'Do'", a name that appears nowhere in the file; now "'D_o', 'D.o' become 'Do'" (98061b2). The two pinned tests fail on the old code and pass on the new.impossible = c(s = "E1")no longer ends in "Every entry needs a name." (only an unnamed list does). A title whose third word is exactly1(Model v 1) loses the title hint by design, becausecatch c1 1is a valid one-line tree;Model v 1 testkeeps the hint.Gates on
255-mpt-import(edba61b)mpt|distributions|conditional|pp_check|update|data_check|helpers-prior|default-priors: 0 failed, 0 errors, 0 skipped, 2503 passed.check()on anrsynccopy: Status OK, 0 errors, 0 warnings, 0 notes.print()of the seven reference models identical except theparameter_info()line of e7c8504, against bb4b584.document(): no drift.git merge-treeagainst develop 222a7ef: clean.Update 2026-10-13: develop through #546 carried up, MPT fits in tests/local (head b1fc508)
This update replaces the "Gates" section above, if any.
tests/local. Nothing else changed in this level's own files.Measured at this head:
document()no drift. The full suite andR CMD checkwere run at the tip of the stack (#489).CI on this head: R-CMD-check passes on macOS, Ubuntu and Windows (Windows check step 9 minutes), tests-source passes within the time budget (suite 570 s), and file-size passes.