Skip to content

Conversation

@justin808
Copy link
Member

@justin808 justin808 commented Nov 9, 2025

Summary

This PR implements comprehensive RBS type checking for both the main gem and Pro package as a follow-up to #1945.

Changes

  1. CI Integration

    • Added rake rbs:validate to main lint workflow
    • Added RBS validation to Pro gem lint workflow
    • Added Steep static type checker to CI pipeline
  2. Runtime Type Checking

    • Configured RSpec to run with RBS runtime checking for gem tests
    • Tests now run with RBS_TEST_TARGET='ReactOnRails::*' RUBYOPT='-rrbs/test/setup'
    • Provides runtime validation of type signatures during test execution
  3. Steep Static Type Checker

    • Added steep gem to development dependencies
    • Created Steepfile configuration for static type analysis
    • Added rake rbs:steep task for running static type checks
    • Added rake rbs:all task to run both validation and steep checks
  4. Pro Gem RBS Types

    • Created sig/ directory structure for Pro gem
    • Added type signatures for:
      • ReactOnRailsPro module
      • Configuration class with all attributes
      • Error, Cache, and Utils modules
    • Foundation for expanding type coverage in Pro package

Implementation Details

This PR follows best practices from Evil Martians' "Climbing Steep Hills" article:

  • Static validation with rbs validate
  • Runtime checking with rbs/test/setup
  • Static analysis with Steep

The combination of these three approaches provides comprehensive type safety:

  • Validation: Ensures RBS signatures are syntactically correct and internally consistent
  • Runtime checking: Verifies actual method calls match their signatures during tests
  • Static analysis: Catches type errors in Ruby code without running it

Test Plan

  • RBS validation passes locally
  • RuboCop passes
  • All files properly formatted
  • CI validates RBS signatures
  • CI runs Steep checks

🤖 Generated with Claude Code


This change is Reviewable

Summary by CodeRabbit

  • Chores

    • CI now validates RBS type signatures during linting; an optional Steep check is present but disabled by default.
    • Development/test dependencies added to support RBS/type-checking tooling.
    • Rake tasks added to run/validate/list RBS checks and to run Steep; test tasks can enable runtime RBS checks when available.
    • Lint config updated to exclude the type-checker config file from filename checks.
  • Documentation

    • Added comprehensive guidance on RBS/Steep usage, runtime type-checking, and reproducing CI checks.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 9, 2025

Warning

Rate limit exceeded

@justin808 has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 15 minutes and 54 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 71f5f84 and ec792e8.

⛔ Files ignored due to path filters (1)
  • Gemfile.lock is excluded by !**/*.lock
📒 Files selected for processing (18)
  • .github/workflows/lint-js-and-ruby.yml (1 hunks)
  • .github/workflows/pro-lint.yml (1 hunks)
  • .rubocop.yml (1 hunks)
  • CLAUDE.md (2 hunks)
  • Gemfile.development_dependencies (1 hunks)
  • Steepfile (1 hunks)
  • rakelib/rbs.rake (3 hunks)
  • rakelib/run_rspec.rake (1 hunks)
  • react_on_rails_pro/Gemfile.development_dependencies (1 hunks)
  • react_on_rails_pro/rakelib/rbs.rake (1 hunks)
  • react_on_rails_pro/sig/react_on_rails_pro.rbs (1 hunks)
  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs (1 hunks)
  • react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs (1 hunks)
  • react_on_rails_pro/sig/react_on_rails_pro/error.rbs (1 hunks)
  • react_on_rails_pro/sig/react_on_rails_pro/utils.rbs (1 hunks)
  • sig/react_on_rails.rbs (1 hunks)
  • spec/react_on_rails/rake_tasks_spec.rb (1 hunks)
  • spec/react_on_rails/rbs_runtime_checking_spec.rb (1 hunks)

Walkthrough

Adds RBS signatures for ReactOnRailsPro, adds RBS/Steep tooling and Steepfile, updates rake tasks and RSpec hooks to run/enable RBS runtime checks, adjusts CI workflows to validate RBS signatures (Steep check kept commented/TODO), and updates docs and dev dependencies.

Changes

Cohort / File(s) Summary
CI / Workflows
\.github/workflows/lint-js-and-ruby.yml, \.github/workflows/pro-lint.yml
Remove dummy-app Yarn install step; add "Validate RBS type signatures" step that runs RBS validation (bundle exec rake rbs:validate / bundle exec rbs -I sig validate); Steep invocation left commented/TODO.
Dev dependencies
Gemfile.development_dependencies, react_on_rails_pro/Gemfile.development_dependencies
Add steep to project dev/test Gemfile (require: false) and rbs to pro dev/test Gemfile (require: false).
Steep configuration
Steepfile
Add Steep configuration targeting lib with signature dir sig, listed core files, diagnostics config, and library includes.
Rake automation & test env
rakelib/rbs.rake, react_on_rails_pro/rakelib/rbs.rake, rakelib/run_rspec.rake
Use Open3.capture3 to run rbs (validate) with captured stdout/stderr and proper exit handling; add rbs:steep (runs steep check), rbs:all (depends on validate + steep), rbs:list and alias rbs:check; add rbs_runtime_env_vars helper to enable RBS runtime hook in RSpec tasks and pass env vars into :gem, :dummy, and :dummy_no_turbolinks tasks.
ReactOnRailsPro RBS signatures
react_on_rails_pro/sig/react_on_rails_pro.rbs, react_on_rails_pro/sig/react_on_rails_pro/cache.rbs, react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs, react_on_rails_pro/sig/react_on_rails_pro/error.rbs, react_on_rails_pro/sig/react_on_rails_pro/utils.rbs
Add RBS signature files declaring ReactOnRailsPro module APIs: Configuration class (constants, typed attrs, initializer, and public methods), Cache class methods, Utils helpers, and Error exception type.
Lint config
\.rubocop.yml
Exclude Steepfile from Naming/FileName cops.
Docs
CLAUDE.md
Add "RBS Type Checking" documentation: quick start, runtime checking behavior, disabling runtime checks, adding signatures, files currently type-checked, Pro package guidance, and CI/pre-commit notes.
Tests
spec/react_on_rails/rake_tasks_spec.rb, spec/react_on_rails/rbs_runtime_checking_spec.rb
Add specs validating presence and prerequisites of rbs rake tasks; add runtime RBS test suite that runs only when RBS is available and enabled, asserting runtime type-check behavior for configuration and helpers.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant CI as CI Job
  participant Rake as Rake
  participant RBS as rbs CLI
  participant Steep as steep CLI
  participant Test as RSpec

  CI->>Rake: invoke rbs validate step (bundle exec rake rbs:validate)
  Rake->>RBS: run `rbs -I sig validate` (Open3.capture3)
  alt RBS passes
    RBS-->>CI: exit 0 (success)
  else RBS fails
    RBS-->>CI: non‑zero exit + captured stdout/stderr
  end

  Note over Rake,Steep: Local developer flow
  Rake->>Rake: `rake rbs:validate` → runs RBS validate
  Rake->>Steep: `rake rbs:steep` → runs `steep check` (captures output)
  Rake->>Rake: `rake rbs:all` depends on validate + steep

  Note over Test: Runtime hook injection
  Rake->>Test: run specs with env from `rbs_runtime_env_vars` (sets RBS_TEST_TARGET, RUBYOPT for RBS test hook)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • Pay extra attention to:
    • react_on_rails_pro/sig/.../configuration.rbs (large API surface, types vs runtime behavior).
    • rakelib/rbs.rake and react_on_rails_pro/rakelib/rbs.rake (Open3 usage, exit handling, message clarity).
    • rakelib/run_rspec.rake (conditional RBS loading, correct RUBYOPT composition, env propagation).
    • CI workflow context/working-directory for running rbs -I sig validate and any cached paths.

Possibly related issues

Possibly related PRs

Suggested labels

review-needed

Suggested reviewers

  • Judahmeek
  • alexeyr-ci
  • AbanoubGhadban

Poem

🐰 I hopped through sigs with nimble feet,
Gems and tasks now line the street,
CI checks types, Rake hums a tune,
Steep waits silent — re-enable soon,
A little rabbit — tests pass sweet! 🥕

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: adding RBS validation and runtime type checking to CI, which is the central focus of all modifications across workflow files, rake tasks, and test configurations.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@justin808 justin808 requested a review from alexeyr-ci2 November 9, 2025 06:19
@claude
Copy link

claude bot commented Nov 9, 2025

Code Review: Add RBS validation and runtime type checking to CI

Overall Assessment

This PR successfully implements comprehensive RBS type checking infrastructure following industry best practices. The implementation is well-structured and provides three complementary layers of type safety: validation, runtime checking, and static analysis.

Strengths

  1. Comprehensive Type Safety Approach - Excellent combination of three type checking methods
  2. CI Integration - Clean integration into existing GitHub Actions workflows
  3. Code Quality - Clean, well-documented rake tasks with proper error handling
  4. Pro Package RBS Signatures - Type signatures appear accurate based on implementation

Critical Issues

1. Runtime Type Checking May Cause CI Failures

Location: rakelib/run_rspec.rake:27

Adding runtime type checking will cause tests to fail if method signatures don't match actual usage. Recommendation: Verify all existing RBS signatures are accurate before merging, or document expected test failures.

2. Type Signature Accuracy

Location: react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs:11-12

DEFAULT_DEPENDENCY_GLOBS and DEFAULT_EXCLUDED_DEPENDENCY_GLOBS use Array[untyped] - these should be Array[String].

3. Pro Package Incomplete

The Pro package only has validation, not runtime checking or Steep analysis. Consider adding the three-layer approach to Pro for consistency.

4. Documentation Gap

No updates about RBS type checking in documentation. Add a section explaining how to run checks and add signatures.

Performance Considerations

  • RBS validation: fast (under 1 second)
  • Steep: may add 30s-2min to CI on large codebases
  • Runtime checking: minimal overhead

Monitor CI times after merge. Consider running Steep only on main/PR branches if it becomes slow.

Pre-Merge Checklist

  • bundle exec rake rbs:validate passes locally
  • bundle exec rake rbs:steep passes locally
  • rake run_rspec:gem with runtime checking passes
  • Fix Array[untyped] in Pro configuration.rbs
  • Verify CI passes with all checks enabled

Summary

High-quality PR adding valuable type safety. Main concern: ensure runtime checking doesn't break tests. Recommendation: Approve with minor changes - fix Array[untyped] and verify tests pass.

Great work! The three-layer approach is exactly right.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
.github/workflows/pro-lint.yml (1)

126-127: Consider using rake task for consistency.

The main lint workflow (.github/workflows/lint-js-and-ruby.yml) uses bundle exec rake rbs:validate and bundle exec rake rbs:steep, but this Pro workflow uses the direct command bundle exec rbs -I sig validate. Additionally, the Steep check is missing here.

Consider updating to match the main workflow for consistency:

-      - name: Validate RBS type signatures
-        run: bundle exec rbs -I sig validate
+      - name: Validate RBS type signatures
+        run: bundle exec rake rbs:validate
+
+      - name: Run Steep type checker
+        run: bundle exec rake rbs:steep

Benefits:

  • Consistent approach across workflows
  • Standardized success/failure messaging from rake tasks
  • Includes Steep static analysis for more comprehensive type checking
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 88f1367 and 71222f6.

⛔ Files ignored due to path filters (1)
  • Gemfile.lock is excluded by !**/*.lock
📒 Files selected for processing (11)
  • .github/workflows/lint-js-and-ruby.yml (1 hunks)
  • .github/workflows/pro-lint.yml (1 hunks)
  • Gemfile.development_dependencies (1 hunks)
  • Steepfile (1 hunks)
  • rakelib/rbs.rake (1 hunks)
  • rakelib/run_rspec.rake (1 hunks)
  • react_on_rails_pro/sig/react_on_rails_pro.rbs (1 hunks)
  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs (1 hunks)
  • react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs (1 hunks)
  • react_on_rails_pro/sig/react_on_rails_pro/error.rbs (1 hunks)
  • react_on_rails_pro/sig/react_on_rails_pro/utils.rbs (1 hunks)
🧰 Additional context used
🧠 Learnings (14)
📓 Common learnings
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: lib/react_on_rails/helper.rb:190-197
Timestamp: 2025-02-18T13:08:01.477Z
Learning: RSC support validation in React on Rails Pro is handled through a chain of validations:
1. Pro version check in `run_stream_inside_fiber`
2. RSC support check during pack generation via `ReactOnRailsPro.configuration.enable_rsc_support`
3. RSC support validation during component registration
This makes additional validation in the helper methods unnecessary.
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1875
File: lib/react_on_rails/utils.rb:112-124
Timestamp: 2025-10-23T17:22:01.074Z
Learning: In React on Rails, when Pro is installed but not licensed, the intended behavior is to raise an error on boot. The `react_on_rails_pro?` method validates licenses and should raise errors early (including during path resolution in methods like `server_bundle?`) to enforce licensing requirements rather than failing later with obscure errors.
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: lib/react_on_rails/helper.rb:190-197
Timestamp: 2025-02-18T13:08:01.477Z
Learning: RSC support validation is handled in deeper level calls of the React on Rails Pro codebase, so it doesn't need to be validated again in the `rsc_payload_react_component` helper method.
📚 Learning: 2025-09-16T08:01:11.146Z
Learnt from: justin808
Repo: shakacode/react_on_rails PR: 1770
File: lib/generators/react_on_rails/templates/base/base/app/javascript/src/HelloWorld/ror_components/HelloWorld.client.jsx:2-2
Timestamp: 2025-09-16T08:01:11.146Z
Learning: React on Rails uses webpack CSS Modules configuration with namedExports: true, which requires the import syntax `import * as style from './file.module.css'` rather than the default export pattern. This configuration enables better tree shaking and bundle size optimization for CSS modules.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/utils.rbs
  • react_on_rails_pro/sig/react_on_rails_pro.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/error.rbs
📚 Learning: 2025-02-12T16:38:06.537Z
Learnt from: Romex91
Repo: shakacode/react_on_rails PR: 1697
File: package-scripts.yml:28-28
Timestamp: 2025-02-12T16:38:06.537Z
Learning: The file `node_package/lib/ReactOnRails.full.js` is autogenerated during the build process and should not be present in the repository.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/utils.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/error.rbs
📚 Learning: 2025-10-23T17:22:01.074Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1875
File: lib/react_on_rails/utils.rb:112-124
Timestamp: 2025-10-23T17:22:01.074Z
Learning: In React on Rails, when Pro is installed but not licensed, the intended behavior is to raise an error on boot. The `react_on_rails_pro?` method validates licenses and should raise errors early (including during path resolution in methods like `server_bundle?`) to enforce licensing requirements rather than failing later with obscure errors.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/utils.rbs
  • react_on_rails_pro/sig/react_on_rails_pro.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/error.rbs
📚 Learning: 2025-09-15T21:24:48.207Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1781
File: node_package/src/ClientSideRenderer.ts:82-95
Timestamp: 2025-09-15T21:24:48.207Z
Learning: In React on Rails, the force_load feature includes both explicit `data-force-load="true"` usage and the ability to hydrate components during the page loading state (`document.readyState === 'loading'`). Both capabilities require a Pro license, so the condition `!railsContext.rorPro && (isComponentForceLoaded || document.readyState === 'loading')` correctly gates both scenarios.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/utils.rbs
  • react_on_rails_pro/sig/react_on_rails_pro.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/error.rbs
📚 Learning: 2025-02-18T13:08:01.477Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: lib/react_on_rails/helper.rb:190-197
Timestamp: 2025-02-18T13:08:01.477Z
Learning: RSC support validation in React on Rails Pro is handled through a chain of validations:
1. Pro version check in `run_stream_inside_fiber`
2. RSC support check during pack generation via `ReactOnRailsPro.configuration.enable_rsc_support`
3. RSC support validation during component registration
This makes additional validation in the helper methods unnecessary.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/utils.rbs
  • react_on_rails_pro/sig/react_on_rails_pro.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/error.rbs
📚 Learning: 2025-04-26T21:55:55.874Z
Learnt from: alexeyr-ci2
Repo: shakacode/react_on_rails PR: 1732
File: spec/dummy/client/app-react16/startup/ReduxSharedStoreApp.client.jsx:40-44
Timestamp: 2025-04-26T21:55:55.874Z
Learning: In the react_on_rails project, files under `app-react16` directories are copied/moved to corresponding `/app` directories during the conversion process (removing the `-react16` suffix), which affects their relative import paths at runtime.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/utils.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs
📚 Learning: 2025-02-13T16:50:47.848Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: node_package/src/clientStartup.ts:18-21
Timestamp: 2025-02-13T16:50:47.848Z
Learning: In the react_on_rails module, the `reactOnRailsPageUnloaded` function in clientStartup.ts is intentionally kept private as it's only used internally as a callback for `onPageUnloaded`.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/error.rbs
📚 Learning: 2025-07-08T05:57:29.630Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1745
File: node_package/src/RSCRequestTracker.ts:8-14
Timestamp: 2025-07-08T05:57:29.630Z
Learning: The global `generateRSCPayload` function in React on Rails Pro (RORP) is provided by the framework during rendering requests, not implemented in application code. The `declare global` statements are used to document the expected interface that RORP will inject at runtime.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/error.rbs
📚 Learning: 2024-06-27T05:22:46.156Z
Learnt from: justin808
Repo: shakacode/react_on_rails PR: 1622
File: spec/dummy/spec/rake/assets_precompile_rake_spec.rb:12-12
Timestamp: 2024-06-27T05:22:46.156Z
Learning: When stubbing environment variables in RSpec tests, use `before` and `after` hooks to ensure that the original values are restored after the tests, preventing any side effects on other tests. Example provided by justin808:
```ruby
describe "My test" do
  before do
    original_value = ENV["VARIABLE_NAME"]
    allow(ENV).to receive(:[]).with("VARIABLE_NAME").and_return("stubbed_value")
  end

  after do
    allow(ENV).to receive(:[]).with("VARIABLE_NAME").and_call_original
    ENV["VARIABLE_NAME"] = original_value
  end

  it "tests something" do
    # Your test code here
  end
end
```
This practice ensures test isolation and reliability.

Applied to files:

  • rakelib/run_rspec.rake
📚 Learning: 2025-02-13T14:29:49.267Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: spec/react_on_rails/utils_spec.rb:218-218
Timestamp: 2025-02-13T14:29:49.267Z
Learning: In RSpec tests, prefer using local variables over constants within test blocks to avoid constant redefinition warnings and maintain better test isolation.

Applied to files:

  • rakelib/run_rspec.rake
📚 Learning: 2024-12-12T13:07:09.929Z
Learnt from: alexeyr-ci
Repo: shakacode/react_on_rails PR: 1644
File: node_package/src/ReactOnRailsRSC.ts:87-87
Timestamp: 2024-12-12T13:07:09.929Z
Learning: When handling errors in 'node_package/src/ReactOnRailsRSC.ts', include the error stack in error messages in development and test environments to aid debugging.

Applied to files:

  • rakelib/run_rspec.rake
  • react_on_rails_pro/sig/react_on_rails_pro/error.rbs
📚 Learning: 2025-02-18T13:08:01.477Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: lib/react_on_rails/helper.rb:190-197
Timestamp: 2025-02-18T13:08:01.477Z
Learning: RSC support validation is handled in deeper level calls of the React on Rails Pro codebase, so it doesn't need to be validated again in the `rsc_payload_react_component` helper method.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs
  • react_on_rails_pro/sig/react_on_rails_pro/error.rbs
📚 Learning: 2025-02-13T16:50:26.861Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: node_package/src/turbolinksUtils.ts:34-36
Timestamp: 2025-02-13T16:50:26.861Z
Learning: In React on Rails, when checking for Turbolinks version 5 using `turbolinksVersion5()`, always ensure `Turbolinks` exists first by checking `turbolinksInstalled()` to prevent TypeError when accessing properties.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/error.rbs
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
  • GitHub Check: build
  • GitHub Check: build-dummy-app-webpack-test-bundles
  • GitHub Check: build-dummy-app-webpack-test-bundles (3.2, 20, minimum)
  • GitHub Check: build-dummy-app-webpack-test-bundles (3.4, 22, latest)
  • GitHub Check: rspec-package-tests (3.4, latest)
  • GitHub Check: rspec-package-tests (3.2, minimum)
  • GitHub Check: lint-js-and-ruby
  • GitHub Check: build-dummy-app-webpack-test-bundles
  • GitHub Check: claude-review
🔇 Additional comments (4)
.github/workflows/lint-js-and-ruby.yml (1)

92-95: LGTM! Well-integrated type checking steps.

The new RBS validation and Steep type checking steps are well-positioned in the workflow after Ruby linting. Using the rake tasks (rbs:validate and rbs:steep) provides consistency and proper error handling.

Steepfile (1)

1-24: LGTM! Solid foundation for type checking.

The Steepfile configuration provides a good starting point for static type analysis. The target definition correctly specifies the source (lib) and signature (sig) directories, and includes commonly-used standard library signatures.

rakelib/rbs.rake (1)

38-57: LGTM! Excellent implementation following established patterns.

The new rbs:steep task follows the same structure as the existing rbs:validate task with:

  • Clear status messages
  • Proper error handling for all cases (success, failure, command not found)
  • Appropriate exit codes

The rbs:all task provides a convenient way to run both checks in sequence.

react_on_rails_pro/sig/react_on_rails_pro.rbs (1)

1-5: LGTM! Method implementations verified.

Both configure and configuration methods exist in the ReactOnRailsPro module at react_on_rails_pro/lib/react_on_rails_pro/configuration.rb. The implementations match their RBS signatures: configure yields a Configuration block (line 4-7), and configuration provides lazy-loaded access to the Configuration instance (line 9+).

attr_accessor prerender_caching: bool?
attr_accessor renderer_http_pool_size: Integer?
attr_accessor renderer_http_pool_timeout: Integer?
attr_accessor renderer_http_pool_warn_timeout: Float?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Allow integers for warn timeout.

Many existing configs (including spec/dummy/config/initializers/react_on_rails_pro.rb) assign config.renderer_http_pool_warn_timeout = 5, which is an Integer. With runtime RBS enabled, this setter will now raise a TypeError because Integer is not a Float. Please widen the type to accept any numeric value. (github.com)

-    attr_accessor renderer_http_pool_warn_timeout: Float?
+    attr_accessor renderer_http_pool_warn_timeout: Numeric?
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
attr_accessor renderer_http_pool_warn_timeout: Float?
attr_accessor renderer_http_pool_warn_timeout: Numeric?
🤖 Prompt for AI Agents
In react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs around line 34,
the attr_accessor currently restricts renderer_http_pool_warn_timeout to Float?
which rejects Integer assignments at runtime; change the type to accept any
numeric value (e.g., Numeric?) while preserving nilability so existing integer
assignments like `= 5` no longer raise a TypeError.

attr_accessor rendering_returns_promises: bool?
attr_accessor remote_bundle_cache_adapter: Module?
attr_accessor ssr_pre_hook_js: String?
attr_accessor assets_to_copy: Array[String]?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Accept Pathname entries in assets_to_copy.

Our sample initializer passes Rails.root.join(...) values to config.assets_to_copy, which yield Pathname objects. The new signature forces String, so runtime checking will blow up immediately. Please permit both String and Pathname. (github.com)

-    attr_accessor assets_to_copy: Array[String]?
+    attr_accessor assets_to_copy: Array[String | Pathname]?
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
attr_accessor assets_to_copy: Array[String]?
attr_accessor assets_to_copy: Array[String | Pathname]?
🤖 Prompt for AI Agents
In react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs around line 40,
the typed signature for assets_to_copy currently only permits Array[String] but
the initializer may pass Pathname objects; update the signature to accept both
String and Pathname (e.g. an array element union) and keep the optionality, so
callers can provide Array[String | Pathname] or nil; adjust any related type
aliases/imports if needed to reference Pathname in the rbs file.

@claude
Copy link

claude bot commented Nov 9, 2025

Code Review: Add RBS validation and runtime type checking to CI

I've reviewed PR #1950 and have the following feedback:

✅ Strengths

  1. Comprehensive Type Checking Approach: The PR implements the three-layer type checking strategy from Evil Martians' "Climbing Steep Hills" article - validation, runtime checking, and static analysis. This is excellent practice.

  2. CI Integration: Adding RBS validation and Steep checks to the CI pipeline ensures type safety is maintained going forward.

  3. Runtime Validation: Enabling RBS runtime checking for the gem tests (RBS_TEST_TARGET='ReactOnRails::*') will catch actual type mismatches during test execution.

  4. Good Documentation: The PR description clearly explains the implementation approach and provides context.

🐛 Issues Found

Critical: RBS Type Signature Errors

1. Cache module signature is incorrect (react_on_rails_pro/sig/react_on_rails_pro/cache.rbs:3)

The signature shows:

def self.fetch: (String key) { () -> String } -> String

But the actual implementation in react_on_rails_pro/lib/react_on_rails_pro/cache.rb shows:

  • Cache is a class, not a module
  • The main public method is fetch_react_component(component_name, options), not fetch(key)
  • It yields a block and can return various types (Hash, String, etc.), not just String
  • There's also enabled? method that doesn't exist in the actual code

Recommendation: The RBS signature should match the actual API:

class ReactOnRailsPro::Cache
  class << self
    def fetch_react_component: (String component_name, Hash[Symbol, untyped] options) { () -> untyped } -> untyped
    def use_cache?: (Hash[Symbol, untyped] options) -> bool
    def base_cache_key: (String type, ?prerender: bool?) -> Array[untyped]
    def dependencies_cache_key: () -> String?
    def react_component_cache_key: (String component_name, Hash[Symbol, untyped] options) -> Array[untyped]
  end
end

2. Utils module has missing methods (react_on_rails_pro/sig/react_on_rails_pro/utils.rbs)

The signature only includes:

def self.bundle_js_file_path: (String bundle_name) -> String
def self.running_on_windows?: () -> bool

But the actual utils.rb has many more public methods including:

  • rorp_puts, rsc_bundle_js_file_path, react_client_manifest_file_path, digest_of_globs, bundle_hash, with_trace, common_form_data, pro_attribution_comment, etc.

Also, bundle_js_file_path doesn't exist in the Pro Utils - it delegates to ReactOnRails::Utils.bundle_js_file_path.

Recommendation: Add comprehensive signatures for all public methods in Utils module.

Minor Issues

3. Steep configuration may need more libraries (Steepfile:18-23)

The Steepfile includes basic libraries but may need additional ones based on the actual dependencies used in the codebase (e.g., digest, rack, rails).

4. Runtime type checking only applies to gem tests (rakelib/run_rspec.rake:24-28)

Runtime checking is only enabled for run_rspec:gem task, not for dummy app or example tests. This is reasonable for performance, but worth documenting.

📋 Suggestions

  1. Validate signatures before merging: Run bundle exec rbs -I sig validate locally in both the main repo and react_on_rails_pro/ to catch signature errors.

  2. Add to CHANGELOG: This appears to be an infrastructure/development improvement. Per CLAUDE.md guidelines, this probably doesn't need a CHANGELOG entry since it's not a user-facing change. However, if type safety becomes a documented feature, consider adding it.

  3. Consider gradual rollout: Since this is the first comprehensive RBS implementation, you might want to:

    • Start with just rbs:validate in CI
    • Add Steep after fixing any initial issues
    • Monitor CI performance impact
  4. Document type coverage: Consider adding a rake task to report RBS coverage (% of classes/modules with signatures).

🔒 Security Considerations

No security concerns identified. Type checking actually improves security by catching type-related bugs.

⚡ Performance Considerations

  • CI Impact: Adding two new steps (RBS validate + Steep check) will increase CI time. Monitor the impact.
  • Runtime checking: Only runs during tests, so no production impact.
  • The implementation correctly uses require: false for development-only gems.

✅ Test Coverage

The PR appropriately integrates type checking into existing test workflows. The runtime checking during gem tests will help validate signatures against actual usage.

Action Items

Before merging:

  1. ✅ Fix Cache RBS signatures to match actual implementation (class vs module, correct method signatures)
  2. ✅ Add missing Utils method signatures
  3. ✅ Run bundle exec rbs -I sig validate locally and fix any errors
  4. ✅ Run bundle exec rake rbs:steep locally and address any issues
  5. ✅ Verify all files have trailing newlines (per CLAUDE.md requirements)
  6. ✅ Run bundle exec rubocop to ensure linting passes

Summary

This is a valuable addition to the project's quality infrastructure. The main issues are incorrect RBS signatures that need to be fixed before the CI checks will pass. Once the signatures accurately reflect the actual code, this will provide excellent type safety guarantees.

Great work on adopting modern Ruby type checking practices! 🎉

@claude
Copy link

claude bot commented Nov 9, 2025

PR Review: Add RBS validation and runtime type checking to CI

Summary

This PR implements comprehensive RBS type checking infrastructure for both the main gem and Pro package. The implementation follows best practices from Evil Martians' "Climbing Steep Hills" article with a three-pronged approach: validation, runtime checking, and static analysis.


Strengths

1. Well-Structured CI Integration ✓

  • Clean separation of validation and static checking in workflows
  • Appropriately disabled Steep checks until signatures are complete (374 errors noted)
  • RBS validation runs in both main and Pro workflows

2. Runtime Type Checking ✓

  • Smart integration with RSpec via RBS_TEST_TARGET
  • Runtime checking catches type mismatches during test execution
  • Non-intrusive approach using environment variables

3. Good Documentation ✓

  • Clear TODO comment explaining why Steep is disabled
  • Well-commented Steepfile explaining file selection strategy
  • Comprehensive RBS signatures for Pro package configuration

4. Incremental Adoption ✓

  • Steepfile only checks files with corresponding signatures
  • Prevents noise from uncovered files
  • Foundation for expanding coverage over time

Issues and Recommendations

1. Potential Issue: Array Type Precision (Minor)

Location: react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs:12-13

Using Array[untyped] defeats the purpose of type checking. These appear to be string arrays based on the attribute signatures.

Recommendation: Change to Array[String]

2. Potential Bug: System Call Error Handling (Medium)

Location: rakelib/rbs.rake:13, rakelib/rbs.rake:43

Direct system calls without bundle exec might fail in environments where gems aren't in PATH. If someone runs bundle install --path vendor/bundle, gem binaries won't be in PATH.

Recommendation: Use bundle exec rbs and bundle exec steep in the system calls.

3. Test Coverage Gap (Medium)

No tests verify that RBS signatures are loaded correctly or that runtime type checking works. Consider adding tests in spec/react_on_rails/ to verify the rake tasks work correctly.

4. Pro Gem Gemfile.lock Missing (Minor)

Added rbs to Pro gem dependencies but Gemfile.lock not updated. Run bundle install in react_on_rails_pro/.


Security Concerns

No security issues identified. This PR improves security through better type validation.


Performance Considerations

  • CI Build Time: +10-30 seconds for RBS validation (minimal)
  • Test Runtime: Runtime type checking adds approximately 5-15% overhead to gem tests
  • Steep Disabled: No impact until re-enabled

Monitor CI build times after merge. When Steep is enabled, it can be slow on large codebases (1-5 minutes).


Code Quality

Strengths:

  • Follows repository conventions
  • Clear task descriptions
  • Proper error handling with exit codes
  • Good comments explaining TODO items

Minor improvements possible:

  • Similar error handling logic in :validate and :steep tasks could be extracted to reduce duplication

Final Recommendation

Status: ✅ Approve with Minor Changes

Required Changes:

  1. Fix Array[untyped] to Array[String] in configuration.rbs
  2. Use bundle exec for system calls in rake tasks
  3. Update Pro gem Gemfile.lock

Suggested Changes:

  1. Add basic RSpec tests for rake tasks
  2. Verify Steepfile library declarations are all necessary
  3. Consider refactoring duplicate error handling code

Overall Assessment:
This is a well-thought-out PR that establishes solid type checking infrastructure. The incremental approach is pragmatic. Once the minor issues are addressed, this will significantly improve code quality and catch bugs earlier.

Great work on following the Evil Martians article and setting up all three type checking approaches!


📝 Review completed by Claude Code

@claude
Copy link

claude bot commented Nov 9, 2025

Code Review: Add RBS validation and runtime type checking to CI

Summary

This PR adds comprehensive RBS type checking infrastructure. Overall well-structured implementation following Ruby type checking best practices, with a few issues to address.

Critical Issues

1. Inconsistency in rbs.rake - bundle exec removed incorrectly
Location: rakelib/rbs.rake:13,40

Problem: Removing bundle exec assumes rake runs in bundle context, but this only happens with 'bundle exec rake'. Direct 'rake' calls will fail if gems aren't in system path.

Recommendation: Keep 'bundle exec' in system calls for both rbs and steep commands.

2. Missing Error Handling for Runtime Type Checking
Location: rakelib/run_rspec.rake:24-26

Problems: Silent failures if RBS fails to load, no opt-out mechanism, environment pollution via RUBYOPT.

Recommendation: Add conditional check with ENV opt-out and RBS availability validation.

High Priority

3. Type Signature Accuracy - react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs:11-12
Using Array[untyped] defeats type checking purpose. Should be Array[String].

4. Cache.fetch Block Signature - react_on_rails_pro/sig/react_on_rails_pro/cache.rbs:3
Verify if block always required and always returns String.

Medium Priority

5. Steepfile Scope - Use 'check lib' with ignore patterns vs listing individual files
6. Pro Gem Validation - Use rake task for consistency with main gem

Positive Observations

  • Excellent PR description and implementation rationale
  • Follows Evil Martians best practices (validate + runtime + Steep)
  • Good CI integration and test coverage strategy
  • Consistent with main gem patterns

Recommendations Summary

Must fix: Issues 1 and 2
Should fix: Issues 3 and 4
Consider for follow-up: Issues 5-6 plus documentation improvements

Conclusion

Solid foundation for type safety. After addressing bundle exec and runtime checking validation, ready to merge. Great work!

Status: Requesting changes for issues 1 and 2

@claude
Copy link

claude bot commented Nov 9, 2025

Pull Request Review: Add RBS validation and runtime type checking to CI

Overall Assessment

This is a well-structured PR that implements comprehensive type checking for the React on Rails gem. The approach is methodical and follows Ruby best practices for gradual type adoption. Overall: ✅ Approved with minor suggestions


✅ Code Quality & Best Practices

Strengths

  1. Excellent incremental adoption strategy: The PR correctly implements RBS validation first, then adds runtime checking, following the "Climbing Steep Hills" article methodology.

  2. Proper separation of concerns: Main gem and Pro package are handled independently with their own RBS signatures and validation workflows.

  3. Good CI integration: RBS validation is added to CI while Steep is appropriately disabled until signatures are complete (374 errors remaining).

  4. Well-documented Steepfile: Clear comments explain why only specific files are checked.

  5. Comprehensive Pro package types: The RBS signatures for ReactOnRailsPro::Configuration are thorough with all 24+ configuration attributes properly typed.


⚠️ Potential Issues & Suggestions

1. Inconsistency in bundle exec usage (rakelib/rbs.rake)

Issue: The rake file comments say "use rbs directly, not bundle exec since we're already in bundle context" but the diff shows:

  • Line 13: Changed FROM bundle exec rbs -I sig validate TO rbs -I sig validate
  • Line 43: Uses steep check (no bundle exec)

Recommendation:

  • When running from rake tasks (already in bundle context), you should NOT use bundle exec to avoid nested bundler calls
  • The change to system("rbs -I sig validate") is correct
  • Similarly for steep: system("steep check") is correct
  • However, verify this works correctly in all environments (local dev, CI, etc.)

2. Missing RBS type signature for the new rake tasks

The new rake tasks in rakelib/rbs.rake should ideally have RBS signatures. While this is a minor issue for internal tooling, it would improve consistency.

3. Runtime type checking only for gem tests (rakelib/run_rspec.rake)

Good addition: Runtime checking is enabled for gem tests with RBS_TEST_TARGET='ReactOnRails::*' RUBYOPT='-rrbs/test/setup'

Concern: Runtime checking is NOT enabled for dummy app tests or example app tests.

Recommendation: Consider adding runtime checking to dummy/example tests in a follow-up PR once the type signatures are more complete.

4. Steepfile library dependencies may be incomplete

The Steepfile only declares 4 library dependencies (pathname, singleton, logger, monitor, securerandom) but the gem uses many more libraries (rails, action_view, active_support, connection_pool).

Recommendation: Add core Rails dependencies explicitly for clarity.

5. Pro gem RBS validation command differs

The Pro gem workflow uses bundle exec rbs -I sig validate while the main gem uses bundle exec rake rbs:validate.

Recommendation: For consistency, the Pro gem should also use a rake task or document why the direct command is preferred.


🔒 Security Concerns

No security issues identified ✅

The changes are purely related to static type checking infrastructure. The runtime type checking happens only in test mode, not in production.


⚡ Performance Considerations

1. Runtime type checking overhead in tests ✅

Runtime checking with rbs/test/setup adds overhead to test execution. However:

  • This is acceptable for test environments
  • The benefit of catching type mismatches at runtime outweighs the cost
  • Only enabled for gem tests (not all tests)

Status: Acceptable trade-off

2. CI pipeline duration ✅

Adding RBS validation to CI adds minimal overhead (~5-10 seconds typically). The PR correctly disables Steep (which would be slower) until signatures are complete.

Status: No concerns


🧪 Test Coverage

Gaps Identified

  1. No tests for the new rake tasks: The rbs.rake file adds new tasks (rbs:validate, rbs:steep, rbs:all) but there are no tests verifying these tasks work correctly.

  2. No verification that runtime checking is actually running: The env vars are set, but there's no test to confirm rbs/test/setup is actually loaded and checking types.

  3. Pro package RBS signatures are not tested: The Pro gem has new RBS files but no indication that they're validated or runtime-checked.


📝 Documentation & Maintainability

Strengths

  1. Excellent PR description: Clear summary, implementation details, and test plan
  2. TODO comments in CI workflow: Clearly documents why Steep is disabled
  3. Steepfile is well-commented: Explains the file selection strategy

Minor Suggestions

  1. Update CLAUDE.md: Consider adding RBS/Steep commands to the development commands section
  2. Add RBS to the MANDATORY BEFORE GIT PUSH section: Include RBS validation alongside RuboCop

🐛 Specific Code Issues

1. Gemfile.lock changes look correct ✅

All new dependencies are properly tracked with no version conflicts detected.

2. RBS signature quality (Pro package) ✅

The react_on_rails_pro/sig/ files are well-structured with proper module nesting and appropriate type usage.

Minor issue: DEFAULT_DEPENDENCY_GLOBS: Array[untyped] - could be more specific as Array[String]

3. GitHub Actions workflow syntax ✅

The YAML changes are syntactically correct and follow existing patterns.


🎯 Recommendations Summary

Must Fix (Blocking)

  • None - PR is ready to merge

Should Fix (High Priority)

  1. Verify the bundle exec removal in rbs.rake is correct
  2. Add RBS validation to CLAUDE.md mandatory checks
  3. Unify Pro gem validation to use rake task for consistency

Nice to Have (Low Priority)

  1. Add tests for new rake tasks
  2. Add runtime checking to dummy/example tests (follow-up PR)
  3. Expand Steepfile library declarations
  4. Add RBS commands to CLAUDE.md development section
  5. Improve type specificity for DEFAULT_DEPENDENCY_GLOBS

✅ Final Verdict

Approved - This is solid work that improves the codebase's type safety without introducing regressions. The incremental approach is exactly right for adopting RBS/Steep in an existing codebase.

The PR demonstrates:

  • Good understanding of Ruby type checking tools
  • Thoughtful CI integration
  • Appropriate scoping (not trying to fix all 374 Steep errors at once)
  • Clear documentation

Great job! 🎉


Review conducted by Claude Code following React on Rails conventions from CLAUDE.md

@claude
Copy link

claude bot commented Nov 9, 2025

Code Review: RBS Validation and Runtime Type Checking

I've reviewed this PR and overall it's a solid implementation of RBS type checking infrastructure. Here's my feedback:


✅ Strengths

  1. Comprehensive Approach: The PR implements all three layers of type checking (validation, runtime, static analysis) following Evil Martians' best practices - this is excellent.

  2. Pragmatic Phasing: Smart decision to temporarily disable Steep in CI while RBS signatures are being completed. This allows incremental adoption without blocking progress.

  3. Good Architecture:

    • Separate RBS signatures for Pro package
    • Proper CI integration in both main and Pro workflows
    • Clear rake task organization
  4. Pro Package Type Signatures: The RBS signatures for the Pro package Configuration class appear complete and match the implementation well.


🔍 Issues & Concerns

1. Critical: Inconsistency in rake tasks (rakelib/rbs.rake)

Issue: The file shows two different versions:

  • Lines 13, 42: Use bundle exec (before the PR changes)
  • The diff shows changes to remove bundle exec (lines 13, 42 in diff)

Current state in repository:

# Line 13 - BEFORE changes
result = system("bundle exec rbs -I sig validate")

# Line 13 - AFTER changes (in diff)
result = system("rbs -I sig validate")

Problem: The comment says "use rbs directly, not bundle exec since we're already in bundle context", but:

  • The rake tasks are typically called via bundle exec rake rbs:validate
  • Removing bundle exec from the inner system call is correct to avoid nested bundler
  • However, the CURRENT file still has bundle exec on line 13, meaning the changes weren't fully applied

Recommendation: Verify that the changes from commit 058d25e were properly applied. The file should use system("rbs -I sig validate") not system("bundle exec rbs -I sig validate").


2. Runtime Type Checking Scope (rakelib/run_rspec.rake:25-27)

Issue: Runtime type checking only runs for :gem task, not for :dummy or other example tests.

Current:

task :gem do
  run_tests_in("",
               rspec_args: File.join("spec", "react_on_rails"),
               env_vars: "RBS_TEST_TARGET='ReactOnRails::*' RUBYOPT='-rrbs/test/setup'")
end

Questions:

  • Is this intentional?
  • Would runtime checking benefit the dummy app tests as well?
  • Performance trade-off: Runtime checking adds overhead - is this acceptable for all test runs?

Recommendation: Document why runtime checking is only enabled for :gem tests, or consider enabling it for more comprehensive validation.


3. Steepfile - Incomplete Library Dependencies

The Steepfile only lists 5 library dependencies:

library "pathname"
library "singleton"
library "logger"
library "monitor"
library "securerandom"

Missing common Rails/Ruby dependencies that the codebase likely uses:

  • activesupport
  • actionview (for helpers)
  • railties (for engines)
  • execjs
  • rainbow
  • webpacker / shakapacker

Impact: Steep may not properly type-check code that uses these libraries, leading to false positives/negatives.

Recommendation:

  1. Run steep check --verbose to identify missing library references
  2. Add the required libraries to Steepfile
  3. Note: Some gems may not have RBS signatures yet - document which ones are missing

4. Type Signature Quality (Pro Package)

Concern: Some type signatures use untyped which defeats the purpose:

DEFAULT_DEPENDENCY_GLOBS: Array[untyped]
DEFAULT_EXCLUDED_DEPENDENCY_GLOBS: Array[untyped]

From implementation (configuration.rb:49-50):

DEFAULT_DEPENDENCY_GLOBS = [].freeze
DEFAULT_EXCLUDED_DEPENDENCY_GLOBS = [].freeze

Recommendation: These should be Array[String] since globs are strings:

DEFAULT_DEPENDENCY_GLOBS: Array[String]
DEFAULT_EXCLUDED_DEPENDENCY_GLOBS: Array[String]

5. Cache Module Type Signature Accuracy

def self.fetch: (String key) { () -> String } -> String

Question: Does the block always return String?

  • If caching React component HTML, yes
  • If caching other values, this might be too restrictive
  • Consider: (String key) { () -> String } -> String if it's correct, or use a type parameter if it can cache different types

Recommendation: Verify the actual usage and ensure the signature matches all use cases.


🎯 Recommendations

High Priority

  1. ✅ Verify rakelib/rbs.rake changes were properly applied (remove nested bundle exec)
  2. ✅ Fix untyped in Pro package signatures to use proper types
  3. ✅ Add missing library dependencies to Steepfile

Medium Priority

  1. Document the runtime checking strategy (why only :gem tests)
  2. Verify Cache.fetch type signature matches actual usage
  3. Consider adding a pre-commit hook to run rake rbs:validate locally

Nice to Have

  1. Add documentation about RBS adoption roadmap (which files are next)
  2. Consider adding # @type comments in Ruby code for complex type scenarios
  3. Add rake task to show RBS coverage percentage

🧪 Testing Recommendations

Before merging:

  • Run bundle exec rake rbs:validate locally and confirm it passes
  • Run bundle exec rake run_rspec:gem and verify runtime type checking works without errors
  • Run cd react_on_rails_pro && bundle exec rbs -I sig validate for Pro package
  • (Future) Run bundle exec rake rbs:steep when ready to see remaining type errors

📚 Documentation Suggestions

Consider adding to CLAUDE.md:

  • How to add RBS signatures for new classes/modules
  • How to run type checking locally
  • Strategy for incremental RBS adoption
  • Known limitations (e.g., Steep disabled until signatures complete)

Security & Performance

No security concerns - type signatures are compile/check-time only
Performance: Runtime checking only in tests (not production) - appropriate
⚠️ CI time: Monitor CI duration - RBS validation + Steep could add overhead


Summary

This is a well-structured PR that lays excellent groundwork for type safety. The main issues are:

  1. Verify rake task changes were applied correctly
  2. Improve type signature specificity (avoid untyped)
  3. Complete Steepfile library dependencies

The phased rollout approach (validation now, Steep later) is pragmatic and reduces risk.

Verdict: ✅ Approve with minor fixes recommended

Great work on adopting modern Ruby type checking! This will significantly improve code quality and IDE support.


🤖 Review by Claude Code

@claude
Copy link

claude bot commented Nov 9, 2025

Code Review - PR #1950: Add RBS validation and runtime type checking to CI

Summary

This PR adds comprehensive RBS type checking infrastructure for both the main gem and Pro package. Overall, this is a solid enhancement that follows Ruby type-checking best practices.


Strengths

1. Excellent Three-Pronged Approach

The implementation follows Evil Martians best practices with static validation, runtime checking, and static analysis with Steep.

2. Well-Structured RBS Signatures for Pro

The Pro gem type signatures are well-crafted with complete coverage of Configuration class and proper use of nilable types.

3. Smart CI Integration Strategy

Pragmatic approach: validation enabled, Steep disabled until signatures are complete (374 errors to fix first).

4. Clear Documentation

The PR description is excellent with clear explanations and test plan.


Issues & Concerns

1. CRITICAL: Inconsistency in run_rspec.rake (lines 24-26)

Issue: The PR diff shows RBS runtime checking added to the :gem task, but the actual file does NOT contain this change. The runtime type checking configuration appears to be missing from the code!

Impact: Tests won't validate RBS signatures at runtime, defeating one of the three pillars of the type-checking strategy.

Action Required: Verify the change is actually applied to the file.

2. Inconsistency: bundle exec (rakelib/rbs.rake:13 vs :steep task)

Issue: rbs:validate uses "bundle exec rbs" but rbs:steep uses "steep check" without bundle exec. This is inconsistent and potentially fragile.

Recommendation: Be consistent - standardize on one approach for reliability.

3. Missing Runtime Checking for Pro Gem

Main gem gets runtime checking via RBS_TEST_TARGET but Pro gem only gets static validation. Consider adding runtime type checking to Pro gem tests for consistency.

4. Missing Test Coverage Verification

Test plan shows CI validation items unchecked. Has this actually been tested in CI?


Performance Considerations

Runtime type checking with rbs/test/setup can slow down test execution 10-30%. Consider documenting this and making it optional via ENV variable.


Recommendations

High Priority (Must Fix):

  1. Resolve run_rspec.rake inconsistency - verify runtime checking code is applied
  2. Fix bundle exec inconsistency - standardize approach
  3. Verify CI actually works

Medium Priority:
4. Add runtime type checking to Pro gem tests
5. Document performance impact
6. Consider making RBS runtime tests optional via ENV

Low Priority:
7. Add rake task to generate RBS signatures
8. Document 374 Steep errors in tracking issue


Conclusion

Overall Assessment: Well-thought-out PR with sound strategy and well-crafted signatures.

Blocking Issues: run_rspec.rake and bundle exec inconsistencies need resolution.

Score: 7.5/10 (would be 9/10 if inconsistencies resolved)

Great work bringing type safety to React on Rails!

run_tests_in("", rspec_args: File.join("spec", "react_on_rails"))
run_tests_in("",
rspec_args: File.join("spec", "react_on_rails"),
env_vars: "RBS_TEST_TARGET='ReactOnRails::*' RUBYOPT='-rrbs/test/setup'")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might make sense to pass these options when starting the server in integration tests too, but not 100% sure.

Comment on lines +11 to +37
check "lib/react_on_rails.rb"
check "lib/react_on_rails/configuration.rb"
check "lib/react_on_rails/controller.rb"
check "lib/react_on_rails/git_utils.rb"
check "lib/react_on_rails/helper.rb"
check "lib/react_on_rails/packer_utils.rb"
check "lib/react_on_rails/server_rendering_pool.rb"
check "lib/react_on_rails/test_helper.rb"
check "lib/react_on_rails/utils.rb"
check "lib/react_on_rails/version_checker.rb"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not have to maintain this list, just iterate over all files in lib and check for each if it has a signature in sig.

@claude
Copy link

claude bot commented Nov 9, 2025

Code Review - PR #1950: Add RBS validation and runtime type checking to CI

Summary

This PR adds comprehensive RBS type checking infrastructure to both the main gem and Pro package. Overall, this is a well-structured and valuable addition that will improve type safety. I've identified several issues that should be addressed before merging.


Critical Issues

1. Inconsistency in rakelib/rbs.rake

Location: rakelib/rbs.rake:13 vs lines 37, 45

Issue: The diff shows a comment change to "use rbs directly, not bundle exec since we're already in bundle context" but the actual code at line 13 still uses bundle exec rbs. Meanwhile, the new steep task uses steep directly without bundle exec.

Impact: This inconsistency is confusing and suggests incomplete refactoring.

Recommendation: Either remove bundle exec from line 13 to match the comment and the steep task, OR update the comment to explain why bundle exec is still needed for RBS.


2. Incomplete Runtime Checking Coverage

Location: rakelib/run_rspec.rake:24-26

Issue: Runtime type checking (RBS_TEST_TARGET) is only enabled for the :gem task, not for :dummy or example tasks.

Impact: Incomplete test coverage. If dummy/example tests exercise gem code paths not covered by gem specs, type errors could slip through.

Recommendation: Consider enabling runtime checking for dummy and example tasks if they exercise ReactOnRails:: code per the Evil Martians article emphasis on comprehensive runtime checking.


High Priority Issues

3. Pro Gem Configuration Types Could Be More Specific

Location: react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs

Issue: Uses Array[untyped] for dependency globs and Module? for cache adapter.

Recommendation: Replace with more specific types where possible (e.g., Array[String] for file path globs) to maximize type safety benefits.


4. Limited Steepfile Scope

Location: Steepfile:10-19

Issue: Only 10 files are checked. The limited positive list makes it easy to forget adding new files.

Recommendation: Add comments documenting which directories/files are intentionally excluded and why. Consider creating a follow-up issue to expand coverage incrementally.


Medium Priority Issues

5. Steep Disabled in CI

Location: .github/workflows/lint-js-and-ruby.yml:94-98

Issue: Steep is commented out with TODO about 374 type errors.

Recommendation: File a follow-up issue with concrete plan to enable Steep in CI. Consider running Steep only on the 10 files listed in Steepfile if those are clean.


6. Pro Gem Missing Steep Configuration

Location: react_on_rails_pro/ directory

Issue: Pro gem gets RBS validation but lacks Steepfile, steep rake tasks, and runtime checking in tests.

Recommendation: Add Steep configuration to Pro gem for consistency.


Positive Aspects

  1. Excellent PR description - Clear, references best practices (Evil Martians article)
  2. Proper RBS structure - Follows conventions with sig/ directories
  3. Comprehensive signatures - Detailed Configuration types covering all attributes
  4. Runtime checking integration - Correct use of rbs/test/setup
  5. Conservative rollout - Pragmatic to disable Steep initially while fixing errors
  6. Consistent formatting - Clean code style

Performance Considerations

Runtime checking adds overhead during tests - monitor test suite execution time. Consider using RBS_TEST_SAMPLE_SIZE if slowdown is significant.

Steep can be slow on large codebases - consider running only on changed files in PRs once enabled, or caching analysis results.


Test Coverage Analysis

✅ RBS validation runs in CI (syntax checking)
✅ Runtime checking enabled for gem specs
❌ Steep disabled in CI (static analysis not enforced)
❌ No runtime checking for dummy/example specs
❌ Pro gem has no runtime checking


Recommendations Summary

Must Fix Before Merge

  1. Fix inconsistency in rakelib/rbs.rake:13 (comment vs code)
  2. Decide on runtime checking for dummy/example tasks

Should Fix Before Merge

  1. Verify Pro gem Configuration constants match implementation
  2. Add documentation about Steepfile file selection strategy
  3. Create follow-up issue for enabling Steep in CI with concrete plan

Nice to Have

  1. Add Steep configuration to Pro gem
  2. Add documentation header to rbs.rake explaining the workflow
  3. Monitor and document test suite performance impact

Conclusion

This is a solid foundation for type safety. The approach follows best practices and implementation is clean. However, the limited scope (Steep disabled, only 10 files checked, no Pro gem Steep config) means this is more scaffolding than complete type safety.

Recommendation: Approve with changes for critical issues. Ensure clear roadmap for:

  1. Enabling Steep in CI
  2. Expanding coverage beyond 10 files
  3. Fixing the 374 type errors mentioned in the TODO

The infrastructure is good - now it needs to be fully utilized. Great work following the Evil Martians guide and setting up comprehensive type checking infrastructure!

@claude
Copy link

claude bot commented Nov 10, 2025

Code Review - PR #1950: Add RBS validation and runtime type checking to CI

I've reviewed this PR implementing comprehensive RBS type checking. Overall, this is a well-structured addition that follows Ruby typing best practices. Here are my findings:

✅ Strengths

  1. Comprehensive Approach: The PR correctly implements the three-pillar approach from Evil Martians:

    • Static validation with rbs validate
    • Runtime checking with rbs/test/setup
    • Static analysis with Steep
  2. Good CI Integration: RBS validation is properly added to both main and Pro gem workflows

  3. Thoughtful Steep Configuration: The Steepfile correctly limits type checking to files with RBS signatures, preventing false positives

  4. Well-Structured Pro Type Signatures: The RBS definitions for ReactOnRailsPro are comprehensive and match the actual implementation

🔍 Issues & Concerns

Critical Issues

  1. Inconsistent bundle exec usage in rake tasks (rakelib/rbs.rake:13, 43)

    The PR diff shows:

    • Line 13: Changed from bundle exec rbs to rbs with comment "use rbs directly, not bundle exec"
    • Line 43: Uses steep check without bundle exec

    Problem: The comment says "we're already in bundle context" but this is incorrect. Rake tasks run via bundle exec rake but spawned shell commands via system() do not inherit the bundle context. They need explicit bundle exec.

    Risk: This will cause CI failures if rbs/steep gems aren't in the system PATH, which they typically aren't.

    Fix: Revert to bundle exec rbs and bundle exec steep

  2. Runtime type checking modification issue (rakelib/run_rspec.rake:24-26)

    The diff shows runtime checking added to run_rspec:gem task, but when I read the current file, it shows:

    task :gem do
      run_tests_in("", rspec_args: File.join("spec", "react_on_rails"))
    end

    The env_vars parameter from the diff is missing in the actual file. This means:

    • Runtime type checking is not actually enabled for gem tests
    • The PR description claims runtime checking is working, but it's not

    Fix: Verify the actual file content matches the PR diff

Moderate Issues

  1. Steep disabled in CI without clear plan

    The TODO comment says "374 type errors need to be fixed first" but provides no:

    • Issue tracking these errors
    • Timeline for fixing them
    • Plan for incremental adoption

    Recommendation: Create a tracking issue and consider enabling Steep for a subset of files first

  2. Pro gem RBS type accuracy concerns (react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs)

    Several type definitions may be too loose:

    • Line 11: DEFAULT_DEPENDENCY_GLOBS: Array[untyped] should be Array[String]
    • Line 12: DEFAULT_EXCLUDED_DEPENDENCY_GLOBS: Array[untyped] should be Array[String]
    • Line 22: remote_bundle_cache_adapter typed as Module? - should this be a more specific interface/duck type?

    Impact: Runtime type checking won't catch type errors in these arrays

  3. Missing Steepfile in repository

    The PR adds a Steepfile but it doesn't exist in the current working directory. This suggests:

    • Either the PR hasn't been merged/applied
    • Or there's a mismatch between the PR diff and actual changes

    Recommendation: Verify file was committed correctly

Minor Issues

  1. Incomplete test coverage verification

    The test plan shows:

    • ✅ Local validations passing
    • ❌ CI validation not confirmed
    • ❌ CI Steep checks not confirmed

    Recommendation: Wait for CI to pass before merging

  2. No changelog entry

    According to CLAUDE.md, user-visible changes should have changelog entries. While RBS is primarily a dev tool, this could be considered a quality/reliability improvement worth documenting.

🎯 Recommendations

  1. Fix Critical Issue TODO for first version #1: Restore bundle exec prefix to all gem commands in rake tasks
  2. Fix Critical Issue Make work with turbolinks and better helper #2: Investigate why run_rspec.rake doesn't match the PR diff
  3. Create tracking issue: For the 374 Steep errors with a plan for incremental fixes
  4. Improve type precision: Change Array[untyped] to Array[String] in Pro RBS signatures
  5. Wait for CI: Ensure all checks pass before merging
  6. Consider changelog: Add entry under "Internal" or "Development" section if appropriate

📊 Overall Assessment

This is a valuable addition that will improve code quality and catch type-related bugs early. However, the critical issues with bundle exec and potentially missing changes need to be resolved before merging to avoid CI failures.

Recommendation: Request changes to address critical issues #1 and #2 before approval.


🤖 Generated with Claude Code

@claude
Copy link

claude bot commented Nov 10, 2025

Code Review: RBS Validation and Runtime Type Checking

Thank you for this comprehensive PR implementing RBS type checking! This is a well-structured addition that follows best practices from the Evil Martians guide. Here's my feedback:

✅ Strengths

  1. Excellent Documentation: The Steepfile has clear comments explaining the incremental adoption strategy with explicit lists of excluded files and their reasons.

  2. Thoughtful Error Handling: The rake tasks properly handle command failures with clear error messages and re-run commands to show actual errors (lines 22-24, 57-59 in rbs.rake).

  3. Opt-in Runtime Checking: The ENABLE_RBS_RUNTIME_CHECKING environment variable approach is wise - prevents breaking existing workflows while allowing gradual adoption.

  4. Comprehensive CI Integration: Both main and Pro gem workflows now validate RBS signatures, ensuring type safety is enforced.

  5. Good Separation of Concerns: Runtime checking only applies to gem/dummy tests, not example apps - this is the right architectural choice.

🔍 Issues & Suggestions

Critical Issues

  1. Duplicate Yarn Install in CI (.github/workflows/lint-js-and-ruby.yml:98)

    • Lines 87 and 98 both run cd spec/dummy && yarn install
    • This is wasteful and increases CI time
    • Fix: Remove the duplicate at line 98
  2. Inconsistent RBS Command Between Main and Pro

    • Main gem uses: bundle exec rake rbs:validate (line 93)
    • Pro gem uses: bundle exec rbs -I sig validate (line 127)
    • Fix: Pro should use the rake task for consistency: bundle exec rake rbs:validate (assuming you add the task to Pro's Rakefile)
  3. Missing Runtime Checking in CI

    • The PR description mentions runtime checking as a key feature
    • However, ENABLE_RBS_RUNTIME_CHECKING is never set in any CI workflow
    • Impact: Runtime type checking won't actually run in CI, defeating part of the purpose
    • Fix: Add ENABLE_RBS_RUNTIME_CHECKING=true to the test workflow environment

Minor Issues

  1. Overly Broad Type Signatures (react_on_rails_pro/sig/react_on_rails_pro/cache.rbs)

    • Line 3: { () -> untyped } -> untyped - both block and return are untyped
    • Line 11: -> Array[untyped] - array elements are untyped
    • Suggestion: These could be more specific. For example:
      • If the block yields component HTML, specify that
      • If the array contains cache key parts, specify Array[String | Symbol]
    • Using untyped reduces the value of type checking
  2. Stderr Redirection May Hide Real Errors (rbs.rake:16, 51)

    • 2>/dev/null suppresses all stderr output
    • If RBS/Steep have warnings (not errors), they'll be hidden
    • Suggestion: Consider using 2>&1 | grep -v "bundler warning" to filter only bundler warnings
  3. Environment Variable Concatenation Risk (run_rspec.rake:64)

    • [rbs_runtime_env_vars, "DISABLE_TURBOLINKS=TRUE"].reject(&:empty?).join(" ")
    • This works but could fail if env vars contain spaces
    • Suggestion: Build env hash instead:
      env_hash = {}
      env_hash.merge\!(parse_env_string(rbs_runtime_env_vars)) if rbs_runtime_env_vars
      env_hash["DISABLE_TURBOLINKS"] = "TRUE"
  4. Missing RBS Gem in Pro Development Dependencies

    • Added gem "rbs" to Pro's Gemfile.development_dependencies ✓
    • But didn't add gem "steep" - inconsistent with main gem
    • Question: Is this intentional? If Pro will use Steep later, add it now

🔒 Security Considerations

  • ✅ No security concerns identified
  • Type signatures don't expose sensitive data
  • No command injection vulnerabilities (proper use of system())

⚡ Performance Considerations

  • RBS Validation: Fast, adds ~1-3 seconds to CI
  • Steep Type Checking: Currently disabled (374 errors), but will add ~10-30 seconds when enabled
  • Runtime Checking: Adds ~5-10% test execution overhead (acceptable for CI)
  • Recommendation: Monitor CI times after enabling runtime checking

🧪 Test Coverage

  • ✅ RBS validation runs in CI
  • ❌ Runtime checking not enabled in CI (see Critical Issue Add linting and CI scripts #3)
  • ❌ No tests for the rake tasks themselves
  • Suggestion: Add a simple smoke test:
    # spec/rake/rbs_spec.rb
    describe 'rake rbs:validate' do
      it 'validates signatures successfully' do
        expect(system('bundle exec rake rbs:validate')).to be true
      end
    end

📋 Code Quality

  • ✅ Follows CLAUDE.md conventions
  • ✅ RuboCop compliant (Steepfile excluded properly)
  • ✅ Excellent inline documentation
  • ✅ Proper error handling
  • ⚠️ Missing trailing newline check (per CLAUDE.md requirements) - verify all .rbs files

🎯 Overall Assessment

This is a high-quality PR that takes a pragmatic approach to gradual type adoption. The implementation is solid, documentation is excellent, and the architecture is sound.

Recommendation: Request changes to address the 3 critical issues, especially enabling runtime checking in CI. The minor issues can be addressed in follow-up PRs.

📝 Action Items

Must fix before merge:

  1. Remove duplicate yarn install
  2. Enable runtime checking in CI
  3. Standardize RBS validation command

Nice to have:
4. Tighten type signatures in Pro Cache class
5. Add Steep to Pro gem dependencies
6. Add rake task tests


Great work on this comprehensive type safety implementation! 🎉

@claude
Copy link

claude bot commented Nov 10, 2025

Code Review - PR #1950: Add RBS validation and runtime type checking to CI

Overview

This PR adds comprehensive RBS type checking infrastructure following Evil Martians best practices. The implementation is well-structured and thoughtful, with excellent documentation. Here's my detailed feedback:


✅ Strengths

1. Excellent Documentation

  • The Steepfile has outstanding inline documentation explaining the positive-list approach
  • Clear rationale for excluded files/directories
  • Step-by-step guide for adding new files to type checking
  • run_rspec.rake has comprehensive comments explaining RBS runtime checking strategy

2. Smart Incremental Approach

  • Using a positive list in Steepfile (explicit file-by-file checking) rather than checking everything at once
  • Only 10 files initially checked, allowing gradual expansion
  • Steep CI check temporarily disabled with clear TODO comment explaining why
  • This prevents the "all or nothing" problem and allows iterative improvement

3. Comprehensive Type Safety

  • Three-layer approach: validation + runtime checking + static analysis (Steep)
  • Runtime checking is opt-in via ENV["ENABLE_RBS_RUNTIME_CHECKING"]
  • Well-reasoned coverage strategy: gem tests + dummy tests, but not example apps

4. Pro Package Integration

  • Separate RBS setup for Pro gem following monorepo structure
  • Good foundation with 5 RBS files covering core modules
  • Pro gem workflow properly configured with RBS validation

🔧 Issues & Recommendations

1. Critical: Inconsistent bundle exec Usage in rbs.rake

Issue: Lines 13 and 51 in rakelib/rbs.rake show inconsistency:

# Line 13 - OLD version (incorrect based on diff)
result = system("bundle exec rbs -I sig validate")

# Line 51 - Should be (based on diff)
result = system("bundle exec steep check 2>/dev/null")

The diff shows these were updated to redirect stderr with 2>/dev/null to suppress bundler warnings, but the current file may not match. However, the comment explains the reasoning perfectly:

IMPORTANT: Always use 'bundle exec' even though rake runs in bundle context
Reason: Direct 'rake' calls (without 'bundle exec rake') won't have gems in path

Recommendation: Verify the final version matches the diff. The stderr redirection pattern is good:

  • First run with 2>/dev/null to suppress warnings
  • On failure, re-run without redirection to show actual errors

2. Potential Bug: Error Handling in rbs_runtime_env_vars

Location: rakelib/run_rspec.rake:38-43

def rbs_runtime_env_vars
  return "" unless ENV["ENABLE_RBS_RUNTIME_CHECKING"]

  begin
    require "rbs"
    "RBS_TEST_TARGET='ReactOnRails::*' RUBYOPT='-rrbs/test/setup'"
  rescue LoadError
    warn "Warning: RBS gem not available, skipping runtime type checking"
    ""
  end
end

Issue: The function silently returns empty string on LoadError. This means if someone explicitly enables runtime checking (ENABLE_RBS_RUNTIME_CHECKING=true) but RBS isn't available, tests will run without type checking and without failing.

Recommendation:

def rbs_runtime_env_vars
  return "" unless ENV["ENABLE_RBS_RUNTIME_CHECKING"]

  begin
    require "rbs"
    "RBS_TEST_TARGET='ReactOnRails::*' RUBYOPT='-rrbs/test/setup'"
  rescue LoadError
    # If user explicitly requested it, fail loudly
    raise "RBS gem not available. Install with 'bundle install' or disable ENABLE_RBS_RUNTIME_CHECKING"
  end
end

This ensures explicit user intent is respected and failures aren't silent.


3. Type Accuracy Issue in Pro Configuration

Location: react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs

The PR description mentions fixing Array[untyped] -> Array[String], which is good. However, I notice several attributes might need more specific types:

attr_accessor assets_to_copy: Array[String]?  # Good
attr_accessor dependency_globs: Array[String]?  # Good
attr_accessor excluded_dependency_globs: Array[String]?  # Good

Question: Should remote_bundle_cache_adapter be Module? or something more specific like a cache adapter interface? Consider if this could be tightened.


4. Cache.rbs Return Type Precision

Location: react_on_rails_pro/sig/react_on_rails_pro/cache.rbs:3

def self.fetch_react_component: (String component_name, Hash[Symbol, untyped] options) { () -> untyped } -> untyped

Issue: Triple untyped - the block params, block return, and method return are all untyped. While this may accurately reflect the current implementation, consider if any can be tightened:

  • Block return: Likely String (HTML)?
  • Method return: Same as block return?
  • Options hash: Could this use a more specific interface?

Recommendation: Add TODO comments if these should be refined later, or document why they must remain untyped.


5. Missing Test Coverage Information

What's Missing:

  • No visible changes to test files to verify RBS signatures are correct
  • No examples of intentionally wrong signatures to verify runtime checking works
  • No documentation of what happens when runtime checking catches an error

Recommendation: Add a test that:

  1. Intentionally violates a type signature
  2. Verifies runtime checking catches it
  3. Documents the error message format

This validates the entire system works as expected.


6. Steepfile Library Dependencies

Location: Steepfile:46-50

library "pathname"
library "singleton"
library "logger"
library "monitor"
library "securerandom"

Question: How were these specific libraries chosen? Consider:

  • Are these comprehensive for the 10 checked files?
  • Should this list be documented in comments?
  • Will new files require updating this list?

Recommendation: Add a comment explaining this list corresponds to stdlib dependencies of the checked files.


7. CI Workflow: Commented-Out Steep Check

Location: .github/workflows/lint-js-and-ruby.yml:94-97

# TODO: Re-enable Steep once RBS signatures are complete for all checked files
# Currently disabled because 374 type errors need to be fixed first
# - name: Run Steep type checker
#   run: bundle exec rake rbs:steep

Concern: The PR description says this adds Steep to CI, but it's commented out. This means:

  • The steep gem is added as a dependency but not used
  • No CI enforcement of type checking beyond validation
  • Easy to forget to re-enable

Recommendations:

  1. Option A (Preferred): Enable Steep CI check now, but only for the 10 files currently in Steepfile. Since those are explicitly listed, they should pass.
  2. Option B: Create a GitHub issue to track re-enabling, link it in the TODO comment
  3. Option C: Add a separate CI job that runs Steep but doesn't fail the build (informational only)

The current approach leaves value on the table since the 10 checked files should already pass Steep.


🔒 Security Considerations

No Major Concerns

  • Type checking is a development/CI-only feature
  • No runtime performance impact (unless explicitly enabled)
  • No user-facing changes

Minor: Environment Variable Behavior

The ENABLE_RBS_RUNTIME_CHECKING environment variable is development-only, but ensure it's documented that:

  • It should NEVER be enabled in production (performance overhead)
  • It's purely for test/development environments

🎯 Performance Considerations

1. Runtime Checking Overhead

Good: Runtime checking is opt-in and only for tests. No production impact.

Question: Has anyone measured the performance impact on test suite duration with runtime checking enabled? Even a 10-20% slowdown might be acceptable for the value, but worth documenting.

2. CI Time Impact

Minor concern: Adding rake rbs:validate to CI adds a step. Based on the diff, this should be fast (~seconds), but monitor CI duration.


🧪 Test Coverage Assessment

What's Tested

  • ✅ RBS validation runs in CI
  • ✅ Pro gem RBS validation configured
  • ✅ Runtime checking available (opt-in)

What's Not Tested

  • ❌ No test verifying runtime checking actually catches violations
  • ❌ No verification that type signatures match actual method behavior
  • ❌ Steep disabled in CI (no static analysis enforcement)

Recommendation: Add at least one integration test demonstrating the runtime checking works.


📋 Checklist Review

From the PR description:

  • RBS validation passes locally ✅
  • RuboCop passes ✅
  • All files properly formatted ✅
  • CI validates RBS signatures ⚠️ (Will pass, but Steep is disabled)
  • CI runs Steep checks ❌ (Commented out)

🎓 Best Practices Alignment

Following Best Practices ✅

  1. Incremental adoption - Perfect positive-list approach
  2. Three-layer type safety - Validation + runtime + static (though Steep is disabled)
  3. Comprehensive documentation - Excellent comments throughout
  4. Opt-in runtime checking - Smart default-off behavior

Could Be Improved

  1. Error handling - Silent failures in rbs_runtime_env_vars
  2. CI enforcement - Steep disabled means only getting validation, not full static checking
  3. Test coverage - No tests proving the system works

🚀 Recommendations Priority

High Priority (Should Address Before Merge)

  1. Fix silent failure in rbs_runtime_env_vars - Change warn to raise when explicitly enabled
  2. Enable Steep CI check - Since only 10 files are checked, they should pass
  3. Verify stderr redirection - Confirm rbs.rake matches the diff

Medium Priority (Consider for This PR)

  1. 🤔 Add runtime checking test - Validate the system catches violations
  2. 🤔 Document performance impact - Add note about test suite overhead when enabled
  3. 🤔 Tighten cache.rbs types - Reduce untyped where possible

Low Priority (Can Defer to Later PRs)

  1. 📝 Document Steepfile library list - Explain how it was chosen
  2. 📝 Create tracking issue - For expanding type coverage to more files
  3. 📝 Production safety note - Document that runtime checking must never be enabled in prod

🎉 Summary

This is excellent foundational work for bringing type safety to React on Rails. The incremental approach, comprehensive documentation, and three-layer strategy are all exemplary.

Main Concerns:

  1. Silent failure when RBS isn't available but explicitly requested
  2. Steep CI check disabled despite having checkable files
  3. No test coverage proving runtime checking works

Recommendation: Address the high-priority items, particularly the error handling and enabling Steep for the files already in Steepfile. The rest can be iterative improvements.

Great work on this PR! 🎉


Review Style Note: This review follows the React on Rails CLAUDE.md guidelines, focusing on:\n- Code quality and best practices ✅\n- Potential bugs (silent failures) ✅ \n- Performance considerations ✅\n- Security concerns ✅\n- Test coverage ✅

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
Steepfile (1)

28-37: Consider the trade-offs of automating the file list.

A past review suggested automating this list by iterating over lib/ files and checking for corresponding signatures in sig/. While this would reduce maintenance burden, the explicit list approach has benefits for incremental adoption:

Pros of automation:

  • No need to manually update the list when adding new typed files
  • Reduces chance of forgetting to add files to Steepfile

Pros of explicit list (current approach):

  • Clear visibility of what's actually being checked
  • Prevents accidentally type-checking files without proper signatures
  • Aligns with the documented "positive list" strategy for gradual rollout

Given the PR's incremental adoption goals, the current approach is reasonable. However, consider automation once type coverage is more mature.

Example automation approach for future consideration:

# Automatically check all files with RBS signatures
Dir.glob("lib/react_on_rails/*.rb").each do |lib_file|
  sig_file = lib_file.sub(%r{^lib/}, "sig/").sub(/\.rb$/, ".rbs")
  check lib_file if File.exist?(sig_file)
end
🧹 Nitpick comments (2)
rakelib/run_rspec.rake (2)

37-47: Consider returning a Hash instead of a string for better composability.

The method returns a space-separated string of environment variables, which requires careful handling downstream (e.g., line 64 uses reject(&:empty?).join(" ")). A Hash would be more robust and composable.

Apply this diff to improve type safety and composability:

-  def rbs_runtime_env_vars
-    return "" unless ENV["ENABLE_RBS_RUNTIME_CHECKING"]
+  def rbs_runtime_env_hash
+    return {} unless ENV["ENABLE_RBS_RUNTIME_CHECKING"]

     begin
       require "rbs"
-      "RBS_TEST_TARGET='ReactOnRails::*' RUBYOPT='-rrbs/test/setup'"
+      {
+        "RBS_TEST_TARGET" => "ReactOnRails::*",
+        "RUBYOPT" => "-rrbs/test/setup"
+      }
     rescue LoadError
       warn "Warning: RBS gem not available, skipping runtime type checking"
-      ""
+      {}
     end
   end

Then update the callers to use the Hash (example for line 53):

run_tests_in("",
             rspec_args: File.join("spec", "react_on_rails"),
             env_hash: rbs_runtime_env_hash)

And update run_tests_in to accept and merge env_hash into the environment variables construction at lines 139-144.


63-68: String concatenation is fragile; consider structured approach.

The array-reject-join pattern works but mixes string manipulation with environment variable composition. This would be cleaner with a Hash-based approach (as suggested for rbs_runtime_env_vars).

With the Hash approach, this would become:

env_hash = rbs_runtime_env_hash.merge("DISABLE_TURBOLINKS" => "TRUE")
run_tests_in(spec_dummy_dir,
             env_hash: env_hash,
             command_name: "dummy_no_turbolinks")
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 058d25e and 9ea26db.

📒 Files selected for processing (6)
  • .rubocop.yml (1 hunks)
  • Steepfile (1 hunks)
  • rakelib/rbs.rake (2 hunks)
  • rakelib/run_rspec.rake (1 hunks)
  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs (1 hunks)
  • react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • .rubocop.yml
🚧 Files skipped from review as they are similar to previous changes (1)
  • react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs
🧰 Additional context used
🧠 Learnings (14)
📓 Common learnings
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: lib/react_on_rails/helper.rb:190-197
Timestamp: 2025-02-18T13:08:01.477Z
Learning: RSC support validation in React on Rails Pro is handled through a chain of validations:
1. Pro version check in `run_stream_inside_fiber`
2. RSC support check during pack generation via `ReactOnRailsPro.configuration.enable_rsc_support`
3. RSC support validation during component registration
This makes additional validation in the helper methods unnecessary.
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1875
File: lib/react_on_rails/utils.rb:112-124
Timestamp: 2025-10-23T17:22:01.074Z
Learning: In React on Rails, when Pro is installed but not licensed, the intended behavior is to raise an error on boot. The `react_on_rails_pro?` method validates licenses and should raise errors early (including during path resolution in methods like `server_bundle?`) to enforce licensing requirements rather than failing later with obscure errors.
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1745
File: node_package/src/RSCRequestTracker.ts:8-14
Timestamp: 2025-07-08T05:57:29.630Z
Learning: The global `generateRSCPayload` function in React on Rails Pro (RORP) is provided by the framework during rendering requests, not implemented in application code. The `declare global` statements are used to document the expected interface that RORP will inject at runtime.
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: lib/react_on_rails/helper.rb:190-197
Timestamp: 2025-02-18T13:08:01.477Z
Learning: RSC support validation is handled in deeper level calls of the React on Rails Pro codebase, so it doesn't need to be validated again in the `rsc_payload_react_component` helper method.
Learnt from: alexeyr-ci
Repo: shakacode/react_on_rails PR: 1644
File: node_package/src/ReactOnRailsRSC.ts:87-87
Timestamp: 2024-12-12T13:07:09.929Z
Learning: When handling errors in 'node_package/src/ReactOnRailsRSC.ts', include the error stack in error messages in development and test environments to aid debugging.
Learnt from: justin808
Repo: shakacode/react_on_rails PR: 1770
File: lib/generators/react_on_rails/templates/base/base/app/javascript/src/HelloWorld/ror_components/HelloWorld.client.jsx:2-2
Timestamp: 2025-09-16T08:01:11.146Z
Learning: React on Rails uses webpack CSS Modules configuration with namedExports: true, which requires the import syntax `import * as style from './file.module.css'` rather than the default export pattern. This configuration enables better tree shaking and bundle size optimization for CSS modules.
📚 Learning: 2025-09-29T14:45:42.687Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1833
File: lib/react_on_rails/dev/process_manager.rb:72-83
Timestamp: 2025-09-29T14:45:42.687Z
Learning: In Ruby bundler contexts, when bundler intercepts system commands for executables not in the Gemfile, both version checks (like `system("foreman", "--version")`) and execution commands (like `system("foreman", "start", ...)`) fail equally, both returning false. This means availability checks using version flags accurately reflect whether execution commands will work in the current bundler context.

Applied to files:

  • rakelib/rbs.rake
📚 Learning: 2025-02-18T13:08:01.477Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: lib/react_on_rails/helper.rb:190-197
Timestamp: 2025-02-18T13:08:01.477Z
Learning: RSC support validation in React on Rails Pro is handled through a chain of validations:
1. Pro version check in `run_stream_inside_fiber`
2. RSC support check during pack generation via `ReactOnRailsPro.configuration.enable_rsc_support`
3. RSC support validation during component registration
This makes additional validation in the helper methods unnecessary.

Applied to files:

  • rakelib/rbs.rake
  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
  • rakelib/run_rspec.rake
📚 Learning: 2025-07-08T05:57:29.630Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1745
File: node_package/src/RSCRequestTracker.ts:8-14
Timestamp: 2025-07-08T05:57:29.630Z
Learning: The global `generateRSCPayload` function in React on Rails Pro (RORP) is provided by the framework during rendering requests, not implemented in application code. The `declare global` statements are used to document the expected interface that RORP will inject at runtime.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
📚 Learning: 2025-09-16T08:01:11.146Z
Learnt from: justin808
Repo: shakacode/react_on_rails PR: 1770
File: lib/generators/react_on_rails/templates/base/base/app/javascript/src/HelloWorld/ror_components/HelloWorld.client.jsx:2-2
Timestamp: 2025-09-16T08:01:11.146Z
Learning: React on Rails uses webpack CSS Modules configuration with namedExports: true, which requires the import syntax `import * as style from './file.module.css'` rather than the default export pattern. This configuration enables better tree shaking and bundle size optimization for CSS modules.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
📚 Learning: 2025-09-15T21:24:48.207Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1781
File: node_package/src/ClientSideRenderer.ts:82-95
Timestamp: 2025-09-15T21:24:48.207Z
Learning: In React on Rails, the force_load feature includes both explicit `data-force-load="true"` usage and the ability to hydrate components during the page loading state (`document.readyState === 'loading'`). Both capabilities require a Pro license, so the condition `!railsContext.rorPro && (isComponentForceLoaded || document.readyState === 'loading')` correctly gates both scenarios.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
📚 Learning: 2025-02-13T16:50:47.848Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: node_package/src/clientStartup.ts:18-21
Timestamp: 2025-02-13T16:50:47.848Z
Learning: In the react_on_rails module, the `reactOnRailsPageUnloaded` function in clientStartup.ts is intentionally kept private as it's only used internally as a callback for `onPageUnloaded`.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
📚 Learning: 2025-02-12T16:38:06.537Z
Learnt from: Romex91
Repo: shakacode/react_on_rails PR: 1697
File: package-scripts.yml:28-28
Timestamp: 2025-02-12T16:38:06.537Z
Learning: The file `node_package/lib/ReactOnRails.full.js` is autogenerated during the build process and should not be present in the repository.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
📚 Learning: 2025-02-18T13:08:01.477Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: lib/react_on_rails/helper.rb:190-197
Timestamp: 2025-02-18T13:08:01.477Z
Learning: RSC support validation is handled in deeper level calls of the React on Rails Pro codebase, so it doesn't need to be validated again in the `rsc_payload_react_component` helper method.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
📚 Learning: 2025-10-23T17:22:01.074Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1875
File: lib/react_on_rails/utils.rb:112-124
Timestamp: 2025-10-23T17:22:01.074Z
Learning: In React on Rails, when Pro is installed but not licensed, the intended behavior is to raise an error on boot. The `react_on_rails_pro?` method validates licenses and should raise errors early (including during path resolution in methods like `server_bundle?`) to enforce licensing requirements rather than failing later with obscure errors.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
📚 Learning: 2025-04-26T21:55:55.874Z
Learnt from: alexeyr-ci2
Repo: shakacode/react_on_rails PR: 1732
File: spec/dummy/client/app-react16/startup/ReduxSharedStoreApp.client.jsx:40-44
Timestamp: 2025-04-26T21:55:55.874Z
Learning: In the react_on_rails project, files under `app-react16` directories are copied/moved to corresponding `/app` directories during the conversion process (removing the `-react16` suffix), which affects their relative import paths at runtime.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
📚 Learning: 2025-06-09T08:01:35.973Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1696
File: node_package/src/RSCProvider.tsx:40-41
Timestamp: 2025-06-09T08:01:35.973Z
Learning: In React components, cache objects (like Map, Record, etc.) that are created within the component's scope or factory function closure are automatically garbage collected when the component unmounts. This pattern is acceptable and doesn't cause memory leaks because the cache is tied to the component lifecycle, unlike global or module-level caches that persist indefinitely.

Applied to files:

  • react_on_rails_pro/sig/react_on_rails_pro/cache.rbs
📚 Learning: 2025-02-13T14:29:49.267Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: spec/react_on_rails/utils_spec.rb:218-218
Timestamp: 2025-02-13T14:29:49.267Z
Learning: In RSpec tests, prefer using local variables over constants within test blocks to avoid constant redefinition warnings and maintain better test isolation.

Applied to files:

  • rakelib/run_rspec.rake
📚 Learning: 2024-06-27T05:22:46.156Z
Learnt from: justin808
Repo: shakacode/react_on_rails PR: 1622
File: spec/dummy/spec/rake/assets_precompile_rake_spec.rb:12-12
Timestamp: 2024-06-27T05:22:46.156Z
Learning: When stubbing environment variables in RSpec tests, use `before` and `after` hooks to ensure that the original values are restored after the tests, preventing any side effects on other tests. Example provided by justin808:
```ruby
describe "My test" do
  before do
    original_value = ENV["VARIABLE_NAME"]
    allow(ENV).to receive(:[]).with("VARIABLE_NAME").and_return("stubbed_value")
  end

  after do
    allow(ENV).to receive(:[]).with("VARIABLE_NAME").and_call_original
    ENV["VARIABLE_NAME"] = original_value
  end

  it "tests something" do
    # Your test code here
  end
end
```
This practice ensures test isolation and reliability.

Applied to files:

  • rakelib/run_rspec.rake
🔇 Additional comments (7)
Steepfile (2)

1-23: Excellent documentation of the incremental adoption strategy.

The comments clearly explain the rationale for using a positive list approach and provide clear instructions for adding new files to type checking. This will help maintainers understand the strategy.


39-51: LGTM! Configuration is appropriate.

The signature directory, diagnostics, and library declarations are properly configured for the files being checked.

rakelib/rbs.rake (2)

43-65: LGTM! Consistent implementation with the validate task.

The steep task follows the same pattern as the validate task, which provides consistency. The error handling appropriately distinguishes between validation failures and command-not-found scenarios.


67-68: LGTM! Useful composite task.

The :all task provides a convenient way to run both RBS validation and Steep type checking in sequence.

rakelib/run_rspec.rake (2)

50-54: LGTM! Gem task properly configured for RBS runtime checking.

The task correctly passes RBS environment variables for unit tests as documented in the coverage strategy.


57-60: LGTM! Integration tests correctly configured with RBS runtime checking.

This addresses the past review comment about whether integration tests should have these options - they do, which aligns with the documented coverage strategy.

react_on_rails_pro/sig/react_on_rails_pro/cache.rbs (1)

1-13: ****

The RBS signatures are correct as written. The return type difference between base_cache_key and react_component_cache_key is intentional and accurate:

  • base_cache_key returns Array[String] because it builds a fixed array of strings: [type, ReactOnRails::VERSION, ReactOnRailsPro::VERSION] (plus optional bundle_hash).
  • react_component_cache_key returns Array[untyped] because it spreads the base key array and appends cache_key_value—an arbitrary value extracted from options[:cache_key] that can be any type (as confirmed by test fixtures showing object instances in the returned array).

The original concern about consistency is unfounded; the signatures correctly reflect the implementation's runtime behavior.

Likely an incorrect or invalid review comment.

@claude
Copy link

claude bot commented Nov 10, 2025

Code Review: RBS Validation and Runtime Type Checking

I've reviewed PR #1950 and overall this is a well-structured implementation that follows industry best practices. Here's my detailed feedback:

✅ Strengths

  1. Comprehensive Type Safety Approach

    • Correctly implements the three-pronged approach from Evil Martians: validation, runtime checking, and static analysis
    • Good use of opt-in runtime checking via environment variable to prevent silent failures
    • Proper separation of concerns between static and runtime checks
  2. CI Integration

    • Appropriately added RBS validation to both main and Pro lint workflows
    • Smart decision to disable Steep by default until signatures are complete (avoiding 374 errors)
    • Clear TODO comments explaining why Steep is disabled
  3. Configuration Quality

    • Steepfile is well-documented with clear explanations of what's excluded and why
    • Explicit positive list approach (checking only files with signatures) prevents false errors
    • Alphabetically ordered file list for easy maintenance
  4. Pro Package RBS Signatures

    • Comprehensive type coverage for Configuration class with all 24 attributes
    • Good use of untyped for complex Hash values where appropriate
    • Proper module structure matching the Ruby code

🔍 Issues & Recommendations

Critical Issues

1. RBS Runtime Checking Not Actually Enabled (rakelib/run_rspec.rake:23-67)

The runtime checking implementation has a logical flaw - ENABLE_RBS_RUNTIME_CHECKING is never set anywhere in the codebase, CI workflows, or documentation. This means runtime checking is effectively never enabled, and one of the three pillars of your type safety strategy is not functioning.

Recommendations:

  • Either: Remove ENV["ENABLE_RBS_RUNTIME_CHECKING"] check and enable by default when RBS gem is available
  • Or: Add the environment variable to CI workflows
  • Or: Document how developers should enable it in CLAUDE.md

2. Silent stderr Suppression May Hide Real Issues (rakelib/rbs.rake:13-16)

The 2>/dev/null suppresses ALL stderr output initially. If there are validation errors mixed with warnings, they might be hidden. Consider using Open3.capture3 for better error handling instead.

Performance & Best Practices

3. Redundant bundle exec in Rake Tasks (rakelib/rbs.rake:13, 42)

The comments claim bundle exec is needed even though rake runs in bundle context, but this is incorrect. When running bundle exec rake, the rake process already runs within the bundle context. The extra bundle exec adds unnecessary overhead.

4. Missing Test Coverage

No tests verify that:

  • RBS signatures are syntactically valid
  • Runtime type checking actually catches type errors
  • The rake tasks work correctly

Consider adding at least one test that intentionally violates a type signature to verify runtime checking works.

Documentation

5. Missing CLAUDE.md Updates

CLAUDE.md doesn't mention how to enable RBS runtime checking, when to update RBS signatures, or how to run RBS validation locally. Consider adding an "RBS Type Checking" section.

📊 Test Plan Assessment

Before merging, verify:

  1. CI actually runs RBS validation (check workflow run logs)
  2. Runtime checking works when enabled (currently untested)
  3. Add at least one test that verifies type checking catches errors

🎯 Summary

Overall Assessment: Solid foundation for type safety, but the runtime checking component needs attention before merge.

Priority Actions:

  1. HIGH: Fix runtime checking to actually work (either enable by default or add ENV var to CI)
  2. MEDIUM: Improve error handling in rake tasks (use Open3 instead of stderr redirect)
  3. MEDIUM: Add test coverage for runtime type checking
  4. LOW: Update CLAUDE.md with RBS workflow documentation

Verdict: Request changes for item #1 (runtime checking), others can be follow-up PRs.

Great work on the comprehensive approach to type safety! The Steepfile documentation and CI integration are particularly well done.

@claude
Copy link

claude bot commented Nov 10, 2025

PR Review: Add RBS validation and runtime type checking to CI

Thank you for this comprehensive PR implementing RBS type checking! This is a well-structured enhancement that follows Ruby typing best practices. Here's my detailed review:


Strengths

  1. Well-designed incremental approach: Using a positive list in Steepfile for files ready for type checking is pragmatic and maintainable.

  2. Comprehensive documentation: The CLAUDE.md additions are thorough and provide clear guidance on:

    • Quick start commands
    • Runtime type checking behavior
    • How to add new signatures
    • Escape hatch with DISABLE_RBS_RUNTIME_CHECKING
  3. Three-layer validation strategy: Following Evil Martians best practices with:

    • Static validation (rbs validate)
    • Runtime checking (rbs/test/setup)
    • Static analysis (Steep)
  4. Thoughtful CI integration: Enabling validation immediately while keeping Steep commented out until errors are fixed is a good balance.

  5. Error handling improvements: Using Open3.capture3 in rake tasks provides better error handling and output separation.

  6. Pro package coverage: Including RBS signatures for Pro package shows commitment to type safety across the monorepo.


🔍 Code Quality Observations

rakelib/rbs.rake (lines 39-54)

The steep task implementation looks good, but consider adding the require "steep" at the top of the task for consistency:

desc "Run Steep type checker"
task :steep do
  require "steep"  # Add explicit require for consistency
  puts "Running Steep type checker..."
  # ... rest of implementation
end

rakelib/run_rspec.rake (lines 37-48)

The rbs_runtime_env_vars helper is well-designed with:

  • ✅ Graceful fallback when RBS unavailable
  • ✅ Opt-out mechanism via environment variable
  • ✅ Clear documentation

However, there's a minor issue: the method is defined at module level but used within tasks. Consider moving it to the ReactOnRails::TaskHelpers module for better organization:

# In task_helpers.rb or at the top of the namespace
module ReactOnRails
  module TaskHelpers
    def rbs_runtime_env_vars
      # ... implementation
    end
  end
end

⚠️ Potential Issues

1. Runtime Type Checking Performance Impact

Runtime RBS checking can slow down tests significantly (10-30% overhead). Consider:

  • Adding a note in CLAUDE.md about expected performance impact
  • Measuring the actual overhead and documenting it
  • Perhaps enabling runtime checking only for a subset of critical tests in CI

Suggestion: Add timing comparison to PR description or documentation:

# Without RBS runtime checking
time rake run_rspec:gem
# With RBS runtime checking  
DISABLE_RBS_RUNTIME_CHECKING=false time rake run_rspec:gem

2. Missing require "steep" guard

In rakelib/rbs.rake:56-57, the :steep task doesn't have a require guard like :validate does. If Steep isn't installed, the task will fail less gracefully:

desc "Run Steep type checker"
task :steep do
  begin
    require "steep"
  rescue LoadError
    puts "⚠️  Steep gem not installed. Run 'bundle install' to install it."
    exit 1
  end
  
  puts "Running Steep type checker..."
  # ...
end

3. Pro Package Gemfile dependency missing

react_on_rails_pro/Gemfile.development_dependencies adds rbs but not steep. For consistency with the main gem, consider adding:

gem "rbs", require: false
gem "steep", require: false  # Add this

🔒 Security Considerations

✅ No security concerns identified. The changes are development/testing infrastructure only.


🧪 Test Coverage

Strengths:

  • Runtime checking enabled for gem unit tests and dummy integration tests
  • Appropriately disabled for example apps (user-facing demos)

Suggestions:

  1. Add RBS validation to pre-commit hooks: Since the PR mentions git hooks run linting automatically, consider adding RBS validation:

    bundle exec rake rbs:validate
  2. Test the opt-out mechanism: Add a CI job that explicitly tests DISABLE_RBS_RUNTIME_CHECKING=true to ensure the escape hatch works.

  3. Verify error reporting: Intentionally introduce a type error in a test to verify that runtime checking catches it and provides useful error messages.


📊 Performance Considerations

  1. Steep disabled in CI: Good decision given 374 type errors. The TODO comment is clear.

  2. Runtime checking overhead: As mentioned above, document the performance impact and consider selective enablement strategies.

  3. CI execution time: Monitor whether the additional rake rbs:validate step significantly increases CI time. Based on typical RBS validation performance, this should be minimal (<10 seconds).


📝 Documentation Quality

Excellent documentation! The CLAUDE.md additions are comprehensive and well-organized. Minor suggestions:

  1. Add troubleshooting section:

    ### Troubleshooting
    
    **RBS validation fails with "unknown type"**: Ensure all referenced types have signatures or are from loaded libraries.
    
    **Runtime checks cause test failures**: Check if type signatures match actual method behavior. Use `DISABLE_RBS_RUNTIME_CHECKING=true` to isolate RBS issues.
  2. Version compatibility: Document minimum RBS and Steep versions if there are specific requirements.


🎯 Final Recommendations

Before Merge:

  1. ✅ Add require "steep" guard in rakelib/rbs.rake:40
  2. ✅ Consider adding Steep to Pro package Gemfile
  3. ✅ Document performance impact of runtime checking
  4. ⚠️ Verify all CI checks pass (couldn't verify due to permission constraints)

Future Enhancements:

  1. Consider selective runtime checking (e.g., only core modules) to reduce test overhead
  2. Add RBS validation to pre-commit hooks
  3. Create a milestone/issues for fixing the 374 Steep errors
  4. Consider adding RBS signatures for generators once core types are stable

📈 Overall Assessment

Score: 8.5/10

This is a high-quality PR that implements type checking infrastructure following industry best practices. The incremental approach, comprehensive documentation, and thoughtful CI integration demonstrate strong engineering practices.

The main areas for improvement are:

  • Minor error handling enhancements
  • Performance impact documentation
  • Small consistency fixes between main and Pro packages

Recommendation: Approve with minor revisions suggested above.


Great work on advancing type safety in React on Rails! 🎉

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9ea26db and dba6179.

📒 Files selected for processing (3)
  • CLAUDE.md (1 hunks)
  • rakelib/rbs.rake (3 hunks)
  • rakelib/run_rspec.rake (1 hunks)
🧰 Additional context used
🧠 Learnings (9)
📓 Common learnings
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: lib/react_on_rails/helper.rb:190-197
Timestamp: 2025-02-18T13:08:01.477Z
Learning: RSC support validation in React on Rails Pro is handled through a chain of validations:
1. Pro version check in `run_stream_inside_fiber`
2. RSC support check during pack generation via `ReactOnRailsPro.configuration.enable_rsc_support`
3. RSC support validation during component registration
This makes additional validation in the helper methods unnecessary.
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1875
File: lib/react_on_rails/utils.rb:112-124
Timestamp: 2025-10-23T17:22:01.074Z
Learning: In React on Rails, when Pro is installed but not licensed, the intended behavior is to raise an error on boot. The `react_on_rails_pro?` method validates licenses and should raise errors early (including during path resolution in methods like `server_bundle?`) to enforce licensing requirements rather than failing later with obscure errors.
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1745
File: node_package/src/RSCRequestTracker.ts:8-14
Timestamp: 2025-07-08T05:57:29.630Z
Learning: The global `generateRSCPayload` function in React on Rails Pro (RORP) is provided by the framework during rendering requests, not implemented in application code. The `declare global` statements are used to document the expected interface that RORP will inject at runtime.
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: lib/react_on_rails/helper.rb:190-197
Timestamp: 2025-02-18T13:08:01.477Z
Learning: RSC support validation is handled in deeper level calls of the React on Rails Pro codebase, so it doesn't need to be validated again in the `rsc_payload_react_component` helper method.
Learnt from: alexeyr-ci
Repo: shakacode/react_on_rails PR: 1644
File: node_package/src/ReactOnRailsRSC.ts:87-87
Timestamp: 2024-12-12T13:07:09.929Z
Learning: When handling errors in 'node_package/src/ReactOnRailsRSC.ts', include the error stack in error messages in development and test environments to aid debugging.
Learnt from: justin808
Repo: shakacode/react_on_rails PR: 1770
File: lib/generators/react_on_rails/templates/base/base/app/javascript/src/HelloWorld/ror_components/HelloWorld.client.jsx:2-2
Timestamp: 2025-09-16T08:01:11.146Z
Learning: React on Rails uses webpack CSS Modules configuration with namedExports: true, which requires the import syntax `import * as style from './file.module.css'` rather than the default export pattern. This configuration enables better tree shaking and bundle size optimization for CSS modules.
📚 Learning: 2025-02-13T14:29:49.267Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: spec/react_on_rails/utils_spec.rb:218-218
Timestamp: 2025-02-13T14:29:49.267Z
Learning: In RSpec tests, prefer using local variables over constants within test blocks to avoid constant redefinition warnings and maintain better test isolation.

Applied to files:

  • rakelib/run_rspec.rake
📚 Learning: 2024-06-27T05:22:46.156Z
Learnt from: justin808
Repo: shakacode/react_on_rails PR: 1622
File: spec/dummy/spec/rake/assets_precompile_rake_spec.rb:12-12
Timestamp: 2024-06-27T05:22:46.156Z
Learning: When stubbing environment variables in RSpec tests, use `before` and `after` hooks to ensure that the original values are restored after the tests, preventing any side effects on other tests. Example provided by justin808:
```ruby
describe "My test" do
  before do
    original_value = ENV["VARIABLE_NAME"]
    allow(ENV).to receive(:[]).with("VARIABLE_NAME").and_return("stubbed_value")
  end

  after do
    allow(ENV).to receive(:[]).with("VARIABLE_NAME").and_call_original
    ENV["VARIABLE_NAME"] = original_value
  end

  it "tests something" do
    # Your test code here
  end
end
```
This practice ensures test isolation and reliability.

Applied to files:

  • rakelib/run_rspec.rake
📚 Learning: 2025-09-29T14:45:42.687Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1833
File: lib/react_on_rails/dev/process_manager.rb:72-83
Timestamp: 2025-09-29T14:45:42.687Z
Learning: In Ruby bundler contexts, when bundler intercepts system commands for executables not in the Gemfile, both version checks (like `system("foreman", "--version")`) and execution commands (like `system("foreman", "start", ...)`) fail equally, both returning false. This means availability checks using version flags accurately reflect whether execution commands will work in the current bundler context.

Applied to files:

  • rakelib/rbs.rake
📚 Learning: 2025-02-18T13:08:01.477Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1644
File: lib/react_on_rails/helper.rb:190-197
Timestamp: 2025-02-18T13:08:01.477Z
Learning: RSC support validation in React on Rails Pro is handled through a chain of validations:
1. Pro version check in `run_stream_inside_fiber`
2. RSC support check during pack generation via `ReactOnRailsPro.configuration.enable_rsc_support`
3. RSC support validation during component registration
This makes additional validation in the helper methods unnecessary.

Applied to files:

  • rakelib/rbs.rake
  • CLAUDE.md
📚 Learning: 2025-02-12T16:38:06.537Z
Learnt from: Romex91
Repo: shakacode/react_on_rails PR: 1697
File: package-scripts.yml:28-28
Timestamp: 2025-02-12T16:38:06.537Z
Learning: The file `node_package/lib/ReactOnRails.full.js` is autogenerated during the build process and should not be present in the repository.

Applied to files:

  • CLAUDE.md
📚 Learning: 2025-10-23T17:22:01.074Z
Learnt from: AbanoubGhadban
Repo: shakacode/react_on_rails PR: 1875
File: lib/react_on_rails/utils.rb:112-124
Timestamp: 2025-10-23T17:22:01.074Z
Learning: In React on Rails, when Pro is installed but not licensed, the intended behavior is to raise an error on boot. The `react_on_rails_pro?` method validates licenses and should raise errors early (including during path resolution in methods like `server_bundle?`) to enforce licensing requirements rather than failing later with obscure errors.

Applied to files:

  • CLAUDE.md
📚 Learning: 2025-09-16T08:01:11.146Z
Learnt from: justin808
Repo: shakacode/react_on_rails PR: 1770
File: lib/generators/react_on_rails/templates/base/base/app/javascript/src/HelloWorld/ror_components/HelloWorld.client.jsx:2-2
Timestamp: 2025-09-16T08:01:11.146Z
Learning: React on Rails uses webpack CSS Modules configuration with namedExports: true, which requires the import syntax `import * as style from './file.module.css'` rather than the default export pattern. This configuration enables better tree shaking and bundle size optimization for CSS modules.

Applied to files:

  • CLAUDE.md
📚 Learning: 2025-04-26T21:55:55.874Z
Learnt from: alexeyr-ci2
Repo: shakacode/react_on_rails PR: 1732
File: spec/dummy/client/app-react16/startup/ReduxSharedStoreApp.client.jsx:40-44
Timestamp: 2025-04-26T21:55:55.874Z
Learning: In the react_on_rails project, files under `app-react16` directories are copied/moved to corresponding `/app` directories during the conversion process (removing the `-react16` suffix), which affects their relative import paths at runtime.

Applied to files:

  • CLAUDE.md

@justin808 justin808 force-pushed the justin808/rbs-ci-validation branch from dba6179 to efca28f Compare November 12, 2025 21:39
@justin808
Copy link
Member Author

@claude review this PR

@claude
Copy link

claude bot commented Nov 12, 2025

Claude finished @justin808's task —— View job


PR Review: Add RBS validation and runtime type checking to CI

I've completed a thorough review of this PR. Overall, this is an excellent implementation of comprehensive RBS type checking that follows industry best practices. The three-layer approach (validation, runtime checking, and static analysis) is exactly right.


✅ Strengths

1. Well-Architected Type Checking Strategy

  • Correctly implements the three-pronged approach from Evil Martians' "Climbing Steep Hills" article
  • Static validation (rbs:validate) ensures signatures are syntactically correct
  • Runtime checking verifies actual method calls match signatures during tests
  • Static analysis (Steep) catches type errors without running code
  • Smart decision to disable Steep in CI initially (374 errors) while keeping validation active

2. Excellent Documentation

  • The Steepfile has outstanding inline documentation explaining the positive-list approach
  • Clear rationale for excluded files/directories with specific reasons
  • Comprehensive CLAUDE.md section on RBS type checking with quick start guide
  • Well-commented runtime checking configuration in run_rspec.rake

3. Thoughtful Runtime Checking Implementation

  • Enabled by default for gem and integration tests (where it matters most)
  • Opt-out mechanism via DISABLE_RBS_RUNTIME_CHECKING environment variable
  • Graceful fallback when RBS gem is unavailable
  • Appropriate coverage strategy: gem + dummy tests, but not example apps

4. Clean CI Integration

  • RBS validation properly integrated into both main and Pro lint workflows
  • Steep appropriately disabled with clear TODO comment
  • No breaking changes to existing workflows

5. Comprehensive Pro Package Signatures

  • Well-structured RBS signatures for Pro package configuration
  • Good coverage of all 24+ configuration attributes
  • Proper use of nilable types and optional parameters

6. Code Quality

  • Follows repository conventions and best practices
  • Uses Open3.capture3 for better error handling in rake tasks
  • Proper error messages and exit codes
  • Clean separation of concerns

🔍 Issues Found

Critical Issues

1. Runtime Checking RUBYOPT Clobbers Existing Flags (run_rspec.rake:42)

Issue: The code sets RUBYOPT='-rrbs/test/setup' which completely replaces any existing RUBYOPT flags a developer may have set (e.g., --enable-yjit, --jit, warnings toggles).

Impact: Developers who have custom RUBYOPT settings will lose them during test runs with RBS enabled.

Current code:

"RBS_TEST_TARGET='ReactOnRails::*' RUBYOPT='-rrbs/test/setup'"

Recommended fix:

existing_rubyopt = ENV["RUBYOPT"]
rubyopt_parts = ["-rrbs/test/setup", existing_rubyopt].compact.reject(&:empty?)
"RBS_TEST_TARGET='ReactOnRails::*' RUBYOPT='#{rubyopt_parts.join(" ")}'"

This preserves any existing RUBYOPT flags while adding the RBS runtime hook.


High Priority Issues

2. Missing bundle exec in Rake Tasks (rakelib/rbs.rake:16, 44)

Issue: The rake tasks use bare rbs and steep commands without bundle exec:

stdout, stderr, status = Open3.capture3("rbs -I sig validate")
stdout, stderr, status = Open3.capture3("steep check")

Impact: This will fail if the gems aren't in the system PATH, which is common in CI environments or when using bundle install --path vendor/bundle.

Recommended fix:

stdout, stderr, status = Open3.capture3("bundle exec rbs -I sig validate")
stdout, stderr, status = Open3.capture3("bundle exec steep check")

The comment "use rbs directly, not bundle exec since we're already in bundle context" is misleading. While the rake process runs in bundle context, spawned shell commands via system() or Open3.capture3() do NOT inherit that context.


Medium Priority Issues

3. Pro Configuration Type Signature May Be Too Restrictive (react_on_rails_pro/sig/react_on_rails_pro/configuration.rbs:34, 40)

Potential Issues:

  • Line 34: renderer_http_pool_warn_timeout: Float? - Should accept Numeric? to allow Integer assignments (e.g., = 5)
  • Line 40: assets_to_copy: Array[String]? - Should accept Array[String | Pathname]? since code uses Rails.root.join(...)

Impact: Runtime type checking will raise TypeError for common usage patterns.

Recommendation: Review actual usage in Pro package and widen types if needed.


4. No Test Coverage for Type Checking Infrastructure

Missing tests:

  • No tests verify RBS signatures load correctly
  • No tests that runtime checking actually catches type violations
  • No tests for the new rake tasks

Recommendation: Add at least one integration test that:

  1. Intentionally violates a type signature
  2. Verifies runtime checking catches it
  3. Documents the error message format

This would validate the entire system works as expected.


5. Performance Impact Not Documented

Issue: Runtime type checking adds overhead to test execution (typically 5-15%).

Recommendation:

  • Document expected performance impact in CLAUDE.md
  • Consider measuring and reporting the actual overhead
  • Add guidance on when to disable for faster test iterations

Minor Issues

6. Duplicate yarn install in CI (.github/workflows/lint-js-and-ruby.yml:87, 98)

Lines 87 and 98 both run cd spec/dummy && yarn install, which is wasteful.

Recommendation: Remove the duplicate at line 98.


7. Inconsistent RBS Command Between Main and Pro

  • Main gem workflow: bundle exec rake rbs:validate (line 93)
  • Pro gem workflow: bundle exec rbs -I sig validate (line 127 in pro-lint.yml)

Recommendation: Use rake tasks consistently for better error messages and standardization.


🎯 Recommendations

Before Merge (High Priority)

  1. Fix RUBYOPT handling to preserve existing flags
  2. Add bundle exec to rbs.rake system calls
  3. ⚠️ Verify Pro type signatures don't break existing code (check the Integer vs Float issue)

After Merge (Medium Priority)

  1. Add test coverage for runtime type checking
  2. Document performance impact
  3. Remove duplicate yarn install in CI
  4. Standardize RBS validation commands across workflows

Future Enhancements

  1. Create tracking issue for fixing the 374 Steep errors
  2. Consider adding RBS validation to pre-commit hooks
  3. Expand type coverage incrementally to more files

🔒 Security & Performance

Security: ✅ No security concerns identified. Type checking is development/test-only.

Performance:

  • ✅ RBS validation: Fast (~1-5 seconds in CI)
  • ✅ Runtime checking: Only in tests, not production (5-15% overhead acceptable)
  • ✅ Steep disabled: No impact until re-enabled

📊 Code Review Summary

Category Rating Notes
Architecture ⭐⭐⭐⭐⭐ Excellent three-layer approach
Documentation ⭐⭐⭐⭐⭐ Outstanding inline docs and guides
Code Quality ⭐⭐⭐⭐ Clean, follows conventions, minor issues
Test Coverage ⭐⭐⭐ Good for main changes, missing type checking tests
CI Integration ⭐⭐⭐⭐⭐ Well-integrated, pragmatic Steep disabling

Overall Score: 4.5/5 - Excellent work with a few fixable issues.


🎉 Conclusion

This is a high-quality PR that establishes solid type checking infrastructure for React on Rails. The incremental adoption strategy is pragmatic and the implementation follows Ruby type checking best practices.

Main concerns:

  1. RUBYOPT handling needs fixing to avoid breaking developer workflows
  2. Missing bundle exec will cause failures in some environments
  3. Pro type signatures may need widening for common usage patterns

Once these issues are addressed, this will provide excellent type safety guarantees and catch bugs early in development.

Recommendation: Request changes for issues #1 and #2, which are straightforward fixes. Issue #3 should be verified against actual Pro package usage.

Great work on adopting modern Ruby type checking practices! 🎉


CI Status: 2 failures in "Main test" and "Pro Integration Tests" - these appear unrelated to the RBS changes based on the test names.


@claude
Copy link

claude bot commented Nov 13, 2025

PR Review: Add RBS validation and runtime type checking to CI

Overall Assessment

This is an excellent PR that implements comprehensive RBS type checking with a well-thought-out incremental approach. The implementation follows best practices from the Evil Martians article and provides solid foundations for type safety. The code quality is high, documentation is thorough, and the testing approach is sound.

✅ Strengths

1. Excellent Incremental Approach

  • Using a positive list in Steepfile rather than checking all files is pragmatic and maintainable
  • Clear documentation of excluded files with rationale (Steepfile:10-16)
  • Steep checks currently disabled in CI (commented out) until errors are resolved - shows good judgment

2. Comprehensive Three-Pronged Strategy

  • Static validation (rbs:validate) - catches syntax errors
  • Runtime checking (enabled by default in tests) - catches actual type violations during execution
  • Static analysis (Steep) - catches type errors without running code
  • This layered approach provides excellent coverage as per Evil Martians best practices

3. Well-Designed Runtime Type Checking

  • Smart graceful degradation when RBS gem unavailable (rakelib/run_rspec.rake:40-51)
  • Properly preserves existing RUBYOPT flags (important for YJIT, etc.)
  • Clear opt-out mechanism via DISABLE_RBS_RUNTIME_CHECKING env var
  • Good documentation of performance impact (5-15%) and when to disable

4. Excellent Testing

  • Added dedicated specs to verify runtime checking works (spec/react_on_rails/rbs_runtime_checking_spec.rb)
  • Tests intentionally trigger type violations to verify RBS catches them - clever approach
  • Added specs to verify rake tasks are properly defined (spec/react_on_rails/rake_tasks_spec.rb)

5. Thorough Documentation

  • CLAUDE.md has comprehensive RBS section with clear quick start, usage patterns, and pro package guidance
  • Inline comments explain rationale throughout (e.g., why bundle exec in rake tasks)
  • Steepfile documents excluded files and how to add new ones

6. Pro Package Integration

  • Created RBS signatures for Pro package (react_on_rails_pro/sig/)
  • Added validation to Pro CI workflow
  • Comprehensive Configuration type signature with all 24 attributes

🔍 Code Quality Observations

Minor Issues & Suggestions

1. Potential Issue: Runtime Checking in Integration Tests

In rakelib/run_rspec.rake:68-72, the env vars are concatenated for the dummy_no_turbolinks task:

env_vars = [rbs_runtime_env_vars, "DISABLE_TURBOLINKS=TRUE"].reject(&:empty?).join(" ")

Concern: If rbs_runtime_env_vars returns "RBS_TEST_TARGET='ReactOnRails::*' RUBYOPT='-rrbs/test/setup'", the join with a space might create issues with shell quoting.

Suggestion: Consider a more robust approach:

env_vars_array = []
env_vars_array << rbs_runtime_env_vars unless rbs_runtime_env_vars.empty?
env_vars_array << "DISABLE_TURBOLINKS=TRUE"
env_vars = env_vars_array.join(" ")

Or ensure the run_tests_in method properly handles complex environment variables.

2. Test Coverage Gap

The runtime checking test (spec/react_on_rails/rbs_runtime_checking_spec.rb:15-24) tests that type violations are caught, but the test assumes server_bundle_js_file is a typed attribute.

Verify: Does the ReactOnRails::Configuration RBS signature actually type this attribute? The test would fail silently (skip) if the signature doesn't exist or doesn't type that specific attribute.

Suggestion: Add a comment indicating which RBS signature file defines this type, or add an assertion that the signature exists before testing violations.

3. Rake Task Error Handling

In rakelib/rbs.rake:18, the code does:

stdout, stderr, status = Open3.capture3("bundle exec rbs -I sig validate")

Minor improvement: Consider adding a timeout to prevent hung processes:

stdout, stderr, status = Open3.capture3("bundle exec rbs -I sig validate", timeout: 60)

This is especially important for CI environments.

4. Pro Package Rake Tasks Missing Steep

The Pro package has rakelib/rbs.rake but it only includes :validate, :check, and :list tasks - no :steep or :all tasks like the main gem.

Question: Is this intentional? Should the Pro package also support Steep type checking? If not, a comment explaining why would be helpful.

5. RuboCop Exclusion

In .rubocop.yml:42, Steepfile is excluded from Naming/FileName:

- '**/Steepfile'

Good catch - this is correct since Steepfile doesn't follow snake_case convention. Consider if any other RBS-related files need exclusions.

🔒 Security Considerations

No security concerns identified. The changes are limited to type checking infrastructure with no user-facing changes or data handling modifications.

⚡ Performance Considerations

Well documented: The 5-15% overhead for runtime type checking during tests is acceptable and properly documented in CLAUDE.md. The opt-out mechanism allows developers to disable it during rapid iteration.

Suggestion: Consider collecting actual metrics from CI runs before/after to validate the estimated overhead and update documentation accordingly.

📊 Test Coverage

Excellent coverage for the new functionality:

  • ✅ Rake tasks verified to exist
  • ✅ Runtime type checking verified to catch violations
  • ✅ Environment configuration verified

Potential gap: No integration test that runs actual RSpec with RBS enabled end-to-end. Consider adding a simple test that:

  1. Sets up RBS environment
  2. Runs a single spec file with runtime checking
  3. Verifies it completes successfully

🎯 Best Practices

The PR follows all the repository conventions from CLAUDE.md:

  • ✅ Files properly formatted (RuboCop configured)
  • ✅ Comprehensive documentation
  • ✅ Tests included
  • ✅ CI integration
  • ✅ Proper use of Open3 for shell commands
  • ✅ Good error messages and user feedback

📋 Checklist Review

From the PR description:

  • RBS validation passes locally ✓
  • RuboCop passes ✓
  • All files properly formatted ✓
  • CI validates RBS signatures - In progress (workflow added)
  • CI runs Steep checks - Intentionally disabled (commented out until errors fixed)

🚀 Recommendations Before Merge

Must Fix

None - the PR is solid as-is.

Should Consider

  1. Verify env var concatenation in run_rspec.rake:69 works correctly with complex RUBYOPT values
  2. Add timeout to Open3.capture3 calls to prevent CI hangs
  3. Clarify Pro package Steep support - add comment if intentionally excluded

Nice to Have

  1. Document actual performance impact from CI runs
  2. Add end-to-end integration test for RBS runtime checking
  3. Consider adding a sample violation test for a Pro package class

🎓 Learning from This PR

This PR is an excellent example of:

  • Incremental type system adoption (positive lists, disabled checks)
  • Comprehensive documentation alongside code
  • Thoughtful error handling and graceful degradation
  • Testing the test infrastructure itself

The approach here can serve as a model for other gradual type adoption efforts.

Final Verdict

LGTM with minor suggestions 🎉

This PR is ready to merge. The suggestions above are minor improvements that could be addressed in follow-up PRs. The core implementation is solid, well-tested, and properly documented.

The intentional decision to disable Steep in CI until errors are fixed (comment in .github/workflows/lint-js-and-ruby.yml:94-97) shows excellent judgment - it's better to add validation incrementally than to have a constantly failing check.

Great work on this comprehensive type checking implementation!

justin808 and others added 6 commits November 12, 2025 15:57
This commit implements comprehensive RBS type checking for both the main gem and Pro package:

1. CI Integration:
   - Added `rake rbs:validate` to main lint workflow
   - Added RBS validation to Pro gem lint workflow
   - Added Steep static type checker to CI pipeline

2. Runtime Type Checking:
   - Configured RSpec to run with RBS runtime checking for gem tests
   - Tests now run with RBS_TEST_TARGET='ReactOnRails::*' RUBYOPT='-rrbs/test/setup'
   - This provides runtime validation of type signatures during test execution

3. Steep Static Type Checker:
   - Added steep gem to development dependencies
   - Created Steepfile configuration for static type analysis
   - Added `rake rbs:steep` task for running static type checks
   - Added `rake rbs:all` task to run both validation and steep checks

4. Pro Gem RBS Types:
   - Created sig/ directory structure for Pro gem
   - Added type signatures for:
     - ReactOnRailsPro module
     - Configuration class with all attributes
     - Error, Cache, and Utils modules
   - Foundation for expanding type coverage in Pro package

5. Documentation:
   - Configured Steepfile to check lib/ directory
   - Set up library dependencies for proper type resolution

Follows best practices from Evil Martians' "Climbing Steep Hills" article:
- Static validation with rbs validate
- Runtime checking with rbs/test/setup
- Static analysis with Steep

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
The Pro gem has its own Gemfile.development_dependencies and needs
rbs added to run RBS validation in CI.
This commit addresses the 1527 Steep type errors that were causing CI to fail.

Changes:
- Updated Steepfile to only check files with complete RBS signatures (10 files)
  instead of the entire lib/ directory (which has 140+ files)
- Temporarily disabled Steep CI check until RBS signatures are complete
  (374 remaining errors in files with partial signatures)
- Fixed rake tasks to avoid nested bundle exec calls that were causing
  bundler errors in local development

The RBS validation step continues to work correctly and validates that
all type signatures are syntactically correct and internally consistent.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Critical Fixes:
- Fix bundle exec consistency in rbs.rake for both rbs and steep tasks
- Add comprehensive error handling for runtime type checking in run_rspec.rake
- Fix type signature accuracy: Array[untyped] -> Array[String] in Pro configuration
- Rewrite cache.rbs to match actual implementation (was describing non-existent method)

Enhanced Runtime Type Checking:
- Extract runtime checking into reusable rbs_runtime_env_vars helper
- Extend coverage to :dummy and :dummy_no_turbolinks tasks
- Add opt-in mechanism via ENV["ENABLE_RBS_RUNTIME_CHECKING"]
- Add comprehensive documentation explaining coverage strategy per Evil Martians best practices

Documentation Improvements:
- Add extensive Steepfile documentation explaining positive-list approach
- Document all intentionally excluded files/directories with reasons
- Add step-by-step guide for adding new files to type checking
- Explain why bundle exec is necessary even in rake context

Configuration:
- Add Steepfile to RuboCop Naming/FileName exclusions
- Alphabetically order checked files for easy maintenance

All changes improve type safety, prevent silent failures, and make the
system more maintainable for future developers.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
CRITICAL FIXES:

1. Runtime Type Checking Now Actually Works
   - Changed from opt-in (never enabled) to enabled-by-default with opt-out
   - Was: ENV["ENABLE_RBS_RUNTIME_CHECKING"] (never set anywhere)
   - Now: Enabled by default, disable with DISABLE_RBS_RUNTIME_CHECKING=true
   - This activates the third pillar of type safety (static, steep, runtime)

2. Improved Error Handling in RBS Validation
   - Replace fragile stderr redirection with Open3.capture3
   - Properly separates validation errors from warnings
   - No longer hides real issues with 2>/dev/null
   - Shows both stdout and stderr when validation fails

3. Remove Redundant bundle exec
   - Removed unnecessary 'bundle exec' from rake tasks
   - When running 'bundle exec rake', already in bundle context
   - Reduces overhead and simplifies implementation
   - Uses direct 'rbs' and 'steep' commands

DOCUMENTATION:

4. Comprehensive RBS Documentation in CLAUDE.md
   - Added "RBS Type Checking" section with quick start
   - Documents runtime checking behavior (enabled by default)
   - Explains how to disable for faster test runs
   - Step-by-step guide for adding new type signatures
   - Lists files currently type-checked
   - Pro package validation instructions

These fixes ensure runtime type checking actually runs in tests, providing
comprehensive type safety validation that catches errors static analysis misses.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Critical fixes:
- Preserve existing RUBYOPT flags instead of clobbering them
- Add bundle exec to rbs and steep commands in rake tasks
- Remove duplicate yarn install in CI workflow
- Standardize RBS commands between main and Pro workflows

Improvements:
- Add test coverage for RBS type checking infrastructure
- Document performance impact (5-15% overhead) in CLAUDE.md
- Create rbs.rake for Pro package for consistency
- Add guidance on when to disable runtime checking

Files modified:
- rakelib/run_rspec.rake: Fix RUBYOPT handling to append vs replace
- rakelib/rbs.rake: Add bundle exec with explanatory comments
- .github/workflows/lint-js-and-ruby.yml: Remove duplicate yarn install
- .github/workflows/pro-lint.yml: Use rake rbs:validate consistently
- react_on_rails_pro/rakelib/rbs.rake: New file for Pro consistency
- spec/react_on_rails/rake_tasks_spec.rb: Test RBS rake tasks
- spec/react_on_rails/rbs_runtime_checking_spec.rb: Test runtime checking
- CLAUDE.md: Add performance impact documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
justin808 and others added 2 commits November 12, 2025 15:57
Minor improvements:
- Fix env vars concatenation for robustness with complex variables
- Document Configuration RBS signature in test (sig/react_on_rails/configuration.rbs:4)
- Add timeout (60s) to Open3.capture3 commands to prevent hung processes in CI
- Document why Pro package omits Steep tasks (doesn't use Steep type checker)

Files modified:
- rakelib/run_rspec.rake: Improve env vars handling for DISABLE_TURBOLINKS
- rakelib/rbs.rake: Add timeout parameter to Open3.capture3 calls
- react_on_rails_pro/rakelib/rbs.rake: Add comment explaining Steep omission + timeout
- spec/react_on_rails/rbs_runtime_checking_spec.rb: Document RBS signature location

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
- Replace Open3.capture3 timeout option with Timeout.timeout wrapper for Ruby 3.0+ compatibility
- Remove duplicate RBS definitions for PrerenderError and JsonParseError from sig/react_on_rails.rbs
- Add comment explaining error class definitions are in separate files
- Improve test documentation with clearer explanation of RBS runtime checking
- Add timeout handling to Pro package RBS rake tasks
- Add RuboCop disable for BlockLength in Pro package RBS rake file

The timeout option for Open3.capture3 was added in Ruby 3.3, but we support Ruby >= 3.0.0.
Using Timeout.timeout provides compatibility across all supported Ruby versions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@justin808 justin808 force-pushed the justin808/rbs-ci-validation branch from 544da89 to bd1265e Compare November 13, 2025 01:58
Change require_relative from "../spec_helper" to "spec_helper" to match
the correct path structure. The spec_helper is located at
spec/react_on_rails/spec_helper.rb, not spec/spec_helper.rb.

This fixes the LoadError in CI minimum dependency tests.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@justin808 justin808 merged commit 5bb036e into master Nov 13, 2025
24 of 25 checks passed
@justin808 justin808 deleted the justin808/rbs-ci-validation branch November 13, 2025 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants