Skip to content

Merging specs from CRuby, JRuby and TruffleRuby

Jump to bottom
Benoit Daloze edited this page Oct 9, 2025 · 2 revisions

Introduction

Once per month, specs from all sources are merged together in this repo and then back to the 3 Ruby implementations, by a ruby/spec maintainer. This is not trivial so if you never did it before you should coordinate with someone who did.

Synchronizing specs between the ruby/spec repository and CRuby/JRuby/TruffleRuby consists of 3 steps. This is only the summary, follow every step on this page from top to bottom.

  1. Upstreaming changes from Ruby implementations:
    1. Pull changes made to mspec and push them to ruby/mspec
    2. Pull changes made to ruby specs and push them to ruby/spec
  2. Downstreaming changes from ruby/mspec and ruby/spec back to Ruby implementations:
    1. Push ruby/mspec & ruby/spec changes to implementations

The whole sync'ing should be done as quickly as possible (at most in the same day), and creating the PRs to the Ruby implementations too. For CRuby the PR/push should be merged as quickly as possible (because CRuby does not have merge commits), for JRuby/TruffleRuby they need to be created as quickly as possible but can be merged later with a merge commit without issues.

The reason for this timing is we essentially rm -rf spec/ruby && git clone ruby/spec in each Ruby implementation, overriding any changes. So if there are any specs committed in these repo between sync'ing to ruby/spec and back, they need to be imported to ruby/spec semi-manually, see Importing specs pushed concurrently.

Therefore when sync'ing specs you should try as much as possible to handle everything yourself, that means fixing the ruby/spec CI (including RuboCop), addressing transient specs, etc.

Most of the work is already automated by the sync-rubyspec.rb script.

Prerequisites

Install chruby

To check locally that new specs pass on all supported Ruby versions, specs will be run on the lowest and the highest supported Ruby versions. To switch between Ruby versions a chruby command is used. Use https://github.com/postmodern/chruby to install it.

Tip

If ZSH is used locally then chruby should be installed not only in ~/.zshrc but also in ~/.zshenv to be available in non-interactive sessions (when the script runs the chruby command).

Install Ruby versions to run specs

Get the lowest and the highest supported (by ruby/spec) Ruby versions from the CI config. Use ruby-install or ruby-build to install them to the ~/.rubies directory . The specs also will be run on the current Ruby head.

Build Ruby head

Ruby head is needed to ensure upstreamed new specs pass on it successfully.

The main instruction is located here. But in short the following commands should be executed locally in the Ruby working directory after cloning Ruby Git repository:

./autogen.sh
mkdir ~/.rubies
./configure --prefix="${HOME}/.rubies/ruby-master" --disable-install-doc
make -j8 install-nodoc

Ruby dev build will be available as ruby-master with chruby command.

Tip

For simplicity the mspec/tool/sync/ruby directory can be reused.

Layout

The MSpec and ruby/spec Git repositories should be cloned and be located outside any Ruby implementation working directory.

It's expected by the sync-rubyspec.rb script that MSpec and ruby/spec working directories are siblings:

mspec
rubyspec

The Ruby, JRuby and TruffleRuby Git repositories will be cloned by the script into the mspec/tool/sync directory:

  • mspec/tool/sync/ruby
  • mspec/tool/sync/jruby
  • mspec/tool/sync/truffleruby

Upstream MSpec changes in MRI, JRuby and TruffleRuby

  1. Check if there are any changes under spec/mspec in TruffleRuby, JRuby, Ruby (and git checkout master && git pull in each)
  2. If so:
    • cd mspec/tool/sync
    • ruby sync-rubyspec.rb --mspec mri (until it succeeds)
    • ruby sync-rubyspec.rb --mspec jruby (until it succeeds)
    • ruby sync-rubyspec.rb --mspec truffleruby (until it succeeds)
  3. cd mspec, git push (or make a PR, and use Rebase and merge)
  4. Check ruby/mspec CI and fix it until gets green

Important

Whenever sync-rubyspec.rb exits with non-zero you should rerun it after e.g. fixing conflicts or fixing specs to pass on every tested version.

Upstream ruby/spec changes in MRI, JRuby and TruffleRuby

  1. cd mspec/tool/sync, ruby sync-rubyspec.rb mri (until it succeeds)
  2. cd mspec/tool/sync, ruby sync-rubyspec.rb jruby (until it succeeds)
  3. cd mspec/tool/sync, ruby sync-rubyspec.rb truffleruby (until it succeeds)

Then push the changes to the ruby/spec Git repository:

$ cd rubyspec
$ git push

or make a PR, and use Rebase and merge

Then check ruby/spec CI and fix until it gets green.

Important

Whenever sync-rubyspec.rb exits with non-zero you should rerun it after e.g. fixing conflicts or fixing specs to pass on every tested version (in the sibling rubyspec directory).

Downstream ruby/spec to MRI, JRuby and TruffleRuby

The Ruby, JRuby and TruffleRuby Git repositories should be cloned (or reuse ones in mspec/tool/sync)

Tip

On macOS pbpaste could be used instead of xsel command.

Ruby

  1. Check if there are any new mspec or ruby/spec changes pushed concurrently with git pull; git log -p spec/ruby; git log -p spec/mspec, by comparing to the commit you just imported in ruby/mspec and ruby/spec. If there are new mspec or ruby/spec changes, see the Importing specs pushed concurrently section.
  2. Run the following commands:
cd ruby
git checkout master
spec/mspec/tool/pull-latest-mspec-spec
make test-spec MSPECOPT="-j"
make test-bundled-gems-spec

  1. Push directly to master or make a PR (if so, make sure to merge it ASAP)

  2. Check the CRuby Slack channel #alerts or https://rubyci.org/ some time after to see if there is any failure related to ruby/spec.

JRuby

  1. Do not git pull, stay on master as when you started the sync and checked for mspec changes. That way if new specs are pushed concurrently they will not be lost and just picked up by the next sync.
  2. Run the following commands:
cd jruby
git checkout -b update-specs
spec/mspec/tool/pull-latest-mspec-spec
  1. Make a PR (Milestone: Non-Release)
  2. Check CI, tag failed specs with xsel -b | spec/mspec/tool/tag_from_output.rb
  3. The PR must be merged with a Merge Commit (to avoid losing specs pushed concurrently).

TruffleRuby

  1. Do not git pull, stay on master as when you started the sync and checked for mspec changes. That way if new specs are pushed concurrently they will not be lost and just picked up by the next sync.
  2. Run the commands:
cd truffleruby
git checkout -b bd/update-specs-YYYYMMDD
spec/mspec/tool/pull-latest-mspec-spec
jt --build test fast
  1. Commit slow tags, generated by jt test fast command (use commit message Add slow tags)
  2. Tag failed specs with xsel -b | spec/mspec/tool/tag_from_output.rb (use commit message Add tags for new failing specs)
  3. Make a PR
  4. The PR must be merged with a Merge Commit (to avoid losing specs pushed concurrently).

Reminder to check RubyCI

See the last step of Ruby.

Importing specs pushed concurrently

When downstreaming ruby/spec to Ruby implementations, we override any changes in spec/ruby and spec/mspec. So if there are any specs committed in these repo between sync'ing to ruby/spec and back, they need to be imported to ruby/spec semi-manually, by using ONLY_FILTER=true ruby sync-rubyspec.rb mri|jruby|truffleruby and cherry-picking commits from the corresponding branch to the master branch.

Clone this wiki locally