Ruby: add overlay annotations to AST/CFG/SSA layers #19989

nickrolfe · 2025-07-07T11:20:07Z

Almost everything in those libraries is marked as overlay[local], with the exception that everything depending on resolution of constants and call targets is overlay[global].

Overlay compilation is currently disabled for Ruby, so the annotations won't currently have any effect on compilation or evaluation.

I've run Ruby DCA experiments on these changes, combined with an additional commit enabling overlay compilation, showing that they give 100% alert accuracy on nightly.qls and a curated source suite of ~200 sources.

The QL-for-QL changes are just to ensure that the compiler is happy; I haven't tested them with actual overlay analysis.

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

jbj · 2025-07-07T16:49:01Z

I think this is technically a breaking change and therefore needs a change note: if a class in a custom query overrides a predicate from a class that's now local, it's a compiler error. Reviewers should check that no supported extension point is becoming local.

hvitved

LGTM, some suggestions for predicates that can be made local.

hvitved · 2025-07-07T19:03:43Z

ruby/ql/lib/codeql/ruby/ast/Call.qll

@@ -38,6 +41,7 @@ class Call extends Expr instanceof CallImpl {
   * foo :bar "baz", qux: 123
   * ```
   */
+  overlay[global]


I think this predicate can reasonably be made local by replacing

p.getKey().getConstantValue().isSymbol(keyword)

below with

keyword = p.getKey().(SymbolLiteral).(StringlikeLiteralImpl).getStringValue()

(requires an private import internal.Literal import).

hvitved · 2025-07-07T19:07:02Z

ruby/ql/lib/codeql/ruby/ast/Expr.qll

@@ -425,6 +429,7 @@ class StringConcatenation extends Expr, TStringConcatenation {
   * "foo" "bar#{ n }"
   * ```
   */
+  overlay[global]


I think this can be made local:

final string getConcatenatedValueText() { forall(StringLiteral c | c = this.getString(_) | exists(c.(StringlikeLiteralImpl).getStringValue()) ) and result = concat(string valueText, int i | valueText = this.getString(i).(StringlikeLiteralImpl).getStringValue() | valueText order by i ) }

(requires private import internal.Literal).

The pragma still needs to be removed.

Unfortunately, the test failure shows that this change isn't equivalent. It no longer gives a result for "foo" "bar#{ 1 * 1 }" 'baz', since StringlikeLiteralImpl.getStringValue() doesn't give a result for the string with the interpolation.

The qldoc for getConcatenatedValueText() is actually wrong then, for the current behaviour.

I think I would prefer backing this change out — along with the other, similar changes — and keeping it global. At least in this PR. What do you think?

ruby/ql/lib/codeql/ruby/ast/Literal.qll

hvitved · 2025-07-07T19:10:36Z

ruby/ql/lib/codeql/ruby/ast/Method.qll

@@ -40,18 +43,22 @@ class MethodBase extends Callable, BodyStmt, Scope, TMethodBase {
   * Holds if this method is public.
   * Methods are public by default.
   */
+  overlay[global]


It feels like all of these should be local, but it is probably not important.

hvitved · 2025-07-07T19:12:32Z

ruby/ql/lib/codeql/ruby/ast/Pattern.qll

@@ -203,6 +206,7 @@ class HashPattern extends CasePattern, THashPattern {
  }

  /** Gets the value for a given key name. */
+  overlay[global]


Can be made local:

key = this.getKey(i).(StringlikeLiteralImpl).getStringValue()

hvitved · 2025-07-07T19:14:29Z

ruby/ql/lib/codeql/ruby/controlflow/CfgNodes.qll

@@ -310,6 +317,7 @@ module ExprNodes {
    final ExprCfgNode getAnArgument() { result = this.getArgument(_) }

    /** Gets the keyword argument whose key is `keyword` of this call. */
+    overlay[global]


Again, this can be made local.

hvitved · 2025-07-07T19:16:02Z

ruby/ql/lib/codeql/ruby/dataflow/internal/SsaImpl.qll

@@ -469,19 +474,24 @@ class ParameterExt extends TParameterExt {
  }
 }

+overlay[global]


What is it that makes this necessary?

There must have been some intermediate state where I needed it to fix compilation errors, but I can't see any reason why it's needed now. I'll remove it.

hvitved · 2025-07-08T11:55:33Z

ruby/ql/lib/codeql/ruby/dataflow/internal/SsaImpl.qll

@@ -375,10 +378,12 @@ private module Cached {
    Impl::uncertainWriteDefinitionInput(def, result)
  }

+  overlay[global]


Is this still needed?

I can remove it here, but the BarrierGuards module below needs to be global (at least to make the compiler happy).

nickrolfe · 2025-07-09T12:56:25Z

As mentioned in the comment above, I backed out some of the requested changes for getting string-like values, so they are global again.

I've also added a change-note.

github-actions bot added Ruby QL-for-QL labels Jul 7, 2025

nickrolfe mentioned this pull request Jul 7, 2025

Ruby: enable overlay compilation #19731

Open

nickrolfe force-pushed the nickrolfe/ruby-annotations branch 3 times, most recently from 4a725ba to 3c4a528 Compare July 7, 2025 14:57

nickrolfe marked this pull request as ready for review July 7, 2025 16:06

Copilot AI review requested due to automatic review settings July 7, 2025 16:06

nickrolfe requested review from a team as code owners July 7, 2025 16:06

nickrolfe added the no-change-note-required This PR does not need a change note label Jul 7, 2025

Copilot AI reviewed Jul 7, 2025

View reviewed changes

hvitved requested changes Jul 7, 2025

View reviewed changes

nickrolfe force-pushed the nickrolfe/ruby-annotations branch from 3c4a528 to 21dc396 Compare July 8, 2025 11:49

hvitved reviewed Jul 8, 2025

View reviewed changes

Ruby: add overlay[local] annotations to AST/CFG/SSA layers

b51940d

nickrolfe force-pushed the nickrolfe/ruby-annotations branch from 21dc396 to 1eda55d Compare July 8, 2025 12:14

Ruby: make resolveConstant overlay[global]

e1f2433

nickrolfe force-pushed the nickrolfe/ruby-annotations branch from 1eda55d to e1f2433 Compare July 9, 2025 10:59

Ruby: add changenote for overlay[local] annotations

c415795

github-actions bot added the documentation label Jul 9, 2025

nickrolfe removed the no-change-note-required This PR does not need a change note label Jul 9, 2025

hvitved approved these changes Jul 10, 2025

View reviewed changes

nickrolfe merged commit 3a0def7 into main Jul 10, 2025
58 checks passed

nickrolfe deleted the nickrolfe/ruby-annotations branch July 10, 2025 15:53

Ruby: add overlay annotations to AST/CFG/SSA layers #19989

Ruby: add overlay annotations to AST/CFG/SSA layers #19989

Conversation

nickrolfe commented Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

jbj commented Jul 7, 2025

Uh oh!

hvitved left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nickrolfe Jul 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nickrolfe commented Jul 9, 2025

Uh oh!

Uh oh!

Uh oh!

nickrolfe commented Jul 7, 2025 •

edited

Loading

nickrolfe Jul 8, 2025 •

edited

Loading