Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Reduce RTS calls in the updateLayout flow #37127

Open
wants to merge 15 commits into
base: release
Choose a base branch
from

Conversation

rishabhrathod01
Copy link
Contributor

@rishabhrathod01 rishabhrathod01 commented Oct 29, 2024

Description

Fixes #37055

Automation

/ok-to-test tags="@tag.All"

🔍 Cypress test results

Tip

🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/11804864728
Commit: 2b0981b
Cypress dashboard.
Tags: @tag.All
Spec:


Tue, 12 Nov 2024 22:04:29 UTC

Communication

Should the DevRel and Marketing teams inform users about this change?

  • Yes
  • No

Summary by CodeRabbit

  • New Features

    • Introduced new constants for entity references and dynamic bindings to enhance functionality.
    • Added observation capabilities to various classes for improved monitoring of operations.
  • Bug Fixes

    • Improved error handling and monitoring for potential issues during entity retrieval.
  • Documentation

    • Updated comments and documentation to reflect new logic and enhance maintainability.

Copy link
Contributor

coderabbitai bot commented Oct 29, 2024

Walkthrough

The changes in this pull request involve the addition of new constants and modifications to several classes within the Appsmith codebase. The OnLoadSpanCE class receives three new constants related to entity references and dynamic bindings. The MustacheHelper and AstServiceImpl classes are updated to include an ObservationRegistry for enhanced monitoring capabilities. Additionally, the OnLoadExecutablesUtilCEImpl and AstServiceCEImpl classes undergo modifications to improve their handling of entity references and dynamic bindings, including the introduction of new methods and updates to existing ones.

Changes

File Change Summary
.../OnLoadSpanCE.java Added constants: GET_POSSIBLE_ENTITY_REFERENCES, GET_POSSIBLE_ENTITY_PARENTS_MAP, GET_POSSIBLE_REFERENCES_FROM_DYNAMIC_BINDING.
.../MustacheHelper.java Updated constructor to accept an ObservationRegistry and added a static field for it.
.../OnLoadExecutablesUtilCEImpl.java Added method getPossibleEntityReferencesMap, updated getPossibleEntityReferences with observation tap, refactored addDirectlyReferencedExecutablesToGraph, and improved error handling in getPossibleEntityParentsMap.
.../AstServiceImpl.java Updated constructor to include ObservationRegistry.
.../AstServiceCEImpl.java Added ObservationRegistry field, updated getPossibleReferencesFromDynamicBinding with observation tap.

Assessment against linked issues

Objective Addressed Explanation
Debug why RTS is taking ~3s per request (#37055) Changes include instrumentation, but impact on RTS timing is unclear.

Possibly related PRs

Suggested labels

JS Objects, Javascript Product

Suggested reviewers

  • sneha122
  • nidhi-nair

🎉 In the realm of code, new constants arise,
With observation tools, we sharpen our eyes.
From entity maps to bindings so fine,
Each change we make helps the system align.
Let's track and observe, with metrics in sight,
In the world of Appsmith, we code with delight! 🚀


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@rishabhrathod01 rishabhrathod01 changed the title Chore/rts-metrics chore: Add RTS metrics Oct 29, 2024
@github-actions github-actions bot added the skip-changelog Adding this label to a PR prevents it from being listed in the changelog label Oct 29, 2024
Copy link

Failed server tests

  • com.appsmith.server.refactors.ce.RefactoringServiceCETest#testRefactorWidgetName_forDefaultWidgetsInList_updatesBothWidgetsAndTemplateReferences

- After this change we reduce the RTS calls from 2*N to 2 where N is the no of bindings in the layout.
Copy link

Failed server tests

  • com.appsmith.server.services.LayoutActionServiceTest#OnLoadActionsWhenActionDependentOnActionViaWidget

@github-actions github-actions bot added Integrations Pod General Issues related to the Integrations Pod that don't fit into other tags. Integrations Pod Integrations Product Issues related to a specific integration Query & JS Pod Issues related to the query & JS Pod Task A simple Todo labels Oct 30, 2024
@rishabhrathod01 rishabhrathod01 added the ok-to-test Required label for CI label Oct 30, 2024
Copy link

Failed server tests

  • com.appsmith.server.services.LayoutActionServiceTest#OnLoadActionsWhenActionDependentOnActionViaWidget

Copy link

Failed server tests

  • com.appsmith.server.services.ce.ActionServiceCE_Test#invalidCreateActionNullName

Copy link

github-actions bot commented Nov 1, 2024

Failed server tests

  • com.appsmith.server.services.LayoutServiceTest#getActionsExecuteOnLoadWithAstLogic

Copy link

github-actions bot commented Nov 1, 2024

Failed server tests

  • com.appsmith.server.services.LayoutServiceTest#getActionsExecuteOnLoadWithAstLogic

1 similar comment
Copy link

github-actions bot commented Nov 1, 2024

Failed server tests

  • com.appsmith.server.services.LayoutServiceTest#getActionsExecuteOnLoadWithAstLogic

@rishabhrathod01 rishabhrathod01 changed the title chore: Add RTS metrics fix: Reduce RTS calls in the updateLayout flow Nov 1, 2024
@github-actions github-actions bot added Bug Something isn't working and removed skip-changelog Adding this label to a PR prevents it from being listed in the changelog labels Nov 1, 2024
Copy link

github-actions bot commented Nov 1, 2024

Failed server tests

  • com.appsmith.server.services.LayoutServiceTest#getActionsExecuteOnLoadWithAstLogic

Copy link

Failed server tests

  • com.appsmith.server.git.ServerSchemaMigrationEnforcerTest#saveGitRepo_ImportAndThenExport_diffOccurs

Copy link

Failed server tests

  • com.appsmith.server.services.LayoutServiceTest#getActionsExecuteOnLoadWithAstLogic

1 similar comment
Copy link

Failed server tests

  • com.appsmith.server.services.LayoutServiceTest#getActionsExecuteOnLoadWithAstLogic

Copy link

Failed server tests

  • com.appsmith.server.services.LayoutServiceTest#getActionsExecuteOnLoadWithAstLogic

Copy link

Failed server tests

  • com.appsmith.server.services.ce.ActionServiceCE_Test#testActionWithGraphQLDatasourceMoustacheBinding

Copy link

Failed server tests

  • com.appsmith.server.git.ServerSchemaMigrationEnforcerTest#saveGitRepo_ImportAndThenExport_diffOccurs

@rishabhrathod01 rishabhrathod01 marked this pull request as ready for review November 12, 2024 23:16
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Outside diff range and nitpick comments (4)
app/server/appsmith-interfaces/src/main/java/com/appsmith/external/constants/spans/ce/OnLoadSpanCE.java (1)

14-14: Consider adding documentation for monitoring points.

Adding Javadoc comments for these new span constants would help other developers understand:

  • The specific operations being monitored
  • Their role in the RTS call optimization
  • Expected impact on performance
+    /**
+     * Span for monitoring entity reference resolution operations.
+     * Added as part of RTS call optimization in the updateLayout flow.
+     */
     public static final String GET_POSSIBLE_ENTITY_REFERENCES = APPSMITH_SPAN_PREFIX + "getPossibleEntityReferences";

+    /**
+     * Span for monitoring entity parent relationship mapping operations.
+     * Used to optimize entity hierarchy traversal.
+     */
     public static final String GET_POSSIBLE_ENTITY_PARENTS_MAP = APPSMITH_SPAN_PREFIX + "getPossibleEntityParentsMap";

+    /**
+     * Span for monitoring dynamic binding reference resolution.
+     * Helps track and optimize binding-related RTS calls.
+     */
     public static final String GET_POSSIBLE_REFERENCES_FROM_DYNAMIC_BINDING =
             APPSMITH_SPAN_PREFIX + "getPossibleReferencesFromDynamicBinding";

Also applies to: 17-17, 23-24

app/server/appsmith-server/src/main/java/com/appsmith/server/services/ce/AstServiceCEImpl.java (1)

Line range hint 55-55: Review performance monitoring thresholds

Given that RTS calls currently take ~3s, the MAX_API_RESPONSE_TIME_IN_MS threshold of 50ms seems too aggressive. Additionally, using debug level for logging these performance issues might make them harder to track.

Consider these improvements:

  1. Adjust MAX_API_RESPONSE_TIME_IN_MS to a more realistic threshold based on current metrics
  2. Upgrade to log.warn for better visibility of performance issues
  3. Add metrics collection for these slow calls
-    private static final long MAX_API_RESPONSE_TIME_IN_MS = 50;
+    private static final long MAX_API_RESPONSE_TIME_IN_MS = 1000; // 1s threshold

     // In elapsed().map block:
-    log.debug("Time elapsed since AST refactor call: {} ms", tuple.getT1());
-    log.debug("This call took longer than expected. The binding was: {}", bindingValue);
+    log.warn("Time elapsed since AST refactor call: {} ms", tuple.getT1());
+    log.warn("This call took longer than expected. The binding was: {}", bindingValue);

Also applies to: 169-173

app/server/appsmith-interfaces/src/main/java/com/appsmith/external/helpers/MustacheHelper.java (1)

ObservationRegistry is declared but not utilized in MustacheHelper.java

  • The observationRegistry is initialized but not used in any methods within the file.
🔗 Analysis chain

Line range hint 39-79: Verify the integration of ObservationRegistry

The ObservationRegistry is added but not utilized in any of the methods. This seems incomplete and doesn't align with the PR objective of reducing RTS calls.

Let's check for other files that might be using this registry:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for ObservationRegistry usage in the codebase
rg -l "ObservationRegistry" --type java

# Search for similar observation patterns
ast-grep --pattern 'new Observation.Context($_, $_)'

Length of output: 6034

app/server/appsmith-server/src/main/java/com/appsmith/server/onload/internal/OnLoadExecutablesUtilCEImpl.java (1)

601-686: Avoid code duplication in overloaded methods

The overloaded getPossibleEntityReferencesMap methods share significant code. Refactoring to eliminate duplication will improve maintainability.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 8773bdd and 2b0981b.

📒 Files selected for processing (5)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/constants/spans/ce/OnLoadSpanCE.java (1 hunks)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/helpers/MustacheHelper.java (3 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/onload/internal/OnLoadExecutablesUtilCEImpl.java (7 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/services/AstServiceImpl.java (1 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/services/ce/AstServiceCEImpl.java (5 hunks)
🔇 Additional comments (7)
app/server/appsmith-server/src/main/java/com/appsmith/server/services/AstServiceImpl.java (2)

7-7: LGTM: Clean import addition

The ObservationRegistry import is properly placed with other framework imports.


15-20: LGTM: Constructor properly updated for observability

The constructor modification correctly integrates ObservationRegistry for monitoring capabilities. This aligns with the PR's goal of optimizing RTS calls.

Let's verify the superclass implementation:

✅ Verification successful

ObservationRegistry integration in superclass confirmed.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the superclass constructor and its usage of ObservationRegistry

# Check the superclass implementation
ast-grep --pattern 'class AstServiceCEImpl {
  $$$
  private final ObservationRegistry observationRegistry;
  $$$
  AstServiceCEImpl($$$) {
    $$$
  }
  $$$
}'

Length of output: 162


Script:

#!/bin/bash
# Description: Find the superclass of AstServiceImpl

rg 'class AstServiceImpl\s+extends\s+(\w+)' --only-matching --replace '$1' app/server/appsmith-server/src/main/java/com/appsmith/server/services/AstServiceImpl.java

Length of output: 183


Script:

#!/bin/bash
# Description: Check if ObservationRegistry is used in AstServiceCEImpl

rg 'ObservationRegistry' app/server/appsmith-server/src/main/java/com/appsmith/server/services/AstServiceCEImpl.java

Length of output: 248


Script:

#!/bin/bash
# Description: Locate AstServiceCEImpl.java in the repository

fd 'AstServiceCEImpl\.java' .

Length of output: 128


Script:

#!/bin/bash
# Description: Check if ObservationRegistry is used in the correct AstServiceCEImpl.java

rg 'ObservationRegistry' app/server/appsmith-server/src/main/java/com/appsmith/server/services/ce/AstServiceCEImpl.java

Length of output: 232

app/server/appsmith-interfaces/src/main/java/com/appsmith/external/constants/spans/ce/OnLoadSpanCE.java (1)

14-14: LGTM! Constants align with performance optimization goals.

The new span constants follow the established naming pattern and provide monitoring points for entity reference operations, which aligns with the PR's objective of reducing RTS calls.

Also applies to: 17-17, 23-24

app/server/appsmith-server/src/main/java/com/appsmith/server/services/ce/AstServiceCEImpl.java (2)

11-11: LGTM: Clean integration of monitoring capabilities

The addition of ObservationRegistry is well-structured and follows proper dependency injection patterns.

Also applies to: 20-20, 45-45


133-134: LGTM: Well-placed observation tap

The Micrometer observation tap is correctly positioned at the end of the reactive chain, enabling proper monitoring of RTS call performance.

app/server/appsmith-server/src/main/java/com/appsmith/server/onload/internal/OnLoadExecutablesUtilCEImpl.java (2)

675-676: ⚠️ Potential issue

Potential issue with map updates inside loops

Updating possibleEntitiesReferencesToBindingMap inside the inner loop may lead to incorrect mappings. Move the map update outside the inner loop to ensure correct aggregation.

Apply this diff to adjust the map update:

-        possibleEntitiesReferencesToBindingMap.put(binding, possibleEntitiesReferences);
     });
+    possibleEntitiesReferencesToBindingMap.put(binding, possibleEntitiesReferences);

Likely invalid or redundant comment.


513-518: ⚠️ Potential issue

Syntax error: Incomplete method declaration

The method getPossibleEntityReferences is missing its parameter list and body.

Apply this diff to fix the syntax error:

-protected Mono<Set<EntityDependencyNode>> getPossibleEntityReferences(
-
-    return getPossibleEntityReferences(executableNameToExecutableMapMono, bindings, evalVersion, null)
-            .name(GET_POSSIBLE_ENTITY_REFERENCES)
-            .tap(Micrometer.observation(observationRegistry));
+protected Mono<Set<EntityDependencyNode>> getPossibleEntityReferences(
+        Mono<Map<String, Executable>> executableNameToExecutableMapMono, Set<String> bindings, int evalVersion) {
+    return getPossibleEntityReferences(executableNameToExecutableMapMono, bindings, evalVersion, null)
+            .name(GET_POSSIBLE_ENTITY_REFERENCES)
+            .tap(Micrometer.observation(observationRegistry));
+}

Likely invalid or redundant comment.

Comment on lines +135 to 137

// TODO: add error handling scenario for when RTS is not accessible in fat container
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

TODO for RTS Accessibility Error Handling Not Addressed

The error handling scenario for RTS accessibility in AstServiceCEImpl.java remains unimplemented.

  • File: app/server/appsmith-server/src/main/java/com/appsmith/server/services/ce/AstServiceCEImpl.java
🔗 Analysis chain

Address the TODO comment regarding RTS accessibility

The TODO comment about error handling for RTS inaccessibility in fat container remains unaddressed. This could be critical for reliability.

Let's check if this scenario is handled elsewhere:

Would you like me to help implement the error handling for this scenario?

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for RTS accessibility handling patterns
rg -l "getIsRtsAccessible|RTS.*accessibility" --type java

Length of output: 575


Script:

#!/bin/bash
# Check for error handling related to RTS accessibility in AstServiceCEImpl.java
rg "RTS.*(error handling|exception)" app/server/appsmith-server/src/main/java/com/appsmith/server/services/ce/AstServiceCEImpl.java

Length of output: 133

Comment on lines +39 to +40
private static ObservationRegistry observationRegistry;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Static field initialization through instance constructor is an anti-pattern

The observationRegistry is declared as static but initialized through an instance constructor. This creates potential thread-safety issues and makes the initialization order unpredictable when multiple instances are created.

Consider this alternative implementation:

-private static ObservationRegistry observationRegistry;
+private final ObservationRegistry observationRegistry;

 public MustacheHelper(ObservationRegistry observationRegistry) {
-    this.observationRegistry = observationRegistry;
+    this.observationRegistry = observationRegistry;
 }

Also applies to: 77-79

Comment on lines +507 to +512
private Mono<Map<String, Set<EntityDependencyNode>>> getPossibleEntityReferencesMap(
Mono<Map<String, Executable>> executableNameToExecutableMapMono, Set<String> bindings, int evalVersion) {
return getPossibleEntityReferencesMap(executableNameToExecutableMapMono, bindings, evalVersion, null)
.name(GET_POSSIBLE_ENTITY_REFERENCES)
.tap(Micrometer.observation(observationRegistry));
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Refactor to reduce code duplication

The method getPossibleEntityReferencesMap duplicates logic from its overloaded version. Consider consolidating common code into a helper method.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working Integrations Pod General Issues related to the Integrations Pod that don't fit into other tags. Integrations Pod Integrations Product Issues related to a specific integration ok-to-test Required label for CI Query & JS Pod Issues related to the query & JS Pod Task A simple Todo
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Task]: Debug why RTS is taking ~3s per request
1 participant