Implement Refine from Notes batch processing and progress indicator

[zeus2611]

What?

Closes #250
See #251

This PR introduces the new "Refine from Notes" feature, enabling users to automatically apply pending editorial feedback/notes to their content using AI.

Why?

Feature request #250 proposed allowing users to generate automated refinement updates based heavily on unresolved editorial notes attached to blocks. This enables a smoother editing workflow where suggestions can be adopted contextually instead of applied manually one-by-one.

We matched the batch UX already provided by "Review Notes" and supply users with a safe, native diff interface via WordPress Revisions to examine and accept/rollback any generated AI changes before publishing.

How?

• Created useRefineNotes hook to loop comprehensively through the document blocks, matching pending and threaded ExistingNote collections to associated parent clientIds.
• Utilized an asynchronous batch progression of 4 block items per network request.
• Published surface-level progress data to a new RefineNotesPlugin.tsx component in the sidebar to dynamically step the button text (e.g. Refining block (1 of 7)...).
• Injected an autosave() trigger to establish a clean DB checkpoint after processing.
• Exposed a "Review in Revisions" action to the success snackbar notification linking directly to revision.php?revision=XXX.

Use of AI Tools

Implementation was planned and executed with assistance from Gemini (Antigravity). All code was reviewed, tested, and validated manually in a local WordPress environment.

Testing Instructions

1. Run and build the plugin locally on WP. Ensure AI experiments are enabled.
2. Create a Post and generate some blocks.
3. Attach multiple Editorial Notes/Comments across various blocks suggesting changes (e.g. "Fix capitalization", "Make this sound professional").
4. Click the "Refine from Notes" button from the main Plugin Sidebar.
5. Verify it updates the button text to show progression through the blocks.
6. Check that the snackbar notification appears with a "Review in Revisions" action upon completion.
7. Click "Review in Revisions" and verify the WP Core diff viewer accurately shows the before & after changes.

Testing Instructions for Keyboard

1. Navigate via Tab to the Plugin Sidebar -> Refine from Notes button.
2. Hit Enter to start the action.
3. Once completed, Tab over to the auto-focused snackbar notification to click "Review in Revisions".

Screenshots or screencast

[jeffpaul]

@zeus2611 I see this is still in draft, but PR description seems like it might be ready for review/testing; mind confirming if this is ready or still in progress?

[codecov]

Codecov Report

❌ Patch coverage is 75.34247% with 36 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.62%. Comparing base (32c5e2d) to head (cbfadf2).

Files with missing lines	Patch %	Lines
includes/Experiments/Refine_Notes/Refine_Notes.php	31.25%	22 Missing ⚠️
includes/Abilities/Refine_Notes/Refine_Notes.php	88.39%	13 Missing ⚠️
...udes/Abilities/Refine_Notes/system-instruction.php	50.00%	1 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##             develop     #289      +/-   ##
=============================================
+ Coverage      57.85%   58.62%   +0.77%     
- Complexity       615      646      +31     
=============================================
  Files             46       49       +3     
  Lines           3165     3311     +146     
=============================================
+ Hits            1831     1941     +110     
- Misses          1334     1370      +36     

Flag	Coverage Δ
unit	58.62% <75.34%> (+0.77%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[zeus2611]

Hey @jeffpaul — the core feature is complete and working as expected. I'm currently working on improving the e2e test coverage before marking this ready for review. Should have that wrapped up shortly and will convert from draft then!

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: zeus2611 <n1schay@git.wordpress.org>
Co-authored-by: dkotter <dkotter@git.wordpress.org>
Co-authored-by: jeffpaul <jeffpaul@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[zeus2611]

Hey @dkotter — the implementation is complete and ready for review. Marking this as ready and removing the Draft status.

[jeffpaul]

@zeus2611 one thing that I think would be a great extension of this in a future, follow-up, PR is leveraging Real Time Collab and having a new WordPress AI (or just AI) user on the site and showing it within the editor interface making these changes. I'll open an issue to document that, but will otherwise review/test this issue without that in mind.

[zeus2611]

Sounds great, looking forward to the scope of the feature.

One thing I've been thinking about while building this: the individual features we're shipping (Review Notes, Refine from Notes, etc.) are really just discrete steps in a larger editorial workflow. There's an opportunity to tie these together as an agentic flow — e.g., a single "Review & Refine" pipeline where the AI first surfaces annotations as editorial notes and then automatically applies accepted suggestions in one pass, rather than requiring two separate manual actions.

I think this connects well to #282 — if the Chat Workspace is going to be the home for multi-step AI interactions, these editor-native workflows could be surfaced there as named agent actions ("Refine my draft", "Run editorial review"), giving users a coherent experience whether they're acting from the sidebar or the chat interface. Worth keeping in mind as the #282 design takes shape.

[jeffpaul]

There's an opportunity to tie these together as an agentic flow — e.g., a single "Review & Refine" pipeline where the AI first surfaces annotations as editorial notes and then automatically applies accepted suggestions in one pass, rather than requiring two separate manual actions.

I'm not certain I want to design a YOLO-mode into this by default, I'd prefer a human step in between the Review and Refine steps where an author or editor could perhaps resolve certain Review Notes that they don't want actioned and then have the AI continue on with the Refine steps.

[dkotter]

Haven't fully tested things yet, overall looking good but have a handful of things I think we need addressed, though let me know if there's any questions on these.

[dkotter]

We'll want to update all of these @since statements to be x.x.x so we can replace those when doing a release (as it may not be in the 0.5.0 release)

[zeus2611]

Updated all @since tags to x.x.x across both PHP files.

[dkotter]

Looks like the test files have these as well so those will need updated

[zeus2611]

Updated both Refine Notes test files.

[dkotter]

Is this needed with our E2E plugin we add? That plugin intercepts all HTTP requests and allows us to return canned responses

[zeus2611]

I added this because the tests were failing locally with prompt_builder_error: No models found that support text_generation for this prompt — the PHP AI client fails before any HTTP call is dispatched, so the pre_http_request mock never fires. I also saw one CI E2E run fail for the same test without the page.route(). The page.route() intercept at the browser network level bypasses the PHP client entirely.

If the E2E plugin setup ensures valid credentials are configured during CI runs, I can update this accordingly. What's the preferred approach?

[dkotter]

So we don't make any valid API requests in our E2E tests. We capture any HTTP request and mock the responses (including the model responses)

PHP AI client fails before any HTTP call is dispatched

I'd be curious on why this is? The client should be making requests to get all models from each configured provider and we do capture that request and return a mocked model response which should mean you never get that error message. But maybe something else is going on here that I'm not aware of?

If the E2E plugin setup ensures valid credentials are configured during CI runs, I can update this accordingly. What's the preferred approach?

So none of our other E2E tests have this page.route() setup in place and they all work fine. I'm not opposed to doing this just seems like it's more complicated than it needs to be

[zeus2611]

Removed the ability endpoint intercept — the PHP mocking plugin handles the downstream OpenAI call. Kept the /wp/v2/comments intercepts since those are browser→WordPress REST calls, not server→external HTTP, so pre_http_request doesn't cover them.

[jeffpaul]

I'm not certain that the flow through the "Review in Revisions" toast message and then historical diff view for Revisions is the most optimal way to get folks through this feature. I'd argue that immediately applying these updates to the post and allowing a user to leverage revisions to roll back any changes they don't like would be better. Alternatively presenting each change for editor/author review where accepting applies the change and resolves the respective Note and rejecting the change skips the change and leaves the respective Note?

[zeus2611]

The current implementation already applies changes directly to the post — the toast is just a convenience link. We could improve this by redirecting users straight to the revisions view after refinement completes, which would pair nicely with 7.0's block-aware visual diffing.

True per-block rollback isn't feasible yet since revisions are still whole-post snapshots — reverting one block's refinement would undo all others from that pass.

The accept/reject per-change approach would give the most granular control but requires a custom inline diff review UI. For this PR, would keeping "apply all + redirect to revisions" as the MVP and opening a follow-up issue for per-block accept/reject work?

[jeffpaul]

The current implementation already applies changes directly to the post — the toast is just a convenience link. We could improve this by redirecting users straight to the revisions view after refinement completes, which would pair nicely with 7.0's block-aware visual diffing.

Ahh ok, yeah let's try linking to the visual revisions view instead of the text-based diff revisions view.

True per-block rollback isn't feasible yet since revisions are still whole-post snapshots — reverting one block's refinement would undo all others from that pass.

For now that's probably fine. I could see a state where each resolved Note gets a comment from "WordPress AI" noting the refinement is done and links to a visual revision reference to see that specific change (which would be we'd need to fire an auto-save after each individual Note refinement).

The accept/reject per-change approach would give the most granular control but requires a custom inline diff review UI. For this PR, would keeping "apply all + redirect to revisions" as the MVP and opening a follow-up issue for per-block accept/reject work?

Yeah, that approach for this version totally makes sense. I'll try to collect all the iterations noted in this PR and open a follow-up issue.

[jeffpaul]

Works well in my testing, so @dkotter feel free to merge when you're good from your testing/review