Skip to content

Add ability to edit existing images in the media library#305

Draft
dkotter wants to merge 38 commits intoWordPress:developfrom
dkotter:feature/media-library-image-edits
Draft

Add ability to edit existing images in the media library#305
dkotter wants to merge 38 commits intoWordPress:developfrom
dkotter:feature/media-library-image-edits

Conversation

@dkotter
Copy link
Collaborator

@dkotter dkotter commented Mar 13, 2026

Important

This PR is branched from #292. Once that PR is merged, I'll open this for review.

What?

Closes #236, #237

Adds the ability to edit existing images in the media library views.

Why?

We already have the ability to generate images and we're adding the ability to refine those images in #292 but we don't have a way to refine/edit existing images. This PR adds that in via a prompt approach and with a few preset options (Remove background, Expand background).

How?

  • Inject our needed component into the WordPress Media Library edit screen when that is loaded (there are no hooks here to use unfortunately)
  • This component will render three preset buttons, Remove background, Expand background, and Refine Image, which will add a prompt textarea when clicked that allows you to make conversational edits to the image
  • When using this prompt, set the flow up to match our image refinement process used elsewhere. Allows you to refine the image and then import the image once done
  • When using a preset option, add built-in prompts that power those. For expanding background, we first build a new canvas that is 1.5x larger than the image. We add the image to the center of this and we send this to the LLM, allowing it to fill in the new blank space
  • Add better state management, both in this new flow and in the existing image refinement flows, so we keep track of each image generation and allow a user to paginate back and forth. This allows them to "go back in time" and import an older generated image or start refinement over from that point.
  • Keep state in place after importing so a user can import another image or continue to refine

Use of AI Tools

Used both Claude Code (running Opus 4.6) and Cursor (running GPT-5.3 Codex) to plan and execute, with final refinement and clean up done by me.

Testing Instructions

  1. Install the AI Provider for Google plugin
  2. Pull this PR down and run npm install && npm run build
  3. Add valid Google credentials
  4. Turn on the Image Generation experiment
  5. Ensure you have an existing image in the Media Library
  6. In the Media Library grid view, click on an image and then in the modal that shows, click the Edit Image button
  7. Ensure you see the new buttons. Click on Refine Image
  8. Add a prompt and click Generate
  9. Ensure an image is generated and that it shows side-by-side with the original image
  10. From here, test out the various buttons below the images, like refining further, running the generation again, starting over and saving the image
  11. Go back to the main edit screen and try out the other buttons
  12. Ensure those work as expected (background is removed and replaced with white or image is expanded)
  13. Run these same tests again but in the block editor. Add an image block and choose Media Library. Choose an image in the modal and click Edit image
  14. If desired, can also test the refinement flow after generating an image (either in the media library or in the editor) and ensure you can refine the image multiple times and paginate back and forth through those results

Screenshots

Image edit screen showing the new Expand Background, Remove Background and Refine Image buttons Showing the refine image textarea after the button is clicked Showing the generate loading state after refining an image Showing the first image refinement Showing the second image refinement Open WordPress Playground Preview

dkotter added 28 commits March 9, 2026 16:10
…n passed, add that as a reference image before making our image generation request
…mage for easier comparison. In that state, ensure refining further refines the refined image instead of the original. And ensure generating another image sends the original image along with the refinement prompt
… all styles into our stylesheet instead of inline. Minor adjustments to spacing and text
… as well as a prompt area for specific edits. Set this up to match the flow we use in our image generation components. Add necessary CSS and a helper function to turn an image URL into a base64 encoded image
…n flow. This allows you to refine an image multiple times, refining the new refined image each time. Ensure we keep track of all prompts used to store with the image. Update text to match our other image generation flows
…wn. Ensure we don't add extra spacing when our tools are hidden
…n to remove a certain color and replace with transparency that we can try out if we want transparent background instead of white background
…he requested image and then sending that along, so the LLM has known boundaries to expand to. Update our remove background prompt to work better with modern LLMs
…t happens, allowing users to paginate back and forth to those images, allowing them to import previously generated images or go back and start the refinment process from a different state
@dkotter dkotter added this to the 0.6.0 milestone Mar 13, 2026
@dkotter dkotter self-assigned this Mar 13, 2026
@codecov
Copy link

codecov bot commented Mar 13, 2026

Codecov Report

❌ Patch coverage is 83.33333% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 57.43%. Comparing base (71f393a) to head (0cf177d).

Files with missing lines Patch % Lines
includes/Abilities/Image/Generate_Image.php 82.14% 5 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             develop     #305      +/-   ##
=============================================
+ Coverage      57.05%   57.43%   +0.37%     
- Complexity       591      596       +5     
=============================================
  Files             42       42              
  Lines           3083     3101      +18     
=============================================
+ Hits            1759     1781      +22     
+ Misses          1324     1320       -4     
Flag Coverage Δ
unit 57.43% <83.33%> (+0.37%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jeffpaul
Copy link
Member

For some reason my Google credentials aren't working, so I can't fully test this (will continue to see what's going on there so I can test more fully, but have some thoughts for now). I would advocate for just removing the AI Exit button press requirements and just add the Expand Background (capital B) and Remove Background buttons on the row below the current Crop and other buttons.

Given that the prompt textarea and Generate button appear to be tied to a "refine" flow, let's perhaps add a Refine Image button that adds the helper text, textarea, and button (perhaps those appear below the image so it doesn't jump the image downward when those are added in. Let's also make the DESCRIBE... text sentence case.

Also for the Expand and Remove and Refine buttons let's ensure the base image is still present, as for now clicking AI Edit replaces the image and renders buttons but would be ideal to keep the image and add in the buttons.

dkotter added 3 commits March 18, 2026 15:40
…. Hide the prompt textarea and add a new Refine Image button that when clicked will show that. Minor text and style updates
@dkotter
Copy link
Collaborator Author

dkotter commented Mar 18, 2026

I would advocate for just removing the AI Exit button press requirements and just add the Expand Background (capital B) and Remove Background buttons on the row below the current Crop and other buttons.

This is done now. Removed the AI Edit button and instead we show the other buttons by default (and capitalized the second word). I've updated screenshots in the PR description showing the new UI.

Given that the prompt textarea and Generate button appear to be tied to a "refine" flow, let's perhaps add a Refine Image button that adds the helper text, textarea, and button (perhaps those appear below the image so it doesn't jump the image downward when those are added in). Let's also make the DESCRIBE... text sentence case.

Added a Refine Image button and clicking that now shows the prompt flow. The text above the textarea is now sentence case instead of ALL UPPERCASE, though noting that style was added by Core so we are overriding that here now.

For now I haven't added the textarea below the image as it's pretty common for an image to be larger than the screen height and so you'd have to scroll down to see those options. So easy to miss those were added in that scenario.

Personally I think this conversational image editing is the most useful thing here (comparing to removing background or expanding the image). My make sense to have that be the first button, at the moment I've left that as the last.

Also for the Expand and Remove and Refine buttons let's ensure the base image is still present, as for now clicking AI Edit replaces the image and renders buttons but would be ideal to keep the image and add in the buttons.

I've left the original image while generation is in place. Once the image is generated, the original image hides as we show that side-by-side with the generated image.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add generative image edit actions (inpaint and outpaint)

2 participants