V15.2 Load Balanced Content Delivery Api Caching Issue

[a-moseley]

Which Umbraco version are you using? (Please write the exact version, example: 10.1.0)

15.2.2

Bug summary

When updating a block grid block the updated value is not returned by the content delivery api in a load balanced environment. All brand new or removed blocks seem to correctly get returned in the content delivery api

Specifics

We started the project as a umbraco 13 project and did not have these issues until we upgrade to 15.2 but as I understand the caching has also changed since.

We are running the application in an AWS EC2 instance and have the setup as our main set as SchedulingPublisher and every other instance is a subscriber by default as per https://docs.umbraco.com/umbraco-cms/fundamentals/setup/server-setup/load-balancing
Additionally all our instances have the umbracoApplicationUrl set to the correct domain as well.

We noticed the issue with caching at first when we upgraded most recently from 15 to 15.2 as we needed all the fixes to route issues with the content delivery api, though truthfully the cache issues may have been before that.

In an attempt to test and fix this locally I setup a new docker setup with two instances, a publisher and subscriber and was able to get the same problem we are seeing on our deployed sites.

Trying a variety of avenues I went through the docs and setup a valkey cache distribute cache, this did not fix the issue. I ensured the correct permissions were provided for the files and folders as per https://docs.umbraco.com/umbraco-cms/fundamentals/setup/server-setup/permissions, this at least allows the logs and data to correctly update but still nothing.

I also setup a sql profiler to see what the subscriber was polling umbracoCacheInstruction and I could see it was updating the lastsynced.txt in the TEMP DistCache to the latest entry in said db table, but again this did not seem to be correctly working.

Restarting the subscriber container does update/clear the cache it seems.

At this point I feel i've exhausted what options exist via the documentation that could explain these issues and started to think this is a bug, hence this raised issue.

Steps to reproduce

1. Create a brand new umbraco 15.2.2 project
2. Follow the details laid out in the documentation for load balancing (https://docs.umbraco.com/umbraco-cms/fundamentals/setup/server-setup/load-balancing), we deploy in a load balanced environment in AWS but I was able to recreate this same issue locally in docker by having two instances, one set as the publisher on localhost:5011 and one as the subscriber at localhost:5012 both using the same sql db, both having UmbracoApplicationUrl set as localhost:5011.
3. Create a basic block grid data type, block grid content page, and a couple of block document types to use in that block grid (we have a variety but a simple headline with a textstring input and rich text with a rich text editor should suffice)
4. Create a quick content page using that block grid and add a few different blocks.
5. Call the endpoint http://localhost:5012/umbraco/delivery/api/v2/content/item/:path with the path being the content page we created (though its worth noting it appears to be on all the endpoints for content delivery api)
6. The first call should return all the details correctly.
7. Change the values of the headline text, the rich text etc. Additionally add a brand new headline/rich text. Save & Publish
8. Call the endpoint again
9. Now you should see the text changes to headline and rich text are not passed through yet the brand new block is. Additionally removing a block will correctly show in the content delivery api.

Expected result / actual result

Expected Result

When I modify a content block in a block grid, save it (publish or not) it correctly updates what is returned from the content delivery api in a load balanced environment.

Acutal Result

When I modify a content block in a block grid, save it (publish or not) it does not send the update values of modified blocks.

[github-actions]

Hi there @a-moseley!

Firstly, a big thank you for raising this issue. Every piece of feedback we receive helps us to make Umbraco better.

We really appreciate your patience while we wait for our team to have a look at this but we wanted to let you know that we see this and share with you the plan for what comes next.

• We'll assess whether this issue relates to something that has already been fixed in a later version of the release that it has been raised for.
• If it's a bug, is it related to a release that we are actively supporting or is it related to a release that's in the end-of-life or security-only phase?
• We'll replicate the issue to ensure that the problem is as described.
• We'll decide whether the behavior is an issue or if the behavior is intended.

We wish we could work with everyone directly and assess your issue immediately but we're in the fortunate position of having lots of contributions to work with and only a few humans who are able to do it. We are making progress though and in the meantime, we will keep you in the loop and let you know when we have any questions.

Thanks, from your friendly Umbraco GitHub bot 🤖 🙂