Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(fhir): don’t materialise anything if the worker is disabled #7123

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

passcod
Copy link
Member

@passcod passcod commented Jan 28, 2025

Changes

Not certain this is a good idea or if this should be done a different way, but currently when fhir.worker.enabled is false, then materialisation jobs are still inserted into the queue, which means the job queue grows infinitely (which triggers alerts, and caused Samoa going down).

Copy link
Contributor

@NavarroEmilioLuis NavarroEmilioLuis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well it kinda makes sense. The downside would be that when we turn it on, we will start having a big queue but I mean, that's expected.

I think I'd be good to have yet a third pair of eyes to confirm before approving? Probably Rohan's?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@passcod I just wanted to double check if this would prevent new jobs from being added to the queue?
There's a fhir.refresh_trigger function which is used as a trigger on any of the tables that affect fhir resources. That function creates a job whenever a record in the table is updated, and I believe it's independent of this code.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ugh, yeah. should we have some kind of setting that the trigger checks?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm yeah maybe. The integrations.fhir.enabled setting in the config does seem like it should the primary on/off switch for fhir stuff. But reading that from the database doesn't work (and even if it was a db setting, might feel a bit awkward to check in a function).

Thoughts on maybe adding a FhirJobQueueCleaner scheduled task or something? So we just create the jobs and separately clean them up. Does feel like extra complexity...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when I originally designed settings I actually added a couple sql functions specifically to do this kinda thing (read from settings in triggers and reports). not sure they're still there?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about a check on app start that checks whether the setting is enabled or disabled, and then turns the trigger on or off to match?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I was thinking along the same lines after seeing chris's workaround! bonus, it would make reverting that unnecessary

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we happy to merge this PR as is and I'll make a ticket for the solution proposed by edwin to work on separate?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes please 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants