Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perf: Refactor to check user role in memory not db #4846

Closed
wants to merge 5,842 commits into from

Conversation

coalest
Copy link
Collaborator

@coalest coalest commented Dec 11, 2024

Doesn't resolve any issue

Description

Since we have multiple current_user.has_role? calls in many controllers/views, let's use Rolify's has_cached_role? instead as this performs the authorization check with user's roles in memory rather than going to the database every single time.

The difference is that has_role? performs a query everytime, while has_cached_role? loads the roles on the user object and performs the check without any queries.

The documentation for this method has a warning:

This method should be used with caution. If you don't preload the roles, the has_cached_role? might return false. In the above example, it would return false for @user.has_cached_role?(:member, Forum), because User.with_role(:admin, Forum) will load only the :admin roles.

I was not able to replicate that behavior personally (like mentioned in this issue). Also with this change we are fetching all roles right after the user is set regardless.

I know that "caching" authorization checks sounds like a bad idea. But it is only "cached" for the length of the request, so I think it's fine. And it would help with performance across the app.

An example is the distributions index page, which went from 6 database queries regarding roles down to 2 (one loading all the roles and one when current_role is called to fetch the first role).

If this approach is rejected, I would propose we set up our own way of caching/storing these rolify checks in views. Because some pages trigger a lot of these queries.

Type of change

  • Refactor (performance)

How Has This Been Tested?

Locally and by running the test suite.

Aaryanpal and others added 30 commits September 14, 2024 18:39
…from_request

4604 - Add Total Quantity header in Distribution Request
* chore(model/purchase): add comment_view

* feat(view/purchases/show): show comment on purchase details

* chore(model/purchase): remove comment_view

* fix(purchases/show): indent html

* chore(purchases_requests_rspec): use expected value on test

* fix(purchases_requests_rspec): use freeze_time on show purchase info test
USER_GUIDE]: Add donation sites user guide
* Add the Delete button on the Partner Group Page

* Replace Destroyed with Deleted

* Resolve Comments

* Added Request Spec for PartnerGroup and remove corresponding system-spec

* Fix lint

* fix the request-spec
…ail-issues

4486: address pickup email issues
coalest and others added 16 commits December 5, 2024 13:42
Fix transfer InventoryError during db seed
* Make purchase helper pick default storage location

Changed the purchase helper to use the default storage location
of the organization first, before using any other storage locations
that are available.

Also added a (failing) spec to check that we can see the option
on the page. Should be fixed in the next commit

* Fix test by using extended format and `subject`

Fixed the failing test by using the extended format
of the test expectation and used `subject` to make
an assertion regarding the response body.

* Add test that new donation has default storage location

Added a test that when we begin to make a new donation,
we have a default storage location set.

* Switch to use helper to set default storage location

Switched to use the helper method from PurchasesHelper
to set the default storage location for a donation

* Address review feedback

Addressed review feedback namely:
* Renaming new_purchase_default_location to set_default_location
and moving it to ApplicationHelper. This allows us to use across
both the donation and purchase forms.
* Updated the tests and forms accordingly
* Added specs to ApplicationHelper to test that we are able
to successfully set the default storage location for purchases
and donations.

* Rename method, have only one test

Renamed the method as per review suggestion
and keep only one test since we only need to test
that the default storage location is successfully set

* Update set_default_location to default_location

* Update method references in view code

* Update spec to match new behaviour

Updated spec to match the new behaviour of
the donation form.
* REFACTOR clean up unused line item trait introduced in #4163

* FIX typo in params in donation and purchase controllers

* Another typo in docs

* DOCS add comment clarifying itemizable works with line items

* REFACTOR rename Item:kits scope to :housing_a_kit, add rspecs

* Clearer name
* Add rspecs to test :housing_a_kit and :loose scopes

* REFACTOR DRY up kit base_item, seed_base_items, prevent kit base_item deletion

* Move seed_base_items code into one static function
* Move kit base item creation code into one static function
* Add code to prevent calling destroy on kit base item and corresponding rspec
* Added comments - not sure about whether other base item request specs are useful or what the purpose of destroy is in the controller if it can't be called

* Add rspec for item.is_in_kit?

* Fix lint

* rename findorcreatebaseitem to add !

* REFACTOR remove unused code childrenservedreportservice

* All SQL code is duplicated in AcquisitionReportService
* RSpec is close enough to what is in AcquisitionReportService Spec that it can be removed without merging in
  (only difference is Diapers - Adult Briefs category is not created, but that shouldn't matter
  because the SQL looks for %diaper% so this category isn't testing anything different)

* Fix bin/setup so it's working with seed_base_item move

* Base Item linting

* Rename to snake case
* Add docs noting base items will be changed

* REFACTOR move seed_base_items into lib util file

* REFACTOR use specify -> instead of it -> in test descriptions in item_spec.rb

* FIX typos, move to Seeds module

* FIX clarify org admins can't view or edit base items in specs

* update bundler version

* remove extra base item route rspecs

---------

Co-authored-by: CL Fisher <[email protected]>
… location (Fixes #4647) (#4804)

* fix item minimum error to consider quantity across organization

* display correct alerts when distribution causes distinct items to dip below recommended and minimum levels

* refine recommended quantity error to occur based on org quantity, not storage location quantity

* fix distribution system specs that failed due to updated error message content

* display minimum and recommended amount error messages on separate lines

* refactor inventory check alert messages and related spec

* fix lint errors
…ce (#4590) (#4806)

* Improve distribution filtering and refactor for performance (#4590)

* Minor refactor

* Refactor with service object

* Return total distribion value not value of filtered items

* Fix default date range bug

* Fix inactive_items N+1

* Fix regression in item category filter
* Adds type enum to request model.

* Adds request type enum to request model.

* Adds request_type to request.

* Adds request_type to request.

* Passes request_type to FamilyRequestCreateService.new

* Passes request_type to FamilyRequestCreateService.new

* Passes request_type to RequestCreateService.new

* Passes request_type to Request.new.

* Adds hidden field in order to pass request_type param on submit.

* Adds request type flag to index and show pages as well as the requests export. Adds filter by request type to index.

* Adds by_request_type scope for filtering by request type.

* Adds request type flag to index and show pages as well as the requests export. Adds filter by request type to index.

* Adds by_request_type filter.

* Updates flag names and removes flag styles based on feedback from ruby for good.

* Adds test for request_type enum.

* Removes unnecessary line.

* Adds Type to expected_headers.

* Adds request_type traits to request factory.

* Adds request_type to family_request_create_service_spec args.

* Adds request_type to args.

* Pass request_type directly.

* Remove request_type partial and enter text directly into reqest_row

* Revert changes to database.yml

* Revert changes to request_spec.rb

* Remove request_type

* Remove hidden request_type fields.

* Change request_type data_type to string, replace request_type string in view to display only the first letter, pull request_type from service instead of defining with an instance var.

* Remove request_type from tests

* Add request_type_label method

* Undo changes to versions

* Add request_type_label method

* Add request_type to initialize

* Fix request_type bug

* Fix linting

* Fix request_type_label method model spec

* Remove unused for_families attribute from RequestCreateService

* Remove leftover print debugging statement

* Fix/add specs for new request_type attribute

* Revert submit button color change for family requests

* Hover over request type abbreviation to see full text

* Show full request type when viewing individual request

* Fix linting

---------

Co-authored-by: nathan <[email protected]>
Co-authored-by: Nathan Thomas <[email protected]>
…ails (#4829)

* clean up social media, request types and pick up emails

* suggested changes
Bumps [bullet](https://github.com/flyerhzm/bullet) from 7.2.0 to 8.0.0.
- [Changelog](https://github.com/flyerhzm/bullet/blob/main/CHANGELOG.md)
- [Commits](flyerhzm/bullet@7.2.0...8.0.0)

---
updated-dependencies:
- dependency-name: bullet
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Daniel Orner <[email protected]>
Bumps [flipper-active_record](https://github.com/flippercloud/flipper) from 1.3.0 to 1.3.2.
- [Release notes](https://github.com/flippercloud/flipper/releases)
- [Changelog](https://github.com/flippercloud/flipper/blob/main/Changelog.md)
- [Commits](flippercloud/flipper@v1.3.0...v1.3.2)

---
updated-dependencies:
- dependency-name: flipper-active_record
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Daniel Orner <[email protected]>
Bumps [coverband](https://github.com/danmayer/coverband) from 6.1.2 to 6.1.4.
- [Release notes](https://github.com/danmayer/coverband/releases)
- [Changelog](https://github.com/danmayer/coverband/blob/main/changes.md)
- [Commits](danmayer/coverband@v6.1.2...v6.1.4)

---
updated-dependencies:
- dependency-name: coverband
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Daniel Orner <[email protected]>
Bumps [newrelic_rpm](https://github.com/newrelic/newrelic-ruby-agent) from 9.13.0 to 9.16.0.
- [Release notes](https://github.com/newrelic/newrelic-ruby-agent/releases)
- [Changelog](https://github.com/newrelic/newrelic-ruby-agent/blob/dev/CHANGELOG.md)
- [Commits](newrelic/newrelic-ruby-agent@9.13.0...9.16.0)

---
updated-dependencies:
- dependency-name: newrelic_rpm
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Daniel Orner <[email protected]>
…#4836)

* Show all active partners on distributions page unrestricted by filter

* Move spec to make PR diff more legible
@coalest coalest requested a review from dorner December 11, 2024 16:36
@coalest coalest force-pushed the reduce-rolify-db-calls branch from 2c50866 to e4349f9 Compare December 11, 2024 16:59
@coalest
Copy link
Collaborator Author

coalest commented Dec 11, 2024

I can hide this change behind a feature flag if we want to extra careful.

@cielf
Copy link
Collaborator

cielf commented Dec 11, 2024

Re: feature flag. We mostly use feature flags for multi-stage changes (like the current 'packs' milestone, or things we would initially release to limited users for beta.

I feel that this is a change that either we are very confident in or we don't put forward to production at all.

It would provide rollback-ability, of course.

@dorner
Copy link
Collaborator

dorner commented Dec 13, 2024

I'm OK with this change. All tests are passing and from what I can read, there don't seem to be any reproductions of the original claim of flakiness (which was added when the feature first was, almost 10 years ago.

I'm good with merging and we can see if anyone complains of issues. @cielf thoughts?

@cielf
Copy link
Collaborator

cielf commented Dec 13, 2024

A question: If this were to go awry, do we know what the expression would be? Banks don't always complain.

and

My knee-jerk response is that I'm a bit iffy about putting it in so close to high new-banks-who-are-using-the-system-for-real season, just because of the higher support load.

But/and my knee-jerk response has always been a bit over-cautious.

@coalest
Copy link
Collaborator Author

coalest commented Dec 13, 2024

Hard to say how it would fail, because it doesn't look like it should.

But if it did fail, it would be bad because it touches both authorization and authentication. So potentially users getting signed out, or not being able to view pages they should have access to. I imagine it would be noticeable in logs though. I would expect a higher number of 302 responses (redirecting users to different pages if they don't have access or redirecting to the sign in page if they don't pass authentication).

So I understand the cautiousness. We can either wait till later or put it behind a feature flag so that we could quickly turn it off.

@cielf
Copy link
Collaborator

cielf commented Dec 14, 2024

Hmm... so this might interact with #4511.

@coalest coalest force-pushed the reduce-rolify-db-calls branch from e4349f9 to 4af094a Compare December 21, 2024 15:44
@cielf
Copy link
Collaborator

cielf commented Dec 21, 2024

What's with the gazillion commits on this one too?

@coalest coalest closed this by deleting the head repository Dec 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.