Skip to content

Don't list users twice #128

Description

@rossjrw

All users are currently iterated twice - once at the start to get everyone who might need notifying:

notifier/notifier/notify.py

Lines 144 to 151 in 5fc4fe5

activation_log_dump.update({"config_start_timestamp": timestamp()})
if dry_run:
logger.info("Dry run: skipping remote config acquisition")
else:
logger.info("Getting remote config...")
get_global_config(config, database, wikidot)
logger.info("Getting user config...")
get_user_config(config, database, wikidot)

and once at the end, during cleanup, to check for pages that have been moved to new urls, as a way of automoderation:

def rename_invalid_user_config_pages(
local_config: LocalConfig, wikidot: Wikidot
) -> None:
"""Prepares invalid user config pages for deletion."""
logger.info("Finding invalid user configs to prepare for deletion")
# Get all user configs and filter out any that are valid
invalid_configs = [
(slug, config)
for slug, config in fetch_user_configs(local_config, wikidot)

These two steps are separated by like 10 minutes at most. The second fetch just wastes time to get data that's only slightly fresher. These fetches are particularly slow, having to iterate through all pages of the users ListPages query.

The data from the first fetch should be retained and used in the rename check instead of calling the listpages endpoint again.

Because the risk of acting on old data would therefore be greater, before any proposed automoderation action is taken, the current state of the page should be re-evaluated to make sure that it's still the case.

Users who become 'automoderatable' during the 10 minute window will be missed, but simple detected on the next run.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    optimisationMake an existing feature faster or smaller

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions