Skip to content

Filter BOT/AI Pull Request Reviews #635

Closed

Description

@samad-yar-khan

Why do we need this ?

  • We currently include Bot/AI reviewers while considering first response time. This decreases the actual First Response time of the PR due to instant bots reveiws.

Acceptance Criteria

  • Identify Reviews by AI/Bots
  • While calculating Pull Request Cycle Time metrics (first response time and overall cycle time) exclude Bot/AI reviews.

Further Comments / References

Activity

  1. changed the title [-]Filter BOT/AI PUll Request Reviews[/-] [+]Filter BOT/AI Pull Request Reviews[/+] on Mar 17, 2025
  2. thesujai commented on Mar 18, 2025

    @thesujai
    Contributor

    Hi @samad-yar-khan Hope you are well:)
    For identifying bots, a new field called actor_type (which indicates either "bot" or "user") should be added to the PullRequestEvent model.
    How will we know whether the review is from a bot or not?
    For GitHub, we can leverage the github.PullRequest.get_reviews() API, where each review's user object also includes a type indicating whether it is a bot or a user. So, just after this line, we will retrieve the actor type as:

    actor_type=actor.get("type", "")
    

    Then, we add it to the PullRequestEvent.

    I'm not sure how GitLab processes this, but the second part will be the same for both platforms:

    The second part is straightforward: simply filter out bot events before any operation in this method. At the beginning of the function, the code will look like:

    pr_events = [
                e
                for e in pr_events
                if not e.actor_type.lower() == "bot" 
            ]
    
  3. Ayyanaruto commented on Apr 2, 2025

    @Ayyanaruto
    Contributor

    Can I work on this issue

  4. samad-yar-khan commented on Apr 6, 2025

    @samad-yar-khan
    ContributorAuthor

    @Ayyanaruto sure, give it a go. Lets discuss how you are planning to solve this ?

  5. Ayyanaruto commented on Apr 8, 2025

    @Ayyanaruto
    Contributor

    Hi @samad-yar-khan sorry for late reply as per my deduction the cycle time is calculated here backend/analytics_server/mhq/service/code/sync/etl_code_analytics.py In the Class etl_code_analytics I will check in the api response includes bot in username or if type is bot or author_association=BOT and filter out if present

  6. Ayyanaruto commented on Apr 13, 2025

    @Ayyanaruto
    Contributor

    Hi @samad-yar-khan,
    I’ve created the PR for this issue. Sorry about the earlier PR — I had pushed the branch with the wrong logic and wasn’t aware of it. All the tests were failing, and I got a bit confused.
    I’ve closed that PR and created a fresh one. When you’re free, please take a look!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestmediumModerate complexity issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions