Skip to content

Conversation

xxdydx
Copy link

@xxdydx xxdydx commented Apr 6, 2025

Description

Created a new feature to automate feedback generation of code submissions in Source Academy, with the help of LLMs. This will help save TAs some time in grading code submissions from students!

  • Added AI comment generation functionality in the backend.
  • Integrated OpenAI API for generating AI comments.
  • Implemented generate_ai_comments endpoint to fetch question details and generate AI-generated comments for submissions.
  • Added save_chosen_comments endpoint to save multiple chosen comments for a submission and question for logging purposes.
  • Added save_final_comment endpoint to save the final comment chosen for a submission for logging purposes.
  • Added new ai_comment_logs table to log various data points from inputs, original student's code, outputs generated by LLM, comments chosen, and final comment.
  • Added AIComments module to handle creation, retrieval, and updates for AI comments, including saving final and chosen comments.
  • Added AICodeAnalysisController to handle AI comment generation, saving final comments, and saving chosen comments.
  • Added test cases for generate_ai_comments, save_final_comment, and save_chosen_comments endpoints in AICodeAnalysisControllerTest.
  • Updated Swagger documentation for the new endpoints.
  • Added necessary migrations to update the database schema.
  • Added encryption and decryption logic for LLM API keys using AES-GCM.

Note: This may require changes to the DB diagram in README.md.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update
  • Code quality improvements

Checklist

  • I have tested this code
  • I have updated the documentation

@xxdydx xxdydx changed the title Feat/add ai generated comments grading AI-powered marking Apr 6, 2025
@xxdydx xxdydx self-assigned this Apr 6, 2025
@coveralls
Copy link

coveralls commented Apr 9, 2025

Coverage Status

coverage: 89.3% (-3.2%) from 92.469%
when pulling 8f8b93a on feat/add-AI-generated-comments-grading
into 009532a on master.

@xxdydx xxdydx marked this pull request as ready for review April 9, 2025 05:06
@xxdydx xxdydx requested a review from GabrielCWT April 9, 2025 05:06
Copy link
Contributor

@GabrielCWT GabrielCWT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this feature! Quite a bit of comments. Please look through, resolve them. I do have some clarification comments as well so please answer those as well.

One question I have is when are these comments used? I couldn't find anytime in which the comments are returned/retrieved to/by the FE.

@Tkaixiang Tkaixiang self-assigned this Sep 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants