🌟 The must-have tool for developers in the AI-Gen era 🌟
Keploy-gen uses LLMs to understand code semantics and generates meaningful unit tests. It's inspired by the Automated Unit Test Improvement using LLM at Meta.
-
Automate unit test generation (UTG): Quickly generate comprehensive unit tests and reduce the redundant manual effort.
-
Improve edge cases: Extend and improve the scope of tests to cover more complex scenarios that are often missed manually.
-
Boost test coverage: As codebase grows, ensuring exhaustive coverage should become feasible.
| Phase | Activities | Tools/Technologies |
|---|---|---|
| Code Analysis | Analyze the code structure and dependencies. | Static analysis tools, LLMs |
| Prompt Engineering | Generation of targeted prompts to guide the LLM in producing relevant tests. | LLMs, Custom scripts |
| Iterative Test Refinement | Cyclic process of refining tests by running them, analyzing coverage, and incorporating feedback. | Testing frameworks (e.g., JUnit, pytest) |
Referred from Meta's research, TestGen-LLM top level architecture.
AI model Setup - Set the environment variable API_KEY.
export API_KEY=xxxx
API_KEY can be from either of one these:
-
OpenAI's GPT-4o directly [preferred].
-
Alternative LLMs via litellm.
-
Azure OpenAI
Install Keploy locally by running the following command:
curl --silent -O -L https://keploy.io/install.sh && source install.sh- Download and move the keploy.exe file to
C:\Windows\System32
-
Ensure you've set the API key, as mentioned in pre-requisites above:
export API_KEY=xxxx -
Ensure Cobertura formatted coverage reports, edit
jest.config.jsorpackage.json:// package.json "jest": { "coverageReporters": ["text", "cobertura"], }
or
// jest.config.js module.exports = { coverageReporters: ["text", "cobertura"], };
-
Run the following command in the root of your application.
-
For Single Test File: If you prefer to test a smaller section of your application or to control costs, consider generating tests for a single source and its corresponding test file:
keploy gen --sourceFilePath="<path to source file>" --testFilePath="<path to test file for above source file>" --testCommand="npm test" --coverageReportPath="<path to coverage.xml>"
-
For Entire Application use the following command to generate tests across:
⚠️ Warning: Executing this command will generate unit tests for all files in the application. Depending on the size of the codebase, this process may take between 20 minutes to an hour and will incur costs related to LLM usage.keploy gen --testCommand="npm test" --testDir="test" --coverageReportPath="<path to coverage.xml>"
🎉 You should see improved test cases and code-coverage. ✅ Enjoy coding with enhanced unit test coverage! 🫰
-
-
Ensure you've set the API key, as mentioned in pre-requisites above:
export API_KEY=xxxx -
To ensure Cobertura formatted coverage reports, add:
go install github.com/axw/gocov/[email protected] go install github.com/AlekSi/[email protected]
-
Run the following command in the root of your application.
-
For Single Test File: If you prefer to test a smaller section of your application or to control costs, consider generating tests for a single source and its corresponding test file:
keploy gen --sourceFilePath="<path to source file>" --testFilePath="<path to test file for above source file>" --testCommand="go test -v ./... -coverprofile=coverage.out && gocov convert coverage.out | gocov-xml > coverage.xml" --coverageReportPath="<path to coverage.xml>"
-
For Entire Application use the following command to generate tests across:
⚠️ Warning: Executing this command will generate unit tests for all files in the application. Depending on the size of the codebase, this process may take between 20 minutes to an hour and will incur costs related to LLM usage.keploy gen --testDir="." --testCommand="go test -v ./... -coverprofile=coverage.out && gocov convert coverage.out | gocov-xml > coverage.xml" --coverageReportPath="<path to coverage.xml>"
🎉 You should see improved test cases and code-coverage. ✅ Enjoy coding with enhanced unit test coverage! 🫰
-
-
Ensure you've set the API key, as mentioned in pre-requisites above:
export API_KEY=xxxx -
Ensure that your unit test report format is Cobertura(it's very common).
-
Generate tests using keploy-gen:
keploy gen --sourceFilePath="<path to source code file>" --testFilePath="<path to existing unit test file>" --testCommand="<cmd to execute unit tests>" --coverageReportPath="<path to cobertura-coverage.xml>"
Configure Keploy using command-line flags:
--sourceFilePath ""
--testFilePath ""
--coverageReportPath "coverage.xml"
--testCommand ""
--coverageFormat "cobertura"
--expectedCoverage 100
--maxIterations 5
--testDir ""
--llmBaseUrl "https://api.openai.com/v1"
--model "gpt-4o"
--llmApiVersion "sourceFilePath: Path to the source file for which tests are to be generated.testFilePath: Path where the generated tests will be saved.coverageReportPath: Path to generate the coverage report.testCommand(required): Command to execute tests and generate the coverage report.coverageFormat: Type of the coverage report (default "cobertura").expectedCoverage: Desired coverage percentage (default 100%).maxIterations: Maximum number of iterations for refining tests (default 5).testDir: Directory where tests will be written.llmBaseUrl: Base url of the llm.model: Specifies the AI model to use (default "gpt-4o").llmApiVersion: API version of the llm if any (default "")
Reach out to us. We're here to answer!
-
: Try a unit-gen on Mux-SQL app
-
: Try a unit-gen on Express-Mongoose app
Other language may be supported, we've not tested them yet. If your coverage reports are of Cobertura format then you should be able to use keploy-gen in any language.
Keploy-gen is not just a project but an attempt to make developers life easier testing applications. It aims to simplify the creation and maintenance of tests, ensuring high coverage, and adapts to the complexity of modern software development.
Referred from Meta's research, the four primary prompts used in the deployment for the December 2023 Instagram and Facebook app test-a-thons
| Prompt Name | Template |
|---|---|
| extend_test | Here is a Kotlin unit test class: {existing_test_class}. Write an extended version of the test class that includes additional tests to cover some extra corner cases. |
| extend_coverage | Here is a Kotlin unit test class and the class that it tests: {existing_test_class} {class_under_test}. Write an extended version of the test class that includes additional unit tests that will increase the test coverage of the class under test. |
| corner_cases | Here is a Kotlin unit test class and the class that it tests: {existing_test_class} {class_under_test}. Write an extended version of the test class that includes additional unit tests that will cover corner cases missed by the original and will increase the test coverage of the class under test. |
| statement_to_complete | Here is a Kotlin class under test {class_under_test} This class under test can be tested with this Kotlin unit test class {existing_test_class}. Here is an extended version of the unit test class that includes additional unit test cases that will cover methods, edge cases, corner cases, and other features of the class under test that were missed by the original unit test class: |
Limitation: This project currently doesn't generate quality fresh tests if there are no existing tests to learn from.
Enjoy coding with enhanced unit test coverage! 🫰