Starter for IntelliJ IDEA based IDE's

Overview

The Starter will help you create and run tests for IntelliJ-based IDEs. It can run an IDE from an existing installer or download one if needed, configure IDE, launch the IDE as an external process, send commands to the IDE, collect the logs, metrics and other IDEs output after test execution.

Additional features include:

• Executing commands inside IDE (see the list of available commands below)
• Implementing custom commands for use in tests
• Executing custom code (without relying on external libraries not available in the IDE)
• CI integration (optional)
• Test artifact collection
• Freeze/exception collection
• Reporting artifacts to CI (optional)
• Running tests with a profiler (not yet included)

More details how it works can be found here

Supported products

• IDEA
• GoLand
• WebStorm
• PhpStorm
• DataGrip
• PyCharm
• RubyMine
• Android Studio

How to use

For the required build.gradle and various test scenarios, refer to the Examples directory.

Available commands

The available commands can be found in the performance-testing-commands package which includes performance-intensive operations and IDE configuration commands.

Examples of included commands:

• waitForSmartMode()
• flushIndexes()
• setupProjectSdk(sdkName: String, sdkType: String, sdkPath: String)]
• openFile(relativePath: String)
• openProject(projectPath: Path)
• reopenProject()
• goto(line: String, column: String)
• findUsages()
• inspectCode()
• checkOnRedCode()
• exitApp(forceExit: Boolean = true)
• memoryDump()
• dumpProjectFiles()
• compareProjectFiles(firstDir: String, secondDir: String)
• cleanCaches()
• doComplete(times: Int)
• openProjectView()
• pressKey(key: String)
• delayType(command: String)
• doLocalInspection()
• altEnter(intention: String)
• callAltEnter(times: Int, intention: String = "")
• createAllServicesAndExtensions()
• runConfiguration(command: String)
• openFileWithTerminate(relativePath: String, terminateIdeInSeconds: Long)
• searchEverywhere(text: String)
• storeIndices()
• compareIndices()
• recoveryAction(action: RecoveryActionType)
• importMavenProject()
• importGradleProject()
• and many more

Performance testing/Metrics collection

Metrics are values collected during test execution, including operation durations, command results (e.g., the number of elements in completion lists), and more.
You can find more information about different approach to collect metrics (with or without Starter) in Starter Readme