Description
When calling the pipeline driver main function from pipeline unit test to simulates a full pipeline run, an error occurs saying unknown connector for request-channel
Steps to Reproduce
Clear, specific, and detailed steps taken to enable reproduction of the bug for investigation.
- Create a pipeline with following pipeline model
{
"name": "SimplePipeline",
"package": "org.test",
"type": {
"name": "data-flow",
"implementation": "data-delivery-spark"
},
"steps": [
{
"name": "Ingest",
"type": "synchronous",
"alerting": {
"enabled": false
},
"dataProfiling": {
"enabled": false
},
"provenance": {
"enabled": false
}
}
]
}
- Repeatedly build the pipeline:
mvn clean install until all the manual actions are finished.
- In the PipelineSteps.java class,
an_action_occurs() function, add below content
// execute your action
String[] args = {};
SimplePipelineDriver.main(args);
- Build the spark pipeline module:
mvn clean install -pl :simple-pipeline -Dmaven.build.cache.skipCache
Expected Behavior
A clear and concise description of the outcome you expected from the actions taken.
5. The build should pass without any error
Actual Behavior
A clear and concise description of the outcome you experienced from the actions taken.
5. Build failed with below error:
[ERROR] Errors:
[ERROR] SRMSG00072: Unknown connector for `request-channel`.
[INFO]
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
- If removing all in-memory kafka topics listed in the
SparkTestHarness class configureMessagingChannels() function, the Failed to construct kafak producer error will occur.
Additional Context
- Log output
- Screenshots (if applicable)
- Solution Baseline Version
- Environment details (local, cloud, Azure, AWS, etc.)
Description
When calling the pipeline driver main function from pipeline unit test to simulates a full pipeline run, an error occurs saying
unknown connector for request-channelSteps to Reproduce
Clear, specific, and detailed steps taken to enable reproduction of the bug for investigation.
mvn clean installuntil all the manual actions are finished.an_action_occurs()function, add below contentmvn clean install -pl :simple-pipeline -Dmaven.build.cache.skipCacheExpected Behavior
A clear and concise description of the outcome you expected from the actions taken.
5. The build should pass without any error
Actual Behavior
A clear and concise description of the outcome you experienced from the actions taken.
5. Build failed with below error:
SparkTestHarnessclassconfigureMessagingChannels()function, theFailed to construct kafak producererror will occur.Additional Context