Skip to content

BUG: unknown connector for error when calling driver main function from unit test #609

Description

@csun-cpointe

Description

When calling the pipeline driver main function from pipeline unit test to simulates a full pipeline run, an error occurs saying unknown connector for request-channel

Steps to Reproduce

Clear, specific, and detailed steps taken to enable reproduction of the bug for investigation.

  1. Create a pipeline with following pipeline model
{
	"name": "SimplePipeline",
	"package": "org.test",
	"type": {
		"name": "data-flow",
		"implementation": "data-delivery-spark"
	},
	"steps": [
		{
			"name": "Ingest",
			"type": "synchronous",
			"alerting": {
				"enabled": false
			},
			"dataProfiling": {
				"enabled": false
			},
			"provenance": {
				"enabled": false
			}
		}
          ]
}
  1. Repeatedly build the pipeline: mvn clean install until all the manual actions are finished.
  2. In the PipelineSteps.java class, an_action_occurs() function, add below content
        // execute your action
        String[] args = {};
        SimplePipelineDriver.main(args);
  1. Build the spark pipeline module: mvn clean install -pl :simple-pipeline -Dmaven.build.cache.skipCache

Expected Behavior

A clear and concise description of the outcome you expected from the actions taken.
5. The build should pass without any error

Actual Behavior

A clear and concise description of the outcome you experienced from the actions taken.
5. Build failed with below error:

[ERROR] Errors: 
[ERROR]   SRMSG00072: Unknown connector for `request-channel`.
[INFO] 
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
  1. If removing all in-memory kafka topics listed in the SparkTestHarness class configureMessagingChannels() function, the Failed to construct kafak producer error will occur.

Additional Context

  • Log output
  • Screenshots (if applicable)
  • Solution Baseline Version
  • Environment details (local, cloud, Azure, AWS, etc.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions