Skip to content

feat(stovepipe): add validate controller forwarding commits to batch#255

Open
mnoah1 wants to merge 1 commit into
mnoah1/stovepipe-start-wiringfrom
mnoah1/stovepipe-validate-controller
Open

feat(stovepipe): add validate controller forwarding commits to batch#255
mnoah1 wants to merge 1 commit into
mnoah1/stovepipe-start-wiringfrom
mnoah1/stovepipe-validate-controller

Conversation

@mnoah1

@mnoah1 mnoah1 commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Set up empty validate controller. This consumes events from the existing start controller, than forwards them to batch if valid. Here, we can implement additional validation logic on a given commit before passing it forward to a batch.

Also wires this into the existing controller

Why?

Building out overall workflow outlined in https://github.com/uber/submitqueue/blob/main/doc/rfc/stovepipe/workflow.md

What?

Validate step in the outline above

Test Plan

  • make local-stovepipe-start
  • Insert a message and confirm flow through the steps:
rchestrator-service-1  | 2026-06-16T22:15:23.665Z	DEBUG	consumer/consumer.go:351	processing delivery	{"controller": "start", "topic_key": "start", "message_id": "manual-1781648123538", "partition_key": "github.com/uber/cadence", "attempt": 1}
orchestrator-service-1  | 2026-06-16T22:15:23.665Z	INFO	start_controller	start/start.go:91	received change event	{"uri": "git://github.com/uber/cadence/refs/heads/main/abc123def456", "attempt": 1, "partition_key": "github.com/uber/cadence"}
orchestrator-service-1  | 2026-06-16T22:15:23.665Z	DEBUG	queue_mysql.message_store	mysql/message_store.go:72	inserting messages	{"topic": "validate", "count": 1, "visible_after": 0}
orchestrator-service-1  | 2026-06-16T22:15:23.669Z	DEBUG	queue_mysql.message_store	mysql/message_store.go:125	inserted messages	{"topic": "validate", "count": 1}
orchestrator-service-1  | 2026-06-16T22:15:23.669Z	DEBUG	queue_mysql.publisher	mysql/publisher.go:65	published message	{"topic": "validate", "message_id": "git://github.com/uber/cadence/refs/heads/main/abc123def456"}
orchestrator-service-1  | 2026-06-16T22:15:23.669Z	INFO	start_controller	start/start.go:106	published commit to validate	{"uri": "git://github.com/uber/cadence/refs/heads/main/abc123def456", "topic_key": "validate"}
orchestrator-service-1  | 2026-06-16T22:15:23.674Z	DEBUG	consumer/consumer.go:479	message processed successfully	{"controller": "start", "topic_key": "start", "message_id": "manual-1781648123538", "partition_key": "github.com/uber/cadence", "attempt": 1, "elapsed_ms": 4}
orchestrator-service-1  | 2026-06-16T22:15:23.757Z	DEBUG	queue_mysql.message_store	mysql/message_store.go:217	fetched messages	{"topic": "start", "partition_key": "github.com/uber/cadence", "count": 1}
orchestrator-service-1  | 2026-06-16T22:15:23.767Z	DEBUG	consumer/consumer.go:351	processing delivery	{"controller": "validate", "topic_key": "validate", "message_id": "git://github.com/uber/cadence/refs/heads/main/abc123def456", "partition_key": "github.com/uber/cadence", "attempt": 1}
orchestrator-service-1  | 2026-06-16T22:15:23.767Z	INFO	validate_controller	validate/validate.go:89	received change event	{"uri": "git://github.com/uber/cadence/refs/heads/main/abc123def456", "attempt": 1, "partition_key": "github.com/uber/cadence"}
orchestrator-service-1  | 2026-06-16T22:15:23.767Z	DEBUG	queue_mysql.message_store	mysql/message_store.go:72	inserting messages	{"topic": "batch", "count": 1, "visible_after": 0}
orchestrator-service-1  | 2026-06-16T22:15:23.770Z	DEBUG	queue_mysql.message_store	mysql/message_store.go:125	inserted messages	{"topic": "batch", "count": 1}
orchestrator-service-1  | 2026-06-16T22:15:23.770Z	DEBUG	queue_mysql.publisher	mysql/publisher.go:65	published message	{"topic": "batch", "message_id": "git://github.com/uber/cadence/refs/heads/main/abc123def456"}
orchestrator-service-1  | 2026-06-16T22:15:23.770Z	INFO	validate_controller	validate/validate.go:104	published commit to batch	{"uri": "git://github.com/uber/cadence/refs/heads/main/abc123def456", "topic_key": "batch"}
orchestrator-service-1  | 2026-06-16T22:15:23.774Z	DEBUG	consumer/consumer.go:479	message processed successfully	{"controller": "validate", "topic_key": "validate", "message_id": "git://github.com/uber/cadence/refs/heads/main/abc123def456", "partition_key": "github.com/uber/cadence", "attempt": 1, "elapsed_ms": 3}

Issue

@mnoah1 mnoah1 marked this pull request as ready for review June 16, 2026 22:20
@mnoah1 mnoah1 requested review from a team, behinddwalls and sbalabanov as code owners June 16, 2026 22:20
Mirror the start controller's shape for the validate pipeline stage: the
validate controller consumes the validate topic, guards the carried
partition key, and forwards the change event to the batch stage. Adds the
batch topic key, wires the controller into the example orchestrator, and
registers the batch topic in the registry.

Currently a forwarding stub; placeholder comments describe the eventual
commit-metadata resolution per the workflow RFC.
@mnoah1 mnoah1 force-pushed the mnoah1/stovepipe-validate-controller branch from 4ca3b54 to 5d2e6de Compare June 17, 2026 03:03
@mnoah1 mnoah1 force-pushed the mnoah1/stovepipe-start-wiring branch from 092eba3 to 07811a3 Compare June 17, 2026 03:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant