Migrate build repair agent to Hackbot by evgenyrp · Pull Request #6232 · mozilla/bugbug

evgenyrp · 2026-06-25T00:55:04Z

The goal is to preserve the same capabilities but run it on the new Hackbot platform. We'll continue iterating on it in the follow-up PRs. It now runs successfully with Docker Compose and evals.

Successful eval + traces

Changes in Hackbot libs

allow cloning a specific commit
better handle Bugzilla errors
allow the agent to build a specific target (helps with build time)
fix "Rust compiler not found"

Changes in the agent

preserve two-stage approach analysis-fix (we can always use analysis even if the fix failed)
ability to pull Bugzilla bug info through MCP
the bug ID is optional since we might want to trigger it without a bug
download Taskcluster failure logs (full log and sanitized errors only log)
ability to build and bootstrap Firefox through MCP
the agent can now iterate on the fix if the MCP-triggered build failed
remove sandbox since we always run in Docker and on isolated machines
relax permissions
it can write mozconfig to make it closer to the environment of the failed build
update Claude models

Changes in evals

separate agent-as-a-judge from the main agent
significantly simplify integration with Weave since it now supports Claude Code SDK natively
reuse the main agent as is

Known issues

the evals now pull the full bug info through MCP which leaks the fix - added todo
the MCP-triggered build always fails first, and then the agent runs bootstrap. We should make the repo bootstrapped before that
I don't remember if TRY-push code was tested before, but I preserved it as a starting point

Pure rename to preserve git history before reworking the contents for the hackbot port. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Pure renames to preserve git history before reworking the contents for the hackbot port: agent.py -> hackbot_agents/build_repair/agent.py prompts.py -> hackbot_agents/build_repair/prompts.py build_repair_eval.py -> buildrepair_eval/eval.py Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Pure renames to preserve git history before reworking the contents for the hackbot port: config.py -> hackbot_agents/build_repair/config.py eval.py -> build-repair/evals/buildrepair_eval/eval.py Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Preserves file history; the prior move staged it as add+delete. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

# Conflicts: # .github/dependabot.yml # pyproject.toml # services/hackbot-api/app/agents.py # services/hackbot-api/app/schemas.py

suhaibmujahid

LGTM! Thank you!

Copilot

Pull request overview

This PR migrates the Firefox build-repair agent from the legacy BugBug service/scripts into the Hackbot platform, wiring it into hackbot-api, Hackbot runtime/source checkout, and a new Docker/Compose-based agent + broker setup (plus a new eval harness).

Changes:

Add hackbot-agent-build-repair workspace package with two-stage agent, Bugzilla broker sidecar, Taskcluster log download/sanitization, optional try-push tooling, and eval harness.
Extend Hackbot runtime to support pinning the source checkout to a specific ref/commit (SOURCE_REF / [source].ref) and extend agent-tools with targeted builds + better Bugzilla error normalization.
Remove the old services/buildrepair implementation and standalone evaluation script, and update compose/dependabot/workspace lockfile accordingly.

Reviewed changes

Copilot reviewed 38 out of 41 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
uv.lock	Adds the new build-repair agent workspace package and its dependencies (incl. eval extras).
services/hackbot-api/tests/test_agents.py	Adds registry + env-serialization tests for the new build-repair agent inputs.
services/hackbot-api/app/schemas.py	Introduces `BuildRepairInputs` schema for hackbot-api runs.
services/hackbot-api/app/agents.py	Registers the `build-repair` agent in `AGENT_REGISTRY`.
services/buildrepair/README.md	Removes legacy build-repair service documentation.
services/buildrepair/pyproject.toml	Removes legacy build-repair service packaging.
services/buildrepair/Dockerfile	Removes legacy build-repair service Docker image.
services/buildrepair/docker-compose.dev.yml	Removes legacy build-repair dev compose config.
scripts/build_repair_eval.py	Removes legacy standalone eval CLI.
libs/hackbot-runtime/tests/test_source.py	Adds tests for pinning source checkout to a specific ref/commit.
libs/hackbot-runtime/tests/test_context.py	Updates context tests to include `ref` / `SOURCE_REF` handling.
libs/hackbot-runtime/hackbot_runtime/source.py	Adds `ref` support and shallow-fetch behavior for pinned commits.
libs/hackbot-runtime/hackbot_runtime/context.py	Plumbs `SOURCE_REF` / config ref into `ensure_source_repo`.
libs/hackbot-runtime/hackbot_runtime/config.py	Adds `[source].ref` to runtime configuration.
libs/agent-tools/agent_tools/firefox/tools/build_firefox.py	Adds optional build `target` and ensures toolchain paths are on `PATH` post-bootstrap.
libs/agent-tools/agent_tools/firefox/init.py	Exposes `target` argument on the public MCP tool wrapper and updates docs.
libs/agent-tools/agent_tools/bugzilla.py	Normalizes more Bugzilla request failures into structured `ToolError`s.
docker-compose.yml	Includes the new build-repair agent compose file at the repo root.
bugbug/tools/build_repair/try_server.py	Removes legacy try-server logic from old agent implementation.
bugbug/tools/build_repair/prompts.py	Removes legacy prompt templates.
bugbug/tools/build_repair/config.py	Removes legacy agent config (models, sandbox/tool allowlist, etc.).
bugbug/tools/build_repair/agent.py	Removes the legacy build-repair agent implementation.
agents/build-repair/README.md	Adds Hackbot-based build-repair agent docs (local + eval usage).
agents/build-repair/pyproject.toml	Adds the new agent package definition (incl. `eval` extra).
agents/build-repair/hackbot.toml	Declares Hackbot source/firefox config for the new agent.
agents/build-repair/hackbot_agents/build_repair/try_push.py	Adds optional try-push MCP tool(s) for build-repair runs.
agents/build-repair/hackbot_agents/build_repair/prompts.py	Adds new Hackbot-oriented analysis/fix prompts (MCP Bugzilla + log paths + targeted build guidance).
agents/build-repair/hackbot_agents/build_repair/logs.py	Adds Taskcluster log download + sanitization utilities.
agents/build-repair/hackbot_agents/build_repair/config.py	Defines models + tool allowlists (Bugzilla, Firefox build, optional try-push).
agents/build-repair/hackbot_agents/build_repair/broker.py	Adds Bugzilla MCP broker sidecar that holds the Bugzilla API key.
agents/build-repair/hackbot_agents/build_repair/agent.py	Implements the two-stage Hackbot agent orchestration + tool result capture.
agents/build-repair/hackbot_agents/build_repair/main.py	Wires runtime context + env inputs into `run_build_repair` (including `SOURCE_REF`).
agents/build-repair/hackbot_agents/build_repair/init.py	Introduces the build-repair agent package module.
agents/build-repair/evals/worktree.py	Implements worktree management for parallel eval trials (Hackbot agent runs don’t need worktrees in prod).
agents/build-repair/evals/verify.py	Adds LLM-as-a-judge verification module for evals (data contamination cutoff logic included).
agents/build-repair/evals/scorer.py	Updates eval scoring outputs (drops token accounting fields).
agents/build-repair/evals/eval.py	Adds the new Weave evaluation harness for the ported agent.
agents/build-repair/evals/init.py	Introduces evals package module.
agents/build-repair/Dockerfile	Adds agent/broker/eval multi-stage images for build-repair.
agents/build-repair/compose.yml	Adds compose services for broker, agent, and profile-gated eval harness.
.github/dependabot.yml	Removes dependabot updates for the deleted legacy buildrepair service.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+        resp = requests.get(url, headers=_HEADERS, timeout=_TIMEOUT)
+        resp.raise_for_status()
+        full_path.write_text(resp.text)
+        sanitized = sanitize_log(resp.text)
+        sanitized_path.write_text(
+            sanitized if sanitized else f"(no ERROR -/FATAL - lines matched in {url})\n"
+        )


evgenyrp · 2026-06-25T23:37:23Z

LGTM! Thank you!

ok, let me address Copilot's comments

evgenyrp · 2026-06-25T23:44:30Z

Ok, I see you marked most of them as resolved. I guess we can skip those

evgenyrp and others added 14 commits June 24, 2026 14:21

Add ability to clone a specific commit

217e6f6

Improve Bugzilla error handling

39d82f5

Add build target and fix Rust path

040944c

Move try_server.py to try_push.py

b043e05

Pure rename to preserve git history before reworking the contents for the hackbot port. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Migrate build repair agent to Hackbot

387acbf

Move eval.py to evals/ via rename

3c626dd

Preserves file history; the prior move staged it as add+delete. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Rewrite eval.py for Hackbot migration

d68aa6c

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Migrate build repair evals to Hackbot

f92c516

Remove old file from docker compose

11750c6

Add eval todo

84e1a2a

Update readme

dbac63f

Merge branch 'refs/heads/master' into build_repair_port_to_hackbot

babcd24

# Conflicts: # .github/dependabot.yml # pyproject.toml # services/hackbot-api/app/agents.py # services/hackbot-api/app/schemas.py

evgenyrp changed the title ~~Build repair port to hackbot~~ Migrate build repair agent to Hackbot Jun 25, 2026

Remove services

b2f6b74

evgenyrp marked this pull request as ready for review June 25, 2026 21:26

evgenyrp requested a review from suhaibmujahid June 25, 2026 21:26

suhaibmujahid previously approved these changes Jun 25, 2026

View reviewed changes

suhaibmujahid requested a review from Copilot June 25, 2026 22:44

Copilot started reviewing on behalf of suhaibmujahid June 25, 2026 22:44 View session

Copilot AI reviewed Jun 25, 2026

View reviewed changes

suhaibmujahid enabled auto-merge (squash) June 25, 2026 23:50

Remove unnecessary git config

d07ac26

evgenyrp dismissed suhaibmujahid’s stale review via d07ac26 June 26, 2026 00:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Migrate build repair agent to Hackbot#6232

Migrate build repair agent to Hackbot#6232
evgenyrp wants to merge 16 commits into
mozilla:masterfrom
evgenyrp:build_repair_port_to_hackbot

evgenyrp commented Jun 25, 2026 •

edited

Loading

Uh oh!

suhaibmujahid left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

evgenyrp commented Jun 25, 2026

Uh oh!

evgenyrp commented Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

evgenyrp commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes in Hackbot libs

Changes in the agent

Changes in evals

Known issues

Uh oh!

suhaibmujahid left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

evgenyrp commented Jun 25, 2026

Uh oh!

evgenyrp commented Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

evgenyrp commented Jun 25, 2026 •

edited

Loading