Skip to content

Account for actions/cache branch scoping in cache-miss misconfiguration detection #39386

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pelikhan merged 8 commits into from
Jun 16, 2026
+602 −36
Merged

Account for actions/cache branch scoping in cache-miss misconfiguration detection #39386

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
904bded
Plan cache-miss misconfiguration update
Copilot Jun 15, 2026
ed45819
Account for actions/cache branch scoping in cache-miss misconfigurati…
Copilot Jun 15, 2026
f6307e5
Refine restore-status step IDs and cache warning naming
Copilot Jun 15, 2026
5812765
Move cache restore-miss gating to JS without extra workflow step
Copilot Jun 15, 2026
8cf9bfc
Merge branch 'main' into copilot/update-cache-misconfiguration-algorithm
github-actions[bot] Jun 15, 2026
f56ada4
Merge branch 'main' into copilot/update-cache-misconfiguration-algorithm
github-actions[bot] Jun 15, 2026
2798a11
Merge branch 'main' into copilot/update-cache-misconfiguration-algorithm
github-actions[bot] Jun 16, 2026
dae0142
Recompile workflows after merge with main
Copilot Jun 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/design-decision-gate.lock.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

24 changes: 15 additions & 9 deletions actions/setup/js/handle_agent_failure.cjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1018,10 +1018,11 @@ function loadMissingDataMessages(items) {
* When cache-memory is enabled and a cache_miss is detected, appends a
* configuration-problem warning to the context.
* @param {boolean} cacheMemoryEnabled - Whether cache-memory is configured for this workflow
* @param {boolean} cacheMemoryRestored - Whether cache restore matched an existing cache key in this run
* @param {Array<any>} [items] - Optional pre-loaded agent output items. When provided, avoids re-reading the output file.
* @returns {string} Formatted missing data context
*/
function buildMissingDataContext(cacheMemoryEnabled, items) {
function buildMissingDataContext(cacheMemoryEnabled, cacheMemoryRestored, items) {
const missingDataMessages = loadMissingDataMessages(items);

if (missingDataMessages.length === 0) {
Expand All @@ -1036,7 +1037,8 @@ function buildMissingDataContext(cacheMemoryEnabled, items) {

// When cache-memory is configured and cache_miss is present, avoid repeating the same
// signal in the generic "Missing Data" section. Keep the specialised cache warning below.
const displayableMissingData = cacheMemoryEnabled && hasCacheMiss ? missingDataMessages.filter(m => m.reason !== "cache_memory_miss") : missingDataMessages;
const shouldShowCacheWarning = cacheMemoryEnabled && cacheMemoryRestored && hasCacheMiss;
const displayableMissingData = shouldShowCacheWarning ? missingDataMessages.filter(m => m.reason !== "cache_memory_miss") : missingDataMessages;

@github-actions github-actions Bot Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cache_memory_miss items now flow into the generic missing-data section when cache was not restored — can trigger missingDataReportAsFailure failure verdicts on expected first-run misses.

💡 Explanation

In the old code, cache_memory_miss was always stripped from displayableMissingData whenever cacheMemoryEnabled && hasCacheMiss, regardless of restore status. The specialist template was then unconditionally shown.

In the new code, when cacheMemoryRestored=false, shouldShowCacheWarning is false, so displayableMissingData includes the cache_memory_miss item unfiltered. It flows into the "Missing Data Reported" block at line 1046 and is visible to the downstream missingDataReportAsFailure guard.

This means that if an operator has missingDataReportAsFailure=true, a completely normal first-run or branch-scoped cache miss (expected behavior) now causes a failure-level verdict in handle_agent_failure, which would have been a no-op before this PR.

The PR description notes this is intentional ("kept in generic missing-data context"), but the downstream failure guard was not updated to exclude cache_memory_miss from missingDataReportAsFailure evaluation when !cacheMemoryRestored. A targeted fix:

// Filter out expected cache misses from failure accounting when cache was not restored
const failureAccountableMissingData = (cacheMemoryEnabled && !cacheMemoryRestored)
  ? missingDataMessages.filter(m => m.reason !== "cache_memory_miss")
  : missingDataMessages;
// use failureAccountableMissingData for hasMissingData / failure-level guards
// use displayableMissingData (current) for the formatted context string


let context = "";
if (displayableMissingData.length > 0) {
Expand All @@ -1046,8 +1048,8 @@ function buildMissingDataContext(cacheMemoryEnabled, items) {
context += "\n\n";
}

if (cacheMemoryEnabled && hasCacheMiss) {
core.info("Cache-miss detected despite cache-memory being available — likely a configuration problem");
if (shouldShowCacheWarning) {
core.info("Cache-miss detected after a successful cache restore — likely a configuration problem");
const templatePath = getPromptPath("cache_memory_miss.md");
context += "\n" + renderTemplateFromFile(templatePath, {}) + "\n";
}
Expand Down Expand Up @@ -2502,6 +2504,7 @@ async function main() {
// Cache-memory availability flag — set when cache-memory is configured for the workflow.
// Used to detect cache-miss misconfigurations reported by the agent.
const cacheMemoryEnabled = process.env.GH_AW_CACHE_MEMORY_ENABLED === "true";
const cacheMemoryRestored = process.env.GH_AW_CACHE_MEMORY_RESTORED === "true";

// Collect repo-memory validation errors from all memory configurations
const repoMemoryValidationErrors = [];
Expand Down Expand Up @@ -2549,6 +2552,7 @@ async function main() {
core.info(`Lockdown check failed: ${hasLockdownCheckFailed}`);
core.info(`Stale lock file check failed: ${hasStaleLockFileFailed}`);
core.info(`Cache memory enabled: ${cacheMemoryEnabled}`);
core.info(`Cache memory restored: ${cacheMemoryRestored}`);
core.info(`Missing tool report-as-failure: ${missingToolReportAsFailure}`);
core.info(`Missing data report-as-failure: ${missingDataReportAsFailure}`);

Expand Down Expand Up @@ -2678,19 +2682,21 @@ async function main() {
// Check for items regardless of agentOutputResult.success so that cache-miss signals
// emitted alongside other output are not missed when the agent job also fails.
let hasCacheMissMisconfiguration = false;
if (cacheMemoryEnabled && agentOutputResult.items) {
if (cacheMemoryEnabled && cacheMemoryRestored && agentOutputResult.items) {
const cacheMissItems = agentOutputResult.items.filter(item => item.type === "missing_data" && item.reason === "cache_memory_miss");
if (cacheMissItems.length > 0) {
hasCacheMissMisconfiguration = true;
core.info(`Cache-miss misconfiguration detected: ${cacheMissItems.length} missing_data item(s) with reason "cache_memory_miss" despite cache-memory being available`);
core.info(`Cache-miss misconfiguration detected: ${cacheMissItems.length} missing_data item(s) with reason "cache_memory_miss" after cache restore matched an existing key`);
}

@github-actions github-actions Bot Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnose] ✅ Positive highlight: the new else if branch correctly logs why misconfiguration detection was skipped. This is exactly the right operator-facing signal — without it, a skipped detection would be invisible in the run logs.

One small follow-up worth considering: the log message references "actions/cache branch scoping and first-run behavior" — consistent with the PR description. Consider also logging the specific number of cache_memory_miss items (matching the style of the hasCacheMissMisconfiguration true-path log above) so operators can see the suppressed count if they need to audit:

�� Optional log improvement 
  const suppressedCount = agentOutputResult.items
    ? agentOutputResult.items.filter(i => i.type === "missing_data" && i.reason === "cache_memory_miss").length
    : 0;
  core.info(`Cache-memory is configured but no cache restore match was found; ${suppressedCount} cache_memory_miss item(s) treated as expected cache miss (actions/cache branch scoping and first-run behavior)`);
}

} else if (cacheMemoryEnabled && !cacheMemoryRestored) {
core.info("Cache-memory is configured but no cache restore match was found; cache_memory_miss is treated as expected cache miss (actions/cache branch scoping and first-run behavior)");
}

// Only proceed if the agent job actually failed OR timed out OR there are assignment errors OR
// create_discussion errors OR code-push failures OR push_repo_memory failed OR missing safe outputs
// OR a GitHub App token minting step failed OR the lockdown check failed OR copilot assignment failed
// OR the stale lock file check failed OR the agent reported task incompletion via report_incomplete
// OR a cache-miss was detected despite cache-memory being available (configuration problem)
// OR a cache-miss was detected after cache restore succeeded (configuration problem)
// OR the agent reported missing tools or missing data (treated as agent failures by default).
// BUT skip if we only have noop outputs (that's a successful no-action scenario)
if (
Expand Down Expand Up @@ -2913,7 +2919,7 @@ async function main() {
const pushRepoMemoryFailureContext = buildPushRepoMemoryFailureContext(hasPushRepoMemoryFailure, repoMemoryPatchSizeExceededIDs, runUrl);

// Build missing_data context (only when report-as-failure is enabled for this signal type)
const missingDataContext = missingDataReportAsFailure ? buildMissingDataContext(cacheMemoryEnabled, agentOutputResult.items) : "";
const missingDataContext = missingDataReportAsFailure ? buildMissingDataContext(cacheMemoryEnabled, cacheMemoryRestored, agentOutputResult.items) : "";

// Build tool-denials-exceeded guard context from events.jsonl
const toolDenialsExceededContext = buildToolDenialsExceededContext(toolDenialsExceededEvents, workflowID);
Expand Down Expand Up @@ -3136,7 +3142,7 @@ async function main() {
const pushRepoMemoryFailureContext = buildPushRepoMemoryFailureContext(hasPushRepoMemoryFailure, repoMemoryPatchSizeExceededIDs, runUrl);

// Build missing_data context (only when report-as-failure is enabled for this signal type)
const missingDataContext = missingDataReportAsFailure ? buildMissingDataContext(cacheMemoryEnabled, agentOutputResult.items) : "";
const missingDataContext = missingDataReportAsFailure ? buildMissingDataContext(cacheMemoryEnabled, cacheMemoryRestored, agentOutputResult.items) : "";

// Build tool-denials-exceeded guard context from events.jsonl
const toolDenialsExceededContext = buildToolDenialsExceededContext(toolDenialsExceededEvents, workflowID);
Expand Down
45 changes: 30 additions & 15 deletions actions/setup/js/handle_agent_failure.test.cjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2578,16 +2578,16 @@ describe("handle_agent_failure", () => {
});

it("returns empty string when agent output file does not exist", () => {

@github-actions github-actions Bot Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] The "returns empty string" test covers (false, false) and (true, false) but omits (true, true) — cache enabled AND restored, but no missing-data items. Adding this case explicitly documents and protects the early-return path for the most operationally-relevant argument combination.

💡 One-line addition 
expect(buildMissingDataContext(true, true)).toBe("");

This is the case that occurs every normal run where cache-memory is healthy and the agent produced no missing-data signals.

expect(buildMissingDataContext(false)).toBe("");
expect(buildMissingDataContext(true)).toBe("");
expect(buildMissingDataContext(false, false)).toBe("");
expect(buildMissingDataContext(true, false)).toBe("");

@github-actions github-actions Bot Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test now only covers (true, false) for the "no output file" case — (true, true) is untested.

💡 Details

The test was updated from buildMissingDataContext(true) to buildMissingDataContext(true, false), but the corresponding (true, true) variant (cache enabled AND restored, file absent) was not added. The early-return on empty missingDataMessages is not exercised when cacheMemoryRestored=true, so a regression that conditionally skips the early return in that branch would go undetected.

Suggested addition:

expect(buildMissingDataContext(true, true)).toBe("");

This is low-risk but cheap to add alongside the existing cases.

});

it("returns empty string when agent output has no missing_data items", () => {
fs.writeFileSync(path.join(tmpDir, "agent_output.json"), JSON.stringify({ items: [{ type: "noop", reason: "done" }] }));
vi.resetModules();
({ buildMissingDataContext } = require("./handle_agent_failure.cjs"));
expect(buildMissingDataContext(false)).toBe("");
expect(buildMissingDataContext(true)).toBe("");
expect(buildMissingDataContext(false, false)).toBe("");
expect(buildMissingDataContext(true, false)).toBe("");
});

it("returns missing data context without cache warning when cacheMemoryEnabled is false", () => {
Expand All @@ -2599,13 +2599,13 @@ describe("handle_agent_failure", () => {
);
vi.resetModules();
({ buildMissingDataContext } = require("./handle_agent_failure.cjs"));
const result = buildMissingDataContext(false);
const result = buildMissingDataContext(false, false);
expect(result).toContain("Missing Data Reported");
expect(result).toContain("cache\\_memory"); // data_type after markdown escaping
expect(result).not.toContain("Cache Configuration Problem");
});

it("appends cache configuration warning when cacheMemoryEnabled is true and cache_memory_miss item present", () => {
it("appends cache configuration warning when cache restore matched and cache_memory_miss item present", () => {
fs.writeFileSync(
path.join(tmpDir, "agent_output.json"),
JSON.stringify({
Expand All @@ -2615,14 +2615,14 @@ describe("handle_agent_failure", () => {
const templateContent =
"> [!WARNING]\n" +
"> <details>\n" +
"> <summary>Cache Configuration Problem: cache miss detected despite cache-memory being configured.</summary>\n>\n" +
"> <summary>Cache Configuration Problem: cache miss detected after cache restore succeeded.</summary>\n>\n" +
"> Review the [cache-memory configuration](https://github.github.com/gh-aw/reference/cache-memory/) and ensure the agent prompt correctly references files inside the cache directory.\n>\n" +
"> **File naming convention:** Cache files are stored at `/tmp/gh-aw/cache-memory/`.\n>\n" +
"> </details>";
fs.writeFileSync(path.join(promptsDir, "cache_memory_miss.md"), templateContent);
vi.resetModules();
({ buildMissingDataContext } = require("./handle_agent_failure.cjs"));
const result = buildMissingDataContext(true);
const result = buildMissingDataContext(true, true);
expect(result).not.toContain("Missing Data Reported");
expect(result).toContain("Cache Configuration Problem");
expect(result).toContain("> [!WARNING]");
Expand All @@ -2640,18 +2640,33 @@ describe("handle_agent_failure", () => {
items: [{ type: "missing_data", reason: "cache_memory_miss" }],
})
);
const templateContent = "> [!WARNING]\n" + "> <details>\n" + "> <summary>Cache Configuration Problem: cache miss detected despite cache-memory being configured.</summary>\n>\n" + "> Details here.\n>\n" + "> </details>";
const templateContent = "> [!WARNING]\n" + "> <details>\n" + "> <summary>Cache Configuration Problem: cache miss detected after cache restore succeeded.</summary>\n>\n" + "> Details here.\n>\n" + "> </details>";
fs.writeFileSync(path.join(promptsDir, "cache_memory_miss.md"), templateContent);
vi.resetModules();
({ buildMissingDataContext } = require("./handle_agent_failure.cjs"));
const result = buildMissingDataContext(true);
const result = buildMissingDataContext(true, true);
expect(result).not.toContain("Missing Data Reported");
expect(result).toContain("Cache Configuration Problem");
expect(result).toContain("> [!WARNING]");
expect(result).toContain("<summary>");
expect(result).toContain("<details>");
});

it("does not append cache configuration warning when cache restore did not match", () => {
fs.writeFileSync(
path.join(tmpDir, "agent_output.json"),
JSON.stringify({
items: [{ type: "missing_data", data_type: "cache_memory", reason: "cache_memory_miss" }],
})
);
vi.resetModules();
({ buildMissingDataContext } = require("./handle_agent_failure.cjs"));
const result = buildMissingDataContext(true, false);
expect(result).toContain("Missing Data Reported");
expect(result).toContain("cache\\_memory");
expect(result).not.toContain("Cache Configuration Problem");
});

it("shows generic missing-data context for non-cache items while still appending cache warning", () => {
fs.writeFileSync(
path.join(tmpDir, "agent_output.json"),
Expand All @@ -2662,11 +2677,11 @@ describe("handle_agent_failure", () => {
],
})
);
const templateContent = "> [!WARNING]\n> <details>\n> <summary>Cache Configuration Problem: cache miss detected despite cache-memory being configured.</summary>\n>\n> Details here.\n>\n> </details>";
const templateContent = "> [!WARNING]\n> <details>\n> <summary>Cache Configuration Problem: cache miss detected after cache restore succeeded.</summary>\n>\n> Details here.\n>\n> </details>";
fs.writeFileSync(path.join(promptsDir, "cache_memory_miss.md"), templateContent);
vi.resetModules();
({ buildMissingDataContext } = require("./handle_agent_failure.cjs"));
const result = buildMissingDataContext(true);
const result = buildMissingDataContext(true, true);
expect(result).toContain("Missing Data Reported");
expect(result).toContain("user\\_data");
expect(result).toContain("Cache Configuration Problem");
Expand All @@ -2684,11 +2699,11 @@ describe("handle_agent_failure", () => {
],
})
);
const templateContent = "> [!WARNING]\n> <details>\n> <summary>Cache Configuration Problem: cache miss detected despite cache-memory being configured.</summary>\n>\n> Details here.\n>\n> </details>";
const templateContent = "> [!WARNING]\n> <details>\n> <summary>Cache Configuration Problem: cache miss detected after cache restore succeeded.</summary>\n>\n> Details here.\n>\n> </details>";
fs.writeFileSync(path.join(promptsDir, "cache_memory_miss.md"), templateContent);
vi.resetModules();
const { buildMissingDataContext: buildMissingDataContextFn, buildReportIncompleteContext } = require("./handle_agent_failure.cjs");
const missingDataResult = buildMissingDataContextFn(true);
const missingDataResult = buildMissingDataContextFn(true, true);
const reportIncompleteResult = buildReportIncompleteContext();
expect(missingDataResult).not.toContain("Missing Data Reported");
expect(missingDataResult).toContain("Cache Configuration Problem");
Expand All @@ -2705,7 +2720,7 @@ describe("handle_agent_failure", () => {
);
vi.resetModules();
({ buildMissingDataContext } = require("./handle_agent_failure.cjs"));
const result = buildMissingDataContext(true);
const result = buildMissingDataContext(true, true);
expect(result).toContain("Missing Data Reported");
expect(result).not.toContain("Cache Configuration Problem");
});
Expand Down
6 changes: 4 additions & 2 deletions actions/setup/md/cache_memory_miss.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
> [!WARNING]
> <details>
> <summary>Cache Configuration Problem: cache miss detected despite cache-memory being configured.</summary>
> <summary>Cache Configuration Problem: cache miss detected after cache restore succeeded.</summary>
>
> The agent reported a cache miss (`missing_data` with `reason: cache_memory_miss`) even though cache-memory is configured and was available. This likely indicates the prompt is misconfigured and the agent cannot locate the correct file path within the cache directory.
> The agent reported a cache miss (`missing_data` with `reason: cache_memory_miss`) after the workflow successfully restored cache-memory for this run. This likely indicates the prompt is misconfigured and the agent cannot locate the correct file path within the cache directory.
>
> This warning is shown only when a cache restore matched an existing key. Cache misses can be expected on first runs and on branches where `actions/cache` has no visible entries due to branch scoping.
>
> Review the [cache-memory configuration](https://github.github.com/gh-aw/reference/cache-memory/) and ensure the agent prompt correctly references files inside the cache directory.
>
Expand Down
Loading
Loading