diff --git a/ba_auto/reference_notes/mapping.md b/ba_auto/reference_notes/mapping.md index 719a880..23ad4c6 100644 --- a/ba_auto/reference_notes/mapping.md +++ b/ba_auto/reference_notes/mapping.md @@ -6,7 +6,7 @@ Maps each local feature to the corresponding `~/repo/baas-reference/module/...` |---|---|---|---|---|---| | Battle Pass | `module/collect_pass_reward.py` | `implement` (main flow), `main_page_to_pass_menu`, `to_page_pass_mission`, `to_page_pass_menu`, `collect_reward` -- `detect_statistics`/`detect_pass_level`/`detect_pass_next_level_point`/`detect_pass_weekly_point` (OCR level/point readout) deliberately not ported, nothing in this project's flow consumes those numbers | `ba_auto/tasks/battle_pass.py` | `core/picture.py::co_detect`'s img_possibles/img_ends (template-image matching against ~20 assets this project doesn't have) → local color probes (`config.BATTLE_PASS_INSIDE_PROBES`/`BATTLE_PASS_MISSION_TAB_PROBES`/`BATTLE_PASS_CLAIM_PROBE`); claim-all button reuses stamina.py's exact bright-yellow-vs-grey enabled/disabled signature and its Enter-then-Enter-to-dismiss-popup convention | Done, live-tested 2026-07-29: real mission-point claim (Lv.3→4) and real pass-level reward claim, both confirmed via screenshots, plus two real `./ba_dailies.sh battle_pass` CLI runs (the first surfaced a real Escape-overshoots-to-home navigation bug, fixed via a self-healing `_ensure_pass_menu` loop rather than a single fixed retry; the second, post-fix, run was clean). `navigation.is_on_subscreen`/`is_modal_open` both read false/false on both battle-pass screens (same false-negative pattern login.py's own pre-home states hit) -- this module carries its own local probes and explicitly Escapes back to home at the end rather than trusting `ba_daily.py`'s generic post-task `return_to_home` cleanup, which would otherwise no-op and leave the game stuck on the pass menu. Also hit live at the very start of this session (not new -- the same full-screen ambient/idle cutscene state Phase 24 already documents, no header/nav chrome at all, cleared by a single click/Enter anywhere): `_ensure_home()`'s pre-task check would misread it as home (same false/false pattern as every other state in that family), so a run starting while it's showing would need that first click/keypress to land correctly. Not specific to this feature, no action taken here. Pure reclaim, no AP/credit/ticket spend -- added to `DEFAULT_ORDER` and the `daily` preset (not `q4h`, matching gem_shop/circle's once-a-day-is-enough placement). | | Mailbox | `module/mail.py` | `to_mail`, `implement` | `ba_auto/tasks/mailbox.py` | tap/click via xdotool, screenshot via scrot, `color.rgb_in_range` → `driver.color_at` pixel-probe check | Migrated: real Python, state-verified via color probe (no legacy bridge) | -| Cafe | `module/cafe_reward.py` | `to_cafe` (its `relationship_rank_up` popup-handling now also ported, see below), `interaction_for_cafe_solve_method3`, `collect`, `invite_girl`/`invite_by_affection`/`checkConfirmInvite` (student invitation, added 2026-07-14) | `ba_auto/tasks/cafe.py` | `picture.co_detect`/`color.rgb_in_range` → `driver.color_at` pixel-probe checks; sparkle template match ported in-process into `ba_auto/detector.py` (`find_cafe_sparkle`, now multi-scale) | Migrated: real Python, state-verified via color probes (no legacy bridge). Pat loop now polls for the full attempt budget instead of stopping on the first miss (see `plan.md` Phase 6 follow-up) — not yet confirmed against a live sparkle since none was available during testing. `_dismiss_rank_up_if_shown` reuses `navigation.is_on_subscreen` to detect and clear the full-screen bond-rank-up cutscene after a pat (see `plan.md` Phase 6 follow-up: rank-up popups) — not yet live-confirmed against a real trigger. **Student invitation (2026-07-14)**: per explicit user direction, invite a student into each room before farming it, preferring highest affection, always skipping (never confirming) a candidate that would swap an already-seated student's costume or move one in from the other room — directly ports `invite_by_affection`/`checkConfirmInvite`'s own logic with the reference's default `cafe_reward_allow_exchange_student`/`cafe_reward_allow_duplicate_invite` both `False` (no local config exists to make either configurable). Live-calibrated against nik-gpu with **zero real tickets spent** — all 3 real dialog variants (normal confirm, same-room costume-swap warning, neighboring-room move warning) found and confirmed live, every one cancelled rather than confirmed during calibration. The heart-shaped affection badge OCR reuses `detector.read_int_on_heart_badge` directly (pixel-confirmed the same widget lesson.py's own badges use — digit pixels sampled RG, matching that function's exact masking assumption) rather than building a second OCR path. Dialog-type detection OCRs the title bar and checks for either warning's own distinctive substring (`衣装`/`隣`) rather than an exact match, mirroring `event_sweep.py`'s own `"終了"` substring-match reasoning. All 5 heart-badge reads and all 3 dialog classifications verified offline against the real saved calibration screenshots before deploying (exact match, zero mismatches). **Confirmed live with real tickets spent, both rooms**: room 1 correctly skipped one 衣装替え (costume-swap) candidate then invited row 1 cleanly; room 2 correctly skipped three consecutive 隣のカフェの生徒を招待 (neighboring-room-move) candidates (expected — room 1's own invite had just taken the account's highest-affection students) then invited row 3 cleanly. Both newly-invited students were immediately patted successfully in the same run (the "newly invited student can be farmed" requirement), income was claimed, and the task returned cleanly to the true home screen with no warnings anywhere in the log. **Horizontal camera panning (2026-07-14)**: per explicit user direction ("due to my screen size... move screen most right and most left then farm"), `_pat_room` now pans the room camera to its rightmost extreme via a new `driver.drag()` primitive (this project's first — distinct from `driver.scroll()`'s wheel-based gesture, which is for list widgets, not a room camera), farms there, pans to the leftmost extreme, farms there too — no vertical panning, per the user's own instruction. Ports the reference's `zoom_out`'s underlying intent (see the whole room regardless of width) via the user's own specified mechanism (panning) rather than the reference's (zoom). Live-confirmed drag-direction-to-reveal-side mapping, that HUD elements (status bar, ticket buttons) stay fixed regardless of pan (no camera reset needed afterward), and that one drag already reaches the true extreme (extra repeats are a confirmed no-op, kept as a safety margin). **Confirmed live with a real game-state change**: a full run patted a real sparkle in room 2 (score 0.997) specifically after panning to an extreme, while room 1 found nothing that run (expected — per-student cooldown). Clean end-to-end completion, no warnings. **Invite ticket cooldown bugfix (2026-07-14)**: real-usage report (with screenshot) that the invite step kept failing once the account's ticket went on cooldown — `_open_invite_list`'s original check (`not is_on_subscreen`) couldn't distinguish the real MomoTalk list opening from a "通知" cooldown notice ("待機時間が経過した後に、再度招待することができます。") opening directly instead, since both dim the header the same way; misread as "list opened", it sent `_ensure_invite_sort`/`_try_invite_row`'s fixed coordinates into a dialog that has none of them. Fixed by checking `navigation.is_modal_open` (the darker real-dialog reading) first — a dialog appearing before any row is clicked can only mean the ticket click raised one directly — and dismissing it via the shared `SWEEP_CONFIRM_BUTTON`, returning `False` so the existing "skip this room's invite" fallback handles it. **Confirmed live**: both rooms correctly detected and dismissed the cooldown notice with no cascading errors, task completed cleanly (exit 0, true home screen confirmed via screenshot). See `plan.md`'s Phase 6 follow-up #4. **Two further real-usage fixes (2026-07-15, `next_fix.md`)**: (1) the cooldown-notice dismiss above now presses Escape and verifies via `navigation.is_modal_open` with bounded retry, instead of a fixed-coordinate click that could miss and leave the notice open under the following pan drags; (2) `_dismiss_rank_up_if_shown` switched from `navigation.is_on_subscreen`'s single-pixel header probe (found live to misread some characters' rank-up cutscene art as bright) to a new `navigation.is_header_bar_visible`, requiring 8 spread-out header-row points to all read bright via one atomic multi-point capture (`driver.colors_at`, added alongside it). Both best-effort — not yet re-confirmed live post-fix. **A real rank-up happened naturally via cron (2026-07-16)**, reported live with a screenshot of the game left stuck on the cutscene — exposed a third bug, a timing gap rather than a threshold problem: `_dismiss_rank_up_if_shown` was only ever called once, immediately after a pat, with no wait beforehand, but the cutscene renders with its own client-side animation delay, so that single check could catch the tail end of the still-normal room view and conclude "clear" a beat before the actual cutscene appeared. Confirmed via a live pixel-check that the resulting stuck cutscene reads under `is_on_subscreen`/`is_modal_open`'s thresholds too (`SUBSCREEN_HEADER_PROBE` read `(180,227,244)`, r=180 < the 200 both checks need), so `navigation.return_to_home`'s generic cleanup also couldn't recover it — every "are we home" check downstream falsely agreed nothing was wrong, and the whole `q4h` cron run finished "successfully" with the game actually left stuck. Fixed by checking on every `_pat_current_view` poll iteration instead of only right after a pat, so a delayed cutscene appearance is caught (and dismissed) roughly a second later instead of never. **Confirmed live**: manually reproduced the exact stuck state left by the incident and confirmed a single Enter (the same action `_dismiss_rank_up_if_shown` already takes) cleared it back to the real cafe room view; the per-iteration check itself has not yet been re-exercised against a fresh live rank-up trigger end-to-end (same "could not force one on demand" caveat as the original fix). **`SPARKLE_CLICK_OFFSET` recalibrated (2026-07-17)**: reported live with a screenshot (`screenshots/cafe/bug_student_close_together.png`) — when two students stand close together, the original `(75, 47)` offset (ported byte-for-byte from `scripts/detect_and_click.py`, never independently verified) overshoots past the intended target's head and lands on a different, closer student instead; the click doesn't register, so `find_cafe_sparkle()` matches the same still-showing sparkle again next iteration, repeatedly clicking nearly the same point for the rest of the room's budget (confirmed live: 13 of 16 "pats" in one run clustered within a ~10px box). First recalibration attempt, `(0, 0)` (raw template-match center), was also wrong — reported live ("clicking the actual sparkle instead of the student head"): a coincidental ambient thought-bubble animation at the test position had been misread as a hover confirmation. Properly recalibrated via numbered candidate-point overlays (screenshots with several labeled offset options drawn on a real live sparkle; the user picked the correct one directly on-screen) against two independent real students in different poses (sitting at an arcade cabinet, lying on a couch) — both converged on the same small offset, `(51, 15)`. **Confirmed live** via the real `./ba_dailies.sh cafe` CLI path: 6 sparkles patted across both rooms at genuinely distinct coordinates with zero repeated-click clustering, clean `[cafe] Done.` and true-home return. **Room 2 now invites lowest affection, plus invited-student name logging (2026-08-13, see `plan.md` Phase 31)**: per explicit user direction, room 1 keeps inviting highest affection (`_invite_student(..., prefer_highest=True)`, the default) while room 2 now invites lowest (`prefer_highest=False`) by sorting the MomoTalk list ascending instead of descending — same row-walking/skip logic either way, no reference equivalent (`invite_by_affection` only ever applies one direction per call and never logs back a name). New `CAFE_INVITE_NAME_RECT_X`/`CAFE_INVITE_NAME_RECT_Y_OFFSET` OCR rect, live-calibrated against a real open invite list; psm=7 (this project's usual single-line default) badly garbled short 2-character names ("ミカ" → `"' ーー、テテー"`), fixed by switching to psm=8 (single word) plus a punctuation strip, re-verified live reading all 5 sampled rows exactly correct. **Real live "screen not confirmed" failure investigated 2026-09-19** (reported by the user: "cafe currently full of resource, never claimed"): every real logged run across two days (4/4 `daily` fires, 10/11 `q4h` fires) hit `_dismiss_rank_up_if_shown`'s "cafe screen not confirmed (rank-up cutscene stuck?)" warning on the very first pat-loop iteration in both rooms, then `_enter_room`'s room-switch also failed, so income was never claimed. Extensive live debugging (superpowers:systematic-debugging — fresh game kill+relaunch, the full `login`→`cafe` sequence, the cooldown-dialog-dismiss path, and a real supervised `q4h` run through the actual cron wrapper, all included) could **not** reproduce the failure even once (3 manual reproductions plus one full live `q4h` run all completed cleanly) despite matching the real preset's task order as closely as possible. Ruled out: `DISPLAY`/`XAUTHORITY` env handling (already hardcoded correctly in `config.ENV`, unaffected by cron's minimal environment), and a "game always cold-launches" theory (both `daily` and `q4h` end with `exit_game`, so every real fire does start from a freshly-relaunched game, but forcing that same cold-launch manually still didn't reproduce it). Real, unrelated things found along the way: the `q4h` crontab's actual schedule (`0 1,10,13,17,21 * * *`) doesn't match its own documented `1,5,9,13,17,21` — confirmed **intentional** by the user (changed to avoid their own active play time), not a bug; and a real screenshot-capture crash (repeated libpng CRC errors, `RuntimeError` after 3 retries) traced to running a manual task while a scheduled `q4h` fire was concurrently active on the same game window — a real hazard of manual testing overlapping a cron fire, not a code bug, left unaddressed per explicit user direction ("No need to fix this, leave it as it be"). Since the exact interrupting state couldn't be captured live, `_dismiss_rank_up_if_shown` now saves a diagnostic screenshot to `scratchpad/` (`cafe_diag_not_confirmed_.png`) on its own final failure, so the next real occurrence leaves hard evidence instead of log text alone. **Sparkle-detection redesigned the same day, per explicit user direction**: the original design took one screenshot and matched only the single best sparkle per click attempt, polling up to a fixed `CAFE_MAX_CLICKS_PER_ROOM` (15) budget regardless of how many sparkles were actually visible at once — correct, but wasteful (confirmed live: ~50s per view, ~100s per room, when nothing was available to pat). `detector.find_cafe_sparkles` (replacing `find_cafe_sparkle`) now finds every sparkle above threshold in one frame via per-scale `np.where` plus a proximity-based dedupe (`SPARKLE_DEDUPE_DISTANCE`, not yet live-calibrated against two genuinely distinct close-together sparkles), and `_pat_current_view` pats every match found in one screenshot before recapturing, stopping once `CAFE_EMPTY_ROUNDS_TO_STOP` (2) consecutive rounds each find nothing rather than exhausting a fixed click budget. Patting a whole batch from one stale screenshot is safe here specifically because cafe sparkles sit on static room students, not a reflowing list like Common Shop's item grid (see that row's own 2026-09-17 incident for the contrasting case). **Confirmed live** via a real `./ba_dailies.sh cafe` run post-fix: both rooms completed correctly (no sparkles happened to be available that run, so the "pat multiple from one screenshot" path itself is implemented-but-not-yet-exercised against a real multi-sparkle frame), but the empty-room case that used to take ~100s per room now completes in ~14s — confirmed via real log timestamps, a >85% reduction for that case. The `q4h` run this same investigation used to try reproducing the original bug also completed with zero anomalies using the code as it stood *before* this redesign, so that run doesn't independently confirm the redesign itself — only the standalone post-deploy run does. | +| Cafe | `module/cafe_reward.py` | `to_cafe` (its `relationship_rank_up` popup-handling now also ported, see below), `interaction_for_cafe_solve_method3`, `collect`, `invite_girl`/`invite_by_affection`/`checkConfirmInvite` (student invitation, added 2026-07-14) | `ba_auto/tasks/cafe.py` | `picture.co_detect`/`color.rgb_in_range` → `driver.color_at` pixel-probe checks; sparkle template match ported in-process into `ba_auto/detector.py` (`find_cafe_sparkle`, now multi-scale) | Migrated: real Python, state-verified via color probes (no legacy bridge). Pat loop now polls for the full attempt budget instead of stopping on the first miss (see `plan.md` Phase 6 follow-up) — not yet confirmed against a live sparkle since none was available during testing. `_dismiss_rank_up_if_shown` reuses `navigation.is_on_subscreen` to detect and clear the full-screen bond-rank-up cutscene after a pat (see `plan.md` Phase 6 follow-up: rank-up popups) — not yet live-confirmed against a real trigger. **Student invitation (2026-07-14)**: per explicit user direction, invite a student into each room before farming it, preferring highest affection, always skipping (never confirming) a candidate that would swap an already-seated student's costume or move one in from the other room — directly ports `invite_by_affection`/`checkConfirmInvite`'s own logic with the reference's default `cafe_reward_allow_exchange_student`/`cafe_reward_allow_duplicate_invite` both `False` (no local config exists to make either configurable). Live-calibrated against nik-gpu with **zero real tickets spent** — all 3 real dialog variants (normal confirm, same-room costume-swap warning, neighboring-room move warning) found and confirmed live, every one cancelled rather than confirmed during calibration. The heart-shaped affection badge OCR reuses `detector.read_int_on_heart_badge` directly (pixel-confirmed the same widget lesson.py's own badges use — digit pixels sampled RG, matching that function's exact masking assumption) rather than building a second OCR path. Dialog-type detection OCRs the title bar and checks for either warning's own distinctive substring (`衣装`/`隣`) rather than an exact match, mirroring `event_sweep.py`'s own `"終了"` substring-match reasoning. All 5 heart-badge reads and all 3 dialog classifications verified offline against the real saved calibration screenshots before deploying (exact match, zero mismatches). **Confirmed live with real tickets spent, both rooms**: room 1 correctly skipped one 衣装替え (costume-swap) candidate then invited row 1 cleanly; room 2 correctly skipped three consecutive 隣のカフェの生徒を招待 (neighboring-room-move) candidates (expected — room 1's own invite had just taken the account's highest-affection students) then invited row 3 cleanly. Both newly-invited students were immediately patted successfully in the same run (the "newly invited student can be farmed" requirement), income was claimed, and the task returned cleanly to the true home screen with no warnings anywhere in the log. **Horizontal camera panning (2026-07-14)**: per explicit user direction ("due to my screen size... move screen most right and most left then farm"), `_pat_room` now pans the room camera to its rightmost extreme via a new `driver.drag()` primitive (this project's first — distinct from `driver.scroll()`'s wheel-based gesture, which is for list widgets, not a room camera), farms there, pans to the leftmost extreme, farms there too — no vertical panning, per the user's own instruction. Ports the reference's `zoom_out`'s underlying intent (see the whole room regardless of width) via the user's own specified mechanism (panning) rather than the reference's (zoom). Live-confirmed drag-direction-to-reveal-side mapping, that HUD elements (status bar, ticket buttons) stay fixed regardless of pan (no camera reset needed afterward), and that one drag already reaches the true extreme (extra repeats are a confirmed no-op, kept as a safety margin). **Confirmed live with a real game-state change**: a full run patted a real sparkle in room 2 (score 0.997) specifically after panning to an extreme, while room 1 found nothing that run (expected — per-student cooldown). Clean end-to-end completion, no warnings. **Invite ticket cooldown bugfix (2026-07-14)**: real-usage report (with screenshot) that the invite step kept failing once the account's ticket went on cooldown — `_open_invite_list`'s original check (`not is_on_subscreen`) couldn't distinguish the real MomoTalk list opening from a "通知" cooldown notice ("待機時間が経過した後に、再度招待することができます。") opening directly instead, since both dim the header the same way; misread as "list opened", it sent `_ensure_invite_sort`/`_try_invite_row`'s fixed coordinates into a dialog that has none of them. Fixed by checking `navigation.is_modal_open` (the darker real-dialog reading) first — a dialog appearing before any row is clicked can only mean the ticket click raised one directly — and dismissing it via the shared `SWEEP_CONFIRM_BUTTON`, returning `False` so the existing "skip this room's invite" fallback handles it. **Confirmed live**: both rooms correctly detected and dismissed the cooldown notice with no cascading errors, task completed cleanly (exit 0, true home screen confirmed via screenshot). See `plan.md`'s Phase 6 follow-up #4. **Two further real-usage fixes (2026-07-15, `next_fix.md`)**: (1) the cooldown-notice dismiss above now presses Escape and verifies via `navigation.is_modal_open` with bounded retry, instead of a fixed-coordinate click that could miss and leave the notice open under the following pan drags; (2) `_dismiss_rank_up_if_shown` switched from `navigation.is_on_subscreen`'s single-pixel header probe (found live to misread some characters' rank-up cutscene art as bright) to a new `navigation.is_header_bar_visible`, requiring 8 spread-out header-row points to all read bright via one atomic multi-point capture (`driver.colors_at`, added alongside it). Both best-effort — not yet re-confirmed live post-fix. **A real rank-up happened naturally via cron (2026-07-16)**, reported live with a screenshot of the game left stuck on the cutscene — exposed a third bug, a timing gap rather than a threshold problem: `_dismiss_rank_up_if_shown` was only ever called once, immediately after a pat, with no wait beforehand, but the cutscene renders with its own client-side animation delay, so that single check could catch the tail end of the still-normal room view and conclude "clear" a beat before the actual cutscene appeared. Confirmed via a live pixel-check that the resulting stuck cutscene reads under `is_on_subscreen`/`is_modal_open`'s thresholds too (`SUBSCREEN_HEADER_PROBE` read `(180,227,244)`, r=180 < the 200 both checks need), so `navigation.return_to_home`'s generic cleanup also couldn't recover it — every "are we home" check downstream falsely agreed nothing was wrong, and the whole `q4h` cron run finished "successfully" with the game actually left stuck. Fixed by checking on every `_pat_current_view` poll iteration instead of only right after a pat, so a delayed cutscene appearance is caught (and dismissed) roughly a second later instead of never. **Confirmed live**: manually reproduced the exact stuck state left by the incident and confirmed a single Enter (the same action `_dismiss_rank_up_if_shown` already takes) cleared it back to the real cafe room view; the per-iteration check itself has not yet been re-exercised against a fresh live rank-up trigger end-to-end (same "could not force one on demand" caveat as the original fix). **`SPARKLE_CLICK_OFFSET` recalibrated (2026-07-17)**: reported live with a screenshot (`screenshots/cafe/bug_student_close_together.png`) — when two students stand close together, the original `(75, 47)` offset (ported byte-for-byte from `scripts/detect_and_click.py`, never independently verified) overshoots past the intended target's head and lands on a different, closer student instead; the click doesn't register, so `find_cafe_sparkle()` matches the same still-showing sparkle again next iteration, repeatedly clicking nearly the same point for the rest of the room's budget (confirmed live: 13 of 16 "pats" in one run clustered within a ~10px box). First recalibration attempt, `(0, 0)` (raw template-match center), was also wrong — reported live ("clicking the actual sparkle instead of the student head"): a coincidental ambient thought-bubble animation at the test position had been misread as a hover confirmation. Properly recalibrated via numbered candidate-point overlays (screenshots with several labeled offset options drawn on a real live sparkle; the user picked the correct one directly on-screen) against two independent real students in different poses (sitting at an arcade cabinet, lying on a couch) — both converged on the same small offset, `(51, 15)`. **Confirmed live** via the real `./ba_dailies.sh cafe` CLI path: 6 sparkles patted across both rooms at genuinely distinct coordinates with zero repeated-click clustering, clean `[cafe] Done.` and true-home return. **Room 2 now invites lowest affection, plus invited-student name logging (2026-08-13, see `plan.md` Phase 31)**: per explicit user direction, room 1 keeps inviting highest affection (`_invite_student(..., prefer_highest=True)`, the default) while room 2 now invites lowest (`prefer_highest=False`) by sorting the MomoTalk list ascending instead of descending — same row-walking/skip logic either way, no reference equivalent (`invite_by_affection` only ever applies one direction per call and never logs back a name). New `CAFE_INVITE_NAME_RECT_X`/`CAFE_INVITE_NAME_RECT_Y_OFFSET` OCR rect, live-calibrated against a real open invite list; psm=7 (this project's usual single-line default) badly garbled short 2-character names ("ミカ" → `"' ーー、テテー"`), fixed by switching to psm=8 (single word) plus a punctuation strip, re-verified live reading all 5 sampled rows exactly correct. **Real live "screen not confirmed" failure investigated 2026-09-19** (reported by the user: "cafe currently full of resource, never claimed"): every real logged run across two days (4/4 `daily` fires, 10/11 `q4h` fires) hit `_dismiss_rank_up_if_shown`'s "cafe screen not confirmed (rank-up cutscene stuck?)" warning on the very first pat-loop iteration in both rooms, then `_enter_room`'s room-switch also failed, so income was never claimed. Extensive live debugging (superpowers:systematic-debugging — fresh game kill+relaunch, the full `login`→`cafe` sequence, the cooldown-dialog-dismiss path, and a real supervised `q4h` run through the actual cron wrapper, all included) could **not** reproduce the failure even once (3 manual reproductions plus one full live `q4h` run all completed cleanly) despite matching the real preset's task order as closely as possible. Ruled out: `DISPLAY`/`XAUTHORITY` env handling (already hardcoded correctly in `config.ENV`, unaffected by cron's minimal environment), and a "game always cold-launches" theory (both `daily` and `q4h` end with `exit_game`, so every real fire does start from a freshly-relaunched game, but forcing that same cold-launch manually still didn't reproduce it). Real, unrelated things found along the way: the `q4h` crontab's actual schedule (`0 1,10,13,17,21 * * *`) doesn't match its own documented `1,5,9,13,17,21` — confirmed **intentional** by the user (changed to avoid their own active play time), not a bug; and a real screenshot-capture crash (repeated libpng CRC errors, `RuntimeError` after 3 retries) traced to running a manual task while a scheduled `q4h` fire was concurrently active on the same game window — a real hazard of manual testing overlapping a cron fire, not a code bug, left unaddressed per explicit user direction ("No need to fix this, leave it as it be"). Since the exact interrupting state couldn't be captured live, `_dismiss_rank_up_if_shown` now saves a diagnostic screenshot to `scratchpad/` (`cafe_diag_not_confirmed_.png`) on its own final failure, so the next real occurrence leaves hard evidence instead of log text alone. **Sparkle-detection redesigned the same day, per explicit user direction**: the original design took one screenshot and matched only the single best sparkle per click attempt, polling up to a fixed `CAFE_MAX_CLICKS_PER_ROOM` (15) budget regardless of how many sparkles were actually visible at once — correct, but wasteful (confirmed live: ~50s per view, ~100s per room, when nothing was available to pat). `detector.find_cafe_sparkles` (replacing `find_cafe_sparkle`) now finds every sparkle above threshold in one frame via per-scale `np.where` plus a proximity-based dedupe (`SPARKLE_DEDUPE_DISTANCE`, not yet live-calibrated against two genuinely distinct close-together sparkles), and `_pat_current_view` pats every match found in one screenshot before recapturing, stopping once `CAFE_EMPTY_ROUNDS_TO_STOP` (2) consecutive rounds each find nothing rather than exhausting a fixed click budget. Patting a whole batch from one stale screenshot is safe here specifically because cafe sparkles sit on static room students, not a reflowing list like Common Shop's item grid (see that row's own 2026-09-17 incident for the contrasting case). **Confirmed live** via a real `./ba_dailies.sh cafe` run post-fix: both rooms completed correctly (no sparkles happened to be available that run, so the "pat multiple from one screenshot" path itself is implemented-but-not-yet-exercised against a real multi-sparkle frame), but the empty-room case that used to take ~100s per room now completes in ~14s — confirmed via real log timestamps, a >85% reduction for that case. The `q4h` run this same investigation used to try reproducing the original bug also completed with zero anomalies using the code as it stood *before* this redesign, so that run doesn't independently confirm the redesign itself — only the standalone post-deploy run does. **Real live bug, 2026-09-24, reported by the user with an exact log** ("it will press Esc many times until it returns home"): reproduced twice in a row on demand (the account's invite ticket happened to be on cooldown both times). Root-caused with a live state trace (screenshot + `is_on_subscreen`/`is_modal_open`/`is_header_bar_visible` check at every step): `is_modal_open`'s single dark-pixel probe read `True` for one extra frame right as the cooldown notice's own closing animation finished — even though the room behind it was ALREADY back to a completely clean, sparkle-visible state on that same frame (`is_on_subscreen` and `is_header_bar_visible` both `True`, confirmed by a saved screenshot). Two places in `_open_invite_list`/`_invite_student` trusted `is_modal_open` alone as their sole signal to press Escape again, and that second, unnecessary Escape — pressed against an already-fine room — is exactly the same class of hazard CLAUDE.md already documents for a stray Escape on the true home screen, just one screen up: it backed the whole flow out of the cafe room entirely, confirmed by a second trace screenshot catching the game's own "Now Loading..." transition back toward home mid-flight. `_dismiss_rank_up_if_shown`'s later, unrelated blind Enter-press loop (correctly trying to recover from what looked like a stuck cutscene) then had nothing real to recover from, since the actual problem was upstream, and gave up as designed — the true fix was upstream of that function entirely, which is why its own diagnostic screenshot (added back in the 2026-09-19 investigation that first suspected but couldn't reproduce this exact cooldown-dialog-dismiss path) kept showing a perfectly clean home screen with nothing wrong on it. Fixed by requiring `is_header_bar_visible` to also read `False` before either location presses Escape again — a much stronger positive signal, since it needs 8 points spread across the full screen width to all read bright, so it stays reliably `False` while a real notice's dimming overlay is up but flips `True` the instant the room is genuinely clear, unlike `is_modal_open`'s single probe point. A separate, real architectural gap found along the way and fixed first: `_invite_student`'s cooldown-notice path used to `return` immediately after dismissing the notice, skipping the same room-state cleanup the normal invite-list path always ran — fixed by restructuring so the cleanup always runs regardless of which path was taken. **Confirmed live end-to-end**: a real `./ba_dailies.sh cafe` run completed perfectly with both fixes in place — room 1's invite correctly skipped (cooldown) with zero extra Escape presses, 4 sparkles patted across both camera positions, a clean room switch, room 2's invite hitting the identical cooldown and handled just as cleanly, 5 more sparkles patted, income claimed, and `Done.` with zero "screen not confirmed" warnings anywhere in the log. | | Stamina/AP | `module/collect_daily_task_power.py` | `to_tasks`/`implement` | `ba_auto/tasks/stamina.py` | `color.rgb_in_range` → `driver.color_at`; reference's per-tab claim loop → live UI's single "一括受取" bulk-claim button + Enter | Migrated: Mission-panel claim done (see `plan.md` Phase 8). The gem shop's own Daily Free Power flow (`module/collect_daily_free_power.py`) is a separate task now — see the "Gem shop daily free package" row below. **Daily gem reward bugfix (2026-07-16)**: reported live by the user against a real account state — the reference's own `implement` checks TWO separate button regions (the main 一括受取 area, then a second, separate "claim daily pyroxenes" button checked independently after), but only the first was ported originally. Fixed by porting the second region too: a new `MISSION_DAILY_GEM_CLAIM_BUTTON` (config.py), reusing the same bright-yellow-vs-grey color-probe style as the existing `MISSION_CLAIM_PROBE`, clicked via `driver.click` (not Enter — unlike 一括受取, this button has no keybind shown on screen) with a click-then-verify retry loop matching this project's established discipline. **Confirmed live**: gem balance went from 12,384 → 12,404 (exactly the reward's stated +20), clean return to the true home screen, no warnings. | | Gem shop daily free package | `module/collect_daily_free_power.py` | `implement`, `to_purchase_pyroxenes_menu`, `to_purchase_type`, `detect_free_power_availability`, `collect_daily_free_power`, `return_to_main_page` | `ba_auto/tasks/gem_shop.py` | Reference detects every step via `core.picture.co_detect` + fixed-region OpenCV template matching (no OCR anywhere in this reference flow). Originally ported here as plain fixed-coordinate clicks + color probes, matching this project's own established equivalent for a simple flat-color state-A/state-B difference (see `cafe.py`'s `CLAIM_DISABLED_RGB`, `stamina.py`'s `MISSION_CLAIM_PROBE`). **Rewritten 2026-09-17**: the left sidebar and the free card's status/buy-button are now located dynamically via OCR (`detector.read_lines`, new primitive) instead of fixed coordinates/colors — see the Status column for why. | Done, rewritten. Original: live-calibrated against nik-gpu 2026-07-15, genuinely claimed the account's real free package for the day (0 yen, confirmed +10 AP / +10,000 credits). Recalibrated 2026-07-29 after a UI update replaced the old 3-top-tab overlay dialog with a real full subscreen (left sidebar + デイリー/ウィークリー/マンスリー sub-tabs under 一般商品) — every coordinate/color re-verified live at that point. **Real live failure, reported by the user 2026-09-17**: `./ba_dailies.sh gem_shop` logged "free card status not recognized after click" 3/3 attempts then aborted. Root-caused (superpowers:systematic-debugging — `scratchpad/probe_gem_shop_careful_nav.py`/`probe_gem_shop_general_tab_fixed.py`, screenshots pulled back for direct visual inspection) to the sidebar gaining a new 限定商品 (NEW) entry above 一般商品 sometime after 2026-07-29, silently shifting 一般商品 from the 3rd to the 4th slot and swapping order with 定額商品 — the old fixed `GEM_SHOP_GENERAL_PRODUCTS_TAB=(196,376)` coordinate landed on 定額商品 every single attempt, reading an unrelated card's content and never matching either status color, exactly matching the reported symptom (a genuine "pressing the wrong button" bug, not a flaky click). Per explicit user request ("make the button press dynamic. Read word then click the correct position of the word"), fixed by adding `detector.read_lines()` (OCR a region, return `(x, y, text)` per detected line) and rewriting the sidebar click, the status read, and the buy-button click to all locate their target text dynamically each run instead of trusting a fixed position — calibrated live against real screenshots (`scratchpad/probe_gem_shop_ocr_regions.py`): the sidebar and the card's own text (`一日にN回まで購入可能`, `毎日無料パッケージ`, `無料`) all OCR cleanly as vertically-stacked lines in their normal (unselected) styling; a selected/highlighted sidebar item's inverted white-on-blue-pill styling was confirmed live to OCR unreliably (garbage text, not just a clean failure) even after trying an inverted threshold, so the design avoids ever needing to re-read a label the task itself just selected (checking for a still-unselected marker instead — see `_on_general_products_tab`, which checks for ウィークリー rather than whichever subtab is selected). The デイリー/ウィークリー/マンスリー subtab row was confirmed NOT the root cause (that exact fixed coordinate landed correctly every time in live testing) and was confirmed live NOT to OCR reliably as one horizontal region (tesseract fragments side-by-side columns far worse than stacked lines), so it deliberately stays a fixed click. **Confirmed live end-to-end** via a real `./ba_dailies.sh gem_shop` run post-fix: OCR-found the sidebar tab, correctly read "available", OCR-clicked the real 無料 button, and genuinely claimed the day's package — credits went from 373,773,507 to 373,783,507 (exactly +10,000) and AP rose accordingly, confirmed via before/after screenshots. Still added to `DEFAULT_ORDER`, unchanged from before — a pure free reclaim with no choice to make. | | Normal/Hard story AP sweep | `module/explore_tasks/sweep_task.py`, `module/explore_tasks/task_utils.py` | `to_region` (ported: OCR region-number readout + delta-click), a scoped-down `swipe_search_target_str` (ported: OCR stage-row label matching), `start_sweep`'s named-outcome contract (ported via `navigation.wait_for_state`, this project's scoped `co_detect` port) | `ba_auto/tasks/story_sweep.py` | OCR region/stage-name matching, ported for real (Phase 10) — replaces Phase 9's "next-region arrow stops advancing, then random stage" heuristic; modal closed via its own X button (Escape doesn't close it; X-button position re-calibrated per stage-layout variant, see Phase 10) | Done (see `plan.md` Phase 10, supersedes Phase 9; rewritten Phase 22 to match story_sweep_hard.py's Phase 21 fixes and add its own campaign guard). Config-driven exact `(region, stage, count)` targets (`config.STORY_SWEEP_TARGETS`), not latest-region/random-stage. Opt-in only, not in default flow. **2026-07-21 rewrite (Phase 22)**: per explicit user direction ("match the hard implementation on best practices"), ported story_sweep_hard.py's fixes back into this module — an explicit Normal-tab guard (`config.NORMAL_TAB`/`_open_normal_tab`, needed now that story_sweep_hard can run earlier in the same `q4h` preset and leave Hard selected), a loop-level self-heal (`_ensure_task_screen`) replacing the old per-path "Escape if on subscreen" cleanup that would have backed out of the region browser after every target (never triggered in production since this module only ever swept one rotation target per run, but a real latent bug), MAX/+ clicks no longer requiring a visible stepper raise as proof of success (same low-AP false-negative class Hard hit), `_click_plus` now resets to a known baseline via a new `SWEEP_MIN_BUTTON` before raising (closing a latent version of bounty.py's real overspend bug that this module's original `_click_plus_and_verify` never guarded against), a newly-added `_confirm_dialog_is_sweep` OCR gate against the two-stacked-button hazard (confirmed live 2026-07-21 that Normal's own tabbed modal has the identical gold 任務開始 button beneath 掃討開始 that config.py's own `SWEEP_MAX_BUTTON` comment already documented but this module never actually guarded against), the same `clicked_any`-gated `_watch_sweep_result` fix, and a `REGION_READ_RETRIES` OCR-read retry. Also added a genuinely new campaign-active guard (`config.TASK_CAMPAIGN_BADGE_RECT`, renamed from `HARD_CAMPAIGN_BADGE_RECT` once shared) — confirmed live to be pixel-identical rect/color to Hard's own banner, just reading "任務Normalで獲得できる報酬量が2倍(+100%)になっています" instead of "Hard". Confirmed live that Normal mode has no per-stage daily clear cap (no "残り回数" indicator anywhere), so Hard's gem-refill hazard does not apply here. `story_sweep_force` added as the override command. **Live-tested for real the same day**: a real MAX sweep of the day's rotation target (30-2) succeeded cleanly end-to-end on the first attempt — campaign guard passed, OCR stage-row match succeeded despite the known leading-digit misread ("20-2" read but the "2" suffix still matched), sweep confirmed and correctly detected as `"swept"` (not the old cosmetic misreport), clean return home. AP dropped from ~139 to 1 and credits rose by 13,966, confirming a real substantial MAX sweep. | diff --git a/ba_auto/tasks/cafe.py b/ba_auto/tasks/cafe.py index f5887f6..5dd2f81 100644 --- a/ba_auto/tasks/cafe.py +++ b/ba_auto/tasks/cafe.py @@ -91,6 +91,33 @@ live screenshot pixel-check: SUBSCREEN_HEADER_PROBE read (180,227,244), r=180 downstream falsely agreed nothing was wrong. Fixed by checking on every _pat_current_view poll iteration instead of only right after a pat -- see that function's own comment. + +A 2026-09-19 investigation (see `_save_diagnostic_screenshot`'s own comment) +found this same "screen not confirmed (rank-up cutscene stuck?)" warning +firing on ~90% of real scheduled runs, but could not reproduce it on demand +across several live debugging sessions -- including explicitly trying the +cooldown-dialog-dismiss path, which never once failed to recover. Diagnostic +screenshot capture was added for the next real occurrence rather than +guessing further. That occurrence came 2026-09-24, reported live by the user +("it will press Esc many times until it returns home") with an exact log, +and reproduced twice in a row on demand this time (the account's invite +ticket happened to be on cooldown both times) -- both saved diagnostics +showed the plain, undamaged TRUE HOME SCREEN, not a rank-up cutscene at all. +Root cause: `_invite_student`'s cooldown-notice path (`_open_invite_list` +returning False) `return`ed immediately after dismissing the notice, +skipping the exact same "confirm we're back in the room, and correct it if +not" cleanup the normal invite-list path always runs. Escaping the notice +reliably closes IT (the 2026-07-15 fix above), but that was never actually +the same thing as confirming the ROOM behind it is also back to a clean +navigable state -- `_pat_room`'s very first call, `_pan_camera`, is a blind +drag with no check of its own, so whatever screen was actually left showing +got dragged across regardless. The real mechanism connecting that to +"reaches home" is `_dismiss_rank_up_if_shown`'s own blind Enter-press loop +(5 attempts, unconditional, believing anything that isn't cafe's header must +be a rank-up cutscene) -- pressing Enter repeatedly against whatever screen +that leftover state actually was, not cafe, evidently backs out through it +and beyond, all the way to home. Fixed by moving the room-state cleanup out +of the success-only branch so it always runs, cooldown or not. """ import os @@ -364,10 +391,32 @@ def _open_invite_list(driver, config): # replaced. So verify with is_modal_open and retry the press # (bounded, matching this file's own click-then-verify # convention) instead of assuming one Escape worked. + # + # Real live bug (2026-09-24, reported live: "it will press Esc + # many times until it returns home"): is_modal_open alone is not + # a safe stop condition here -- a live state trace (screenshots + # at every step) caught it reading True for one extra frame + # right as the notice's own closing animation finished, even + # though the room behind it was ALREADY back to normal by then + # (is_on_subscreen and is_header_bar_visible both True on that + # same frame, confirmed by a saved screenshot of a completely + # clean, sparkle-visible room). The loop trusted only + # is_modal_open and pressed Escape a second, unnecessary time -- + # and Escape on an already-fine room is exactly the "go back" + # press CLAUDE.md already warns is hazardous on home, just one + # level up: it backed all the way out of the cafe, confirmed by + # a second screenshot catching the game's own "Now Loading..." + # transition back to the home screen mid-flight. Fixed by also + # accepting is_header_bar_visible as a stop condition -- + # requires 8 points spread across the full screen width to all + # read bright, so it stays reliably False while a real notice's + # dimming overlay is still up, but flips True the instant the + # room is genuinely clear again, unlike is_modal_open's single + # probe point. for dismiss_attempt in range(1, ROOM_OPEN_RETRIES + 1): driver.keypress("Escape") driver.wait(1) - if not navigation.is_modal_open(driver): + if not navigation.is_modal_open(driver) or navigation.is_header_bar_visible(driver): break print(f"[cafe] cooldown notice still open after Escape (attempt {dismiss_attempt}/{ROOM_OPEN_RETRIES})") else: @@ -487,27 +536,49 @@ def _invite_student(driver, config, prefer_highest=True): any candidate that would swap an already-seated student's costume or move one in from the other room. """ - if not _open_invite_list(driver, config): + if _open_invite_list(driver, config): + _ensure_invite_sort(driver, config, descending=prefer_highest) + + invited = False + for row_index in range(len(config.CAFE_INVITE_ROW_Y)): + result, name = _try_invite_row(driver, config, row_index) + if result == "invited": + print(f"[cafe] invited {name or f'row {row_index}'} (row {row_index})") + invited = True + break + if result == "no_dialog": + break + + if not invited: + print("[cafe] no candidate could be invited without swapping/moving an existing student") + else: print("[cafe] could not open the invitation ticket list -- no ticket available or a click missed, skipping invite") - return - - _ensure_invite_sort(driver, config, descending=prefer_highest) - - invited = False - for row_index in range(len(config.CAFE_INVITE_ROW_Y)): - result, name = _try_invite_row(driver, config, row_index) - if result == "invited": - print(f"[cafe] invited {name or f'row {row_index}'} (row {row_index})") - invited = True - break - if result == "no_dialog": - break - - if not invited: - print("[cafe] no candidate could be invited without swapping/moving an existing student") # Return to the plain room view regardless of outcome. - if navigation.is_modal_open(driver): + # + # Real live bug (2026-09-24): this used to be skipped entirely on the + # cooldown-notice path above, which returned right after dismissing the + # notice without ever re-confirming the room state. Escaping the notice + # reliably clears it (see this module's own 2026-07-15 fix), but that + # alone was never verified to also leave the room itself in a clean + # is_on_subscreen state -- confirmed live: `_pat_room`'s very next call, + # `_pan_camera`, is a blind drag gesture with no state check of its own, + # and performing it against whatever screen we actually landed on + # (reproduced twice in a row, byte-for-byte identical log both times) + # cascaded through `_dismiss_rank_up_if_shown`'s blind Enter-press loop + # all the way back to the true home screen, confirmed via its own saved + # diagnostic screenshot -- not a rank-up cutscene at all. Moving this + # check out of the `if` branch so it always runs, cooldown or not, + # gives `_pat_room` a confirmed-clean room to work with either way. + # is_modal_open alone is not a safe trigger for a stray Escape here + # either -- see _open_invite_list's own 2026-09-24 fix for why: the same + # stale-for-one-frame reading, on an already-clean room, cost a real + # live run this exact "one extra Escape backs out of the whole room" + # failure a second time, from THIS check specifically, even after the + # first fix stopped _open_invite_list's own retry loop from firing it. + # Same guard applied here: only escape if the room doesn't already read + # unambiguously clear. + if navigation.is_modal_open(driver) and not navigation.is_header_bar_visible(driver): driver.keypress("Escape") driver.wait(1) if not navigation.is_on_subscreen(driver):