diff --git a/CLAUDE.md b/CLAUDE.md index cec228c..43cad29 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -659,6 +659,8 @@ Current project state: mailbox, cafe, stamina, story_sweep, shop_common, shop_ta - `ba_auto/tasks/lesson.py` sweeps every unlocked region's schedule grid (a scrollable list of 12 named regions, each opening a grid modal of up to 9 location cards), picking the highest-affection available lesson each time until lesson tickets or lessons run out - per-cell affection is read via `detector.read_int_on_heart_badge`, a dedicated OCR path for the pink/magenta heart-shaped badge — the project's normal grayscale-threshold OCR (`read_int`) misreads it, because the badge's own outline stroke survives the same threshold as the digit glyph; a "done today" portrait keeps its number and gets a green checkmark added alongside it rather than losing the number, so done-ness is checked via that checkmark's color, not inferred from a failed OCR read - lesson was live-tested with real tickets spent (see `plan.md` Phase 12), which surfaced two real bugs from that assumption gap plus an OCR contamination issue — both fixed; see Phase 12 for the full writeup +- a later regression broke `LESSON_TICKET_OCR_RECT` entirely (a stray katakana fragment at the crop's left edge made tesseract drop the leading digit, `"7/7"` reading as `"/7"`, aborting every run) — fixed by tightening the rect; re-validated live with 7 real tickets spent and correct re-reads after every schedule. See `plan.md`'s "Phase 12 follow-up" +- a second regression then surfaced: clicking the schedule icon doesn't always land on the Location Select list — the game can resume directly on whichever region's per-region isometric map was last open (a previous run's Ctrl-C interruption left it stuck there), which broke navigation for every region identically since the list's scroll/row-click logic doesn't apply to that screen. Fixed via `lesson._ensure_location_select_list`, which detects the per-region map's own "all schedules" button already showing and returns via the back button before sweeping. Validated by deliberately reproducing the stuck state and confirming recovery with a real ticket spend. See `plan.md`'s "Phase 12 follow-up #2" - `ba_auto/tasks/arena.py` fights exactly one ranked Tactical Challenge (Arena) battle per invocation (not "spend every ticket" — the reference itself only fights one per call, relying on its own background scheduler for pacing, which this project has no equivalent for), then collects both reward slots - Tactical Challenge is reached via a card inside the お仕事 (Work) hub, not a bottom-nav icon; the reference's separate opponent-info and formation-edit screens are merged into one modal here with a live ticket-count preview confirming the real fight-commit click - arena was live-tested for real across all 5 of the account's daily tickets (2 WIN, 1 LOSE, 2 spent debugging), which surfaced three real bugs: a level-OCR crop too small for tesseract despite looking legible to the eye, level text being bright-on-dark unlike every other OCR read in this project (fixed via a new `detector.read_int_white_on_dark`), and — most importantly — the post-fight WIN/LOSE result modal proving undetectable by precisely locating its own confirm button (WIN and LOSE are different heights; widening the search region to cover both then caught stray cyan-ish pixels in the opponent list's own portrait art, false-positive-clicking into an unrelated opponent's info modal). Fixed by abandoning per-button color detection for a bounded blind-Enter-press loop (matching `lesson.py`'s own `_run_one_schedule` pattern), gated by a hard safety check against the one modal where Enter is genuinely dangerous — the opponent-info modal's own attack-formation button is also Enter-bound and spends a real ticket. See `plan.md` Phase 13 for the full writeup diff --git a/ba_auto/config.py b/ba_auto/config.py index c04a555..c1a77c2 100644 --- a/ba_auto/config.py +++ b/ba_auto/config.py @@ -288,6 +288,13 @@ TACTICAL_SHOP_TARGETS = [ LESSON_ICON = (314, 1100) # bottom nav "スケジュール" icon on the home screen LESSON_BACK_BUTTON = (85, 55) # shared by both the region-list and per-region map screens +# Clicking LESSON_ICON does not always land on the Location Select list -- +# confirmed live: the game remembers the last-viewed region and reopens +# directly to its per-region isometric map instead (e.g. after a previous run +# was interrupted mid-region). lesson.py's _ensure_location_select_list +# detects and recovers from this via LESSON_ALL_SCHEDULES_BUTTON below +# already being visible before any row has been clicked. + LESSON_REGION_NAMES = [ "シャーレオフィス", "シャーレ居住区", "ゲヘナ学園・中央区", "アビドス高等学校", "ミレニアム・スタディーエリア", "トリニティ・スクエア", "レッドウインター連邦学園", @@ -340,7 +347,15 @@ LESSON_GRID_CHECKMARK_HALF_SIZE = (14, 13) LESSON_GRID_BADGE_MAX_PLAUSIBLE = 99 # "保有チケット N/M" readout, top-left of the region-list/map screens. -LESSON_TICKET_OCR_RECT = (295, 133, 385, 172) +# The original rect (295, 133, 385, 172) clipped in a stray fragment of the +# "チケット" label's own trailing katakana glyph a few pixels left of the +# first digit -- confirmed live: it read back as "/7" (whole leading digit +# dropped) instead of "7/7", reproducible across a fresh screenshot every +# time, not a one-off render glitch. Tightening the left edge past that +# fragment (and the right edge in to match) fixed it; re-confirmed stable +# across 6 consecutive fresh reads. See Handoff.md for the live debugging +# session that found this. +LESSON_TICKET_OCR_RECT = (315, 135, 375, 170) # Per-cell info panel ("スケジュール情報"), opened by clicking a grid cell. # Its Start button and the post-schedule report's OK button (below) are the diff --git a/ba_auto/reference_notes/mapping.md b/ba_auto/reference_notes/mapping.md index 0012721..8ee6e63 100644 --- a/ba_auto/reference_notes/mapping.md +++ b/ba_auto/reference_notes/mapping.md @@ -14,6 +14,6 @@ Maps each local feature to the corresponding `~/repo/baas-reference/module/...` | Arena | `module/arena.py` | `implement` (main flow), `to_tactical_challenge` (nav from main page), `get_tickets` (ticket-count OCR), `choose_enemy` (self/opponent level OCR + bounded refresh-reroll loop), `check_skip_button` (skip-toggle color probe), `fight` (click fight, wait for win/lose), `collect_tactical_challenge_reward` (two reward-slot color probes) | `ba_auto/tasks/arena.py` | Ticket count/self level/opponent level/rank are plain digit OCR (`detector.read_int`/`read_text`, plus a new `read_int_white_on_dark` for the profile card's bright-on-dark level text). Skip-toggle state and the two reward-slot claimed-vs-claimable colors are plain pixel-color probes (`driver.color_at`/`_color_in_range`). The post-fight WIN/LOSE result modal and an unrelated list-refresh-expired notice are NOT detected by precisely locating a button (a color-region search proved unreliable — see Status); they're dismissed via a bounded blind-Enter-press loop matching `lesson.py`'s own `_run_one_schedule` pattern, gated by a hard safety check against the one modal where Enter is dangerous (opponent-info's own attack-formation button, checked via its fixed-position gold button). `choose_enemy`'s refresh-reroll loop is direct Python control flow, bounded by `maxArenaRefreshTimes`. Config knobs carried over from reference defaults: `ArenaComponentNumber`=1, `ArenaLevelDiff`=0, `maxArenaRefreshTimes`=10, `ArenaStopFightWhenRank1`=False | Done. Live-tested for real across all 5 of the account's daily tickets (2 WIN, 1 LOSE, 2 spent debugging the result-modal detection — see `plan.md` Phase 13 for the full writeup). Real navigation differences confirmed live: Tactical Challenge is a Work-hub card, not a bottom-nav icon; the reference's separate opponent-info and formation-edit screens are merged into one modal here with a live ticket-preview; `navigation.is_modal_open`'s shared probe reads *inverted* on this screen (own `_is_modal_open` via `config.ARENA_MODAL_PROBE`). Three real bugs fixed: a level-OCR crop too small for tesseract despite looking legible (fixed by widening the crop, not the pipeline); level text being bright-on-dark unlike every other OCR read in this project (fixed via `read_int_white_on_dark`); and the result-modal detection cycling through two failed color-based designs (fixed by switching to bounded blind-Enter dismissal with a hard safety gate — a real near-miss of the same "mistimed keypress" hazard class `CLAUDE.md` already documents from story_sweep). Deliberately opt-in only, never in `DEFAULT_ORDER` — unlike every other opt-in task so far (which spend a known-safe resource on a config-driven target list), this one fights a real ranked PvP battle that can win or lose and moves the account's actual arena rank. Per explicit user decision: fights exactly one battle per invocation, matching the reference's own per-call pacing (its `next_time = 55` background-thread rescheduling has no equivalent in this project's one-shot CLI). Not yet exercised live: the "no ticket" mid-flow race, an actual reroll click (every opponent offered was already an acceptable level), and `ArenaStopFightWhenRank1`'s rank-1 stop condition — all implemented per the reference's logic, just not yet hit by real game state. `detector.find_template`/`template_visible` (generalized named-template matcher, built during scaffolding) ended up unused — state detection stayed OCR/color-probe-driven throughout, like every other task in this project. | | Common Shop | `module/shop/common_shop.py`, `module/shop/shop_utils.py` | `implement`, `to_common_shop`, `get_item_position`/`ensure_choose`/`buy` (shared, see Tactical Shop row) | `ba_auto/tasks/shop_common.py`, `ba_auto/tasks/shop_utils.py` | `get_item_position`'s color+template item-state scan → fixed grid-position targets (`config.COMMON_SHOP_TARGETS`) + price-digit OCR verify, since the reference's own item-identification here indexes an external static price table (`self.static_config.common_shop_price_list`, fetched from a remote resource) this repo doesn't have — not per-item OCR, so this isn't an OCR-avoidance shortcut. Purchase-confirm dialog + reward-acquired banner handled via a single overlay-darkness probe (`config.SHOP_OVERLAY_PROBE`) instead of tracking each dialog's own layout | Done. Live-tested with real purchases (all 8 configured targets bought, cost matched exactly). Discovered live: these items have a per-refresh-cycle purchase cap not shown as a visible counter (unlike the 青輝石 tab's "あと1回購入可能" labels) — confirmed by re-running the task after purchase and observing it correctly detect the now-unselectable items (checkbox + individual 購入 button both unresponsive) and safely decline rather than guess. A fresh, everything-available run hasn't been re-verified since the account had already exhausted this cycle's purchases via that same test | | Tactical Shop | `module/shop/tactical_challenge_shop.py`, `module/shop/shop_utils.py` | `implement`, `goto_shop_by_name`, shared `get_item_position`/`ensure_choose`/`buy` | `ba_auto/tasks/shop_tactical.py`, `ba_auto/tasks/shop_utils.py` | `goto_shop_by_name`'s OCR swipe-search over the shop-tab list → fixed click (`config.SHOP_TAB_TACTICAL`): this account's tab list is only 7 entries and fits on screen with no scroll needed, confirmed live, so there's nothing to search for — not an OCR-avoidance shortcut. Same grid-position + price-OCR-verify + overlay-probe design as Common Shop, sharing `shop_utils.run_shop_tab` | Done. Live-tested with real purchases (both configured AP-recovery drinks bought; AP and tactical-coin balance changes matched exactly) | -| Lesson/Schedule | `module/lesson.py` | `implement`, `to_lesson_location_select`/`to_select_location`/`to_all_locations` (nav state machine), `get_lesson_region_num`/`switch_lesson_region_page`/`to_lesson_region` (paged region nav), `get_lesson_each_region_status`+`check_region_availability` (per-cell status via isometric-parallelogram pixel scan), `get_lesson_relationship_counts` (per-cell affection pip count via color count), `choose_lesson` (selection policy), `execute_lesson`/`to_location_info`/`start_lesson` (click cell -> info panel -> start -> result) | `ba_auto/tasks/lesson.py` | `picture.co_detect` -> `navigation.wait_for_state`-style bounded Enter-press loop (see below); the reference's paged-arrow region nav (needing OCR to know current position) -> this client renders the 12 regions as a scrollable list instead, which only ever settles at two scroll positions (`config.LESSON_REGION_ROW_Y`), so navigation is direct index-based clicking with nothing to OCR-locate; the reference's isometric `Parallelogram`/`Triangle` per-cell scan (tuned to the reference's own screen layout) -> reading each portrait's heart-shaped affection badge via a dedicated OCR path (`detector.read_int_on_heart_badge`) needs no isometric geometry at all | Done. Config-driven scope only in the sense of the *policy* (affection-first selection, sweep every unlocked region until tickets/lessons run out, no ticket purchasing, no favor-student targeting) -- unlike shop, no user-specific target list was needed since the reference's own `lesson_region_name.JP` (embedded directly in its `default_config.py`, not externally fetched) already names all 12 regions, used here only for logging. Live-tested for real: 5 real tickets spent across 3 regions with correct outcomes (ticket count, cleanup navigation, home-screen return all verified). Two real bugs were found and fixed from that run -- see below and `plan.md`'s Lesson phase | +| Lesson/Schedule | `module/lesson.py` | `implement`, `to_lesson_location_select`/`to_select_location`/`to_all_locations` (nav state machine), `get_lesson_region_num`/`switch_lesson_region_page`/`to_lesson_region` (paged region nav), `get_lesson_each_region_status`+`check_region_availability` (per-cell status via isometric-parallelogram pixel scan), `get_lesson_relationship_counts` (per-cell affection pip count via color count), `choose_lesson` (selection policy), `execute_lesson`/`to_location_info`/`start_lesson` (click cell -> info panel -> start -> result) | `ba_auto/tasks/lesson.py` | `picture.co_detect` -> `navigation.wait_for_state`-style bounded Enter-press loop (see below); the reference's paged-arrow region nav (needing OCR to know current position) -> this client renders the 12 regions as a scrollable list instead, which only ever settles at two scroll positions (`config.LESSON_REGION_ROW_Y`), so navigation is direct index-based clicking with nothing to OCR-locate; the reference's isometric `Parallelogram`/`Triangle` per-cell scan (tuned to the reference's own screen layout) -> reading each portrait's heart-shaped affection badge via a dedicated OCR path (`detector.read_int_on_heart_badge`) needs no isometric geometry at all | Done. Config-driven scope only in the sense of the *policy* (affection-first selection, sweep every unlocked region until tickets/lessons run out, no ticket purchasing, no favor-student targeting) -- unlike shop, no user-specific target list was needed since the reference's own `lesson_region_name.JP` (embedded directly in its `default_config.py`, not externally fetched) already names all 12 regions, used here only for logging. Live-tested for real: 5 real tickets spent across 3 regions with correct outcomes (ticket count, cleanup navigation, home-screen return all verified). Two real bugs were found and fixed from that run -- see below and `plan.md`'s Lesson phase. **Regression fix (Phase 12 follow-up)**: `LESSON_TICKET_OCR_RECT`'s left edge clipped in a stray katakana fragment next to the first digit, making tesseract drop the whole leading digit (`"7/7"` -> `"/7"`) and aborting every run outright; fixed by tightening the rect, re-validated with a full real run (7 tickets spent, correct count re-read after every schedule, clean stop and home-screen return). **Regression fix (Phase 12 follow-up #2)**: clicking the schedule icon doesn't always land on the Location Select list -- the game can resume directly on whichever region's per-region isometric map was last open (confirmed live: a previous run's Ctrl-C interruption left it stuck there, breaking `_open_region_grid` for every region in the next run identically). Fixed via a new `lesson._ensure_location_select_list` recovery check (detects the per-region map's own "すべてのスケジュール" button already showing before any row's been clicked, and returns via the back button if so); validated by deliberately reproducing the stuck state and confirming a real run recovered, spent the account's real remaining ticket, and finished cleanly | Do not implement a feature without filling at least the relevant row. diff --git a/ba_auto/tasks/lesson.py b/ba_auto/tasks/lesson.py index 030fe40..3a96b16 100644 --- a/ba_auto/tasks/lesson.py +++ b/ba_auto/tasks/lesson.py @@ -53,12 +53,41 @@ def _read_ticket_count(driver, config): return int(digits) if digits else None +def _ensure_location_select_list(driver, config): + """Recover to the Location Select list if the schedule icon resumed + directly on a specific region's isometric map instead. + + Confirmed live: the game remembers the last-viewed region and reopens + directly to its per-region map when the schedule icon is clicked again, + rather than always landing on the Location Select list -- e.g. after a + previous run was interrupted (Ctrl-C) mid-region. Every region-index-based + click in this module assumes it's starting from the list, so a resumed + per-region map silently breaks navigation for every region, not just the + one that was open (confirmed live: this made `_open_region_grid` fail all + 3 retries for every single region in the sweep, since the list's + scroll/row-click coordinates don't do anything useful on that screen). + + Detected via the per-region map's own "すべてのスケジュール" button + already being visible before any row here has been clicked -- that button + only exists on the per-region screen, never on the list (confirmed by + direct pixel sample: the same coordinate reads as plain dark background on + the list screen). + """ + for attempt in range(1, OPEN_RETRIES + 1): + if not _is_action_button_showing(driver, config, config.LESSON_ALL_SCHEDULES_BUTTON): + return True + print(f"[lesson] schedule screen resumed on a specific region's map instead of the Location Select list -- returning (attempt {attempt}/{OPEN_RETRIES})") + driver.click(*config.LESSON_BACK_BUTTON) + driver.wait(1.5) + return not _is_action_button_showing(driver, config, config.LESSON_ALL_SCHEDULES_BUTTON) + + def _open_schedule_screen(driver, config): for attempt in range(1, OPEN_RETRIES + 1): driver.click(*config.LESSON_ICON) driver.wait(2) if navigation.is_on_subscreen(driver): - return True + return _ensure_location_select_list(driver, config) print(f"[lesson] schedule screen not detected after click (attempt {attempt}/{OPEN_RETRIES})") return False diff --git a/plan.md b/plan.md index 33b357e..ec3a08e 100644 --- a/plan.md +++ b/plan.md @@ -408,6 +408,30 @@ Also confirmed live: the "保有チケット N/M" ticket counter is directly vis **Not verified**: a region with more than 9 currently-unlocked locations (would need scroll support inside the grid modal — not implemented, not yet seen on this account, both regions tested topped out at 7-8); the "no lesson tickets" abort-immediately path (tickets hit exactly 0 mid-sweep during the real test, not at the start); a full 12-region sweep in one run (the test's 5 tickets ran out partway through region 3 of 12); a truly fresh "everything available, nothing done yet" run (this account had already done some lessons manually during calibration before the automated run started). Worth re-running `~/ba_dailies.sh lesson` after tickets next refill to exercise the untested tail of the region list. +### Phase 12 follow-up: ticket-count OCR regression fix + +Between sessions, `~/ba_dailies.sh lesson` started failing immediately every run with `[lesson] could not OCR ticket count, aborting without pressing further keys` — a full regression of a read that worked during Phase 12's own live test. + +Root cause, confirmed by pulling a live screenshot back to `scratchpad/` and testing `detector.read_text` against the exact configured rect in isolation: `LESSON_TICKET_OCR_RECT`'s left edge (`x=295`) clipped in a stray few pixels of the "チケット" label's own trailing katakana glyph, immediately adjacent to the first digit. That fragment was enough to make tesseract drop the whole leading digit from its read — `"7/7"` came back as `"/7"`, confirmed reproducible across a fresh screenshot every single time (not a one-off render glitch, not a transient game-side layout change). Splitting `"/7"` on `/` gives an empty head, so `_read_ticket_count` returned `None`, exactly matching the failure. + +Fixed by tightening the rect to `(315, 135, 375, 170)` — pixel-column analysis of the crop located the actual digit glyphs' bounding box and the new rect starts past the stray fragment. Re-confirmed stable across 6 consecutive fresh OCR reads before deploying, then validated with a full real run: 7 real lesson tickets correctly spent, ticket count correctly re-read after every single schedule (7→6→5→4→3→2→1→0), clean stop at 0, and a clean return to the home screen confirmed via a final screenshot. + +This is the same class of fragility as the heart-badge OCR contamination fix from the original Phase 12 writeup above (a plausible-looking crop still picking up unrelated UI content at its edge) — worth keeping in mind for any other tightly-cropped OCR rect in this project if a similarly "worked before, fails now" regression shows up elsewhere. + +### Phase 12 follow-up #2: schedule-icon navigation regression fix ("stuck on a region map") + +Immediately after the ticket-OCR fix above, a full real run (`./ba_dailies.sh lesson`) still failed completely — this time past ticket reading (`starting tickets: 1` printed correctly), but `[lesson] schedule grid not detected for region index N (attempt 1-3/3)` for literally every region in sequence, 0 through 8, until the user Ctrl-C'd. The user's own suspicion was that `driver.click` was doing a click-and-hold instead of a single click; `driver.click`'s implementation was checked and is an ordinary `mousemove` + `click 1`, unchanged — that wasn't it. + +Root cause, confirmed live by screenshotting the actual screen right after clicking `LESSON_ICON`: **the game does not always land on the Location Select list when the schedule icon is clicked.** It remembers the last-viewed region and reopens directly to that region's per-region isometric map instead — confirmed by reproducing the exact scenario (manually opening a region's map, then running the real task without returning to the list first) and seeing the identical failure signature. The previous session's run had been left stuck exactly like this by its own Ctrl-C interruption, mid-sweep, on some region's map — every subsequent `_open_region_grid` call for every region_index then fired its scroll/row-click sequence against that same stuck per-region map screen, which doesn't respond to any of it the way the list does, so nothing ever matched and every region failed identically. + +This isn't a one-off leftover-state fluke either: since the game itself decides whether the schedule icon resumes on a region or resets to the list, any future interruption (or even manual browsing) before a run could reproduce it again. + +Fixed with a new `lesson._ensure_location_select_list`, called right after `_open_schedule_screen` confirms a subscreen is open: it checks whether `LESSON_ALL_SCHEDULES_BUTTON`'s position already shows that button's known color *before any row here has been clicked* — that button only exists on the per-region map, never on the list (confirmed by direct pixel sample: same coordinate reads as plain dark background on the list) — and if so, presses `LESSON_BACK_BUTTON` (bounded, `OPEN_RETRIES` attempts) to return to the list before the sweep begins. + +Validated live in two passes: first, a full real run from the recovered list (7→...→0 tickets from the earlier ticket-OCR fix test) confirmed the list-based flow itself was never broken. Second, the actual regression was reproduced on purpose — manually left the game on a region's per-region map, then ran the real task — and the new recovery step printed `schedule screen resumed on a specific region's map instead of the Location Select list -- returning`, correctly returned to the list, read the account's 1 remaining real ticket, spent it on a real schedule, and finished cleanly with a confirmed clean return to the home screen. + +This is the same class of finding CLAUDE.md already documents from mailbox/cafe's fixed-coordinate click flakiness and story_sweep's OCR-based navigation: a screen's identity can't be safely assumed from "what button did we intend to click here," it has to be verified — this task's `_open_schedule_screen` had gotten away without that check only because the list had always been the landing screen in every session up to now. + ### Phase 13: Arena / Tactical Challenge **Status: Done.** Ported `module/arena.py`'s `implement` flow (`get_tickets`, `choose_enemy`, `check_skip_button`, `fight`, `collect_tactical_challenge_reward`) to `ba_auto/tasks/arena.py`. Live-tested for real across all 5 of the account's daily tickets — 3 real fights (2 WIN, 1 LOSE), plus real reward claims (credits, pyroxene, tactical coin all changed as expected).