From dc0e8403a16108d9ef571d8ed15aea395c88fe1b Mon Sep 17 00:00:00 2001 From: Nik Afiq Date: Wed, 15 Jul 2026 18:22:33 +0900 Subject: [PATCH] feat: Implement gem shop daily free package automation - Added `ba_auto/tasks/gem_shop.py` to automate claiming the daily free package from the gem shop. - Integrated color probes for state detection instead of template matching. - Updated `ba_daily.py` to include the new gem shop task in the default execution order. - Documented the implementation and calibration process in `plan.md`, including live testing results and fixes for dialog handling. - Addressed potential bugs related to dialog state verification and tab layout changes. --- ba_auto/config.py | 103 +++++++++++++++++++++ ba_auto/reference_notes/mapping.md | 3 +- ba_auto/tasks/gem_shop.py | 143 +++++++++++++++++++++++++++++ ba_daily.py | 10 +- plan.md | 37 ++++++++ 5 files changed, 292 insertions(+), 4 deletions(-) create mode 100644 ba_auto/tasks/gem_shop.py diff --git a/ba_auto/config.py b/ba_auto/config.py index 3d96c5b..9de89fe 100644 --- a/ba_auto/config.py +++ b/ba_auto/config.py @@ -1045,3 +1045,106 @@ BOUNTY_RESULT_BUTTON_MIN_PIXELS = 3000 # areas, choose random (mod%3), run the latest stage available." Same # date-ordinal-modulo scheme as STORY_SWEEP_ROTATION_*/EVENT_SWEEP_ROTATION_*. BOUNTY_SWEEP_COUNT = "max" + +# Gem shop daily free package (毎日無料パッケージ), ported from +# module/collect_daily_free_power.py. Reference reads: home-screen icon -> +# purchase-pyroxenes dialog -> パッケージ (package) tab -> the FREE card at a +# fixed position -> confirm-purchase notice -> reward. The reference detects +# every step via image template matching (no OCR anywhere in this flow) -- +# this port uses plain color probes instead, matching this project's own +# established equivalent for simple enabled/disabled or state-A/state-B +# visual differences (see cafe.py's CLAIM_DISABLED_RGB, stamina.py's +# MISSION_CLAIM_PROBE) rather than building new template assets, since every +# state below reduces to a clean, high-contrast flat color rather than a +# complex shape needing find_cafe_sparkle-style matching. +# +# All coordinates/colors below are pixel-scanned from real scrot captures on +# nik-gpu at the native 1920x1200 (not estimated from the non-native-resolution +# screenshots/gem_shop/*.png reference photos -- confirmed those are NOT 1:1 +# with real game coordinates, same finding as screenshots/cafe/student/'s own +# calibration history). Live-calibrated 2026-07-15 by manually driving the +# real dialog end-to-end (see scratchpad/gem_shop_*.png for the captures) -- +# this walk-through also genuinely claimed the account's real free package +# for the day (0 yen, +10 AP / +10,000 credits, confirmed via before/after +# counter values), so the two states below are both real, not guessed. +GEM_SHOP_ICON = (204, 345) +# パッケージ (package) tab, third of three (期間限定/青輝石/パッケージ). +# +# KNOWN FUTURE-BREAKAGE RISK (flagged in review, not yet fixable): this is a +# single fixed coordinate calibrated only against the CURRENT 3-tab layout, +# where 期間限定 (time-limited) is present as the first tab. The reference's +# own to_purchase_type branches on exactly this -- it picks a different +# パッケージ click x depending on whether 期間限定 is currently showing, +# because removing it reflows the remaining tabs. screenshots/gem_shop/2.png +# (captured live the same day as this calibration) shows 期間限定's own +# on-screen countdown ("終了まであと6日" / "26.06.24〜26.07.29"), meaning +# this account's 3-tab layout is itself temporary and due to change around +# 2026-07-21 -- not a theoretical edge case. No real screenshot of the +# resulting 2-tab (青輝石/パッケージ) layout exists yet to calibrate the +# correct alternate x, so a second coordinate has NOT been guessed in here +# rather than grounded in a real capture, per this project's own +# no-guessing-at-pixel-values discipline. +# +# This is safe to leave as a known gap rather than a live hazard: if this +# coordinate goes stale, _open_package_tab's own status-probe check reads +# "unknown" (neither the available-navy nor claimed-red color, since it'd +# be looking at the wrong tab's content) and run() aborts cleanly via +# _close_gem_shop's verified close WITHOUT ever clicking a purchase button +# on whatever tab it actually landed on -- no currency-risk path exists +# through this failure mode. Re-calibrate against a real capture once the +# tab count actually changes (or the countdown resets on a new campaign +# and this account still shows 3 tabs, meaning it wasn't actually an issue). +GEM_SHOP_PACKAGE_TAB = (1345, 330) +# 毎日無料パッケージ card's own 購入 (purchase) button -- always the +# leftmost card, top row, of the package tab (fixed position, matching the +# reference's own fixed (385, 479)-style offset rather than a search). +GEM_SHOP_FREE_CARD_BUY_BUTTON = (573, 795) + +# The free card's own status bar (just below its artwork, above the price +# button) reads a flat, highly distinct color depending on claim state -- +# confirmed live by claiming the real package and diffing before/after +# captures pixel-for-pixel at this exact point: +# available ("一日に1回まで購入可能"): flat dark navy ~(41, 65, 90) +# claimed ("一日に0回まで購入可能"): flat dark red ~(144, 38, 47) +# r vs b relationship alone cleanly separates them (navy: b>r by ~50; red: +# r>b by ~100), so no OCR of the "0"/"1" count text is needed -- the +# reference itself doesn't OCR this either, it template-matches two whole +# separate "purchasable"/"non-purchasable" card images. +GEM_SHOP_FREE_CARD_STATUS_PROBE = (420, 710) +GEM_SHOP_STATUS_AVAILABLE_RGB = (41, 65, 90) +GEM_SHOP_STATUS_CLAIMED_RGB = (144, 38, 47) +GEM_SHOP_STATUS_TOLERANCE = 20 + +# Generic "is the 青輝石購入 dialog (or a notice/confirm card nested on top +# of it) currently showing" check -- navigation.is_on_subscreen/is_modal_open +# both proved unreliable here, confirmed live: this dialog overlays directly +# on the home screen (keeping its own header bar visible, unlike mailbox/ +# cafe/shop's own full subscreen headers) so is_on_subscreen's header probe +# stays dark in both states; is_modal_open's (960, 200) probe lands on the +# dialog's own opaque white card interior rather than a dimmed backdrop, so +# it reads bright (not-dark) whether the dialog is open or not -- the exact +# same class of default-probe mismatch story_sweep.py/bounty.py already +# document for their own wide modals. +# +# These 4 points sit in the dialog card's plain gutter background (confirmed +# identical whichever of the 3 tabs is active, and also identical whether +# the base dialog or a nested confirm/notice card is showing on top) and +# read a tight, near-white, near-equal-RGB cluster there in every dialog +# state captured; the true home screen's own character art was confirmed +# NOT to coincidentally match this closely at ALL 4 spread-out points +# simultaneously (same multi-point-beats-single-point reasoning as +# navigation.is_header_bar_visible, added the same day for an unrelated +# cafe bug -- reused here independently since this dialog's false-positive +# risk is a different shape, not the header row). +GEM_SHOP_DIALOG_PROBES = ((960, 200), (960, 250), (1500, 600), (420, 600)) +GEM_SHOP_DIALOG_PROBE_MIN_CHANNEL = 220 +GEM_SHOP_DIALOG_PROBE_MAX_SPREAD = 15 + +GEM_SHOP_ICON_RETRIES = 3 +# Bounds the "press Enter, re-check the free card's status" loop that +# advances through the confirm-purchase notice and the "報酬獲得!" reward +# banner after clicking 購入 -- confirmed live the reward banner's own entry +# animation did not accept input on the first 1-2 presses while its sparkle +# animation was still playing, so this needs real patience, not just 1-2 +# tries. +GEM_SHOP_CLAIM_MAX_ATTEMPTS = 6 diff --git a/ba_auto/reference_notes/mapping.md b/ba_auto/reference_notes/mapping.md index 3d4aae4..89e2e2a 100644 --- a/ba_auto/reference_notes/mapping.md +++ b/ba_auto/reference_notes/mapping.md @@ -6,7 +6,8 @@ Maps each local feature to the corresponding `~/repo/baas-reference/module/...` |---|---|---|---|---|---| | Mailbox | `module/mail.py` | `to_mail`, `implement` | `ba_auto/tasks/mailbox.py` | tap/click via xdotool, screenshot via scrot, `color.rgb_in_range` → `driver.color_at` pixel-probe check | Migrated: real Python, state-verified via color probe (no legacy bridge) | | Cafe | `module/cafe_reward.py` | `to_cafe` (its `relationship_rank_up` popup-handling now also ported, see below), `interaction_for_cafe_solve_method3`, `collect`, `invite_girl`/`invite_by_affection`/`checkConfirmInvite` (student invitation, added 2026-07-14) | `ba_auto/tasks/cafe.py` | `picture.co_detect`/`color.rgb_in_range` → `driver.color_at` pixel-probe checks; sparkle template match ported in-process into `ba_auto/detector.py` (`find_cafe_sparkle`, now multi-scale) | Migrated: real Python, state-verified via color probes (no legacy bridge). Pat loop now polls for the full attempt budget instead of stopping on the first miss (see `plan.md` Phase 6 follow-up) — not yet confirmed against a live sparkle since none was available during testing. `_dismiss_rank_up_if_shown` reuses `navigation.is_on_subscreen` to detect and clear the full-screen bond-rank-up cutscene after a pat (see `plan.md` Phase 6 follow-up: rank-up popups) — not yet live-confirmed against a real trigger. **Student invitation (2026-07-14)**: per explicit user direction, invite a student into each room before farming it, preferring highest affection, always skipping (never confirming) a candidate that would swap an already-seated student's costume or move one in from the other room — directly ports `invite_by_affection`/`checkConfirmInvite`'s own logic with the reference's default `cafe_reward_allow_exchange_student`/`cafe_reward_allow_duplicate_invite` both `False` (no local config exists to make either configurable). Live-calibrated against nik-gpu with **zero real tickets spent** — all 3 real dialog variants (normal confirm, same-room costume-swap warning, neighboring-room move warning) found and confirmed live, every one cancelled rather than confirmed during calibration. The heart-shaped affection badge OCR reuses `detector.read_int_on_heart_badge` directly (pixel-confirmed the same widget lesson.py's own badges use — digit pixels sampled RG, matching that function's exact masking assumption) rather than building a second OCR path. Dialog-type detection OCRs the title bar and checks for either warning's own distinctive substring (`衣装`/`隣`) rather than an exact match, mirroring `event_sweep.py`'s own `"終了"` substring-match reasoning. All 5 heart-badge reads and all 3 dialog classifications verified offline against the real saved calibration screenshots before deploying (exact match, zero mismatches). **Confirmed live with real tickets spent, both rooms**: room 1 correctly skipped one 衣装替え (costume-swap) candidate then invited row 1 cleanly; room 2 correctly skipped three consecutive 隣のカフェの生徒を招待 (neighboring-room-move) candidates (expected — room 1's own invite had just taken the account's highest-affection students) then invited row 3 cleanly. Both newly-invited students were immediately patted successfully in the same run (the "newly invited student can be farmed" requirement), income was claimed, and the task returned cleanly to the true home screen with no warnings anywhere in the log. **Horizontal camera panning (2026-07-14)**: per explicit user direction ("due to my screen size... move screen most right and most left then farm"), `_pat_room` now pans the room camera to its rightmost extreme via a new `driver.drag()` primitive (this project's first — distinct from `driver.scroll()`'s wheel-based gesture, which is for list widgets, not a room camera), farms there, pans to the leftmost extreme, farms there too — no vertical panning, per the user's own instruction. Ports the reference's `zoom_out`'s underlying intent (see the whole room regardless of width) via the user's own specified mechanism (panning) rather than the reference's (zoom). Live-confirmed drag-direction-to-reveal-side mapping, that HUD elements (status bar, ticket buttons) stay fixed regardless of pan (no camera reset needed afterward), and that one drag already reaches the true extreme (extra repeats are a confirmed no-op, kept as a safety margin). **Confirmed live with a real game-state change**: a full run patted a real sparkle in room 2 (score 0.997) specifically after panning to an extreme, while room 1 found nothing that run (expected — per-student cooldown). Clean end-to-end completion, no warnings. **Invite ticket cooldown bugfix (2026-07-14)**: real-usage report (with screenshot) that the invite step kept failing once the account's ticket went on cooldown — `_open_invite_list`'s original check (`not is_on_subscreen`) couldn't distinguish the real MomoTalk list opening from a "通知" cooldown notice ("待機時間が経過した後に、再度招待することができます。") opening directly instead, since both dim the header the same way; misread as "list opened", it sent `_ensure_invite_sort`/`_try_invite_row`'s fixed coordinates into a dialog that has none of them. Fixed by checking `navigation.is_modal_open` (the darker real-dialog reading) first — a dialog appearing before any row is clicked can only mean the ticket click raised one directly — and dismissing it via the shared `SWEEP_CONFIRM_BUTTON`, returning `False` so the existing "skip this room's invite" fallback handles it. **Confirmed live**: both rooms correctly detected and dismissed the cooldown notice with no cascading errors, task completed cleanly (exit 0, true home screen confirmed via screenshot). See `plan.md`'s Phase 6 follow-up #4. **Two further real-usage fixes (2026-07-15, `next_fix.md`)**: (1) the cooldown-notice dismiss above now presses Escape and verifies via `navigation.is_modal_open` with bounded retry, instead of a fixed-coordinate click that could miss and leave the notice open under the following pan drags; (2) `_dismiss_rank_up_if_shown` switched from `navigation.is_on_subscreen`'s single-pixel header probe (found live to misread some characters' rank-up cutscene art as bright) to a new `navigation.is_header_bar_visible`, requiring 8 spread-out header-row points to all read bright via one atomic multi-point capture (`driver.colors_at`, added alongside it). Both best-effort — not yet re-confirmed live post-fix. | -| Stamina/AP | `module/collect_daily_free_power.py`, `module/collect_daily_task_power.py` | `to_tasks`/`implement` (task-power, ported); `to_purchase_pyroxenes_menu` (free-power, not ported) | `ba_auto/tasks/stamina.py` | `color.rgb_in_range` → `driver.color_at`; reference's per-tab claim loop → live UI's single "一括受取" bulk-claim button + Enter | Partially migrated: Mission-panel claim done (see `plan.md` Phase 8). Daily Free Power (real-money purchase menu) deliberately not automated | +| Stamina/AP | `module/collect_daily_task_power.py` | `to_tasks`/`implement` | `ba_auto/tasks/stamina.py` | `color.rgb_in_range` → `driver.color_at`; reference's per-tab claim loop → live UI's single "一括受取" bulk-claim button + Enter | Migrated: Mission-panel claim done (see `plan.md` Phase 8). The gem shop's own Daily Free Power flow (`module/collect_daily_free_power.py`) is a separate task now — see the "Gem shop daily free package" row below | +| Gem shop daily free package | `module/collect_daily_free_power.py` | `implement`, `to_purchase_pyroxenes_menu`, `to_purchase_type`, `detect_free_power_availability`, `collect_daily_free_power`, `return_to_main_page` | `ba_auto/tasks/gem_shop.py` | Reference detects every step via `core.picture.co_detect` + fixed-region OpenCV template matching (no OCR anywhere in this reference flow) — ported here as plain color probes instead, matching this project's own established equivalent for a simple flat-color state-A/state-B difference (see `cafe.py`'s `CLAIM_DISABLED_RGB`, `stamina.py`'s `MISSION_CLAIM_PROBE`) rather than building new template assets. The free card's own status bar reads flat dark navy (~(41,65,90)) when available vs flat dark red (~(144,38,47)) once claimed today — a clean, high-contrast signal pixel-scanned from real captures, no OCR of the "0"/"1" count text needed. A dedicated 4-point "is this dialog (or a nested confirm/notice card) showing" check (`GEM_SHOP_DIALOG_PROBES`) was needed because `navigation.is_on_subscreen`/`is_modal_open`'s default probes both proved unreliable on this dialog (it overlays the home screen directly rather than being a full subscreen, and its own opaque white card sits right on top of `is_modal_open`'s probe point) — same class of default-probe mismatch `story_sweep.py`/`bounty.py` already documented for their own wide modals, fixed the same multi-point-beats-single-point way `navigation.is_header_bar_visible` was fixed for cafe's rank-up cutscene the same day. | Done. Live-calibrated against nik-gpu 2026-07-15 by manually driving the real dialog end-to-end via raw xdotool/scrot (see `scratchpad/gem_shop_*.png`) — this **genuinely claimed the account's real free package for the day** (0 yen, confirmed +10 AP / +10,000 credits via before/after counter values: AP 64→74, credits 161,391,144→161,401,144), which calibrated both the "available" and "already claimed" visual states from real data. The actual `gem_shop.py` module was then live-tested for real against the resulting "already claimed today" state. The "available → claim" code path itself uses the same coordinates/logic already confirmed live via the manual walkthrough, but has not yet been exercised by the module itself end-to-end — worth a follow-up live check the next time the package resets and hasn't been claimed yet. Added to `DEFAULT_ORDER` (unlike most recent additions): it only ever reclaims a genuinely free, once-per-day resource with no choice to make, matching mailbox/cafe/stamina's own "reclaim something free" category rather than the "spends a resource, needs opt-in" category. | | Normal/Hard story AP sweep | `module/explore_tasks/sweep_task.py`, `module/explore_tasks/task_utils.py` | `to_region` (ported: OCR region-number readout + delta-click), a scoped-down `swipe_search_target_str` (ported: OCR stage-row label matching), `start_sweep`'s named-outcome contract (ported via `navigation.wait_for_state`, this project's scoped `co_detect` port) | `ba_auto/tasks/story_sweep.py` | OCR region/stage-name matching, ported for real (Phase 10) — replaces Phase 9's "next-region arrow stops advancing, then random stage" heuristic; MAX click verified via `SWEEP_MINUS_BUTTON_PROBE` color check (reused, still correct), modal closed via its own X button (Escape doesn't close it; X-button position re-calibrated per stage-layout variant, see Phase 10) | Done (see `plan.md` Phase 10, supersedes Phase 9). Config-driven exact `(region, stage, count)` targets (`config.STORY_SWEEP_TARGETS`), not latest-region/random-stage. Opt-in only, not in default flow | | Event sweep | `module/sweep_activity.py`, `module/activities/activity_utils.py` | `activity_sweep` (main flow), `to_activity` (nav to the event's Story/Mission/Challenge tabs), `check_sweep_availability`/`color.check_sweep_availability` (SSS gate), `start_sweep`'s named-outcome contract (shared with story_sweep's, ported the same way via `navigation.wait_for_state`) | `ba_auto/tasks/event_sweep.py`, `ba_auto/navigation.py` (`return_to_home`) | `to_activity`'s bottom-nav-icon entry -> this client's home-screen event badge (`config.EVENT_BADGE_ICON`, confirmed live to be a rotating carousel -- see Status); the reference's config-string sweep-list parsing (arbitrary stage lists, per-stage float/fraction counts via `preprocess_activity_region`/`preprocess_activity_sweep_times`) -> a single date-ordinal-modulo rotation target over a fixed 9-12 sub-range, mirroring `story_sweep.py`'s own rotation, per explicit user direction; stage-number OCR (`detector.read_int`) replaces the reference's `swipe_search_target_str` template-button search, since this client's stage list only ever needs its bottom scroll extreme for the 9-12 target range | Done, with a live-reported bug fixed (see `plan.md`'s Phase 14 follow-up). Live-calibrated against nik-gpu 2026-07-10 against the currently-running "鉄道爆走事件" event (12 stages) -- zero real AP spent during calibration (every confirm dialog reached was cancelled via Escape, verified by the AP counter). The stage-info modal is structurally identical to story_sweep's (MIN/-/+/MAX stepper, same AP-confirm dialog reusing `SWEEP_CONFIRM_*`/`SWEEP_RESULT_BUTTON_REGION`), but simpler: no region navigation, one fixed modal layout confirmed across two different stages (09 and 12), and the modal closes on Escape (story_sweep's doesn't). The reference's SSS-availability gate for a never-cleared stage was never exercised (every stage 9-12 on this account was already 3-starred) -- `event_sweep.py` handles that gate the same defensive way story_sweep handles an unavailable target: if the MAX-button count-raise can't be verified, it aborts without spending AP rather than guessing. **Bug found on a real run**: the home-screen event badge turned out to be a rotating carousel (cycles between the current event's countdown and other notices, e.g. a finished event's leftover reward-claim reminder), so a click could land on a stale event's page instead. Fixed via a new shared `navigation.return_to_home` primitive (bounded press-back-until-home loop, built generically so other tasks can reuse it, per explicit user request) plus wrong-page detection in `_find_stage_row` (zero stage-row numbers OCR'd at all -> retry via `return_to_home`, up to 3 attempts). **Second bug found on the next real run** (after the above fix correctly recovered and correctly OCR'd the target row): the row's own 入場 (enter) button click had no retry, unlike every other click-then-confirm step in this module -- missed once, aborted the whole run. Fixed via `_open_stage_modal`, the same click-then-verify-then-retry pattern already used everywhere else in the file. **Third bug found on a third real run**: `_find_stage_row`'s wrong-page detection false-positived twice (a fresh navigation's list hadn't finished rendering on the first OCR pass) before self-correcting on the 3rd attempt, wasting expensive return-home retries on what was really just a timing race -- fixed with a cheap in-place rescan before concluding "wrong page." Also, once past that, the 掃討開始 (start sweep) click turned out to be the last bare, unretried click in the file -- fixed via `_click_sweep_start_and_verify`. **Fourth round, self-driven live iteration per explicit user direction (fix/deploy/run/screenshot/diagnose in a loop, no stopping to report)**: found two more root causes and reached the first confirmed real live sweep. (1) The badge carousel's auto-rotate timer is far slower than the retry window -- a run hit "wrong page" on all 3 attempts genuinely, confirmed by screenshot; fixed by discovering and clicking the carousel's own pagination dots directly (`config.EVENT_BADGE_DOT_X`) instead of hoping the ambiguous badge shows the right item. (2) A genuine cold-start settle delay (up to ~20s for the Quest tab's stage list to populate after a fresh navigation, most likely a one-time server round-trip), not a flaky race -- confirmed via standalone probe scripts polling the real OCR pipeline once/second for a minute; fixed by widening `STAGE_ROW_SCAN_ATTEMPTS`/`_RETRY_WAIT` to match. With both fixes, a real run completed end-to-end: 200 AP spent (10x MAX sweep), credits gained, clean return home, confirmed by screenshot -- see `plan.md`'s Phase 14 follow-up #4 for the full writeup. **Fifth round, reported by the user a day later**: the exact same wrong-page-looping symptom recurred. Per the user's explicit request, added a DIRECT check for the finished event's own "イベント期間が終了しました" text (Japanese OCR, newly installed on nik-gpu since this project previously only needed digit/English reads) instead of the indirect "zero valid rows" heuristic -- `config.EVENT_FINISHED_TEXT_RECT`/`_is_finished_event_page`, live-calibrated and confirmed both ways (exact phrase match on a real finished page, no false positive on the correct page). This immediately proved the badge carousel's dot-clicking (follow-up #4's fix) is NOT reliably controllable after all -- sometimes worked, sometimes didn't, on the identical badge state -- while simply waiting longer let the carousel's own auto-rotate timer land on the correct item independently. Fixed by widening `WRONG_PAGE_RETRIES`/`_WAIT` (3x3s -> 6x12s) to give the timer real room to cycle, keeping dot-clicking as a harmless supplementary nudge. Two more real sweeps confirmed this fully working (11x MAX sweep, 219 AP spent, credits gained, clean home return, confirmed by screenshot -- three real confirmed sweeps total across this whole investigation). Also found and fixed the real root cause of the lingering `"unrecognized_state"` cosmetic bug (not a budget issue after all): `_watch_sweep_result`'s "swept" condition required the stage modal to still be open, but this event's flow can auto-return all the way to the Quest list instead, a terminal state it never anticipated -- fixed with a second `ends` condition gated on having clicked at least one result button first. Not yet re-verified live (AP exhausted by the successful sweep). See `plan.md`'s Phase 14 follow-up #5 for the full writeup. **Sixth round (2026-07-11)**: the daily rotation picked stage 9 for the first time, hitting a previously-flagged-but-never-exercised gap -- rows 366/538 ("08"/"09") consistently misread by OCR ("2" and empty/None) on every psm mode, even though the crop looked completely clean by eye; confirmed NOT a navigation/timing bug since rows 710/883/1055 ("10"/"11"/"12") read fine in the same run. Root cause (`scratchpad/probe_ocr_fix_08_09.py`): tesseract's segmentation fails on this tight edge-to-edge crop (no whitespace margin) for a leading-zero digit pair specifically -- adding a plain white border around the upscaled crop before OCR fixed both digits exactly, at every psm mode, without affecting "10". Fixed via new `detector.read_int_bordered`, now used for all `EVENT_STAGE_ROW_Y` reads. **Confirmed live**: stage 9 found, sweep executed for real (AP 233->14, credits +5,892, screenshot-confirmed safe return home) -- the originally reported bug is fixed. The same run again logged `unrecognized_state`, proving follow-up #5's `clicked_any`-gate fix addressed a real but secondary issue, not the full story. Diagnosed at zero AP cost (`scratchpad/probe_result_button_fp.py`, checks `_find_result_button` against the plain Quest list with no sweep running): the shared `SWEEP_RESULT_BUTTON_REGION` (borrowed from `story_sweep.py`) reaches into this event's own character-art panel and false-positive-matches `SWEEP_CONFIRM_CYAN` there with no dialog showing at all, confirmed on both a wrong event page and the correct one's own plain list -- so `_watch_sweep_result` kept "finding" a result button after the real one was already dismissed, and its "modal closed, no result button" end condition could never match. Fixed with a new event_sweep-only `config.EVENT_SWEEP_RESULT_BUTTON_REGION` (x narrowed to exclude the character-art panel, still comfortably covering the real buttons), confirmed live against the actual false-positive condition (now returns `None` where it previously didn't) -- not yet re-confirmed via a fresh full sweep since AP was too low that day. See `plan.md`'s Phase 14 follow-up #6. | | Group/Club AP | `module/group.py` | Need to inspect | `ba_auto/tasks/group.py` | fixed click + state check via local driver | Not started | diff --git a/ba_auto/tasks/gem_shop.py b/ba_auto/tasks/gem_shop.py new file mode 100644 index 0000000..da01ed7 --- /dev/null +++ b/ba_auto/tasks/gem_shop.py @@ -0,0 +1,143 @@ +"""Gem shop (青輝石購入) daily free package. Ports module/collect_daily_free_power.py. + +Reference flow (`implement`): to_main_page -> to_purchase_pyroxenes_menu +(open the dialog via the home-screen 青輝石購入 icon) -> to_purchase_type +("package") -> detect_free_power_availability -> if purchasable, +collect_daily_free_power (click the free card, confirm the 0-yen purchase, +wait for the reward) else log "already collected" -> return_to_main_page +(close the dialog). Skips entirely on CN server (JP/Global only) -- no +equivalent server concept in this project, so not ported. + +The reference detects every step via image template matching (no OCR +anywhere in this flow) -- this port uses plain color probes for the same +states instead (see config.py's "Gem shop daily free package" section for +the full live-calibration writeup and exact pixel values), matching this +project's own established equivalent for a simple, high-contrast +state-A/state-B visual difference, same as cafe.py's CLAIM_DISABLED_RGB or +stamina.py's MISSION_CLAIM_PROBE. + +Live-calibrated 2026-07-15 by manually driving the real dialog end-to-end +on nik-gpu, which genuinely claimed the account's real free package for the +day (0 yen, confirmed +10 AP / +10,000 credits via before/after counter +values) -- see scratchpad/gem_shop_*.png for the captures. That exercised +the "available -> claim it" path for real, via raw xdotool/scrot rather +than this actual module. This module was then live-tested for real +against the resulting "already claimed today" state (see plan.md's Gem +shop phase) -- the "available -> claim" code path itself is implemented +against the same coordinates/logic already confirmed live, but has not +yet been exercised by this actual module end-to-end; treat as +implemented-but-unverified for that one path until a run happens naturally +on a day the package hasn't been claimed yet. + +A `reference-parity-reviewer` pass the same day caught a real bug in the +first version of this module: the final dialog-close step verified success +via `navigation.is_on_subscreen`, but config.py's own writeup on +`GEM_SHOP_DIALOG_PROBES` already documents that probe as blind to this +specific dialog (reads the same whether it's open or closed) -- so that +check could never fail, making a stuck-open dialog unrecoverable by this +module (and, per the same blindness, by `navigation.return_to_home`'s +shared fallback too). Fixed by adding `_close_gem_shop`, which verifies +against the module's own correctly-calibrated `_dialog_open` instead, with +a bounded retry -- reused for both closing paths (the "unknown status" +abort and the normal end-of-run close), replacing the original's second, +separately-broken blind Escape in the abort path too. +""" + +def _dialog_open(driver, config): + for x, y in config.GEM_SHOP_DIALOG_PROBES: + r, g, b = driver.color_at(x, y) + spread = max(r, g, b) - min(r, g, b) + if not (min(r, g, b) > config.GEM_SHOP_DIALOG_PROBE_MIN_CHANNEL and spread < config.GEM_SHOP_DIALOG_PROBE_MAX_SPREAD): + return False + return True + + +def _open_gem_shop(driver, config): + for attempt in range(1, config.GEM_SHOP_ICON_RETRIES + 1): + driver.click(*config.GEM_SHOP_ICON) + driver.wait(2) + if _dialog_open(driver, config): + return True + print(f"[gem_shop] dialog not detected after click (attempt {attempt}/{config.GEM_SHOP_ICON_RETRIES})") + return False + + +def _open_package_tab(driver, config): + for attempt in range(1, config.GEM_SHOP_ICON_RETRIES + 1): + driver.click(*config.GEM_SHOP_PACKAGE_TAB) + driver.wait(1.5) + status = _read_free_card_status(driver, config) + if status != "unknown": + return status + print(f"[gem_shop] package tab's free card not recognized after click (attempt {attempt}/{config.GEM_SHOP_ICON_RETRIES})") + return "unknown" + + +def _color_matches(rgb, target, tolerance): + return all(abs(c - t) <= tolerance for c, t in zip(rgb, target)) + + +def _read_free_card_status(driver, config): + rgb = driver.color_at(*config.GEM_SHOP_FREE_CARD_STATUS_PROBE) + if _color_matches(rgb, config.GEM_SHOP_STATUS_CLAIMED_RGB, config.GEM_SHOP_STATUS_TOLERANCE): + return "claimed" + if _color_matches(rgb, config.GEM_SHOP_STATUS_AVAILABLE_RGB, config.GEM_SHOP_STATUS_TOLERANCE): + return "available" + return "unknown" + + +def _close_gem_shop(driver, config): + # Verify with the module's own _dialog_open probe rather than + # navigation.is_on_subscreen/is_modal_open -- both are documented above + # (see the "Generic... dialog... showing" comment in config.py) to read + # the same regardless of whether this specific dialog is open or closed, + # so a check against either can never fail and would silently mask a + # stuck-open dialog (caught in review before this was live-tested: the + # original version of this function used is_on_subscreen for exactly + # that dead check). + for attempt in range(1, config.GEM_SHOP_ICON_RETRIES + 1): + driver.keypress("Escape") + driver.wait(1.5) + if not _dialog_open(driver, config): + return True + print(f"[gem_shop] dialog still open after Escape (attempt {attempt}/{config.GEM_SHOP_ICON_RETRIES})") + return False + + +def _claim_free_package(driver, config): + driver.click(*config.GEM_SHOP_FREE_CARD_BUY_BUTTON) + driver.wait(2) + for _ in range(config.GEM_SHOP_CLAIM_MAX_ATTEMPTS): + if _read_free_card_status(driver, config) == "claimed": + return True + driver.keypress("Return") + driver.wait(2) + return _read_free_card_status(driver, config) == "claimed" + + +def run(driver, config): + driver.focus_game() + + if not _open_gem_shop(driver, config): + print("[gem_shop] could not confirm the gem shop dialog opened, aborting without pressing further keys") + return + + status = _open_package_tab(driver, config) + if status == "unknown": + print("[gem_shop] could not read the free package card's status, aborting without pressing further keys") + if not _close_gem_shop(driver, config): + print("[gem_shop] warning: could not confirm the dialog closed") + return + + if status == "claimed": + print("[gem_shop] daily free package already collected today") + else: + print("[gem_shop] claiming daily free package (+10 AP, +10,000 credits)") + if _claim_free_package(driver, config): + print("[gem_shop] claimed daily free package") + else: + print("[gem_shop] warning: could not confirm the free package was claimed after retries") + + if not _close_gem_shop(driver, config): + print("[gem_shop] warning: could not confirm the dialog closed") + print("[gem_shop] Done.") diff --git a/ba_daily.py b/ba_daily.py index aadf6f9..4e5dd21 100644 --- a/ba_daily.py +++ b/ba_daily.py @@ -3,12 +3,13 @@ import sys from ba_auto import config, driver, navigation -from ba_auto.tasks import arena, bounty, cafe, event_sweep, lesson, mailbox, shop_common, shop_tactical, stamina, story_sweep +from ba_auto.tasks import arena, bounty, cafe, event_sweep, gem_shop, lesson, mailbox, shop_common, shop_tactical, stamina, story_sweep TASKS = { "mailbox": mailbox.run, "cafe": cafe.run, "stamina": stamina.run, + "gem_shop": gem_shop.run, "story_sweep": story_sweep.run, "event_sweep": event_sweep.run, "shop_common": shop_common.run, @@ -23,8 +24,11 @@ TASKS = { # rather than reclaiming something free, which is a real resource decision # the default unattended run shouldn't make blindly. Arena specifically # fights a real ranked PvP battle each run -- see ba_auto/tasks/arena.py's -# module docstring. -DEFAULT_ORDER = ["mailbox", "cafe", "stamina"] +# module docstring. gem_shop is the opposite case -- like mailbox/cafe/ +# stamina, it only ever reclaims a genuinely free (0 yen), once-per-day +# resource with no choice to make (claim it or don't, nothing to select), +# so it belongs in the default flow rather than opt-in. +DEFAULT_ORDER = ["mailbox", "cafe", "stamina", "gem_shop"] # How many times _ensure_home retries navigation.return_to_home as a whole # (not to be confused with that function's own internal diff --git a/plan.md b/plan.md index 86036b3..9fadd70 100644 --- a/plan.md +++ b/plan.md @@ -344,6 +344,18 @@ Fix: check `navigation.is_modal_open` (the darker "real dialog" reading, not the **Confirmed live** (`live-test-runner` agent, real `cafe` run with the ticket confirmed still on cooldown for both rooms): both rooms logged `invite ticket click opened a dialog instead of the list (title: '...') -- likely on cooldown, treating invitation as unavailable` immediately, no cascading errors or misclicks followed, and the task proceeded normally to farming/income-claim and ended cleanly (exit 0, true home screen confirmed via screenshot). OCR of the dialog title came back noisy (`'通和 X'` instead of `'通知'`) but this doesn't matter — the fix's branch doesn't depend on the title text, only on `is_modal_open` firing before any row was clicked. +#### Phase 6 follow-up #5: rank-up cutscene misread + cooldown-notice dismiss misalignment (2026-07-15) + +Two real bugs reported together in `next_fix.md` after a live `cafe` run: + +**Bug 1**: a pat that triggered a real 絆ランクアップ (bond rank-up) cutscene caused the run to skip the rest of that room's farming instead of self-healing through it, matching a gap CLAUDE.md had already flagged as theoretical ("is_on_subscreen's fixed header probe reads inconsistently across different characters' cutscene art") — now confirmed live. Root cause: `_dismiss_rank_up_if_shown` checked `navigation.is_on_subscreen`, a single header pixel at `(500,10)`; for this run's character, that pixel happened to read bright, so the loop believed the cutscene had already cleared without ever pressing Enter, then `_pat_current_view`'s loop kept polling `find_cafe_sparkle()` against the still-showing cutscene for the rest of its budget. Fixed by adding `navigation.is_header_bar_visible` (8 spread-out header-row x-positions, all must read bright — a real header bar is uniformly flat-colored across its width, unlike a photo-real character composition) and switching `_dismiss_rank_up_if_shown` to use it. Best-effort: could not force a live rank-up on demand to confirm end-to-end this session. + +**Bug 2**: when the invite ticket is on cooldown, the cooldown-notice dismiss (added in follow-up #4 above) clicked `SWEEP_CONFIRM_BUTTON`'s fixed coordinate, which could miss this single-OK notice's own button (not necessarily laid out like the two-button dialogs that coordinate was calibrated against) — a missed click left the notice open, and the camera-pan drags that followed landed on the still-open dialog instead of the room view. First fix attempt switched to a plain `driver.keypress("Escape")`; an independent `reference-parity-reviewer` pass caught that this was itself unverified (an unconfirmed keypress is no more trustworthy than the unconfirmed click it replaced, and this project's own `config.py` documents at least one dialog where Escape does *not* close it) — so the actual fix presses Escape and verifies with `navigation.is_modal_open`, retrying up to `ROOM_OPEN_RETRIES` times and logging clearly if it's still stuck, rather than assuming one press worked. + +A related efficiency finding from the same review: `is_header_bar_visible`'s 8-point check was calling `driver.color_at()` 8 times (8 full-screenshot round-trips) for one logical check. Added `driver.colors_at(points)` — one screenshot, multiple points sampled from it, also atomic (no drift across sequentially-captured frames) — and switched `is_header_bar_visible` to use it. + +Both fixes deployed to nik-gpu; the reviewer independently pixel-verified `is_header_bar_visible` against real captured screenshots (`screenshots/cafe/room1.png`, `screenshots/cafe/student/04_*.png`) and found no false-negative risk against normal cafe/dialog screens. Neither fix has been re-confirmed against a fresh live rank-up or cooldown notice post-deploy — worth watching the next time either occurs naturally. + ### Phase 7: setup.sh update **Status: Done — `setup.sh` deploys `ba_daily.py` and `ba_auto/`.** `scripts/ba_dailies_legacy.sh` and `scripts/detect_and_click.py` were deleted once mailbox and cafe both migrated off them (Phases 5–6); `setup.sh` no longer references either. @@ -372,6 +384,10 @@ Two real bugs found and fixed during live calibration, both worth remembering fo Not yet done: Group/Club AP, and Daily Free Power (see above). +#### Phase 8 follow-up: Daily Free Power built as its own task (2026-07-15, see Phase 16) + +The 青輝石購入 menu deliberately left unautomated above turned out to gate a real-money purchase UI in general, but the specific card this was always about (`module/collect_daily_free_power.py`'s 毎日無料パッケージ, 0 yen, AP+credits) is safe to automate on its own — it's a fixed, non-selectable, always-free claim with no purchase decision to make, same category as this phase's own Mission-panel claim. Built as a separate `ba_auto/tasks/gem_shop.py` rather than folded into `stamina.py`, since it's a different reference file/UI entry point. See Phase 16. + ### Phase 9: Normal/Hard story AP sweep **Status: Done.** Read `module/explore_tasks/sweep_task.py` and `module/explore_tasks/task_utils.py` — the reference flow reads the current region number and matches stage-name text via OCR (`swipe_search_target_str`) to navigate to a configured target stage, then runs a per-stage claim loop. This client exposes a much simpler path to the same goal (burn AP via already-3-starred stages) that avoids porting the OCR-based lookup entirely: each stage's own 任務情報 (task info) modal has a self-contained 掃討 (sweep) sub-panel with a MIN/-/+/MAX count stepper and a start button. @@ -661,6 +677,27 @@ With that fixed, the user approved spending the account's one remaining ticket o **Confirmed real**: 2 live sweeps total, credits gained both times (+180,000 then +36,000), clean automatic return home both times, zero manual intervention, zero real Pyroxene spent despite the near-miss. **Not yet re-confirmed live**: a fresh sweep with today's fix deployed (account is at 0/6 tickets as of session end) and any bulk/MAX-count sweep's result-screen flow specifically (only count=1 was ever tested, since ticket scarcity forced it — a bulk sweep may show a SKIP-then-OK sequence like story_sweep/event_sweep rather than the single-OK dialog confirmed here). +### Phase 16: Gem shop daily free package (2026-07-15) + +Reference: `module/collect_daily_free_power.py`. Local: `ba_auto/tasks/gem_shop.py`. Requested directly by the user with reference screenshots (`screenshots/gem_shop/1-4.png`): claim the 毎日無料パッケージ (daily free package, +10 AP / +10,000 credits, 0 yen, once per day) inside the 青輝石購入 (gem purchase) dialog. This is the exact piece Phase 8 deliberately deferred (see its follow-up above) — that phase stopped at "this menu has real-money purchase buttons visible immediately," but the specific free card is a fixed, non-selectable, no-decision claim, same category as Phase 8's own Mission-panel claim. + +Reference flow (`implement`): `to_main_page` → `to_purchase_pyroxenes_menu` (open the dialog via the home-screen icon) → `to_purchase_type("package")` → `detect_free_power_availability` → if purchasable, `collect_daily_free_power` (click the free card, confirm the 0-yen purchase, wait for the reward) else log already-collected → `return_to_main_page` (close the dialog). Every step is detected via `core.picture.co_detect` + fixed-region OpenCV template matching against per-locale PNG assets (`main_page_purchase-pyroxenes-{menu,package-selected-*,daily-free-purchasable,daily-free-non-purchasable,confirm-purchase-notice}.png`) — no OCR anywhere in this reference flow. + +Ported as plain color probes instead of new template assets, matching this project's own established equivalent for a simple, high-contrast state-A/state-B visual difference (`cafe.py`'s `CLAIM_DISABLED_RGB`, `stamina.py`'s `MISSION_CLAIM_PROBE`) rather than building a template-match asset pipeline for what reduces to flat colors. **Live-calibrated 2026-07-15** by manually driving the real dialog end-to-end on nik-gpu via raw `xdotool`/`scrot` (not estimated from the user's own non-native-resolution reference screenshots — confirmed those aren't 1:1 with real game coordinates, same finding as `screenshots/cafe/student/`'s calibration history) — this walkthrough **genuinely claimed the account's real free package for the day** (AP 64→74, credits 161,391,144→161,401,144, both confirmed via before/after counter reads), which calibrated both the "available" and "already claimed" states from real data in one pass: + +- The free card's own status bar reads flat dark navy `~(41,65,90)` when available ("一日に1回まで購入可能") vs flat dark red `~(144,38,47)` once claimed ("一日に0回まで購入可能") — a clean, high-contrast signal (`r` vs `b` alone separates them by ~50-100), no OCR of the count digit needed. +- `navigation.is_on_subscreen`/`is_modal_open`'s default probes both proved unreliable on this dialog specifically: it overlays the home screen directly (keeping the home header visible) rather than being a full subscreen, so `is_on_subscreen`'s header probe stays dark whether the dialog is open or not; `is_modal_open`'s `(960,200)` probe lands on the dialog's own opaque white card interior rather than a dimmed backdrop, so it reads bright regardless of dialog state too — the same class of default-probe mismatch `story_sweep.py`/`bounty.py` already document for their own wide modals. Fixed with a dedicated 4-point "is this dialog (or a nested confirm/notice card) showing" check (`GEM_SHOP_DIALOG_PROBES`), reusing the same multi-point-beats-single-point reasoning `navigation.is_header_bar_visible` was built on the same day for an unrelated cafe bug (see Phase 6 follow-up #5) — independently applied here since this dialog's false-positive risk is a different shape (character-art home screen vs. the dialog's own near-white card gutters), not the header row. +- The post-purchase "報酬獲得!" (reward acquired) banner did not accept input on its first 1-2 Enter presses while its own entry animation was still playing — `_claim_free_package` polls the free card's status after each press rather than assuming one press is enough, bounded by `GEM_SHOP_CLAIM_MAX_ATTEMPTS` (6). + +Added to `ba_daily.py`'s `TASKS` dict and, unlike every task added since Phase 8, also to `DEFAULT_ORDER` — it only ever reclaims a genuinely free, once-per-day resource with no choice to make (unlike story_sweep/shop/lesson/arena/bounty, which all spend something on an automated choice), the same "reclaim something free" category as mailbox/cafe/stamina. + +**Live-tested for real** via the actual `./ba_dailies.sh gem_shop` CLI path (not just the manual calibration walkthrough): correctly recovered from a flaky first icon click (retried and succeeded on attempt 2, the same "first click sometimes misses" pattern already documented for the mailbox/cafe icons), correctly read the real "already claimed today" state left over from the calibration walkthrough, logged it, and returned cleanly to the confirmed true home screen. **Not yet live-tested**: the "available → claim" code path itself, since the account's package was already claimed for the day by the manual calibration walkthrough before the module existed — it uses the same coordinates/logic already confirmed live via that manual walkthrough, but hasn't been exercised by the actual module end-to-end. Worth a follow-up check the next time the package resets and hasn't been claimed yet. + +**A `reference-parity-reviewer` pass the same day, before this was reported done, caught a real bug and a real future-breakage risk:** + +1. **Dead close-verification (fixed).** The first version's final dialog-close step verified success via `navigation.is_on_subscreen` — but that probe is documented (in the same session's config.py writeup) to read identically whether this specific dialog is open or closed, since the dialog overlays the home screen directly rather than being a full subscreen. The check could therefore never fail, silently masking a stuck-open dialog that neither this module nor `navigation.return_to_home`'s shared fallback (same blind-probe class) could then detect — the exact "unverified action assumed to work" shape the cafe.py bugs earlier in this same session were also caught in. Fixed by adding `_close_gem_shop`, which verifies with the module's own correctly-calibrated `_dialog_open` check instead, bounded-retry, reused for both the normal end-of-run close and the "unknown status" abort path (which had the same unverified-Escape bug separately). **Re-confirmed live** after the fix: same real "already claimed" run via the actual CLI, no warnings, confirmed via screenshot back at the true home screen. +2. **`GEM_SHOP_PACKAGE_TAB` is a single coordinate calibrated only against today's temporary 3-tab layout (documented, not yet fixable).** The reference's own `to_purchase_type` branches on whether the 期間限定 (time-limited) tab is present, because removing it reflows the remaining tabs — and `screenshots/gem_shop/2.png` (captured live the same session) shows that tab's own countdown ("終了まであと6日", 26.06.24〜26.07.29), meaning this account's 3-tab layout is itself due to change around 2026-07-21. No real screenshot of the resulting 2-tab layout exists yet, so a second coordinate was deliberately NOT guessed in rather than grounded in a real capture — flagged with a dated comment in `config.py` instead. Contained, not a live hazard: if this goes stale, `_open_package_tab`'s existing status-probe check reads "unknown" and aborts cleanly via the now-fixed verified close, without ever clicking a purchase button on whatever tab it actually landed on. Needs a real re-calibration pass once the tab count actually changes. + ### Return-to-home audit (2026-07-12) Per explicit user request: "I want all the script to return to home page after the script ended, no matter the script ended in success or failure. You can use the Esc button to return or use module to reuse the logic."