Nik Afiq 0ff3dca0b9 Implement dynamic OCR for gem shop sidebar navigation and free card status
- Added `read_lines` function to `detector.py` for OCR of multiple text lines in a specified region, improving the ability to dynamically locate text in the UI.
- Rewrote the gem shop task in `gem_shop.py` to utilize OCR for sidebar navigation and free card status checks, replacing fixed coordinates with dynamic text detection.
- Introduced helper functions `_find_sidebar_label`, `_on_general_products_tab`, `_ensure_general_products_tab`, `_read_card_lines`, `_read_free_card_status`, and `_find_buy_button` to streamline the OCR process and improve reliability against UI changes.
- Removed outdated color probe methods for checking the free card status, enhancing the robustness of the gem shop interaction flow.
- Live-calibrated the new OCR functionality to ensure accurate detection of UI elements and their states.
2026-09-17 10:37:24 +09:00

205 lines
9.3 KiB
Python

"""Gem shop (青輝石購入) daily free package. Ports module/collect_daily_free_power.py.
Reference flow (`implement`): to_main_page -> to_purchase_pyroxenes_menu
(open the dialog via the home-screen 青輝石購入 icon) -> to_purchase_type
("package") -> detect_free_power_availability -> if purchasable,
collect_daily_free_power (click the free card, confirm the 0-yen purchase,
wait for the reward) else log "already collected" -> return_to_main_page
(close the dialog). Skips entirely on CN server (JP/Global only) -- no
equivalent server concept in this project, so not ported.
The reference detects every step via image template matching (no OCR
anywhere in this flow). This port originally used plain color probes for
the same states instead (see config.py's "Gem shop daily free package"
section for the original 2026-07-15/07-29 live-calibration writeup),
matching this project's own established equivalent for a simple,
high-contrast state-A/state-B visual difference, same as cafe.py's
CLAIM_DISABLED_RGB or stamina.py's MISSION_CLAIM_PROBE.
REWRITTEN 2026-09-17 after a real live failure: the sidebar gained a new
限定商品 entry above 一般商品 sometime after the 2026-07-29 recalibration,
silently shifting 一般商品 down one slot and making the old fixed
GEM_SHOP_GENERAL_PRODUCTS_TAB coordinate land on 定額商品 instead --
confirmed live via scratchpad/probe_gem_shop_careful_nav.py and
probe_gem_shop_general_tab_fixed.py, which walked the real screen step by
step and captured the wrong-tab landing directly. Every retry just
re-clicked the same wrong position, matching the reported symptom exactly
("free card status not recognized after click", 3/3 attempts).
Per explicit user request ("make the button press dynamic. Read word then
click the correct position of the word"), the sidebar navigation, the
status read, and the buy-button click are now all OCR-driven via
detector.read_lines() rather than fixed coordinates/colors -- each one
finds its target text wherever it actually rendered this run and
reads/clicks that position, so a future sidebar reorder can't silently
break this the same way again. See config.py's "RECALIBRATED AGAIN
2026-09-17" note and scratchpad/probe_gem_shop_ocr_regions.py for the live
OCR calibration this was based on. The デイリー/ウィークリー/マンスリー
subtab row is the one piece that stays a fixed click -- it's a horizontal
layout that doesn't OCR-segment reliably as one region (see
detector.read_lines' own docstring), and it was not the coordinate that
actually broke.
Live-calibrated 2026-07-15 by manually driving the real dialog end-to-end
on nik-gpu, which genuinely claimed the account's real free package for the
day (0 yen, confirmed +10 AP / +10,000 credits via before/after counter
values) -- see scratchpad/gem_shop_*.png for the captures. That exercised
the "available -> claim it" path for real, via raw xdotool/scrot rather
than this actual module.
RECALIBRATED 2026-07-29 after a real Blue Archive client UI update replaced
the old 3-top-tab overlay dialog (期間限定/青輝石/パッケージ) with a real
full subscreen (left sidebar + デイリー/ウィークリー/マンスリー sub-tabs
under 一般商品). See config.py for the full writeup of what was
re-verified live and what's still an inferred value. The old
`_dialog_open`/`_close_gem_shop` custom probes existed only because the
pre-redesign overlay dialog was invisible to
navigation.is_on_subscreen/is_modal_open; the redesigned screen is a real
subscreen, confirmed live, so this module shares the same
is_on_subscreen/return_to_home machinery every other subscreen-based task
(mailbox/cafe/shop/lesson) already uses.
A `reference-parity-reviewer` pass on the original version caught a real
bug: the final dialog-close step verified success via
`navigation.is_on_subscreen`, which the pre-redesign overlay dialog read
identically whether open or closed -- so that check could never fail,
making a stuck-open dialog unrecoverable. That specific blindness no longer
applies post-redesign (is_on_subscreen now genuinely distinguishes this
screen from home, confirmed live), which is what made switching to the
shared navigation.return_to_home safe here.
"""
import re
from ba_auto import detector, navigation
def _open_gem_shop(driver, config):
for attempt in range(1, config.GEM_SHOP_ICON_RETRIES + 1):
driver.click(*config.GEM_SHOP_ICON)
driver.wait(2)
if navigation.is_on_subscreen(driver):
return True
print(f"[gem_shop] subscreen not detected after click (attempt {attempt}/{config.GEM_SHOP_ICON_RETRIES})")
return False
def _find_sidebar_label(driver, config, label):
"""OCR the gem shop's own left sidebar for `label` and return its
actual on-screen position, or None if not found in its normal
(unselected) styling."""
for x, y, text in detector.read_lines(config.GEM_SHOP_SIDEBAR_REGION):
if label in text:
return (x, y)
return None
def _on_general_products_tab(driver, config):
"""The デイリー/ウィークリー/マンスリー subtab row only renders under
一般商品 -- checking for ウィークリー specifically (rather than
whichever subtab gem_shop.py itself just selected) avoids ever needing
to OCR a selected/highlighted pill, which reads unreliably (see
detector.read_lines' docstring)."""
return "ウィークリー" in detector.read_text(config.GEM_SHOP_WEEKLY_SUBTAB_LABEL_REGION, lang="jpn", psm=7)
def _ensure_general_products_tab(driver, config):
if _on_general_products_tab(driver, config):
return True
for attempt in range(1, config.GEM_SHOP_ICON_RETRIES + 1):
pos = _find_sidebar_label(driver, config, "一般商品")
if pos is None:
print(f"[gem_shop] could not OCR '一般商品' in the sidebar (attempt {attempt}/{config.GEM_SHOP_ICON_RETRIES})")
driver.wait(1)
continue
driver.click(*pos)
driver.wait(1.5)
if _on_general_products_tab(driver, config):
return True
print(f"[gem_shop] general products tab not confirmed after click (attempt {attempt}/{config.GEM_SHOP_ICON_RETRIES})")
return False
def _read_card_lines(driver, config):
return detector.read_lines(config.GEM_SHOP_FREE_CARD_REGION)
def _read_free_card_status(driver, config):
for _, _, text in _read_card_lines(driver, config):
if "購入可能" not in text:
continue
counts = re.findall(r"(\d+)回", text)
if counts:
return "claimed" if counts[0] == "0" else "available"
return "unknown"
def _find_buy_button(driver, config):
"""The free card's own 無料 (free) buy button, located by OCR rather
than a fixed coordinate. "無料" can also appear as a substring of the
card's title (毎日無料パッケージ) in the same region -- picking the
shortest matching line prefers the standalone button label over that
longer title line."""
candidates = [(x, y, text) for x, y, text in _read_card_lines(driver, config) if "無料" in text]
if not candidates:
return None
x, y, _ = min(candidates, key=lambda c: len(c[2]))
return (x, y)
def _open_daily_general_tab(driver, config):
if not _ensure_general_products_tab(driver, config):
print("[gem_shop] could not confirm the general products tab opened")
return "unknown"
for attempt in range(1, config.GEM_SHOP_ICON_RETRIES + 1):
driver.click(*config.GEM_SHOP_DAILY_SUBTAB)
driver.wait(1.5)
status = _read_free_card_status(driver, config)
if status != "unknown":
return status
print(f"[gem_shop] free card status not recognized after click (attempt {attempt}/{config.GEM_SHOP_ICON_RETRIES})")
return "unknown"
def _claim_free_package(driver, config):
pos = _find_buy_button(driver, config)
if pos is None:
print("[gem_shop] could not OCR the free package's buy button")
return False
driver.click(*pos)
driver.wait(2)
for _ in range(config.GEM_SHOP_CLAIM_MAX_ATTEMPTS):
if _read_free_card_status(driver, config) == "claimed":
return True
driver.keypress("Return")
driver.wait(2)
return _read_free_card_status(driver, config) == "claimed"
def run(driver, config):
driver.focus_game()
if not _open_gem_shop(driver, config):
print("[gem_shop] could not confirm the gem shop screen opened, aborting without pressing further keys")
return
status = _open_daily_general_tab(driver, config)
if status == "unknown":
print("[gem_shop] could not read the free package card's status, aborting without pressing further keys")
if not navigation.return_to_home(driver):
print("[gem_shop] warning: could not confirm return to home")
return
if status == "claimed":
print("[gem_shop] daily free package already collected today")
else:
print("[gem_shop] claiming daily free package (+10 AP, +10,000 credits)")
if _claim_free_package(driver, config):
print("[gem_shop] claimed daily free package")
else:
print("[gem_shop] warning: could not confirm the free package was claimed after retries")
if not navigation.return_to_home(driver):
print("[gem_shop] warning: could not confirm return to home")
print("[gem_shop] Done.")