diff --git a/CLAUDE.md b/CLAUDE.md index 11a262e..c0238d7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,29 +8,55 @@ This repository is a personal Blue Archive JP daily-automation project. It controls the real Blue Archive PC/Steam/Proton client running on a Linux machine named `nik-gpu`. The development machine is a MacBook named `nik-macbookair`. -**The original goal is a backend port, not a rewrite.** The reference project (`~/repo/baas-reference/`) only works against an Android emulator via ADB/uiautomator2, typically run on a Windows PC. This project's job is to make that same automation logic work against the real JP client running under Proton on Linux instead — porting the reference's decision logic, navigation, and state-detection approach (including its use of OCR) as faithfully as practical, and replacing only the parts that are genuinely Android-specific (ADB/uiautomator2 input and UI-object-tree queries) with local desktop equivalents (xdotool, scrot, OpenCV, OCR). Inventing a different, simpler approach to avoid porting a piece of reference logic (most notably OCR — see below) is a deviation from that goal, not a shortcut toward it. +**The original goal is a backend port, not a rewrite.** The reference project (`~/repo/baas-reference/`) only works against an Android emulator via ADB/uiautomator2, typically run on a Windows PC. This project's job is to make that same automation logic work against the real JP client running under Proton on Linux instead — porting the reference's decision logic, navigation, and state-detection approach, including its use of OCR, as faithfully as practical, and replacing only the parts that are genuinely Android-specific with local desktop equivalents. + +Android-specific parts include: + +- ADB input +- uiautomator2 taps +- Android UI-object-tree queries +- emulator-specific assumptions + +Local desktop equivalents include: + +- `xdotool` +- `scrot` +- OpenCV +- OCR against screenshots +- local image/color/template detection + +Inventing a different, simpler approach to avoid porting a piece of reference logic is a deviation from the project goal, not a shortcut toward it. The automation backend is local desktop control: - `xdotool` for mouse/keyboard/window control - `scrot` for screenshots - Python/OpenCV for image matching and color/template detection -- OCR (Tesseract or PaddleOCR), ported wherever the reference implementation uses it for a feature — see "OCR policy" below +- OCR, using Tesseract or PaddleOCR, wherever the reference implementation uses OCR for a feature - no Android emulator control - no ADB - no uiautomator2 ### OCR policy -OCR is **not** an Android-specific concern — the reference's OCR-driven region/stage-name/currency matching runs against a screenshot and would work identically against a `scrot` capture on this backend. Only the reference's *input* (ADB/uiautomator2 taps) is Android-specific. +OCR is **not** an Android-specific concern. The reference's OCR-driven region, stage-name, and currency matching runs against a screenshot and can be ported to this backend using `scrot` captures. Only the reference's input mechanism is Android-specific. -Two features (Stamina/AP mission claim, Normal/Hard story AP sweep) were originally built without OCR by substituting ad hoc pixel-probes, fixed coordinates, or randomized selection for the reference's OCR-driven navigation. For story sweep, that substitution produced more real bugs during live testing than porting the reference's actual approach would have — a wrong modal-open probe, an unverified button click that silently under-spent AP, a modal that doesn't close on Escape, and a latent hazard where a mistimed keypress could have started a real battle (see `plan.md` Phase 9's retrospective for the full writeup). None of those bugs would exist if the reference's deterministic, OCR-based stage targeting had been ported instead of replaced with a heuristic guess. **This has since been fixed** — `story_sweep.py` now ports the reference's actual OCR-based region/stage targeting (see `plan.md` Phase 10) — but the retrospective stays here as the concrete, lived reason for the policy below, not a description of the current state of that task. +Two features, Stamina/AP mission claim and Normal/Hard story AP sweep, were originally built without OCR by substituting ad hoc pixel probes, fixed coordinates, or randomized selection for the reference's OCR-driven navigation. For story sweep, that substitution produced more real bugs during live testing than porting the reference's actual approach would have: -Going forward: when a reference feature's control flow depends on OCR, set up OCR and port that logic, rather than inventing a non-OCR workaround to avoid the setup cost. Only skip OCR for a specific step if the reference itself doesn't use OCR there. +- a wrong modal-open probe +- an unverified button click that silently under-spent AP +- a modal that does not close on Escape +- a latent hazard where a mistimed keypress could have started a real battle + +See `plan.md` Phase 9's retrospective for the full writeup. + +That issue has since been fixed: `story_sweep.py` now ports the reference's actual OCR-based region/stage targeting. See `plan.md` Phase 10. The retrospective stays here as the concrete, lived reason for the OCR policy, not as a description of the current state of that task. + +Going forward: when a reference feature's control flow depends on OCR, set up OCR and port that logic. Do not invent a non-OCR workaround just to avoid the setup cost. Only skip OCR for a specific step if the reference itself does not use OCR there. The reference project is located at: -``` +```text ~/repo/baas-reference/ ``` @@ -48,7 +74,7 @@ All feature logic must live in Python. The intended structure is: -``` +```text ba-auto-daily/ ├── ba_dailies.sh ├── ba_daily.py @@ -69,12 +95,13 @@ ba-auto-daily/ │ └── mapping.md ├── assets/ ├── screenshots/ +├── scratchpad/ ├── setup.sh ├── plan.md └── CLAUDE.md ``` -(`scripts/` previously held one-off Bash/Python helpers; it was deleted once mailbox and cafe migrated off it — see "Existing features" below. Do not recreate it as a place to stash feature logic.) +`scripts/` previously held one-off Bash/Python helpers. It was deleted once mailbox and cafe migrated off it. Do not recreate `scripts/` as a place to stash feature logic. The exact layout can evolve, but the architectural rule should not change: @@ -87,7 +114,7 @@ The exact layout can evolve, but the architectural rule should not change: Before implementing any new feature, inspect the matching reference implementation in: -``` +```text ~/repo/baas-reference/module/ ``` @@ -105,7 +132,7 @@ For every feature, first identify: Then implement the feature in Python under: -``` +```text ba_auto/tasks/ ``` @@ -128,17 +155,30 @@ The reference repository is **read-only**. - edit files in `~/repo/baas-reference/` - reimplement a reference feature from scratch in Bash - create a local solution that ignores the reference flow when a reference implementation already exists -- substitute a pixel-probe, fixed-coordinate, or randomized shortcut for reference logic that uses OCR, just to avoid setting up OCR (see "OCR policy" above) +- substitute a pixel probe, fixed coordinate, or randomized shortcut for reference logic that uses OCR, just to avoid setting up OCR ## Two-machine architecture -Development happens on: `nik-macbookair` +Development happens on: -Runtime happens on: `nik-gpu` +```text +nik-macbookair +``` + +Runtime happens on: + +```text +nik-gpu +``` The Blue Archive client and X display live on `nik-gpu`. -Assume: (`DISPLAY=:0`, GDM `XAUTHORITY` under `/run/user/1000`) +Assume: + +```text +DISPLAY=:0 +GDM XAUTHORITY under /run/user/1000 +``` The game runs under Steam/Proton on the Linux desktop. @@ -150,7 +190,7 @@ During iteration, files are pushed from `nik-macbookair` to `nik-gpu`. Typical paths on `nik-gpu`: -``` +```text ~/ba_dailies.sh ~/ba_daily.py ~/ba_auto/ @@ -160,7 +200,7 @@ Typical paths on `nik-gpu`: The current setup may still contain older paths such as: -``` +```text ~/ba_scripts/detect_and_click.py ``` @@ -170,12 +210,28 @@ When refactoring, prefer consolidating Python code into `ba_auto/`. For a fresh checkout on `nik-gpu`, run from the repo root: -``` +```bash ./setup.sh ``` After initial setup, individual changes may be pushed with `scp` or `rsync`. +Prefer syncing the project from the repository root: + +```bash +rsync -av \ + --exclude='.git/' \ + --exclude='__pycache__/' \ + --exclude='*.pyc' \ + --exclude='.claude/settings.local.json' \ + --exclude='graphify-out/' \ + --exclude='screenshots/' \ + --exclude='scratchpad/' \ + ./ nik-gpu:~/repo/ba-auto-daily/ +``` + +Do not sync Claude Code internal temporary folders. + ## Runtime dependencies on nik-gpu These are host-level dependencies. Confirm they exist before assuming a bug is in the project code. @@ -190,20 +246,28 @@ Required Python packages: - `opencv-python` or `opencv-python-headless` - `numpy` +- `pytesseract` Expected venv: -``` +```text ~/.venvs/ba-auto-daily/bin/python3 ``` Quick check: -``` +```bash ssh nik-gpu "which xdotool scrot tesseract && ~/.venvs/ba-auto-daily/bin/python3 -c 'import cv2, numpy, pytesseract; print(cv2.__version__)'" ``` -OCR engine dependency: **set up** (Phase 10) — Tesseract via the `tesseract-ocr` apt package (needs an interactive `sudo`, so `setup.sh` checks for it but can't install it for you) plus `pytesseract` in the venv. `ba_auto/detector.py`'s `read_text()`/`read_int()` wrap it for occasional single-crop reads; see `story_sweep.py` for the first real usage. +OCR engine dependency is set up as of Phase 10: + +- Tesseract via the `tesseract-ocr` apt package +- `pytesseract` in the venv + +Installing `tesseract-ocr` needs interactive `sudo`, so `setup.sh` should check for it but should not assume it can install it automatically. + +`ba_auto/detector.py`'s `read_text()` and `read_int()` wrap OCR for occasional single-crop reads. See `story_sweep.py` for the first real usage. ## Bash policy @@ -214,8 +278,10 @@ Preferred shape: ```bash #!/usr/bin/env bash set -euo pipefail + VENV_PYTHON="${VENV_PYTHON:-$HOME/.venvs/ba-auto-daily/bin/python3}" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + exec "$VENV_PYTHON" "$SCRIPT_DIR/ba_daily.py" "$@" ``` @@ -244,13 +310,13 @@ If an existing Bash function exists, migrate it to Python rather than extending The intended Python CLI entry point is: -``` +```text ba_daily.py ``` It should support commands such as: -``` +```bash ./ba_dailies.sh ./ba_dailies.sh mailbox ./ba_dailies.sh cafe @@ -261,7 +327,7 @@ It should support commands such as: No argument should run the default daily flow. -Current default flow (`ba_daily.py`'s `DEFAULT_ORDER`): +Current default flow in `ba_daily.py`'s `DEFAULT_ORDER`: 1. focus game 2. mailbox @@ -269,15 +335,19 @@ Current default flow (`ba_daily.py`'s `DEFAULT_ORDER`): 4. stamina 5. future daily tasks -`story_sweep` spends AP rather than reclaiming something free, so it is deliberately excluded from the default flow — it must be invoked explicitly. +`story_sweep` spends AP rather than reclaiming something free, so it is deliberately excluded from the default flow. It must be invoked explicitly. -The CLI should dispatch into task modules under `ba_auto/tasks/`. +The CLI should dispatch into task modules under: + +```text +ba_auto/tasks/ +``` ## Driver layer Create and maintain a driver layer in: -``` +```text ba_auto/driver.py ``` @@ -285,14 +355,15 @@ The driver layer should wrap the PC/Steam/Proton backend. It should provide reusable primitives such as: -``` +```python focus_game() screenshot() click(x, y) double_click(x, y) -drag/swipe(start, end, duration) +drag(start, end, duration) +swipe(start, end, duration) keypress(key) -sleep/wait +wait(seconds) wait_until(...) color_at(...) region_average_color(...) @@ -300,7 +371,7 @@ template_match(...) find_and_click_template(...) ``` -Feature modules should not directly shell out to `xdotool` or `scrot` unless a driver primitive is missing and being added. +Feature modules should not directly shell out to `xdotool` or `scrot` unless a driver primitive is missing and is being added. Prefer: @@ -311,16 +382,22 @@ driver.click(x, y) over: ```python -subprocess.run(["xdotool", "click", ...]) +subprocess.run(["xdotool", "click", "1"]) ``` This keeps the project close to the reference architecture while replacing only the backend-control layer. ## Detector layer -Image matching and color matching should live in `ba_auto/detector.py` or in clearly named helper classes/functions. +Image matching, color matching, and OCR wrappers should live in: -The cafe sparkle-matching logic formerly in `scripts/detect_and_click.py` (now retired) has been migrated here as `find_cafe_sparkle()`, called in-process rather than as a per-click subprocess. +```text +ba_auto/detector.py +``` + +or in clearly named helper classes/functions. + +The cafe sparkle-matching logic formerly in `scripts/detect_and_click.py` has been migrated into `ba_auto/detector.py` as `find_cafe_sparkle()`. It should be called in-process rather than as a per-click subprocess. The detector should support: @@ -329,13 +406,18 @@ The detector should support: - threshold tuning - masked matching - click-offset handling +- OCR crop reads - debug image output to `scratchpad/` Avoid one Python cold start per click attempt where possible. Prefer long-running Python task logic that can take repeated screenshots and click repeatedly from one process. ## Navigation layer -Common navigation should live in `ba_auto/navigation.py`. +Common navigation should live in: + +```text +ba_auto/navigation.py +``` Use this for shared flows such as: @@ -347,16 +429,21 @@ Use this for shared flows such as: - opening lesson/schedule - closing popups - generic back/escape handling +- waiting for known UI states Do not duplicate navigation click sequences inside every task if they can be shared. ## Task modules -Each feature should have a task module: `ba_auto/tasks/.py` - -Example: +Each feature should have a task module: +```text +ba_auto/tasks/.py ``` + +Examples: + +```text ba_auto/tasks/mailbox.py ba_auto/tasks/cafe.py ba_auto/tasks/stamina.py @@ -366,13 +453,13 @@ ba_auto/tasks/group.py Each task module should expose a clear function such as: -``` +```python run(driver, config) ``` or: -``` +```python run_mailbox(driver, config) ``` @@ -382,7 +469,7 @@ Keep task files feature-focused. Maintain a mapping file at: -``` +```text ba_auto/reference_notes/mapping.md ``` @@ -390,78 +477,231 @@ Before or during implementation of a feature, update the mapping. Use this format: +```markdown | Local feature | Reference file | Reference functions/classes | Local file | Backend replacements | Status | |---|---|---|---|---|---| | Cafe | `module/cafe.py` or relevant file | `...` | `ba_auto/tasks/cafe.py` | uiautomator2 tap -> xdotool click, screenshot -> scrot/OpenCV | In progress | +``` ## Working conventions -Use `scratchpad/` (create if missing) in the project root for temporary or intermediate files. +Always work from the repository root: + +```bash +cd ~/repo/ba-auto-daily +``` + +Use `scratchpad/` in the project root for all temporary, generated, diagnostic, calibration, and investigation files. Examples: -- cropped calibration images, e.g. from `screenshots/cafe/sparkle/` +- cropped calibration images - debug screenshots - annotated match results +- temporary Python probes - temporary investigation notes +- one-off image-analysis scripts +- temporary OCR experiments +- screenshots copied back from `nik-gpu` for analysis -Do not use `/tmp` or `/private/tmp` unless there is a strong reason. +Create it if missing: + +```bash +mkdir -p scratchpad +``` + +Do **not** use Claude Code's internal temporary directories such as: + +```text +/private/tmp/claude-* +/tmp/claude-* +``` + +Do **not** run probes from `/private/tmp/claude-*` or write temporary scripts there. These paths are outside the repository and often trigger extra permission prompts, interrupting the development flow. + +When a temporary Python probe is needed, write it into `scratchpad/` first, then run it from the repository root. + +Preferred pattern: + +```bash +cat > scratchpad/probe_image.py <<'PY' +from PIL import Image + +img = Image.open("scratchpad/shop_probe_04_checked.png") +print(img.size) + +points = [ + ("checkbox_checked_center", (972, 297)), + ("checkbox_checked_bg", (975, 300)), + ("card_border_left", (940, 380)), + ("card_border_unselected", (1165, 380)), + ("buy_button", (1751, 1112)), + ("cancel_button", (1525, 1112)), +] + +for name, xy in points: + print(name, xy, img.getpixel(xy)) +PY + +python3 scratchpad/probe_image.py +``` + +Avoid this pattern: + +```bash +cd /private/tmp/claude-*/scratchpad && python3 -c "..." +``` + +Also avoid large multiline `python3 -c "..."` commands when they contain comments or complex quoting. Prefer a temporary script under `scratchpad/` because it is easier to inspect, rerun, and keep within workspace permissions. `screenshots/` contains human reference captures. They are useful for calibration and documentation, but they are not necessarily automated test fixtures. `assets/` contains local template images. These should be captured from the local game setup where possible. +## Temporary probe policy + +Temporary probes are allowed, but they must be workspace-local. + +For non-trivial investigation, use: + +```text +scratchpad/.py +``` + +Examples: + +```text +scratchpad/probe_shop_checkbox.py +scratchpad/probe_cafe_income.py +scratchpad/probe_story_modal.py +scratchpad/probe_ocr_region.py +``` + +A temporary probe should: + +- run from the repository root +- read inputs from `scratchpad/`, `screenshots/`, or `assets/` +- write outputs to `scratchpad/` +- avoid `/tmp`, `/private/tmp`, and Claude Code internal paths +- avoid relying on absolute `/private/tmp/claude-*` paths +- be deletable once the investigation is complete + +Prefer: + +```bash +python3 scratchpad/probe_shop_checkbox.py +``` + +Do not prefer: + +```bash +python3 -c "large multiline script..." +``` + +For very small one-liners, `python3 -c` is acceptable only when it does not require changing directories into `/private/tmp/claude-*`. + ## Testing and checks Because the game only runs on `nik-gpu`, local macOS testing is limited. Before deploying, run static/syntax checks locally: -``` +```bash bash -n ba_dailies.sh python3 -m py_compile ba_daily.py python3 -m py_compile ba_auto/*.py python3 -m py_compile ba_auto/tasks/*.py ``` +For temporary investigation scripts, use: + +```bash +python3 scratchpad/.py +``` + +Do not run temporary investigation scripts from Claude Code's internal `/private/tmp/claude-*` workspace. + On `nik-gpu`, run real integration tests against the live game. Example: -``` -ssh nik-gpu "~/ba_dailies.sh cafe" +```bash +ssh nik-gpu "~/repo/ba-auto-daily/ba_dailies.sh cafe" ``` -When debugging image matching, write debug images to `scratchpad/`. +When debugging image matching, write debug images to: + +```text +scratchpad/ +``` + +When debugging on `nik-gpu`, copy relevant screenshots or debug images back into the local project `scratchpad/` if they need local analysis. ## Existing features -Current project state (mailbox, cafe, stamina, and story_sweep all migrated to real Python; no Bash feature logic remains): +Current project state: mailbox, cafe, stamina, and story_sweep are all migrated to real Python. No Bash feature logic remains. - `ba_dailies.sh` is a thin launcher that execs `ba_daily.py` -- `ba_daily.py` dispatches `mailbox`/`cafe`/`stamina`/`story_sweep`/default flow to `ba_auto/tasks/`. Default flow (`DEFAULT_ORDER`) is `mailbox`, `cafe`, `stamina` — `story_sweep` is opt-in only since it spends AP rather than reclaiming something free -- `ba_auto/tasks/mailbox.py` and `ba_auto/tasks/cafe.py` click with `ba_auto/driver.py` primitives and verify state with `driver.color_at`/`ba_auto/navigation.py` (ported from `module/mail.py` and `module/cafe_reward.py`'s `rgb_in_range`/`co_detect` pattern) before pressing further keys — no legacy bridge remains -- `ba_auto/tasks/stamina.py` claims the Mission panel's bulk "一括受取" button (see `plan.md` Phase 8) -- `ba_auto/tasks/story_sweep.py` sweeps a config-driven list of exact `(region, stage, count)` targets (`config.STORY_SWEEP_TARGETS`) for AP, navigating to each via OCR (region-number readout + delta-click, stage-label matching) rather than a random pick (see `plan.md` Phase 10, which replaced Phase 9's random-pick design per Phase 9's own OCR-avoidance retrospective — this task is the concrete example behind the "OCR policy" above) +- `ba_daily.py` dispatches `mailbox`, `cafe`, `stamina`, `story_sweep`, and default flow to `ba_auto/tasks/` +- default flow is `mailbox`, `cafe`, `stamina` +- `story_sweep` is opt-in only since it spends AP rather than reclaiming something free +- `ba_auto/tasks/mailbox.py` and `ba_auto/tasks/cafe.py` click with `ba_auto/driver.py` primitives and verify state with `driver.color_at` and `ba_auto/navigation.py` +- mailbox and cafe were ported from the reference patterns around `module/mail.py` and `module/cafe_reward.py` +- no legacy bridge remains +- `ba_auto/tasks/stamina.py` claims the Mission panel's bulk `一括受取` button +- see `plan.md` Phase 8 for stamina details +- `ba_auto/tasks/story_sweep.py` sweeps a config-driven list of exact `(region, stage, count)` targets from `config.STORY_SWEEP_TARGETS` +- story sweep navigates via OCR, using region-number readout and stage-label matching, rather than random selection +- see `plan.md` Phase 10 for the OCR-based story sweep port - `ba_auto/detector.py` has `find_cafe_sparkle()`, the sparkle template-match ported in-process from the now-deleted `scripts/detect_and_click.py` -- `scripts/ba_dailies_legacy.sh` and `scripts/detect_and_click.py` have been deleted — nothing references them anymore, and the `scripts/` directory itself no longer exists -- `ba_auto/driver.py` primitives (`run_command`, `focus_game`, `click`, `move_mouse`, `scroll`, `keypress`, `screenshot`, `wait`, `color_at`) are wired into all four task modules -- `ba_auto/navigation.py` has two shared state probes used across tasks: `is_on_subscreen` (any mailbox/cafe/shop-style panel vs. the home screen) and `is_modal_open` (a dimmed dialog overlay), plus `wait_for_state()` (a scoped port of the reference's `core/picture.py::co_detect` — watch for any of several named states, react to known non-terminal ones, stop on a recognized terminal one; see `plan.md` Phase 10). `story_sweep.py` additionally has its own local modal probes/close logic because the stage-info modal is wide enough to break `is_modal_open`'s default probe point, and does not close on Escape at all (see `plan.md` Phase 9); its modal also renders at least two different internal layouts (a plain one for the bonus "-A" stage, a taller tabbed one for regular numbered stages) whose button coordinates differ, discovered live in Phase 10 -- Live testing found both the mailbox-icon and cafe-icon fixed coordinates were flaky (missed the first click, worked on retry) and that neither task verified anything before proceeding, so a missed click cascaded into blind actions and could reach an unverified Escape press on the home screen — which triggers Blue Archive's own "exit the game?" confirmation. See `plan.md` Phases 5–6 for the full writeup. This is the concrete reason every task now verifies state before acting rather than trusting fixed coordinates or a single click blindly -- Rank-up popups mid-pat-loop are now handled: a pat that crosses an affection-rank threshold shows a full-screen "絆ランクアップ!" cutscene with no cafe header at all, which `find_cafe_sparkle()` can never recognize (it's nothing like the sparkle template) — the loop used to just spin uselessly against it for the rest of the room's click budget. `cafe.py`'s `_dismiss_rank_up_if_shown()` reuses the existing `navigation.is_on_subscreen` header-brightness probe (confirmed against `screenshots/cafe/student/01-02`: it reads r<200 during the cutscene vs. r>200 on the normal cafe screen) to detect it and press Enter until it clears, porting the reference's own `to_cafe()`-navigation handling of `relationship_rank_up` (`module/cafe_reward.py`). Not yet live-confirmed against a real rank-up trigger (it's semi-random, tied to hitting an affection threshold) — the fix is grounded in real captured screenshots, not a live end-to-end run; treat as implemented-but-unverified until one happens naturally during a real run -- Not yet verified for cafe: whether camera zoom/pan can drift over a long unattended run (the reference project zooms out before detecting; ours does not, and testing didn't reproduce a failure from skipping it — see `plan.md` Phase 6 "Not verified" list) +- `scripts/ba_dailies_legacy.sh` and `scripts/detect_and_click.py` have been deleted +- the `scripts/` directory itself no longer exists +- `ba_auto/driver.py` primitives are wired into all migrated task modules +- existing driver primitives include `run_command`, `focus_game`, `click`, `move_mouse`, `scroll`, `keypress`, `screenshot`, `wait`, and `color_at` +- `ba_auto/navigation.py` has shared state probes used across tasks: + - `is_on_subscreen` + - `is_modal_open` + - `wait_for_state()` -Migration goal: +`wait_for_state()` is a scoped port of the reference's `core/picture.py::co_detect`: watch for any of several named states, react to known non-terminal ones, and stop on a recognized terminal one. -- preserve existing behavior -- move feature logic from Bash into Python -- keep `ba_dailies.sh` as launcher only -- make detection logic reusable -- prepare for future features by copying the reference project's structure where appropriate +`story_sweep.py` additionally has its own local modal probes and close logic because the stage-info modal is wide enough to break `is_modal_open`'s default probe point, and it does not close on Escape. See `plan.md` Phase 9. + +The story sweep modal renders at least two different internal layouts: + +- a plain one for the bonus `-A` stage +- a taller tabbed one for regular numbered stages + +Their button coordinates differ. This was discovered live in Phase 10. + +Live testing found both the mailbox-icon and cafe-icon fixed coordinates were flaky. They missed the first click and worked on retry. Neither task originally verified state before proceeding, so a missed click cascaded into blind actions and could reach an unverified Escape press on the home screen, which triggers Blue Archive's own "exit the game?" confirmation. + +See `plan.md` Phases 5–6 for the full writeup. This is the concrete reason every task now verifies state before acting rather than trusting fixed coordinates or a single click blindly. + +Rank-up popups mid-pat-loop are now handled. + +A pat that crosses an affection-rank threshold shows a full-screen `絆ランクアップ!` cutscene with no cafe header. `find_cafe_sparkle()` can never recognize this because it is nothing like the sparkle template. The loop used to spin uselessly against it for the rest of the room's click budget. + +`cafe.py`'s `_dismiss_rank_up_if_shown()` reuses the existing `navigation.is_on_subscreen` header-brightness probe. This was confirmed against `screenshots/cafe/student/01-02`: + +- rank-up cutscene: header probe reads `r < 200` +- normal cafe screen: header probe reads `r > 200` + +It presses Enter until the cutscene clears, porting the reference's own `to_cafe()` navigation handling of `relationship_rank_up` from `module/cafe_reward.py`. + +This is not yet live-confirmed against a real rank-up trigger. It is grounded in real captured screenshots, but treat it as implemented-but-unverified until one happens naturally during a real run. + +Not yet verified for cafe: + +- whether camera zoom/pan can drift over a long unattended run +- the reference project zooms out before detecting +- this project currently does not +- testing did not reproduce a failure from skipping it +- see `plan.md` Phase 6 "Not verified" list ## Known cafe gaps to verify during migration -When migrating cafe logic, verify these manually against the actual current behavior: +When migrating or modifying cafe logic, verify these manually against the actual current behavior: - whether student rotation popups are handled - whether rank-up popups are handled @@ -473,13 +713,23 @@ When migrating cafe logic, verify these manually against the actual current beha Do not assume the old Bash implementation handles these correctly. +## Migration goal + +The migration goal is: + +- preserve existing behavior +- move feature logic from Bash into Python +- keep `ba_dailies.sh` as launcher only +- make detection logic reusable +- prepare for future features by copying the reference project's structure where appropriate + ## Anti-patterns Do not: - add new `do_` Bash functions - build a giant Bash automation script -- recreate reference logic as fixed coordinate Bash click chains +- recreate reference logic as fixed-coordinate Bash click chains - skip reading the reference module before implementing a feature - put OCR in Bash - put OpenCV state machines in Bash @@ -488,8 +738,13 @@ Do not: - copy reference image assets blindly - edit `~/repo/baas-reference/` - implement event-specific features before the generic reusable machinery exists -- invent a pixel-probe, fixed-coordinate, or randomized substitute for reference logic that uses OCR, just to avoid setting up OCR -- treat OCR as an Android-specific concern that can be deferred indefinitely — it isn't, and deferring it has already produced more bugs than it saved (see "OCR policy" above) +- invent a pixel probe, fixed-coordinate, or randomized substitute for reference logic that uses OCR, just to avoid setting up OCR +- treat OCR as an Android-specific concern that can be deferred indefinitely +- run temporary probes from `/private/tmp/claude-*` instead of the repository `scratchpad/` +- create investigation scripts outside the repository when they can live in `scratchpad/` +- use multiline `python3 -c "..."` commands for non-trivial probes +- rely on Claude Code internal temporary paths in committed code, notes, commands, or debugging workflow +- write generated debug files into the repository root when they belong in `scratchpad/` ## Feature implementation workflow @@ -497,7 +752,7 @@ For every new feature: 1. Read the relevant `~/repo/baas-reference/module/...` file. 2. Summarize the upstream feature flow in notes or comments. -3. Add/update the reference mapping table. +3. Add or update the reference mapping table. 4. Identify missing local driver primitives. 5. Implement or improve those primitives in `ba_auto/driver.py` or `ba_auto/detector.py`. 6. Implement the feature in `ba_auto/tasks/.py`. @@ -507,17 +762,20 @@ For every new feature: 10. Deploy to `nik-gpu`. 11. Test against the live game. 12. Update `plan.md` status. +13. Move temporary investigation files into `scratchpad/`, or delete them if they are no longer useful. ## Priority when uncertain When uncertain, prefer this order: 1. Preserve existing working behavior. -2. Follow the reference project's control flow — including its use of OCR. Do not deprioritize OCR-based navigation/matching in favor of a simpler non-OCR substitute; that has already produced more bugs than the OCR setup would have cost (see "OCR policy" above). -3. Keep logic in Python. -4. Add reusable driver primitives instead of feature-specific hacks. -5. Use local screenshots/assets only when backend differences require it. -6. Avoid large rewrites that do not move the project closer to reference-driven Python architecture. +2. Follow the reference project's control flow, including its use of OCR. +3. Do not deprioritize OCR-based navigation/matching in favor of a simpler non-OCR substitute. +4. Keep logic in Python. +5. Add reusable driver primitives instead of feature-specific hacks. +6. Use local screenshots/assets only when backend differences require it. +7. Use `scratchpad/` for temporary work and diagnostics. +8. Avoid large rewrites that do not move the project closer to reference-driven Python architecture. ## User preference @@ -525,4 +783,6 @@ The user wants this project to be as close to the original Blue Archive Auto Scr The user specifically does not want Claude Code to keep converting feature work into Bash. -Respect that preference. +The user also wants Claude Code to avoid using `/private/tmp/claude-*` for development probes when a repository-local `scratchpad/` directory can be used instead, because those paths trigger permission prompts and interrupt the development flow. + +Respect those preferences. \ No newline at end of file