Nik Afiq
a8ff95d6f5
- Added `wait_for_state` function in `navigation.py` for state monitoring and reaction handling. - Updated `mapping.md` to reflect changes in story sweep implementation and OCR usage. - Refactored `story_sweep.py` to utilize OCR for region and stage identification, replacing random selection with configured targets. - Enhanced modal handling and confirmation checks for AP usage in `story_sweep.py`. - Updated setup script to require `tesseract` for OCR functionality and included installation instructions. - Revised `plan.md` to document the transition from heuristic to OCR-based stage targeting and the associated findings from live testing.
117 lines
7.2 KiB
Markdown
117 lines
7.2 KiB
Markdown
# ba-auto-daily
|
||
|
||
Personal automation for Blue Archive JP daily tasks (mailbox, cafe, ...), driven by desktop control (`xdotool`/`scrot`) against the real PC/Steam/Proton client.
|
||
|
||
## How it's wired together
|
||
|
||
Two machines are involved:
|
||
|
||
- **`nik-macbookair`** (this Mac) — where the code is edited. The game does not run here and none of this can be tested locally.
|
||
- **`nik-gpu`** (Linux) — where the actual Blue Archive client runs under Steam/Proton, and where every command below actually executes.
|
||
|
||
Call path once deployed to `nik-gpu`:
|
||
|
||
```
|
||
~/ba_dailies.sh [command] <- thin Bash launcher, no game logic
|
||
|
|
||
v
|
||
~/ba_daily.py [command] <- Python CLI, dispatches to a task
|
||
|
|
||
v
|
||
~/ba_auto/tasks/<command>.py <- the actual click/verify logic
|
||
```
|
||
|
||
`ba_dailies.sh` only picks the venv Python and execs `ba_daily.py`. All real logic — clicking, screenshotting, verifying game state — lives in Python under `ba_auto/`.
|
||
|
||
Current task status:
|
||
|
||
| Command | Implementation |
|
||
|---|---|
|
||
| `mailbox` | Real Python (`ba_auto/tasks/mailbox.py`). Verifies the mailbox panel actually opened (via a pixel-color probe, `driver.color_at`) before clicking "claim all" or pressing any further keys. Retries the open-click up to 3 times before giving up safely. |
|
||
| `cafe` | Real Python (`ba_auto/tasks/cafe.py`). Verifies each room/dialog transition the same way as `mailbox` before acting; sparkle detection runs in-process via `ba_auto/detector.py` instead of shelling out per click. See "Fixed: the exit-game dialog bug" below for what this replaced, and `plan.md` Phase 6 follow-up for the multi-scale detection + persistent-polling changes made after a "farming affection doesn't happen" report. |
|
||
| `stamina` | Real Python (`ba_auto/tasks/stamina.py`). Opens the Mission panel and claims via its bulk "一括受取" button (Enter key) when enabled. Does **not** touch the Pyroxene Purchase (青輝石購入) menu's free-AP claim — that's a real-money purchase screen and was deliberately left unautomated; see `plan.md` Phase 8. |
|
||
| `story_sweep` | Real Python (`ba_auto/tasks/story_sweep.py`). **Spends AP** — opt-in only, not part of the default flow. Sweeps a config-driven list of exact `(region, stage, count)` targets (`config.STORY_SWEEP_TARGETS` — edit this before running for real, it ships with a placeholder), navigating to each via OCR (region-number readout + stage-label matching) rather than a random pick. Verifies the MAX/`+` click actually raised the count before starting the sweep, clicks through the AP-usage-confirmation dialog, and closes the stage-info modal via its own X button afterward (Escape doesn't close it — confirmed live). See `plan.md` Phase 10. |
|
||
|
||
## Prerequisites on nik-gpu
|
||
|
||
One-time, or after a dependency change:
|
||
|
||
```bash
|
||
ssh nik-gpu
|
||
which xdotool scrot tesseract # all three must be installed
|
||
```
|
||
|
||
`tesseract` (the OCR engine `story_sweep` uses for region/stage-label reads) needs `sudo apt install tesseract-ocr` — `setup.sh` checks for it but can't install it for you, since sudo needs an interactive password. `setup.sh` (see below) creates the Python venv (including `pytesseract`) and checks all three tools for you.
|
||
|
||
## Deploying your changes
|
||
|
||
From `nik-macbookair`, in the repo root:
|
||
|
||
```bash
|
||
rsync -av --delete \
|
||
--exclude='.git/' --exclude='__pycache__/' --exclude='*.pyc' \
|
||
--exclude='.claude/settings.local.json' --exclude='graphify-out/' \
|
||
--exclude='screenshots/' \
|
||
./ nik-gpu:~/repo/ba-auto-daily/
|
||
|
||
ssh nik-gpu "cd ~/repo/ba-auto-daily && ./setup.sh"
|
||
```
|
||
|
||
`setup.sh` copies the synced files into the fixed runtime paths (`~/ba_dailies.sh`, `~/ba_daily.py`, `~/ba_auto/`, `~/ba_assets/`) and (re)installs the venv. Re-run both commands any time you change code — there is no auto-deploy.
|
||
|
||
## Running it
|
||
|
||
The game must already be running on `nik-gpu` (window titled `BlueArchive`). Then, on `nik-gpu`:
|
||
|
||
```bash
|
||
~/ba_dailies.sh # default: mailbox, cafe, then stamina
|
||
~/ba_dailies.sh mailbox # just mailbox
|
||
~/ba_dailies.sh cafe # just cafe
|
||
~/ba_dailies.sh stamina # just the Mission-panel bulk claim
|
||
~/ba_dailies.sh story_sweep # spends AP -- not in the default flow, run explicitly
|
||
```
|
||
|
||
You can also run these remotely without a separate `ssh` login step:
|
||
|
||
```bash
|
||
ssh nik-gpu "~/ba_dailies.sh mailbox"
|
||
```
|
||
|
||
Exit code `0` means the script ran to completion without a Python exception — it does **not** by itself guarantee the in-game action succeeded. `mailbox`, `cafe`, and `stamina` all log what they actually detected and did (or why they safely aborted), so check the log output, not just the exit code.
|
||
|
||
## How to watch/verify it
|
||
|
||
- **Easiest: watch it live.** If you have Moonlight (or similar) streaming `nik-gpu`'s desktop, just open that and run the command from another terminal — you'll see the clicks happen in real time.
|
||
- **No stream handy: pull a screenshot after the fact.**
|
||
```bash
|
||
ssh nik-gpu "DISPLAY=:0 XAUTHORITY=/run/user/1000/gdm/Xauthority scrot -o /tmp/check.png"
|
||
scp nik-gpu:/tmp/check.png .
|
||
open check.png
|
||
```
|
||
- **Read the log output.** Each task prints what it's doing, e.g. `mailbox` prints `panel not detected after click (attempt N/3)` if a click misses, and `nothing to claim` vs `claiming all` depending on what it found.
|
||
|
||
## Fixed: the exit-game dialog bug
|
||
|
||
Live-testing both `mailbox` and `cafe` surfaced the same real bug in the *original*, unverified click sequences (see `plan.md` Phases 5–6 for the full writeup): a slightly-off icon coordinate occasionally caused a missed click, and the fixed sequence blindly kept going — clicking, pressing Enter, pressing Escape — with no idea whether any of it landed. The trailing Escape ended up hitting the bare home screen, which triggers Blue Archive's own **"Exit the game?"** confirmation dialog. `cafe`'s sequence is much longer than `mailbox`'s (open → pat loop ×15 → switch room → pat loop ×15 → claim income → exit), so a missed click there had more room to cascade.
|
||
|
||
Both `mailbox.py` and `cafe.py` now guard against this: they verify state via pixel-color probes before acting, retry a bounded number of times on a missed click, and abort safely (no further keypresses) instead of guessing. This was verified against the live game, including an actual income claim (gold and AP increased as expected) and a real sparkle detect-and-click.
|
||
|
||
That said, this is one round of live testing, not exhaustive coverage — see `plan.md` Phase 6 "Not verified" for open risks (rank-up popups mid-loop, long-run camera zoom/pan drift). If you ever see an unexpected confirmation dialog while running either command, press **Escape/Cancel**, never Enter/OK, until you've confirmed what it's asking.
|
||
|
||
## Local checks (nik-macbookair)
|
||
|
||
The game can't run here, so this only catches syntax errors, not behavior:
|
||
|
||
```bash
|
||
bash -n ba_dailies.sh
|
||
python3 -m py_compile ba_daily.py ba_auto/*.py ba_auto/tasks/*.py
|
||
```
|
||
|
||
Real verification only happens by actually running against the live game on `nik-gpu`, per "How to watch/verify it" above.
|
||
|
||
## More detail
|
||
|
||
- `CLAUDE.md` — architecture rules and conventions for this repo.
|
||
- `plan.md` — feature-by-feature migration status and backlog, including the mailbox/cafe exit-game-dialog bug writeup.
|
||
- `ba_auto/reference_notes/mapping.md` — maps each feature to its `baas-reference` source.
|