docs: Update CLAUDE.md to clarify project goals, OCR policy, and migration details
This commit is contained in:
parent
b2ed571908
commit
4842d575a5
410
CLAUDE.md
410
CLAUDE.md
@ -8,29 +8,55 @@ This repository is a personal Blue Archive JP daily-automation project.
|
||||
|
||||
It controls the real Blue Archive PC/Steam/Proton client running on a Linux machine named `nik-gpu`. The development machine is a MacBook named `nik-macbookair`.
|
||||
|
||||
**The original goal is a backend port, not a rewrite.** The reference project (`~/repo/baas-reference/`) only works against an Android emulator via ADB/uiautomator2, typically run on a Windows PC. This project's job is to make that same automation logic work against the real JP client running under Proton on Linux instead — porting the reference's decision logic, navigation, and state-detection approach (including its use of OCR) as faithfully as practical, and replacing only the parts that are genuinely Android-specific (ADB/uiautomator2 input and UI-object-tree queries) with local desktop equivalents (xdotool, scrot, OpenCV, OCR). Inventing a different, simpler approach to avoid porting a piece of reference logic (most notably OCR — see below) is a deviation from that goal, not a shortcut toward it.
|
||||
**The original goal is a backend port, not a rewrite.** The reference project (`~/repo/baas-reference/`) only works against an Android emulator via ADB/uiautomator2, typically run on a Windows PC. This project's job is to make that same automation logic work against the real JP client running under Proton on Linux instead — porting the reference's decision logic, navigation, and state-detection approach, including its use of OCR, as faithfully as practical, and replacing only the parts that are genuinely Android-specific with local desktop equivalents.
|
||||
|
||||
Android-specific parts include:
|
||||
|
||||
- ADB input
|
||||
- uiautomator2 taps
|
||||
- Android UI-object-tree queries
|
||||
- emulator-specific assumptions
|
||||
|
||||
Local desktop equivalents include:
|
||||
|
||||
- `xdotool`
|
||||
- `scrot`
|
||||
- OpenCV
|
||||
- OCR against screenshots
|
||||
- local image/color/template detection
|
||||
|
||||
Inventing a different, simpler approach to avoid porting a piece of reference logic is a deviation from the project goal, not a shortcut toward it.
|
||||
|
||||
The automation backend is local desktop control:
|
||||
|
||||
- `xdotool` for mouse/keyboard/window control
|
||||
- `scrot` for screenshots
|
||||
- Python/OpenCV for image matching and color/template detection
|
||||
- OCR (Tesseract or PaddleOCR), ported wherever the reference implementation uses it for a feature — see "OCR policy" below
|
||||
- OCR, using Tesseract or PaddleOCR, wherever the reference implementation uses OCR for a feature
|
||||
- no Android emulator control
|
||||
- no ADB
|
||||
- no uiautomator2
|
||||
|
||||
### OCR policy
|
||||
|
||||
OCR is **not** an Android-specific concern — the reference's OCR-driven region/stage-name/currency matching runs against a screenshot and would work identically against a `scrot` capture on this backend. Only the reference's *input* (ADB/uiautomator2 taps) is Android-specific.
|
||||
OCR is **not** an Android-specific concern. The reference's OCR-driven region, stage-name, and currency matching runs against a screenshot and can be ported to this backend using `scrot` captures. Only the reference's input mechanism is Android-specific.
|
||||
|
||||
Two features (Stamina/AP mission claim, Normal/Hard story AP sweep) were originally built without OCR by substituting ad hoc pixel-probes, fixed coordinates, or randomized selection for the reference's OCR-driven navigation. For story sweep, that substitution produced more real bugs during live testing than porting the reference's actual approach would have — a wrong modal-open probe, an unverified button click that silently under-spent AP, a modal that doesn't close on Escape, and a latent hazard where a mistimed keypress could have started a real battle (see `plan.md` Phase 9's retrospective for the full writeup). None of those bugs would exist if the reference's deterministic, OCR-based stage targeting had been ported instead of replaced with a heuristic guess. **This has since been fixed** — `story_sweep.py` now ports the reference's actual OCR-based region/stage targeting (see `plan.md` Phase 10) — but the retrospective stays here as the concrete, lived reason for the policy below, not a description of the current state of that task.
|
||||
Two features, Stamina/AP mission claim and Normal/Hard story AP sweep, were originally built without OCR by substituting ad hoc pixel probes, fixed coordinates, or randomized selection for the reference's OCR-driven navigation. For story sweep, that substitution produced more real bugs during live testing than porting the reference's actual approach would have:
|
||||
|
||||
Going forward: when a reference feature's control flow depends on OCR, set up OCR and port that logic, rather than inventing a non-OCR workaround to avoid the setup cost. Only skip OCR for a specific step if the reference itself doesn't use OCR there.
|
||||
- a wrong modal-open probe
|
||||
- an unverified button click that silently under-spent AP
|
||||
- a modal that does not close on Escape
|
||||
- a latent hazard where a mistimed keypress could have started a real battle
|
||||
|
||||
See `plan.md` Phase 9's retrospective for the full writeup.
|
||||
|
||||
That issue has since been fixed: `story_sweep.py` now ports the reference's actual OCR-based region/stage targeting. See `plan.md` Phase 10. The retrospective stays here as the concrete, lived reason for the OCR policy, not as a description of the current state of that task.
|
||||
|
||||
Going forward: when a reference feature's control flow depends on OCR, set up OCR and port that logic. Do not invent a non-OCR workaround just to avoid the setup cost. Only skip OCR for a specific step if the reference itself does not use OCR there.
|
||||
|
||||
The reference project is located at:
|
||||
|
||||
```
|
||||
```text
|
||||
~/repo/baas-reference/
|
||||
```
|
||||
|
||||
@ -48,7 +74,7 @@ All feature logic must live in Python.
|
||||
|
||||
The intended structure is:
|
||||
|
||||
```
|
||||
```text
|
||||
ba-auto-daily/
|
||||
├── ba_dailies.sh
|
||||
├── ba_daily.py
|
||||
@ -69,12 +95,13 @@ ba-auto-daily/
|
||||
│ └── mapping.md
|
||||
├── assets/
|
||||
├── screenshots/
|
||||
├── scratchpad/
|
||||
├── setup.sh
|
||||
├── plan.md
|
||||
└── CLAUDE.md
|
||||
```
|
||||
|
||||
(`scripts/` previously held one-off Bash/Python helpers; it was deleted once mailbox and cafe migrated off it — see "Existing features" below. Do not recreate it as a place to stash feature logic.)
|
||||
`scripts/` previously held one-off Bash/Python helpers. It was deleted once mailbox and cafe migrated off it. Do not recreate `scripts/` as a place to stash feature logic.
|
||||
|
||||
The exact layout can evolve, but the architectural rule should not change:
|
||||
|
||||
@ -87,7 +114,7 @@ The exact layout can evolve, but the architectural rule should not change:
|
||||
|
||||
Before implementing any new feature, inspect the matching reference implementation in:
|
||||
|
||||
```
|
||||
```text
|
||||
~/repo/baas-reference/module/
|
||||
```
|
||||
|
||||
@ -105,7 +132,7 @@ For every feature, first identify:
|
||||
|
||||
Then implement the feature in Python under:
|
||||
|
||||
```
|
||||
```text
|
||||
ba_auto/tasks/
|
||||
```
|
||||
|
||||
@ -128,17 +155,30 @@ The reference repository is **read-only**.
|
||||
- edit files in `~/repo/baas-reference/`
|
||||
- reimplement a reference feature from scratch in Bash
|
||||
- create a local solution that ignores the reference flow when a reference implementation already exists
|
||||
- substitute a pixel-probe, fixed-coordinate, or randomized shortcut for reference logic that uses OCR, just to avoid setting up OCR (see "OCR policy" above)
|
||||
- substitute a pixel probe, fixed coordinate, or randomized shortcut for reference logic that uses OCR, just to avoid setting up OCR
|
||||
|
||||
## Two-machine architecture
|
||||
|
||||
Development happens on: `nik-macbookair`
|
||||
Development happens on:
|
||||
|
||||
Runtime happens on: `nik-gpu`
|
||||
```text
|
||||
nik-macbookair
|
||||
```
|
||||
|
||||
Runtime happens on:
|
||||
|
||||
```text
|
||||
nik-gpu
|
||||
```
|
||||
|
||||
The Blue Archive client and X display live on `nik-gpu`.
|
||||
|
||||
Assume: (`DISPLAY=:0`, GDM `XAUTHORITY` under `/run/user/1000`)
|
||||
Assume:
|
||||
|
||||
```text
|
||||
DISPLAY=:0
|
||||
GDM XAUTHORITY under /run/user/1000
|
||||
```
|
||||
|
||||
The game runs under Steam/Proton on the Linux desktop.
|
||||
|
||||
@ -150,7 +190,7 @@ During iteration, files are pushed from `nik-macbookair` to `nik-gpu`.
|
||||
|
||||
Typical paths on `nik-gpu`:
|
||||
|
||||
```
|
||||
```text
|
||||
~/ba_dailies.sh
|
||||
~/ba_daily.py
|
||||
~/ba_auto/
|
||||
@ -160,7 +200,7 @@ Typical paths on `nik-gpu`:
|
||||
|
||||
The current setup may still contain older paths such as:
|
||||
|
||||
```
|
||||
```text
|
||||
~/ba_scripts/detect_and_click.py
|
||||
```
|
||||
|
||||
@ -170,12 +210,28 @@ When refactoring, prefer consolidating Python code into `ba_auto/`.
|
||||
|
||||
For a fresh checkout on `nik-gpu`, run from the repo root:
|
||||
|
||||
```
|
||||
```bash
|
||||
./setup.sh
|
||||
```
|
||||
|
||||
After initial setup, individual changes may be pushed with `scp` or `rsync`.
|
||||
|
||||
Prefer syncing the project from the repository root:
|
||||
|
||||
```bash
|
||||
rsync -av \
|
||||
--exclude='.git/' \
|
||||
--exclude='__pycache__/' \
|
||||
--exclude='*.pyc' \
|
||||
--exclude='.claude/settings.local.json' \
|
||||
--exclude='graphify-out/' \
|
||||
--exclude='screenshots/' \
|
||||
--exclude='scratchpad/' \
|
||||
./ nik-gpu:~/repo/ba-auto-daily/
|
||||
```
|
||||
|
||||
Do not sync Claude Code internal temporary folders.
|
||||
|
||||
## Runtime dependencies on nik-gpu
|
||||
|
||||
These are host-level dependencies. Confirm they exist before assuming a bug is in the project code.
|
||||
@ -190,20 +246,28 @@ Required Python packages:
|
||||
|
||||
- `opencv-python` or `opencv-python-headless`
|
||||
- `numpy`
|
||||
- `pytesseract`
|
||||
|
||||
Expected venv:
|
||||
|
||||
```
|
||||
```text
|
||||
~/.venvs/ba-auto-daily/bin/python3
|
||||
```
|
||||
|
||||
Quick check:
|
||||
|
||||
```
|
||||
```bash
|
||||
ssh nik-gpu "which xdotool scrot tesseract && ~/.venvs/ba-auto-daily/bin/python3 -c 'import cv2, numpy, pytesseract; print(cv2.__version__)'"
|
||||
```
|
||||
|
||||
OCR engine dependency: **set up** (Phase 10) — Tesseract via the `tesseract-ocr` apt package (needs an interactive `sudo`, so `setup.sh` checks for it but can't install it for you) plus `pytesseract` in the venv. `ba_auto/detector.py`'s `read_text()`/`read_int()` wrap it for occasional single-crop reads; see `story_sweep.py` for the first real usage.
|
||||
OCR engine dependency is set up as of Phase 10:
|
||||
|
||||
- Tesseract via the `tesseract-ocr` apt package
|
||||
- `pytesseract` in the venv
|
||||
|
||||
Installing `tesseract-ocr` needs interactive `sudo`, so `setup.sh` should check for it but should not assume it can install it automatically.
|
||||
|
||||
`ba_auto/detector.py`'s `read_text()` and `read_int()` wrap OCR for occasional single-crop reads. See `story_sweep.py` for the first real usage.
|
||||
|
||||
## Bash policy
|
||||
|
||||
@ -214,8 +278,10 @@ Preferred shape:
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
VENV_PYTHON="${VENV_PYTHON:-$HOME/.venvs/ba-auto-daily/bin/python3}"
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
exec "$VENV_PYTHON" "$SCRIPT_DIR/ba_daily.py" "$@"
|
||||
```
|
||||
|
||||
@ -244,13 +310,13 @@ If an existing Bash function exists, migrate it to Python rather than extending
|
||||
|
||||
The intended Python CLI entry point is:
|
||||
|
||||
```
|
||||
```text
|
||||
ba_daily.py
|
||||
```
|
||||
|
||||
It should support commands such as:
|
||||
|
||||
```
|
||||
```bash
|
||||
./ba_dailies.sh
|
||||
./ba_dailies.sh mailbox
|
||||
./ba_dailies.sh cafe
|
||||
@ -261,7 +327,7 @@ It should support commands such as:
|
||||
|
||||
No argument should run the default daily flow.
|
||||
|
||||
Current default flow (`ba_daily.py`'s `DEFAULT_ORDER`):
|
||||
Current default flow in `ba_daily.py`'s `DEFAULT_ORDER`:
|
||||
|
||||
1. focus game
|
||||
2. mailbox
|
||||
@ -269,15 +335,19 @@ Current default flow (`ba_daily.py`'s `DEFAULT_ORDER`):
|
||||
4. stamina
|
||||
5. future daily tasks
|
||||
|
||||
`story_sweep` spends AP rather than reclaiming something free, so it is deliberately excluded from the default flow — it must be invoked explicitly.
|
||||
`story_sweep` spends AP rather than reclaiming something free, so it is deliberately excluded from the default flow. It must be invoked explicitly.
|
||||
|
||||
The CLI should dispatch into task modules under `ba_auto/tasks/`.
|
||||
The CLI should dispatch into task modules under:
|
||||
|
||||
```text
|
||||
ba_auto/tasks/
|
||||
```
|
||||
|
||||
## Driver layer
|
||||
|
||||
Create and maintain a driver layer in:
|
||||
|
||||
```
|
||||
```text
|
||||
ba_auto/driver.py
|
||||
```
|
||||
|
||||
@ -285,14 +355,15 @@ The driver layer should wrap the PC/Steam/Proton backend.
|
||||
|
||||
It should provide reusable primitives such as:
|
||||
|
||||
```
|
||||
```python
|
||||
focus_game()
|
||||
screenshot()
|
||||
click(x, y)
|
||||
double_click(x, y)
|
||||
drag/swipe(start, end, duration)
|
||||
drag(start, end, duration)
|
||||
swipe(start, end, duration)
|
||||
keypress(key)
|
||||
sleep/wait
|
||||
wait(seconds)
|
||||
wait_until(...)
|
||||
color_at(...)
|
||||
region_average_color(...)
|
||||
@ -300,7 +371,7 @@ template_match(...)
|
||||
find_and_click_template(...)
|
||||
```
|
||||
|
||||
Feature modules should not directly shell out to `xdotool` or `scrot` unless a driver primitive is missing and being added.
|
||||
Feature modules should not directly shell out to `xdotool` or `scrot` unless a driver primitive is missing and is being added.
|
||||
|
||||
Prefer:
|
||||
|
||||
@ -311,16 +382,22 @@ driver.click(x, y)
|
||||
over:
|
||||
|
||||
```python
|
||||
subprocess.run(["xdotool", "click", ...])
|
||||
subprocess.run(["xdotool", "click", "1"])
|
||||
```
|
||||
|
||||
This keeps the project close to the reference architecture while replacing only the backend-control layer.
|
||||
|
||||
## Detector layer
|
||||
|
||||
Image matching and color matching should live in `ba_auto/detector.py` or in clearly named helper classes/functions.
|
||||
Image matching, color matching, and OCR wrappers should live in:
|
||||
|
||||
The cafe sparkle-matching logic formerly in `scripts/detect_and_click.py` (now retired) has been migrated here as `find_cafe_sparkle()`, called in-process rather than as a per-click subprocess.
|
||||
```text
|
||||
ba_auto/detector.py
|
||||
```
|
||||
|
||||
or in clearly named helper classes/functions.
|
||||
|
||||
The cafe sparkle-matching logic formerly in `scripts/detect_and_click.py` has been migrated into `ba_auto/detector.py` as `find_cafe_sparkle()`. It should be called in-process rather than as a per-click subprocess.
|
||||
|
||||
The detector should support:
|
||||
|
||||
@ -329,13 +406,18 @@ The detector should support:
|
||||
- threshold tuning
|
||||
- masked matching
|
||||
- click-offset handling
|
||||
- OCR crop reads
|
||||
- debug image output to `scratchpad/`
|
||||
|
||||
Avoid one Python cold start per click attempt where possible. Prefer long-running Python task logic that can take repeated screenshots and click repeatedly from one process.
|
||||
|
||||
## Navigation layer
|
||||
|
||||
Common navigation should live in `ba_auto/navigation.py`.
|
||||
Common navigation should live in:
|
||||
|
||||
```text
|
||||
ba_auto/navigation.py
|
||||
```
|
||||
|
||||
Use this for shared flows such as:
|
||||
|
||||
@ -347,16 +429,21 @@ Use this for shared flows such as:
|
||||
- opening lesson/schedule
|
||||
- closing popups
|
||||
- generic back/escape handling
|
||||
- waiting for known UI states
|
||||
|
||||
Do not duplicate navigation click sequences inside every task if they can be shared.
|
||||
|
||||
## Task modules
|
||||
|
||||
Each feature should have a task module: `ba_auto/tasks/<feature>.py`
|
||||
|
||||
Example:
|
||||
Each feature should have a task module:
|
||||
|
||||
```text
|
||||
ba_auto/tasks/<feature>.py
|
||||
```
|
||||
|
||||
Examples:
|
||||
|
||||
```text
|
||||
ba_auto/tasks/mailbox.py
|
||||
ba_auto/tasks/cafe.py
|
||||
ba_auto/tasks/stamina.py
|
||||
@ -366,13 +453,13 @@ ba_auto/tasks/group.py
|
||||
|
||||
Each task module should expose a clear function such as:
|
||||
|
||||
```
|
||||
```python
|
||||
run(driver, config)
|
||||
```
|
||||
|
||||
or:
|
||||
|
||||
```
|
||||
```python
|
||||
run_mailbox(driver, config)
|
||||
```
|
||||
|
||||
@ -382,7 +469,7 @@ Keep task files feature-focused.
|
||||
|
||||
Maintain a mapping file at:
|
||||
|
||||
```
|
||||
```text
|
||||
ba_auto/reference_notes/mapping.md
|
||||
```
|
||||
|
||||
@ -390,78 +477,231 @@ Before or during implementation of a feature, update the mapping.
|
||||
|
||||
Use this format:
|
||||
|
||||
```markdown
|
||||
| Local feature | Reference file | Reference functions/classes | Local file | Backend replacements | Status |
|
||||
|---|---|---|---|---|---|
|
||||
| Cafe | `module/cafe.py` or relevant file | `...` | `ba_auto/tasks/cafe.py` | uiautomator2 tap -> xdotool click, screenshot -> scrot/OpenCV | In progress |
|
||||
```
|
||||
|
||||
## Working conventions
|
||||
|
||||
Use `scratchpad/` (create if missing) in the project root for temporary or intermediate files.
|
||||
Always work from the repository root:
|
||||
|
||||
```bash
|
||||
cd ~/repo/ba-auto-daily
|
||||
```
|
||||
|
||||
Use `scratchpad/` in the project root for all temporary, generated, diagnostic, calibration, and investigation files.
|
||||
|
||||
Examples:
|
||||
|
||||
- cropped calibration images, e.g. from `screenshots/cafe/sparkle/`
|
||||
- cropped calibration images
|
||||
- debug screenshots
|
||||
- annotated match results
|
||||
- temporary Python probes
|
||||
- temporary investigation notes
|
||||
- one-off image-analysis scripts
|
||||
- temporary OCR experiments
|
||||
- screenshots copied back from `nik-gpu` for analysis
|
||||
|
||||
Do not use `/tmp` or `/private/tmp` unless there is a strong reason.
|
||||
Create it if missing:
|
||||
|
||||
```bash
|
||||
mkdir -p scratchpad
|
||||
```
|
||||
|
||||
Do **not** use Claude Code's internal temporary directories such as:
|
||||
|
||||
```text
|
||||
/private/tmp/claude-*
|
||||
/tmp/claude-*
|
||||
```
|
||||
|
||||
Do **not** run probes from `/private/tmp/claude-*` or write temporary scripts there. These paths are outside the repository and often trigger extra permission prompts, interrupting the development flow.
|
||||
|
||||
When a temporary Python probe is needed, write it into `scratchpad/` first, then run it from the repository root.
|
||||
|
||||
Preferred pattern:
|
||||
|
||||
```bash
|
||||
cat > scratchpad/probe_image.py <<'PY'
|
||||
from PIL import Image
|
||||
|
||||
img = Image.open("scratchpad/shop_probe_04_checked.png")
|
||||
print(img.size)
|
||||
|
||||
points = [
|
||||
("checkbox_checked_center", (972, 297)),
|
||||
("checkbox_checked_bg", (975, 300)),
|
||||
("card_border_left", (940, 380)),
|
||||
("card_border_unselected", (1165, 380)),
|
||||
("buy_button", (1751, 1112)),
|
||||
("cancel_button", (1525, 1112)),
|
||||
]
|
||||
|
||||
for name, xy in points:
|
||||
print(name, xy, img.getpixel(xy))
|
||||
PY
|
||||
|
||||
python3 scratchpad/probe_image.py
|
||||
```
|
||||
|
||||
Avoid this pattern:
|
||||
|
||||
```bash
|
||||
cd /private/tmp/claude-*/scratchpad && python3 -c "..."
|
||||
```
|
||||
|
||||
Also avoid large multiline `python3 -c "..."` commands when they contain comments or complex quoting. Prefer a temporary script under `scratchpad/` because it is easier to inspect, rerun, and keep within workspace permissions.
|
||||
|
||||
`screenshots/` contains human reference captures. They are useful for calibration and documentation, but they are not necessarily automated test fixtures.
|
||||
|
||||
`assets/` contains local template images. These should be captured from the local game setup where possible.
|
||||
|
||||
## Temporary probe policy
|
||||
|
||||
Temporary probes are allowed, but they must be workspace-local.
|
||||
|
||||
For non-trivial investigation, use:
|
||||
|
||||
```text
|
||||
scratchpad/<short_descriptive_name>.py
|
||||
```
|
||||
|
||||
Examples:
|
||||
|
||||
```text
|
||||
scratchpad/probe_shop_checkbox.py
|
||||
scratchpad/probe_cafe_income.py
|
||||
scratchpad/probe_story_modal.py
|
||||
scratchpad/probe_ocr_region.py
|
||||
```
|
||||
|
||||
A temporary probe should:
|
||||
|
||||
- run from the repository root
|
||||
- read inputs from `scratchpad/`, `screenshots/`, or `assets/`
|
||||
- write outputs to `scratchpad/`
|
||||
- avoid `/tmp`, `/private/tmp`, and Claude Code internal paths
|
||||
- avoid relying on absolute `/private/tmp/claude-*` paths
|
||||
- be deletable once the investigation is complete
|
||||
|
||||
Prefer:
|
||||
|
||||
```bash
|
||||
python3 scratchpad/probe_shop_checkbox.py
|
||||
```
|
||||
|
||||
Do not prefer:
|
||||
|
||||
```bash
|
||||
python3 -c "large multiline script..."
|
||||
```
|
||||
|
||||
For very small one-liners, `python3 -c` is acceptable only when it does not require changing directories into `/private/tmp/claude-*`.
|
||||
|
||||
## Testing and checks
|
||||
|
||||
Because the game only runs on `nik-gpu`, local macOS testing is limited.
|
||||
|
||||
Before deploying, run static/syntax checks locally:
|
||||
|
||||
```
|
||||
```bash
|
||||
bash -n ba_dailies.sh
|
||||
python3 -m py_compile ba_daily.py
|
||||
python3 -m py_compile ba_auto/*.py
|
||||
python3 -m py_compile ba_auto/tasks/*.py
|
||||
```
|
||||
|
||||
For temporary investigation scripts, use:
|
||||
|
||||
```bash
|
||||
python3 scratchpad/<script_name>.py
|
||||
```
|
||||
|
||||
Do not run temporary investigation scripts from Claude Code's internal `/private/tmp/claude-*` workspace.
|
||||
|
||||
On `nik-gpu`, run real integration tests against the live game.
|
||||
|
||||
Example:
|
||||
|
||||
```
|
||||
ssh nik-gpu "~/ba_dailies.sh cafe"
|
||||
```bash
|
||||
ssh nik-gpu "~/repo/ba-auto-daily/ba_dailies.sh cafe"
|
||||
```
|
||||
|
||||
When debugging image matching, write debug images to `scratchpad/`.
|
||||
When debugging image matching, write debug images to:
|
||||
|
||||
```text
|
||||
scratchpad/
|
||||
```
|
||||
|
||||
When debugging on `nik-gpu`, copy relevant screenshots or debug images back into the local project `scratchpad/` if they need local analysis.
|
||||
|
||||
## Existing features
|
||||
|
||||
Current project state (mailbox, cafe, stamina, and story_sweep all migrated to real Python; no Bash feature logic remains):
|
||||
Current project state: mailbox, cafe, stamina, and story_sweep are all migrated to real Python. No Bash feature logic remains.
|
||||
|
||||
- `ba_dailies.sh` is a thin launcher that execs `ba_daily.py`
|
||||
- `ba_daily.py` dispatches `mailbox`/`cafe`/`stamina`/`story_sweep`/default flow to `ba_auto/tasks/`. Default flow (`DEFAULT_ORDER`) is `mailbox`, `cafe`, `stamina` — `story_sweep` is opt-in only since it spends AP rather than reclaiming something free
|
||||
- `ba_auto/tasks/mailbox.py` and `ba_auto/tasks/cafe.py` click with `ba_auto/driver.py` primitives and verify state with `driver.color_at`/`ba_auto/navigation.py` (ported from `module/mail.py` and `module/cafe_reward.py`'s `rgb_in_range`/`co_detect` pattern) before pressing further keys — no legacy bridge remains
|
||||
- `ba_auto/tasks/stamina.py` claims the Mission panel's bulk "一括受取" button (see `plan.md` Phase 8)
|
||||
- `ba_auto/tasks/story_sweep.py` sweeps a config-driven list of exact `(region, stage, count)` targets (`config.STORY_SWEEP_TARGETS`) for AP, navigating to each via OCR (region-number readout + delta-click, stage-label matching) rather than a random pick (see `plan.md` Phase 10, which replaced Phase 9's random-pick design per Phase 9's own OCR-avoidance retrospective — this task is the concrete example behind the "OCR policy" above)
|
||||
- `ba_daily.py` dispatches `mailbox`, `cafe`, `stamina`, `story_sweep`, and default flow to `ba_auto/tasks/`
|
||||
- default flow is `mailbox`, `cafe`, `stamina`
|
||||
- `story_sweep` is opt-in only since it spends AP rather than reclaiming something free
|
||||
- `ba_auto/tasks/mailbox.py` and `ba_auto/tasks/cafe.py` click with `ba_auto/driver.py` primitives and verify state with `driver.color_at` and `ba_auto/navigation.py`
|
||||
- mailbox and cafe were ported from the reference patterns around `module/mail.py` and `module/cafe_reward.py`
|
||||
- no legacy bridge remains
|
||||
- `ba_auto/tasks/stamina.py` claims the Mission panel's bulk `一括受取` button
|
||||
- see `plan.md` Phase 8 for stamina details
|
||||
- `ba_auto/tasks/story_sweep.py` sweeps a config-driven list of exact `(region, stage, count)` targets from `config.STORY_SWEEP_TARGETS`
|
||||
- story sweep navigates via OCR, using region-number readout and stage-label matching, rather than random selection
|
||||
- see `plan.md` Phase 10 for the OCR-based story sweep port
|
||||
- `ba_auto/detector.py` has `find_cafe_sparkle()`, the sparkle template-match ported in-process from the now-deleted `scripts/detect_and_click.py`
|
||||
- `scripts/ba_dailies_legacy.sh` and `scripts/detect_and_click.py` have been deleted — nothing references them anymore, and the `scripts/` directory itself no longer exists
|
||||
- `ba_auto/driver.py` primitives (`run_command`, `focus_game`, `click`, `move_mouse`, `scroll`, `keypress`, `screenshot`, `wait`, `color_at`) are wired into all four task modules
|
||||
- `ba_auto/navigation.py` has two shared state probes used across tasks: `is_on_subscreen` (any mailbox/cafe/shop-style panel vs. the home screen) and `is_modal_open` (a dimmed dialog overlay), plus `wait_for_state()` (a scoped port of the reference's `core/picture.py::co_detect` — watch for any of several named states, react to known non-terminal ones, stop on a recognized terminal one; see `plan.md` Phase 10). `story_sweep.py` additionally has its own local modal probes/close logic because the stage-info modal is wide enough to break `is_modal_open`'s default probe point, and does not close on Escape at all (see `plan.md` Phase 9); its modal also renders at least two different internal layouts (a plain one for the bonus "-A" stage, a taller tabbed one for regular numbered stages) whose button coordinates differ, discovered live in Phase 10
|
||||
- Live testing found both the mailbox-icon and cafe-icon fixed coordinates were flaky (missed the first click, worked on retry) and that neither task verified anything before proceeding, so a missed click cascaded into blind actions and could reach an unverified Escape press on the home screen — which triggers Blue Archive's own "exit the game?" confirmation. See `plan.md` Phases 5–6 for the full writeup. This is the concrete reason every task now verifies state before acting rather than trusting fixed coordinates or a single click blindly
|
||||
- Rank-up popups mid-pat-loop are now handled: a pat that crosses an affection-rank threshold shows a full-screen "絆ランクアップ!" cutscene with no cafe header at all, which `find_cafe_sparkle()` can never recognize (it's nothing like the sparkle template) — the loop used to just spin uselessly against it for the rest of the room's click budget. `cafe.py`'s `_dismiss_rank_up_if_shown()` reuses the existing `navigation.is_on_subscreen` header-brightness probe (confirmed against `screenshots/cafe/student/01-02`: it reads r<200 during the cutscene vs. r>200 on the normal cafe screen) to detect it and press Enter until it clears, porting the reference's own `to_cafe()`-navigation handling of `relationship_rank_up` (`module/cafe_reward.py`). Not yet live-confirmed against a real rank-up trigger (it's semi-random, tied to hitting an affection threshold) — the fix is grounded in real captured screenshots, not a live end-to-end run; treat as implemented-but-unverified until one happens naturally during a real run
|
||||
- Not yet verified for cafe: whether camera zoom/pan can drift over a long unattended run (the reference project zooms out before detecting; ours does not, and testing didn't reproduce a failure from skipping it — see `plan.md` Phase 6 "Not verified" list)
|
||||
- `scripts/ba_dailies_legacy.sh` and `scripts/detect_and_click.py` have been deleted
|
||||
- the `scripts/` directory itself no longer exists
|
||||
- `ba_auto/driver.py` primitives are wired into all migrated task modules
|
||||
- existing driver primitives include `run_command`, `focus_game`, `click`, `move_mouse`, `scroll`, `keypress`, `screenshot`, `wait`, and `color_at`
|
||||
- `ba_auto/navigation.py` has shared state probes used across tasks:
|
||||
- `is_on_subscreen`
|
||||
- `is_modal_open`
|
||||
- `wait_for_state()`
|
||||
|
||||
Migration goal:
|
||||
`wait_for_state()` is a scoped port of the reference's `core/picture.py::co_detect`: watch for any of several named states, react to known non-terminal ones, and stop on a recognized terminal one.
|
||||
|
||||
- preserve existing behavior
|
||||
- move feature logic from Bash into Python
|
||||
- keep `ba_dailies.sh` as launcher only
|
||||
- make detection logic reusable
|
||||
- prepare for future features by copying the reference project's structure where appropriate
|
||||
`story_sweep.py` additionally has its own local modal probes and close logic because the stage-info modal is wide enough to break `is_modal_open`'s default probe point, and it does not close on Escape. See `plan.md` Phase 9.
|
||||
|
||||
The story sweep modal renders at least two different internal layouts:
|
||||
|
||||
- a plain one for the bonus `-A` stage
|
||||
- a taller tabbed one for regular numbered stages
|
||||
|
||||
Their button coordinates differ. This was discovered live in Phase 10.
|
||||
|
||||
Live testing found both the mailbox-icon and cafe-icon fixed coordinates were flaky. They missed the first click and worked on retry. Neither task originally verified state before proceeding, so a missed click cascaded into blind actions and could reach an unverified Escape press on the home screen, which triggers Blue Archive's own "exit the game?" confirmation.
|
||||
|
||||
See `plan.md` Phases 5–6 for the full writeup. This is the concrete reason every task now verifies state before acting rather than trusting fixed coordinates or a single click blindly.
|
||||
|
||||
Rank-up popups mid-pat-loop are now handled.
|
||||
|
||||
A pat that crosses an affection-rank threshold shows a full-screen `絆ランクアップ!` cutscene with no cafe header. `find_cafe_sparkle()` can never recognize this because it is nothing like the sparkle template. The loop used to spin uselessly against it for the rest of the room's click budget.
|
||||
|
||||
`cafe.py`'s `_dismiss_rank_up_if_shown()` reuses the existing `navigation.is_on_subscreen` header-brightness probe. This was confirmed against `screenshots/cafe/student/01-02`:
|
||||
|
||||
- rank-up cutscene: header probe reads `r < 200`
|
||||
- normal cafe screen: header probe reads `r > 200`
|
||||
|
||||
It presses Enter until the cutscene clears, porting the reference's own `to_cafe()` navigation handling of `relationship_rank_up` from `module/cafe_reward.py`.
|
||||
|
||||
This is not yet live-confirmed against a real rank-up trigger. It is grounded in real captured screenshots, but treat it as implemented-but-unverified until one happens naturally during a real run.
|
||||
|
||||
Not yet verified for cafe:
|
||||
|
||||
- whether camera zoom/pan can drift over a long unattended run
|
||||
- the reference project zooms out before detecting
|
||||
- this project currently does not
|
||||
- testing did not reproduce a failure from skipping it
|
||||
- see `plan.md` Phase 6 "Not verified" list
|
||||
|
||||
## Known cafe gaps to verify during migration
|
||||
|
||||
When migrating cafe logic, verify these manually against the actual current behavior:
|
||||
When migrating or modifying cafe logic, verify these manually against the actual current behavior:
|
||||
|
||||
- whether student rotation popups are handled
|
||||
- whether rank-up popups are handled
|
||||
@ -473,13 +713,23 @@ When migrating cafe logic, verify these manually against the actual current beha
|
||||
|
||||
Do not assume the old Bash implementation handles these correctly.
|
||||
|
||||
## Migration goal
|
||||
|
||||
The migration goal is:
|
||||
|
||||
- preserve existing behavior
|
||||
- move feature logic from Bash into Python
|
||||
- keep `ba_dailies.sh` as launcher only
|
||||
- make detection logic reusable
|
||||
- prepare for future features by copying the reference project's structure where appropriate
|
||||
|
||||
## Anti-patterns
|
||||
|
||||
Do not:
|
||||
|
||||
- add new `do_<feature>` Bash functions
|
||||
- build a giant Bash automation script
|
||||
- recreate reference logic as fixed coordinate Bash click chains
|
||||
- recreate reference logic as fixed-coordinate Bash click chains
|
||||
- skip reading the reference module before implementing a feature
|
||||
- put OCR in Bash
|
||||
- put OpenCV state machines in Bash
|
||||
@ -488,8 +738,13 @@ Do not:
|
||||
- copy reference image assets blindly
|
||||
- edit `~/repo/baas-reference/`
|
||||
- implement event-specific features before the generic reusable machinery exists
|
||||
- invent a pixel-probe, fixed-coordinate, or randomized substitute for reference logic that uses OCR, just to avoid setting up OCR
|
||||
- treat OCR as an Android-specific concern that can be deferred indefinitely — it isn't, and deferring it has already produced more bugs than it saved (see "OCR policy" above)
|
||||
- invent a pixel probe, fixed-coordinate, or randomized substitute for reference logic that uses OCR, just to avoid setting up OCR
|
||||
- treat OCR as an Android-specific concern that can be deferred indefinitely
|
||||
- run temporary probes from `/private/tmp/claude-*` instead of the repository `scratchpad/`
|
||||
- create investigation scripts outside the repository when they can live in `scratchpad/`
|
||||
- use multiline `python3 -c "..."` commands for non-trivial probes
|
||||
- rely on Claude Code internal temporary paths in committed code, notes, commands, or debugging workflow
|
||||
- write generated debug files into the repository root when they belong in `scratchpad/`
|
||||
|
||||
## Feature implementation workflow
|
||||
|
||||
@ -497,7 +752,7 @@ For every new feature:
|
||||
|
||||
1. Read the relevant `~/repo/baas-reference/module/...` file.
|
||||
2. Summarize the upstream feature flow in notes or comments.
|
||||
3. Add/update the reference mapping table.
|
||||
3. Add or update the reference mapping table.
|
||||
4. Identify missing local driver primitives.
|
||||
5. Implement or improve those primitives in `ba_auto/driver.py` or `ba_auto/detector.py`.
|
||||
6. Implement the feature in `ba_auto/tasks/<feature>.py`.
|
||||
@ -507,17 +762,20 @@ For every new feature:
|
||||
10. Deploy to `nik-gpu`.
|
||||
11. Test against the live game.
|
||||
12. Update `plan.md` status.
|
||||
13. Move temporary investigation files into `scratchpad/`, or delete them if they are no longer useful.
|
||||
|
||||
## Priority when uncertain
|
||||
|
||||
When uncertain, prefer this order:
|
||||
|
||||
1. Preserve existing working behavior.
|
||||
2. Follow the reference project's control flow — including its use of OCR. Do not deprioritize OCR-based navigation/matching in favor of a simpler non-OCR substitute; that has already produced more bugs than the OCR setup would have cost (see "OCR policy" above).
|
||||
3. Keep logic in Python.
|
||||
4. Add reusable driver primitives instead of feature-specific hacks.
|
||||
5. Use local screenshots/assets only when backend differences require it.
|
||||
6. Avoid large rewrites that do not move the project closer to reference-driven Python architecture.
|
||||
2. Follow the reference project's control flow, including its use of OCR.
|
||||
3. Do not deprioritize OCR-based navigation/matching in favor of a simpler non-OCR substitute.
|
||||
4. Keep logic in Python.
|
||||
5. Add reusable driver primitives instead of feature-specific hacks.
|
||||
6. Use local screenshots/assets only when backend differences require it.
|
||||
7. Use `scratchpad/` for temporary work and diagnostics.
|
||||
8. Avoid large rewrites that do not move the project closer to reference-driven Python architecture.
|
||||
|
||||
## User preference
|
||||
|
||||
@ -525,4 +783,6 @@ The user wants this project to be as close to the original Blue Archive Auto Scr
|
||||
|
||||
The user specifically does not want Claude Code to keep converting feature work into Bash.
|
||||
|
||||
Respect that preference.
|
||||
The user also wants Claude Code to avoid using `/private/tmp/claude-*` for development probes when a repository-local `scratchpad/` directory can be used instead, because those paths trigger permission prompts and interrupt the development flow.
|
||||
|
||||
Respect those preferences.
|
||||
Loading…
x
Reference in New Issue
Block a user