Compare commits
10 Commits
398b936c64
...
9f4d8f4361
| Author | SHA1 | Date | |
|---|---|---|---|
| 9f4d8f4361 | |||
| 51ad297f69 | |||
| 1bb2ccdd6f | |||
| 4842d575a5 | |||
| b2ed571908 | |||
| a8ff95d6f5 | |||
| 0707487934 | |||
| 0b16399948 | |||
| a04ef5d0c5 | |||
| 3a4534f205 |
1
.gitignore
vendored
1
.gitignore
vendored
@ -3,3 +3,4 @@ graphify-out/cost.json
|
|||||||
graphify-out/cache/
|
graphify-out/cache/
|
||||||
__pycache__/
|
__pycache__/
|
||||||
*.pyc
|
*.pyc
|
||||||
|
scratchpad/
|
||||||
|
|||||||
425
CLAUDE.md
425
CLAUDE.md
@ -8,19 +8,55 @@ This repository is a personal Blue Archive JP daily-automation project.
|
|||||||
|
|
||||||
It controls the real Blue Archive PC/Steam/Proton client running on a Linux machine named `nik-gpu`. The development machine is a MacBook named `nik-macbookair`.
|
It controls the real Blue Archive PC/Steam/Proton client running on a Linux machine named `nik-gpu`. The development machine is a MacBook named `nik-macbookair`.
|
||||||
|
|
||||||
|
**The original goal is a backend port, not a rewrite.** The reference project (`~/repo/baas-reference/`) only works against an Android emulator via ADB/uiautomator2, typically run on a Windows PC. This project's job is to make that same automation logic work against the real JP client running under Proton on Linux instead — porting the reference's decision logic, navigation, and state-detection approach, including its use of OCR, as faithfully as practical, and replacing only the parts that are genuinely Android-specific with local desktop equivalents.
|
||||||
|
|
||||||
|
Android-specific parts include:
|
||||||
|
|
||||||
|
- ADB input
|
||||||
|
- uiautomator2 taps
|
||||||
|
- Android UI-object-tree queries
|
||||||
|
- emulator-specific assumptions
|
||||||
|
|
||||||
|
Local desktop equivalents include:
|
||||||
|
|
||||||
|
- `xdotool`
|
||||||
|
- `scrot`
|
||||||
|
- OpenCV
|
||||||
|
- OCR against screenshots
|
||||||
|
- local image/color/template detection
|
||||||
|
|
||||||
|
Inventing a different, simpler approach to avoid porting a piece of reference logic is a deviation from the project goal, not a shortcut toward it.
|
||||||
|
|
||||||
The automation backend is local desktop control:
|
The automation backend is local desktop control:
|
||||||
|
|
||||||
- `xdotool` for mouse/keyboard/window control
|
- `xdotool` for mouse/keyboard/window control
|
||||||
- `scrot` for screenshots
|
- `scrot` for screenshots
|
||||||
- Python/OpenCV for image matching and color/template detection
|
- Python/OpenCV for image matching and color/template detection
|
||||||
- OCR later, when needed
|
- OCR, using Tesseract or PaddleOCR, wherever the reference implementation uses OCR for a feature
|
||||||
- no Android emulator control
|
- no Android emulator control
|
||||||
- no ADB
|
- no ADB
|
||||||
- no uiautomator2
|
- no uiautomator2
|
||||||
|
|
||||||
|
### OCR policy
|
||||||
|
|
||||||
|
OCR is **not** an Android-specific concern. The reference's OCR-driven region, stage-name, and currency matching runs against a screenshot and can be ported to this backend using `scrot` captures. Only the reference's input mechanism is Android-specific.
|
||||||
|
|
||||||
|
Two features, Stamina/AP mission claim and Normal/Hard story AP sweep, were originally built without OCR by substituting ad hoc pixel probes, fixed coordinates, or randomized selection for the reference's OCR-driven navigation. For story sweep, that substitution produced more real bugs during live testing than porting the reference's actual approach would have:
|
||||||
|
|
||||||
|
- a wrong modal-open probe
|
||||||
|
- an unverified button click that silently under-spent AP
|
||||||
|
- a modal that does not close on Escape
|
||||||
|
- a latent hazard where a mistimed keypress could have started a real battle
|
||||||
|
|
||||||
|
See `plan.md` Phase 9's retrospective for the full writeup.
|
||||||
|
|
||||||
|
That issue has since been fixed: `story_sweep.py` now ports the reference's actual OCR-based region/stage targeting. See `plan.md` Phase 10. The retrospective stays here as the concrete, lived reason for the OCR policy, not as a description of the current state of that task.
|
||||||
|
|
||||||
|
Going forward: when a reference feature's control flow depends on OCR, set up OCR and port that logic. Do not invent a non-OCR workaround just to avoid the setup cost. Only skip OCR for a specific step if the reference itself does not use OCR there.
|
||||||
|
|
||||||
The reference project is located at:
|
The reference project is located at:
|
||||||
|
|
||||||
```
|
```text
|
||||||
~/repo/baas-reference/
|
~/repo/baas-reference/
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -38,7 +74,7 @@ All feature logic must live in Python.
|
|||||||
|
|
||||||
The intended structure is:
|
The intended structure is:
|
||||||
|
|
||||||
```
|
```text
|
||||||
ba-auto-daily/
|
ba-auto-daily/
|
||||||
├── ba_dailies.sh
|
├── ba_dailies.sh
|
||||||
├── ba_daily.py
|
├── ba_daily.py
|
||||||
@ -59,12 +95,14 @@ ba-auto-daily/
|
|||||||
│ └── mapping.md
|
│ └── mapping.md
|
||||||
├── assets/
|
├── assets/
|
||||||
├── screenshots/
|
├── screenshots/
|
||||||
├── scripts/
|
├── scratchpad/
|
||||||
├── setup.sh
|
├── setup.sh
|
||||||
├── plan.md
|
├── plan.md
|
||||||
└── CLAUDE.md
|
└── CLAUDE.md
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`scripts/` previously held one-off Bash/Python helpers. It was deleted once mailbox and cafe migrated off it. Do not recreate `scripts/` as a place to stash feature logic.
|
||||||
|
|
||||||
The exact layout can evolve, but the architectural rule should not change:
|
The exact layout can evolve, but the architectural rule should not change:
|
||||||
|
|
||||||
- Bash launches Python.
|
- Bash launches Python.
|
||||||
@ -76,7 +114,7 @@ The exact layout can evolve, but the architectural rule should not change:
|
|||||||
|
|
||||||
Before implementing any new feature, inspect the matching reference implementation in:
|
Before implementing any new feature, inspect the matching reference implementation in:
|
||||||
|
|
||||||
```
|
```text
|
||||||
~/repo/baas-reference/module/
|
~/repo/baas-reference/module/
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -94,7 +132,7 @@ For every feature, first identify:
|
|||||||
|
|
||||||
Then implement the feature in Python under:
|
Then implement the feature in Python under:
|
||||||
|
|
||||||
```
|
```text
|
||||||
ba_auto/tasks/
|
ba_auto/tasks/
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -117,16 +155,30 @@ The reference repository is **read-only**.
|
|||||||
- edit files in `~/repo/baas-reference/`
|
- edit files in `~/repo/baas-reference/`
|
||||||
- reimplement a reference feature from scratch in Bash
|
- reimplement a reference feature from scratch in Bash
|
||||||
- create a local solution that ignores the reference flow when a reference implementation already exists
|
- create a local solution that ignores the reference flow when a reference implementation already exists
|
||||||
|
- substitute a pixel probe, fixed coordinate, or randomized shortcut for reference logic that uses OCR, just to avoid setting up OCR
|
||||||
|
|
||||||
## Two-machine architecture
|
## Two-machine architecture
|
||||||
|
|
||||||
Development happens on: `nik-macbookair`
|
Development happens on:
|
||||||
|
|
||||||
Runtime happens on: `nik-gpu`
|
```text
|
||||||
|
nik-macbookair
|
||||||
|
```
|
||||||
|
|
||||||
|
Runtime happens on:
|
||||||
|
|
||||||
|
```text
|
||||||
|
nik-gpu
|
||||||
|
```
|
||||||
|
|
||||||
The Blue Archive client and X display live on `nik-gpu`.
|
The Blue Archive client and X display live on `nik-gpu`.
|
||||||
|
|
||||||
Assume: (`DISPLAY=:0`, GDM `XAUTHORITY` under `/run/user/1000`)
|
Assume:
|
||||||
|
|
||||||
|
```text
|
||||||
|
DISPLAY=:0
|
||||||
|
GDM XAUTHORITY under /run/user/1000
|
||||||
|
```
|
||||||
|
|
||||||
The game runs under Steam/Proton on the Linux desktop.
|
The game runs under Steam/Proton on the Linux desktop.
|
||||||
|
|
||||||
@ -138,7 +190,7 @@ During iteration, files are pushed from `nik-macbookair` to `nik-gpu`.
|
|||||||
|
|
||||||
Typical paths on `nik-gpu`:
|
Typical paths on `nik-gpu`:
|
||||||
|
|
||||||
```
|
```text
|
||||||
~/ba_dailies.sh
|
~/ba_dailies.sh
|
||||||
~/ba_daily.py
|
~/ba_daily.py
|
||||||
~/ba_auto/
|
~/ba_auto/
|
||||||
@ -148,7 +200,7 @@ Typical paths on `nik-gpu`:
|
|||||||
|
|
||||||
The current setup may still contain older paths such as:
|
The current setup may still contain older paths such as:
|
||||||
|
|
||||||
```
|
```text
|
||||||
~/ba_scripts/detect_and_click.py
|
~/ba_scripts/detect_and_click.py
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -158,12 +210,28 @@ When refactoring, prefer consolidating Python code into `ba_auto/`.
|
|||||||
|
|
||||||
For a fresh checkout on `nik-gpu`, run from the repo root:
|
For a fresh checkout on `nik-gpu`, run from the repo root:
|
||||||
|
|
||||||
```
|
```bash
|
||||||
./setup.sh
|
./setup.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
After initial setup, individual changes may be pushed with `scp` or `rsync`.
|
After initial setup, individual changes may be pushed with `scp` or `rsync`.
|
||||||
|
|
||||||
|
Prefer syncing the project from the repository root:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
rsync -av \
|
||||||
|
--exclude='.git/' \
|
||||||
|
--exclude='__pycache__/' \
|
||||||
|
--exclude='*.pyc' \
|
||||||
|
--exclude='.claude/settings.local.json' \
|
||||||
|
--exclude='graphify-out/' \
|
||||||
|
--exclude='screenshots/' \
|
||||||
|
--exclude='scratchpad/' \
|
||||||
|
./ nik-gpu:~/repo/ba-auto-daily/
|
||||||
|
```
|
||||||
|
|
||||||
|
Do not sync Claude Code internal temporary folders.
|
||||||
|
|
||||||
## Runtime dependencies on nik-gpu
|
## Runtime dependencies on nik-gpu
|
||||||
|
|
||||||
These are host-level dependencies. Confirm they exist before assuming a bug is in the project code.
|
These are host-level dependencies. Confirm they exist before assuming a bug is in the project code.
|
||||||
@ -178,22 +246,28 @@ Required Python packages:
|
|||||||
|
|
||||||
- `opencv-python` or `opencv-python-headless`
|
- `opencv-python` or `opencv-python-headless`
|
||||||
- `numpy`
|
- `numpy`
|
||||||
|
- `pytesseract`
|
||||||
|
|
||||||
Expected venv:
|
Expected venv:
|
||||||
|
|
||||||
```
|
```text
|
||||||
~/.venvs/ba-auto-daily/bin/python3
|
~/.venvs/ba-auto-daily/bin/python3
|
||||||
```
|
```
|
||||||
|
|
||||||
Quick check:
|
Quick check:
|
||||||
|
|
||||||
```
|
```bash
|
||||||
ssh nik-gpu "which xdotool scrot && ~/.venvs/ba-auto-daily/bin/python3 -c 'import cv2, numpy; print(cv2.__version__)'"
|
ssh nik-gpu "which xdotool scrot tesseract && ~/.venvs/ba-auto-daily/bin/python3 -c 'import cv2, numpy, pytesseract; print(cv2.__version__)'"
|
||||||
```
|
```
|
||||||
|
|
||||||
Future dependency: OCR engine, likely Tesseract or PaddleOCR.
|
OCR engine dependency is set up as of Phase 10:
|
||||||
|
|
||||||
Do not introduce OCR casually. Add it only when implementing a feature that actually needs OCR.
|
- Tesseract via the `tesseract-ocr` apt package
|
||||||
|
- `pytesseract` in the venv
|
||||||
|
|
||||||
|
Installing `tesseract-ocr` needs interactive `sudo`, so `setup.sh` should check for it but should not assume it can install it automatically.
|
||||||
|
|
||||||
|
`ba_auto/detector.py`'s `read_text()` and `read_int()` wrap OCR for occasional single-crop reads. See `story_sweep.py` for the first real usage.
|
||||||
|
|
||||||
## Bash policy
|
## Bash policy
|
||||||
|
|
||||||
@ -204,8 +278,10 @@ Preferred shape:
|
|||||||
```bash
|
```bash
|
||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
VENV_PYTHON="${VENV_PYTHON:-$HOME/.venvs/ba-auto-daily/bin/python3}"
|
VENV_PYTHON="${VENV_PYTHON:-$HOME/.venvs/ba-auto-daily/bin/python3}"
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
|
||||||
exec "$VENV_PYTHON" "$SCRIPT_DIR/ba_daily.py" "$@"
|
exec "$VENV_PYTHON" "$SCRIPT_DIR/ba_daily.py" "$@"
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -234,36 +310,44 @@ If an existing Bash function exists, migrate it to Python rather than extending
|
|||||||
|
|
||||||
The intended Python CLI entry point is:
|
The intended Python CLI entry point is:
|
||||||
|
|
||||||
```
|
```text
|
||||||
ba_daily.py
|
ba_daily.py
|
||||||
```
|
```
|
||||||
|
|
||||||
It should support commands such as:
|
It should support commands such as:
|
||||||
|
|
||||||
```
|
```bash
|
||||||
./ba_dailies.sh
|
./ba_dailies.sh
|
||||||
./ba_dailies.sh mailbox
|
./ba_dailies.sh mailbox
|
||||||
./ba_dailies.sh cafe
|
./ba_dailies.sh cafe
|
||||||
./ba_dailies.sh stamina
|
./ba_dailies.sh stamina
|
||||||
|
./ba_dailies.sh story_sweep
|
||||||
./ba_dailies.sh group
|
./ba_dailies.sh group
|
||||||
```
|
```
|
||||||
|
|
||||||
No argument should run the default daily flow.
|
No argument should run the default daily flow.
|
||||||
|
|
||||||
Example default flow:
|
Current default flow in `ba_daily.py`'s `DEFAULT_ORDER`:
|
||||||
|
|
||||||
1. focus game
|
1. focus game
|
||||||
2. mailbox
|
2. mailbox
|
||||||
3. cafe
|
3. cafe
|
||||||
4. future daily tasks
|
4. stamina
|
||||||
|
5. future daily tasks
|
||||||
|
|
||||||
The CLI should dispatch into task modules under `ba_auto/tasks/`.
|
`story_sweep` spends AP rather than reclaiming something free, so it is deliberately excluded from the default flow. It must be invoked explicitly.
|
||||||
|
|
||||||
|
The CLI should dispatch into task modules under:
|
||||||
|
|
||||||
|
```text
|
||||||
|
ba_auto/tasks/
|
||||||
|
```
|
||||||
|
|
||||||
## Driver layer
|
## Driver layer
|
||||||
|
|
||||||
Create and maintain a driver layer in:
|
Create and maintain a driver layer in:
|
||||||
|
|
||||||
```
|
```text
|
||||||
ba_auto/driver.py
|
ba_auto/driver.py
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -271,14 +355,15 @@ The driver layer should wrap the PC/Steam/Proton backend.
|
|||||||
|
|
||||||
It should provide reusable primitives such as:
|
It should provide reusable primitives such as:
|
||||||
|
|
||||||
```
|
```python
|
||||||
focus_game()
|
focus_game()
|
||||||
screenshot()
|
screenshot()
|
||||||
click(x, y)
|
click(x, y)
|
||||||
double_click(x, y)
|
double_click(x, y)
|
||||||
drag/swipe(start, end, duration)
|
drag(start, end, duration)
|
||||||
|
swipe(start, end, duration)
|
||||||
keypress(key)
|
keypress(key)
|
||||||
sleep/wait
|
wait(seconds)
|
||||||
wait_until(...)
|
wait_until(...)
|
||||||
color_at(...)
|
color_at(...)
|
||||||
region_average_color(...)
|
region_average_color(...)
|
||||||
@ -286,7 +371,7 @@ template_match(...)
|
|||||||
find_and_click_template(...)
|
find_and_click_template(...)
|
||||||
```
|
```
|
||||||
|
|
||||||
Feature modules should not directly shell out to `xdotool` or `scrot` unless a driver primitive is missing and being added.
|
Feature modules should not directly shell out to `xdotool` or `scrot` unless a driver primitive is missing and is being added.
|
||||||
|
|
||||||
Prefer:
|
Prefer:
|
||||||
|
|
||||||
@ -297,16 +382,22 @@ driver.click(x, y)
|
|||||||
over:
|
over:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
subprocess.run(["xdotool", "click", ...])
|
subprocess.run(["xdotool", "click", "1"])
|
||||||
```
|
```
|
||||||
|
|
||||||
This keeps the project close to the reference architecture while replacing only the backend-control layer.
|
This keeps the project close to the reference architecture while replacing only the backend-control layer.
|
||||||
|
|
||||||
## Detector layer
|
## Detector layer
|
||||||
|
|
||||||
Image matching and color matching should live in `ba_auto/detector.py` or in clearly named helper classes/functions.
|
Image matching, color matching, and OCR wrappers should live in:
|
||||||
|
|
||||||
Existing logic from `scripts/detect_and_click.py` should be migrated into reusable Python functions.
|
```text
|
||||||
|
ba_auto/detector.py
|
||||||
|
```
|
||||||
|
|
||||||
|
or in clearly named helper classes/functions.
|
||||||
|
|
||||||
|
The cafe sparkle-matching logic formerly in `scripts/detect_and_click.py` has been migrated into `ba_auto/detector.py` as `find_cafe_sparkle()`. It should be called in-process rather than as a per-click subprocess.
|
||||||
|
|
||||||
The detector should support:
|
The detector should support:
|
||||||
|
|
||||||
@ -315,13 +406,18 @@ The detector should support:
|
|||||||
- threshold tuning
|
- threshold tuning
|
||||||
- masked matching
|
- masked matching
|
||||||
- click-offset handling
|
- click-offset handling
|
||||||
- debug image output to `./scratchpad`
|
- OCR crop reads
|
||||||
|
- debug image output to `scratchpad/`
|
||||||
|
|
||||||
Avoid one Python cold start per click attempt where possible. Prefer long-running Python task logic that can take repeated screenshots and click repeatedly from one process.
|
Avoid one Python cold start per click attempt where possible. Prefer long-running Python task logic that can take repeated screenshots and click repeatedly from one process.
|
||||||
|
|
||||||
## Navigation layer
|
## Navigation layer
|
||||||
|
|
||||||
Common navigation should live in `ba_auto/navigation.py`.
|
Common navigation should live in:
|
||||||
|
|
||||||
|
```text
|
||||||
|
ba_auto/navigation.py
|
||||||
|
```
|
||||||
|
|
||||||
Use this for shared flows such as:
|
Use this for shared flows such as:
|
||||||
|
|
||||||
@ -333,31 +429,37 @@ Use this for shared flows such as:
|
|||||||
- opening lesson/schedule
|
- opening lesson/schedule
|
||||||
- closing popups
|
- closing popups
|
||||||
- generic back/escape handling
|
- generic back/escape handling
|
||||||
|
- waiting for known UI states
|
||||||
|
|
||||||
Do not duplicate navigation click sequences inside every task if they can be shared.
|
Do not duplicate navigation click sequences inside every task if they can be shared.
|
||||||
|
|
||||||
## Task modules
|
## Task modules
|
||||||
|
|
||||||
Each feature should have a task module: `ba_auto/tasks/<feature>.py`
|
Each feature should have a task module:
|
||||||
|
|
||||||
Example:
|
|
||||||
|
|
||||||
|
```text
|
||||||
|
ba_auto/tasks/<feature>.py
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
```text
|
||||||
ba_auto/tasks/mailbox.py
|
ba_auto/tasks/mailbox.py
|
||||||
ba_auto/tasks/cafe.py
|
ba_auto/tasks/cafe.py
|
||||||
ba_auto/tasks/group.py
|
|
||||||
ba_auto/tasks/stamina.py
|
ba_auto/tasks/stamina.py
|
||||||
|
ba_auto/tasks/story_sweep.py
|
||||||
|
ba_auto/tasks/group.py
|
||||||
```
|
```
|
||||||
|
|
||||||
Each task module should expose a clear function such as:
|
Each task module should expose a clear function such as:
|
||||||
|
|
||||||
```
|
```python
|
||||||
run(driver, config)
|
run(driver, config)
|
||||||
```
|
```
|
||||||
|
|
||||||
or:
|
or:
|
||||||
|
|
||||||
```
|
```python
|
||||||
run_mailbox(driver, config)
|
run_mailbox(driver, config)
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -367,7 +469,7 @@ Keep task files feature-focused.
|
|||||||
|
|
||||||
Maintain a mapping file at:
|
Maintain a mapping file at:
|
||||||
|
|
||||||
```
|
```text
|
||||||
ba_auto/reference_notes/mapping.md
|
ba_auto/reference_notes/mapping.md
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -375,74 +477,239 @@ Before or during implementation of a feature, update the mapping.
|
|||||||
|
|
||||||
Use this format:
|
Use this format:
|
||||||
|
|
||||||
|
```markdown
|
||||||
| Local feature | Reference file | Reference functions/classes | Local file | Backend replacements | Status |
|
| Local feature | Reference file | Reference functions/classes | Local file | Backend replacements | Status |
|
||||||
|---|---|---|---|---|---|
|
|---|---|---|---|---|---|
|
||||||
| Cafe | `module/cafe.py` or relevant file | `...` | `ba_auto/tasks/cafe.py` | uiautomator2 tap -> xdotool click, screenshot -> scrot/OpenCV | In progress |
|
| Cafe | `module/cafe.py` or relevant file | `...` | `ba_auto/tasks/cafe.py` | uiautomator2 tap -> xdotool click, screenshot -> scrot/OpenCV | In progress |
|
||||||
|
```
|
||||||
|
|
||||||
## Working conventions
|
## Working conventions
|
||||||
|
|
||||||
Use `./scratchpad` for temporary or intermediate files.
|
Always work from the repository root:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/repo/ba-auto-daily
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `scratchpad/` in the project root for all temporary, generated, diagnostic, calibration, and investigation files.
|
||||||
|
|
||||||
Examples:
|
Examples:
|
||||||
|
|
||||||
- cropped calibration images
|
- cropped calibration images
|
||||||
- debug screenshots
|
- debug screenshots
|
||||||
- annotated match results
|
- annotated match results
|
||||||
|
- temporary Python probes
|
||||||
- temporary investigation notes
|
- temporary investigation notes
|
||||||
|
- one-off image-analysis scripts
|
||||||
|
- temporary OCR experiments
|
||||||
|
- screenshots copied back from `nik-gpu` for analysis
|
||||||
|
|
||||||
Do not use `/tmp` or `/private/tmp` unless there is a strong reason.
|
Create it if missing:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mkdir -p scratchpad
|
||||||
|
```
|
||||||
|
|
||||||
|
Do **not** use Claude Code's internal temporary directories such as:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/private/tmp/claude-*
|
||||||
|
/tmp/claude-*
|
||||||
|
```
|
||||||
|
|
||||||
|
Do **not** run probes from `/private/tmp/claude-*` or write temporary scripts there. These paths are outside the repository and often trigger extra permission prompts, interrupting the development flow.
|
||||||
|
|
||||||
|
When a temporary Python probe is needed, write it into `scratchpad/` first, then run it from the repository root.
|
||||||
|
|
||||||
|
Preferred pattern:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cat > scratchpad/probe_image.py <<'PY'
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
img = Image.open("scratchpad/shop_probe_04_checked.png")
|
||||||
|
print(img.size)
|
||||||
|
|
||||||
|
points = [
|
||||||
|
("checkbox_checked_center", (972, 297)),
|
||||||
|
("checkbox_checked_bg", (975, 300)),
|
||||||
|
("card_border_left", (940, 380)),
|
||||||
|
("card_border_unselected", (1165, 380)),
|
||||||
|
("buy_button", (1751, 1112)),
|
||||||
|
("cancel_button", (1525, 1112)),
|
||||||
|
]
|
||||||
|
|
||||||
|
for name, xy in points:
|
||||||
|
print(name, xy, img.getpixel(xy))
|
||||||
|
PY
|
||||||
|
|
||||||
|
python3 scratchpad/probe_image.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Avoid this pattern:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /private/tmp/claude-*/scratchpad && python3 -c "..."
|
||||||
|
```
|
||||||
|
|
||||||
|
Also avoid large multiline `python3 -c "..."` commands when they contain comments or complex quoting. Prefer a temporary script under `scratchpad/` because it is easier to inspect, rerun, and keep within workspace permissions.
|
||||||
|
|
||||||
`screenshots/` contains human reference captures. They are useful for calibration and documentation, but they are not necessarily automated test fixtures.
|
`screenshots/` contains human reference captures. They are useful for calibration and documentation, but they are not necessarily automated test fixtures.
|
||||||
|
|
||||||
`assets/` contains local template images. These should be captured from the local game setup where possible.
|
`assets/` contains local template images. These should be captured from the local game setup where possible.
|
||||||
|
|
||||||
|
## Temporary probe policy
|
||||||
|
|
||||||
|
Temporary probes are allowed, but they must be workspace-local.
|
||||||
|
|
||||||
|
For non-trivial investigation, use:
|
||||||
|
|
||||||
|
```text
|
||||||
|
scratchpad/<short_descriptive_name>.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
```text
|
||||||
|
scratchpad/probe_shop_checkbox.py
|
||||||
|
scratchpad/probe_cafe_income.py
|
||||||
|
scratchpad/probe_story_modal.py
|
||||||
|
scratchpad/probe_ocr_region.py
|
||||||
|
```
|
||||||
|
|
||||||
|
A temporary probe should:
|
||||||
|
|
||||||
|
- run from the repository root
|
||||||
|
- read inputs from `scratchpad/`, `screenshots/`, or `assets/`
|
||||||
|
- write outputs to `scratchpad/`
|
||||||
|
- avoid `/tmp`, `/private/tmp`, and Claude Code internal paths
|
||||||
|
- avoid relying on absolute `/private/tmp/claude-*` paths
|
||||||
|
- be deletable once the investigation is complete
|
||||||
|
|
||||||
|
Prefer:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scratchpad/probe_shop_checkbox.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Do not prefer:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 -c "large multiline script..."
|
||||||
|
```
|
||||||
|
|
||||||
|
For very small one-liners, `python3 -c` is acceptable only when it does not require changing directories into `/private/tmp/claude-*`.
|
||||||
|
|
||||||
## Testing and checks
|
## Testing and checks
|
||||||
|
|
||||||
Because the game only runs on `nik-gpu`, local macOS testing is limited.
|
Because the game only runs on `nik-gpu`, local macOS testing is limited.
|
||||||
|
|
||||||
Before deploying, run static/syntax checks locally:
|
Before deploying, run static/syntax checks locally:
|
||||||
|
|
||||||
```
|
```bash
|
||||||
bash -n ba_dailies.sh
|
bash -n ba_dailies.sh
|
||||||
python3 -m py_compile ba_daily.py
|
python3 -m py_compile ba_daily.py
|
||||||
python3 -m py_compile ba_auto/*.py
|
python3 -m py_compile ba_auto/*.py
|
||||||
python3 -m py_compile ba_auto/tasks/*.py
|
python3 -m py_compile ba_auto/tasks/*.py
|
||||||
```
|
```
|
||||||
|
|
||||||
|
For temporary investigation scripts, use:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scratchpad/<script_name>.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Do not run temporary investigation scripts from Claude Code's internal `/private/tmp/claude-*` workspace.
|
||||||
|
|
||||||
On `nik-gpu`, run real integration tests against the live game.
|
On `nik-gpu`, run real integration tests against the live game.
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
|
|
||||||
```
|
```bash
|
||||||
ssh nik-gpu "~/ba_dailies.sh cafe"
|
ssh nik-gpu "~/repo/ba-auto-daily/ba_dailies.sh cafe"
|
||||||
```
|
```
|
||||||
|
|
||||||
When debugging image matching, write debug images to `./scratchpad`.
|
When debugging image matching, write debug images to:
|
||||||
|
|
||||||
|
```text
|
||||||
|
scratchpad/
|
||||||
|
```
|
||||||
|
|
||||||
|
When debugging on `nik-gpu`, copy relevant screenshots or debug images back into the local project `scratchpad/` if they need local analysis.
|
||||||
|
|
||||||
## Existing features
|
## Existing features
|
||||||
|
|
||||||
Current project state (Phase 1 skeleton in place, mailbox migrated):
|
Current project state: mailbox, cafe, stamina, story_sweep, shop_common, shop_tactical, and lesson are all migrated to real Python. No Bash feature logic remains.
|
||||||
|
|
||||||
- `ba_dailies.sh` is a thin launcher that execs `ba_daily.py`
|
- `ba_dailies.sh` is a thin launcher that execs `ba_daily.py`
|
||||||
- `ba_daily.py` dispatches `mailbox`/`cafe`/default flow to `ba_auto/tasks/`
|
- `ba_daily.py` dispatches `mailbox`, `cafe`, `stamina`, `story_sweep`, `shop_common`, `shop_tactical`, `lesson`, and default flow to `ba_auto/tasks/`
|
||||||
- `ba_auto/tasks/mailbox.py` is real Python: it clicks with `ba_auto/driver.py` primitives and verifies state with `driver.color_at` (ported from `module/mail.py`'s `rgb_in_range` pattern) before pressing further keys — no legacy bridge
|
- default flow is `mailbox`, `cafe`, `stamina`
|
||||||
- `ba_auto/tasks/cafe.py` still bridges to `scripts/ba_dailies_legacy.sh` (the pre-migration click sequence), marked with a TODO — the real logic has not moved into Python yet
|
- `story_sweep`, `shop_common`, `shop_tactical`, and `lesson` are opt-in only since they spend AP/credits/tactical coin/lesson tickets rather than reclaiming something free
|
||||||
- cafe's bridge still shells out to `scripts/detect_and_click.py` for OpenCV sparkle detection, unchanged
|
- `ba_auto/tasks/mailbox.py` and `ba_auto/tasks/cafe.py` click with `ba_auto/driver.py` primitives and verify state with `driver.color_at` and `ba_auto/navigation.py`
|
||||||
- `ba_auto/driver.py` has the primitives (`run_command`, `focus_game`, `click`, `keypress`, `screenshot`, `wait`, `color_at`) wired into `mailbox.py`; not yet used by `cafe.py`
|
- mailbox and cafe were ported from the reference patterns around `module/mail.py` and `module/cafe_reward.py`
|
||||||
- `ba_auto/detector.py` and `ba_auto/navigation.py` are placeholders
|
- no legacy bridge remains
|
||||||
- Live testing found the old fixed mailbox-icon coordinate was marginal and could miss, cascading into an unverified Escape press that triggers Blue Archive's own "exit the game?" confirmation — see `plan.md` Phase 5 for details. This is the concrete reason `mailbox.py` verifies state before acting rather than trusting fixed coordinates blindly.
|
- `ba_auto/tasks/stamina.py` claims the Mission panel's bulk `一括受取` button
|
||||||
|
- see `plan.md` Phase 8 for stamina details
|
||||||
|
- `ba_auto/tasks/story_sweep.py` sweeps a config-driven list of exact `(region, stage, count)` targets from `config.STORY_SWEEP_TARGETS`, plus one daily-rotating target (`config.STORY_SWEEP_ROTATION_*`) that cycles through a fixed region's stages one per day
|
||||||
|
- story sweep navigates via OCR, using region-number readout and stage-label matching, rather than random selection
|
||||||
|
- see `plan.md` Phase 10 for the OCR-based story sweep port, and its "Phase 10 follow-up" for the rotation-target fix
|
||||||
|
- `ba_auto/tasks/shop_common.py` and `ba_auto/tasks/shop_tactical.py` share a checkbox-grid-then-bulk-buy flow (`ba_auto/tasks/shop_utils.py`) against config-driven `(row, col, name, expected_price)` targets (`config.COMMON_SHOP_TARGETS` / `config.TACTICAL_SHOP_TARGETS`)
|
||||||
|
- item identification is by fixed grid position, not per-item OCR — the reference's own `get_item_position` indexes an external static price table this repo doesn't have, so a locally pixel-scanned position table is the faithful port, not an OCR-avoidance shortcut; price-digit OCR is layered on top as an extra catalog-drift safety check the reference doesn't even do per-item
|
||||||
|
- both shops were live-tested with real purchases (see `plan.md` Phase 11), which also surfaced a real, previously-unknown per-refresh-cycle purchase cap on these items (not shown as a visible counter) — the task correctly detected the now-unselectable items and safely declined rather than misfiring
|
||||||
|
- `ba_auto/tasks/lesson.py` sweeps every unlocked region's schedule grid (a scrollable list of 12 named regions, each opening a grid modal of up to 9 location cards), picking the highest-affection available lesson each time until lesson tickets or lessons run out
|
||||||
|
- per-cell affection is read via `detector.read_int_on_heart_badge`, a dedicated OCR path for the pink/magenta heart-shaped badge — the project's normal grayscale-threshold OCR (`read_int`) misreads it, because the badge's own outline stroke survives the same threshold as the digit glyph; a "done today" portrait keeps its number and gets a green checkmark added alongside it rather than losing the number, so done-ness is checked via that checkmark's color, not inferred from a failed OCR read
|
||||||
|
- lesson was live-tested with real tickets spent (see `plan.md` Phase 12), which surfaced two real bugs from that assumption gap plus an OCR contamination issue — both fixed; see Phase 12 for the full writeup
|
||||||
|
- `ba_auto/detector.py` has `find_cafe_sparkle()`, the sparkle template-match ported in-process from the now-deleted `scripts/detect_and_click.py`
|
||||||
|
- `scripts/ba_dailies_legacy.sh` and `scripts/detect_and_click.py` have been deleted
|
||||||
|
- the `scripts/` directory itself no longer exists
|
||||||
|
- `ba_auto/driver.py` primitives are wired into all migrated task modules
|
||||||
|
- existing driver primitives include `run_command`, `focus_game`, `click`, `move_mouse`, `scroll`, `keypress`, `screenshot`, `wait`, and `color_at`
|
||||||
|
- `ba_auto/navigation.py` has shared state probes used across tasks:
|
||||||
|
- `is_on_subscreen`
|
||||||
|
- `is_modal_open`
|
||||||
|
- `wait_for_state()`
|
||||||
|
|
||||||
Migration goal:
|
`wait_for_state()` is a scoped port of the reference's `core/picture.py::co_detect`: watch for any of several named states, react to known non-terminal ones, and stop on a recognized terminal one.
|
||||||
|
|
||||||
- preserve existing behavior
|
`story_sweep.py` additionally has its own local modal probes and close logic because the stage-info modal is wide enough to break `is_modal_open`'s default probe point, and it does not close on Escape. See `plan.md` Phase 9.
|
||||||
- move feature logic from Bash into Python
|
|
||||||
- keep `ba_dailies.sh` as launcher only
|
The story sweep modal renders at least two different internal layouts:
|
||||||
- make detection logic reusable
|
|
||||||
- prepare for future features by copying the reference project's structure where appropriate
|
- a plain one for the bonus `-A` stage
|
||||||
|
- a taller tabbed one for regular numbered stages
|
||||||
|
|
||||||
|
Their button coordinates differ. This was discovered live in Phase 10.
|
||||||
|
|
||||||
|
Live testing found both the mailbox-icon and cafe-icon fixed coordinates were flaky. They missed the first click and worked on retry. Neither task originally verified state before proceeding, so a missed click cascaded into blind actions and could reach an unverified Escape press on the home screen, which triggers Blue Archive's own "exit the game?" confirmation.
|
||||||
|
|
||||||
|
See `plan.md` Phases 5–6 for the full writeup. This is the concrete reason every task now verifies state before acting rather than trusting fixed coordinates or a single click blindly.
|
||||||
|
|
||||||
|
Rank-up popups mid-pat-loop are now handled.
|
||||||
|
|
||||||
|
A pat that crosses an affection-rank threshold shows a full-screen `絆ランクアップ!` cutscene with no cafe header. `find_cafe_sparkle()` can never recognize this because it is nothing like the sparkle template. The loop used to spin uselessly against it for the rest of the room's click budget.
|
||||||
|
|
||||||
|
`cafe.py`'s `_dismiss_rank_up_if_shown()` reuses the existing `navigation.is_on_subscreen` header-brightness probe. This was confirmed against `screenshots/cafe/student/01-02`:
|
||||||
|
|
||||||
|
- rank-up cutscene: header probe reads `r < 200`
|
||||||
|
- normal cafe screen: header probe reads `r > 200`
|
||||||
|
|
||||||
|
It presses Enter until the cutscene clears, porting the reference's own `to_cafe()` navigation handling of `relationship_rank_up` from `module/cafe_reward.py`.
|
||||||
|
|
||||||
|
This is not yet live-confirmed against a real cafe-pat rank-up trigger specifically. It is grounded in real captured screenshots, but treat it as implemented-but-unverified until one happens naturally during a real cafe run.
|
||||||
|
|
||||||
|
The same full-screen `絆ランクアップ!` cutscene *was* confirmed live during Lesson/Schedule work (see `plan.md` Phase 12), and that work found `is_on_subscreen`'s fixed header probe reads inconsistently across different characters' cutscene art — bright for one, which would make cafe's "keep pressing Enter while NOT on subscreen" loop think the cutscene had already cleared when it hadn't. Not necessarily hazardous on its own (it would just stop dismissing early and let the pat loop retry against a still-showing cutscene), but worth a more robust probe if cafe's rank-up path ever misbehaves live.
|
||||||
|
|
||||||
|
Not yet verified for cafe:
|
||||||
|
|
||||||
|
- whether camera zoom/pan can drift over a long unattended run
|
||||||
|
- the reference project zooms out before detecting
|
||||||
|
- this project currently does not
|
||||||
|
- testing did not reproduce a failure from skipping it
|
||||||
|
- see `plan.md` Phase 6 "Not verified" list
|
||||||
|
|
||||||
## Known cafe gaps to verify during migration
|
## Known cafe gaps to verify during migration
|
||||||
|
|
||||||
When migrating cafe logic, verify these manually against the actual current behavior:
|
When migrating or modifying cafe logic, verify these manually against the actual current behavior:
|
||||||
|
|
||||||
- whether student rotation popups are handled
|
- whether student rotation popups are handled
|
||||||
- whether rank-up popups are handled
|
- whether rank-up popups are handled
|
||||||
@ -454,13 +721,23 @@ When migrating cafe logic, verify these manually against the actual current beha
|
|||||||
|
|
||||||
Do not assume the old Bash implementation handles these correctly.
|
Do not assume the old Bash implementation handles these correctly.
|
||||||
|
|
||||||
|
## Migration goal
|
||||||
|
|
||||||
|
The migration goal is:
|
||||||
|
|
||||||
|
- preserve existing behavior
|
||||||
|
- move feature logic from Bash into Python
|
||||||
|
- keep `ba_dailies.sh` as launcher only
|
||||||
|
- make detection logic reusable
|
||||||
|
- prepare for future features by copying the reference project's structure where appropriate
|
||||||
|
|
||||||
## Anti-patterns
|
## Anti-patterns
|
||||||
|
|
||||||
Do not:
|
Do not:
|
||||||
|
|
||||||
- add new `do_<feature>` Bash functions
|
- add new `do_<feature>` Bash functions
|
||||||
- build a giant Bash automation script
|
- build a giant Bash automation script
|
||||||
- recreate reference logic as fixed coordinate Bash click chains
|
- recreate reference logic as fixed-coordinate Bash click chains
|
||||||
- skip reading the reference module before implementing a feature
|
- skip reading the reference module before implementing a feature
|
||||||
- put OCR in Bash
|
- put OCR in Bash
|
||||||
- put OpenCV state machines in Bash
|
- put OpenCV state machines in Bash
|
||||||
@ -469,6 +746,13 @@ Do not:
|
|||||||
- copy reference image assets blindly
|
- copy reference image assets blindly
|
||||||
- edit `~/repo/baas-reference/`
|
- edit `~/repo/baas-reference/`
|
||||||
- implement event-specific features before the generic reusable machinery exists
|
- implement event-specific features before the generic reusable machinery exists
|
||||||
|
- invent a pixel probe, fixed-coordinate, or randomized substitute for reference logic that uses OCR, just to avoid setting up OCR
|
||||||
|
- treat OCR as an Android-specific concern that can be deferred indefinitely
|
||||||
|
- run temporary probes from `/private/tmp/claude-*` instead of the repository `scratchpad/`
|
||||||
|
- create investigation scripts outside the repository when they can live in `scratchpad/`
|
||||||
|
- use multiline `python3 -c "..."` commands for non-trivial probes
|
||||||
|
- rely on Claude Code internal temporary paths in committed code, notes, commands, or debugging workflow
|
||||||
|
- write generated debug files into the repository root when they belong in `scratchpad/`
|
||||||
|
|
||||||
## Feature implementation workflow
|
## Feature implementation workflow
|
||||||
|
|
||||||
@ -476,7 +760,7 @@ For every new feature:
|
|||||||
|
|
||||||
1. Read the relevant `~/repo/baas-reference/module/...` file.
|
1. Read the relevant `~/repo/baas-reference/module/...` file.
|
||||||
2. Summarize the upstream feature flow in notes or comments.
|
2. Summarize the upstream feature flow in notes or comments.
|
||||||
3. Add/update the reference mapping table.
|
3. Add or update the reference mapping table.
|
||||||
4. Identify missing local driver primitives.
|
4. Identify missing local driver primitives.
|
||||||
5. Implement or improve those primitives in `ba_auto/driver.py` or `ba_auto/detector.py`.
|
5. Implement or improve those primitives in `ba_auto/driver.py` or `ba_auto/detector.py`.
|
||||||
6. Implement the feature in `ba_auto/tasks/<feature>.py`.
|
6. Implement the feature in `ba_auto/tasks/<feature>.py`.
|
||||||
@ -486,17 +770,20 @@ For every new feature:
|
|||||||
10. Deploy to `nik-gpu`.
|
10. Deploy to `nik-gpu`.
|
||||||
11. Test against the live game.
|
11. Test against the live game.
|
||||||
12. Update `plan.md` status.
|
12. Update `plan.md` status.
|
||||||
|
13. Move temporary investigation files into `scratchpad/`, or delete them if they are no longer useful.
|
||||||
|
|
||||||
## Priority when uncertain
|
## Priority when uncertain
|
||||||
|
|
||||||
When uncertain, prefer this order:
|
When uncertain, prefer this order:
|
||||||
|
|
||||||
1. Preserve existing working behavior.
|
1. Preserve existing working behavior.
|
||||||
2. Follow the reference project's control flow.
|
2. Follow the reference project's control flow, including its use of OCR.
|
||||||
3. Keep logic in Python.
|
3. Do not deprioritize OCR-based navigation/matching in favor of a simpler non-OCR substitute.
|
||||||
4. Add reusable driver primitives instead of feature-specific hacks.
|
4. Keep logic in Python.
|
||||||
5. Use local screenshots/assets only when backend differences require it.
|
5. Add reusable driver primitives instead of feature-specific hacks.
|
||||||
6. Avoid large rewrites that do not move the project closer to reference-driven Python architecture.
|
6. Use local screenshots/assets only when backend differences require it.
|
||||||
|
7. Use `scratchpad/` for temporary work and diagnostics.
|
||||||
|
8. Avoid large rewrites that do not move the project closer to reference-driven Python architecture.
|
||||||
|
|
||||||
## User preference
|
## User preference
|
||||||
|
|
||||||
@ -504,4 +791,6 @@ The user wants this project to be as close to the original Blue Archive Auto Scr
|
|||||||
|
|
||||||
The user specifically does not want Claude Code to keep converting feature work into Bash.
|
The user specifically does not want Claude Code to keep converting feature work into Bash.
|
||||||
|
|
||||||
Respect that preference.
|
The user also wants Claude Code to avoid using `/private/tmp/claude-*` for development probes when a repository-local `scratchpad/` directory can be used instead, because those paths trigger permission prompts and interrupt the development flow.
|
||||||
|
|
||||||
|
Respect those preferences.
|
||||||
37
README.md
37
README.md
@ -28,7 +28,12 @@ Current task status:
|
|||||||
| Command | Implementation |
|
| Command | Implementation |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `mailbox` | Real Python (`ba_auto/tasks/mailbox.py`). Verifies the mailbox panel actually opened (via a pixel-color probe, `driver.color_at`) before clicking "claim all" or pressing any further keys. Retries the open-click up to 3 times before giving up safely. |
|
| `mailbox` | Real Python (`ba_auto/tasks/mailbox.py`). Verifies the mailbox panel actually opened (via a pixel-color probe, `driver.color_at`) before clicking "claim all" or pressing any further keys. Retries the open-click up to 3 times before giving up safely. |
|
||||||
| `cafe` | Still a bridge: `ba_auto/tasks/cafe.py` shells out to `scripts/ba_dailies_legacy.sh cafe`, the original unverified fixed-coordinate click sequence, unchanged. Not yet hardened — see "Known issue" below. |
|
| `cafe` | Real Python (`ba_auto/tasks/cafe.py`). Verifies each room/dialog transition the same way as `mailbox` before acting; sparkle detection runs in-process via `ba_auto/detector.py` instead of shelling out per click. See "Fixed: the exit-game dialog bug" below for what this replaced, and `plan.md` Phase 6 follow-up for the multi-scale detection + persistent-polling changes made after a "farming affection doesn't happen" report. |
|
||||||
|
| `stamina` | Real Python (`ba_auto/tasks/stamina.py`). Opens the Mission panel and claims via its bulk "一括受取" button (Enter key) when enabled. Does **not** touch the Pyroxene Purchase (青輝石購入) menu's free-AP claim — that's a real-money purchase screen and was deliberately left unautomated; see `plan.md` Phase 8. |
|
||||||
|
| `story_sweep` | Real Python (`ba_auto/tasks/story_sweep.py`). **Spends AP** — opt-in only, not part of the default flow. Sweeps a config-driven list of exact `(region, stage, count)` targets (`config.STORY_SWEEP_TARGETS` — edit this before running for real, it ships with a placeholder), navigating to each via OCR (region-number readout + stage-label matching) rather than a random pick. Verifies the MAX/`+` click actually raised the count before starting the sweep, clicks through the AP-usage-confirmation dialog, and closes the stage-info modal via its own X button afterward (Escape doesn't close it — confirmed live). See `plan.md` Phase 10. |
|
||||||
|
| `shop_common` | Real Python (`ba_auto/tasks/shop_common.py`). **Spends credits** — opt-in only, not part of the default flow. Buys a config-driven list of exact `(row, col, name, expected_price)` targets (`config.COMMON_SHOP_TARGETS`) from the 通常アイテム tab: OCR-verifies each item's price before checking its box, then clicks the bulk 購入 button and confirms through the purchase dialog. Live-tested with real purchases. See `plan.md` Phase 11. |
|
||||||
|
| `shop_tactical` | Real Python (`ba_auto/tasks/shop_tactical.py`). **Spends tactical coin** — opt-in only, not part of the default flow. Same flow as `shop_common` against the 戦術対抗戦 tab and `config.TACTICAL_SHOP_TARGETS`. Live-tested with real purchases. See `plan.md` Phase 11. |
|
||||||
|
| `lesson` | Real Python (`ba_auto/tasks/lesson.py`). **Spends lesson tickets** — opt-in only, not part of the default flow. Sweeps every unlocked region's schedule grid, picking the highest-affection available lesson each time (via `detector.read_int_on_heart_badge` OCR on each portrait's heart-shaped badge) until tickets or lessons run out. Live-tested with real tickets spent. See `plan.md` Phase 12. |
|
||||||
|
|
||||||
## Prerequisites on nik-gpu
|
## Prerequisites on nik-gpu
|
||||||
|
|
||||||
@ -36,10 +41,10 @@ One-time, or after a dependency change:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
ssh nik-gpu
|
ssh nik-gpu
|
||||||
which xdotool scrot # both must be installed
|
which xdotool scrot tesseract # all three must be installed
|
||||||
```
|
```
|
||||||
|
|
||||||
`setup.sh` (see below) creates the Python venv and checks these for you.
|
`tesseract` (the OCR engine `story_sweep` uses for region/stage-label reads) needs `sudo apt install tesseract-ocr` — `setup.sh` checks for it but can't install it for you, since sudo needs an interactive password. `setup.sh` (see below) creates the Python venv (including `pytesseract`) and checks all three tools for you.
|
||||||
|
|
||||||
## Deploying your changes
|
## Deploying your changes
|
||||||
|
|
||||||
@ -55,16 +60,21 @@ rsync -av --delete \
|
|||||||
ssh nik-gpu "cd ~/repo/ba-auto-daily && ./setup.sh"
|
ssh nik-gpu "cd ~/repo/ba-auto-daily && ./setup.sh"
|
||||||
```
|
```
|
||||||
|
|
||||||
`setup.sh` copies the synced files into the fixed runtime paths (`~/ba_dailies.sh`, `~/ba_daily.py`, `~/ba_auto/`, `~/ba_scripts/`, `~/ba_assets/`) and (re)installs the venv. Re-run both commands any time you change code — there is no auto-deploy.
|
`setup.sh` copies the synced files into the fixed runtime paths (`~/ba_dailies.sh`, `~/ba_daily.py`, `~/ba_auto/`, `~/ba_assets/`) and (re)installs the venv. Re-run both commands any time you change code — there is no auto-deploy.
|
||||||
|
|
||||||
## Running it
|
## Running it
|
||||||
|
|
||||||
The game must already be running on `nik-gpu` (window titled `BlueArchive`). Then, on `nik-gpu`:
|
The game must already be running on `nik-gpu` (window titled `BlueArchive`). Then, on `nik-gpu`:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
~/ba_dailies.sh # default: mailbox, then cafe
|
~/ba_dailies.sh # default: mailbox, cafe, then stamina
|
||||||
~/ba_dailies.sh mailbox # just mailbox
|
~/ba_dailies.sh mailbox # just mailbox
|
||||||
~/ba_dailies.sh cafe # just cafe
|
~/ba_dailies.sh cafe # just cafe
|
||||||
|
~/ba_dailies.sh stamina # just the Mission-panel bulk claim
|
||||||
|
~/ba_dailies.sh story_sweep # spends AP -- not in the default flow, run explicitly
|
||||||
|
~/ba_dailies.sh shop_common # spends credits -- not in the default flow, run explicitly
|
||||||
|
~/ba_dailies.sh shop_tactical # spends tactical coin -- not in the default flow, run explicitly
|
||||||
|
~/ba_dailies.sh lesson # spends lesson tickets -- not in the default flow, run explicitly
|
||||||
```
|
```
|
||||||
|
|
||||||
You can also run these remotely without a separate `ssh` login step:
|
You can also run these remotely without a separate `ssh` login step:
|
||||||
@ -73,7 +83,7 @@ You can also run these remotely without a separate `ssh` login step:
|
|||||||
ssh nik-gpu "~/ba_dailies.sh mailbox"
|
ssh nik-gpu "~/ba_dailies.sh mailbox"
|
||||||
```
|
```
|
||||||
|
|
||||||
Exit code `0` means the script ran to completion — it does **not** by itself guarantee the in-game action succeeded (`mailbox` checks and logs this explicitly; `cafe` currently does not).
|
Exit code `0` means the script ran to completion without a Python exception — it does **not** by itself guarantee the in-game action succeeded. `mailbox`, `cafe`, and `stamina` all log what they actually detected and did (or why they safely aborted), so check the log output, not just the exit code.
|
||||||
|
|
||||||
## How to watch/verify it
|
## How to watch/verify it
|
||||||
|
|
||||||
@ -86,11 +96,13 @@ Exit code `0` means the script ran to completion — it does **not** by itself g
|
|||||||
```
|
```
|
||||||
- **Read the log output.** Each task prints what it's doing, e.g. `mailbox` prints `panel not detected after click (attempt N/3)` if a click misses, and `nothing to claim` vs `claiming all` depending on what it found.
|
- **Read the log output.** Each task prints what it's doing, e.g. `mailbox` prints `panel not detected after click (attempt N/3)` if a click misses, and `nothing to claim` vs `claiming all` depending on what it found.
|
||||||
|
|
||||||
## Known issue: don't trust `cafe` unattended yet
|
## Fixed: the exit-game dialog bug
|
||||||
|
|
||||||
Live-testing `mailbox` surfaced a real bug in the *original*, unverified click sequence (see `plan.md` Phase 5 for the full writeup): a slightly-off icon coordinate caused a missed click, and the fixed sequence blindly kept going — clicking, pressing Enter, pressing Escape — with no idea whether any of it landed. The trailing Escape ended up hitting the bare home screen, which triggers Blue Archive's own **"Exit the game?"** confirmation dialog.
|
Live-testing both `mailbox` and `cafe` surfaced the same real bug in the *original*, unverified click sequences (see `plan.md` Phases 5–6 for the full writeup): a slightly-off icon coordinate occasionally caused a missed click, and the fixed sequence blindly kept going — clicking, pressing Enter, pressing Escape — with no idea whether any of it landed. The trailing Escape ended up hitting the bare home screen, which triggers Blue Archive's own **"Exit the game?"** confirmation dialog. `cafe`'s sequence is much longer than `mailbox`'s (open → pat loop ×15 → switch room → pat loop ×15 → claim income → exit), so a missed click there had more room to cascade.
|
||||||
|
|
||||||
`mailbox.py` now guards against this (verifies state before acting, retries, aborts safely instead of guessing). **`cafe` still uses the old unverified bridge and has the same failure mode.** If you run `~/ba_dailies.sh cafe` and something looks off, check the screen before pressing anything — if you see an unexpected confirmation dialog, press **Escape/Cancel**, never Enter/OK, until you've confirmed what it's asking.
|
Both `mailbox.py` and `cafe.py` now guard against this: they verify state via pixel-color probes before acting, retry a bounded number of times on a missed click, and abort safely (no further keypresses) instead of guessing. This was verified against the live game, including an actual income claim (gold and AP increased as expected) and a real sparkle detect-and-click.
|
||||||
|
|
||||||
|
That said, this is one round of live testing, not exhaustive coverage — see `plan.md` Phase 6 "Not verified" for open risks (rank-up popups mid-loop, long-run camera zoom/pan drift). If you ever see an unexpected confirmation dialog while running either command, press **Escape/Cancel**, never Enter/OK, until you've confirmed what it's asking.
|
||||||
|
|
||||||
## Local checks (nik-macbookair)
|
## Local checks (nik-macbookair)
|
||||||
|
|
||||||
@ -98,7 +110,6 @@ The game can't run here, so this only catches syntax errors, not behavior:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
bash -n ba_dailies.sh
|
bash -n ba_dailies.sh
|
||||||
bash -n scripts/ba_dailies_legacy.sh
|
|
||||||
python3 -m py_compile ba_daily.py ba_auto/*.py ba_auto/tasks/*.py
|
python3 -m py_compile ba_daily.py ba_auto/*.py ba_auto/tasks/*.py
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -107,5 +118,5 @@ Real verification only happens by actually running against the live game on `nik
|
|||||||
## More detail
|
## More detail
|
||||||
|
|
||||||
- `CLAUDE.md` — architecture rules and conventions for this repo.
|
- `CLAUDE.md` — architecture rules and conventions for this repo.
|
||||||
- `plan.md` — feature-by-feature migration status and backlog, including the mailbox bug writeup.
|
- `plan.md` — feature-by-feature migration status and backlog, including the mailbox/cafe exit-game-dialog bug writeup.
|
||||||
- `ba_auto/reference_notes/mapping.md` — maps each feature to its `baas-reference` source.
|
- `ba_auto/reference_notes/mapping.md` — maps each feature to its `baas-reference` source.
|
||||||
|
|||||||
@ -6,11 +6,368 @@ XAUTHORITY = "/run/user/1000/gdm/Xauthority"
|
|||||||
ENV = {**os.environ, "DISPLAY": DISPLAY, "XAUTHORITY": XAUTHORITY}
|
ENV = {**os.environ, "DISPLAY": DISPLAY, "XAUTHORITY": XAUTHORITY}
|
||||||
|
|
||||||
WINDOW_NAME = "BlueArchive"
|
WINDOW_NAME = "BlueArchive"
|
||||||
|
ASSET_DIR = os.path.expanduser("~/ba_assets")
|
||||||
|
|
||||||
# TODO: remove once cafe is ported off the legacy Bash bridge (mailbox no longer uses this).
|
# Runtime working files (probe/detection screenshots); never /tmp, per CLAUDE.md.
|
||||||
LEGACY_SCRIPT = os.path.expanduser("~/ba_scripts/ba_dailies_legacy.sh")
|
SCRATCHPAD_DIR = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "scratchpad")
|
||||||
|
os.makedirs(SCRATCHPAD_DIR, exist_ok=True)
|
||||||
|
|
||||||
# Refined from (1726, 60): that coordinate sat on the edge of the icon's
|
# Refined from (1726, 60): that coordinate sat on the edge of the icon's
|
||||||
# hitbox and intermittently missed during live testing.
|
# hitbox and intermittently missed during live testing.
|
||||||
MAILBOX_ICON = (1732, 50)
|
MAILBOX_ICON = (1732, 50)
|
||||||
CLAIM_ALL = (1691, 1128)
|
CLAIM_ALL = (1691, 1128)
|
||||||
|
|
||||||
|
MISSION_ICON = (75, 350)
|
||||||
|
# Background pixel inside the "一括受取" (claim all) button on the Mission
|
||||||
|
# panel: bright yellow (r~250, r-b~180+) when something is claimable, flat
|
||||||
|
# grey (r-b<20) when not. Calibrated live at 1920x1200.
|
||||||
|
MISSION_CLAIM_PROBE = (1600, 1100)
|
||||||
|
|
||||||
|
CAFE_ICON = (165, 1100)
|
||||||
|
CAFE_ROOM_SWITCH = (190, 160)
|
||||||
|
CAFE_INCOME = (1780, 1105)
|
||||||
|
# Max sparkle-detection attempts per room (hits and misses both count --
|
||||||
|
# sparkles are on a per-student cooldown, so most checks legitimately find
|
||||||
|
# nothing and the loop keeps polling rather than giving up after one miss).
|
||||||
|
CAFE_MAX_CLICKS_PER_ROOM = 15
|
||||||
|
CAFE_SPARKLE_TEMPLATE = os.path.join(ASSET_DIR, "cafe_sparkle.png")
|
||||||
|
# A pat that crosses an affection-rank threshold shows a full-screen "絆ラン
|
||||||
|
# クアップ!" (Bond Rank Up!) cutscene with no cafe header visible at all --
|
||||||
|
# confirmed against screenshots/cafe/student/01-02: navigation.is_on_subscreen's
|
||||||
|
# header probe reads (183,220,240) there (r<200, fails) vs (248,249,250) on
|
||||||
|
# the normal cafe screen (r>200, passes), so the existing header-brightness
|
||||||
|
# check already tells the two apart. Bounds how many Enter presses
|
||||||
|
# _dismiss_rank_up_if_shown will try before giving up.
|
||||||
|
CAFE_RANK_UP_DISMISS_RETRIES = 5
|
||||||
|
|
||||||
|
# Home -> お仕事 (Work hub) -> 任務 (Task) card -> Normal/Hard story region browser.
|
||||||
|
WORK_ICON = (1793, 1138)
|
||||||
|
# Moved up from the original (1370, 450): that point sat close enough to the
|
||||||
|
# 任務 card's bottom edge that a live Phase 10 run missed and landed on the
|
||||||
|
# "総力戦" (Total War) card in the row below instead -- confirmed live via
|
||||||
|
# screenshot, not just a hunch. (1250, 380) sits solidly mid-card, on the
|
||||||
|
# "任務" title text itself, well clear of every edge.
|
||||||
|
TASK_CARD = (1250, 380)
|
||||||
|
REGION_RIGHT_ARROW = (1862, 598)
|
||||||
|
# Pixel-scanline-scanned (not visually estimated -- see plan.md Phase 8's
|
||||||
|
# lesson) from scratchpad/stage_info.png: the "<" chevron's navy-blue pixel
|
||||||
|
# centroid was (66, 597), mirroring REGION_RIGHT_ARROW. Used for the
|
||||||
|
# OCR-driven to_region port (ba_auto/tasks/story_sweep.py) to step backward
|
||||||
|
# when the current region is past the target.
|
||||||
|
REGION_LEFT_ARROW = (66, 598)
|
||||||
|
# Bounds the "read region, click delta, re-check" loop in
|
||||||
|
# ba_auto/tasks/story_sweep.py's _go_to_region. A correct read normally
|
||||||
|
# converges in one round; this just guards against a stuck OCR misread.
|
||||||
|
REGION_NAV_MAX_ATTEMPTS = 8
|
||||||
|
|
||||||
|
# Region-number readout on the region browser's left panel (the "Area 30"
|
||||||
|
# card's big digits, below the smaller "Area" label). Rect pixel-scanned from
|
||||||
|
# scratchpad/stage_info.png: the "Area" label occupies roughly y 295-325, the
|
||||||
|
# number itself y 330-372 -- this rect isolates just the digits. Replaces
|
||||||
|
# _go_to_latest_region's "spam the arrow and hope" (see plan.md Phase 9's
|
||||||
|
# retrospective) with task_utils.py::to_region's actual OCR-read-and-click-
|
||||||
|
# the-exact-delta approach.
|
||||||
|
REGION_NUMBER_OCR_RECT = (175, 325, 250, 380)
|
||||||
|
|
||||||
|
# Stage list panel (right side of the region browser). Scrolling to either
|
||||||
|
# extreme always shows exactly 4 full stage rows, since every region has at
|
||||||
|
# least 5 stages -- scrolling past either end is a harmless no-op (verified
|
||||||
|
# live), so a generous bounded click count is safe. Each stage row's "入場"
|
||||||
|
# (enter) button sits at STAGE_ENTER_X across both scroll extremes; the two
|
||||||
|
# row-position sets below were measured at each extreme.
|
||||||
|
STAGE_LIST_SCROLL_POINT = (1400, 700)
|
||||||
|
# 10 was the original (Phase 9) calibration, but live re-testing during
|
||||||
|
# Phase 10 found it insufficient to reach the opposite extreme when the list
|
||||||
|
# was already scrolled near the other end (it undershot, landing between the
|
||||||
|
# two calibrated row-position sets and producing garbled OCR reads) -- 20
|
||||||
|
# reliably reached either extreme regardless of starting position. Scrolling
|
||||||
|
# past either end remains a harmless no-op (verified live both phases).
|
||||||
|
STAGE_LIST_SCROLL_CLICKS = 20
|
||||||
|
STAGE_ENTER_X = 1683
|
||||||
|
STAGE_ROWS_AT_TOP_Y = (424, 570, 718, 866)
|
||||||
|
STAGE_ROWS_AT_BOTTOM_Y = (483, 630, 778, 926)
|
||||||
|
|
||||||
|
# Stage-label OCR rect (e.g. "30-1", "30-A"), offset from a row's known
|
||||||
|
# center y above. Pixel-scanned across all eight row positions (both
|
||||||
|
# STAGE_ROWS_AT_TOP_Y and _BOTTOM_Y) in live captures -- x[1030,1150],
|
||||||
|
# y[row_y-40, row_y-4] consistently isolates just the label text line above
|
||||||
|
# the row's star-rating icons. The symmetric row_y+/-40 crop tried first
|
||||||
|
# during calibration included the stars below and reliably broke OCR (empty
|
||||||
|
# or garbled reads) even with a character whitelist -- see plan.md Phase 10.
|
||||||
|
# Replaces _pick_random_stage_row's "scroll to an extreme, grab a random one
|
||||||
|
# of the 4 rows" with a scoped-down port of the reference's
|
||||||
|
# swipe_search_target_str: since this client's stage list only ever has 2
|
||||||
|
# relevant scroll positions (top/bottom extremes, both already known-good),
|
||||||
|
# full swipe-and-retry generality isn't needed -- just OCR each of the 4
|
||||||
|
# visible rows at each extreme and match by label text.
|
||||||
|
STAGE_LABEL_OCR_X = (1030, 1150)
|
||||||
|
STAGE_LABEL_OCR_Y_PAD = (40, 4)
|
||||||
|
|
||||||
|
# 任務情報 (stage info) modal's sweep sub-panel. This modal is wide enough
|
||||||
|
# that navigation.MODAL_DIM_PROBE (960, 200) lands on the modal's own white
|
||||||
|
# card instead of the dimmed backdrop -- use a corner point that's outside
|
||||||
|
# the card in either scroll/region state instead.
|
||||||
|
STAGE_MODAL_PROBE = (1870, 600)
|
||||||
|
# Regular numbered stages (30-1..30-5) render an extra "集中指揮"/"簡易攻略"
|
||||||
|
# tab row and a manual "任務開始" panel below the sweep panel that the
|
||||||
|
# bonus "-A" stage Phase 9 originally calibrated against does not have --
|
||||||
|
# discovered live during Phase 10 when Phase 9's coordinates (calibrated
|
||||||
|
# only against 30-A) missed the MAX button on 30-3 by ~43px vertically.
|
||||||
|
# These values are pixel-scanned against 30-3's tabbed layout and are what
|
||||||
|
# config.STORY_SWEEP_TARGETS will hit in the common case (sweeping a
|
||||||
|
# regular numbered stage, not the "-A" bonus stage). If a target's stage is
|
||||||
|
# "A", these may be off by the same ~43px the old (untabbed) calibration
|
||||||
|
# used -- not yet re-confirmed against an actual "-A" stage since this fix;
|
||||||
|
# see plan.md Phase 10.
|
||||||
|
SWEEP_MAX_BUTTON = (1620, 550)
|
||||||
|
SWEEP_START_BUTTON = (1400, 710)
|
||||||
|
# The "-" stepper button next to the sweep count: flat grey (240,240,239)
|
||||||
|
# while count is still at its default of 1, vivid orange (255,111,0) once
|
||||||
|
# MAX (or any +) has raised it. Used to verify the MAX click actually landed
|
||||||
|
# instead of trusting a single click blindly, since this gates real AP spend.
|
||||||
|
SWEEP_MINUS_BUTTON_PROBE = (1285, 555)
|
||||||
|
# The modal's own "X" close icon (top-right corner of the white card).
|
||||||
|
# Escape does NOT close this modal (confirmed live: two Escape presses left
|
||||||
|
# it open with focus on the live "任務開始"/start-mission button) -- must
|
||||||
|
# click this explicitly. Pinned via pixel-scanline scan of the glyph's
|
||||||
|
# crossing point, not visual estimation.
|
||||||
|
#
|
||||||
|
# The whole card (not just the sweep sub-panel) is vertically centered on
|
||||||
|
# its own content height rather than anchored at a fixed absolute position
|
||||||
|
# -- confirmed live during Phase 10: the tabbed regular-stage layout (see
|
||||||
|
# SWEEP_MAX_BUTTON above) is taller than the "-A" bonus-stage layout this
|
||||||
|
# was originally calibrated against, and its X button sits ~46px higher
|
||||||
|
# on screen (225 vs the old 271) as a result. This value is re-measured
|
||||||
|
# against the taller, tabbed layout.
|
||||||
|
STAGE_MODAL_CLOSE_BUTTON = (1691, 225)
|
||||||
|
# "+" stepper button, for configured exact (non-"max") sweep counts. Pinned
|
||||||
|
# via color-scan (bright cyan glyph centroid) against 30-3's tabbed layout
|
||||||
|
# (see SWEEP_MAX_BUTTON above) -- unlike SWEEP_MAX_BUTTON/
|
||||||
|
# SWEEP_MINUS_BUTTON_PROBE this specific button has NOT been live-clicked
|
||||||
|
# yet; confirm it before relying on a non-"max" configured count (see
|
||||||
|
# plan.md Phase 10).
|
||||||
|
SWEEP_PLUS_BUTTON = (1520, 550)
|
||||||
|
|
||||||
|
# Clicking 掃討開始 (start sweep) always raises an "AP<N>使用して、掃討を
|
||||||
|
# <M>回行いますか?" usage-confirmation dialog before the sweep actually
|
||||||
|
# runs -- discovered live during Phase 10; the previous design had no
|
||||||
|
# handling for this dialog at all, which is what a crude early placeholder
|
||||||
|
# probe was misreading as "inadequate_ap" on every sweep, successful or not.
|
||||||
|
#
|
||||||
|
# If AP is too low for even one sweep (confirmed live by deliberately
|
||||||
|
# emptying the count via MAX/"+" at low AP), a dialog that looks the same
|
||||||
|
# and sits at the *same* OK-button position appears instead, but titled
|
||||||
|
# "AP購入" (real-currency AP purchase) with a gold/yellow OK button instead
|
||||||
|
# of cyan. The two are told apart by that color, not by position.
|
||||||
|
SWEEP_CONFIRM_BUTTON = (1150, 810)
|
||||||
|
SWEEP_CONFIRM_CANCEL_BUTTON = (770, 810)
|
||||||
|
SWEEP_CONFIRM_CYAN = ((90, 190, 230), (200, 240, 256))
|
||||||
|
SWEEP_CONFIRM_GOLD = ((200, 200, 50), (256, 256, 150))
|
||||||
|
|
||||||
|
# Region spanning every button this task clicks through after 掃討開始: the
|
||||||
|
# AP-usage-confirm OK above, and the "掃討完了" (sweep complete) results
|
||||||
|
# screen's "SKIP" (first, skips the reward-reveal animation) and final "OK"
|
||||||
|
# (after full reward totals appear) buttons. SKIP and the final OK share
|
||||||
|
# SWEEP_CONFIRM_CYAN's color but sit ~120px apart vertically, so this task
|
||||||
|
# finds whichever one is showing by color within this region instead of
|
||||||
|
# hardcoding each dialog's exact Y position.
|
||||||
|
SWEEP_RESULT_BUTTON_REGION = (700, 700, 1300, 1050)
|
||||||
|
|
||||||
|
# (region, stage, count) targets to sweep, mirroring the reference's
|
||||||
|
# unfinished_normal_tasks shape (module/explore_tasks/sweep_task.py). `stage`
|
||||||
|
# is 1-5, or the string "A" for the bonus stage that only exists when
|
||||||
|
# region % 3 == 0. `count` is a positive int (uses the "+" stepper) or the
|
||||||
|
# literal string "max" (uses the in-game MAX button).
|
||||||
|
#
|
||||||
|
# Empty by default -- the earlier placeholder here was (1, 1, "max"), which
|
||||||
|
# a real run then dutifully swept region 1 stage 1 instead of the account's
|
||||||
|
# actual last region, since story_sweep has no "find the latest region"
|
||||||
|
# heuristic anymore (see plan.md Phase 9's retrospective for why that
|
||||||
|
# heuristic was removed). Add entries here for any additional fixed targets
|
||||||
|
# you want swept every run, on top of the daily rotation target below.
|
||||||
|
STORY_SWEEP_TARGETS = []
|
||||||
|
|
||||||
|
# Daily-rotating target: sweeps a single region, cycling through its stages
|
||||||
|
# one per day rather than grinding the same stage every run, per explicit
|
||||||
|
# user direction. `ROTATION_STAGE_COUNT` is how many stages that region has
|
||||||
|
# (1..N); which one runs today is `today's date -> N` via a plain date
|
||||||
|
# ordinal modulo, not the calendar day-of-year, so the cycle doesn't skip or
|
||||||
|
# repeat around a year boundary. Set STORY_SWEEP_ROTATION_REGION to None to
|
||||||
|
# disable this and only sweep STORY_SWEEP_TARGETS.
|
||||||
|
STORY_SWEEP_ROTATION_REGION = 30
|
||||||
|
STORY_SWEEP_ROTATION_STAGE_COUNT = 6
|
||||||
|
STORY_SWEEP_ROTATION_COUNT = "max"
|
||||||
|
|
||||||
|
# Common Shop / Tactical Challenge Shop. Both tabs share the same underlying
|
||||||
|
# checkbox-grid-then-bulk-buy UI (the live equivalent of the reference's
|
||||||
|
# module/shop/shop_utils.py get_item_position/ensure_choose/buy pattern);
|
||||||
|
# see ba_auto/tasks/shop_utils.py for the shared control flow.
|
||||||
|
SHOP_ICON = (1155, 1085) # bottom nav "ショップ" icon on the home screen
|
||||||
|
SHOP_BACK_BUTTON = (85, 55)
|
||||||
|
SHOP_TAB_COMMON = (160, 208) # 通常アイテム tab (credit-point items)
|
||||||
|
# 戦術対抗戦 tab (tactical-coin items). The reference reaches this via
|
||||||
|
# goto_shop_by_name's OCR swipe-search over the shop-type tab list
|
||||||
|
# (module/shop/shop_utils.py) because that list can require scrolling on
|
||||||
|
# some accounts/versions. Confirmed live here: this account's tab list is
|
||||||
|
# only 7 entries and all fit on screen with no scroll needed, so a fixed
|
||||||
|
# click is faithful (there is nothing to search for), not a shortcut around
|
||||||
|
# the OCR the reference would otherwise need.
|
||||||
|
SHOP_TAB_TACTICAL = (160, 915)
|
||||||
|
|
||||||
|
# Item grid checkbox top-left-ish click point per (row, col), pixel-scanned
|
||||||
|
# live against the shop's real catalog (see plan.md's shop phase). Confirmed
|
||||||
|
# identical across both shop tabs -- it's the same shared UI component.
|
||||||
|
SHOP_ITEM_COL_X = [972, 1197, 1423, 1649]
|
||||||
|
SHOP_ITEM_ROW_Y = [297, 674]
|
||||||
|
# The checkbox glyph renders this vivid yellow-green only once checked
|
||||||
|
# (plain white/grey otherwise) -- pixel-sampled from a live checked vs.
|
||||||
|
# unchecked capture of the same card.
|
||||||
|
SHOP_CHECKED_RGB = ((60, 130, 80), (220, 245, 115))
|
||||||
|
# Each item's price-digit crop, as an (x1, y1, x2, y2) offset added to that
|
||||||
|
# item's own (col_x, row_y). Pixel-scanned and OCR-tested live against all 8
|
||||||
|
# configured targets' actual prices (12,500 up to 500,000) -- wide enough
|
||||||
|
# for the largest configured price without bleeding into the neighboring
|
||||||
|
# column's card.
|
||||||
|
SHOP_PRICE_OCR_OFFSET = (48, 166, 145, 195)
|
||||||
|
|
||||||
|
SHOP_BUY_BUTTON = (1751, 1112)
|
||||||
|
SHOP_CANCEL_BUTTON = (1525, 1112)
|
||||||
|
# A corner point that both the purchase-confirm dialog and the post-purchase
|
||||||
|
# "報酬獲得!" (reward acquired) banner dim away from pure white as they
|
||||||
|
# cover the screen -- confirmed live across both, and confirmed to stay pure
|
||||||
|
# white with no dialog open (across tab switches and scrolling). Cheaper and
|
||||||
|
# more robust than tracking each dialog's own layout individually.
|
||||||
|
SHOP_OVERLAY_PROBE = (100, 600)
|
||||||
|
SHOP_OVERLAY_IDLE_MIN_CHANNEL = 200
|
||||||
|
# Bounds shop_utils.confirm_purchase's "press Enter until idle" loop.
|
||||||
|
# Confirmed live: exactly 2 presses clears both the confirm dialog and the
|
||||||
|
# reward banner in one purchase; this leaves headroom for any additional
|
||||||
|
# one-time popup (e.g. a first-time notice) without spinning forever if the
|
||||||
|
# game is ever in a state this project doesn't recognize.
|
||||||
|
SHOP_PURCHASE_MAX_ENTER_PRESSES = 6
|
||||||
|
|
||||||
|
# Top status bar credit-point balance -- present on every screen, not
|
||||||
|
# shop-specific. Rect excludes the currency icon on the left, which OCR
|
||||||
|
# otherwise misreads as a spurious leading digit (confirmed live).
|
||||||
|
CREDIT_BALANCE_OCR_RECT = (1040, 15, 1290, 55)
|
||||||
|
# The Tactical Challenge Shop's own coin-balance readout, shown inline above
|
||||||
|
# the item grid on that tab specifically (not in the top status bar).
|
||||||
|
TACTICAL_COIN_OCR_RECT = (1090, 100, 1290, 150)
|
||||||
|
|
||||||
|
# (row, col, item name (log/debug only -- identification is by grid
|
||||||
|
# position, see shop_utils.py's module docstring for why), expected
|
||||||
|
# credit-point price). Buy-list confirmed with the user; all 8 are visible
|
||||||
|
# without scrolling.
|
||||||
|
COMMON_SHOP_TARGETS = [
|
||||||
|
(0, 0, "初級レポート", 12500),
|
||||||
|
(0, 1, "中級レポート", 125000),
|
||||||
|
(0, 2, "上級レポート", 300000),
|
||||||
|
(0, 3, "最上級レポート", 500000),
|
||||||
|
(1, 0, "初級強化珠", 10000),
|
||||||
|
(1, 1, "中級強化珠", 40000),
|
||||||
|
(1, 2, "上級強化珠", 96000),
|
||||||
|
(1, 3, "最上級強化珠", 128000),
|
||||||
|
]
|
||||||
|
|
||||||
|
# (row, col, item name, expected tactical-coin price). Buy-list confirmed
|
||||||
|
# with the user; both visible without scrolling.
|
||||||
|
TACTICAL_SHOP_TARGETS = [
|
||||||
|
(0, 0, "初級栄養ドリンク(AP30)", 15),
|
||||||
|
(0, 1, "中級栄養ドリンク(AP60)", 30),
|
||||||
|
]
|
||||||
|
|
||||||
|
# Lesson/Schedule (module/lesson.py). This client renders the reference's
|
||||||
|
# paged single-region view as a scrollable "Location Select" list of 12
|
||||||
|
# named regions instead -- these names are the reference's own
|
||||||
|
# lesson_region_name.JP list (core/config/default_config.py), embedded there
|
||||||
|
# directly rather than fetched externally, so hardcoding it locally isn't an
|
||||||
|
# external-data problem the way the shop price table was. They're used only
|
||||||
|
# for logging here, not for OCR identification -- unlike the reference's
|
||||||
|
# paged arrows, this list's scroll position is deterministic (see
|
||||||
|
# REGION_ROW_Y below), so there's nothing to OCR-locate.
|
||||||
|
LESSON_ICON = (314, 1100) # bottom nav "スケジュール" icon on the home screen
|
||||||
|
LESSON_BACK_BUTTON = (85, 55) # shared by both the region-list and per-region map screens
|
||||||
|
|
||||||
|
LESSON_REGION_NAMES = [
|
||||||
|
"シャーレオフィス", "シャーレ居住区", "ゲヘナ学園・中央区", "アビドス高等学校",
|
||||||
|
"ミレニアム・スタディーエリア", "トリニティ・スクエア", "レッドウインター連邦学園",
|
||||||
|
"百鬼夜行中心部", "D.U.シラトリ区", "山海経中央特区", "春葉原", "ワイルドハント総合芸術地区",
|
||||||
|
]
|
||||||
|
# The region list only ever settles at two scroll positions -- scrolled fully
|
||||||
|
# to top (regions 0-5 visible) or fully to bottom (regions 6-11) -- confirmed
|
||||||
|
# live: 15 scroll-down clicks always lands on the same bottom state, it
|
||||||
|
# doesn't keep scrolling past it. Each row's card is clickable at this
|
||||||
|
# center-ish Y regardless of which of the two scroll states is showing.
|
||||||
|
LESSON_REGION_ROW_Y = [265, 420, 585, 745, 900, 1060]
|
||||||
|
LESSON_REGION_LIST_SCROLL_POINT = (1400, 700)
|
||||||
|
LESSON_REGION_LIST_SCROLL_CLICKS = 15
|
||||||
|
LESSON_REGION_ROW_X = 1400
|
||||||
|
|
||||||
|
# Per-region isometric map screen -- opens the "全てのスケジュール" grid
|
||||||
|
# modal (this client's rendering of the reference's per-region 3x3
|
||||||
|
# get_lesson_each_region_status/get_lesson_relationship_counts grid).
|
||||||
|
LESSON_ALL_SCHEDULES_BUTTON = (1770, 1118)
|
||||||
|
LESSON_GRID_MODAL_CLOSE_BUTTON = (1710, 211)
|
||||||
|
|
||||||
|
# Grid modal cell layout: up to 3 columns x 3 rows of location cards, each
|
||||||
|
# showing up to 3 student portraits with a heart-shaped affection-count
|
||||||
|
# badge at bottom-right. Pixel-scanned live against two different regions'
|
||||||
|
# modals (Gehenna Central: 8 cells, Schale Office: 7 cells) -- consistent
|
||||||
|
# across both. A region with more than 9 currently-unlocked locations would
|
||||||
|
# need scrolling inside this modal, which isn't implemented (not yet seen
|
||||||
|
# live on this account; see plan.md).
|
||||||
|
LESSON_GRID_COL_X = [270, 786, 1302] # portrait-slot-0 center per column
|
||||||
|
LESSON_GRID_ROW_HEADER_Y = [380, 608, 836] # click target to open a cell's info panel
|
||||||
|
LESSON_GRID_ROW_PORTRAIT_Y = [486, 713, 940]
|
||||||
|
LESSON_GRID_PORTRAIT_STEP_X = 109 # slot 1/2 center = slot 0 center + N * this
|
||||||
|
# Badge center sits below-right of each portrait-slot's own center point.
|
||||||
|
LESSON_GRID_BADGE_OFFSET = (38, 24)
|
||||||
|
LESSON_GRID_BADGE_OCR_HALF_SIZE = (26, 20) # crop half-width/half-height around badge center
|
||||||
|
# A green checkmark can appear at top-right of a portrait ALONGSIDE its
|
||||||
|
# unchanged heart badge number, not instead of it (confirmed live -- an
|
||||||
|
# earlier assumption that "done" always blanks the number was wrong) --
|
||||||
|
# this is the only reliable "already done today" signal, so it's checked
|
||||||
|
# separately rather than inferred from the badge OCR.
|
||||||
|
LESSON_GRID_CHECKMARK_OFFSET = (41, -31)
|
||||||
|
LESSON_GRID_CHECKMARK_RGB = ((130, 190, 60), (220, 255, 160))
|
||||||
|
LESSON_GRID_CHECKMARK_HALF_SIZE = (14, 13)
|
||||||
|
# A masked, digit-only OCR read of the heart badge can still occasionally
|
||||||
|
# fuse a stray leading digit from portrait art bleeding into the crop's left
|
||||||
|
# edge (confirmed live: "13" -> "413", "18" -> "418", reproducible across
|
||||||
|
# every psm mode -- see detector.read_int_on_heart_badge). Real affection
|
||||||
|
# values never reach this range in practice, so treat anything this large
|
||||||
|
# as contamination and discard it rather than trust it.
|
||||||
|
LESSON_GRID_BADGE_MAX_PLAUSIBLE = 99
|
||||||
|
|
||||||
|
# "保有チケット N/M" readout, top-left of the region-list/map screens.
|
||||||
|
LESSON_TICKET_OCR_RECT = (295, 133, 385, 172)
|
||||||
|
|
||||||
|
# Per-cell info panel ("スケジュール情報"), opened by clicking a grid cell.
|
||||||
|
# Its Start button and the post-schedule report's OK button (below) are the
|
||||||
|
# same bright-cyan gradient button at nearly the same position -- confirmed
|
||||||
|
# by direct pixel sample, sharing one color range.
|
||||||
|
LESSON_ACTION_BUTTON_RGB = ((90, 195, 235), (150, 240, 255))
|
||||||
|
LESSON_INFO_START_BUTTON = (960, 890)
|
||||||
|
LESSON_INFO_CLOSE_BUTTON = (1435, 240)
|
||||||
|
|
||||||
|
# After clicking Start, a variable sequence of intermediate screens can
|
||||||
|
# appear before settling back on the grid modal -- a bond-rank-up full-
|
||||||
|
# screen cutscene (confirmed live, see screenshots/lesson/ -- its art is
|
||||||
|
# character-dependent, so no fixed color/position reliably identifies it)
|
||||||
|
# and/or a "スケジュールレポート" results modal. A transient "run the
|
||||||
|
# schedule twice" campaign multiplier was also observed live, doubling how
|
||||||
|
# many of these screens appear in a row -- rather than special-case any of
|
||||||
|
# this, `lesson.py` presses Enter in a bounded loop and re-checks two fixed
|
||||||
|
# markers each round:
|
||||||
|
#
|
||||||
|
# - the grid modal's own title underline (this exact yellow-gold, confirmed
|
||||||
|
# live) is visible ONLY when the grid modal is frontmost and idle -- both
|
||||||
|
# the report modal and the cutscene cover it, confirmed live against all
|
||||||
|
# three states.
|
||||||
|
LESSON_GRID_IDLE_PROBE_RECT = (790, 244, 1130, 252)
|
||||||
|
LESSON_GRID_IDLE_RGB = ((235, 220, 70), (255, 250, 130))
|
||||||
|
# - the report modal's OK button -- same color as LESSON_ACTION_BUTTON_RGB,
|
||||||
|
# its own confirmed fixed position -- clicked directly rather than folded
|
||||||
|
# into the blind Enter-press fallback, since we can verify it precisely.
|
||||||
|
LESSON_REPORT_OK_BUTTON = (960, 895)
|
||||||
|
LESSON_POST_SCHEDULE_MAX_ENTER_PRESSES = 8
|
||||||
|
|||||||
@ -1,3 +1,162 @@
|
|||||||
"""Image/color matching helpers (OpenCV-based); not wired up yet."""
|
"""Image/color matching helpers (OpenCV-based). Ported from scripts/detect_and_click.py."""
|
||||||
|
import os
|
||||||
|
|
||||||
# TODO: migrate scripts/detect_and_click.py sparkle-matching logic here.
|
import cv2
|
||||||
|
import numpy as np
|
||||||
|
import pytesseract
|
||||||
|
|
||||||
|
from ba_auto import config, driver
|
||||||
|
|
||||||
|
OCR_SHOT_PATH = os.path.join(config.SCRATCHPAD_DIR, "ocr_live.png")
|
||||||
|
OCR_UPSCALE = 3
|
||||||
|
|
||||||
|
SPARKLE_SHOT_PATH = os.path.join(config.SCRATCHPAD_DIR, "cafe_live.png")
|
||||||
|
SPARKLE_CLICK_OFFSET = (75, 47)
|
||||||
|
SPARKLE_THRESHOLD = 0.97
|
||||||
|
|
||||||
|
# The cafe camera's zoom level isn't reset before farming, and the sparkle
|
||||||
|
# icon's on-screen size scales with it (see screenshots/cafe/sparkle/
|
||||||
|
# 01_sparke_zoomed_centered.png vs 03_sparkle_zoomed_out.png) -- a single
|
||||||
|
# fixed-scale template match misses whenever the camera isn't at the exact
|
||||||
|
# zoom the template was captured at. Try a spread of scales instead.
|
||||||
|
SPARKLE_SCALES = (0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2)
|
||||||
|
|
||||||
|
|
||||||
|
def _masked_template(template):
|
||||||
|
b, g, r = cv2.split(template.astype(np.int16))
|
||||||
|
yellow_white = ((r > 180) & (g > 140) & (r - b > 60)) | ((r > 200) & (g > 200) & (b > 200))
|
||||||
|
mask_plane = (yellow_white.astype(np.uint8)) * 255
|
||||||
|
return cv2.merge([mask_plane, mask_plane, mask_plane])
|
||||||
|
|
||||||
|
|
||||||
|
def find_cafe_sparkle():
|
||||||
|
driver.screenshot(SPARKLE_SHOT_PATH)
|
||||||
|
template_full = cv2.imread(config.CAFE_SPARKLE_TEMPLATE)
|
||||||
|
img = cv2.imread(SPARKLE_SHOT_PATH)
|
||||||
|
th0, tw0 = template_full.shape[:2]
|
||||||
|
|
||||||
|
best = None
|
||||||
|
for scale in SPARKLE_SCALES:
|
||||||
|
tw, th = max(1, round(tw0 * scale)), max(1, round(th0 * scale))
|
||||||
|
template = cv2.resize(template_full, (tw, th))
|
||||||
|
mask = _masked_template(template)
|
||||||
|
|
||||||
|
result = cv2.matchTemplate(img, template, cv2.TM_CCORR_NORMED, mask=mask)
|
||||||
|
_, max_val, _, max_loc = cv2.minMaxLoc(result)
|
||||||
|
if max_val >= SPARKLE_THRESHOLD and (best is None or max_val > best[0]):
|
||||||
|
best = (max_val, max_loc[0], max_loc[1], tw, th, scale)
|
||||||
|
|
||||||
|
if best is None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
score, x, y, tw, th, scale = best
|
||||||
|
ox, oy = SPARKLE_CLICK_OFFSET
|
||||||
|
return (x + tw // 2 + round(ox * scale), y + th // 2 + round(oy * scale), score)
|
||||||
|
|
||||||
|
|
||||||
|
def _color_mask(region, rgb_min, rgb_max):
|
||||||
|
x1, y1, x2, y2 = region
|
||||||
|
driver.screenshot(OCR_SHOT_PATH)
|
||||||
|
img = cv2.imread(OCR_SHOT_PATH)
|
||||||
|
crop = img[y1:y2, x1:x2]
|
||||||
|
b, g, r = crop[:, :, 0].astype(np.int16), crop[:, :, 1].astype(np.int16), crop[:, :, 2].astype(np.int16)
|
||||||
|
(r_lo, g_lo, b_lo), (r_hi, g_hi, b_hi) = rgb_min, rgb_max
|
||||||
|
return (r >= r_lo) & (r <= r_hi) & (g >= g_lo) & (g <= g_hi) & (b >= b_lo) & (b <= b_hi)
|
||||||
|
|
||||||
|
|
||||||
|
def region_contains_color(region, rgb_min, rgb_max):
|
||||||
|
"""Whether any pixel within `region` (x1, y1, x2, y2) falls in the given
|
||||||
|
RGB range. Useful for presence checks on small, non-convex glyphs (e.g.
|
||||||
|
an arrow chevron) where a single fixed-point probe can land in the
|
||||||
|
glyph's own concave gap -- confirmed live: a centroid-based single point
|
||||||
|
for story_sweep's region-arrow chevron fell squarely in the notch
|
||||||
|
between its two strokes, reading as "absent" even while the arrow was
|
||||||
|
clearly rendered a few pixels away. See plan.md Phase 10.
|
||||||
|
"""
|
||||||
|
return bool(_color_mask(region, rgb_min, rgb_max).any())
|
||||||
|
|
||||||
|
|
||||||
|
def find_color_centroid(region, rgb_min, rgb_max):
|
||||||
|
"""Centroid `(x, y)` of every pixel within `region` (x1, y1, x2, y2)
|
||||||
|
falling in the given RGB range, or None if none match. Useful for
|
||||||
|
clicking a known-colored button whose exact position varies between
|
||||||
|
otherwise-similar dialogs -- e.g. story_sweep's sweep-result screen
|
||||||
|
shows the same cyan confirm-button color for its "SKIP" and final "OK"
|
||||||
|
states, ~120px apart vertically; finding it by color avoids hardcoding
|
||||||
|
both positions. See plan.md Phase 10.
|
||||||
|
"""
|
||||||
|
x1, y1, x2, y2 = region
|
||||||
|
mask = _color_mask(region, rgb_min, rgb_max)
|
||||||
|
ys, xs = np.nonzero(mask)
|
||||||
|
if len(xs) == 0:
|
||||||
|
return None
|
||||||
|
return (x1 + int(xs.mean()), y1 + int(ys.mean()))
|
||||||
|
|
||||||
|
|
||||||
|
def _ocr_crop(region):
|
||||||
|
x1, y1, x2, y2 = region
|
||||||
|
driver.screenshot(OCR_SHOT_PATH)
|
||||||
|
img = cv2.imread(OCR_SHOT_PATH)
|
||||||
|
crop = img[y1:y2, x1:x2]
|
||||||
|
gray = cv2.cvtColor(crop, cv2.COLOR_BGR2GRAY)
|
||||||
|
# This UI's text is consistently dark-on-light -- a hard black/white
|
||||||
|
# threshold measurably fixed real misreads during live calibration
|
||||||
|
# (e.g. a stage label's digit misread as a stray extra digit) that
|
||||||
|
# persisted across every psm mode until the anti-aliased grey edges were
|
||||||
|
# removed. Confirmed live against scratchpad/scroll_up20.png row labels.
|
||||||
|
_, thresh = cv2.threshold(gray, 150, 255, cv2.THRESH_BINARY)
|
||||||
|
return cv2.resize(thresh, None, fx=OCR_UPSCALE, fy=OCR_UPSCALE, interpolation=cv2.INTER_CUBIC)
|
||||||
|
|
||||||
|
|
||||||
|
def read_text(region, whitelist=None, psm=7, lang="eng"):
|
||||||
|
"""OCR a pixel rectangle `(x1, y1, x2, y2)` from a fresh screenshot.
|
||||||
|
|
||||||
|
Local equivalent of the reference's core/ocr/ocr.py Baas_ocr client
|
||||||
|
(get_region_res) -- without its socket/shared-memory server, which exists
|
||||||
|
there to make OCR fast across thousands of automation steps. This project
|
||||||
|
only needs occasional single-crop reads (a region number, a stage label),
|
||||||
|
so a plain in-process pytesseract call is enough; see CLAUDE.md's "OCR
|
||||||
|
policy" and Handoff.md. `lang="eng"` is sufficient for the digit/dash
|
||||||
|
labels this project reads (region numbers, "30-1"/"30-A" stage labels) --
|
||||||
|
no Japanese trained data is needed for those specific reads.
|
||||||
|
"""
|
||||||
|
crop = _ocr_crop(region)
|
||||||
|
tess_config = f"--psm {psm}"
|
||||||
|
if whitelist:
|
||||||
|
tess_config += f" -c tessedit_char_whitelist={whitelist}"
|
||||||
|
return pytesseract.image_to_string(crop, lang=lang, config=tess_config).strip()
|
||||||
|
|
||||||
|
|
||||||
|
def read_int(region, psm=7):
|
||||||
|
"""Local equivalent of the reference's recognize_int."""
|
||||||
|
digits = "".join(ch for ch in read_text(region, whitelist="0123456789", psm=psm) if ch.isdigit())
|
||||||
|
return int(digits) if digits else None
|
||||||
|
|
||||||
|
|
||||||
|
def read_int_on_heart_badge(region, psm=7):
|
||||||
|
"""OCR a small dark-navy digit rendered on lesson.py's pink/magenta
|
||||||
|
heart-shaped affection badge.
|
||||||
|
|
||||||
|
read_int's plain grayscale threshold (tuned for this UI's normal
|
||||||
|
dark-text-on-light-card look) misreads these: the heart's own outline
|
||||||
|
stroke is a darker, saturated magenta whose grayscale value happens to
|
||||||
|
fall on the same side of the threshold as the digit glyph, so it
|
||||||
|
survives as stray black marks tesseract sometimes fuses into extra
|
||||||
|
digits -- confirmed live, "13" -> "113", "19" -> "119". The digit color
|
||||||
|
is reliably R < G (navy/blue-grey) while every pink/magenta badge tone
|
||||||
|
sampled (fill and outline, light and dark) is R > G, so masking on that
|
||||||
|
channel relationship instead of raw brightness cleanly drops the badge
|
||||||
|
shape and keeps just the glyph.
|
||||||
|
"""
|
||||||
|
x1, y1, x2, y2 = region
|
||||||
|
driver.screenshot(OCR_SHOT_PATH)
|
||||||
|
img = cv2.imread(OCR_SHOT_PATH)
|
||||||
|
crop = img[y1:y2, x1:x2]
|
||||||
|
b, g, r = crop[:, :, 0].astype(np.int16), crop[:, :, 1].astype(np.int16), crop[:, :, 2].astype(np.int16)
|
||||||
|
ink = (r < g) & (np.maximum(np.maximum(r, g), b) < 170)
|
||||||
|
binary = np.where(ink, 0, 255).astype(np.uint8)
|
||||||
|
upscaled = cv2.resize(binary, None, fx=OCR_UPSCALE, fy=OCR_UPSCALE, interpolation=cv2.INTER_CUBIC)
|
||||||
|
tess_config = f"--psm {psm} -c tessedit_char_whitelist=0123456789"
|
||||||
|
text = pytesseract.image_to_string(upscaled, lang="eng", config=tess_config).strip()
|
||||||
|
digits = "".join(ch for ch in text if ch.isdigit())
|
||||||
|
return int(digits) if digits else None
|
||||||
|
|||||||
@ -1,4 +1,5 @@
|
|||||||
"""Local PC/Steam/Proton control backend (xdotool/scrot wrappers)."""
|
"""Local PC/Steam/Proton control backend (xdotool/scrot wrappers)."""
|
||||||
|
import os
|
||||||
import subprocess
|
import subprocess
|
||||||
import time
|
import time
|
||||||
|
|
||||||
@ -6,7 +7,7 @@ import cv2
|
|||||||
|
|
||||||
from ba_auto import config
|
from ba_auto import config
|
||||||
|
|
||||||
PROBE_SHOT_PATH = "/tmp/ba_auto_probe.png"
|
PROBE_SHOT_PATH = os.path.join(config.SCRATCHPAD_DIR, "probe.png")
|
||||||
|
|
||||||
|
|
||||||
def run_command(args, **kwargs):
|
def run_command(args, **kwargs):
|
||||||
@ -28,10 +29,33 @@ def focus_game():
|
|||||||
|
|
||||||
|
|
||||||
def click(x, y):
|
def click(x, y):
|
||||||
run_command(["xdotool", "mousemove", str(x), str(y), "click", "1"])
|
# A combined "mousemove X Y click 1" invocation is unreliable here --
|
||||||
|
# empirically (see plan.md's click-flakiness writeups) the game's input
|
||||||
|
# handler sometimes misses clicks fired before it's processed the move.
|
||||||
|
# Splitting into separate commands with a short pause between fixes it.
|
||||||
|
run_command(["xdotool", "mousemove", str(x), str(y)])
|
||||||
|
wait(0.2)
|
||||||
|
run_command(["xdotool", "click", "1"])
|
||||||
wait(0.5)
|
wait(0.5)
|
||||||
|
|
||||||
|
|
||||||
|
def move_mouse(x, y):
|
||||||
|
run_command(["xdotool", "mousemove", str(x), str(y)])
|
||||||
|
|
||||||
|
|
||||||
|
def scroll(x, y, direction, clicks=1):
|
||||||
|
# xdotool button 4/5 = scroll wheel up/down. A drag (mousedown/move/mouseup)
|
||||||
|
# does not register as a list-scroll gesture in this Proton client; the
|
||||||
|
# wheel does.
|
||||||
|
button = "4" if direction == "up" else "5"
|
||||||
|
run_command(["xdotool", "mousemove", str(x), str(y)])
|
||||||
|
wait(0.2)
|
||||||
|
for _ in range(clicks):
|
||||||
|
run_command(["xdotool", "click", button])
|
||||||
|
wait(0.15)
|
||||||
|
wait(0.3)
|
||||||
|
|
||||||
|
|
||||||
def keypress(key):
|
def keypress(key):
|
||||||
run_command(["xdotool", "key", key])
|
run_command(["xdotool", "key", key])
|
||||||
wait(0.5)
|
wait(0.5)
|
||||||
|
|||||||
@ -1,3 +1,56 @@
|
|||||||
"""Shared navigation helpers (home, menu, popups, back/escape); not wired up yet."""
|
"""Shared navigation helpers (home, menu, popups, back/escape)."""
|
||||||
|
|
||||||
# TODO: extract common navigation flows here as tasks are ported off Bash.
|
# The mailbox/cafe/shop-style header bar renders a plain light background
|
||||||
|
# here; the home screen shows character art instead.
|
||||||
|
SUBSCREEN_HEADER_PROBE = (500, 10)
|
||||||
|
SUBSCREEN_HEADER_MIN_CHANNEL = 200
|
||||||
|
|
||||||
|
# Any modal dialog dims the screen behind it to roughly this darkness.
|
||||||
|
MODAL_DIM_PROBE = (960, 200)
|
||||||
|
MODAL_DIM_MAX_CHANNEL = 150
|
||||||
|
|
||||||
|
|
||||||
|
def is_on_subscreen(driver):
|
||||||
|
r, g, b = driver.color_at(*SUBSCREEN_HEADER_PROBE)
|
||||||
|
return r > SUBSCREEN_HEADER_MIN_CHANNEL and g > SUBSCREEN_HEADER_MIN_CHANNEL and b > SUBSCREEN_HEADER_MIN_CHANNEL
|
||||||
|
|
||||||
|
|
||||||
|
def is_modal_open(driver):
|
||||||
|
r, g, b = driver.color_at(*MODAL_DIM_PROBE)
|
||||||
|
return r < MODAL_DIM_MAX_CHANNEL and g < MODAL_DIM_MAX_CHANNEL and b < MODAL_DIM_MAX_CHANNEL
|
||||||
|
|
||||||
|
|
||||||
|
def wait_for_state(driver, config, reactions, ends, max_iterations=30, poll_interval=1.0):
|
||||||
|
"""Generic "watch the screen, react to anything recognized, stop once a
|
||||||
|
recognized destination is reached" loop -- the local equivalent of the
|
||||||
|
reference's core/picture.py::co_detect, scoped to what this project
|
||||||
|
actually needs (a handful of named checks) rather than co_detect's full
|
||||||
|
generality (which spans the whole reference project via ~20 image
|
||||||
|
template assets this project doesn't have).
|
||||||
|
|
||||||
|
`ends`: {check_fn(driver, config) -> bool: outcome_name}. Checked first,
|
||||||
|
every iteration; the first match stops the loop and returns its name.
|
||||||
|
|
||||||
|
`reactions`: {check_fn(driver, config) -> bool: action(driver)}. Checked
|
||||||
|
if no end matched; the first match runs its action (a click, a keypress,
|
||||||
|
whatever the recognized state calls for) and the loop continues.
|
||||||
|
|
||||||
|
If neither an end nor a reaction matches, the loop just waits and retries
|
||||||
|
-- it never falls back to a blind click/keypress guess (see CLAUDE.md's
|
||||||
|
exit-game-dialog writeup for why that was a real bug elsewhere).
|
||||||
|
|
||||||
|
Returns the matched end's outcome name, or None once max_iterations is
|
||||||
|
exhausted without reaching a recognized end -- callers should treat None
|
||||||
|
as "unrecognized state, abort safely."
|
||||||
|
"""
|
||||||
|
for _ in range(max_iterations):
|
||||||
|
for check_fn, outcome_name in ends.items():
|
||||||
|
if check_fn(driver, config):
|
||||||
|
return outcome_name
|
||||||
|
for check_fn, action in reactions.items():
|
||||||
|
if check_fn(driver, config):
|
||||||
|
action(driver)
|
||||||
|
break
|
||||||
|
else:
|
||||||
|
driver.wait(poll_interval)
|
||||||
|
return None
|
||||||
|
|||||||
@ -5,14 +5,15 @@ Maps each local feature to the corresponding `~/repo/baas-reference/module/...`
|
|||||||
| Local feature | Reference file | Reference functions/classes | Local file | Backend replacements | Status |
|
| Local feature | Reference file | Reference functions/classes | Local file | Backend replacements | Status |
|
||||||
|---|---|---|---|---|---|
|
|---|---|---|---|---|---|
|
||||||
| Mailbox | `module/mail.py` | `to_mail`, `implement` | `ba_auto/tasks/mailbox.py` | tap/click via xdotool, screenshot via scrot, `color.rgb_in_range` → `driver.color_at` pixel-probe check | Migrated: real Python, state-verified via color probe (no legacy bridge) |
|
| Mailbox | `module/mail.py` | `to_mail`, `implement` | `ba_auto/tasks/mailbox.py` | tap/click via xdotool, screenshot via scrot, `color.rgb_in_range` → `driver.color_at` pixel-probe check | Migrated: real Python, state-verified via color probe (no legacy bridge) |
|
||||||
| Cafe | Need to confirm in reference | Need to inspect | `ba_auto/tasks/cafe.py` | template matching via OpenCV, click via xdotool | Bridged to `scripts/ba_dailies_legacy.sh` + `scripts/detect_and_click.py`; Python port pending (plan.md Phase 6) |
|
| Cafe | `module/cafe_reward.py` | `to_cafe` (its `relationship_rank_up` popup-handling now also ported, see below), `interaction_for_cafe_solve_method3`, `collect` | `ba_auto/tasks/cafe.py` | `picture.co_detect`/`color.rgb_in_range` → `driver.color_at` pixel-probe checks; sparkle template match ported in-process into `ba_auto/detector.py` (`find_cafe_sparkle`, now multi-scale) | Migrated: real Python, state-verified via color probes (no legacy bridge). Pat loop now polls for the full attempt budget instead of stopping on the first miss (see `plan.md` Phase 6 follow-up) — not yet confirmed against a live sparkle since none was available during testing. `_dismiss_rank_up_if_shown` reuses `navigation.is_on_subscreen` to detect and clear the full-screen bond-rank-up cutscene after a pat (see `plan.md` Phase 6 follow-up: rank-up popups) — not yet live-confirmed against a real trigger |
|
||||||
| Stamina/AP | `module/collect_daily_free_power.py`, `module/collect_daily_task_power.py` | Need to inspect | `ba_auto/tasks/stamina.py` | color checks/clicks via local driver | Not started |
|
| Stamina/AP | `module/collect_daily_free_power.py`, `module/collect_daily_task_power.py` | `to_tasks`/`implement` (task-power, ported); `to_purchase_pyroxenes_menu` (free-power, not ported) | `ba_auto/tasks/stamina.py` | `color.rgb_in_range` → `driver.color_at`; reference's per-tab claim loop → live UI's single "一括受取" bulk-claim button + Enter | Partially migrated: Mission-panel claim done (see `plan.md` Phase 8). Daily Free Power (real-money purchase menu) deliberately not automated |
|
||||||
|
| Normal/Hard story AP sweep | `module/explore_tasks/sweep_task.py`, `module/explore_tasks/task_utils.py` | `to_region` (ported: OCR region-number readout + delta-click), a scoped-down `swipe_search_target_str` (ported: OCR stage-row label matching), `start_sweep`'s named-outcome contract (ported via `navigation.wait_for_state`, this project's scoped `co_detect` port) | `ba_auto/tasks/story_sweep.py` | OCR region/stage-name matching, ported for real (Phase 10) — replaces Phase 9's "next-region arrow stops advancing, then random stage" heuristic; MAX click verified via `SWEEP_MINUS_BUTTON_PROBE` color check (reused, still correct), modal closed via its own X button (Escape doesn't close it; X-button position re-calibrated per stage-layout variant, see Phase 10) | Done (see `plan.md` Phase 10, supersedes Phase 9). Config-driven exact `(region, stage, count)` targets (`config.STORY_SWEEP_TARGETS`), not latest-region/random-stage. Opt-in only, not in default flow |
|
||||||
| Group/Club AP | `module/group.py` | Need to inspect | `ba_auto/tasks/group.py` | fixed click + state check via local driver | Not started |
|
| Group/Club AP | `module/group.py` | Need to inspect | `ba_auto/tasks/group.py` | fixed click + state check via local driver | Not started |
|
||||||
| Bounty | `module/rewarded_task.py` | Need to inspect | `ba_auto/tasks/bounty.py` | sweep/color/OCR adaptation | Not started |
|
| Bounty | `module/rewarded_task.py` | Need to inspect | `ba_auto/tasks/bounty.py` | sweep/color/OCR adaptation | Not started |
|
||||||
| Commissions | `module/clear_special_task_power.py` | Need to inspect | `ba_auto/tasks/commission.py` | sweep/color adaptation | Not started |
|
| Commissions | `module/clear_special_task_power.py` | Need to inspect | `ba_auto/tasks/commission.py` | sweep/color adaptation | Not started |
|
||||||
| Arena | `module/arena.py` | Need to inspect | `ba_auto/tasks/arena.py` | auto-fight + OCR + local driver | Not started |
|
| Arena | `module/arena.py` | Need to inspect | `ba_auto/tasks/arena.py` | auto-fight + OCR + local driver | Not started |
|
||||||
| Common Shop | `module/shop/common_shop.py`, `module/shop/shop_utils.py` | Need to inspect | `ba_auto/tasks/shop_common.py` | OCR + tab navigation + local clicks | Not started |
|
| Common Shop | `module/shop/common_shop.py`, `module/shop/shop_utils.py` | `implement`, `to_common_shop`, `get_item_position`/`ensure_choose`/`buy` (shared, see Tactical Shop row) | `ba_auto/tasks/shop_common.py`, `ba_auto/tasks/shop_utils.py` | `get_item_position`'s color+template item-state scan → fixed grid-position targets (`config.COMMON_SHOP_TARGETS`) + price-digit OCR verify, since the reference's own item-identification here indexes an external static price table (`self.static_config.common_shop_price_list`, fetched from a remote resource) this repo doesn't have — not per-item OCR, so this isn't an OCR-avoidance shortcut. Purchase-confirm dialog + reward-acquired banner handled via a single overlay-darkness probe (`config.SHOP_OVERLAY_PROBE`) instead of tracking each dialog's own layout | Done. Live-tested with real purchases (all 8 configured targets bought, cost matched exactly). Discovered live: these items have a per-refresh-cycle purchase cap not shown as a visible counter (unlike the 青輝石 tab's "あと1回購入可能" labels) — confirmed by re-running the task after purchase and observing it correctly detect the now-unselectable items (checkbox + individual 購入 button both unresponsive) and safely decline rather than guess. A fresh, everything-available run hasn't been re-verified since the account had already exhausted this cycle's purchases via that same test |
|
||||||
| Tactical Shop | `module/shop/tactical_challenge_shop.py`, `module/shop/shop_utils.py` | Need to inspect | `ba_auto/tasks/shop_tactical.py` | OCR + tab navigation + local clicks | Not started |
|
| Tactical Shop | `module/shop/tactical_challenge_shop.py`, `module/shop/shop_utils.py` | `implement`, `goto_shop_by_name`, shared `get_item_position`/`ensure_choose`/`buy` | `ba_auto/tasks/shop_tactical.py`, `ba_auto/tasks/shop_utils.py` | `goto_shop_by_name`'s OCR swipe-search over the shop-tab list → fixed click (`config.SHOP_TAB_TACTICAL`): this account's tab list is only 7 entries and fits on screen with no scroll needed, confirmed live, so there's nothing to search for — not an OCR-avoidance shortcut. Same grid-position + price-OCR-verify + overlay-probe design as Common Shop, sharing `shop_utils.run_shop_tab` | Done. Live-tested with real purchases (both configured AP-recovery drinks bought; AP and tactical-coin balance changes matched exactly) |
|
||||||
| Lesson/Schedule | `module/lesson.py` | Need to inspect | `ba_auto/tasks/lesson.py` | OCR + template/portrait search + local driver | Not started |
|
| Lesson/Schedule | `module/lesson.py` | `implement`, `to_lesson_location_select`/`to_select_location`/`to_all_locations` (nav state machine), `get_lesson_region_num`/`switch_lesson_region_page`/`to_lesson_region` (paged region nav), `get_lesson_each_region_status`+`check_region_availability` (per-cell status via isometric-parallelogram pixel scan), `get_lesson_relationship_counts` (per-cell affection pip count via color count), `choose_lesson` (selection policy), `execute_lesson`/`to_location_info`/`start_lesson` (click cell -> info panel -> start -> result) | `ba_auto/tasks/lesson.py` | `picture.co_detect` -> `navigation.wait_for_state`-style bounded Enter-press loop (see below); the reference's paged-arrow region nav (needing OCR to know current position) -> this client renders the 12 regions as a scrollable list instead, which only ever settles at two scroll positions (`config.LESSON_REGION_ROW_Y`), so navigation is direct index-based clicking with nothing to OCR-locate; the reference's isometric `Parallelogram`/`Triangle` per-cell scan (tuned to the reference's own screen layout) -> reading each portrait's heart-shaped affection badge via a dedicated OCR path (`detector.read_int_on_heart_badge`) needs no isometric geometry at all | Done. Config-driven scope only in the sense of the *policy* (affection-first selection, sweep every unlocked region until tickets/lessons run out, no ticket purchasing, no favor-student targeting) -- unlike shop, no user-specific target list was needed since the reference's own `lesson_region_name.JP` (embedded directly in its `default_config.py`, not externally fetched) already names all 12 regions, used here only for logging. Live-tested for real: 5 real tickets spent across 3 regions with correct outcomes (ticket count, cleanup navigation, home-screen return all verified). Two real bugs were found and fixed from that run -- see below and `plan.md`'s Lesson phase |
|
||||||
|
|
||||||
Do not implement a feature without filling at least the relevant row.
|
Do not implement a feature without filling at least the relevant row.
|
||||||
|
|||||||
@ -1,6 +1,124 @@
|
|||||||
"""Cafe daily task."""
|
"""Cafe daily task. Ported from baas-reference module/cafe_reward.py's state-probe pattern."""
|
||||||
|
|
||||||
|
from ba_auto import detector, navigation
|
||||||
|
|
||||||
|
ROOM_OPEN_RETRIES = 3
|
||||||
|
|
||||||
|
# "受取" (claim) renders as flat grey when there is nothing to collect yet.
|
||||||
|
CLAIM_PROBE = (960, 850)
|
||||||
|
CLAIM_DISABLED_RGB = (218, 218, 218)
|
||||||
|
CLAIM_DISABLED_TOLERANCE = 15
|
||||||
|
|
||||||
|
|
||||||
|
def _claim_disabled(driver):
|
||||||
|
r, g, b = driver.color_at(*CLAIM_PROBE)
|
||||||
|
tr, tg, tb = CLAIM_DISABLED_RGB
|
||||||
|
return (
|
||||||
|
abs(r - tr) <= CLAIM_DISABLED_TOLERANCE
|
||||||
|
and abs(g - tg) <= CLAIM_DISABLED_TOLERANCE
|
||||||
|
and abs(b - tb) <= CLAIM_DISABLED_TOLERANCE
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _enter_room(driver, coords):
|
||||||
|
for attempt in range(1, ROOM_OPEN_RETRIES + 1):
|
||||||
|
driver.click(*coords)
|
||||||
|
driver.wait(3)
|
||||||
|
if navigation.is_on_subscreen(driver):
|
||||||
|
# dismiss the "visited student list" notice shown on room entry
|
||||||
|
driver.keypress("Return")
|
||||||
|
driver.wait(1)
|
||||||
|
return True
|
||||||
|
print(f"[cafe] room not detected after click (attempt {attempt}/{ROOM_OPEN_RETRIES})")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _dismiss_rank_up_if_shown(driver, config):
|
||||||
|
# The reference's own to_cafe() navigation (module/cafe_reward.py) treats
|
||||||
|
# 'relationship_rank_up' as a recognized, reactively-dismissed popup
|
||||||
|
# after every pat round -- this loop's original port had no equivalent,
|
||||||
|
# so a rank-up cutscene just sat there while find_cafe_sparkle() kept
|
||||||
|
# returning None against it (a full-screen character portrait, nothing
|
||||||
|
# like the sparkle template) for the rest of the room's click budget.
|
||||||
|
# That's the "freeze" -- not a timing fluke, a genuinely unhandled state.
|
||||||
|
for _ in range(config.CAFE_RANK_UP_DISMISS_RETRIES):
|
||||||
|
if navigation.is_on_subscreen(driver):
|
||||||
|
return True
|
||||||
|
driver.keypress("Return")
|
||||||
|
driver.wait(1.5)
|
||||||
|
return navigation.is_on_subscreen(driver)
|
||||||
|
|
||||||
|
|
||||||
|
def _pat_room(driver, config):
|
||||||
|
# Sparkles appear on a per-student cooldown, so most single checks find
|
||||||
|
# nothing -- the old Bash loop (and an earlier version of this one) gave
|
||||||
|
# up on the very first miss, which meant it essentially never farmed.
|
||||||
|
# Keep polling for the full budget instead of bailing early.
|
||||||
|
patted = 0
|
||||||
|
for _ in range(config.CAFE_MAX_CLICKS_PER_ROOM):
|
||||||
|
match = detector.find_cafe_sparkle()
|
||||||
|
if match is None:
|
||||||
|
driver.wait(1)
|
||||||
|
continue
|
||||||
|
x, y, score = match
|
||||||
|
driver.click(x, y)
|
||||||
|
driver.wait(1)
|
||||||
|
driver.keypress("Return")
|
||||||
|
# park the cursor away from the sparkle area so it can't occlude the
|
||||||
|
# next detection screenshot (see screenshots/cafe/sparkle/02_*_cursor_on_head.png)
|
||||||
|
driver.move_mouse(10, 1190)
|
||||||
|
if not _dismiss_rank_up_if_shown(driver, config):
|
||||||
|
print("[cafe] warning: cafe screen not confirmed after a pat (rank-up cutscene stuck?) -- stopping this room's pat loop rather than clicking blindly")
|
||||||
|
break
|
||||||
|
patted += 1
|
||||||
|
print(f"[cafe] patted sparkle at ({x}, {y}), score={score:.3f}")
|
||||||
|
print(f"[cafe] patted {patted} sparkle(s)" if patted else "[cafe] no sparkle found")
|
||||||
|
|
||||||
|
|
||||||
|
def _claim_income(driver, config):
|
||||||
|
driver.click(*config.CAFE_INCOME)
|
||||||
|
driver.wait(2)
|
||||||
|
if not navigation.is_modal_open(driver):
|
||||||
|
print("[cafe] income panel not detected, skipping claim")
|
||||||
|
return
|
||||||
|
|
||||||
|
if _claim_disabled(driver):
|
||||||
|
print("[cafe] nothing to claim")
|
||||||
|
else:
|
||||||
|
print("[cafe] claiming income")
|
||||||
|
driver.keypress("Return")
|
||||||
|
driver.wait(2)
|
||||||
|
driver.keypress("Return")
|
||||||
|
driver.wait(2)
|
||||||
|
|
||||||
|
if navigation.is_modal_open(driver):
|
||||||
|
driver.keypress("Escape")
|
||||||
|
driver.wait(1.5)
|
||||||
|
|
||||||
|
|
||||||
def run(driver, config):
|
def run(driver, config):
|
||||||
# TODO: migrate old Bash cafe logic into this Python task.
|
driver.focus_game()
|
||||||
driver.run_command([config.LEGACY_SCRIPT, "cafe"])
|
|
||||||
|
if not _enter_room(driver, config.CAFE_ICON):
|
||||||
|
print("[cafe] could not confirm cafe is open, aborting without pressing further keys")
|
||||||
|
return
|
||||||
|
|
||||||
|
print("[cafe] room 1: farming affection")
|
||||||
|
_pat_room(driver, config)
|
||||||
|
|
||||||
|
if not _enter_room(driver, config.CAFE_ROOM_SWITCH):
|
||||||
|
print("[cafe] could not confirm room switch, stopping before income claim")
|
||||||
|
if navigation.is_on_subscreen(driver):
|
||||||
|
driver.keypress("Escape")
|
||||||
|
driver.wait(1.5)
|
||||||
|
return
|
||||||
|
|
||||||
|
print("[cafe] room 2: farming affection")
|
||||||
|
_pat_room(driver, config)
|
||||||
|
|
||||||
|
_claim_income(driver, config)
|
||||||
|
|
||||||
|
if navigation.is_on_subscreen(driver):
|
||||||
|
driver.keypress("Escape")
|
||||||
|
driver.wait(1.5)
|
||||||
|
print("[cafe] Done.")
|
||||||
|
|||||||
247
ba_auto/tasks/lesson.py
Normal file
247
ba_auto/tasks/lesson.py
Normal file
@ -0,0 +1,247 @@
|
|||||||
|
"""Lesson/Schedule. Reference: baas-reference/module/lesson.py.
|
||||||
|
|
||||||
|
Scoped for v1 per plan.md's "Suggested first version" and explicit user
|
||||||
|
direction: affection-first selection (mirrors the reference's
|
||||||
|
lesson_relationship_first=True), sweep every unlocked region in a fixed
|
||||||
|
order until either lesson tickets or scoreable lessons run out, no
|
||||||
|
lesson-ticket purchasing (same real-currency-adjacent caution as Daily Free
|
||||||
|
Power / the shop's manual refresh button -- see CLAUDE.md), no favor-student
|
||||||
|
targeting (deferred, matching plan.md).
|
||||||
|
|
||||||
|
This client renders the reference's paged single-region view + isometric
|
||||||
|
3x3 status grid (get_lesson_each_region_status/get_lesson_relationship_counts,
|
||||||
|
built on an isometric Parallelogram/Triangle pixel scan tuned to the
|
||||||
|
reference's own screen layout) as a two-level UI instead: a scrollable list
|
||||||
|
of the same 12 named regions (config.LESSON_REGION_NAMES, taken directly
|
||||||
|
from the reference's own lesson_region_name.JP list -- used here only for
|
||||||
|
logging, since unlike the reference's paged arrows this list's scroll
|
||||||
|
position is deterministic and needs no OCR to locate), each opening a grid
|
||||||
|
modal of up to 9 location cards. Each card shows up to 3 student portraits
|
||||||
|
with a heart-shaped affection-count badge -- reading those via OCR (this
|
||||||
|
project's local equivalent of the reference's pip-counting
|
||||||
|
get_lesson_relationship_counts) needs no isometric geometry at all. A locked
|
||||||
|
location isn't listed in the grid modal at all, so there's nothing to do
|
||||||
|
there; a "no relationship yet" portrait has no badge and its OCR read comes
|
||||||
|
back empty, scoring as a non-candidate. An "already done today" portrait
|
||||||
|
was initially assumed to also blank its badge, but live testing showed that
|
||||||
|
assumption was wrong: it keeps showing its (unchanged) number and gets a
|
||||||
|
green checkmark added at top-right instead -- so done-ness is checked
|
||||||
|
separately via that checkmark's color, not inferred from the OCR read.
|
||||||
|
|
||||||
|
Reading the badge itself also needed its own OCR path
|
||||||
|
(detector.read_int_on_heart_badge): a plain grayscale-threshold read (this
|
||||||
|
project's usual approach) misreads it, because the heart's own outline
|
||||||
|
stroke happens to survive the same threshold as the digit glyph and
|
||||||
|
tesseract sometimes fuses the two into extra digits (see that function's
|
||||||
|
docstring, and config.py's LESSON_GRID_BADGE_MAX_PLAUSIBLE for the
|
||||||
|
plausibility-cap backstop this project keeps as a second line of defense).
|
||||||
|
"""
|
||||||
|
from ba_auto import detector, navigation
|
||||||
|
|
||||||
|
OPEN_RETRIES = 3
|
||||||
|
REGIONS_PER_SCREEN = 6
|
||||||
|
TOTAL_REGIONS = 12
|
||||||
|
GRID_ROWS = 3
|
||||||
|
GRID_COLS = 3
|
||||||
|
GRID_SLOTS = 3
|
||||||
|
|
||||||
|
|
||||||
|
def _read_ticket_count(driver, config):
|
||||||
|
text = detector.read_text(config.LESSON_TICKET_OCR_RECT, whitelist="0123456789/", psm=7)
|
||||||
|
head = text.split("/")[0] if "/" in text else text
|
||||||
|
digits = "".join(ch for ch in head if ch.isdigit())
|
||||||
|
return int(digits) if digits else None
|
||||||
|
|
||||||
|
|
||||||
|
def _open_schedule_screen(driver, config):
|
||||||
|
for attempt in range(1, OPEN_RETRIES + 1):
|
||||||
|
driver.click(*config.LESSON_ICON)
|
||||||
|
driver.wait(2)
|
||||||
|
if navigation.is_on_subscreen(driver):
|
||||||
|
return True
|
||||||
|
print(f"[lesson] schedule screen not detected after click (attempt {attempt}/{OPEN_RETRIES})")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _scroll_region_list(driver, config, to_bottom):
|
||||||
|
x, y = config.LESSON_REGION_LIST_SCROLL_POINT
|
||||||
|
direction = "down" if to_bottom else "up"
|
||||||
|
driver.scroll(x, y, direction, config.LESSON_REGION_LIST_SCROLL_CLICKS)
|
||||||
|
driver.wait(0.8)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_grid_idle(driver, config):
|
||||||
|
lo, hi = config.LESSON_GRID_IDLE_RGB
|
||||||
|
return detector.region_contains_color(config.LESSON_GRID_IDLE_PROBE_RECT, lo, hi)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_action_button_showing(driver, config, position):
|
||||||
|
r, g, b = driver.color_at(*position)
|
||||||
|
lo, hi = config.LESSON_ACTION_BUTTON_RGB
|
||||||
|
return lo[0] <= r <= hi[0] and lo[1] <= g <= hi[1] and lo[2] <= b <= hi[2]
|
||||||
|
|
||||||
|
|
||||||
|
def _open_region_grid(driver, config, region_index):
|
||||||
|
# Retries the whole click-row -> click-all-schedules sequence from
|
||||||
|
# scratch rather than trying to separately verify "did the isometric map
|
||||||
|
# open" -- that screen shares the same bright header as the plain list
|
||||||
|
# (navigation.is_on_subscreen can't tell them apart), so the grid modal
|
||||||
|
# actually opening is the only reliable signal either step worked.
|
||||||
|
screen_bottom = region_index >= REGIONS_PER_SCREEN
|
||||||
|
row = region_index % REGIONS_PER_SCREEN
|
||||||
|
for attempt in range(1, OPEN_RETRIES + 1):
|
||||||
|
_scroll_region_list(driver, config, to_bottom=screen_bottom)
|
||||||
|
driver.click(config.LESSON_REGION_ROW_X, config.LESSON_REGION_ROW_Y[row])
|
||||||
|
driver.wait(1.5)
|
||||||
|
driver.click(*config.LESSON_ALL_SCHEDULES_BUTTON)
|
||||||
|
driver.wait(1.2)
|
||||||
|
if _is_grid_idle(driver, config):
|
||||||
|
return True
|
||||||
|
print(f"[lesson] schedule grid not detected for region index {region_index} (attempt {attempt}/{OPEN_RETRIES})")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _close_grid_modal(driver, config):
|
||||||
|
driver.click(*config.LESSON_GRID_MODAL_CLOSE_BUTTON)
|
||||||
|
driver.wait(1)
|
||||||
|
|
||||||
|
|
||||||
|
def _slot_center(config, row, col, slot):
|
||||||
|
cx = config.LESSON_GRID_COL_X[col] + slot * config.LESSON_GRID_PORTRAIT_STEP_X
|
||||||
|
cy = config.LESSON_GRID_ROW_PORTRAIT_Y[row]
|
||||||
|
return cx, cy
|
||||||
|
|
||||||
|
|
||||||
|
def _badge_rect(config, row, col, slot):
|
||||||
|
cx, cy = _slot_center(config, row, col, slot)
|
||||||
|
ox, oy = config.LESSON_GRID_BADGE_OFFSET
|
||||||
|
hx, hy = config.LESSON_GRID_BADGE_OCR_HALF_SIZE
|
||||||
|
cx, cy = cx + ox, cy + oy
|
||||||
|
return (cx - hx, cy - hy, cx + hx, cy + hy)
|
||||||
|
|
||||||
|
|
||||||
|
def _checkmark_rect(config, row, col, slot):
|
||||||
|
cx, cy = _slot_center(config, row, col, slot)
|
||||||
|
ox, oy = config.LESSON_GRID_CHECKMARK_OFFSET
|
||||||
|
hx, hy = config.LESSON_GRID_CHECKMARK_HALF_SIZE
|
||||||
|
cx, cy = cx + ox, cy + oy
|
||||||
|
return (cx - hx, cy - hy, cx + hx, cy + hy)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_slot_already_done(driver, config, row, col, slot):
|
||||||
|
lo, hi = config.LESSON_GRID_CHECKMARK_RGB
|
||||||
|
return detector.region_contains_color(_checkmark_rect(config, row, col, slot), lo, hi)
|
||||||
|
|
||||||
|
|
||||||
|
def _read_slot_affection(driver, config, row, col, slot):
|
||||||
|
if _is_slot_already_done(driver, config, row, col, slot):
|
||||||
|
return None
|
||||||
|
value = detector.read_int_on_heart_badge(_badge_rect(config, row, col, slot))
|
||||||
|
if value is not None and value > config.LESSON_GRID_BADGE_MAX_PLAUSIBLE:
|
||||||
|
# Contamination from portrait art bleeding into the crop's edge,
|
||||||
|
# not a real affection value -- see config.py's comment.
|
||||||
|
return None
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _find_best_cell(driver, config):
|
||||||
|
best_score, best_cell = -1, None
|
||||||
|
for row in range(GRID_ROWS):
|
||||||
|
for col in range(GRID_COLS):
|
||||||
|
cell_score = -1
|
||||||
|
for slot in range(GRID_SLOTS):
|
||||||
|
value = _read_slot_affection(driver, config, row, col, slot)
|
||||||
|
if value is not None and value > cell_score:
|
||||||
|
cell_score = value
|
||||||
|
if cell_score > best_score:
|
||||||
|
best_score, best_cell = cell_score, (row, col)
|
||||||
|
return best_cell, best_score
|
||||||
|
|
||||||
|
|
||||||
|
def _click_cell(driver, config, row, col):
|
||||||
|
driver.click(config.LESSON_GRID_COL_X[col], config.LESSON_GRID_ROW_HEADER_Y[row])
|
||||||
|
driver.wait(1.2)
|
||||||
|
|
||||||
|
|
||||||
|
def _run_one_schedule(driver, config, row, col):
|
||||||
|
"""Click a grid cell through to a completed schedule. Returns True once
|
||||||
|
settled back at the idle grid modal, False if it never got there."""
|
||||||
|
_click_cell(driver, config, row, col)
|
||||||
|
if not _is_action_button_showing(driver, config, config.LESSON_INFO_START_BUTTON):
|
||||||
|
print("[lesson] info panel start button not detected, aborting this cell without pressing further keys")
|
||||||
|
return False
|
||||||
|
|
||||||
|
driver.click(*config.LESSON_INFO_START_BUTTON)
|
||||||
|
driver.wait(1.5)
|
||||||
|
|
||||||
|
# A variable number of intermediate screens can follow (a bond-rank-up
|
||||||
|
# cutscene, a results modal, occasionally both twice under a "2x
|
||||||
|
# schedule" campaign multiplier -- see config.py's comment). Neither
|
||||||
|
# intermediate screen has a reliable fixed marker of its own except the
|
||||||
|
# results modal's OK button, so anything that isn't "back at the idle
|
||||||
|
# grid" or "OK button showing" gets a plain Enter press, bounded.
|
||||||
|
for _ in range(config.LESSON_POST_SCHEDULE_MAX_ENTER_PRESSES):
|
||||||
|
if _is_grid_idle(driver, config):
|
||||||
|
return True
|
||||||
|
if _is_action_button_showing(driver, config, config.LESSON_REPORT_OK_BUTTON):
|
||||||
|
driver.click(*config.LESSON_REPORT_OK_BUTTON)
|
||||||
|
else:
|
||||||
|
driver.keypress("Return")
|
||||||
|
driver.wait(1.5)
|
||||||
|
|
||||||
|
return _is_grid_idle(driver, config)
|
||||||
|
|
||||||
|
|
||||||
|
def _sweep_region(driver, config, region_index, remaining_tickets):
|
||||||
|
name = config.LESSON_REGION_NAMES[region_index]
|
||||||
|
|
||||||
|
if not _open_region_grid(driver, config, region_index):
|
||||||
|
print(f"[lesson] could not confirm schedule grid opened for {name}, skipping")
|
||||||
|
return remaining_tickets
|
||||||
|
|
||||||
|
while remaining_tickets > 0:
|
||||||
|
cell, score = _find_best_cell(driver, config)
|
||||||
|
if cell is None:
|
||||||
|
print(f"[lesson] no schedulable lesson found in {name}")
|
||||||
|
break
|
||||||
|
row, col = cell
|
||||||
|
print(f"[lesson] {name}: best available affection {score} at row {row} col {col}")
|
||||||
|
if not _run_one_schedule(driver, config, row, col):
|
||||||
|
print(f"[lesson] warning: could not confirm return to the schedule grid after {name} row {row} col {col} -- stopping this region")
|
||||||
|
break
|
||||||
|
new_count = _read_ticket_count(driver, config)
|
||||||
|
remaining_tickets = new_count if new_count is not None else remaining_tickets - 1
|
||||||
|
print(f"[lesson] tickets remaining: {remaining_tickets}")
|
||||||
|
|
||||||
|
_close_grid_modal(driver, config)
|
||||||
|
driver.wait(0.5)
|
||||||
|
driver.click(*config.LESSON_BACK_BUTTON)
|
||||||
|
driver.wait(1.5)
|
||||||
|
return remaining_tickets
|
||||||
|
|
||||||
|
|
||||||
|
def run(driver, config):
|
||||||
|
driver.focus_game()
|
||||||
|
|
||||||
|
if not _open_schedule_screen(driver, config):
|
||||||
|
print("[lesson] could not confirm schedule screen is open, aborting without pressing further keys")
|
||||||
|
return
|
||||||
|
|
||||||
|
tickets = _read_ticket_count(driver, config)
|
||||||
|
if tickets is None:
|
||||||
|
print("[lesson] could not OCR ticket count, aborting without pressing further keys")
|
||||||
|
return
|
||||||
|
|
||||||
|
print(f"[lesson] starting tickets: {tickets}")
|
||||||
|
if tickets <= 0:
|
||||||
|
print("[lesson] no lesson tickets available, nothing to do")
|
||||||
|
else:
|
||||||
|
for region_index in range(TOTAL_REGIONS):
|
||||||
|
if tickets <= 0:
|
||||||
|
print("[lesson] out of lesson tickets -- stopping")
|
||||||
|
break
|
||||||
|
tickets = _sweep_region(driver, config, region_index, tickets)
|
||||||
|
|
||||||
|
driver.click(*config.LESSON_BACK_BUTTON)
|
||||||
|
driver.wait(1.5)
|
||||||
|
print("[lesson] Done.")
|
||||||
@ -1,9 +1,6 @@
|
|||||||
"""Mailbox daily task. Ported from baas-reference module/mail.py's color-probe pattern."""
|
"""Mailbox daily task. Ported from baas-reference module/mail.py's color-probe pattern."""
|
||||||
|
|
||||||
# (500, 10) sits on the mailbox panel's plain header background; the home
|
from ba_auto import navigation
|
||||||
# screen shows character art there instead, so this tells open vs not-open.
|
|
||||||
HEADER_PROBE = (500, 10)
|
|
||||||
HEADER_OPEN_MIN_CHANNEL = 200
|
|
||||||
|
|
||||||
# "Claim all" renders as flat grey (153,153,153) when there is nothing to claim.
|
# "Claim all" renders as flat grey (153,153,153) when there is nothing to claim.
|
||||||
CLAIM_ALL_PROBE = (1710, 1128)
|
CLAIM_ALL_PROBE = (1710, 1128)
|
||||||
@ -13,11 +10,6 @@ CLAIM_ALL_DISABLED_TOLERANCE = 12
|
|||||||
OPEN_RETRIES = 3
|
OPEN_RETRIES = 3
|
||||||
|
|
||||||
|
|
||||||
def _is_open(driver):
|
|
||||||
r, g, b = driver.color_at(*HEADER_PROBE)
|
|
||||||
return r > HEADER_OPEN_MIN_CHANNEL and g > HEADER_OPEN_MIN_CHANNEL and b > HEADER_OPEN_MIN_CHANNEL
|
|
||||||
|
|
||||||
|
|
||||||
def _claim_all_disabled(driver):
|
def _claim_all_disabled(driver):
|
||||||
r, g, b = driver.color_at(*CLAIM_ALL_PROBE)
|
r, g, b = driver.color_at(*CLAIM_ALL_PROBE)
|
||||||
tr, tg, tb = CLAIM_ALL_DISABLED_RGB
|
tr, tg, tb = CLAIM_ALL_DISABLED_RGB
|
||||||
@ -35,7 +27,7 @@ def run(driver, config):
|
|||||||
for attempt in range(1, OPEN_RETRIES + 1):
|
for attempt in range(1, OPEN_RETRIES + 1):
|
||||||
driver.click(*config.MAILBOX_ICON)
|
driver.click(*config.MAILBOX_ICON)
|
||||||
driver.wait(1.5)
|
driver.wait(1.5)
|
||||||
if _is_open(driver):
|
if navigation.is_on_subscreen(driver):
|
||||||
opened = True
|
opened = True
|
||||||
break
|
break
|
||||||
print(f"[mailbox] panel not detected after click (attempt {attempt}/{OPEN_RETRIES})")
|
print(f"[mailbox] panel not detected after click (attempt {attempt}/{OPEN_RETRIES})")
|
||||||
|
|||||||
27
ba_auto/tasks/shop_common.py
Normal file
27
ba_auto/tasks/shop_common.py
Normal file
@ -0,0 +1,27 @@
|
|||||||
|
"""Common Shop (通常アイテム) daily task. Ported from baas-reference
|
||||||
|
module/shop/common_shop.py's implement() -- see shop_utils.py for the
|
||||||
|
shared checkbox-grid-then-bulk-buy control flow both shop tabs use."""
|
||||||
|
from ba_auto import navigation
|
||||||
|
from ba_auto.tasks import shop_utils
|
||||||
|
|
||||||
|
|
||||||
|
def run(driver, config):
|
||||||
|
driver.focus_game()
|
||||||
|
|
||||||
|
driver.click(*config.SHOP_ICON)
|
||||||
|
driver.wait(2)
|
||||||
|
if not navigation.is_on_subscreen(driver):
|
||||||
|
print("[shop_common] could not confirm shop opened, aborting without pressing further keys")
|
||||||
|
return
|
||||||
|
|
||||||
|
shop_utils.run_shop_tab(
|
||||||
|
driver, config,
|
||||||
|
tab_button=config.SHOP_TAB_COMMON,
|
||||||
|
targets=config.COMMON_SHOP_TARGETS,
|
||||||
|
balance_rect=config.CREDIT_BALANCE_OCR_RECT,
|
||||||
|
currency_label="credits",
|
||||||
|
)
|
||||||
|
|
||||||
|
driver.click(*config.SHOP_BACK_BUTTON)
|
||||||
|
driver.wait(1.5)
|
||||||
|
print("[shop_common] Done.")
|
||||||
34
ba_auto/tasks/shop_tactical.py
Normal file
34
ba_auto/tasks/shop_tactical.py
Normal file
@ -0,0 +1,34 @@
|
|||||||
|
"""Tactical Challenge Shop (戦術対抗戦) daily task. Ported from baas-reference
|
||||||
|
module/shop/tactical_challenge_shop.py's implement() -- see shop_utils.py for
|
||||||
|
the shared checkbox-grid-then-bulk-buy control flow both shop tabs use.
|
||||||
|
|
||||||
|
The reference reaches this tab via goto_shop_by_name's OCR swipe-search over
|
||||||
|
the shop-type tab list (module/shop/shop_utils.py), needed there because that
|
||||||
|
list can require scrolling. This account's tab list is only 7 entries and
|
||||||
|
fits on screen with no scrolling, confirmed live -- config.SHOP_TAB_TACTICAL
|
||||||
|
is a fixed click on the already-visible tab, not a shortcut around OCR (there
|
||||||
|
is nothing to search for)."""
|
||||||
|
from ba_auto import navigation
|
||||||
|
from ba_auto.tasks import shop_utils
|
||||||
|
|
||||||
|
|
||||||
|
def run(driver, config):
|
||||||
|
driver.focus_game()
|
||||||
|
|
||||||
|
driver.click(*config.SHOP_ICON)
|
||||||
|
driver.wait(2)
|
||||||
|
if not navigation.is_on_subscreen(driver):
|
||||||
|
print("[shop_tactical] could not confirm shop opened, aborting without pressing further keys")
|
||||||
|
return
|
||||||
|
|
||||||
|
shop_utils.run_shop_tab(
|
||||||
|
driver, config,
|
||||||
|
tab_button=config.SHOP_TAB_TACTICAL,
|
||||||
|
targets=config.TACTICAL_SHOP_TARGETS,
|
||||||
|
balance_rect=config.TACTICAL_COIN_OCR_RECT,
|
||||||
|
currency_label="tactical coin",
|
||||||
|
)
|
||||||
|
|
||||||
|
driver.click(*config.SHOP_BACK_BUTTON)
|
||||||
|
driver.wait(1.5)
|
||||||
|
print("[shop_tactical] Done.")
|
||||||
127
ba_auto/tasks/shop_utils.py
Normal file
127
ba_auto/tasks/shop_utils.py
Normal file
@ -0,0 +1,127 @@
|
|||||||
|
"""Shared Common Shop / Tactical Shop grid logic.
|
||||||
|
|
||||||
|
Ported from baas-reference module/shop/shop_utils.py's get_item_position/
|
||||||
|
ensure_choose/buy pattern -- the checkbox-select-then-bulk-buy grid shared by
|
||||||
|
both shop tabs in the live client. Adapted from the reference's
|
||||||
|
color+template item-state scan to this project's fixed-grid-position +
|
||||||
|
price-OCR-verify design: unlike story_sweep, the reference's own
|
||||||
|
item-identification here isn't OCR at all -- it indexes into
|
||||||
|
self.static_config.common_shop_price_list, a table sourced from an external
|
||||||
|
resource this repo doesn't have. So a locally pixel-scanned, name-commented
|
||||||
|
position table (config.COMMON_SHOP_TARGETS / config.TACTICAL_SHOP_TARGETS) is
|
||||||
|
the faithful local equivalent, not a shortcut around OCR the reference uses.
|
||||||
|
Price-digit OCR -- something the reference doesn't even do per-item -- is
|
||||||
|
layered on top as an extra safety net against catalog drift, consistent with
|
||||||
|
this project's verify-before-spend pattern elsewhere (mailbox/cafe/story_sweep).
|
||||||
|
"""
|
||||||
|
from ba_auto import detector, navigation
|
||||||
|
|
||||||
|
|
||||||
|
def _checkbox_center(config, row, col):
|
||||||
|
return config.SHOP_ITEM_COL_X[col], config.SHOP_ITEM_ROW_Y[row]
|
||||||
|
|
||||||
|
|
||||||
|
def _checkbox_region(config, row, col):
|
||||||
|
x, y = _checkbox_center(config, row, col)
|
||||||
|
return (x - 15, y - 15, x + 15, y + 15)
|
||||||
|
|
||||||
|
|
||||||
|
def _price_rect(config, row, col):
|
||||||
|
x, y = _checkbox_center(config, row, col)
|
||||||
|
x1, y1, x2, y2 = config.SHOP_PRICE_OCR_OFFSET
|
||||||
|
return (x + x1, y + y1, x + x2, y + y2)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_checked(driver, config, row, col):
|
||||||
|
lo, hi = config.SHOP_CHECKED_RGB
|
||||||
|
return detector.region_contains_color(_checkbox_region(config, row, col), lo, hi)
|
||||||
|
|
||||||
|
|
||||||
|
def select_targets(driver, config, targets):
|
||||||
|
"""OCR-verify each target's price, then click its checkbox and confirm
|
||||||
|
it registers as checked. Returns (selected, skipped); a price mismatch
|
||||||
|
or an unconfirmed checkbox skips just that target rather than aborting
|
||||||
|
the whole run (mirrors story_sweep's per-target skip-not-abort design).
|
||||||
|
"""
|
||||||
|
selected = []
|
||||||
|
skipped = []
|
||||||
|
for row, col, name, expected_price in targets:
|
||||||
|
price = detector.read_int(_price_rect(config, row, col))
|
||||||
|
if price != expected_price:
|
||||||
|
print(f"[shop] '{name}' price read as {price}, expected {expected_price} -- skipping (catalog may have changed)")
|
||||||
|
skipped.append((name, "price_mismatch"))
|
||||||
|
continue
|
||||||
|
x, y = _checkbox_center(config, row, col)
|
||||||
|
driver.click(x, y)
|
||||||
|
driver.wait(0.3)
|
||||||
|
if not _is_checked(driver, config, row, col):
|
||||||
|
print(f"[shop] '{name}' checkbox did not register as checked -- skipping")
|
||||||
|
skipped.append((name, "checkbox_not_confirmed"))
|
||||||
|
continue
|
||||||
|
selected.append((name, expected_price))
|
||||||
|
return selected, skipped
|
||||||
|
|
||||||
|
|
||||||
|
def confirm_purchase(driver, config):
|
||||||
|
"""Click the bulk Buy button, then click through the purchase-confirm
|
||||||
|
dialog and the reward-acquired banner. Both dim
|
||||||
|
config.SHOP_OVERLAY_PROBE away from pure white; press Enter until it
|
||||||
|
reads idle again, bounded by config.SHOP_PURCHASE_MAX_ENTER_PRESSES --
|
||||||
|
this project doesn't press keys past a bound against an unrecognized
|
||||||
|
state (see navigation.wait_for_state). Returns True once idle, False if
|
||||||
|
it never clears within the bound.
|
||||||
|
"""
|
||||||
|
driver.click(*config.SHOP_BUY_BUTTON)
|
||||||
|
driver.wait(1.5)
|
||||||
|
min_ch = config.SHOP_OVERLAY_IDLE_MIN_CHANNEL
|
||||||
|
for _ in range(config.SHOP_PURCHASE_MAX_ENTER_PRESSES):
|
||||||
|
r, g, b = driver.color_at(*config.SHOP_OVERLAY_PROBE)
|
||||||
|
if r > min_ch and g > min_ch and b > min_ch:
|
||||||
|
return True
|
||||||
|
driver.keypress("Return")
|
||||||
|
driver.wait(1.5)
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def run_shop_tab(driver, config, *, tab_button, targets, balance_rect, currency_label):
|
||||||
|
"""Shared control flow for both shop tabs: open the tab, verify enough
|
||||||
|
currency for everything configured, select+buy, report the outcome.
|
||||||
|
Ported from the reference's common_shop.py/tactical_challenge_shop.py
|
||||||
|
implement() -- both are this same shape (read assets, calculate cost,
|
||||||
|
buy, verify), differing only in tab entry point and currency.
|
||||||
|
"""
|
||||||
|
if not targets:
|
||||||
|
print(f"[shop] no targets configured for {currency_label}, nothing to do")
|
||||||
|
return "no_targets"
|
||||||
|
|
||||||
|
driver.click(*tab_button)
|
||||||
|
driver.wait(1.5)
|
||||||
|
if not navigation.is_on_subscreen(driver):
|
||||||
|
print("[shop] could not confirm shop tab opened, aborting without pressing further keys")
|
||||||
|
return "navigation_failed"
|
||||||
|
|
||||||
|
balance = detector.read_int(balance_rect)
|
||||||
|
if balance is None:
|
||||||
|
print(f"[shop] could not read {currency_label} balance, aborting without spending")
|
||||||
|
return "balance_unreadable"
|
||||||
|
|
||||||
|
total_cost = sum(price for _, _, _, price in targets)
|
||||||
|
if balance < total_cost:
|
||||||
|
print(f"[shop] insufficient {currency_label}: have {balance}, need {total_cost} -- stopping, nothing bought")
|
||||||
|
return "inadequate_assets"
|
||||||
|
|
||||||
|
selected, skipped = select_targets(driver, config, targets)
|
||||||
|
if not selected:
|
||||||
|
print("[shop] nothing selected (all targets skipped), cancelling")
|
||||||
|
return "nothing_selected"
|
||||||
|
|
||||||
|
if not confirm_purchase(driver, config):
|
||||||
|
print("[shop] purchase confirmation did not resolve to an idle screen within the retry bound -- stopping, check the game manually")
|
||||||
|
return "unrecognized_state"
|
||||||
|
|
||||||
|
spent = sum(price for _, price in selected)
|
||||||
|
msg = f"[shop] bought {len(selected)} item(s) for {spent} {currency_label}"
|
||||||
|
if skipped:
|
||||||
|
msg += f", skipped {len(skipped)}"
|
||||||
|
print(msg)
|
||||||
|
return "purchased"
|
||||||
59
ba_auto/tasks/stamina.py
Normal file
59
ba_auto/tasks/stamina.py
Normal file
@ -0,0 +1,59 @@
|
|||||||
|
"""Mission/task-menu AP+pyroxene claim. Ported from baas-reference module/collect_daily_task_power.py's claim-loop pattern (its rgb_in_range checks replaced with driver.color_at probes)."""
|
||||||
|
|
||||||
|
from ba_auto import navigation
|
||||||
|
|
||||||
|
OPEN_RETRIES = 3
|
||||||
|
# One claim-all round is normally enough (button goes grey right after), but
|
||||||
|
# bound it in case multiple reward reveals need dismissing in sequence.
|
||||||
|
CLAIM_MAX_ROUNDS = 5
|
||||||
|
|
||||||
|
# Yellow "one-click claim all" button vs. its own flat-grey disabled state --
|
||||||
|
# same background-pixel-probe idea as mailbox.CLAIM_ALL_PROBE/cafe.CLAIM_PROBE,
|
||||||
|
# but distinguished by hue (yellow has a big r-b gap; grey doesn't) since the
|
||||||
|
# button's grey isn't a single fixed RGB the way mailbox/cafe's are.
|
||||||
|
CLAIM_ENABLED_MIN_R = 230
|
||||||
|
CLAIM_ENABLED_MIN_RB_GAP = 100
|
||||||
|
|
||||||
|
|
||||||
|
def _claim_enabled(driver, config):
|
||||||
|
r, g, b = driver.color_at(*config.MISSION_CLAIM_PROBE)
|
||||||
|
return r > CLAIM_ENABLED_MIN_R and (r - b) > CLAIM_ENABLED_MIN_RB_GAP
|
||||||
|
|
||||||
|
|
||||||
|
def run(driver, config):
|
||||||
|
driver.focus_game()
|
||||||
|
|
||||||
|
opened = False
|
||||||
|
for attempt in range(1, OPEN_RETRIES + 1):
|
||||||
|
driver.focus_game()
|
||||||
|
driver.click(*config.MISSION_ICON)
|
||||||
|
driver.wait(2)
|
||||||
|
if navigation.is_on_subscreen(driver):
|
||||||
|
opened = True
|
||||||
|
break
|
||||||
|
print(f"[stamina] mission panel not detected after click (attempt {attempt}/{OPEN_RETRIES})")
|
||||||
|
|
||||||
|
if not opened:
|
||||||
|
print("[stamina] could not confirm mission panel is open, aborting without pressing further keys")
|
||||||
|
return
|
||||||
|
|
||||||
|
claimed_any = False
|
||||||
|
for _ in range(CLAIM_MAX_ROUNDS):
|
||||||
|
if not _claim_enabled(driver, config):
|
||||||
|
break
|
||||||
|
print("[stamina] claiming mission rewards")
|
||||||
|
driver.keypress("Return")
|
||||||
|
driver.wait(1.5)
|
||||||
|
# dismiss the reward-reveal popup ("TOUCH" prompt); harmless no-op if
|
||||||
|
# nothing is actually showing, same assumption as cafe's room-entry dismiss
|
||||||
|
driver.keypress("Return")
|
||||||
|
driver.wait(1.5)
|
||||||
|
claimed_any = True
|
||||||
|
|
||||||
|
if not claimed_any:
|
||||||
|
print("[stamina] nothing to claim")
|
||||||
|
|
||||||
|
if navigation.is_on_subscreen(driver):
|
||||||
|
driver.keypress("Escape")
|
||||||
|
driver.wait(1.5)
|
||||||
|
print("[stamina] Done.")
|
||||||
379
ba_auto/tasks/story_sweep.py
Normal file
379
ba_auto/tasks/story_sweep.py
Normal file
@ -0,0 +1,379 @@
|
|||||||
|
"""Normal story AP sweep. Reference: baas-reference/module/explore_tasks/sweep_task.py
|
||||||
|
and task_utils.py.
|
||||||
|
|
||||||
|
Ported per CLAUDE.md's "OCR policy" and Handoff.md: sweeps a config-driven
|
||||||
|
list of exact (region, stage, count) targets (config.STORY_SWEEP_TARGETS),
|
||||||
|
navigating to each one deterministically instead of the previous "latest
|
||||||
|
unlocked region, then a random stage" heuristic -- see plan.md Phase 9's
|
||||||
|
retrospective for why that heuristic was a mistake.
|
||||||
|
|
||||||
|
`_rotation_target` adds a second, date-derived target on top of that static
|
||||||
|
list, per explicit user direction: rather than grinding one fixed stage in
|
||||||
|
their current last region every run, it cycles through that region's stages
|
||||||
|
one per day (a plain date-ordinal modulo, so the cycle doesn't skip or
|
||||||
|
repeat around a year boundary). See config.py's STORY_SWEEP_ROTATION_*
|
||||||
|
constants.
|
||||||
|
|
||||||
|
- `_go_to_region` ports task_utils.py::to_region: OCR the current region
|
||||||
|
number, click the exact delta, re-check, bounded loop.
|
||||||
|
- `_find_stage_row` is a scoped-down port of core/image.py's
|
||||||
|
swipe_search_target_str: OCR each visible stage row's label and match it
|
||||||
|
against the configured target, rather than grabbing a random row.
|
||||||
|
- `_watch_sweep_result` is built on navigation.wait_for_state, this
|
||||||
|
project's scoped port of core/picture.py::co_detect, and returns a named
|
||||||
|
outcome ("swept", "inadequate_ap", "unrecognized_state", ...) the way the
|
||||||
|
reference's start_sweep does, instead of a single generic "Done".
|
||||||
|
|
||||||
|
The MAX-button click-then-verify and the modal's own X-button close (both
|
||||||
|
calibrated and confirmed live in Phase 9) are reused unchanged -- see
|
||||||
|
plan.md Phase 9/10.
|
||||||
|
"""
|
||||||
|
import datetime
|
||||||
|
|
||||||
|
from ba_auto import detector, navigation
|
||||||
|
|
||||||
|
OPEN_RETRIES = 3
|
||||||
|
POST_SWEEP_DISMISS_ROUNDS = 6
|
||||||
|
STAGE_MODAL_DIM_MAX_CHANNEL = 150
|
||||||
|
MAX_BUTTON_RETRIES = 3
|
||||||
|
MODAL_CLOSE_RETRIES = 3
|
||||||
|
|
||||||
|
|
||||||
|
def _is_stage_modal_open(driver, config):
|
||||||
|
r, g, b = driver.color_at(*config.STAGE_MODAL_PROBE)
|
||||||
|
return r < STAGE_MODAL_DIM_MAX_CHANNEL and g < STAGE_MODAL_DIM_MAX_CHANNEL and b < STAGE_MODAL_DIM_MAX_CHANNEL
|
||||||
|
|
||||||
|
|
||||||
|
def _color_in_range(rgb, rgb_range):
|
||||||
|
lo, hi = rgb_range
|
||||||
|
r, g, b = rgb
|
||||||
|
return lo[0] <= r <= hi[0] and lo[1] <= g <= hi[1] and lo[2] <= b <= hi[2]
|
||||||
|
|
||||||
|
|
||||||
|
def _is_sweep_usage_confirm(driver, config):
|
||||||
|
# 掃討開始 always raises a "use N AP to sweep M times?" confirmation
|
||||||
|
# before actually sweeping. Its OK button is this bright cyan.
|
||||||
|
return _color_in_range(driver.color_at(*config.SWEEP_CONFIRM_BUTTON), config.SWEEP_CONFIRM_CYAN)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_ap_purchase_prompt(driver, config):
|
||||||
|
# If AP is too low for even one sweep, a visually similar dialog appears
|
||||||
|
# at the *same* OK-button position but titled "AP購入" (spend real
|
||||||
|
# Pyroxene to buy more AP) with a gold/yellow OK instead of cyan --
|
||||||
|
# confirmed live by deliberately emptying the sweep count at low AP.
|
||||||
|
# Told apart from the safe confirm above by color, not position.
|
||||||
|
return _color_in_range(driver.color_at(*config.SWEEP_CONFIRM_BUTTON), config.SWEEP_CONFIRM_GOLD)
|
||||||
|
|
||||||
|
|
||||||
|
def _find_result_button(driver, config):
|
||||||
|
# The "掃討完了" (sweep complete) results screen shows a SKIP button
|
||||||
|
# (skips the reward-reveal animation) and then, once settled, a final
|
||||||
|
# OK button -- both the same cyan as the usage-confirm OK, but ~120px
|
||||||
|
# apart vertically. Finding whichever is showing by color avoids
|
||||||
|
# hardcoding both positions.
|
||||||
|
return detector.find_color_centroid(config.SWEEP_RESULT_BUTTON_REGION, *config.SWEEP_CONFIRM_CYAN)
|
||||||
|
|
||||||
|
|
||||||
|
def _count_raised_above_one(driver, config):
|
||||||
|
# The "-" stepper button is flat grey while count == 1 (its default,
|
||||||
|
# disabled at the minimum) and turns vivid orange once raised -- cheap
|
||||||
|
# way to confirm a MAX/"+" click actually registered without needing OCR
|
||||||
|
# on the count itself.
|
||||||
|
r, g, b = driver.color_at(*config.SWEEP_MINUS_BUTTON_PROBE)
|
||||||
|
return r > 200 and g < 180 and b < 100
|
||||||
|
|
||||||
|
|
||||||
|
def _open_task_screen(driver, config):
|
||||||
|
for attempt in range(1, OPEN_RETRIES + 1):
|
||||||
|
driver.click(*config.WORK_ICON)
|
||||||
|
driver.wait(2)
|
||||||
|
if navigation.is_on_subscreen(driver):
|
||||||
|
break
|
||||||
|
print(f"[story_sweep] work hub not detected after click (attempt {attempt}/{OPEN_RETRIES})")
|
||||||
|
else:
|
||||||
|
return False
|
||||||
|
|
||||||
|
for attempt in range(1, OPEN_RETRIES + 1):
|
||||||
|
driver.click(*config.TASK_CARD)
|
||||||
|
driver.wait(2)
|
||||||
|
if navigation.is_on_subscreen(driver):
|
||||||
|
return True
|
||||||
|
print(f"[story_sweep] task screen not detected after click (attempt {attempt}/{OPEN_RETRIES})")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _read_current_region(driver, config):
|
||||||
|
return detector.read_int(config.REGION_NUMBER_OCR_RECT)
|
||||||
|
|
||||||
|
|
||||||
|
def _region_arrow_visible(driver, config, center):
|
||||||
|
# Both arrows render as a solid navy-blue "<"/">" chevron on a
|
||||||
|
# light-blue backdrop when present. The last region in a given direction
|
||||||
|
# (or a locked one, per the reference's own "region-unavailable"
|
||||||
|
# template check) simply omits the arrow rather than greying it out --
|
||||||
|
# confirmed live: at region 30 (this account's current last region), the
|
||||||
|
# spot where the right arrow would be was plain background.
|
||||||
|
#
|
||||||
|
# Scans a small box around `center` rather than probing a single fixed
|
||||||
|
# point: a chevron is concave, and a centroid-derived single point
|
||||||
|
# landed in the notch between its two strokes -- reading as "absent"
|
||||||
|
# even while the glyph was clearly rendered a few pixels away. See
|
||||||
|
# plan.md Phase 10.
|
||||||
|
cx, cy = center
|
||||||
|
rect = (cx - 40, cy - 35, cx + 40, cy + 35)
|
||||||
|
return detector.region_contains_color(rect, (40, 70, 120), (100, 130, 190))
|
||||||
|
|
||||||
|
|
||||||
|
def _go_to_region(driver, config, target_region):
|
||||||
|
cur = _read_current_region(driver, config)
|
||||||
|
if cur is None:
|
||||||
|
print("[story_sweep] could not OCR the current region number")
|
||||||
|
return False
|
||||||
|
print(f"[story_sweep] current region {cur}, target region {target_region}")
|
||||||
|
|
||||||
|
for attempt in range(1, config.REGION_NAV_MAX_ATTEMPTS + 1):
|
||||||
|
if cur == target_region:
|
||||||
|
return True
|
||||||
|
going_left = cur > target_region
|
||||||
|
arrow_pos = config.REGION_LEFT_ARROW if going_left else config.REGION_RIGHT_ARROW
|
||||||
|
if not _region_arrow_visible(driver, config, arrow_pos):
|
||||||
|
direction = "left" if going_left else "right"
|
||||||
|
print(f"[story_sweep] region {target_region} unreachable -- no {direction} arrow at region {cur}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
clicks = abs(cur - target_region)
|
||||||
|
for _ in range(clicks):
|
||||||
|
driver.click(*arrow_pos)
|
||||||
|
driver.wait(1)
|
||||||
|
|
||||||
|
new_cur = _read_current_region(driver, config)
|
||||||
|
if new_cur is None or new_cur == cur:
|
||||||
|
print(f"[story_sweep] region number unchanged after {clicks} click(s) (attempt {attempt}/{config.REGION_NAV_MAX_ATTEMPTS})")
|
||||||
|
return False
|
||||||
|
cur = new_cur
|
||||||
|
|
||||||
|
print(f"[story_sweep] gave up navigating to region {target_region} after {config.REGION_NAV_MAX_ATTEMPTS} attempts")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_label(text):
|
||||||
|
# OCR sometimes reads the row's dash as a different dash-like glyph, or
|
||||||
|
# picks up stray whitespace -- normalize before comparing.
|
||||||
|
return text.strip().upper().replace("—", "-").replace("–", "-").replace(" ", "")
|
||||||
|
|
||||||
|
|
||||||
|
def _label_suffix(label):
|
||||||
|
# Compare only the part after the dash (e.g. "2" in "30-2", "A" in
|
||||||
|
# "30-A"), not the full "{region}-{stage}" string. Confirmed live: this
|
||||||
|
# font's leading region-number digit is read unreliably by OCR (e.g. "3"
|
||||||
|
# misread as "2") even after threshold preprocessing, while the stage
|
||||||
|
# suffix after the dash reads correctly across every row tested -- and we
|
||||||
|
# don't need the region digit anyway, since _go_to_region has already
|
||||||
|
# independently confirmed we're in the right region. See plan.md Phase 10.
|
||||||
|
parts = [p for p in _normalize_label(label).split("-") if p]
|
||||||
|
return parts[-1] if parts else ""
|
||||||
|
|
||||||
|
|
||||||
|
def _row_label_rect(config, row_y):
|
||||||
|
x1, x2 = config.STAGE_LABEL_OCR_X
|
||||||
|
top_pad, bottom_pad = config.STAGE_LABEL_OCR_Y_PAD
|
||||||
|
return (x1, row_y - top_pad, x2, row_y - bottom_pad)
|
||||||
|
|
||||||
|
|
||||||
|
def _find_stage_row(driver, config, region, stage):
|
||||||
|
# Scoped-down swipe_search_target_str (see module docstring): this
|
||||||
|
# client's stage list only ever needs the two already-calibrated scroll
|
||||||
|
# extremes checked, not arbitrary swipe-and-retry.
|
||||||
|
target_suffix = str(stage)
|
||||||
|
x, y = config.STAGE_LIST_SCROLL_POINT
|
||||||
|
|
||||||
|
driver.scroll(x, y, "up", config.STAGE_LIST_SCROLL_CLICKS)
|
||||||
|
driver.wait(0.5)
|
||||||
|
for row_y in config.STAGE_ROWS_AT_TOP_Y:
|
||||||
|
label = detector.read_text(_row_label_rect(config, row_y), whitelist="0123456789-A")
|
||||||
|
print(f"[story_sweep] row @ {row_y} (scrolled up): read '{label}'")
|
||||||
|
if _label_suffix(label) == target_suffix:
|
||||||
|
return row_y
|
||||||
|
|
||||||
|
driver.scroll(x, y, "down", config.STAGE_LIST_SCROLL_CLICKS)
|
||||||
|
driver.wait(0.5)
|
||||||
|
for row_y in config.STAGE_ROWS_AT_BOTTOM_Y:
|
||||||
|
label = detector.read_text(_row_label_rect(config, row_y), whitelist="0123456789-A")
|
||||||
|
print(f"[story_sweep] row @ {row_y} (scrolled down): read '{label}'")
|
||||||
|
if _label_suffix(label) == target_suffix:
|
||||||
|
return row_y
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _click_max_and_verify(driver, config):
|
||||||
|
for attempt in range(1, MAX_BUTTON_RETRIES + 1):
|
||||||
|
driver.click(*config.SWEEP_MAX_BUTTON)
|
||||||
|
driver.wait(0.8)
|
||||||
|
if _count_raised_above_one(driver, config):
|
||||||
|
return True
|
||||||
|
print(f"[story_sweep] MAX click not detected (attempt {attempt}/{MAX_BUTTON_RETRIES})")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _click_plus_and_verify(driver, config, count):
|
||||||
|
for attempt in range(1, MAX_BUTTON_RETRIES + 1):
|
||||||
|
for _ in range(count - 1):
|
||||||
|
driver.click(*config.SWEEP_PLUS_BUTTON)
|
||||||
|
driver.wait(0.8)
|
||||||
|
if _count_raised_above_one(driver, config):
|
||||||
|
return True
|
||||||
|
print(f"[story_sweep] count-raise via '+' not detected (attempt {attempt}/{MAX_BUTTON_RETRIES})")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _set_sweep_count(driver, config, count):
|
||||||
|
if count == "max":
|
||||||
|
return _click_max_and_verify(driver, config)
|
||||||
|
return _click_plus_and_verify(driver, config, count)
|
||||||
|
|
||||||
|
|
||||||
|
def _close_stage_modal(driver, config):
|
||||||
|
# Escape does not close this modal (confirmed live: it stayed open with
|
||||||
|
# focus on the live "任務開始" button after two Escape presses) -- the
|
||||||
|
# only reliable close path is clicking its own X button, verified.
|
||||||
|
for _ in range(MODAL_CLOSE_RETRIES):
|
||||||
|
if not _is_stage_modal_open(driver, config):
|
||||||
|
return True
|
||||||
|
driver.click(*config.STAGE_MODAL_CLOSE_BUTTON)
|
||||||
|
driver.wait(1)
|
||||||
|
return not _is_stage_modal_open(driver, config)
|
||||||
|
|
||||||
|
|
||||||
|
def _watch_sweep_result(driver, config):
|
||||||
|
# The reference's start_sweep returns one of "inadequate_ap",
|
||||||
|
# "charge_challenge_counts", or "sweep_complete" so its caller reacts
|
||||||
|
# appropriately -- this ports that same named-outcome contract via
|
||||||
|
# navigation.wait_for_state instead of the old single generic "Done".
|
||||||
|
#
|
||||||
|
# Called only after the usage-confirm dialog is already accepted (see
|
||||||
|
# _sweep_target), so from here it's purely "click through the
|
||||||
|
# 掃討完了 SKIP/OK screens until the bare stage-info modal reappears."
|
||||||
|
# Clicking the found button by color (not a keypress) means this never
|
||||||
|
# risks landing on the underlying "任務開始" button the way a blind
|
||||||
|
# Enter-press loop would.
|
||||||
|
def click_result_button(d):
|
||||||
|
pos = _find_result_button(d, config)
|
||||||
|
if pos:
|
||||||
|
d.click(*pos)
|
||||||
|
d.wait(1.5)
|
||||||
|
|
||||||
|
ends = {
|
||||||
|
(lambda d, c: _is_stage_modal_open(d, c) and _find_result_button(d, c) is None): "swept",
|
||||||
|
}
|
||||||
|
reactions = {
|
||||||
|
(lambda d, c: _find_result_button(d, c) is not None): click_result_button,
|
||||||
|
}
|
||||||
|
outcome = navigation.wait_for_state(
|
||||||
|
driver, config, reactions, ends,
|
||||||
|
max_iterations=POST_SWEEP_DISMISS_ROUNDS, poll_interval=1.5,
|
||||||
|
)
|
||||||
|
return outcome or "unrecognized_state"
|
||||||
|
|
||||||
|
|
||||||
|
def _sweep_target(driver, config, region, stage, count):
|
||||||
|
print(f"[story_sweep] --- target {region}-{stage} x {count} ---")
|
||||||
|
|
||||||
|
if not _go_to_region(driver, config, region):
|
||||||
|
return "region_unavailable"
|
||||||
|
|
||||||
|
row_y = _find_stage_row(driver, config, region, stage)
|
||||||
|
if row_y is None:
|
||||||
|
print(f"[story_sweep] stage {region}-{stage} not found in the visible stage list")
|
||||||
|
return "stage_not_found"
|
||||||
|
|
||||||
|
driver.click(config.STAGE_ENTER_X, row_y)
|
||||||
|
driver.wait(2)
|
||||||
|
|
||||||
|
if not _is_stage_modal_open(driver, config):
|
||||||
|
print("[story_sweep] stage info panel not detected, aborting")
|
||||||
|
if navigation.is_on_subscreen(driver):
|
||||||
|
driver.keypress("Escape")
|
||||||
|
driver.wait(1.5)
|
||||||
|
return "unrecognized_state"
|
||||||
|
|
||||||
|
if not _set_sweep_count(driver, config, count):
|
||||||
|
print("[story_sweep] could not confirm sweep count was raised, aborting without spending AP")
|
||||||
|
_close_stage_modal(driver, config)
|
||||||
|
if navigation.is_on_subscreen(driver):
|
||||||
|
driver.keypress("Escape")
|
||||||
|
driver.wait(1.5)
|
||||||
|
return "unrecognized_state"
|
||||||
|
|
||||||
|
driver.click(*config.SWEEP_START_BUTTON)
|
||||||
|
driver.wait(1.5)
|
||||||
|
|
||||||
|
if _is_ap_purchase_prompt(driver, config):
|
||||||
|
print("[story_sweep] insufficient AP for this sweep -- cancelling without purchasing")
|
||||||
|
driver.click(*config.SWEEP_CONFIRM_CANCEL_BUTTON)
|
||||||
|
driver.wait(1)
|
||||||
|
_close_stage_modal(driver, config)
|
||||||
|
if navigation.is_on_subscreen(driver):
|
||||||
|
driver.keypress("Escape")
|
||||||
|
driver.wait(1.5)
|
||||||
|
return "inadequate_ap"
|
||||||
|
|
||||||
|
if not _is_sweep_usage_confirm(driver, config):
|
||||||
|
print("[story_sweep] sweep-usage confirmation not detected, aborting without further input")
|
||||||
|
_close_stage_modal(driver, config)
|
||||||
|
if navigation.is_on_subscreen(driver):
|
||||||
|
driver.keypress("Escape")
|
||||||
|
driver.wait(1.5)
|
||||||
|
return "unrecognized_state"
|
||||||
|
|
||||||
|
driver.click(*config.SWEEP_CONFIRM_BUTTON)
|
||||||
|
driver.wait(1.5)
|
||||||
|
print("[story_sweep] sweep confirmed, waiting for results")
|
||||||
|
outcome = _watch_sweep_result(driver, config)
|
||||||
|
print(f"[story_sweep] result: {outcome}")
|
||||||
|
|
||||||
|
if not _close_stage_modal(driver, config):
|
||||||
|
print("[story_sweep] warning: could not confirm stage info modal closed -- leaving it open rather than pressing further keys blindly")
|
||||||
|
return outcome
|
||||||
|
|
||||||
|
if navigation.is_on_subscreen(driver):
|
||||||
|
driver.keypress("Escape")
|
||||||
|
driver.wait(1.5)
|
||||||
|
return outcome
|
||||||
|
|
||||||
|
|
||||||
|
def _rotation_target(config):
|
||||||
|
region = getattr(config, "STORY_SWEEP_ROTATION_REGION", None)
|
||||||
|
if not region:
|
||||||
|
return None
|
||||||
|
stage_count = config.STORY_SWEEP_ROTATION_STAGE_COUNT
|
||||||
|
stage = (datetime.date.today().toordinal() % stage_count) + 1
|
||||||
|
return (region, stage, config.STORY_SWEEP_ROTATION_COUNT)
|
||||||
|
|
||||||
|
|
||||||
|
def run(driver, config):
|
||||||
|
driver.focus_game()
|
||||||
|
|
||||||
|
targets = list(config.STORY_SWEEP_TARGETS)
|
||||||
|
rotation = _rotation_target(config)
|
||||||
|
if rotation:
|
||||||
|
print(f"[story_sweep] today's rotation target: region {rotation[0]} stage {rotation[1]}")
|
||||||
|
targets.append(rotation)
|
||||||
|
|
||||||
|
if not targets:
|
||||||
|
print("[story_sweep] no targets configured (config.STORY_SWEEP_TARGETS is empty and rotation is disabled), nothing to do")
|
||||||
|
return
|
||||||
|
|
||||||
|
if not _open_task_screen(driver, config):
|
||||||
|
print("[story_sweep] could not confirm task screen is open, aborting without pressing further keys")
|
||||||
|
return
|
||||||
|
|
||||||
|
for region, stage, count in targets:
|
||||||
|
outcome = _sweep_target(driver, config, region, stage, count)
|
||||||
|
if outcome == "inadequate_ap":
|
||||||
|
print("[story_sweep] insufficient AP -- stopping, not attempting remaining targets")
|
||||||
|
break
|
||||||
|
if outcome != "swept":
|
||||||
|
print(f"[story_sweep] target {region}-{stage} ended in '{outcome}' -- skipping to next target")
|
||||||
|
|
||||||
|
print("[story_sweep] Done.")
|
||||||
13
ba_daily.py
13
ba_daily.py
@ -3,13 +3,22 @@
|
|||||||
import sys
|
import sys
|
||||||
|
|
||||||
from ba_auto import config, driver
|
from ba_auto import config, driver
|
||||||
from ba_auto.tasks import cafe, mailbox
|
from ba_auto.tasks import cafe, lesson, mailbox, shop_common, shop_tactical, stamina, story_sweep
|
||||||
|
|
||||||
TASKS = {
|
TASKS = {
|
||||||
"mailbox": mailbox.run,
|
"mailbox": mailbox.run,
|
||||||
"cafe": cafe.run,
|
"cafe": cafe.run,
|
||||||
|
"stamina": stamina.run,
|
||||||
|
"story_sweep": story_sweep.run,
|
||||||
|
"shop_common": shop_common.run,
|
||||||
|
"shop_tactical": shop_tactical.run,
|
||||||
|
"lesson": lesson.run,
|
||||||
}
|
}
|
||||||
DEFAULT_ORDER = ["mailbox", "cafe"]
|
# story_sweep, both shop tasks, and lesson are opt-in only (not in the
|
||||||
|
# default flow): they spend AP/credits/tactical coin/lesson tickets on an
|
||||||
|
# automated choice rather than reclaiming something free, which is a real
|
||||||
|
# resource decision the default unattended run shouldn't make blindly.
|
||||||
|
DEFAULT_ORDER = ["mailbox", "cafe", "stamina"]
|
||||||
|
|
||||||
|
|
||||||
def main(argv):
|
def main(argv):
|
||||||
|
|||||||
File diff suppressed because one or more lines are too long
@ -348,7 +348,7 @@
|
|||||||
"norm_label": "screenshots/ reference-capture directory"
|
"norm_label": "screenshots/ reference-capture directory"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"label": "./scratchpad working-file convention",
|
"label": "scratchpad/ working-file convention",
|
||||||
"file_type": "concept",
|
"file_type": "concept",
|
||||||
"source_file": "CLAUDE.md",
|
"source_file": "CLAUDE.md",
|
||||||
"source_location": "L38",
|
"source_location": "L38",
|
||||||
@ -358,7 +358,7 @@
|
|||||||
"contributor": null,
|
"contributor": null,
|
||||||
"id": "claude_scratchpad_dir",
|
"id": "claude_scratchpad_dir",
|
||||||
"community": 0,
|
"community": 0,
|
||||||
"norm_label": "./scratchpad working-file convention"
|
"norm_label": "scratchpad/ working-file convention"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"label": "~/.venvs/ba-auto-daily Python venv (opencv-python, numpy)",
|
"label": "~/.venvs/ba-auto-daily Python venv (opencv-python, numpy)",
|
||||||
|
|||||||
308
plan.md
308
plan.md
@ -8,7 +8,7 @@ This project controls the PC/Steam/Proton Blue Archive client running on `nik-gp
|
|||||||
- scrot
|
- scrot
|
||||||
- Python
|
- Python
|
||||||
- OpenCV
|
- OpenCV
|
||||||
- later OCR when needed
|
- OCR, ported wherever the reference implementation uses it for a feature (see `CLAUDE.md` → "OCR policy" — this is no longer a "later, when needed" deferral)
|
||||||
|
|
||||||
Development happens on `nik-macbookair`.
|
Development happens on `nik-macbookair`.
|
||||||
|
|
||||||
@ -76,13 +76,12 @@ This is the intended direction. It does not have to be completed all at once.
|
|||||||
| `~/repo/ba-auto-daily/ba_dailies.sh` | Thin launcher only. It should call the Python entry point. Do not add new feature logic here. |
|
| `~/repo/ba-auto-daily/ba_dailies.sh` | Thin launcher only. It should call the Python entry point. Do not add new feature logic here. |
|
||||||
| `~/repo/ba-auto-daily/ba_daily.py` | Main Python CLI entry point. Dispatches tasks such as mailbox, cafe, stamina, group, etc. |
|
| `~/repo/ba-auto-daily/ba_daily.py` | Main Python CLI entry point. Dispatches tasks such as mailbox, cafe, stamina, group, etc. |
|
||||||
| `~/repo/ba-auto-daily/ba_auto/driver.py` | Local PC/Steam/Proton control backend. Wraps xdotool, scrot, waits, clicks, swipes, keypresses, screenshots, and window focus. |
|
| `~/repo/ba-auto-daily/ba_auto/driver.py` | Local PC/Steam/Proton control backend. Wraps xdotool, scrot, waits, clicks, swipes, keypresses, screenshots, and window focus. |
|
||||||
| `~/repo/ba-auto-daily/ba_auto/detector.py` | OpenCV/template/color matching helpers. Existing `scripts/detect_and_click.py` logic should be migrated here. |
|
| `~/repo/ba-auto-daily/ba_auto/detector.py` | OpenCV/template/color matching helpers. Currently has `find_cafe_sparkle()`, ported in-process from the retired `scripts/detect_and_click.py`. |
|
||||||
| `~/repo/ba-auto-daily/ba_auto/navigation.py` | Shared navigation helpers: home, menu, close popup, back, open feature screens. |
|
| `~/repo/ba-auto-daily/ba_auto/navigation.py` | Shared navigation/state-probe helpers: `is_on_subscreen`, `is_modal_open`, used by both `mailbox.py` and `cafe.py`. |
|
||||||
| `~/repo/ba-auto-daily/ba_auto/tasks/` | Feature implementations. Each task should adapt the relevant `baas-reference/module/...` logic where possible. |
|
| `~/repo/ba-auto-daily/ba_auto/tasks/` | Feature implementations. Each task should adapt the relevant `baas-reference/module/...` logic where possible. |
|
||||||
| `~/repo/ba-auto-daily/ba_auto/reference_notes/mapping.md` | Reference mapping table: local feature → reference module → local implementation → driver gaps. |
|
| `~/repo/ba-auto-daily/ba_auto/reference_notes/mapping.md` | Reference mapping table: local feature → reference module → local implementation → driver gaps. |
|
||||||
| `~/repo/ba-auto-daily/assets/` | Locally captured template images, such as cafe sparkle. Do not blindly copy assets from the reference repo. |
|
| `~/repo/ba-auto-daily/assets/` | Locally captured template images, such as cafe sparkle. Do not blindly copy assets from the reference repo. |
|
||||||
| `~/repo/ba-auto-daily/screenshots/` | Human reference screenshots, mostly Moonlight/game captures, used for calibration and debugging. |
|
| `~/repo/ba-auto-daily/screenshots/` | Human reference screenshots, mostly Moonlight/game captures, used for calibration and debugging. |
|
||||||
| `~/repo/ba-auto-daily/scripts/` | Transitional scripts. Long-term reusable Python logic should move into `ba_auto/`. |
|
|
||||||
| `~/repo/ba-auto-daily/setup.sh` | Bootstrap/deploy helper for `nik-gpu`. Should install/check dependencies and copy runtime files. |
|
| `~/repo/ba-auto-daily/setup.sh` | Bootstrap/deploy helper for `nik-gpu`. Should install/check dependencies and copy runtime files. |
|
||||||
| `~/repo/baas-reference/` | Read-only GPL-3.0 reference clone. Study and adapt. Never edit. |
|
| `~/repo/baas-reference/` | Read-only GPL-3.0 reference clone. Study and adapt. Never edit. |
|
||||||
|
|
||||||
@ -98,11 +97,7 @@ Preferred runtime layout:
|
|||||||
| `nik-gpu:~/ba_assets/` | Runtime assets/templates. |
|
| `nik-gpu:~/ba_assets/` | Runtime assets/templates. |
|
||||||
| `nik-gpu:~/.venvs/ba-auto-daily/` | Python virtual environment. |
|
| `nik-gpu:~/.venvs/ba-auto-daily/` | Python virtual environment. |
|
||||||
|
|
||||||
Current older layout may include:
|
`nik-gpu:~/ba_scripts/` may still contain `detect_and_click.py` and `ba_dailies_legacy.sh` left over from before both mailbox and cafe were migrated off them. Neither is deployed or referenced by anything anymore (`setup.sh` stopped copying them once Phase 6 landed) — safe to delete manually on `nik-gpu`, just not automated here.
|
||||||
|
|
||||||
| Path | What it is |
|
|
||||||
|---|---|
|
|
||||||
| `nik-gpu:~/ba_scripts/detect_and_click.py` | Old standalone detector helper. Should eventually be replaced by `ba_auto/detector.py`. |
|
|
||||||
|
|
||||||
## Implementation strategy
|
## Implementation strategy
|
||||||
|
|
||||||
@ -140,8 +135,9 @@ Initial seed:
|
|||||||
| Local feature | Reference file | Reference functions/classes | Local file | Backend replacements | Status |
|
| Local feature | Reference file | Reference functions/classes | Local file | Backend replacements | Status |
|
||||||
|---|---|---|---|---|---|
|
|---|---|---|---|---|---|
|
||||||
| Mailbox | Need to confirm in reference | Need to inspect | `ba_auto/tasks/mailbox.py` | tap/click via xdotool, screenshot via scrot | Existing Bash behavior; migrate to Python |
|
| Mailbox | Need to confirm in reference | Need to inspect | `ba_auto/tasks/mailbox.py` | tap/click via xdotool, screenshot via scrot | Existing Bash behavior; migrate to Python |
|
||||||
| Cafe | Need to confirm in reference | Need to inspect | `ba_auto/tasks/cafe.py` | template matching via OpenCV, click via xdotool | Existing Bash + helper behavior; migrate to Python |
|
| Cafe | `module/cafe_reward.py` | `to_cafe`, `interaction_for_cafe_solve_method3`, `collect` | `ba_auto/tasks/cafe.py` | `picture.co_detect`/`color.rgb_in_range` → `driver.color_at` pixel-probe checks; sparkle template match ported in-process into `ba_auto/detector.py` | Migrated: real Python, state-verified via color probes, no legacy bridge |
|
||||||
| Stamina/AP | `module/collect_daily_free_power.py`, `module/collect_daily_task_power.py` | Need to inspect | `ba_auto/tasks/stamina.py` | color checks/clicks via local driver | Not started |
|
| Stamina/AP | `module/collect_daily_free_power.py`, `module/collect_daily_task_power.py` | `to_tasks`/`implement` (task-power, ported); `to_purchase_pyroxenes_menu`/`detect_free_power_availability` (free-power, not ported — real-money purchase menu) | `ba_auto/tasks/stamina.py` | `color.rgb_in_range` → `driver.color_at`; reference's per-tab claim loop replaced by the live UI's single "一括受取" (claim-all) button, triggered via Enter | Partially migrated (Phase 8): Mission-panel task/weekly/achievement claim done. Daily Free Power deliberately not implemented. |
|
||||||
|
| Normal/Hard story AP sweep | `module/explore_tasks/sweep_task.py`, `module/explore_tasks/task_utils.py` | `to_region`/`to_normal_event` + OCR-driven per-stage claim loop (ported, Phase 10) | `ba_auto/tasks/story_sweep.py` | OCR-based region/stage-name matching, ported for real (Phase 10); reference's per-stage claim loop → this client's stage-info modal's self-contained 掃討 (sweep) sub-panel (MIN/-/+/MAX stepper + start button) | Done (Phase 10, supersedes Phase 9's random-pick design) |
|
||||||
| Group/Club AP | `module/group.py` | Need to inspect | `ba_auto/tasks/group.py` | fixed click + state check via local driver | Not started |
|
| Group/Club AP | `module/group.py` | Need to inspect | `ba_auto/tasks/group.py` | fixed click + state check via local driver | Not started |
|
||||||
| Bounty | `module/rewarded_task.py` | Need to inspect | `ba_auto/tasks/bounty.py` | sweep/color/OCR adaptation | Not started |
|
| Bounty | `module/rewarded_task.py` | Need to inspect | `ba_auto/tasks/bounty.py` | sweep/color/OCR adaptation | Not started |
|
||||||
| Commissions | `module/clear_special_task_power.py` | Need to inspect | `ba_auto/tasks/commission.py` | sweep/color adaptation | Not started |
|
| Commissions | `module/clear_special_task_power.py` | Need to inspect | `ba_auto/tasks/commission.py` | sweep/color adaptation | Not started |
|
||||||
@ -157,9 +153,13 @@ Do not implement a feature without filling at least the relevant row.
|
|||||||
| Feature | Current status | Target status |
|
| Feature | Current status | Target status |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Mailbox claim | Migrated: `ba_auto/tasks/mailbox.py` uses `driver.color_at` to verify the panel opened before acting (found live-testing bug: a marginal icon coordinate could miss and cascade into pressing Escape on the home screen, which triggers Blue Archive's own exit-game confirmation) | Done |
|
| Mailbox claim | Migrated: `ba_auto/tasks/mailbox.py` uses `driver.color_at` to verify the panel opened before acting (found live-testing bug: a marginal icon coordinate could miss and cascade into pressing Escape on the home screen, which triggers Blue Archive's own exit-game confirmation) | Done |
|
||||||
| Cafe pats + income | Bridged: `ba_auto/tasks/cafe.py` calls `scripts/ba_dailies_legacy.sh cafe` (which still shells out to `scripts/detect_and_click.py`) | Migrate to `ba_auto/tasks/cafe.py` and `ba_auto/detector.py` (Phase 6) |
|
| Cafe pats + income | Migrated: `ba_auto/tasks/cafe.py` verifies each room/dialog opened via `driver.color_at` before acting; sparkle detection now runs in-process via `ba_auto/detector.py` instead of a per-click subprocess | Done |
|
||||||
| Shared driver | `ba_auto/driver.py` built (`run_command`, `focus_game`, `click`, `keypress`, `screenshot`, `wait`); not yet wired into tasks | Wire into mailbox/cafe as they migrate off the Bash bridge |
|
| Stamina/AP (mission claim) | Migrated (partial, Phase 8): `ba_auto/tasks/stamina.py` claims the Mission panel's bulk "一括受取" button. Daily Free Power (real-money purchase menu) intentionally not implemented | Daily Free Power still not started |
|
||||||
| Python CLI | Built: `ba_daily.py` dispatches `mailbox`/`cafe`/default flow | Extend as new tasks are added |
|
| Normal/Hard story AP sweep | Done (Phase 10, supersedes Phase 9's random-pick design): `ba_auto/tasks/story_sweep.py` sweeps a config-driven list of exact `(region, stage, count)` targets (`config.STORY_SWEEP_TARGETS`), navigating to each via OCR (region-number read + delta-click, stage-label OCR match) instead of "latest region, random stage." Opt-in only (`story_sweep` command), not part of the default daily flow | Done |
|
||||||
|
| Common Shop / Tactical Shop | Done (Phase 11): `ba_auto/tasks/shop_common.py` / `shop_tactical.py` share a checkbox-grid-then-bulk-buy flow (`ba_auto/tasks/shop_utils.py`) against config-driven `(row, col, name, expected_price)` targets, price-OCR-verified before each click. Live-tested with real purchases in both shops. Opt-in only (`shop_common`/`shop_tactical` commands), not part of the default daily flow | Done |
|
||||||
|
| Lesson/Schedule | Done (Phase 12): `ba_auto/tasks/lesson.py` sweeps every unlocked region's schedule grid, picking the highest-affection available lesson each time via a dedicated heart-badge OCR read (`detector.read_int_on_heart_badge`) until tickets or lessons run out. Live-tested with real tickets spent; two real bugs (checkmark-doesn't-blank-the-number, badge OCR misreads) found and fixed. Opt-in only (`lesson` command), not part of the default daily flow | Done |
|
||||||
|
| Shared driver | `ba_auto/driver.py` built (`run_command`, `focus_game`, `click`, `move_mouse`, `scroll`, `keypress`, `screenshot`, `wait`, `color_at`); `click()` now splits `mousemove`/`click` into two xdotool calls (Phase 8 finding — fixes a real source of click flakiness); wired into `mailbox.py`, `cafe.py`, `stamina.py`, `story_sweep.py`, `shop_common.py`, `shop_tactical.py`, `lesson.py` | Extend with new primitives as future tasks need them |
|
||||||
|
| Python CLI | Built: `ba_daily.py` dispatches `mailbox`/`cafe`/`stamina`/`story_sweep`/`shop_common`/`shop_tactical`/`lesson`/default flow | Extend as new tasks are added |
|
||||||
| Reference mapping | Built: `ba_auto/reference_notes/mapping.md` | Fill in reference file/function columns per feature |
|
| Reference mapping | Built: `ba_auto/reference_notes/mapping.md` | Fill in reference file/function columns per feature |
|
||||||
| Everything else | Not started | Implement reference-first in Python |
|
| Everything else | Not started | Implement reference-first in Python |
|
||||||
|
|
||||||
@ -210,7 +210,7 @@ Keep compatibility with:
|
|||||||
|
|
||||||
### Phase 3: Driver extraction
|
### Phase 3: Driver extraction
|
||||||
|
|
||||||
**Status: Primitives added to `ba_auto/driver.py`, not yet wired into task modules.**
|
**Status: Done.** Primitives (including `color_at`, added during the mailbox/cafe hardening work) are wired into both `ba_auto/tasks/mailbox.py` and `ba_auto/tasks/cafe.py`.
|
||||||
|
|
||||||
Move shell interactions into `ba_auto/driver.py`.
|
Move shell interactions into `ba_auto/driver.py`.
|
||||||
|
|
||||||
@ -229,11 +229,9 @@ wait_until(...)
|
|||||||
|
|
||||||
### Phase 4: Detector extraction
|
### Phase 4: Detector extraction
|
||||||
|
|
||||||
**Status: Not started — `ba_auto/detector.py` is currently a placeholder.**
|
**Status: Done (scoped).** `scripts/detect_and_click.py`'s sparkle-matching logic (masked template match against `assets/cafe_sparkle.png`) was ported into `ba_auto/detector.py` as `find_cafe_sparkle()`, called in-process from `ba_auto/tasks/cafe.py` — this removed the old per-click Python cold start (a fresh `cv2`/`numpy` import per subprocess call) that CLAUDE.md's driver-layer guidance specifically warns against. `scripts/detect_and_click.py` had no remaining callers once this landed, so it was deleted rather than kept as a compatibility wrapper. The more generic primitives listed below (`load_template`, `match_template`, etc.) have not been built — only the one concrete sparkle-matching function needed so far exists; generalize when a second detector use case actually needs it.
|
||||||
|
|
||||||
Move `scripts/detect_and_click.py` logic into `ba_auto/detector.py`.
|
Detector primitives, generalize later if needed:
|
||||||
|
|
||||||
Detector primitives should include:
|
|
||||||
|
|
||||||
```
|
```
|
||||||
load_template(...)
|
load_template(...)
|
||||||
@ -244,8 +242,6 @@ color_mask(...)
|
|||||||
debug_write_match(...)
|
debug_write_match(...)
|
||||||
```
|
```
|
||||||
|
|
||||||
The old script may remain as a compatibility wrapper temporarily, but the reusable logic should live under `ba_auto/`.
|
|
||||||
|
|
||||||
### Phase 5: Mailbox migration
|
### Phase 5: Mailbox migration
|
||||||
|
|
||||||
**Status: Done.** Live testing surfaced a real bug: the old `MAILBOX_ICON` coordinate `(1726, 60)` sat on the edge of the icon's hitbox and intermittently missed, and the fixed click sequence had no way to notice — it cascaded into pressing Escape on the bare home screen, which triggers Blue Archive's own "exit the game?" confirmation (dismissed safely with Cancel during testing; no game state was lost). The Python port in `ba_auto/tasks/mailbox.py` fixes the coordinate and, following `module/mail.py`'s `rgb_in_range` pattern, verifies the panel actually opened (and whether "claim all" is disabled) via `driver.color_at` before pressing any further keys, with a bounded retry and a safe abort if the panel never appears.
|
**Status: Done.** Live testing surfaced a real bug: the old `MAILBOX_ICON` coordinate `(1726, 60)` sat on the edge of the icon's hitbox and intermittently missed, and the fixed click sequence had no way to notice — it cascaded into pressing Escape on the bare home screen, which triggers Blue Archive's own "exit the game?" confirmation (dismissed safely with Cancel during testing; no game state was lost). The Python port in `ba_auto/tasks/mailbox.py` fixes the coordinate and, following `module/mail.py`'s `rgb_in_range` pattern, verifies the panel actually opened (and whether "claim all" is disabled) via `driver.color_at` before pressing any further keys, with a bounded retry and a safe abort if the panel never appears.
|
||||||
@ -260,27 +256,54 @@ The CLI should call it through Python.
|
|||||||
|
|
||||||
### Phase 6: Cafe migration
|
### Phase 6: Cafe migration
|
||||||
|
|
||||||
**Status: Not started — `ba_auto/tasks/cafe.py` currently bridges to `scripts/ba_dailies_legacy.sh`.**
|
**Status: Done.** Same root cause as the mailbox bug (Phase 5), confirmed by step-by-step live replay with screenshots: `CAFE_ICON` clicks are flaky (missed on the first attempt, worked on retry at the identical coordinate — this is xdotool/Proton click-registration flakiness, not a coordinate-precision problem), and the old sequence had zero verification across its ~12 steps (open → dismiss notice → pat loop ×15 → switch room → dismiss notice → pat loop ×15 → claim income ×2 Enter → ×2 Escape). A missed click anywhere cascades into blind actions on whatever screen is actually showing, which — same as mailbox — very likely ends with an unverified Escape hitting the home screen and triggering Blue Archive's own exit-game confirmation.
|
||||||
|
|
||||||
Move cafe logic from Bash to:
|
`ba_auto/tasks/cafe.py` now verifies state at every transition using `driver.color_at`, following `module/cafe_reward.py`'s `picture.co_detect`/`rgb_in_range` pattern:
|
||||||
|
|
||||||
```
|
- opening the cafe icon and the room-switch button both retry (bounded) and confirm the panel actually opened via the same subscreen-header probe as mailbox, now shared in `ba_auto/navigation.is_on_subscreen`
|
||||||
ba_auto/tasks/cafe.py
|
- the "visited student list" notice that appears on every room entry is dismissed with Enter; this is harmless as a no-op if no popup is actually present (verified live), so no separate presence check was needed there
|
||||||
```
|
- the income dialog's own dimmed-overlay backdrop is checked (`navigation.is_modal_open`) before pressing Enter to claim, and the "receive" button's disabled-grey color is checked before attempting to claim at all (mirrors `collect()`'s `rgb_in_range` gate in the reference)
|
||||||
|
- the closing Escape(s) only fire when a subscreen/modal is confirmed still open, never blindly
|
||||||
|
|
||||||
During migration, verify:
|
Verified live (two full runs against the real game, plus a manual step-by-step replay of every transition):
|
||||||
|
|
||||||
- both rooms still work
|
- both rooms open and pat correctly
|
||||||
- sparkle detection still works
|
- sparkle detection still works and now runs in-process (see Phase 4) instead of shelling out per click
|
||||||
- cafe income claim still works
|
- cafe income claim works (confirmed gold +81,251 / AP +61 on an actual claim) and correctly no-ops when there's nothing to collect
|
||||||
- rank-up popups are handled or explicitly documented as not handled
|
- the reference's `zoom_out` step (camera zoom before sparkle detection — CLAUDE.md's "view centering/zoom" gap) was **not** ported: detection matched at 0.99 confidence without it in live testing, so it wasn't reproducibly broken here. Left as a documented open risk below rather than added speculatively.
|
||||||
- student rotation popups are handled or explicitly documented as not handled
|
|
||||||
- view centering/zoom state is robust
|
Not verified / open risks:
|
||||||
- repeated detection does not suffer from Python cold-start delay
|
|
||||||
|
- whether zoom/pan state could drift over a long unattended run and eventually break sparkle detection (see above — no evidence of this yet, but the reference project treats it as necessary)
|
||||||
|
|
||||||
|
#### Phase 6 follow-up: rank-up popups mid-pat-loop ("it will freeze a bit")
|
||||||
|
|
||||||
|
A user report during real usage: a pat that causes a bond-rank-up makes the loop "freeze a bit." Confirmed as a real, previously-unhandled gap, not a timing fluke — `find_cafe_sparkle()` was being asked to recognize a full-screen "絆ランクアップ!" cutscene (no cafe header, no chrome at all — see `screenshots/cafe/student/01`/`02`) as if it were the sparkle template, which it obviously never matches, so the loop just spun `driver.wait(1)` uselessly for the rest of the room's click budget. The reference's own `to_cafe()` navigation (`module/cafe_reward.py`) already treats `relationship_rank_up` as a recognized, reactively-dismissed popup checked after every pat round — this project's port had never carried that over.
|
||||||
|
|
||||||
|
Fix: `cafe.py`'s `_dismiss_rank_up_if_shown()`, called after every pat (click + Enter + move-mouse), reuses the *existing* `navigation.is_on_subscreen` header-brightness probe rather than adding a new one — directly confirmed against the user-provided screenshots: the header probe point reads `(183, 220, 240)` during the cutscene (r<200, fails the check) vs. `(248, 249, 250)` on the normal cafe screen (r>200, passes). Presses Enter (bounded, `config.CAFE_RANK_UP_DISMISS_RETRIES = 5`) until `is_on_subscreen` confirms the cafe room is back, rather than assuming one Enter is enough; if it never clears, the pat loop stops rather than continuing to click blindly.
|
||||||
|
|
||||||
|
Not yet live-confirmed against a real rank-up trigger — it's semi-random (tied to hitting an affection threshold) and didn't happen to occur during this session's testing. The fix is grounded in the user's own captured screenshots (a real observed state, precisely measured), not a guess, but a live run actually hitting this path and recovering cleanly is still open.
|
||||||
|
|
||||||
|
#### Phase 6 follow-up: "farming affection doesn't happen" report
|
||||||
|
|
||||||
|
A later report claimed pats weren't landing at all, with the original `ba_dailies.sh` `do_cafe_room`/`do_cafe` pasted as the expected-behavior reference. Re-reading that Bash carefully changed the diagnosis: the original `detect_and_click.py` did one screenshot → detect → click per invocation and the Bash loop only kept calling it back-to-back while hits kept landing, breaking immediately on the first miss (`grep -q "^MATCH" || break`) — i.e. give-up-on-first-miss was the *original design*, not a regression introduced by the Python port. Detection math (mask, threshold `0.97`, click offset `(75, 47)`) ported over byte-for-byte identical.
|
||||||
|
|
||||||
|
Changes made this round:
|
||||||
|
|
||||||
|
- `ba_auto/detector.py`: `find_cafe_sparkle()` now tries multiple template scales (`SPARKLE_SCALES`) instead of one fixed size, since the cafe camera's zoom isn't reset before farming and isn't guaranteed to match whatever zoom the template was captured at. Strictly more permissive than the original single-scale match — no observed downside — but not confirmed as the actual root cause of the report (no live zoom-mismatch case was reproduced/observed).
|
||||||
|
- `ba_auto/tasks/cafe.py`: `_pat_room` now polls for the full `CAFE_MAX_CLICKS_PER_ROOM` budget with a 1s wait between misses instead of breaking on the very first miss. This is a deliberate deviation from the original design (see above) — cheap (adds at most ~15s per room when nothing is available) and covers the case where a screenshot lands mid-animation right after the room transition.
|
||||||
|
- `driver.move_mouse` added and called after each pat to park the cursor away from the sparkle area, per `screenshots/cafe/sparkle/02_*_cursor_on_head.png` showing the cursor can occlude the icon.
|
||||||
|
|
||||||
|
What was directly verified live after these changes:
|
||||||
|
|
||||||
|
- the room-entry/no-modal state probes (`navigation.is_on_subscreen`, `is_modal_open`) read correctly on real captured frames from both rooms
|
||||||
|
- neither room had a visible sparkle on any student at the time of testing (confirmed by eye on the actual screenshots, not inferred from the "no sparkle found" log) — this is the most likely explanation for why a same-session automated run kept reporting no matches: this session's own manual+automated testing had already consumed the available per-student affection interactions, which regenerate on a real-world cooldown far longer than one room visit
|
||||||
|
|
||||||
|
**Not yet verified**: an actual end-to-end pat (detect → click → affection-up dialog dismissed) succeeding after this round's changes, because no interactable sparkle was available during testing to exercise it against. Re-run `~/ba_dailies.sh cafe` once interactions have had time to regenerate and confirm `[cafe] patted N sparkle(s)` appears with N > 0.
|
||||||
|
|
||||||
### Phase 7: setup.sh update
|
### Phase 7: setup.sh update
|
||||||
|
|
||||||
**Status: Done — `setup.sh` now deploys `ba_daily.py`, `ba_auto/`, and `scripts/ba_dailies_legacy.sh`.**
|
**Status: Done — `setup.sh` deploys `ba_daily.py` and `ba_auto/`.** `scripts/ba_dailies_legacy.sh` and `scripts/detect_and_click.py` were deleted once mailbox and cafe both migrated off them (Phases 5–6); `setup.sh` no longer references either.
|
||||||
|
|
||||||
Update `setup.sh` so it deploys:
|
Update `setup.sh` so it deploys:
|
||||||
|
|
||||||
@ -293,11 +316,103 @@ assets/
|
|||||||
|
|
||||||
to the expected runtime paths on `nik-gpu`.
|
to the expected runtime paths on `nik-gpu`.
|
||||||
|
|
||||||
|
### Phase 8: Stamina/AP mission claim
|
||||||
|
|
||||||
|
**Status: Partially done.** Read `module/collect_daily_task_power.py` (the "Tasks" menu claim loop — `to_tasks` + `rgb_in_range` checks against two fixed pixel pairs, click, dismiss, repeat) and `module/collect_daily_free_power.py` (a `picture.co_detect` state-machine walk into the Pyroxene Purchase menu's Package tab to claim a genuinely free 10 AP item). Live reconnaissance on the home screen found the direct local equivalents: a `ミッション` (Mission) icon opening a panel with per-tab claim buttons *and* a single bulk "一括受取" (claim all) button whose keyboard shortcut is literally Enter — much simpler than porting the reference's per-tab color-probe loop. `ba_auto/tasks/stamina.py` opens the Mission panel, checks whether "一括受取" is enabled (bright yellow vs. flat grey background probe at `config.MISSION_CLAIM_PROBE`), and if so presses Enter to claim, Enter again to dismiss the reward-reveal card (same "harmless no-op if absent" assumption as cafe's room-entry dismiss), bounded to a few rounds in case multiple rewards queue up.
|
||||||
|
|
||||||
|
The 青輝石購入 (Pyroxene Purchase) icon — reference's Daily Free Power entry point — was opened once to confirm the free-claim flow's location, but turned out to be a real-money purchase menu (¥3,000–¥4,900 package buttons visible immediately) with the genuinely-free item buried in a further tab. Given `plan.md`'s own purchase-safety rules ("avoid unbounded spending", "avoid buying unknown items"), this was **deliberately not automated this round** — the dialog was closed without navigating further. Treat this as a separate, explicitly-confirmed piece of future work, not an oversight.
|
||||||
|
|
||||||
|
Two real bugs found and fixed during live calibration, both worth remembering for future coordinate-hunting:
|
||||||
|
|
||||||
|
- **Visual gridline coordinate estimates were wrong twice in a row.** Reading icon bounds off a scaled/annotated screenshot crop by eye put the Mission icon's center at `(146, 352)` — which is actually in the dead space between the Mission and Pyroxene-Purchase icons, close enough to the latter's edge that clicks there landed on Pyroxene Purchase instead. The fix was sampling actual pixel colors along a scanline (`img.getpixel`) to find each icon's true left/right edge against the background, rather than eyeballing gridlines — this put the real center at `(75, 350)`, squarely inside the icon graphic, confirmed live. Lesson: for icon coordinates, prefer a pixel-boundary scan over a visual grid-overlay estimate.
|
||||||
|
- **`driver.click()`'s combined `xdotool mousemove X Y click 1` invocation is unreliable; splitting it fixed a chunk of this project's long-documented click flakiness.** Repeated single-click tests at a *verified-correct* coordinate still missed intermittently until the mousemove and click were issued as two separate `xdotool` calls with a short (0.2s) pause between them — after that, every subsequent click registered. This plausibly explains some of the "icon click missed on the first attempt, worked on retry" flakiness documented in Phases 5–6 (mailbox/cafe icons). Applied to `driver.click()` itself (project-wide, since all tasks share it) rather than special-cased in `stamina.py`; regression-tested live against `mailbox` and `cafe` after the change — both still work.
|
||||||
|
|
||||||
|
Not yet done: Group/Club AP, and Daily Free Power (see above).
|
||||||
|
|
||||||
|
### Phase 9: Normal/Hard story AP sweep
|
||||||
|
|
||||||
|
**Status: Done.** Read `module/explore_tasks/sweep_task.py` and `module/explore_tasks/task_utils.py` — the reference flow reads the current region number and matches stage-name text via OCR (`swipe_search_target_str`) to navigate to a configured target stage, then runs a per-stage claim loop. This client exposes a much simpler path to the same goal (burn AP via already-3-starred stages) that avoids porting the OCR-based lookup entirely: each stage's own 任務情報 (task info) modal has a self-contained 掃討 (sweep) sub-panel with a MIN/-/+/MAX count stepper and a start button.
|
||||||
|
|
||||||
|
Per explicit user direction on target selection: rather than a fixed configured stage (the plan's original "suggested first version"), `ba_auto/tasks/story_sweep.py` gets the *latest unlocked* region by spamming the "next region" arrow until it stops advancing (a plain state-change check, no OCR — clicking past the last region is a harmless no-op, verified live), then picks one of that region's stages essentially at random (`_pick_random_stage_row`: scroll the stage list to one of its two extremes at random, then pick a random one of the 4 visible rows there — not perfectly uniform since middle stages are reachable from both extremes, but avoids OCR/generic scroll-enumeration). AP spend is bounded by the in-game MAX button per explicit user direction (no additional cap layered on top).
|
||||||
|
|
||||||
|
Three real bugs were found and fixed during live calibration, all specific to the fact that this task spends real AP (unlike every other task so far, which only claims free rewards):
|
||||||
|
|
||||||
|
- **`navigation.is_modal_open`'s default probe `(960, 200)` false-negatives on this modal.** The 任務情報 modal is wide enough that `(960, 200)` lands on the modal's own white card, not the dimmed backdrop. Fixed with a task-specific `config.STAGE_MODAL_PROBE = (1870, 600)` and a local `_is_stage_modal_open()` check. First live test aborted safely on this false negative (correctly spent 0 AP) before the fix.
|
||||||
|
- **The MAX button click was never verified, and silently under-delivered.** A live run completed without error but only spent ~10 AP (one sweep) instead of the ~190 AP a real MAX (19 sweeps) should cost — diagnosed by comparing the actual AP/gold delta against the AP preview text seen during manual calibration. Root cause: the same general click-flakiness documented in Phase 8, just unverified here because nothing checked it. Fixed with `_count_raised_above_one()`: the sweep count's "-" stepper button is flat grey at the default count of 1 and turns vivid orange once raised, so probing `config.SWEEP_MINUS_BUTTON_PROBE` after the MAX click cheaply confirms it landed, without needing OCR on the count itself. Wrapped in a bounded retry (`MAX_BUTTON_RETRIES = 3`), aborting with zero AP spent if it never confirms. A subsequent live test hit 0/3 on this retry (a real flakiness cluster, not a logic bug) and correctly aborted without spending; the very next live run succeeded on attempt 1 with a genuine MAX (count 1→19, AP 191→1 confirmed by screenshot), so the retry+verify mechanism does its job on both sides — safe abort on failure, correct spend on success.
|
||||||
|
- **Escape does not close this modal, and the fallback dismiss loop was a latent hazard.** After a real sweep, the post-sweep dismiss loop pressed Enter a fixed number of times to clear reward-summary popups; live testing showed that once those popups run out, the *same* underlying 任務情報 modal reappears — and its Enter hotkey is bound to the live "任務開始" (start manual mission) button, not a no-op. The original fixed round count (3) happened to land exactly on the modal's reappearance without going further, but a different reward-popup count on another run could just as easily have pressed one Enter too many and started a real manual battle attempt. Two fixes: `_dismiss_sweep_result` now checks `_is_stage_modal_open` before every Enter press and stops immediately once the modal reappears, instead of trusting a fixed count; and closing now happens via a new `_close_stage_modal()` that clicks the modal's own X button (`config.STAGE_MODAL_CLOSE_BUTTON`, pinned via pixel-scanline scan of the glyph, not visual estimate) with a bounded retry+verify, since two Escape presses were confirmed live to leave the modal open. If the X-click ever fails to confirm closed, the task logs a warning and stops rather than pressing any further keys blindly.
|
||||||
|
|
||||||
|
Verified live: work-hub → task-screen navigation, latest-region advance, random stage pick, stage-modal-open detection, MAX click+verify, a genuine MAX sweep (19 runs, AP 191→1, gold +9,144), and the modal-close-via-X-button fix (confirmed via direct scripted click that it reliably closes and returns to the stage list). Not yet re-verified end-to-end in one single run: the fixed dismiss-loop-then-X-close sequence together, since AP was down to 1/240 after the successful test and there wasn't a further real sweep available to test against before the fix was deployed — each half was verified independently instead. Re-run `~/ba_dailies.sh story_sweep` once AP has regenerated to confirm the full sequence end-to-end.
|
||||||
|
|
||||||
|
`story_sweep` is deliberately **not** in `ba_daily.py`'s `DEFAULT_ORDER` — it spends AP on a randomly-picked stage rather than reclaiming something free, which is a real resource decision the default unattended run shouldn't make blindly. It must be invoked explicitly (`~/ba_dailies.sh story_sweep`).
|
||||||
|
|
||||||
|
**Retrospective — OCR avoidance was a mistake here.** Three of this phase's four live bugs (wrong modal probe, unverified MAX click, Escape-doesn't-close-modal plus the latent accidental-battle-start hazard) trace back to one decision: avoiding the reference's OCR-driven, deterministic stage targeting in favor of a heuristic substitute (random-pick + pixel-probes). A deterministic "go to configured stage X" flow, ported from the reference the way `module/explore_tasks/sweep_task.py`/`task_utils.py` actually do it, would not have needed to guess whether a modal opened via an easily-mismatched color probe, nor would it have left ambiguity about what's under the cursor when dismissing reward popups. This project's policy is now to port the reference's OCR-driven logic when the reference uses OCR for a feature, rather than inventing a non-OCR substitute to avoid the setup cost (see `CLAUDE.md` → "OCR policy"). `story_sweep.py`'s random-stage-pick design is not being reverted retroactively without user direction, but any future rework of this task should prefer porting the reference's actual region/stage-name OCR matching over the current random-pick approach.
|
||||||
|
|
||||||
|
### Phase 10: story_sweep OCR/state-machine port (supersedes Phase 9's random-pick design)
|
||||||
|
|
||||||
|
**Status: Done.** Acted on Phase 9's retrospective: set up OCR for real (`pytesseract` + the `tesseract-ocr` apt package) and ported the reference's actual deterministic stage targeting, replacing the random-pick heuristic. See `Handoff.md`'s history (deleted once this phase landed) for the full brief; summary of what changed:
|
||||||
|
|
||||||
|
- **OCR primitive**: `ba_auto/detector.py`'s `read_text()`/`read_int()` crop a screenshot to a pixel rect, threshold it to pure black/white (this measurably fixed real digit misreads that survived every `psm` mode when left anti-aliased — see below), upscale 3x, and run `pytesseract`. `lang="eng"` is enough; the region-number and stage-label reads are pure digits/dashes, no Japanese trained data needed.
|
||||||
|
- **Config-driven targets**: `config.STORY_SWEEP_TARGETS = [(region, stage, count_or_"max"), ...]`, mirroring the reference's `unfinished_normal_tasks` shape. Ships with a placeholder `(1, 1, "max")` entry per explicit user direction — edit it to your own already-cleared stage(s) before running for real.
|
||||||
|
- **Deterministic region navigation** (`_go_to_region`, porting `task_utils.py::to_region`): OCR the region-number readout, click the exact left/right-arrow delta, re-check, bounded loop. Region-arrow presence (locked/last-region detection) is a `detector.region_contains_color` box scan, not a single fixed point — a centroid-derived single point landed in the concave notch of the "<"/">" chevron and read "absent" even while the arrow was clearly rendered a few pixels away.
|
||||||
|
- **Deterministic stage search** (`_find_stage_row`, a scoped-down `swipe_search_target_str`): OCR each of the 4 visible stage-row labels at both already-calibrated scroll extremes, matching by the label's suffix after the dash (e.g. "2" in "30-2") rather than the full string — the font's leading region digit reads unreliably even after threshold preprocessing (e.g. "3" as "2"), but the suffix read correctly on every row tested, and the region digit is redundant anyway since `_go_to_region` already confirmed it independently.
|
||||||
|
- **Scoped `co_detect` port**: `navigation.wait_for_state(driver, config, reactions, ends, max_iterations)` — checks named `ends` first each iteration (stop, return the name), then named `reactions` (run an action, keep polling), else waits and retries up to a bound. Generic and reusable beyond this task.
|
||||||
|
- **Named outcomes**: `_sweep_target` returns `"swept"`, `"inadequate_ap"`, `"region_unavailable"`, `"stage_not_found"`, or `"unrecognized_state"` instead of a single generic "Done".
|
||||||
|
|
||||||
|
Four real, live-discovered findings, on top of what Phase 9 already found:
|
||||||
|
|
||||||
|
- **Regular numbered stages (30-1..30-5) render a different, taller modal layout than the "-A" bonus stage Phase 9 exclusively calibrated against.** Regular stages add a "集中指揮"/"簡易攻略" tab row and a manual "任務開始" panel below the sweep sub-panel that "-A" doesn't have. Phase 9's `SWEEP_MAX_BUTTON`/`SWEEP_MINUS_BUTTON_PROBE`/`SWEEP_START_BUTTON` all missed by ~40-46px vertically against a real numbered stage (30-3) — caught live when the MAX-click retry correctly failed 3/3 and aborted without spending AP, rather than silently misfiring. Re-calibrated against the tabbed layout via pixel-scanning (not eyeballing); the "-A" layout's original Phase 9 coordinates are no longer what these constants hold, so a future sweep of a "-A" stage specifically would need its own re-check.
|
||||||
|
- **The modal's own X-close button also moves with the layout.** Not just the sweep sub-panel — the whole card is vertically positioned by its own content height rather than anchored at a fixed absolute position, so the X button sits at a different absolute Y (225 vs Phase 9's 271) in the taller tabbed layout. Caught live: `_close_stage_modal` correctly reported "not closed" (3/3 retries) against the stale coordinate, rather than silently believing it had closed.
|
||||||
|
- **Clicking 掃討開始 always raises an AP-usage-confirmation dialog the design had never accounted for at all.** ("APを`N`使用して、掃討を`M`回行いますか?", OK/Cancel.) A first live attempt at porting the "wait for outcome" step read this dialog's dimmed backdrop as a false "inadequate_ap" through an early, unverified placeholder probe — worth remembering: an uncalibrated placeholder check can be actively *wrong*, not just inert, if given a chance to run before it's confirmed. Fixed by explicitly clicking through this confirmation before watching for the real result.
|
||||||
|
- **Genuine insufficient-AP is a visually near-identical dialog at the exact same OK-button position, told apart only by color.** Deliberately triggered live (by emptying the sweep count via MAX/"+" at low AP) rather than guessed: a real "AP不足" case shows a dialog titled "AP購入" (spend real Pyroxene to buy more AP) whose OK button is gold/yellow, vs. the safe usage-confirm's cyan — same position, different color. `_is_ap_purchase_prompt`/`_is_sweep_usage_confirm` tell them apart by that color and only ever click the cyan one; the gold one is always cancelled, never clicked, matching this project's purchase-safety rules.
|
||||||
|
|
||||||
|
Also fixed in passing, found only because live testing exercised the actual home-screen click path repeatedly: `config.TASK_CARD`'s old coordinate `(1370, 450)` sat close enough to the 任務 card's bottom edge that one run missed and landed on the "総力戦" (Total War) card below it instead — confirmed via screenshot, safely backed out with zero AP spent, moved to `(1250, 380)` (squarely on the "任務" title text).
|
||||||
|
|
||||||
|
Verified live end-to-end at least once, real AP spent: a genuine 5x sweep of 30-3 (AP 53→~5, confirmed via the "掃討完了" results screen's reward totals), including clicking through the usage-confirm dialog, the SKIP animation-skip screen, and the final reward-totals OK, landing back on the bare stage-info modal afterward. The genuine insufficient-AP path was also verified live (correctly cancelled the real "AP購入" purchase prompt without spending Pyroxene). Not yet re-verified end-to-end with the final rewritten code specifically (the color-based dynamic button-finding in `_watch_sweep_result`) at a nonzero AP balance — the manual walkthrough that discovered the dialogs used direct scripted clicks before the code was rewritten to match; the rewritten code's color-matching logic was separately verified offline against the exact screenshots captured live (all four dialog states correctly classified), but a fresh live run once AP regenerates would close that last gap. Not verified: sweeping a "-A" bonus stage (needs its own layout re-check, see above), an integer (non-"max") configured count actually being clicked via `SWEEP_PLUS_BUTTON`, and Hard-mode tab stages.
|
||||||
|
|
||||||
|
**Phase 10 follow-up (user-reported):** the user ran `story_sweep` for real and it spent AP on region 1 stage 1 instead of region 30 (their actual current last region). Not a code bug — `config.STORY_SWEEP_TARGETS` still held the literal placeholder `(1, 1, "max")` shipped with this phase, and the user hadn't edited it yet. Rather than just filling in one static `(30, N, "max")` entry, the user asked for the stage within region 30 to rotate daily across all 6 of that region's stages instead of grinding one fixed stage every run. Added `config.STORY_SWEEP_ROTATION_REGION`/`STORY_SWEEP_ROTATION_STAGE_COUNT`/`STORY_SWEEP_ROTATION_COUNT` and `story_sweep._rotation_target()`, which computes `(region, stage, count)` from `datetime.date.today().toordinal() % stage_count` — a plain date-ordinal modulo rather than calendar day-of-year, so the 6-day cycle doesn't skip or repeat around a year boundary. This target is appended to (not a replacement for) whatever's in `STORY_SWEEP_TARGETS`, which is now empty by default. Verified the computed target offline (region 30, stage 6, on the date this was fixed) but not yet re-run against the live game since AP hadn't regenerated.
|
||||||
|
|
||||||
|
### Phase 11: Common Shop + Tactical Shop
|
||||||
|
|
||||||
|
**Status: Done.** Ported `module/shop/common_shop.py` / `module/shop/tactical_challenge_shop.py`'s `implement()` and the shared `module/shop/shop_utils.py` (`to_common_shop`, `get_item_position`/`ensure_choose`/`buy`) to `ba_auto/tasks/shop_common.py` / `shop_tactical.py`, sharing control flow through a new `ba_auto/tasks/shop_utils.py`.
|
||||||
|
|
||||||
|
Key design call, made after live-capturing both shop tabs before writing any code: the reference's own item identification inside the grid is **not** OCR-based. `get_item_position` scans fixed pixel columns and matches a purchasable-state color plus a currency-icon template, then maps that grid position to an item identity via `self.static_config.common_shop_price_list` — a table sourced from an external resource this repo doesn't contain (the reference dataclass just declares the field; the actual values are fetched elsewhere, at BAAS's own runtime). So porting this feature couldn't mean "OCR the item names" (the reference doesn't do that either) — it meant building our own local equivalent of that static table by live-capturing the real catalog and letting the user pick their buy list from it, same shape as `STORY_SWEEP_TARGETS`. `config.COMMON_SHOP_TARGETS` / `config.TACTICAL_SHOP_TARGETS` are `(row, col, item name (comment only), expected price)` tuples; identification is by fixed grid position, with price-digit OCR (something the reference doesn't even do per-item) layered on as an extra live-catalog-drift safety net, consistent with this project's verify-before-spend pattern elsewhere.
|
||||||
|
|
||||||
|
What was found live, captured before writing any code (see the session's live-capture screenshots, not kept in the repo):
|
||||||
|
|
||||||
|
- **Both shop tabs share one UI**: a 4-column checkbox grid per item, then a single bulk "購入" (Buy) button that appears once ≥1 item is checked, rather than the reference's per-item purchase flow. Checked state renders a distinct vivid yellow-green on the checkbox glyph, easily told apart from the plain white/grey unchecked state by `detector.region_contains_color` — no template matching needed.
|
||||||
|
- **The tactical shop's tab list (7 entries) fits on screen with no scrolling**, so `config.SHOP_TAB_TACTICAL` is a fixed click rather than a port of the reference's `goto_shop_by_name` OCR swipe-search — there's nothing to search for on this account, so a fixed click is the faithful choice here, not a shortcut around OCR (see `CLAUDE.md`'s OCR policy: only skip OCR where the reference's own need for it doesn't apply).
|
||||||
|
- **Price-digit OCR needed real calibration**, same as Phase 10's stage labels: a rect wide enough for the widest configured price (500,000) without also catching the neighboring column's card, and narrow enough on the left to exclude the currency icon (which OCR otherwise misreads as a spurious leading digit — confirmed live, e.g. the top-bar credit balance read "454920755" instead of "154920755" until the icon was excluded).
|
||||||
|
- **One shared corner-pixel probe (`config.SHOP_OVERLAY_PROBE`, a bottom-left point) detects both the purchase-confirm dialog and the post-purchase "報酬獲得!" (reward acquired) banner** — both dim it away from pure white; it stays pure white with nothing open, confirmed stable across tab switches and scrolling. `shop_utils.confirm_purchase` presses Enter in a bounded loop until it reads idle again, rather than tracking each dialog's own layout individually.
|
||||||
|
- **Live-tested with real purchases, not just offline-verified.** With the user's explicit buy lists confirmed first (8 Common Shop items: 初級/中級/上級/最上級レポート + 初級/中級/上級/最上級強化珠; 2 Tactical Shop items: 初級/中級栄養ドリンク), both shops were run for real: Common Shop total cost matched the pre-calculated 1,211,500 credits exactly (confirmed against the game's own confirmation-dialog total); Tactical Shop's AP gain (+90) and coin spend (-45) matched exactly. Both purchases resolved cleanly back to an idle screen via the overlay-probe loop.
|
||||||
|
- **Discovered live, not anticipated going in: these shop items have a per-refresh-cycle purchase cap that isn't shown as a visible counter** (unlike the Pyroxene-shop tab's explicit "あと1回購入可能" labels — Common Shop items just look normal until you've already bought them, then go quietly unresponsive). Found by re-running the actual `shop_common` CLI task shortly after the manual purchase above: every configured item's checkbox failed to register as checked, and a direct test of the individual per-item "購入" button and the page's own "全て選択" (select-all) control confirmed those specific items are genuinely non-interactive right now (select-all successfully picked up *other*, not-yet-purchased items further down the list), while credits stayed unchanged throughout. The task correctly reported "nothing selected, cancelling" and spent nothing, rather than misfiring — a real edge case the price-verify + checkbox-confirm design caught safely, not a bug in it. Net effect: a fully "fresh, everything-available" unattended run hasn't been re-verified end-to-end today, since this account's cycle allowance for these specific items was already spent via the same session's manual calibration. Next run after the shop's own refresh timer (~5h, shown in-game as "更新まで") should exercise that path for real.
|
||||||
|
- **Not verified**: non-fully-visible target rows requiring a scroll (both current buy lists happen to be fully visible without scrolling, so `shop_utils` has no scroll/pagination logic yet — would need porting `buy()`'s `last_checked_idx`/swipe-diff tracking if a future buy list needs it); the pre-purchase "insufficient assets" abort path (both currencies were comfortably sufficient this run); the paid manual shop-refresh flow (`更新` button) — deliberately not automated for v1, same reasoning as Daily Free Power (real-currency spend needs explicit human intent, not a default unattended path).
|
||||||
|
|
||||||
|
### Phase 12: Lesson/Schedule
|
||||||
|
|
||||||
|
**Status: Done.** Ported `module/lesson.py`'s control flow (`implement`, the `to_*` navigation state machine, `get_lesson_each_region_status`/`get_lesson_relationship_counts`, `choose_lesson`, `execute_lesson`) to `ba_auto/tasks/lesson.py`.
|
||||||
|
|
||||||
|
Scope, decided with the user before writing any code (see the two `AskUserQuestion` answers): affection-first selection (mirrors `lesson_relationship_first=True` — matches this feature's own "affection farming" framing, not raw reward-tier grinding), sweep every unlocked region in a fixed order until either lesson tickets or scoreable lessons run out (no per-region target list needed from the user, unlike shop's per-item buy list), no lesson-ticket purchasing and no favor-student targeting (both deferred, matching plan.md's original "Suggested first version").
|
||||||
|
|
||||||
|
Key finding before writing any code: **this client's UI is structurally the same two-level layout the reference describes (12 named regions, each with up to 9 individual lesson locations) but rendered completely differently** — a scrollable list of regions instead of the reference's paged single-region swipe view, and a clean "すべてのスケジュール" grid-card modal instead of the reference's raw isometric map + `Parallelogram`/`Triangle` pixel-scanned status grid. Two consequences:
|
||||||
|
|
||||||
|
- **No OCR is needed for region navigation at all.** The reference OCRs the current region name because its paged arrows leave position ambiguous. This client's region list only ever settles at two scroll positions (scrolled to top: regions 0-5; scrolled to bottom: regions 6-11, confirmed live — repeated scroll-down clicks don't keep scrolling past this), so navigation is just "scroll to the right state, click the row at a fixed Y" — deterministic, nothing to locate. The reference's own 12 JP region names (`core/config/default_config.py`'s `lesson_region_name.JP` — embedded directly there, not fetched externally like the shop price table) are kept locally purely for log readability, confirmed to match this account's list exactly, row for row.
|
||||||
|
- **No isometric geometry is needed for per-cell status either.** Each grid-modal card shows up to 3 student portraits with a heart-shaped affection-count badge — reading that number via OCR needs no geometry at all, replacing the reference's isometric pixel-scan outright.
|
||||||
|
|
||||||
|
Three real bugs were found and fixed via a live test run (5 real tickets spent across 3 regions, ticket accounting and cleanup navigation both confirmed correct throughout):
|
||||||
|
|
||||||
|
- **A checkmark does not blank the badge number — it renders alongside it.** The initial design assumed an "already done today" portrait would fail the digit-whitelisted OCR read (no number left to read), the same way a "no relationship yet" portrait does, and treated both as identically "not a candidate." Live testing showed this was wrong: a done portrait keeps its unchanged number and gets a small green checkmark added at top-right instead. Without separately checking for that checkmark, the same already-done cell could be re-picked immediately after being completed. Fixed by detecting the checkmark directly via its own fixed color/offset (`config.LESSON_GRID_CHECKMARK_*`, `lesson._is_slot_already_done`) rather than inferring done-ness from the OCR read.
|
||||||
|
- **The badge's own OCR read was unreliable, and not for the reason first suspected.** Reusing the project's existing generic `detector.read_int` (grayscale + hard threshold, tuned for this UI's normal dark-text-on-light-card look) on the pink/magenta heart badge produced wildly oversized results live — "13" read as "113", "19" as "119", "18" as "418". Root cause, confirmed by rendering the exact same threshold step locally: the heart's own darker outline stroke has a grayscale value that happens to land on the same side of the threshold as the digit glyph, surviving as stray black marks that tesseract sometimes fuses into extra leading digits. Fixed with a dedicated `detector.read_int_on_heart_badge`, which masks on the color relationship "R < G" (true for the navy digit glyph in every sample, false for every pink/magenta badge tone, light or dark) instead of raw brightness — this alone fixed most cases. A few slots still occasionally fuse a stray digit from the character's own portrait art bleeding into the crop's edge (this is character-art-dependent, not fixable by OCR config alone — confirmed reproducible across every psm mode tried). Rather than chase perfect crop geometry per-character, `config.LESSON_GRID_BADGE_MAX_PLAUSIBLE` discards any reading of 100+ as certain contamination (real affection values never reach that range in practice) — a second line of defense, not the primary fix.
|
||||||
|
- **A transient "2x schedule" campaign event (active on this account, ~6 days remaining) doubles how many intermediate screens appear after starting a lesson**, and a bond-rank-up cutscene can appear too (same full-screen "絆ランクアップ!" style already handled in `cafe.py`, confirmed live here for the first time against a real trigger). Rather than special-case either, `lesson._run_one_schedule` presses Enter in a bounded loop, checking two fixed markers each round: the grid modal's own title underline color (visible only when it's frontmost and idle — both the results modal and the cutscene cover it, confirmed live against screenshots of all three states) as the "done" signal, and the results modal's OK button color (shared with the info panel's Start button, confirmed by direct pixel sample) as the one intermediate state worth clicking precisely rather than folding into the blind-Enter fallback.
|
||||||
|
|
||||||
|
Also confirmed live: the "保有チケット N/M" ticket counter is directly visible on the region-list screen (no submenu needed, unlike the reference's `to_purchase_lesson_ticket`/OCR-a-modal approach) — `lesson._read_ticket_count` just OCRs it directly and re-reads it after every schedule to track spend, rather than assuming exactly 1 ticket per schedule (the 2x campaign event doesn't change the ticket cost, only the reward/cutscene count, but re-reading rather than assuming keeps this robust either way).
|
||||||
|
|
||||||
|
**Not verified**: a region with more than 9 currently-unlocked locations (would need scroll support inside the grid modal — not implemented, not yet seen on this account, both regions tested topped out at 7-8); the "no lesson tickets" abort-immediately path (tickets hit exactly 0 mid-sweep during the real test, not at the start); a full 12-region sweep in one run (the test's 5 tickets ran out partway through region 3 of 12); a truly fresh "everything available, nothing done yet" run (this account had already done some lessons manually during calibration before the automated run started). Worth re-running `~/ba_dailies.sh lesson` after tickets next refill to exercise the untested tail of the region list.
|
||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
### OCR
|
### OCR
|
||||||
|
|
||||||
Not set up yet.
|
**Set up (Phase 10): `pytesseract` + the `tesseract-ocr` apt package.** `ba_auto/detector.py`'s `read_text()`/`read_int()` wrap it for occasional single-crop reads (a region number, a stage label) — no need for the reference's own socket/shared-memory PaddleOCR server, which exists there to make OCR fast across thousands of automation steps; this project's usage volume doesn't need that.
|
||||||
|
|
||||||
Needed for:
|
Needed for:
|
||||||
|
|
||||||
@ -314,8 +429,6 @@ Candidates:
|
|||||||
- Tesseract
|
- Tesseract
|
||||||
- PaddleOCR
|
- PaddleOCR
|
||||||
|
|
||||||
Do not add OCR until a feature needs it.
|
|
||||||
|
|
||||||
### Auto-fight primitive
|
### Auto-fight primitive
|
||||||
|
|
||||||
Needed for:
|
Needed for:
|
||||||
@ -355,11 +468,9 @@ This should become a reusable primitive, not arena-specific code.
|
|||||||
|
|
||||||
### 1. Migration to Python-first
|
### 1. Migration to Python-first
|
||||||
|
|
||||||
This is the highest priority.
|
**Status: Done.** See Phases 1–7 above for the detailed history, including the mailbox and cafe exit-game-dialog bug and its fix.
|
||||||
|
|
||||||
Do this before adding new features.
|
Goal (all done):
|
||||||
|
|
||||||
Goal:
|
|
||||||
|
|
||||||
- Bash launcher only
|
- Bash launcher only
|
||||||
- Python CLI
|
- Python CLI
|
||||||
@ -369,20 +480,14 @@ Goal:
|
|||||||
- cafe migrated
|
- cafe migrated
|
||||||
- reference mapping started
|
- reference mapping started
|
||||||
|
|
||||||
Reference:
|
|
||||||
|
|
||||||
- Current local implementation
|
|
||||||
- Existing `ba_dailies.sh`
|
|
||||||
- Existing `scripts/detect_and_click.py`
|
|
||||||
|
|
||||||
### 2. Stamina/AP sweep
|
### 2. Stamina/AP sweep
|
||||||
|
|
||||||
|
**Status: Partially done — see Phase 8.**
|
||||||
|
|
||||||
Claim:
|
Claim:
|
||||||
|
|
||||||
- daily free AP purchase
|
- ~~daily free AP purchase~~ — deferred; entry point is a real-money purchase menu, needs explicit confirmation before automating
|
||||||
- daily task-menu AP/pyroxene rewards
|
- daily task-menu AP/pyroxene rewards — done, via the Mission panel's bulk claim button
|
||||||
|
|
||||||
Likely mostly color/state detection and fixed clicks.
|
|
||||||
|
|
||||||
Reference:
|
Reference:
|
||||||
|
|
||||||
@ -393,7 +498,7 @@ Reference:
|
|||||||
|
|
||||||
Local target: `ba_auto/tasks/stamina.py`
|
Local target: `ba_auto/tasks/stamina.py`
|
||||||
|
|
||||||
OCR: Probably not needed for first version.
|
OCR: Not needed — turned out to be a single bulk-claim button + Enter, no per-item detection required.
|
||||||
|
|
||||||
### 3. Club/Group AP claim
|
### 3. Club/Group AP claim
|
||||||
|
|
||||||
@ -411,6 +516,8 @@ OCR: Not expected.
|
|||||||
|
|
||||||
### 4. Normal/Hard story AP sweep
|
### 4. Normal/Hard story AP sweep
|
||||||
|
|
||||||
|
**Status: Done — see Phase 10 (supersedes Phase 9's random-pick design).**
|
||||||
|
|
||||||
Sweep already-cleared main story stages to burn AP.
|
Sweep already-cleared main story stages to burn AP.
|
||||||
|
|
||||||
Reference:
|
Reference:
|
||||||
@ -422,19 +529,13 @@ Reference:
|
|||||||
|
|
||||||
Local target: `ba_auto/tasks/story_sweep.py`
|
Local target: `ba_auto/tasks/story_sweep.py`
|
||||||
|
|
||||||
OCR: Likely needed for current region/stage detection unless using fixed configured targets.
|
OCR: Used, for real, as of Phase 10 — region-number readout OCR + delta-click (porting `to_region`), and stage-row label OCR matching (a scoped-down `swipe_search_target_str`), replacing Phase 9's "next region arrow stops advancing, then a random stage" heuristic. See Phase 10's write-up for what was actually found live (a second, taller modal layout for regular numbered stages; an AP-usage-confirmation dialog the design hadn't accounted for; a genuine insufficient-AP dialog told apart from it only by button color).
|
||||||
|
|
||||||
Suggested first version:
|
Implemented version:
|
||||||
|
|
||||||
- user-configured fixed stage
|
- exact configured `(region, stage, count)` targets (`config.STORY_SWEEP_TARGETS`), not a fixed single stage nor a random pick
|
||||||
- no region search
|
- `count` is `"max"` or a specific int (the latter calibrated but not yet live-clicked — see Phase 10)
|
||||||
- no dynamic OCR
|
- opt-in only, not in the default daily flow
|
||||||
- sweep configured mission only
|
|
||||||
|
|
||||||
Later version:
|
|
||||||
|
|
||||||
- fuzzy stage/region selection
|
|
||||||
- OCR-assisted navigation
|
|
||||||
|
|
||||||
### 5. Bounty
|
### 5. Bounty
|
||||||
|
|
||||||
@ -465,7 +566,7 @@ Reference:
|
|||||||
|
|
||||||
Local target: `ba_auto/tasks/commission.py`
|
Local target: `ba_auto/tasks/commission.py`
|
||||||
|
|
||||||
OCR: Probably avoidable for first version if configured fixed sweep target is used.
|
OCR: Port the reference's approach if it uses OCR here — do not default to a fixed-target workaround just to avoid OCR (see "OCR policy" in `CLAUDE.md` and the Phase 9 retrospective above).
|
||||||
|
|
||||||
### 7. Arena
|
### 7. Arena
|
||||||
|
|
||||||
@ -489,6 +590,8 @@ Arena is high-value but more risky than fixed claim tasks.
|
|||||||
|
|
||||||
### 8. Common Shop + Tactical Shop
|
### 8. Common Shop + Tactical Shop
|
||||||
|
|
||||||
|
**Status: Done — see Phase 11.**
|
||||||
|
|
||||||
Auto-buy configured items.
|
Auto-buy configured items.
|
||||||
|
|
||||||
Reference:
|
Reference:
|
||||||
@ -504,18 +607,18 @@ Local targets:
|
|||||||
```
|
```
|
||||||
ba_auto/tasks/shop_common.py
|
ba_auto/tasks/shop_common.py
|
||||||
ba_auto/tasks/shop_tactical.py
|
ba_auto/tasks/shop_tactical.py
|
||||||
|
ba_auto/tasks/shop_utils.py
|
||||||
```
|
```
|
||||||
|
|
||||||
Needs:
|
Implemented version:
|
||||||
|
|
||||||
- OCR for currency balances
|
- OCR for currency balances (top-bar credits, in-panel tactical coin) — done
|
||||||
- shop tab detection
|
- shop tab detection — done via a fixed click (tactical tab list fits on screen with no scroll needed on this account, confirmed live); no scroll/pagination logic yet since both current buy lists are fully visible without scrolling
|
||||||
- configured buy list
|
- configured buy list — done, `config.COMMON_SHOP_TARGETS`/`config.TACTICAL_SHOP_TARGETS`, fixed `(row, col, name, expected_price)` per item (see Phase 11 for why identification is by grid position + price-OCR-verify rather than per-item OCR — the reference doesn't OCR item names here either)
|
||||||
- safe purchase confirmation logic
|
- safe purchase confirmation logic — done, a single overlay-darkness probe covers both the confirm dialog and the reward-acquired banner
|
||||||
|
- no-refresh (the paid manual `更新` refresh button is deliberately not automated, same reasoning as Daily Free Power)
|
||||||
|
|
||||||
Start with a no-refresh, fixed configured buy list.
|
### 9. Lesson / Schedule — Done (Phase 12)
|
||||||
|
|
||||||
### 9. Lesson / Schedule
|
|
||||||
|
|
||||||
Affection farming via classes.
|
Affection farming via classes.
|
||||||
|
|
||||||
@ -525,22 +628,18 @@ Reference:
|
|||||||
~/repo/baas-reference/module/lesson.py
|
~/repo/baas-reference/module/lesson.py
|
||||||
```
|
```
|
||||||
|
|
||||||
Local target: `ba_auto/tasks/lesson.py`
|
Local target: `ba_auto/tasks/lesson.py` — implemented, live-tested with real tickets. See Phase 12 above for the full writeup.
|
||||||
|
|
||||||
Complexity: High
|
- region/area identification — done without OCR: this client's region list only settles at two fixed scroll positions, so navigation is deterministic index-based clicking, not the reference's OCR-a-name-then-page approach
|
||||||
|
- multi-page swipe search — not needed for the same reason
|
||||||
|
- student detection/portrait matching for specific students — deferred, per the original suggested scope below
|
||||||
|
- isometric grid location logic — done without porting the reference's geometry: per-cell status/affection reads via `detector.read_int_on_heart_badge` OCR + a checkmark color probe instead
|
||||||
|
|
||||||
Needs:
|
Suggested first version (as originally scoped, and what shipped):
|
||||||
|
|
||||||
- OCR for region/area names
|
- pick a fixed region — expanded to all 12, swept in order
|
||||||
- multi-page swipe search
|
- select available/highest visible lesson — affection-first, per explicit user direction
|
||||||
- student detection/portrait matching if targeting specific students
|
- avoid favorite-student targeting at first — still deferred
|
||||||
- isometric grid location logic if following reference fully
|
|
||||||
|
|
||||||
Suggested first version:
|
|
||||||
|
|
||||||
- pick a fixed region
|
|
||||||
- select available/highest visible lesson
|
|
||||||
- avoid favorite-student targeting at first
|
|
||||||
|
|
||||||
## Low priority backlog
|
## Low priority backlog
|
||||||
|
|
||||||
@ -714,7 +813,7 @@ server = JP
|
|||||||
game_window_name = BlueArchive
|
game_window_name = BlueArchive
|
||||||
display = :0
|
display = :0
|
||||||
asset_dir = ~/ba_assets
|
asset_dir = ~/ba_assets
|
||||||
screenshot_dir = ./scratchpad
|
screenshot_dir = scratchpad/
|
||||||
cafe_max_clicks_per_room
|
cafe_max_clicks_per_room
|
||||||
story_sweep_target
|
story_sweep_target
|
||||||
shop_buy_list
|
shop_buy_list
|
||||||
@ -727,7 +826,7 @@ Keep config explicit. Do not bury user-specific settings deep inside task logic.
|
|||||||
|
|
||||||
## Debugging conventions
|
## Debugging conventions
|
||||||
|
|
||||||
Use `./scratchpad` for:
|
Use `scratchpad/` for:
|
||||||
|
|
||||||
- temporary screenshots
|
- temporary screenshots
|
||||||
- cropped templates
|
- cropped templates
|
||||||
@ -772,21 +871,23 @@ should run the default daily sequence.
|
|||||||
|
|
||||||
## Near-term recommended task order
|
## Near-term recommended task order
|
||||||
|
|
||||||
1. Rewrite `ba_dailies.sh` as a thin launcher.
|
1. Rewrite `ba_dailies.sh` as a thin launcher. — Done
|
||||||
2. Add `ba_daily.py`.
|
2. Add `ba_daily.py`. — Done
|
||||||
3. Add `ba_auto/driver.py`.
|
3. Add `ba_auto/driver.py`. — Done
|
||||||
4. Add `ba_auto/detector.py`.
|
4. Add `ba_auto/detector.py`. — Done
|
||||||
5. Add `ba_auto/navigation.py`.
|
5. Add `ba_auto/navigation.py`. — Done
|
||||||
6. Move mailbox logic to `ba_auto/tasks/mailbox.py`.
|
6. Move mailbox logic to `ba_auto/tasks/mailbox.py`. — Done
|
||||||
7. Move cafe logic to `ba_auto/tasks/cafe.py`.
|
7. Move cafe logic to `ba_auto/tasks/cafe.py`. — Done
|
||||||
8. Update `setup.sh`.
|
8. Update `setup.sh`. — Done
|
||||||
9. Add `ba_auto/reference_notes/mapping.md`.
|
9. Add `ba_auto/reference_notes/mapping.md`. — Done
|
||||||
10. Verify existing mailbox and cafe still work.
|
10. Verify existing mailbox and cafe still work. — Done
|
||||||
11. Implement stamina/AP.
|
11. Implement stamina/AP. — Done (Phase 8)
|
||||||
12. Implement group/club AP.
|
12. Implement Normal/Hard story AP sweep. — Done (Phase 9 built a random-pick heuristic; Phase 10 replaced it with the reference's actual OCR-based deterministic stage targeting, per Phase 9's own retrospective)
|
||||||
13. Implement fixed-target sweep features.
|
13. Implement group/club AP. — Not started
|
||||||
14. Add OCR only when needed.
|
14. Set up OCR and port it for whichever remaining feature's reference implementation depends on it — not a blanket "only when needed" deferral; see `CLAUDE.md` → "OCR policy"
|
||||||
15. Attempt Arena/Shop/Lesson after the framework is stable.
|
15. Implement Common Shop + Tactical Shop. — Done (Phase 11)
|
||||||
|
16. Implement Lesson/Schedule. — Done (Phase 12)
|
||||||
|
17. Attempt Arena once OCR is in place, since its reference implementation depends on it.
|
||||||
|
|
||||||
## Claude Code guidance summary
|
## Claude Code guidance summary
|
||||||
|
|
||||||
@ -796,5 +897,6 @@ When Claude Code works on this repo, it should follow this rule:
|
|||||||
> Python first.
|
> Python first.
|
||||||
> Driver primitives before feature hacks.
|
> Driver primitives before feature hacks.
|
||||||
> Bash launcher only.
|
> Bash launcher only.
|
||||||
|
> Port OCR when the reference uses it — don't invent non-OCR substitutes to avoid the setup cost.
|
||||||
|
|
||||||
Do not turn this project into a Bash recreation of Blue Archive Auto Script.
|
Do not turn this project into a Bash recreation of Blue Archive Auto Script.
|
||||||
|
|||||||
@ -1,122 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
# Legacy pre-migration click sequences; bridged from ba_auto/tasks/*.py until ported to Python (see CLAUDE.md).
|
|
||||||
export DISPLAY=:0
|
|
||||||
export XAUTHORITY=/run/user/1000/gdm/Xauthority
|
|
||||||
|
|
||||||
WIN_NAME="BlueArchive"
|
|
||||||
MAILBOX_ICON="1726 60"
|
|
||||||
CLAIM_ALL="1691 1128"
|
|
||||||
|
|
||||||
CAFE_ICON="165 1100"
|
|
||||||
CAFE_ROOM_SWITCH="190 160"
|
|
||||||
CAFE_INCOME="1780 1105"
|
|
||||||
VENV_PYTHON="$HOME/.venvs/ba-auto-daily/bin/python3"
|
|
||||||
DETECT_SPARKLE="$HOME/ba_scripts/detect_and_click.py"
|
|
||||||
CAFE_MAX_CLICKS_PER_ROOM=15
|
|
||||||
|
|
||||||
get_window() {
|
|
||||||
xdotool search --name "$WIN_NAME" | head -1
|
|
||||||
}
|
|
||||||
|
|
||||||
click() {
|
|
||||||
local coords="$1"
|
|
||||||
xdotool mousemove $coords click 1
|
|
||||||
sleep 0.5
|
|
||||||
}
|
|
||||||
|
|
||||||
press_enter() {
|
|
||||||
xdotool key Return
|
|
||||||
sleep 0.5
|
|
||||||
}
|
|
||||||
|
|
||||||
press_esc() {
|
|
||||||
xdotool key Escape
|
|
||||||
sleep 0.5
|
|
||||||
}
|
|
||||||
|
|
||||||
focus_game() {
|
|
||||||
local win
|
|
||||||
win=$(get_window)
|
|
||||||
if [ -z "$win" ]; then
|
|
||||||
echo "ERROR: Blue Archive window not found. Is the game running?"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
xdotool windowactivate "$win"
|
|
||||||
sleep 0.5
|
|
||||||
}
|
|
||||||
|
|
||||||
do_mailbox() {
|
|
||||||
echo "[mailbox] Opening mailbox..."
|
|
||||||
click "$MAILBOX_ICON"
|
|
||||||
sleep 1.5
|
|
||||||
echo "[mailbox] Claiming all..."
|
|
||||||
click "$CLAIM_ALL"
|
|
||||||
sleep 1
|
|
||||||
echo "[mailbox] Confirming claim..."
|
|
||||||
press_enter
|
|
||||||
sleep 1
|
|
||||||
echo "[mailbox] Closing mailbox..."
|
|
||||||
press_esc
|
|
||||||
sleep 1
|
|
||||||
echo "[mailbox] Done."
|
|
||||||
}
|
|
||||||
|
|
||||||
do_cafe_room() {
|
|
||||||
local i
|
|
||||||
for ((i = 0; i < CAFE_MAX_CLICKS_PER_ROOM; i++)); do
|
|
||||||
if ! "$VENV_PYTHON" "$DETECT_SPARKLE" | grep -q "^MATCH"; then
|
|
||||||
break
|
|
||||||
fi
|
|
||||||
sleep 1
|
|
||||||
press_enter
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
do_cafe() {
|
|
||||||
echo "[cafe] Opening cafe..."
|
|
||||||
click "$CAFE_ICON"
|
|
||||||
sleep 3
|
|
||||||
press_enter
|
|
||||||
echo "[cafe] Room 1: farming affection..."
|
|
||||||
do_cafe_room
|
|
||||||
echo "[cafe] Switching to room 2..."
|
|
||||||
click "$CAFE_ROOM_SWITCH"
|
|
||||||
sleep 3
|
|
||||||
press_enter
|
|
||||||
echo "[cafe] Room 2: farming affection..."
|
|
||||||
do_cafe_room
|
|
||||||
echo "[cafe] Claiming cafe income..."
|
|
||||||
click "$CAFE_INCOME"
|
|
||||||
sleep 2
|
|
||||||
press_enter
|
|
||||||
sleep 2
|
|
||||||
press_enter
|
|
||||||
sleep 2
|
|
||||||
echo "[cafe] Exiting cafe..."
|
|
||||||
press_esc
|
|
||||||
sleep 1.5
|
|
||||||
press_esc
|
|
||||||
sleep 1.5
|
|
||||||
echo "[cafe] Done."
|
|
||||||
}
|
|
||||||
|
|
||||||
case "$1" in
|
|
||||||
mailbox)
|
|
||||||
focus_game
|
|
||||||
do_mailbox
|
|
||||||
;;
|
|
||||||
cafe)
|
|
||||||
focus_game
|
|
||||||
do_cafe
|
|
||||||
;;
|
|
||||||
"")
|
|
||||||
focus_game
|
|
||||||
do_mailbox
|
|
||||||
do_cafe
|
|
||||||
echo "All done."
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
echo "Unknown phase: $1"
|
|
||||||
exit 1
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
@ -1,69 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Screenshot the game window, find a cafe affection sparkle, click it.
|
|
||||||
|
|
||||||
Runs entirely on nik-gpu (screenshot -> detect -> click all local) so the
|
|
||||||
whole cycle finishes in well under a second - roaming students move fast
|
|
||||||
enough that a multi-hop SSH/scp pipeline misses the click.
|
|
||||||
|
|
||||||
Prints "MATCH x y score" and exits 0 if a sparkle was found and clicked,
|
|
||||||
or prints "NO_MATCH" and exits 1 otherwise.
|
|
||||||
"""
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
|
|
||||||
import cv2
|
|
||||||
import numpy as np
|
|
||||||
|
|
||||||
TEMPLATE_PATH = "/home/nik/ba_assets/cafe_sparkle.png"
|
|
||||||
SHOT_PATH = "/tmp/ba_live.png"
|
|
||||||
OFFSET_X = 75
|
|
||||||
OFFSET_Y = 47
|
|
||||||
THRESHOLD = 0.97
|
|
||||||
|
|
||||||
ENV = {"DISPLAY": ":0", "XAUTHORITY": "/run/user/1000/gdm/Xauthority"}
|
|
||||||
|
|
||||||
|
|
||||||
def screenshot():
|
|
||||||
subprocess.run(
|
|
||||||
["scrot", "-a", "0,0,1920,1200", "-o", SHOT_PATH],
|
|
||||||
env=ENV, check=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def find_sparkles():
|
|
||||||
template = cv2.imread(TEMPLATE_PATH)
|
|
||||||
b, g, r = cv2.split(template.astype(np.int16))
|
|
||||||
yellow_white = ((r > 180) & (g > 140) & (r - b > 60)) | ((r > 200) & (g > 200) & (b > 200))
|
|
||||||
mask_plane = (yellow_white.astype(np.uint8)) * 255
|
|
||||||
mask = cv2.merge([mask_plane, mask_plane, mask_plane])
|
|
||||||
th, tw = template.shape[:2]
|
|
||||||
|
|
||||||
img = cv2.imread(SHOT_PATH)
|
|
||||||
result = cv2.matchTemplate(img, template, cv2.TM_CCORR_NORMED, mask=mask)
|
|
||||||
locs = np.where(result >= THRESHOLD)
|
|
||||||
points = sorted(zip(*locs[::-1]), key=lambda p: -result[p[1], p[0]])
|
|
||||||
|
|
||||||
merged = []
|
|
||||||
for x, y in points:
|
|
||||||
if all(abs(x - mx) > tw // 2 or abs(y - my) > th // 2 for mx, my, _ in merged):
|
|
||||||
merged.append((x, y, result[y, x]))
|
|
||||||
return [(x + tw // 2, y + th // 2, score) for x, y, score in merged]
|
|
||||||
|
|
||||||
|
|
||||||
def click(x, y):
|
|
||||||
subprocess.run(
|
|
||||||
["xdotool", "mousemove", str(x), str(y), "click", "1"],
|
|
||||||
env=ENV, check=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
screenshot()
|
|
||||||
sparkles = find_sparkles()
|
|
||||||
if not sparkles:
|
|
||||||
print("NO_MATCH")
|
|
||||||
sys.exit(1)
|
|
||||||
x, y, score = sparkles[0]
|
|
||||||
click(x + OFFSET_X, y + OFFSET_Y)
|
|
||||||
print(f"MATCH {x} {y} {score:.4f}")
|
|
||||||
sys.exit(0)
|
|
||||||
31
setup.sh
31
setup.sh
@ -5,13 +5,12 @@
|
|||||||
# not from macOS, since it installs into nik-gpu-local paths.
|
# not from macOS, since it installs into nik-gpu-local paths.
|
||||||
#
|
#
|
||||||
# It does NOT touch the game or take any screenshots; it only installs the
|
# It does NOT touch the game or take any screenshots; it only installs the
|
||||||
# venv and copies files to the fixed paths ba_dailies.sh/ba_daily.py/
|
# venv and copies files to the fixed paths ba_dailies.sh/ba_daily.py/ba_auto
|
||||||
# detect_and_click.py expect (outside the repo, per the two-machine deploy
|
# expect (outside the repo, per the two-machine deploy convention in
|
||||||
# convention in CLAUDE.md).
|
# CLAUDE.md).
|
||||||
set -e
|
set -e
|
||||||
|
|
||||||
VENV_DIR="$HOME/.venvs/ba-auto-daily"
|
VENV_DIR="$HOME/.venvs/ba-auto-daily"
|
||||||
SCRIPTS_DIR="$HOME/ba_scripts"
|
|
||||||
ASSETS_DIR="$HOME/ba_assets"
|
ASSETS_DIR="$HOME/ba_assets"
|
||||||
|
|
||||||
echo "== Checking host tools =="
|
echo "== Checking host tools =="
|
||||||
@ -28,19 +27,29 @@ if [ "$missing" = 1 ]; then
|
|||||||
fi
|
fi
|
||||||
echo "xdotool, scrot: OK"
|
echo "xdotool, scrot: OK"
|
||||||
|
|
||||||
|
if ! command -v tesseract >/dev/null 2>&1; then
|
||||||
|
echo "MISSING: tesseract (OCR engine binary -- e.g. sudo apt install tesseract-ocr)"
|
||||||
|
echo "story_sweep's region/stage-label OCR (see CLAUDE.md \"OCR policy\") needs this."
|
||||||
|
echo "This requires a password-interactive sudo, so it isn't installed for you here --"
|
||||||
|
echo "install it yourself, then re-run this script."
|
||||||
|
missing=1
|
||||||
|
fi
|
||||||
|
if [ "$missing" = 1 ]; then
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "tesseract: OK"
|
||||||
|
|
||||||
echo "== Setting up Python venv at $VENV_DIR =="
|
echo "== Setting up Python venv at $VENV_DIR =="
|
||||||
if [ ! -x "$VENV_DIR/bin/python3" ]; then
|
if [ ! -x "$VENV_DIR/bin/python3" ]; then
|
||||||
python3 -m venv "$VENV_DIR"
|
python3 -m venv "$VENV_DIR"
|
||||||
fi
|
fi
|
||||||
"$VENV_DIR/bin/pip" install --quiet --upgrade pip
|
"$VENV_DIR/bin/pip" install --quiet --upgrade pip
|
||||||
"$VENV_DIR/bin/pip" install --quiet opencv-python-headless numpy
|
"$VENV_DIR/bin/pip" install --quiet opencv-python-headless numpy pytesseract
|
||||||
"$VENV_DIR/bin/python3" -c "import cv2, numpy; print('opencv', cv2.__version__, '/ numpy', numpy.__version__)"
|
"$VENV_DIR/bin/python3" -c "import cv2, numpy; print('opencv', cv2.__version__, '/ numpy', numpy.__version__)"
|
||||||
|
"$VENV_DIR/bin/python3" -c "import pytesseract; print('pytesseract', pytesseract.get_tesseract_version())"
|
||||||
|
|
||||||
echo "== Deploying scripts + assets to fixed paths =="
|
echo "== Deploying assets to fixed paths =="
|
||||||
mkdir -p "$SCRIPTS_DIR" "$ASSETS_DIR"
|
mkdir -p "$ASSETS_DIR"
|
||||||
cp scripts/detect_and_click.py "$SCRIPTS_DIR/detect_and_click.py"
|
|
||||||
cp scripts/ba_dailies_legacy.sh "$SCRIPTS_DIR/ba_dailies_legacy.sh"
|
|
||||||
chmod +x "$SCRIPTS_DIR/ba_dailies_legacy.sh"
|
|
||||||
cp assets/cafe_sparkle.png "$ASSETS_DIR/cafe_sparkle.png"
|
cp assets/cafe_sparkle.png "$ASSETS_DIR/cafe_sparkle.png"
|
||||||
|
|
||||||
echo "== Deploying Python-first entry point =="
|
echo "== Deploying Python-first entry point =="
|
||||||
@ -51,5 +60,5 @@ rm -rf "$HOME/ba_auto"
|
|||||||
cp -r ba_auto "$HOME/ba_auto"
|
cp -r ba_auto "$HOME/ba_auto"
|
||||||
|
|
||||||
echo "== Done =="
|
echo "== Done =="
|
||||||
echo "Run with: ~/ba_dailies.sh [mailbox|cafe]"
|
echo "Run with: ~/ba_dailies.sh [mailbox|cafe|stamina|story_sweep|shop_common|shop_tactical|lesson]"
|
||||||
echo "(requires the game already running, window titled 'BlueArchive')"
|
echo "(requires the game already running, window titled 'BlueArchive')"
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user