Nik Afiq
a04ef5d0c5
- Changed references from './scratchpad' to '.scratchpad/' in graph.json and plan.md for consistency. - Expanded Phase 6 follow-up section in plan.md to clarify changes made to the pat detection logic: - Updated `find_cafe_sparkle()` to utilize multiple template scales for improved detection. - Modified `_pat_room` to allow polling for maximum clicks instead of breaking on the first miss. - Added mouse movement after each pat to prevent cursor occlusion of sparkles. - Verified that room entry and modal state checks function correctly, but end-to-end pat success remains untested due to lack of available interactions.
804 lines
26 KiB
Markdown
804 lines
26 KiB
Markdown
# ba-auto-daily implementation plan
|
||
|
||
Personal Blue Archive JP daily-automation project.
|
||
|
||
This project controls the PC/Steam/Proton Blue Archive client running on `nik-gpu` through local desktop automation:
|
||
|
||
- xdotool
|
||
- scrot
|
||
- Python
|
||
- OpenCV
|
||
- later OCR when needed
|
||
|
||
Development happens on `nik-macbookair`.
|
||
|
||
The reference implementation lives at:
|
||
|
||
```
|
||
~/repo/baas-reference/
|
||
```
|
||
|
||
The reference project should be treated as the behavioral blueprint. This project should avoid recreating feature logic from scratch when the reference already implements it.
|
||
|
||
## Core project direction
|
||
|
||
This project is now Python-first.
|
||
|
||
The goal is not to grow a large Bash script.
|
||
|
||
The goal is to build a small local Python automation framework that adapts the reference project's Blue Archive logic to this user's unique PC/Steam/Proton environment.
|
||
|
||
`ba_dailies.sh` should only be a launcher.
|
||
|
||
Feature logic should live in Python.
|
||
|
||
## Target architecture
|
||
|
||
```
|
||
~/repo/ba-auto-daily/
|
||
├── ba_dailies.sh
|
||
├── ba_daily.py
|
||
├── ba_auto/
|
||
│ ├── __init__.py
|
||
│ ├── config.py
|
||
│ ├── driver.py
|
||
│ ├── detector.py
|
||
│ ├── navigation.py
|
||
│ ├── tasks/
|
||
│ │ ├── __init__.py
|
||
│ │ ├── mailbox.py
|
||
│ │ ├── cafe.py
|
||
│ │ ├── stamina.py
|
||
│ │ ├── group.py
|
||
│ │ ├── bounty.py
|
||
│ │ ├── commission.py
|
||
│ │ ├── arena.py
|
||
│ │ ├── shop_common.py
|
||
│ │ ├── shop_tactical.py
|
||
│ │ ├── lesson.py
|
||
│ │ └── ...
|
||
│ └── reference_notes/
|
||
│ └── mapping.md
|
||
├── assets/
|
||
├── screenshots/
|
||
├── scripts/
|
||
├── setup.sh
|
||
├── CLAUDE.md
|
||
└── plan.md
|
||
```
|
||
|
||
This is the intended direction. It does not have to be completed all at once.
|
||
|
||
## Project layout
|
||
|
||
| Path | What it is |
|
||
|---|---|
|
||
| `~/repo/ba-auto-daily/ba_dailies.sh` | Thin launcher only. It should call the Python entry point. Do not add new feature logic here. |
|
||
| `~/repo/ba-auto-daily/ba_daily.py` | Main Python CLI entry point. Dispatches tasks such as mailbox, cafe, stamina, group, etc. |
|
||
| `~/repo/ba-auto-daily/ba_auto/driver.py` | Local PC/Steam/Proton control backend. Wraps xdotool, scrot, waits, clicks, swipes, keypresses, screenshots, and window focus. |
|
||
| `~/repo/ba-auto-daily/ba_auto/detector.py` | OpenCV/template/color matching helpers. Currently has `find_cafe_sparkle()`, ported in-process from the retired `scripts/detect_and_click.py`. |
|
||
| `~/repo/ba-auto-daily/ba_auto/navigation.py` | Shared navigation/state-probe helpers: `is_on_subscreen`, `is_modal_open`, used by both `mailbox.py` and `cafe.py`. |
|
||
| `~/repo/ba-auto-daily/ba_auto/tasks/` | Feature implementations. Each task should adapt the relevant `baas-reference/module/...` logic where possible. |
|
||
| `~/repo/ba-auto-daily/ba_auto/reference_notes/mapping.md` | Reference mapping table: local feature → reference module → local implementation → driver gaps. |
|
||
| `~/repo/ba-auto-daily/assets/` | Locally captured template images, such as cafe sparkle. Do not blindly copy assets from the reference repo. |
|
||
| `~/repo/ba-auto-daily/screenshots/` | Human reference screenshots, mostly Moonlight/game captures, used for calibration and debugging. |
|
||
| `~/repo/ba-auto-daily/setup.sh` | Bootstrap/deploy helper for `nik-gpu`. Should install/check dependencies and copy runtime files. |
|
||
| `~/repo/baas-reference/` | Read-only GPL-3.0 reference clone. Study and adapt. Never edit. |
|
||
|
||
## Runtime paths on nik-gpu
|
||
|
||
Preferred runtime layout:
|
||
|
||
| Path | What it is |
|
||
|---|---|
|
||
| `nik-gpu:~/ba_dailies.sh` | Thin launcher. |
|
||
| `nik-gpu:~/ba_daily.py` | Python CLI entry point. |
|
||
| `nik-gpu:~/ba_auto/` | Python package copied from this repo. |
|
||
| `nik-gpu:~/ba_assets/` | Runtime assets/templates. |
|
||
| `nik-gpu:~/.venvs/ba-auto-daily/` | Python virtual environment. |
|
||
|
||
`nik-gpu:~/ba_scripts/` may still contain `detect_and_click.py` and `ba_dailies_legacy.sh` left over from before both mailbox and cafe were migrated off them. Neither is deployed or referenced by anything anymore (`setup.sh` stopped copying them once Phase 6 landed) — safe to delete manually on `nik-gpu`, just not automated here.
|
||
|
||
## Implementation strategy
|
||
|
||
For each feature:
|
||
|
||
1. Read the matching `~/repo/baas-reference/module/...` file.
|
||
2. Summarize the reference flow.
|
||
3. Identify reusable logic:
|
||
- state checks
|
||
- retry loops
|
||
- navigation sequence
|
||
- battle/sweep/shop rules
|
||
- detection method
|
||
- failure handling
|
||
4. Identify backend-specific calls that cannot be reused directly.
|
||
5. Implement missing generic primitives in `ba_auto/driver.py` or `ba_auto/detector.py`.
|
||
6. Implement the feature in `ba_auto/tasks/<feature>.py`.
|
||
7. Add CLI command dispatch in `ba_daily.py`.
|
||
8. Keep `ba_dailies.sh` unchanged unless launcher behavior changes.
|
||
9. Test syntax locally.
|
||
10. Deploy to `nik-gpu`.
|
||
11. Run against the live game.
|
||
12. Update this plan.
|
||
|
||
The intended result is not a Bash automation script.
|
||
|
||
The intended result is a Python automation framework using the reference repository as the behavioral blueprint.
|
||
|
||
## Reference mapping table
|
||
|
||
Maintain this table in `ba_auto/reference_notes/mapping.md`.
|
||
|
||
Initial seed:
|
||
|
||
| Local feature | Reference file | Reference functions/classes | Local file | Backend replacements | Status |
|
||
|---|---|---|---|---|---|
|
||
| Mailbox | Need to confirm in reference | Need to inspect | `ba_auto/tasks/mailbox.py` | tap/click via xdotool, screenshot via scrot | Existing Bash behavior; migrate to Python |
|
||
| Cafe | `module/cafe_reward.py` | `to_cafe`, `interaction_for_cafe_solve_method3`, `collect` | `ba_auto/tasks/cafe.py` | `picture.co_detect`/`color.rgb_in_range` → `driver.color_at` pixel-probe checks; sparkle template match ported in-process into `ba_auto/detector.py` | Migrated: real Python, state-verified via color probes, no legacy bridge |
|
||
| Stamina/AP | `module/collect_daily_free_power.py`, `module/collect_daily_task_power.py` | Need to inspect | `ba_auto/tasks/stamina.py` | color checks/clicks via local driver | Not started |
|
||
| Group/Club AP | `module/group.py` | Need to inspect | `ba_auto/tasks/group.py` | fixed click + state check via local driver | Not started |
|
||
| Bounty | `module/rewarded_task.py` | Need to inspect | `ba_auto/tasks/bounty.py` | sweep/color/OCR adaptation | Not started |
|
||
| Commissions | `module/clear_special_task_power.py` | Need to inspect | `ba_auto/tasks/commission.py` | sweep/color adaptation | Not started |
|
||
| Arena | `module/arena.py` | Need to inspect | `ba_auto/tasks/arena.py` | auto-fight + OCR + local driver | Not started |
|
||
| Common Shop | `module/shop/common_shop.py`, `module/shop/shop_utils.py` | Need to inspect | `ba_auto/tasks/shop_common.py` | OCR + tab navigation + local clicks | Not started |
|
||
| Tactical Shop | `module/shop/tactical_challenge_shop.py`, `module/shop/shop_utils.py` | Need to inspect | `ba_auto/tasks/shop_tactical.py` | OCR + tab navigation + local clicks | Not started |
|
||
| Lesson/Schedule | `module/lesson.py` | Need to inspect | `ba_auto/tasks/lesson.py` | OCR + template/portrait search + local driver | Not started |
|
||
|
||
Do not implement a feature without filling at least the relevant row.
|
||
|
||
## Status snapshot
|
||
|
||
| Feature | Current status | Target status |
|
||
|---|---|---|
|
||
| Mailbox claim | Migrated: `ba_auto/tasks/mailbox.py` uses `driver.color_at` to verify the panel opened before acting (found live-testing bug: a marginal icon coordinate could miss and cascade into pressing Escape on the home screen, which triggers Blue Archive's own exit-game confirmation) | Done |
|
||
| Cafe pats + income | Migrated: `ba_auto/tasks/cafe.py` verifies each room/dialog opened via `driver.color_at` before acting; sparkle detection now runs in-process via `ba_auto/detector.py` instead of a per-click subprocess | Done |
|
||
| Shared driver | `ba_auto/driver.py` built (`run_command`, `focus_game`, `click`, `keypress`, `screenshot`, `wait`, `color_at`); wired into both `mailbox.py` and `cafe.py` | Extend with new primitives as future tasks need them |
|
||
| Python CLI | Built: `ba_daily.py` dispatches `mailbox`/`cafe`/default flow | Extend as new tasks are added |
|
||
| Reference mapping | Built: `ba_auto/reference_notes/mapping.md` | Fill in reference file/function columns per feature |
|
||
| Everything else | Not started | Implement reference-first in Python |
|
||
|
||
## Migration phase
|
||
|
||
Before adding new game features, migrate the existing working implementation.
|
||
|
||
### Phase 1: Python skeleton
|
||
|
||
**Status: Done.**
|
||
|
||
Create:
|
||
|
||
```
|
||
ba_daily.py
|
||
ba_auto/__init__.py
|
||
ba_auto/config.py
|
||
ba_auto/driver.py
|
||
ba_auto/detector.py
|
||
ba_auto/navigation.py
|
||
ba_auto/tasks/__init__.py
|
||
ba_auto/tasks/mailbox.py
|
||
ba_auto/tasks/cafe.py
|
||
ba_auto/reference_notes/mapping.md
|
||
```
|
||
|
||
### Phase 2: Launcher
|
||
|
||
**Status: Done.**
|
||
|
||
Change `ba_dailies.sh` into a thin launcher:
|
||
|
||
```bash
|
||
#!/usr/bin/env bash
|
||
set -euo pipefail
|
||
VENV_PYTHON="${VENV_PYTHON:-$HOME/.venvs/ba-auto-daily/bin/python3}"
|
||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||
exec "$VENV_PYTHON" "$SCRIPT_DIR/ba_daily.py" "$@"
|
||
```
|
||
|
||
Keep compatibility with:
|
||
|
||
```
|
||
./ba_dailies.sh
|
||
./ba_dailies.sh mailbox
|
||
./ba_dailies.sh cafe
|
||
```
|
||
|
||
### Phase 3: Driver extraction
|
||
|
||
**Status: Done.** Primitives (including `color_at`, added during the mailbox/cafe hardening work) are wired into both `ba_auto/tasks/mailbox.py` and `ba_auto/tasks/cafe.py`.
|
||
|
||
Move shell interactions into `ba_auto/driver.py`.
|
||
|
||
Driver primitives should include:
|
||
|
||
```
|
||
focus_game()
|
||
click(x, y)
|
||
double_click(x, y)
|
||
keypress(key)
|
||
screenshot(path=None)
|
||
swipe(...)
|
||
wait(seconds)
|
||
wait_until(...)
|
||
```
|
||
|
||
### Phase 4: Detector extraction
|
||
|
||
**Status: Done (scoped).** `scripts/detect_and_click.py`'s sparkle-matching logic (masked template match against `assets/cafe_sparkle.png`) was ported into `ba_auto/detector.py` as `find_cafe_sparkle()`, called in-process from `ba_auto/tasks/cafe.py` — this removed the old per-click Python cold start (a fresh `cv2`/`numpy` import per subprocess call) that CLAUDE.md's driver-layer guidance specifically warns against. `scripts/detect_and_click.py` had no remaining callers once this landed, so it was deleted rather than kept as a compatibility wrapper. The more generic primitives listed below (`load_template`, `match_template`, etc.) have not been built — only the one concrete sparkle-matching function needed so far exists; generalize when a second detector use case actually needs it.
|
||
|
||
Detector primitives, generalize later if needed:
|
||
|
||
```
|
||
load_template(...)
|
||
match_template(...)
|
||
find_best_match(...)
|
||
find_and_click_template(...)
|
||
color_mask(...)
|
||
debug_write_match(...)
|
||
```
|
||
|
||
### Phase 5: Mailbox migration
|
||
|
||
**Status: Done.** Live testing surfaced a real bug: the old `MAILBOX_ICON` coordinate `(1726, 60)` sat on the edge of the icon's hitbox and intermittently missed, and the fixed click sequence had no way to notice — it cascaded into pressing Escape on the bare home screen, which triggers Blue Archive's own "exit the game?" confirmation (dismissed safely with Cancel during testing; no game state was lost). The Python port in `ba_auto/tasks/mailbox.py` fixes the coordinate and, following `module/mail.py`'s `rgb_in_range` pattern, verifies the panel actually opened (and whether "claim all" is disabled) via `driver.color_at` before pressing any further keys, with a bounded retry and a safe abort if the panel never appears.
|
||
|
||
Move mailbox logic from Bash to:
|
||
|
||
```
|
||
ba_auto/tasks/mailbox.py
|
||
```
|
||
|
||
The CLI should call it through Python.
|
||
|
||
### Phase 6: Cafe migration
|
||
|
||
**Status: Done.** Same root cause as the mailbox bug (Phase 5), confirmed by step-by-step live replay with screenshots: `CAFE_ICON` clicks are flaky (missed on the first attempt, worked on retry at the identical coordinate — this is xdotool/Proton click-registration flakiness, not a coordinate-precision problem), and the old sequence had zero verification across its ~12 steps (open → dismiss notice → pat loop ×15 → switch room → dismiss notice → pat loop ×15 → claim income ×2 Enter → ×2 Escape). A missed click anywhere cascades into blind actions on whatever screen is actually showing, which — same as mailbox — very likely ends with an unverified Escape hitting the home screen and triggering Blue Archive's own exit-game confirmation.
|
||
|
||
`ba_auto/tasks/cafe.py` now verifies state at every transition using `driver.color_at`, following `module/cafe_reward.py`'s `picture.co_detect`/`rgb_in_range` pattern:
|
||
|
||
- opening the cafe icon and the room-switch button both retry (bounded) and confirm the panel actually opened via the same subscreen-header probe as mailbox, now shared in `ba_auto/navigation.is_on_subscreen`
|
||
- the "visited student list" notice that appears on every room entry is dismissed with Enter; this is harmless as a no-op if no popup is actually present (verified live), so no separate presence check was needed there
|
||
- the income dialog's own dimmed-overlay backdrop is checked (`navigation.is_modal_open`) before pressing Enter to claim, and the "receive" button's disabled-grey color is checked before attempting to claim at all (mirrors `collect()`'s `rgb_in_range` gate in the reference)
|
||
- the closing Escape(s) only fire when a subscreen/modal is confirmed still open, never blindly
|
||
|
||
Verified live (two full runs against the real game, plus a manual step-by-step replay of every transition):
|
||
|
||
- both rooms open and pat correctly
|
||
- sparkle detection still works and now runs in-process (see Phase 4) instead of shelling out per click
|
||
- cafe income claim works (confirmed gold +81,251 / AP +61 on an actual claim) and correctly no-ops when there's nothing to collect
|
||
- the reference's `zoom_out` step (camera zoom before sparkle detection — CLAUDE.md's "view centering/zoom" gap) was **not** ported: detection matched at 0.99 confidence without it in live testing, so it wasn't reproducibly broken here. Left as a documented open risk below rather than added speculatively.
|
||
|
||
Not verified / open risks:
|
||
|
||
- rank-up popups: not observed during testing (no student ranked up while testing), still unhandled if one appears mid-loop
|
||
- whether zoom/pan state could drift over a long unattended run and eventually break sparkle detection (see above — no evidence of this yet, but the reference project treats it as necessary)
|
||
|
||
#### Phase 6 follow-up: "farming affection doesn't happen" report
|
||
|
||
A later report claimed pats weren't landing at all, with the original `ba_dailies.sh` `do_cafe_room`/`do_cafe` pasted as the expected-behavior reference. Re-reading that Bash carefully changed the diagnosis: the original `detect_and_click.py` did one screenshot → detect → click per invocation and the Bash loop only kept calling it back-to-back while hits kept landing, breaking immediately on the first miss (`grep -q "^MATCH" || break`) — i.e. give-up-on-first-miss was the *original design*, not a regression introduced by the Python port. Detection math (mask, threshold `0.97`, click offset `(75, 47)`) ported over byte-for-byte identical.
|
||
|
||
Changes made this round:
|
||
|
||
- `ba_auto/detector.py`: `find_cafe_sparkle()` now tries multiple template scales (`SPARKLE_SCALES`) instead of one fixed size, since the cafe camera's zoom isn't reset before farming and isn't guaranteed to match whatever zoom the template was captured at. Strictly more permissive than the original single-scale match — no observed downside — but not confirmed as the actual root cause of the report (no live zoom-mismatch case was reproduced/observed).
|
||
- `ba_auto/tasks/cafe.py`: `_pat_room` now polls for the full `CAFE_MAX_CLICKS_PER_ROOM` budget with a 1s wait between misses instead of breaking on the very first miss. This is a deliberate deviation from the original design (see above) — cheap (adds at most ~15s per room when nothing is available) and covers the case where a screenshot lands mid-animation right after the room transition.
|
||
- `driver.move_mouse` added and called after each pat to park the cursor away from the sparkle area, per `screenshots/cafe/sparkle/02_*_cursor_on_head.png` showing the cursor can occlude the icon.
|
||
|
||
What was directly verified live after these changes:
|
||
|
||
- the room-entry/no-modal state probes (`navigation.is_on_subscreen`, `is_modal_open`) read correctly on real captured frames from both rooms
|
||
- neither room had a visible sparkle on any student at the time of testing (confirmed by eye on the actual screenshots, not inferred from the "no sparkle found" log) — this is the most likely explanation for why a same-session automated run kept reporting no matches: this session's own manual+automated testing had already consumed the available per-student affection interactions, which regenerate on a real-world cooldown far longer than one room visit
|
||
|
||
**Not yet verified**: an actual end-to-end pat (detect → click → affection-up dialog dismissed) succeeding after this round's changes, because no interactable sparkle was available during testing to exercise it against. Re-run `~/ba_dailies.sh cafe` once interactions have had time to regenerate and confirm `[cafe] patted N sparkle(s)` appears with N > 0.
|
||
|
||
### Phase 7: setup.sh update
|
||
|
||
**Status: Done — `setup.sh` deploys `ba_daily.py` and `ba_auto/`.** `scripts/ba_dailies_legacy.sh` and `scripts/detect_and_click.py` were deleted once mailbox and cafe both migrated off them (Phases 5–6); `setup.sh` no longer references either.
|
||
|
||
Update `setup.sh` so it deploys:
|
||
|
||
```
|
||
ba_dailies.sh
|
||
ba_daily.py
|
||
ba_auto/
|
||
assets/
|
||
```
|
||
|
||
to the expected runtime paths on `nik-gpu`.
|
||
|
||
## Prerequisites
|
||
|
||
### OCR
|
||
|
||
Not set up yet.
|
||
|
||
Needed for:
|
||
|
||
- currency readouts
|
||
- ticket counts
|
||
- region/tab name matching
|
||
- some shop logic
|
||
- some lesson/schedule logic
|
||
- arena ticket/rank/level checks
|
||
- bounty coin balance if auto-refresh is implemented
|
||
|
||
Candidates:
|
||
|
||
- Tesseract
|
||
- PaddleOCR
|
||
|
||
Do not add OCR until a feature needs it.
|
||
|
||
### Auto-fight primitive
|
||
|
||
Needed for:
|
||
|
||
- Arena
|
||
- future main story push
|
||
- some battle automation
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/main_story.py
|
||
```
|
||
|
||
Look for:
|
||
|
||
```
|
||
auto_fight
|
||
enter_battle
|
||
```
|
||
|
||
Target local module may be:
|
||
|
||
```
|
||
ba_auto/tasks/battle.py
|
||
```
|
||
|
||
or:
|
||
|
||
```
|
||
ba_auto/battle.py
|
||
```
|
||
|
||
This should become a reusable primitive, not arena-specific code.
|
||
|
||
## High priority backlog
|
||
|
||
### 1. Migration to Python-first
|
||
|
||
**Status: Done.** See Phases 1–7 above for the detailed history, including the mailbox and cafe exit-game-dialog bug and its fix.
|
||
|
||
Goal (all done):
|
||
|
||
- Bash launcher only
|
||
- Python CLI
|
||
- Python driver
|
||
- Python detector
|
||
- mailbox migrated
|
||
- cafe migrated
|
||
- reference mapping started
|
||
|
||
### 2. Stamina/AP sweep
|
||
|
||
Claim:
|
||
|
||
- daily free AP purchase
|
||
- daily task-menu AP/pyroxene rewards
|
||
|
||
Likely mostly color/state detection and fixed clicks.
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/collect_daily_free_power.py
|
||
~/repo/baas-reference/module/collect_daily_task_power.py
|
||
```
|
||
|
||
Local target: `ba_auto/tasks/stamina.py`
|
||
|
||
OCR: Probably not needed for first version.
|
||
|
||
### 3. Club/Group AP claim
|
||
|
||
Claim AP from club/group.
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/group.py
|
||
```
|
||
|
||
Local target: `ba_auto/tasks/group.py`
|
||
|
||
OCR: Not expected.
|
||
|
||
### 4. Normal/Hard story AP sweep
|
||
|
||
Sweep already-cleared main story stages to burn AP.
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/explore_tasks/sweep_task.py
|
||
~/repo/baas-reference/module/explore_tasks/task_utils.py
|
||
```
|
||
|
||
Local target: `ba_auto/tasks/story_sweep.py`
|
||
|
||
OCR: Likely needed for current region/stage detection unless using fixed configured targets.
|
||
|
||
Suggested first version:
|
||
|
||
- user-configured fixed stage
|
||
- no region search
|
||
- no dynamic OCR
|
||
- sweep configured mission only
|
||
|
||
Later version:
|
||
|
||
- fuzzy stage/region selection
|
||
- OCR-assisted navigation
|
||
|
||
### 5. Bounty
|
||
|
||
Three sub-areas and sweep availability.
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/rewarded_task.py
|
||
```
|
||
|
||
Local target: `ba_auto/tasks/bounty.py`
|
||
|
||
OCR: Optional for coin balance/refresh logic. Can skip refresh for first version.
|
||
|
||
### 6. Commissions
|
||
|
||
Two sub-dungeons:
|
||
|
||
- Base Defense
|
||
- Item Retrieval
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/clear_special_task_power.py
|
||
```
|
||
|
||
Local target: `ba_auto/tasks/commission.py`
|
||
|
||
OCR: Probably avoidable for first version if configured fixed sweep target is used.
|
||
|
||
### 7. Arena
|
||
|
||
Fight until out of tickets or until configured stopping condition.
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/arena.py
|
||
```
|
||
|
||
Local target: `ba_auto/tasks/arena.py`
|
||
|
||
Needs:
|
||
|
||
- auto-fight primitive
|
||
- OCR or visual detection for ticket/rank state
|
||
- careful safety limits
|
||
|
||
Arena is high-value but more risky than fixed claim tasks.
|
||
|
||
### 8. Common Shop + Tactical Shop
|
||
|
||
Auto-buy configured items.
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/shop/common_shop.py
|
||
~/repo/baas-reference/module/shop/tactical_challenge_shop.py
|
||
~/repo/baas-reference/module/shop/shop_utils.py
|
||
```
|
||
|
||
Local targets:
|
||
|
||
```
|
||
ba_auto/tasks/shop_common.py
|
||
ba_auto/tasks/shop_tactical.py
|
||
```
|
||
|
||
Needs:
|
||
|
||
- OCR for currency balances
|
||
- shop tab detection
|
||
- configured buy list
|
||
- safe purchase confirmation logic
|
||
|
||
Start with a no-refresh, fixed configured buy list.
|
||
|
||
### 9. Lesson / Schedule
|
||
|
||
Affection farming via classes.
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/lesson.py
|
||
```
|
||
|
||
Local target: `ba_auto/tasks/lesson.py`
|
||
|
||
Complexity: High
|
||
|
||
Needs:
|
||
|
||
- OCR for region/area names
|
||
- multi-page swipe search
|
||
- student detection/portrait matching if targeting specific students
|
||
- isometric grid location logic if following reference fully
|
||
|
||
Suggested first version:
|
||
|
||
- pick a fixed region
|
||
- select available/highest visible lesson
|
||
- avoid favorite-student targeting at first
|
||
|
||
## Low priority backlog
|
||
|
||
### Scrimmage
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/scrimmage.py
|
||
```
|
||
|
||
Local target: `ba_auto/tasks/scrimmage.py`
|
||
|
||
Similar shape to Bounty/Commissions.
|
||
|
||
### Crafting
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/create.py
|
||
```
|
||
|
||
Local target: `ba_auto/tasks/crafting.py`
|
||
|
||
Very complex. Contains:
|
||
|
||
- material selection
|
||
- priority lists
|
||
- rarity tiers
|
||
- stepper/quantity UI
|
||
- filtering/sorting
|
||
- OCR-like decision points
|
||
|
||
Do not start until the framework and OCR are mature.
|
||
|
||
### Battle Pass claim
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/collect_pass_reward.py
|
||
```
|
||
|
||
Local target: `ba_auto/tasks/battle_pass.py`
|
||
|
||
Should be simpler than crafting.
|
||
|
||
OCR only needed for optional stats.
|
||
|
||
### Momo Talk
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/momo_talk.py
|
||
```
|
||
|
||
Local target: `ba_auto/tasks/momo_talk.py`
|
||
|
||
Potentially useful because it runs on a different cadence from daily reset.
|
||
|
||
Likely no OCR. Mostly state scanning and click flow.
|
||
|
||
### Main story push
|
||
|
||
This means clearing new uncleared stages, not sweeping already-cleared stages.
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/main_story.py
|
||
~/repo/baas-reference/module/explore_tasks/explore_task.py
|
||
```
|
||
|
||
Low priority because full grid-mode support requires lots of per-stage scripting.
|
||
|
||
A simple auto-fight-only mode can be added later.
|
||
|
||
### Group Story / Mini Story
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/group_story.py
|
||
~/repo/baas-reference/module/mini_story.py
|
||
```
|
||
|
||
Convenience only.
|
||
|
||
### Event content
|
||
|
||
Reference:
|
||
|
||
```
|
||
~/repo/baas-reference/module/activities/activity_utils.py
|
||
~/repo/baas-reference/module/sweep_activity.py
|
||
```
|
||
|
||
Do not prebuild specific event scripts.
|
||
|
||
Build only generic event AP sweep if it can reuse story sweep logic.
|
||
|
||
Event-specific content expires and is not worth hardcoding unless the user asks for a specific live event.
|
||
|
||
## Skip list
|
||
|
||
| Feature | Why skip |
|
||
|---|---|
|
||
| Total Assault / Raid | Low value and risky to automate. Reference support may be limited/stubbed. |
|
||
| Joint Firing Drill | Not worth prioritizing for JP if reference has server-specific limitations. |
|
||
| De-clothes localization toggle | CN-only / irrelevant. |
|
||
| Restart / refresh-uiautomator2 | Android/ADB backend maintenance, not applicable to PC/Steam/Proton. |
|
||
| Auto-unfriend | Risky, low value, destructive. |
|
||
| Daily minigame dispatcher | Event-specific and unstable. Handle ad hoc only. |
|
||
|
||
## Automation cadence notes
|
||
|
||
Some tasks decay on different schedules.
|
||
|
||
| Feature | Suggested cadence |
|
||
|---|---|
|
||
| Cafe income / affection | Every few hours |
|
||
| Momo Talk | Every few hours |
|
||
| Arena | Around reset windows / twice daily if implemented |
|
||
| Mailbox | Daily or with default run |
|
||
| AP/stamina/task rewards | Daily |
|
||
| Group AP | Daily |
|
||
| Bounty/Commissions/Scrimmage | Daily |
|
||
| Shops | Daily, after reset |
|
||
| Lesson/Schedule | Daily |
|
||
|
||
Scheduling should be handled outside the feature logic.
|
||
|
||
Feature code should perform one safe run and exit.
|
||
|
||
## Safety and robustness rules
|
||
|
||
Every task should have:
|
||
|
||
- maximum retry count
|
||
- timeout where appropriate
|
||
- safe failure mode
|
||
- clear stdout logging
|
||
- no infinite click loops
|
||
- no unbounded spending
|
||
- config guard for purchases
|
||
- dry-run or debug mode when useful
|
||
|
||
For purchases:
|
||
|
||
- default to conservative behavior
|
||
- avoid refresh loops until OCR/currency detection is reliable
|
||
- require explicit configured item list
|
||
- avoid buying unknown items
|
||
|
||
For battle features:
|
||
|
||
- require clear stop conditions
|
||
- avoid continuing blindly after unexpected state
|
||
- prefer returning failure over clicking randomly
|
||
|
||
## Configuration direction
|
||
|
||
Future config may live in `ba_auto/config.py` or `config.yaml`.
|
||
|
||
Possible config values:
|
||
|
||
```
|
||
server = JP
|
||
game_window_name = BlueArchive
|
||
display = :0
|
||
asset_dir = ~/ba_assets
|
||
screenshot_dir = .scratchpad/
|
||
cafe_max_clicks_per_room
|
||
story_sweep_target
|
||
shop_buy_list
|
||
arena_stop_condition
|
||
ocr_enabled
|
||
debug_enabled
|
||
```
|
||
|
||
Keep config explicit. Do not bury user-specific settings deep inside task logic.
|
||
|
||
## Debugging conventions
|
||
|
||
Use `.scratchpad/` for:
|
||
|
||
- temporary screenshots
|
||
- cropped templates
|
||
- annotated match images
|
||
- OCR debug output
|
||
- one-off notes
|
||
|
||
Do not use `/tmp` or `/private/tmp` unless unavoidable.
|
||
|
||
When detector behavior changes, save debug outputs with clear names, for example:
|
||
|
||
```
|
||
scratchpad/cafe_match_2026-07-05_001.png
|
||
scratchpad/shop_ocr_debug_001.png
|
||
```
|
||
|
||
## Local validation commands
|
||
|
||
On `nik-macbookair`:
|
||
|
||
```
|
||
bash -n ba_dailies.sh
|
||
python3 -m py_compile ba_daily.py
|
||
python3 -m py_compile ba_auto/*.py
|
||
python3 -m py_compile ba_auto/tasks/*.py
|
||
```
|
||
|
||
On `nik-gpu`:
|
||
|
||
```
|
||
~/ba_dailies.sh mailbox
|
||
~/ba_dailies.sh cafe
|
||
```
|
||
|
||
After migration:
|
||
|
||
```
|
||
~/ba_dailies.sh
|
||
```
|
||
|
||
should run the default daily sequence.
|
||
|
||
## Near-term recommended task order
|
||
|
||
1. Rewrite `ba_dailies.sh` as a thin launcher.
|
||
2. Add `ba_daily.py`.
|
||
3. Add `ba_auto/driver.py`.
|
||
4. Add `ba_auto/detector.py`.
|
||
5. Add `ba_auto/navigation.py`.
|
||
6. Move mailbox logic to `ba_auto/tasks/mailbox.py`.
|
||
7. Move cafe logic to `ba_auto/tasks/cafe.py`.
|
||
8. Update `setup.sh`.
|
||
9. Add `ba_auto/reference_notes/mapping.md`.
|
||
10. Verify existing mailbox and cafe still work.
|
||
11. Implement stamina/AP.
|
||
12. Implement group/club AP.
|
||
13. Implement fixed-target sweep features.
|
||
14. Add OCR only when needed.
|
||
15. Attempt Arena/Shop/Lesson after the framework is stable.
|
||
|
||
## Claude Code guidance summary
|
||
|
||
When Claude Code works on this repo, it should follow this rule:
|
||
|
||
> Reference first.
|
||
> Python first.
|
||
> Driver primitives before feature hacks.
|
||
> Bash launcher only.
|
||
|
||
Do not turn this project into a Bash recreation of Blue Archive Auto Script.
|