ba-auto-daily/plan.md

# ba-auto-daily implementation plan

Personal Blue Archive JP daily-automation project.

This project controls the PC/Steam/Proton Blue Archive client running on `nik-gpu` through local desktop automation:

- xdotool
- scrot
- Python
- OpenCV
- later OCR when needed

Development happens on `nik-macbookair`.

The reference implementation lives at:

```
~/repo/baas-reference/
```

The reference project should be treated as the behavioral blueprint. This project should avoid recreating feature logic from scratch when the reference already implements it.

## Core project direction

This project is now Python-first.

The goal is not to grow a large Bash script.

The goal is to build a small local Python automation framework that adapts the reference project's Blue Archive logic to this user's unique PC/Steam/Proton environment.

`ba_dailies.sh` should only be a launcher.

Feature logic should live in Python.

## Target architecture

```
~/repo/ba-auto-daily/
├── ba_dailies.sh
├── ba_daily.py
├── ba_auto/
│   ├── __init__.py
│   ├── config.py
│   ├── driver.py
│   ├── detector.py
│   ├── navigation.py
│   ├── tasks/
│   │   ├── __init__.py
│   │   ├── mailbox.py
│   │   ├── cafe.py
│   │   ├── stamina.py
│   │   ├── group.py
│   │   ├── bounty.py
│   │   ├── commission.py
│   │   ├── arena.py
│   │   ├── shop_common.py
│   │   ├── shop_tactical.py
│   │   ├── lesson.py
│   │   └── ...
│   └── reference_notes/
│       └── mapping.md
├── assets/
├── screenshots/
├── scripts/
├── setup.sh
├── CLAUDE.md
└── plan.md
```

This is the intended direction. It does not have to be completed all at once.

## Project layout

| Path | What it is |
|---|---|
| `~/repo/ba-auto-daily/ba_dailies.sh` | Thin launcher only. It should call the Python entry point. Do not add new feature logic here. |
| `~/repo/ba-auto-daily/ba_daily.py` | Main Python CLI entry point. Dispatches tasks such as mailbox, cafe, stamina, group, etc. |
| `~/repo/ba-auto-daily/ba_auto/driver.py` | Local PC/Steam/Proton control backend. Wraps xdotool, scrot, waits, clicks, swipes, keypresses, screenshots, and window focus. |
| `~/repo/ba-auto-daily/ba_auto/detector.py` | OpenCV/template/color matching helpers. Currently has `find_cafe_sparkle()`, ported in-process from the retired `scripts/detect_and_click.py`. |
| `~/repo/ba-auto-daily/ba_auto/navigation.py` | Shared navigation/state-probe helpers: `is_on_subscreen`, `is_modal_open`, used by both `mailbox.py` and `cafe.py`. |
| `~/repo/ba-auto-daily/ba_auto/tasks/` | Feature implementations. Each task should adapt the relevant `baas-reference/module/...` logic where possible. |
| `~/repo/ba-auto-daily/ba_auto/reference_notes/mapping.md` | Reference mapping table: local feature → reference module → local implementation → driver gaps. |
| `~/repo/ba-auto-daily/assets/` | Locally captured template images, such as cafe sparkle. Do not blindly copy assets from the reference repo. |
| `~/repo/ba-auto-daily/screenshots/` | Human reference screenshots, mostly Moonlight/game captures, used for calibration and debugging. |
| `~/repo/ba-auto-daily/setup.sh` | Bootstrap/deploy helper for `nik-gpu`. Should install/check dependencies and copy runtime files. |
| `~/repo/baas-reference/` | Read-only GPL-3.0 reference clone. Study and adapt. Never edit. |

## Runtime paths on nik-gpu

Preferred runtime layout:

| Path | What it is |
|---|---|
| `nik-gpu:~/ba_dailies.sh` | Thin launcher. |
| `nik-gpu:~/ba_daily.py` | Python CLI entry point. |
| `nik-gpu:~/ba_auto/` | Python package copied from this repo. |
| `nik-gpu:~/ba_assets/` | Runtime assets/templates. |
| `nik-gpu:~/.venvs/ba-auto-daily/` | Python virtual environment. |

`nik-gpu:~/ba_scripts/` may still contain `detect_and_click.py` and `ba_dailies_legacy.sh` left over from before both mailbox and cafe were migrated off them. Neither is deployed or referenced by anything anymore (`setup.sh` stopped copying them once Phase 6 landed) — safe to delete manually on `nik-gpu`, just not automated here.

## Implementation strategy

For each feature:

1. Read the matching `~/repo/baas-reference/module/...` file.
2. Summarize the reference flow.
3. Identify reusable logic:
   - state checks
   - retry loops
   - navigation sequence
   - battle/sweep/shop rules
   - detection method
   - failure handling
4. Identify backend-specific calls that cannot be reused directly.
5. Implement missing generic primitives in `ba_auto/driver.py` or `ba_auto/detector.py`.
6. Implement the feature in `ba_auto/tasks/<feature>.py`.
7. Add CLI command dispatch in `ba_daily.py`.
8. Keep `ba_dailies.sh` unchanged unless launcher behavior changes.
9. Test syntax locally.
10. Deploy to `nik-gpu`.
11. Run against the live game.
12. Update this plan.

The intended result is not a Bash automation script.

The intended result is a Python automation framework using the reference repository as the behavioral blueprint.

## Reference mapping table

Maintain this table in `ba_auto/reference_notes/mapping.md`.

Initial seed:

| Local feature | Reference file | Reference functions/classes | Local file | Backend replacements | Status |
|---|---|---|---|---|---|
| Mailbox | Need to confirm in reference | Need to inspect | `ba_auto/tasks/mailbox.py` | tap/click via xdotool, screenshot via scrot | Existing Bash behavior; migrate to Python |
| Cafe | `module/cafe_reward.py` | `to_cafe`, `interaction_for_cafe_solve_method3`, `collect` | `ba_auto/tasks/cafe.py` | `picture.co_detect`/`color.rgb_in_range` → `driver.color_at` pixel-probe checks; sparkle template match ported in-process into `ba_auto/detector.py` | Migrated: real Python, state-verified via color probes, no legacy bridge |
| Stamina/AP | `module/collect_daily_free_power.py`, `module/collect_daily_task_power.py` | `to_tasks`/`implement` (task-power, ported); `to_purchase_pyroxenes_menu`/`detect_free_power_availability` (free-power, not ported — real-money purchase menu) | `ba_auto/tasks/stamina.py` | `color.rgb_in_range` → `driver.color_at`; reference's per-tab claim loop replaced by the live UI's single "一括受取" (claim-all) button, triggered via Enter | Partially migrated (Phase 8): Mission-panel task/weekly/achievement claim done. Daily Free Power deliberately not implemented. |
| Group/Club AP | `module/group.py` | Need to inspect | `ba_auto/tasks/group.py` | fixed click + state check via local driver | Not started |
| Bounty | `module/rewarded_task.py` | Need to inspect | `ba_auto/tasks/bounty.py` | sweep/color/OCR adaptation | Not started |
| Commissions | `module/clear_special_task_power.py` | Need to inspect | `ba_auto/tasks/commission.py` | sweep/color adaptation | Not started |
| Arena | `module/arena.py` | Need to inspect | `ba_auto/tasks/arena.py` | auto-fight + OCR + local driver | Not started |
| Common Shop | `module/shop/common_shop.py`, `module/shop/shop_utils.py` | Need to inspect | `ba_auto/tasks/shop_common.py` | OCR + tab navigation + local clicks | Not started |
| Tactical Shop | `module/shop/tactical_challenge_shop.py`, `module/shop/shop_utils.py` | Need to inspect | `ba_auto/tasks/shop_tactical.py` | OCR + tab navigation + local clicks | Not started |
| Lesson/Schedule | `module/lesson.py` | Need to inspect | `ba_auto/tasks/lesson.py` | OCR + template/portrait search + local driver | Not started |

Do not implement a feature without filling at least the relevant row.

## Status snapshot

| Feature | Current status | Target status |
|---|---|---|
| Mailbox claim | Migrated: `ba_auto/tasks/mailbox.py` uses `driver.color_at` to verify the panel opened before acting (found live-testing bug: a marginal icon coordinate could miss and cascade into pressing Escape on the home screen, which triggers Blue Archive's own exit-game confirmation) | Done |
| Cafe pats + income | Migrated: `ba_auto/tasks/cafe.py` verifies each room/dialog opened via `driver.color_at` before acting; sparkle detection now runs in-process via `ba_auto/detector.py` instead of a per-click subprocess | Done |
| Stamina/AP (mission claim) | Migrated (partial, Phase 8): `ba_auto/tasks/stamina.py` claims the Mission panel's bulk "一括受取" button. Daily Free Power (real-money purchase menu) intentionally not implemented | Daily Free Power still not started |
| Shared driver | `ba_auto/driver.py` built (`run_command`, `focus_game`, `click`, `move_mouse`, `keypress`, `screenshot`, `wait`, `color_at`); `click()` now splits `mousemove`/`click` into two xdotool calls (Phase 8 finding — fixes a real source of click flakiness); wired into `mailbox.py`, `cafe.py`, `stamina.py` | Extend with new primitives as future tasks need them |
| Python CLI | Built: `ba_daily.py` dispatches `mailbox`/`cafe`/`stamina`/default flow | Extend as new tasks are added |
| Reference mapping | Built: `ba_auto/reference_notes/mapping.md` | Fill in reference file/function columns per feature |
| Everything else | Not started | Implement reference-first in Python |

## Migration phase

Before adding new game features, migrate the existing working implementation.

### Phase 1: Python skeleton

**Status: Done.**

Create:

```
ba_daily.py
ba_auto/__init__.py
ba_auto/config.py
ba_auto/driver.py
ba_auto/detector.py
ba_auto/navigation.py
ba_auto/tasks/__init__.py
ba_auto/tasks/mailbox.py
ba_auto/tasks/cafe.py
ba_auto/reference_notes/mapping.md
```

### Phase 2: Launcher

**Status: Done.**

Change `ba_dailies.sh` into a thin launcher:

```bash
#!/usr/bin/env bash
set -euo pipefail
VENV_PYTHON="${VENV_PYTHON:-$HOME/.venvs/ba-auto-daily/bin/python3}"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$VENV_PYTHON" "$SCRIPT_DIR/ba_daily.py" "$@"
```

Keep compatibility with:

```
./ba_dailies.sh
./ba_dailies.sh mailbox
./ba_dailies.sh cafe
```

### Phase 3: Driver extraction

**Status: Done.** Primitives (including `color_at`, added during the mailbox/cafe hardening work) are wired into both `ba_auto/tasks/mailbox.py` and `ba_auto/tasks/cafe.py`.

Move shell interactions into `ba_auto/driver.py`.

Driver primitives should include:

```
focus_game()
click(x, y)
double_click(x, y)
keypress(key)
screenshot(path=None)
swipe(...)
wait(seconds)
wait_until(...)
```

### Phase 4: Detector extraction

**Status: Done (scoped).** `scripts/detect_and_click.py`'s sparkle-matching logic (masked template match against `assets/cafe_sparkle.png`) was ported into `ba_auto/detector.py` as `find_cafe_sparkle()`, called in-process from `ba_auto/tasks/cafe.py` — this removed the old per-click Python cold start (a fresh `cv2`/`numpy` import per subprocess call) that CLAUDE.md's driver-layer guidance specifically warns against. `scripts/detect_and_click.py` had no remaining callers once this landed, so it was deleted rather than kept as a compatibility wrapper. The more generic primitives listed below (`load_template`, `match_template`, etc.) have not been built — only the one concrete sparkle-matching function needed so far exists; generalize when a second detector use case actually needs it.

Detector primitives, generalize later if needed:

```
load_template(...)
match_template(...)
find_best_match(...)
find_and_click_template(...)
color_mask(...)
debug_write_match(...)
```

### Phase 5: Mailbox migration

**Status: Done.** Live testing surfaced a real bug: the old `MAILBOX_ICON` coordinate `(1726, 60)` sat on the edge of the icon's hitbox and intermittently missed, and the fixed click sequence had no way to notice — it cascaded into pressing Escape on the bare home screen, which triggers Blue Archive's own "exit the game?" confirmation (dismissed safely with Cancel during testing; no game state was lost). The Python port in `ba_auto/tasks/mailbox.py` fixes the coordinate and, following `module/mail.py`'s `rgb_in_range` pattern, verifies the panel actually opened (and whether "claim all" is disabled) via `driver.color_at` before pressing any further keys, with a bounded retry and a safe abort if the panel never appears.

Move mailbox logic from Bash to:

```
ba_auto/tasks/mailbox.py
```

The CLI should call it through Python.

### Phase 6: Cafe migration

**Status: Done.** Same root cause as the mailbox bug (Phase 5), confirmed by step-by-step live replay with screenshots: `CAFE_ICON` clicks are flaky (missed on the first attempt, worked on retry at the identical coordinate — this is xdotool/Proton click-registration flakiness, not a coordinate-precision problem), and the old sequence had zero verification across its ~12 steps (open → dismiss notice → pat loop ×15 → switch room → dismiss notice → pat loop ×15 → claim income ×2 Enter → ×2 Escape). A missed click anywhere cascades into blind actions on whatever screen is actually showing, which — same as mailbox — very likely ends with an unverified Escape hitting the home screen and triggering Blue Archive's own exit-game confirmation.

`ba_auto/tasks/cafe.py` now verifies state at every transition using `driver.color_at`, following `module/cafe_reward.py`'s `picture.co_detect`/`rgb_in_range` pattern:

- opening the cafe icon and the room-switch button both retry (bounded) and confirm the panel actually opened via the same subscreen-header probe as mailbox, now shared in `ba_auto/navigation.is_on_subscreen`
- the "visited student list" notice that appears on every room entry is dismissed with Enter; this is harmless as a no-op if no popup is actually present (verified live), so no separate presence check was needed there
- the income dialog's own dimmed-overlay backdrop is checked (`navigation.is_modal_open`) before pressing Enter to claim, and the "receive" button's disabled-grey color is checked before attempting to claim at all (mirrors `collect()`'s `rgb_in_range` gate in the reference)
- the closing Escape(s) only fire when a subscreen/modal is confirmed still open, never blindly

Verified live (two full runs against the real game, plus a manual step-by-step replay of every transition):

- both rooms open and pat correctly
- sparkle detection still works and now runs in-process (see Phase 4) instead of shelling out per click
- cafe income claim works (confirmed gold +81,251 / AP +61 on an actual claim) and correctly no-ops when there's nothing to collect
- the reference's `zoom_out` step (camera zoom before sparkle detection — CLAUDE.md's "view centering/zoom" gap) was **not** ported: detection matched at 0.99 confidence without it in live testing, so it wasn't reproducibly broken here. Left as a documented open risk below rather than added speculatively.

Not verified / open risks:

- rank-up popups: not observed during testing (no student ranked up while testing), still unhandled if one appears mid-loop
- whether zoom/pan state could drift over a long unattended run and eventually break sparkle detection (see above — no evidence of this yet, but the reference project treats it as necessary)

#### Phase 6 follow-up: "farming affection doesn't happen" report

A later report claimed pats weren't landing at all, with the original `ba_dailies.sh` `do_cafe_room`/`do_cafe` pasted as the expected-behavior reference. Re-reading that Bash carefully changed the diagnosis: the original `detect_and_click.py` did one screenshot → detect → click per invocation and the Bash loop only kept calling it back-to-back while hits kept landing, breaking immediately on the first miss (`grep -q "^MATCH" || break`) — i.e. give-up-on-first-miss was the *original design*, not a regression introduced by the Python port. Detection math (mask, threshold `0.97`, click offset `(75, 47)`) ported over byte-for-byte identical.

Changes made this round:

- `ba_auto/detector.py`: `find_cafe_sparkle()` now tries multiple template scales (`SPARKLE_SCALES`) instead of one fixed size, since the cafe camera's zoom isn't reset before farming and isn't guaranteed to match whatever zoom the template was captured at. Strictly more permissive than the original single-scale match — no observed downside — but not confirmed as the actual root cause of the report (no live zoom-mismatch case was reproduced/observed).
- `ba_auto/tasks/cafe.py`: `_pat_room` now polls for the full `CAFE_MAX_CLICKS_PER_ROOM` budget with a 1s wait between misses instead of breaking on the very first miss. This is a deliberate deviation from the original design (see above) — cheap (adds at most ~15s per room when nothing is available) and covers the case where a screenshot lands mid-animation right after the room transition.
- `driver.move_mouse` added and called after each pat to park the cursor away from the sparkle area, per `screenshots/cafe/sparkle/02_*_cursor_on_head.png` showing the cursor can occlude the icon.

What was directly verified live after these changes:

- the room-entry/no-modal state probes (`navigation.is_on_subscreen`, `is_modal_open`) read correctly on real captured frames from both rooms
- neither room had a visible sparkle on any student at the time of testing (confirmed by eye on the actual screenshots, not inferred from the "no sparkle found" log) — this is the most likely explanation for why a same-session automated run kept reporting no matches: this session's own manual+automated testing had already consumed the available per-student affection interactions, which regenerate on a real-world cooldown far longer than one room visit

**Not yet verified**: an actual end-to-end pat (detect → click → affection-up dialog dismissed) succeeding after this round's changes, because no interactable sparkle was available during testing to exercise it against. Re-run `~/ba_dailies.sh cafe` once interactions have had time to regenerate and confirm `[cafe] patted N sparkle(s)` appears with N > 0.

### Phase 8: Stamina/AP mission claim

**Status: Partially done.** Read `module/collect_daily_task_power.py` (the "Tasks" menu claim loop — `to_tasks` + `rgb_in_range` checks against two fixed pixel pairs, click, dismiss, repeat) and `module/collect_daily_free_power.py` (a `picture.co_detect` state-machine walk into the Pyroxene Purchase menu's Package tab to claim a genuinely free 10 AP item). Live reconnaissance on the home screen found the direct local equivalents: a `ミッション` (Mission) icon opening a panel with per-tab claim buttons *and* a single bulk "一括受取" (claim all) button whose keyboard shortcut is literally Enter — much simpler than porting the reference's per-tab color-probe loop. `ba_auto/tasks/stamina.py` opens the Mission panel, checks whether "一括受取" is enabled (bright yellow vs. flat grey background probe at `config.MISSION_CLAIM_PROBE`), and if so presses Enter to claim, Enter again to dismiss the reward-reveal card (same "harmless no-op if absent" assumption as cafe's room-entry dismiss), bounded to a few rounds in case multiple rewards queue up.

The 青輝石購入 (Pyroxene Purchase) icon — reference's Daily Free Power entry point — was opened once to confirm the free-claim flow's location, but turned out to be a real-money purchase menu (¥3,000–¥4,900 package buttons visible immediately) with the genuinely-free item buried in a further tab. Given `plan.md`'s own purchase-safety rules ("avoid unbounded spending", "avoid buying unknown items"), this was **deliberately not automated this round** — the dialog was closed without navigating further. Treat this as a separate, explicitly-confirmed piece of future work, not an oversight.

Two real bugs found and fixed during live calibration, both worth remembering for future coordinate-hunting:

- **Visual gridline coordinate estimates were wrong twice in a row.** Reading icon bounds off a scaled/annotated screenshot crop by eye put the Mission icon's center at `(146, 352)` — which is actually in the dead space between the Mission and Pyroxene-Purchase icons, close enough to the latter's edge that clicks there landed on Pyroxene Purchase instead. The fix was sampling actual pixel colors along a scanline (`img.getpixel`) to find each icon's true left/right edge against the background, rather than eyeballing gridlines — this put the real center at `(75, 350)`, squarely inside the icon graphic, confirmed live. Lesson: for icon coordinates, prefer a pixel-boundary scan over a visual grid-overlay estimate.
- **`driver.click()`'s combined `xdotool mousemove X Y click 1` invocation is unreliable; splitting it fixed a chunk of this project's long-documented click flakiness.** Repeated single-click tests at a *verified-correct* coordinate still missed intermittently until the mousemove and click were issued as two separate `xdotool` calls with a short (0.2s) pause between them — after that, every subsequent click registered. This plausibly explains some of the "icon click missed on the first attempt, worked on retry" flakiness documented in Phases 5–6 (mailbox/cafe icons). Applied to `driver.click()` itself (project-wide, since all tasks share it) rather than special-cased in `stamina.py`; regression-tested live against `mailbox` and `cafe` after the change — both still work.

Not yet done: Group/Club AP, and Daily Free Power (see above).

### Phase 7: setup.sh update

**Status: Done — `setup.sh` deploys `ba_daily.py` and `ba_auto/`.** `scripts/ba_dailies_legacy.sh` and `scripts/detect_and_click.py` were deleted once mailbox and cafe both migrated off them (Phases 5–6); `setup.sh` no longer references either.

Update `setup.sh` so it deploys:

```
ba_dailies.sh
ba_daily.py
ba_auto/
assets/
```

to the expected runtime paths on `nik-gpu`.

## Prerequisites

### OCR

Not set up yet.

Needed for:

- currency readouts
- ticket counts
- region/tab name matching
- some shop logic
- some lesson/schedule logic
- arena ticket/rank/level checks
- bounty coin balance if auto-refresh is implemented

Candidates:

- Tesseract
- PaddleOCR

Do not add OCR until a feature needs it.

### Auto-fight primitive

Needed for:

- Arena
- future main story push
- some battle automation

Reference:

```
~/repo/baas-reference/module/main_story.py
```

Look for:

```
auto_fight
enter_battle
```

Target local module may be:

```
ba_auto/tasks/battle.py
```

or:

```
ba_auto/battle.py
```

This should become a reusable primitive, not arena-specific code.

## High priority backlog

### 1. Migration to Python-first

**Status: Done.** See Phases 1–7 above for the detailed history, including the mailbox and cafe exit-game-dialog bug and its fix.

Goal (all done):

- Bash launcher only
- Python CLI
- Python driver
- Python detector
- mailbox migrated
- cafe migrated
- reference mapping started

### 2. Stamina/AP sweep

**Status: Partially done — see Phase 8.**

Claim:

- ~~daily free AP purchase~~ — deferred; entry point is a real-money purchase menu, needs explicit confirmation before automating
- daily task-menu AP/pyroxene rewards — done, via the Mission panel's bulk claim button

Reference:

```
~/repo/baas-reference/module/collect_daily_free_power.py
~/repo/baas-reference/module/collect_daily_task_power.py
```

Local target: `ba_auto/tasks/stamina.py`

OCR: Not needed — turned out to be a single bulk-claim button + Enter, no per-item detection required.

### 3. Club/Group AP claim

Claim AP from club/group.

Reference:

```
~/repo/baas-reference/module/group.py
```

Local target: `ba_auto/tasks/group.py`

OCR: Not expected.

### 4. Normal/Hard story AP sweep

Sweep already-cleared main story stages to burn AP.

Reference:

```
~/repo/baas-reference/module/explore_tasks/sweep_task.py
~/repo/baas-reference/module/explore_tasks/task_utils.py
```

Local target: `ba_auto/tasks/story_sweep.py`

OCR: Likely needed for current region/stage detection unless using fixed configured targets.

Suggested first version:

- user-configured fixed stage
- no region search
- no dynamic OCR
- sweep configured mission only

Later version:

- fuzzy stage/region selection
- OCR-assisted navigation

### 5. Bounty

Three sub-areas and sweep availability.

Reference:

```
~/repo/baas-reference/module/rewarded_task.py
```

Local target: `ba_auto/tasks/bounty.py`

OCR: Optional for coin balance/refresh logic. Can skip refresh for first version.

### 6. Commissions

Two sub-dungeons:

- Base Defense
- Item Retrieval

Reference:

```
~/repo/baas-reference/module/clear_special_task_power.py
```

Local target: `ba_auto/tasks/commission.py`

OCR: Probably avoidable for first version if configured fixed sweep target is used.

### 7. Arena

Fight until out of tickets or until configured stopping condition.

Reference:

```
~/repo/baas-reference/module/arena.py
```

Local target: `ba_auto/tasks/arena.py`

Needs:

- auto-fight primitive
- OCR or visual detection for ticket/rank state
- careful safety limits

Arena is high-value but more risky than fixed claim tasks.

### 8. Common Shop + Tactical Shop

Auto-buy configured items.

Reference:

```
~/repo/baas-reference/module/shop/common_shop.py
~/repo/baas-reference/module/shop/tactical_challenge_shop.py
~/repo/baas-reference/module/shop/shop_utils.py
```

Local targets:

```
ba_auto/tasks/shop_common.py
ba_auto/tasks/shop_tactical.py
```

Needs:

- OCR for currency balances
- shop tab detection
- configured buy list
- safe purchase confirmation logic

Start with a no-refresh, fixed configured buy list.

### 9. Lesson / Schedule

Affection farming via classes.

Reference:

```
~/repo/baas-reference/module/lesson.py
```

Local target: `ba_auto/tasks/lesson.py`

Complexity: High

Needs:

- OCR for region/area names
- multi-page swipe search
- student detection/portrait matching if targeting specific students
- isometric grid location logic if following reference fully

Suggested first version:

- pick a fixed region
- select available/highest visible lesson
- avoid favorite-student targeting at first

## Low priority backlog

### Scrimmage

Reference:

```
~/repo/baas-reference/module/scrimmage.py
```

Local target: `ba_auto/tasks/scrimmage.py`

Similar shape to Bounty/Commissions.

### Crafting

Reference:

```
~/repo/baas-reference/module/create.py
```

Local target: `ba_auto/tasks/crafting.py`

Very complex. Contains:

- material selection
- priority lists
- rarity tiers
- stepper/quantity UI
- filtering/sorting
- OCR-like decision points

Do not start until the framework and OCR are mature.

### Battle Pass claim

Reference:

```
~/repo/baas-reference/module/collect_pass_reward.py
```

Local target: `ba_auto/tasks/battle_pass.py`

Should be simpler than crafting.

OCR only needed for optional stats.

### Momo Talk

Reference:

```
~/repo/baas-reference/module/momo_talk.py
```

Local target: `ba_auto/tasks/momo_talk.py`

Potentially useful because it runs on a different cadence from daily reset.

Likely no OCR. Mostly state scanning and click flow.

### Main story push

This means clearing new uncleared stages, not sweeping already-cleared stages.

Reference:

```
~/repo/baas-reference/module/main_story.py
~/repo/baas-reference/module/explore_tasks/explore_task.py
```

Low priority because full grid-mode support requires lots of per-stage scripting.

A simple auto-fight-only mode can be added later.

### Group Story / Mini Story

Reference:

```
~/repo/baas-reference/module/group_story.py
~/repo/baas-reference/module/mini_story.py
```

Convenience only.

### Event content

Reference:

```
~/repo/baas-reference/module/activities/activity_utils.py
~/repo/baas-reference/module/sweep_activity.py
```

Do not prebuild specific event scripts.

Build only generic event AP sweep if it can reuse story sweep logic.

Event-specific content expires and is not worth hardcoding unless the user asks for a specific live event.

## Skip list

| Feature | Why skip |
|---|---|
| Total Assault / Raid | Low value and risky to automate. Reference support may be limited/stubbed. |
| Joint Firing Drill | Not worth prioritizing for JP if reference has server-specific limitations. |
| De-clothes localization toggle | CN-only / irrelevant. |
| Restart / refresh-uiautomator2 | Android/ADB backend maintenance, not applicable to PC/Steam/Proton. |
| Auto-unfriend | Risky, low value, destructive. |
| Daily minigame dispatcher | Event-specific and unstable. Handle ad hoc only. |

## Automation cadence notes

Some tasks decay on different schedules.

| Feature | Suggested cadence |
|---|---|
| Cafe income / affection | Every few hours |
| Momo Talk | Every few hours |
| Arena | Around reset windows / twice daily if implemented |
| Mailbox | Daily or with default run |
| AP/stamina/task rewards | Daily |
| Group AP | Daily |
| Bounty/Commissions/Scrimmage | Daily |
| Shops | Daily, after reset |
| Lesson/Schedule | Daily |

Scheduling should be handled outside the feature logic.

Feature code should perform one safe run and exit.

## Safety and robustness rules

Every task should have:

- maximum retry count
- timeout where appropriate
- safe failure mode
- clear stdout logging
- no infinite click loops
- no unbounded spending
- config guard for purchases
- dry-run or debug mode when useful

For purchases:

- default to conservative behavior
- avoid refresh loops until OCR/currency detection is reliable
- require explicit configured item list
- avoid buying unknown items

For battle features:

- require clear stop conditions
- avoid continuing blindly after unexpected state
- prefer returning failure over clicking randomly

## Configuration direction

Future config may live in `ba_auto/config.py` or `config.yaml`.

Possible config values:

```
server = JP
game_window_name = BlueArchive
display = :0
asset_dir = ~/ba_assets
screenshot_dir = scratchpad/
cafe_max_clicks_per_room
story_sweep_target
shop_buy_list
arena_stop_condition
ocr_enabled
debug_enabled
```

Keep config explicit. Do not bury user-specific settings deep inside task logic.

## Debugging conventions

Use `scratchpad/` for:

- temporary screenshots
- cropped templates
- annotated match images
- OCR debug output
- one-off notes

Do not use `/tmp` or `/private/tmp` unless unavoidable.

When detector behavior changes, save debug outputs with clear names, for example:

```
scratchpad/cafe_match_2026-07-05_001.png
scratchpad/shop_ocr_debug_001.png
```

## Local validation commands

On `nik-macbookair`:

```
bash -n ba_dailies.sh
python3 -m py_compile ba_daily.py
python3 -m py_compile ba_auto/*.py
python3 -m py_compile ba_auto/tasks/*.py
```

On `nik-gpu`:

```
~/ba_dailies.sh mailbox
~/ba_dailies.sh cafe
```

After migration:

```
~/ba_dailies.sh
```

should run the default daily sequence.

## Near-term recommended task order

1. Rewrite `ba_dailies.sh` as a thin launcher.
2. Add `ba_daily.py`.
3. Add `ba_auto/driver.py`.
4. Add `ba_auto/detector.py`.
5. Add `ba_auto/navigation.py`.
6. Move mailbox logic to `ba_auto/tasks/mailbox.py`.
7. Move cafe logic to `ba_auto/tasks/cafe.py`.
8. Update `setup.sh`.
9. Add `ba_auto/reference_notes/mapping.md`.
10. Verify existing mailbox and cafe still work.
11. Implement stamina/AP.
12. Implement group/club AP.
13. Implement fixed-target sweep features.
14. Add OCR only when needed.
15. Attempt Arena/Shop/Lesson after the framework is stable.

## Claude Code guidance summary

When Claude Code works on this repo, it should follow this rule:

> Reference first.
> Python first.
> Driver primitives before feature hacks.
> Bash launcher only.

Do not turn this project into a Bash recreation of Blue Archive Auto Script.