ba-auto-daily/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code when working with code in this repository.

## What this project is

This repository is a personal Blue Archive JP daily-automation project.

It controls the real Blue Archive PC/Steam/Proton client running on a Linux machine named `nik-gpu`. The development machine is a MacBook named `nik-macbookair`.

**The original goal is a backend port, not a rewrite.** The reference project (`~/repo/baas-reference/`) only works against an Android emulator via ADB/uiautomator2, typically run on a Windows PC. This project's job is to make that same automation logic work against the real JP client running under Proton on Linux instead — porting the reference's decision logic, navigation, and state-detection approach, including its use of OCR, as faithfully as practical, and replacing only the parts that are genuinely Android-specific with local desktop equivalents.

Android-specific parts include:

- ADB input
- uiautomator2 taps
- Android UI-object-tree queries
- emulator-specific assumptions

Local desktop equivalents include:

- `xdotool`
- `scrot`
- OpenCV
- OCR against screenshots
- local image/color/template detection

Inventing a different, simpler approach to avoid porting a piece of reference logic is a deviation from the project goal, not a shortcut toward it.

The automation backend is local desktop control:

- `xdotool` for mouse/keyboard/window control
- `scrot` for screenshots
- Python/OpenCV for image matching and color/template detection
- OCR, using Tesseract or PaddleOCR, wherever the reference implementation uses OCR for a feature
- no Android emulator control
- no ADB
- no uiautomator2

### OCR policy

OCR is **not** an Android-specific concern. The reference's OCR-driven region, stage-name, and currency matching runs against a screenshot and can be ported to this backend using `scrot` captures. Only the reference's input mechanism is Android-specific.

Two features, Stamina/AP mission claim and Normal/Hard story AP sweep, were originally built without OCR by substituting ad hoc pixel probes, fixed coordinates, or randomized selection for the reference's OCR-driven navigation. For story sweep, that substitution produced more real bugs during live testing than porting the reference's actual approach would have:

- a wrong modal-open probe
- an unverified button click that silently under-spent AP
- a modal that does not close on Escape
- a latent hazard where a mistimed keypress could have started a real battle

See `plan.md` Phase 9's retrospective for the full writeup.

That issue has since been fixed: `story_sweep.py` now ports the reference's actual OCR-based region/stage targeting. See `plan.md` Phase 10. The retrospective stays here as the concrete, lived reason for the OCR policy, not as a description of the current state of that task.

Going forward: when a reference feature's control flow depends on OCR, set up OCR and port that logic. Do not invent a non-OCR workaround just to avoid the setup cost. Only skip OCR for a specific step if the reference itself does not use OCR there.

The reference project is located at:

```text
~/repo/baas-reference/
```

That repository contains the full Blue Archive Auto Script implementation. It should be treated as the behavioral reference for this project.

## Core architecture decision

This project must be Python-first.

**Do not implement new Blue Archive automation logic in `ba_dailies.sh`.**

`ba_dailies.sh` is only a thin runtime/deployment launcher for convenience on `nik-gpu`.

All feature logic must live in Python.

The intended structure is:

```text
ba-auto-daily/
├── ba_dailies.sh
├── ba_daily.py
├── ba_auto/
│   ├── __init__.py
│   ├── driver.py
│   ├── detector.py
│   ├── navigation.py
│   ├── config.py
│   ├── tasks/
│   │   ├── __init__.py
│   │   ├── mailbox.py
│   │   ├── cafe.py
│   │   ├── stamina.py
│   │   ├── group.py
│   │   └── ...
│   └── reference_notes/
│       └── mapping.md
├── assets/
├── screenshots/
├── scratchpad/
├── setup.sh
├── plan.md
└── CLAUDE.md
```

`scripts/` previously held one-off Bash/Python helpers. It was deleted once mailbox and cafe migrated off it. Do not recreate `scripts/` as a place to stash feature logic.

The exact layout can evolve, but the architectural rule should not change:

- Bash launches Python.
- Python owns automation logic.
- The reference repository guides feature behavior.
- Local driver primitives adapt that behavior to the PC/Steam/Proton setup.

## Reference-first rule

Before implementing any new feature, inspect the matching reference implementation in:

```text
~/repo/baas-reference/module/
```

Do not start by inventing a Bash click sequence.

For every feature, first identify:

1. Which reference file implements it.
2. Which class/function contains the main control flow.
3. What the reference uses for state detection.
4. What the reference uses for retries/failure handling.
5. Which parts depend on Android/uiautomator2 and must be replaced.
6. Which parts can be ported directly as Python control flow.
7. Which local driver primitives are missing.

Then implement the feature in Python under:

```text
ba_auto/tasks/
```

## Reference repository usage

The reference repository is **read-only**.

### Allowed

- read reference modules
- inspect control flow
- reuse architecture ideas
- reuse retry/state-machine structure
- reuse task decomposition ideas
- reuse constants/config concepts when appropriate
- write local notes describing how a reference module maps to this project

### Not allowed

- edit files in `~/repo/baas-reference/`
- reimplement a reference feature from scratch in Bash
- create a local solution that ignores the reference flow when a reference implementation already exists
- substitute a pixel probe, fixed coordinate, or randomized shortcut for reference logic that uses OCR, just to avoid setting up OCR

## Two-machine architecture

Development happens on:

```text
nik-macbookair
```

Runtime happens on:

```text
nik-gpu
```

The Blue Archive client and X display live on `nik-gpu`.

Assume:

```text
DISPLAY=:0
GDM XAUTHORITY under /run/user/1000
```

The game runs under Steam/Proton on the Linux desktop.

There is no reliable local execution path on macOS. Anything that interacts with the game must be deployed to `nik-gpu`.

## Deployment model

During iteration, files are pushed from `nik-macbookair` to `nik-gpu`.

Typical paths on `nik-gpu`:

```text
~/ba_dailies.sh
~/ba_daily.py
~/ba_auto/
~/ba_assets/
~/.venvs/ba-auto-daily/
```

The current setup may still contain older paths such as:

```text
~/ba_scripts/detect_and_click.py
```

When refactoring, prefer consolidating Python code into `ba_auto/`.

`setup.sh` should bootstrap or update the runtime layout on `nik-gpu`.

For a fresh checkout on `nik-gpu`, run from the repo root:

```bash
./setup.sh
```

After initial setup, individual changes may be pushed with `scp` or `rsync`.

Prefer syncing the project from the repository root:

```bash
rsync -av \
  --exclude='.git/' \
  --exclude='__pycache__/' \
  --exclude='*.pyc' \
  --exclude='.claude/settings.local.json' \
  --exclude='graphify-out/' \
  --exclude='screenshots/' \
  --exclude='scratchpad/' \
  ./ nik-gpu:~/repo/ba-auto-daily/
```

Do not sync Claude Code internal temporary folders.

## Runtime dependencies on nik-gpu

These are host-level dependencies. Confirm they exist before assuming a bug is in the project code.

Required now:

- `xdotool`
- `scrot`
- `python3`

Required Python packages:

- `opencv-python` or `opencv-python-headless`
- `numpy`
- `pytesseract`

Expected venv:

```text
~/.venvs/ba-auto-daily/bin/python3
```

Quick check:

```bash
ssh nik-gpu "which xdotool scrot tesseract && ~/.venvs/ba-auto-daily/bin/python3 -c 'import cv2, numpy, pytesseract; print(cv2.__version__)'"
```

OCR engine dependency is set up as of Phase 10:

- Tesseract via the `tesseract-ocr` apt package
- `pytesseract` in the venv

Installing `tesseract-ocr` needs interactive `sudo`, so `setup.sh` should check for it but should not assume it can install it automatically.

`ba_auto/detector.py`'s `read_text()` and `read_int()` wrap OCR for occasional single-crop reads. See `story_sweep.py` for the first real usage.

## Bash policy

`ba_dailies.sh` should be a thin launcher only.

Preferred shape:

```bash
#!/usr/bin/env bash
set -euo pipefail

VENV_PYTHON="${VENV_PYTHON:-$HOME/.venvs/ba-auto-daily/bin/python3}"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

exec "$VENV_PYTHON" "$SCRIPT_DIR/ba_daily.py" "$@"
```

Acceptable Bash responsibilities:

- choose Python binary
- set environment variables
- call Python entry point
- provide compatibility with old command names
- fail early if the Python entry point is missing

Not acceptable in Bash:

- feature state machines
- long click sequences
- OpenCV logic
- OCR logic
- retry loops for game states
- feature-specific navigation
- shop/sweep/battle logic
- new `do_<feature>` game automation functions

If an existing Bash function exists, migrate it to Python rather than extending it.

## Python entry point

The intended Python CLI entry point is:

```text
ba_daily.py
```

It should support commands such as:

```bash
./ba_dailies.sh
./ba_dailies.sh mailbox
./ba_dailies.sh cafe
./ba_dailies.sh stamina
./ba_dailies.sh story_sweep
./ba_dailies.sh group
```

No argument should run the default daily flow.

Current default flow in `ba_daily.py`'s `DEFAULT_ORDER`:

1. focus game
2. mailbox
3. cafe
4. stamina
5. future daily tasks

`story_sweep` spends AP rather than reclaiming something free, so it is deliberately excluded from the default flow. It must be invoked explicitly.

The CLI should dispatch into task modules under:

```text
ba_auto/tasks/
```

## Driver layer

Create and maintain a driver layer in:

```text
ba_auto/driver.py
```

The driver layer should wrap the PC/Steam/Proton backend.

It should provide reusable primitives such as:

```python
focus_game()
screenshot()
click(x, y)
double_click(x, y)
drag(start, end, duration)
swipe(start, end, duration)
keypress(key)
wait(seconds)
wait_until(...)
color_at(...)
region_average_color(...)
template_match(...)
find_and_click_template(...)
```

Feature modules should not directly shell out to `xdotool` or `scrot` unless a driver primitive is missing and is being added.

Prefer:

```python
driver.click(x, y)
```

over:

```python
subprocess.run(["xdotool", "click", "1"])
```

This keeps the project close to the reference architecture while replacing only the backend-control layer.

## Detector layer

Image matching, color matching, and OCR wrappers should live in:

```text
ba_auto/detector.py
```

or in clearly named helper classes/functions.

The cafe sparkle-matching logic formerly in `scripts/detect_and_click.py` has been migrated into `ba_auto/detector.py` as `find_cafe_sparkle()`. It should be called in-process rather than as a per-click subprocess.

The detector should support:

- screenshot input
- template matching
- threshold tuning
- masked matching
- click-offset handling
- OCR crop reads
- debug image output to `scratchpad/`

Avoid one Python cold start per click attempt where possible. Prefer long-running Python task logic that can take repeated screenshots and click repeatedly from one process.

## Navigation layer

Common navigation should live in:

```text
ba_auto/navigation.py
```

Use this for shared flows such as:

- returning home
- opening main menu
- opening mailbox
- opening cafe
- opening shop
- opening lesson/schedule
- closing popups
- generic back/escape handling
- waiting for known UI states

Do not duplicate navigation click sequences inside every task if they can be shared.

## Task modules

Each feature should have a task module:

```text
ba_auto/tasks/<feature>.py
```

Examples:

```text
ba_auto/tasks/mailbox.py
ba_auto/tasks/cafe.py
ba_auto/tasks/stamina.py
ba_auto/tasks/story_sweep.py
ba_auto/tasks/group.py
```

Each task module should expose a clear function such as:

```python
run(driver, config)
```

or:

```python
run_mailbox(driver, config)
```

Keep task files feature-focused.

## Reference mapping notes

Maintain a mapping file at:

```text
ba_auto/reference_notes/mapping.md
```

Before or during implementation of a feature, update the mapping.

Use this format:

```markdown
| Local feature | Reference file | Reference functions/classes | Local file | Backend replacements | Status |
|---|---|---|---|---|---|
| Cafe | `module/cafe.py` or relevant file | `...` | `ba_auto/tasks/cafe.py` | uiautomator2 tap -> xdotool click, screenshot -> scrot/OpenCV | In progress |
```

## Working conventions

Always work from the repository root:

```bash
cd ~/repo/ba-auto-daily
```

Use `scratchpad/` in the project root for all temporary, generated, diagnostic, calibration, and investigation files.

Examples:

- cropped calibration images
- debug screenshots
- annotated match results
- temporary Python probes
- temporary investigation notes
- one-off image-analysis scripts
- temporary OCR experiments
- screenshots copied back from `nik-gpu` for analysis

Create it if missing:

```bash
mkdir -p scratchpad
```

Do **not** use Claude Code's internal temporary directories such as:

```text
/private/tmp/claude-*
/tmp/claude-*
```

Do **not** run probes from `/private/tmp/claude-*` or write temporary scripts there. These paths are outside the repository and often trigger extra permission prompts, interrupting the development flow.

When a temporary Python probe is needed, write it into `scratchpad/` first, then run it from the repository root.

Preferred pattern:

```bash
cat > scratchpad/probe_image.py <<'PY'
from PIL import Image

img = Image.open("scratchpad/shop_probe_04_checked.png")
print(img.size)

points = [
    ("checkbox_checked_center", (972, 297)),
    ("checkbox_checked_bg", (975, 300)),
    ("card_border_left", (940, 380)),
    ("card_border_unselected", (1165, 380)),
    ("buy_button", (1751, 1112)),
    ("cancel_button", (1525, 1112)),
]

for name, xy in points:
    print(name, xy, img.getpixel(xy))
PY

python3 scratchpad/probe_image.py
```

Avoid this pattern:

```bash
cd /private/tmp/claude-*/scratchpad && python3 -c "..."
```

Also avoid large multiline `python3 -c "..."` commands when they contain comments or complex quoting. Prefer a temporary script under `scratchpad/` because it is easier to inspect, rerun, and keep within workspace permissions.

`screenshots/` contains human reference captures. They are useful for calibration and documentation, but they are not necessarily automated test fixtures.

`assets/` contains local template images. These should be captured from the local game setup where possible.

## Temporary probe policy

Temporary probes are allowed, but they must be workspace-local.

For non-trivial investigation, use:

```text
scratchpad/<short_descriptive_name>.py
```

Examples:

```text
scratchpad/probe_shop_checkbox.py
scratchpad/probe_cafe_income.py
scratchpad/probe_story_modal.py
scratchpad/probe_ocr_region.py
```

A temporary probe should:

- run from the repository root
- read inputs from `scratchpad/`, `screenshots/`, or `assets/`
- write outputs to `scratchpad/`
- avoid `/tmp`, `/private/tmp`, and Claude Code internal paths
- avoid relying on absolute `/private/tmp/claude-*` paths
- be deletable once the investigation is complete

Prefer:

```bash
python3 scratchpad/probe_shop_checkbox.py
```

Do not prefer:

```bash
python3 -c "large multiline script..."
```

For very small one-liners, `python3 -c` is acceptable only when it does not require changing directories into `/private/tmp/claude-*`.

## Testing and checks

Because the game only runs on `nik-gpu`, local macOS testing is limited.

Before deploying, run static/syntax checks locally:

```bash
bash -n ba_dailies.sh
python3 -m py_compile ba_daily.py
python3 -m py_compile ba_auto/*.py
python3 -m py_compile ba_auto/tasks/*.py
```

For temporary investigation scripts, use:

```bash
python3 scratchpad/<script_name>.py
```

Do not run temporary investigation scripts from Claude Code's internal `/private/tmp/claude-*` workspace.

On `nik-gpu`, run real integration tests against the live game.

Example:

```bash
ssh nik-gpu "~/repo/ba-auto-daily/ba_dailies.sh cafe"
```

When debugging image matching, write debug images to:

```text
scratchpad/
```

When debugging on `nik-gpu`, copy relevant screenshots or debug images back into the local project `scratchpad/` if they need local analysis.

## Existing features

Current project state: mailbox, cafe, stamina, and story_sweep are all migrated to real Python. No Bash feature logic remains.

- `ba_dailies.sh` is a thin launcher that execs `ba_daily.py`
- `ba_daily.py` dispatches `mailbox`, `cafe`, `stamina`, `story_sweep`, and default flow to `ba_auto/tasks/`
- default flow is `mailbox`, `cafe`, `stamina`
- `story_sweep` is opt-in only since it spends AP rather than reclaiming something free
- `ba_auto/tasks/mailbox.py` and `ba_auto/tasks/cafe.py` click with `ba_auto/driver.py` primitives and verify state with `driver.color_at` and `ba_auto/navigation.py`
- mailbox and cafe were ported from the reference patterns around `module/mail.py` and `module/cafe_reward.py`
- no legacy bridge remains
- `ba_auto/tasks/stamina.py` claims the Mission panel's bulk `一括受取` button
- see `plan.md` Phase 8 for stamina details
- `ba_auto/tasks/story_sweep.py` sweeps a config-driven list of exact `(region, stage, count)` targets from `config.STORY_SWEEP_TARGETS`
- story sweep navigates via OCR, using region-number readout and stage-label matching, rather than random selection
- see `plan.md` Phase 10 for the OCR-based story sweep port
- `ba_auto/detector.py` has `find_cafe_sparkle()`, the sparkle template-match ported in-process from the now-deleted `scripts/detect_and_click.py`
- `scripts/ba_dailies_legacy.sh` and `scripts/detect_and_click.py` have been deleted
- the `scripts/` directory itself no longer exists
- `ba_auto/driver.py` primitives are wired into all migrated task modules
- existing driver primitives include `run_command`, `focus_game`, `click`, `move_mouse`, `scroll`, `keypress`, `screenshot`, `wait`, and `color_at`
- `ba_auto/navigation.py` has shared state probes used across tasks:
  - `is_on_subscreen`
  - `is_modal_open`
  - `wait_for_state()`

`wait_for_state()` is a scoped port of the reference's `core/picture.py::co_detect`: watch for any of several named states, react to known non-terminal ones, and stop on a recognized terminal one.

`story_sweep.py` additionally has its own local modal probes and close logic because the stage-info modal is wide enough to break `is_modal_open`'s default probe point, and it does not close on Escape. See `plan.md` Phase 9.

The story sweep modal renders at least two different internal layouts:

- a plain one for the bonus `-A` stage
- a taller tabbed one for regular numbered stages

Their button coordinates differ. This was discovered live in Phase 10.

Live testing found both the mailbox-icon and cafe-icon fixed coordinates were flaky. They missed the first click and worked on retry. Neither task originally verified state before proceeding, so a missed click cascaded into blind actions and could reach an unverified Escape press on the home screen, which triggers Blue Archive's own "exit the game?" confirmation.

See `plan.md` Phases 5–6 for the full writeup. This is the concrete reason every task now verifies state before acting rather than trusting fixed coordinates or a single click blindly.

Rank-up popups mid-pat-loop are now handled.

A pat that crosses an affection-rank threshold shows a full-screen `絆ランクアップ!` cutscene with no cafe header. `find_cafe_sparkle()` can never recognize this because it is nothing like the sparkle template. The loop used to spin uselessly against it for the rest of the room's click budget.

`cafe.py`'s `_dismiss_rank_up_if_shown()` reuses the existing `navigation.is_on_subscreen` header-brightness probe. This was confirmed against `screenshots/cafe/student/01-02`:

- rank-up cutscene: header probe reads `r < 200`
- normal cafe screen: header probe reads `r > 200`

It presses Enter until the cutscene clears, porting the reference's own `to_cafe()` navigation handling of `relationship_rank_up` from `module/cafe_reward.py`.

This is not yet live-confirmed against a real rank-up trigger. It is grounded in real captured screenshots, but treat it as implemented-but-unverified until one happens naturally during a real run.

Not yet verified for cafe:

- whether camera zoom/pan can drift over a long unattended run
- the reference project zooms out before detecting
- this project currently does not
- testing did not reproduce a failure from skipping it
- see `plan.md` Phase 6 "Not verified" list

## Known cafe gaps to verify during migration

When migrating or modifying cafe logic, verify these manually against the actual current behavior:

- whether student rotation popups are handled
- whether rank-up popups are handled
- whether the view is zoomed/centered before sparkle detection
- whether detection still works when text overlaps the student
- whether one Python process can handle repeated sparkle detection faster than repeated cold starts
- whether both cafe rooms are handled consistently
- whether cafe income claim is robust against popup timing

Do not assume the old Bash implementation handles these correctly.

## Migration goal

The migration goal is:

- preserve existing behavior
- move feature logic from Bash into Python
- keep `ba_dailies.sh` as launcher only
- make detection logic reusable
- prepare for future features by copying the reference project's structure where appropriate

## Anti-patterns

Do not:

- add new `do_<feature>` Bash functions
- build a giant Bash automation script
- recreate reference logic as fixed-coordinate Bash click chains
- skip reading the reference module before implementing a feature
- put OCR in Bash
- put OpenCV state machines in Bash
- make every feature shell out independently to `xdotool`
- duplicate common navigation in every task
- copy reference image assets blindly
- edit `~/repo/baas-reference/`
- implement event-specific features before the generic reusable machinery exists
- invent a pixel probe, fixed-coordinate, or randomized substitute for reference logic that uses OCR, just to avoid setting up OCR
- treat OCR as an Android-specific concern that can be deferred indefinitely
- run temporary probes from `/private/tmp/claude-*` instead of the repository `scratchpad/`
- create investigation scripts outside the repository when they can live in `scratchpad/`
- use multiline `python3 -c "..."` commands for non-trivial probes
- rely on Claude Code internal temporary paths in committed code, notes, commands, or debugging workflow
- write generated debug files into the repository root when they belong in `scratchpad/`

## Feature implementation workflow

For every new feature:

1. Read the relevant `~/repo/baas-reference/module/...` file.
2. Summarize the upstream feature flow in notes or comments.
3. Add or update the reference mapping table.
4. Identify missing local driver primitives.
5. Implement or improve those primitives in `ba_auto/driver.py` or `ba_auto/detector.py`.
6. Implement the feature in `ba_auto/tasks/<feature>.py`.
7. Add CLI dispatch in `ba_daily.py`.
8. Keep `ba_dailies.sh` unchanged unless launcher behavior changes.
9. Run syntax checks.
10. Deploy to `nik-gpu`.
11. Test against the live game.
12. Update `plan.md` status.
13. Move temporary investigation files into `scratchpad/`, or delete them if they are no longer useful.

## Priority when uncertain

When uncertain, prefer this order:

1. Preserve existing working behavior.
2. Follow the reference project's control flow, including its use of OCR.
3. Do not deprioritize OCR-based navigation/matching in favor of a simpler non-OCR substitute.
4. Keep logic in Python.
5. Add reusable driver primitives instead of feature-specific hacks.
6. Use local screenshots/assets only when backend differences require it.
7. Use `scratchpad/` for temporary work and diagnostics.
8. Avoid large rewrites that do not move the project closer to reference-driven Python architecture.

## User preference

The user wants this project to be as close to the original Blue Archive Auto Script architecture as practical, without recreating logic that already exists.

The user specifically does not want Claude Code to keep converting feature work into Bash.

The user also wants Claude Code to avoid using `/private/tmp/claude-*` for development probes when a repository-local `scratchpad/` directory can be used instead, because those paths trigger permission prompts and interrupt the development flow.

Respect those preferences.