ba-auto-daily/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code when working with code in this repository.

What this project is

This repository is a personal Blue Archive JP daily-automation project.

It controls the real Blue Archive PC/Steam/Proton client running on a Linux machine named nik-gpu. The development machine is a MacBook named nik-macbookair.

The original goal is a backend port, not a rewrite. The reference project (~/repo/baas-reference/) only works against an Android emulator via ADB/uiautomator2, typically run on a Windows PC. This project's job is to make that same automation logic work against the real JP client running under Proton on Linux instead — porting the reference's decision logic, navigation, and state-detection approach (including its use of OCR) as faithfully as practical, and replacing only the parts that are genuinely Android-specific (ADB/uiautomator2 input and UI-object-tree queries) with local desktop equivalents (xdotool, scrot, OpenCV, OCR). Inventing a different, simpler approach to avoid porting a piece of reference logic (most notably OCR — see below) is a deviation from that goal, not a shortcut toward it.

The automation backend is local desktop control:

• xdotool for mouse/keyboard/window control
• scrot for screenshots
• Python/OpenCV for image matching and color/template detection
• OCR (Tesseract or PaddleOCR), ported wherever the reference implementation uses it for a feature — see "OCR policy" below
• no Android emulator control
• no ADB
• no uiautomator2

OCR policy

OCR is not an Android-specific concern — the reference's OCR-driven region/stage-name/currency matching runs against a screenshot and would work identically against a scrot capture on this backend. Only the reference's input (ADB/uiautomator2 taps) is Android-specific.

Two features (Stamina/AP mission claim, Normal/Hard story AP sweep) were built without OCR by substituting ad hoc pixel-probes, fixed coordinates, or randomized selection for the reference's OCR-driven navigation. For story sweep, that substitution produced more real bugs during live testing than porting the reference's actual approach would have — a wrong modal-open probe, an unverified button click that silently under-spent AP, a modal that doesn't close on Escape, and a latent hazard where a mistimed keypress could have started a real battle (see plan.md Phase 9's retrospective for the full writeup). None of those bugs would exist if the reference's deterministic, OCR-based stage targeting had been ported instead of replaced with a heuristic guess.

Going forward: when a reference feature's control flow depends on OCR, set up OCR and port that logic, rather than inventing a non-OCR workaround to avoid the setup cost. Only skip OCR for a specific step if the reference itself doesn't use OCR there.

The reference project is located at:

~/repo/baas-reference/

That repository contains the full Blue Archive Auto Script implementation. It should be treated as the behavioral reference for this project.

Core architecture decision

This project must be Python-first.

Do not implement new Blue Archive automation logic in ba_dailies.sh.

ba_dailies.sh is only a thin runtime/deployment launcher for convenience on nik-gpu.

All feature logic must live in Python.

The intended structure is:

ba-auto-daily/
├── ba_dailies.sh
├── ba_daily.py
├── ba_auto/
│   ├── __init__.py
│   ├── driver.py
│   ├── detector.py
│   ├── navigation.py
│   ├── config.py
│   ├── tasks/
│   │   ├── __init__.py
│   │   ├── mailbox.py
│   │   ├── cafe.py
│   │   ├── stamina.py
│   │   ├── group.py
│   │   └── ...
│   └── reference_notes/
│       └── mapping.md
├── assets/
├── screenshots/
├── setup.sh
├── plan.md
└── CLAUDE.md

(scripts/ previously held one-off Bash/Python helpers; it was deleted once mailbox and cafe migrated off it — see "Existing features" below. Do not recreate it as a place to stash feature logic.)

The exact layout can evolve, but the architectural rule should not change:

• Bash launches Python.
• Python owns automation logic.
• The reference repository guides feature behavior.
• Local driver primitives adapt that behavior to the PC/Steam/Proton setup.

Reference-first rule

Before implementing any new feature, inspect the matching reference implementation in:

~/repo/baas-reference/module/

Do not start by inventing a Bash click sequence.

For every feature, first identify:

1. Which reference file implements it.
2. Which class/function contains the main control flow.
3. What the reference uses for state detection.
4. What the reference uses for retries/failure handling.
5. Which parts depend on Android/uiautomator2 and must be replaced.
6. Which parts can be ported directly as Python control flow.
7. Which local driver primitives are missing.

Then implement the feature in Python under:

ba_auto/tasks/

Reference repository usage

The reference repository is read-only.

Allowed

• read reference modules
• inspect control flow
• reuse architecture ideas
• reuse retry/state-machine structure
• reuse task decomposition ideas
• reuse constants/config concepts when appropriate
• write local notes describing how a reference module maps to this project

Not allowed

• edit files in ~/repo/baas-reference/
• reimplement a reference feature from scratch in Bash
• create a local solution that ignores the reference flow when a reference implementation already exists
• substitute a pixel-probe, fixed-coordinate, or randomized shortcut for reference logic that uses OCR, just to avoid setting up OCR (see "OCR policy" above)

Two-machine architecture

Development happens on: nik-macbookair

Runtime happens on: nik-gpu

The Blue Archive client and X display live on nik-gpu.

Assume: (DISPLAY=:0, GDM XAUTHORITY under /run/user/1000)

The game runs under Steam/Proton on the Linux desktop.

There is no reliable local execution path on macOS. Anything that interacts with the game must be deployed to nik-gpu.

Deployment model

During iteration, files are pushed from nik-macbookair to nik-gpu.

Typical paths on nik-gpu:

~/ba_dailies.sh
~/ba_daily.py
~/ba_auto/
~/ba_assets/
~/.venvs/ba-auto-daily/

The current setup may still contain older paths such as:

~/ba_scripts/detect_and_click.py

When refactoring, prefer consolidating Python code into ba_auto/.

setup.sh should bootstrap or update the runtime layout on nik-gpu.

For a fresh checkout on nik-gpu, run from the repo root:

./setup.sh

After initial setup, individual changes may be pushed with scp or rsync.

Runtime dependencies on nik-gpu

These are host-level dependencies. Confirm they exist before assuming a bug is in the project code.

Required now:

• xdotool
• scrot
• python3

Required Python packages:

• opencv-python or opencv-python-headless
• numpy

Expected venv:

~/.venvs/ba-auto-daily/bin/python3

Quick check:

ssh nik-gpu "which xdotool scrot && ~/.venvs/ba-auto-daily/bin/python3 -c 'import cv2, numpy; print(cv2.__version__)'"

OCR engine dependency: likely Tesseract or PaddleOCR. Not set up yet, but no longer something to defer casually — set it up as soon as a feature's reference implementation depends on it, rather than inventing a non-OCR workaround (see "OCR policy" above).

Bash policy

ba_dailies.sh should be a thin launcher only.

Preferred shape:

#!/usr/bin/env bash
set -euo pipefail
VENV_PYTHON="${VENV_PYTHON:-$HOME/.venvs/ba-auto-daily/bin/python3}"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$VENV_PYTHON" "$SCRIPT_DIR/ba_daily.py" "$@"

Acceptable Bash responsibilities:

• choose Python binary
• set environment variables
• call Python entry point
• provide compatibility with old command names
• fail early if the Python entry point is missing

Not acceptable in Bash:

• feature state machines
• long click sequences
• OpenCV logic
• OCR logic
• retry loops for game states
• feature-specific navigation
• shop/sweep/battle logic
• new do_<feature> game automation functions

If an existing Bash function exists, migrate it to Python rather than extending it.

Python entry point

The intended Python CLI entry point is:

ba_daily.py

It should support commands such as:

./ba_dailies.sh
./ba_dailies.sh mailbox
./ba_dailies.sh cafe
./ba_dailies.sh stamina
./ba_dailies.sh story_sweep
./ba_dailies.sh group

No argument should run the default daily flow.

Current default flow (ba_daily.py's DEFAULT_ORDER):

1. focus game
2. mailbox
3. cafe
4. stamina
5. future daily tasks

story_sweep spends AP rather than reclaiming something free, so it is deliberately excluded from the default flow — it must be invoked explicitly.

The CLI should dispatch into task modules under ba_auto/tasks/.

Driver layer

Create and maintain a driver layer in:

ba_auto/driver.py

The driver layer should wrap the PC/Steam/Proton backend.

It should provide reusable primitives such as:

focus_game()
screenshot()
click(x, y)
double_click(x, y)
drag/swipe(start, end, duration)
keypress(key)
sleep/wait
wait_until(...)
color_at(...)
region_average_color(...)
template_match(...)
find_and_click_template(...)

Feature modules should not directly shell out to xdotool or scrot unless a driver primitive is missing and being added.

Prefer:

driver.click(x, y)

over:

subprocess.run(["xdotool", "click", ...])

This keeps the project close to the reference architecture while replacing only the backend-control layer.

Detector layer

Image matching and color matching should live in ba_auto/detector.py or in clearly named helper classes/functions.

The cafe sparkle-matching logic formerly in scripts/detect_and_click.py (now retired) has been migrated here as find_cafe_sparkle(), called in-process rather than as a per-click subprocess.

The detector should support:

• screenshot input
• template matching
• threshold tuning
• masked matching
• click-offset handling
• debug image output to scratchpad/

Avoid one Python cold start per click attempt where possible. Prefer long-running Python task logic that can take repeated screenshots and click repeatedly from one process.

Navigation layer

Common navigation should live in ba_auto/navigation.py.

Use this for shared flows such as:

• returning home
• opening main menu
• opening mailbox
• opening cafe
• opening shop
• opening lesson/schedule
• closing popups
• generic back/escape handling

Do not duplicate navigation click sequences inside every task if they can be shared.

Task modules

Each feature should have a task module: ba_auto/tasks/<feature>.py

Example:

ba_auto/tasks/mailbox.py
ba_auto/tasks/cafe.py
ba_auto/tasks/stamina.py
ba_auto/tasks/story_sweep.py
ba_auto/tasks/group.py

Each task module should expose a clear function such as:

run(driver, config)

or:

run_mailbox(driver, config)

Keep task files feature-focused.

Reference mapping notes

Maintain a mapping file at:

ba_auto/reference_notes/mapping.md

Before or during implementation of a feature, update the mapping.

Use this format:

Local feature	Reference file	Reference functions/classes	Local file	Backend replacements	Status
Cafe	module/cafe.py or relevant file	...	ba_auto/tasks/cafe.py	uiautomator2 tap -> xdotool click, screenshot -> scrot/OpenCV	In progress

Working conventions

Use scratchpad/ (create if missing) in the project root for temporary or intermediate files.

Examples:

• cropped calibration images, e.g. from screenshots/cafe/sparkle/
• debug screenshots
• annotated match results
• temporary investigation notes

Do not use /tmp or /private/tmp unless there is a strong reason.

screenshots/ contains human reference captures. They are useful for calibration and documentation, but they are not necessarily automated test fixtures.

assets/ contains local template images. These should be captured from the local game setup where possible.

Testing and checks

Because the game only runs on nik-gpu, local macOS testing is limited.

Before deploying, run static/syntax checks locally:

bash -n ba_dailies.sh
python3 -m py_compile ba_daily.py
python3 -m py_compile ba_auto/*.py
python3 -m py_compile ba_auto/tasks/*.py

On nik-gpu, run real integration tests against the live game.

Example:

ssh nik-gpu "~/ba_dailies.sh cafe"

When debugging image matching, write debug images to scratchpad/.

Existing features

Current project state (mailbox, cafe, stamina, and story_sweep all migrated to real Python; no Bash feature logic remains):

• ba_dailies.sh is a thin launcher that execs ba_daily.py
• ba_daily.py dispatches mailbox/cafe/stamina/story_sweep/default flow to ba_auto/tasks/. Default flow (DEFAULT_ORDER) is mailbox, cafe, stamina — story_sweep is opt-in only since it spends AP rather than reclaiming something free
• ba_auto/tasks/mailbox.py and ba_auto/tasks/cafe.py click with ba_auto/driver.py primitives and verify state with driver.color_at/ba_auto/navigation.py (ported from module/mail.py and module/cafe_reward.py's rgb_in_range/co_detect pattern) before pressing further keys — no legacy bridge remains
• ba_auto/tasks/stamina.py claims the Mission panel's bulk "一括受取" button (see plan.md Phase 8)
• ba_auto/tasks/story_sweep.py sweeps the latest unlocked region's stages for AP (see plan.md Phase 9, including its OCR-avoidance retrospective — this task is the concrete example behind the "OCR policy" above)
• ba_auto/detector.py has find_cafe_sparkle(), the sparkle template-match ported in-process from the now-deleted scripts/detect_and_click.py
• scripts/ba_dailies_legacy.sh and scripts/detect_and_click.py have been deleted — nothing references them anymore, and the scripts/ directory itself no longer exists
• ba_auto/driver.py primitives (run_command, focus_game, click, move_mouse, scroll, keypress, screenshot, wait, color_at) are wired into all four task modules
• ba_auto/navigation.py has two shared state probes used across tasks: is_on_subscreen (any mailbox/cafe/shop-style panel vs. the home screen) and is_modal_open (a dimmed dialog overlay). story_sweep.py additionally has its own local modal probes/close logic because the stage-info modal is wide enough to break is_modal_open's default probe point, and does not close on Escape at all (see plan.md Phase 9)
• Live testing found both the mailbox-icon and cafe-icon fixed coordinates were flaky (missed the first click, worked on retry) and that neither task verified anything before proceeding, so a missed click cascaded into blind actions and could reach an unverified Escape press on the home screen — which triggers Blue Archive's own "exit the game?" confirmation. See plan.md Phases 5–6 for the full writeup. This is the concrete reason every task now verifies state before acting rather than trusting fixed coordinates or a single click blindly
• Not yet verified for cafe: rank-up popups mid-pat-loop, and whether camera zoom/pan can drift over a long unattended run (the reference project zooms out before detecting; ours does not, and testing didn't reproduce a failure from skipping it — see plan.md Phase 6 "Not verified" list)

Migration goal:

• preserve existing behavior
• move feature logic from Bash into Python
• keep ba_dailies.sh as launcher only
• make detection logic reusable
• prepare for future features by copying the reference project's structure where appropriate

Known cafe gaps to verify during migration

When migrating cafe logic, verify these manually against the actual current behavior:

• whether student rotation popups are handled
• whether rank-up popups are handled
• whether the view is zoomed/centered before sparkle detection
• whether detection still works when text overlaps the student
• whether one Python process can handle repeated sparkle detection faster than repeated cold starts
• whether both cafe rooms are handled consistently
• whether cafe income claim is robust against popup timing

Do not assume the old Bash implementation handles these correctly.

Anti-patterns

Do not:

• add new do_<feature> Bash functions
• build a giant Bash automation script
• recreate reference logic as fixed coordinate Bash click chains
• skip reading the reference module before implementing a feature
• put OCR in Bash
• put OpenCV state machines in Bash
• make every feature shell out independently to xdotool
• duplicate common navigation in every task
• copy reference image assets blindly
• edit ~/repo/baas-reference/
• implement event-specific features before the generic reusable machinery exists
• invent a pixel-probe, fixed-coordinate, or randomized substitute for reference logic that uses OCR, just to avoid setting up OCR
• treat OCR as an Android-specific concern that can be deferred indefinitely — it isn't, and deferring it has already produced more bugs than it saved (see "OCR policy" above)

Feature implementation workflow

For every new feature:

1. Read the relevant ~/repo/baas-reference/module/... file.
2. Summarize the upstream feature flow in notes or comments.
3. Add/update the reference mapping table.
4. Identify missing local driver primitives.
5. Implement or improve those primitives in ba_auto/driver.py or ba_auto/detector.py.
6. Implement the feature in ba_auto/tasks/<feature>.py.
7. Add CLI dispatch in ba_daily.py.
8. Keep ba_dailies.sh unchanged unless launcher behavior changes.
9. Run syntax checks.
10. Deploy to nik-gpu.
11. Test against the live game.
12. Update plan.md status.

Priority when uncertain

When uncertain, prefer this order:

1. Preserve existing working behavior.
2. Follow the reference project's control flow — including its use of OCR. Do not deprioritize OCR-based navigation/matching in favor of a simpler non-OCR substitute; that has already produced more bugs than the OCR setup would have cost (see "OCR policy" above).
3. Keep logic in Python.
4. Add reusable driver primitives instead of feature-specific hacks.
5. Use local screenshots/assets only when backend differences require it.
6. Avoid large rewrites that do not move the project closer to reference-driven Python architecture.

User preference

The user wants this project to be as close to the original Blue Archive Auto Script architecture as practical, without recreating logic that already exists.

The user specifically does not want Claude Code to keep converting feature work into Bash.

Respect that preference.