nik/ba-auto-daily
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor project structure and add setup script

Browse Source 
- Transitioned the project to a Python-first architecture, moving feature logic from Bash to Python.
- Created a new `setup.sh` script to bootstrap the environment on `nik-gpu`, ensuring necessary tools are installed and setting up a Python virtual environment.
- Updated project layout in `plan.md` to reflect the new structure and clarify the purpose of each component.
- Established a reference mapping table for feature implementation based on the existing reference project.
- Outlined a migration phase to transition existing Bash functionality to Python tasks.
This commit is contained in:
Nik Afiq 2026-07-05 15:24:45 +09:00
parent 0ee1d495c8
commit ebce31156b
3 changed files with 1270 additions and 161 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

509
CLAUDE.md
View File

@ -1,59 +1,504 @@
# CLAUDE.md # CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. This file provides guidance to Claude Code when working with code in this repository.
## What this is ## What this project is
Automation for daily chores in the mobile game Blue Archive: claiming mailbox rewards and farming/collecting cafe affection income. It drives the actual game client via simulated mouse clicks and keypresses (`xdotool`) against fixed screen coordinates, plus one template-matching detector for a dynamic (moving) UI element. This repository is a personal Blue Archive JP daily-automation project.
It controls the real Blue Archive PC/Steam/Proton client running on a Linux machine named `nik-gpu`. The development machine is a MacBook named `nik-macbookair`.
The automation backend is local desktop control:
- `xdotool` for mouse/keyboard/window control
- `scrot` for screenshots
- Python/OpenCV for image matching and color/template detection
- OCR later, when needed
- no Android emulator control
- no ADB
- no uiautomator2
The reference project is located at:
```
~/repo/baas-reference/
```
That repository contains the full Blue Archive Auto Script implementation. It should be treated as the behavioral reference for this project.
## Core architecture decision
This project must be Python-first.
**Do not implement new Blue Archive automation logic in `ba_dailies.sh`.**
`ba_dailies.sh` is only a thin runtime/deployment launcher for convenience on `nik-gpu`.
All feature logic must live in Python.
The intended structure is:
```
ba-auto-daily/
├── ba_dailies.sh
├── ba_daily.py
├── ba_auto/
│ ├── __init__.py
│ ├── driver.py
│ ├── detector.py
│ ├── navigation.py
│ ├── config.py
│ ├── tasks/
│ │ ├── __init__.py
│ │ ├── mailbox.py
│ │ ├── cafe.py
│ │ ├── stamina.py
│ │ ├── group.py
│ │ └── ...
│ └── reference_notes/
│ └── mapping.md
├── assets/
├── screenshots/
├── scripts/
├── setup.sh
├── plan.md
└── CLAUDE.md
```
The exact layout can evolve, but the architectural rule should not change:
- Bash launches Python.
- Python owns automation logic.
- The reference repository guides feature behavior.
- Local driver primitives adapt that behavior to the PC/Steam/Proton setup.
## Reference-first rule
Before implementing any new feature, inspect the matching reference implementation in:
```
~/repo/baas-reference/module/
```
Do not start by inventing a Bash click sequence.
For every feature, first identify:
1. Which reference file implements it.
2. Which class/function contains the main control flow.
3. What the reference uses for state detection.
4. What the reference uses for retries/failure handling.
5. Which parts depend on Android/uiautomator2 and must be replaced.
6. Which parts can be ported directly as Python control flow.
7. Which local driver primitives are missing.
Then implement the feature in Python under:
```
ba_auto/tasks/
```
## Reference repository usage
The reference repository is **read-only**.
### Allowed
- read reference modules
- inspect control flow
- reuse architecture ideas
- reuse retry/state-machine structure
- reuse task decomposition ideas
- reuse constants/config concepts when appropriate
- write local notes describing how a reference module maps to this project
### Not allowed
- edit files in `~/repo/baas-reference/`
- reimplement a reference feature from scratch in Bash
- create a local solution that ignores the reference flow when a reference implementation already exists
## Two-machine architecture ## Two-machine architecture
This repo is developed on macOS (`nik-macbookair`) but the code only runs on a separate Linux box, `nik-gpu`, where the Blue Archive client and its X display actually live (`DISPLAY=:0`, GDM `XAUTHORITY` under `/run/user/1000`). There is no local way to execute or test these scripts — they must be deployed to `nik-gpu` to run for real. Development happens on: `nik-macbookair`
- `ba_dailies.sh` and `scripts/detect_and_click.py` are pushed to `nik-gpu` with `scp`/`rsync` (e.g. `scp ba_dailies.sh nik-gpu:~/ba_dailies.sh`, `scp scripts/detect_and_click.py nik-gpu:~/ba_scripts/detect_and_click.py`) and invoked there over `ssh`. Runtime happens on: `nik-gpu`
- `assets/cafe_sparkle.png` (the template image) is likewise copied to `nik-gpu:~/ba_assets/cafe_sparkle.png`.
- `detect_and_click.py` runs a Python venv on `nik-gpu` at `~/.venvs/ba-auto-daily/bin/python3` (needs `opencv-python`/`numpy`); `ba_dailies.sh` invokes it via that fixed path (`VENV_PYTHON` in the script).
- Screenshot -> detect -> click for the cafe sparkle happens entirely on `nik-gpu` in one local pipeline (see the docstring in `scripts/detect_and_click.py`) rather than round-tripping images over SSH, because the sparkle target moves fast enough that a multi-hop pipeline would miss the click.
- `screenshots/` holds reference captures (taken on `nik-gpu`, pulled back for inspection) used to work out coordinates and template thresholds when coordinates drift after a game UI update — they are not test fixtures consumed by any script.
Because there's no local execution path, "testing a change" means syntax-checking locally and then deploying to `nik-gpu` and running it against the live game there: The Blue Archive client and X display live on `nik-gpu`.
```bash
bash -n ba_dailies.sh Assume: (`DISPLAY=:0`, GDM `XAUTHORITY` under `/run/user/1000`)
python3 -m py_compile scripts/detect_and_click.py
The game runs under Steam/Proton on the Linux desktop.
There is no reliable local execution path on macOS. Anything that interacts with the game must be deployed to `nik-gpu`.
## Deployment model
During iteration, files are pushed from `nik-macbookair` to `nik-gpu`.
Typical paths on `nik-gpu`:
```
~/ba_dailies.sh
~/ba_daily.py
~/ba_auto/
~/ba_assets/
~/.venvs/ba-auto-daily/
``` ```
## Dependencies on `nik-gpu` The current setup may still contain older paths such as:
These are host-level prerequisites, not managed by this repo — confirm they're present before assuming a failure is a code bug: ```
- `xdotool` (window focus, clicks, keypresses) ~/ba_scripts/detect_and_click.py
- `scrot` (screenshot capture for the detector) ```
- A venv at `~/.venvs/ba-auto-daily/` with `opencv-python` and `numpy` installed, Python binary at `~/.venvs/ba-auto-daily/bin/python3`
When refactoring, prefer consolidating Python code into `ba_auto/`.
`setup.sh` should bootstrap or update the runtime layout on `nik-gpu`.
For a fresh checkout on `nik-gpu`, run from the repo root:
```
./setup.sh
```
After initial setup, individual changes may be pushed with `scp` or `rsync`.
## Runtime dependencies on nik-gpu
These are host-level dependencies. Confirm they exist before assuming a bug is in the project code.
Required now:
- `xdotool`
- `scrot`
- `python3`
Required Python packages:
- `opencv-python` or `opencv-python-headless`
- `numpy`
Expected venv:
```
~/.venvs/ba-auto-daily/bin/python3
```
Quick check: Quick check:
```bash
```
ssh nik-gpu "which xdotool scrot && ~/.venvs/ba-auto-daily/bin/python3 -c 'import cv2, numpy; print(cv2.__version__)'" ssh nik-gpu "which xdotool scrot && ~/.venvs/ba-auto-daily/bin/python3 -c 'import cv2, numpy; print(cv2.__version__)'"
``` ```
Future dependency: OCR engine, likely Tesseract or PaddleOCR.
Do not introduce OCR casually. Add it only when implementing a feature that actually needs OCR.
## Bash policy
`ba_dailies.sh` should be a thin launcher only.
Preferred shape:
```bash
#!/usr/bin/env bash
set -euo pipefail
VENV_PYTHON="${VENV_PYTHON:-$HOME/.venvs/ba-auto-daily/bin/python3}"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$VENV_PYTHON" "$SCRIPT_DIR/ba_daily.py" "$@"
```
Acceptable Bash responsibilities:
- choose Python binary
- set environment variables
- call Python entry point
- provide compatibility with old command names
- fail early if the Python entry point is missing
Not acceptable in Bash:
- feature state machines
- long click sequences
- OpenCV logic
- OCR logic
- retry loops for game states
- feature-specific navigation
- shop/sweep/battle logic
- new `do_<feature>` game automation functions
If an existing Bash function exists, migrate it to Python rather than extending it.
## Python entry point
The intended Python CLI entry point is:
```
ba_daily.py
```
It should support commands such as:
```
./ba_dailies.sh
./ba_dailies.sh mailbox
./ba_dailies.sh cafe
./ba_dailies.sh stamina
./ba_dailies.sh group
```
No argument should run the default daily flow.
Example default flow:
1. focus game
2. mailbox
3. cafe
4. future daily tasks
The CLI should dispatch into task modules under `ba_auto/tasks/`.
## Driver layer
Create and maintain a driver layer in:
```
ba_auto/driver.py
```
The driver layer should wrap the PC/Steam/Proton backend.
It should provide reusable primitives such as:
```
focus_game()
screenshot()
click(x, y)
double_click(x, y)
drag/swipe(start, end, duration)
keypress(key)
sleep/wait
wait_until(...)
color_at(...)
region_average_color(...)
template_match(...)
find_and_click_template(...)
```
Feature modules should not directly shell out to `xdotool` or `scrot` unless a driver primitive is missing and being added.
Prefer:
```python
driver.click(x, y)
```
over:
```python
subprocess.run(["xdotool", "click", ...])
```
This keeps the project close to the reference architecture while replacing only the backend-control layer.
## Detector layer
Image matching and color matching should live in `ba_auto/detector.py` or in clearly named helper classes/functions.
Existing logic from `scripts/detect_and_click.py` should be migrated into reusable Python functions.
The detector should support:
- screenshot input
- template matching
- threshold tuning
- masked matching
- click-offset handling
- debug image output to `./scratchpad`
Avoid one Python cold start per click attempt where possible. Prefer long-running Python task logic that can take repeated screenshots and click repeatedly from one process.
## Navigation layer
Common navigation should live in `ba_auto/navigation.py`.
Use this for shared flows such as:
- returning home
- opening main menu
- opening mailbox
- opening cafe
- opening shop
- opening lesson/schedule
- closing popups
- generic back/escape handling
Do not duplicate navigation click sequences inside every task if they can be shared.
## Task modules
Each feature should have a task module: `ba_auto/tasks/<feature>.py`
Example:
```
ba_auto/tasks/mailbox.py
ba_auto/tasks/cafe.py
ba_auto/tasks/group.py
ba_auto/tasks/stamina.py
```
Each task module should expose a clear function such as:
```
run(driver, config)
```
or:
```
run_mailbox(driver, config)
```
Keep task files feature-focused.
## Reference mapping notes
Maintain a mapping file at:
```
ba_auto/reference_notes/mapping.md
```
Before or during implementation of a feature, update the mapping.
Use this format:
| Local feature | Reference file | Reference functions/classes | Local file | Backend replacements | Status |
|---|---|---|---|---|---|
| Cafe | `module/cafe.py` or relevant file | `...` | `ba_auto/tasks/cafe.py` | uiautomator2 tap -> xdotool click, screenshot -> scrot/OpenCV | In progress |
## Working conventions ## Working conventions
Use `./scratchpad` (create if missing) in the project root for temporary/intermediate files — e.g. cropped calibration images from `screenshots/cafe/sparkle/`, one-off debug output. Never write to `/tmp` or `/private/tmp`. Use `./scratchpad` for temporary or intermediate files.
## `ba_dailies.sh` Examples:
Entry point, run on `nik-gpu` as `./ba_dailies.sh [mailbox|cafe]`: - cropped calibration images
- No argument: focuses the game window, runs mailbox claim, then the full cafe routine. - debug screenshots
- `mailbox`: focus + claim mailbox only. - annotated match results
- `cafe`: focus + cafe routine only (both rooms + income claim). - temporary investigation notes
All interaction points (icon/button coordinates, max click-attempts per cafe room) are top-of-file constants — when the in-game UI shifts or the window resolution changes, update the coordinates there rather than inline in the functions. `focus_game` finds the window via `xdotool search --name "BlueArchive"` and hard-fails if the game isn't running. Do not use `/tmp` or `/private/tmp` unless there is a strong reason.
The cafe routine (`do_cafe`) alternates between two cafe rooms; for each room it repeatedly calls into `detect_and_click.py` to find and click affection sparkles (up to `CAFE_MAX_CLICKS_PER_ROOM` times) before moving on, then claims cafe income at the end. `screenshots/` contains human reference captures. They are useful for calibration and documentation, but they are not necessarily automated test fixtures.
**Known gap — needs verification:** the manual cafe flow this automates includes a few conditional steps not obviously covered above: closing a popup that only appears if a student is "rotated," zooming out/centering the view before detection starts, and closing a rank-up popup that can appear after a successful click. Confirm these are actually handled somewhere in `do_cafe` (or decide they're unnecessary in practice) rather than assuming coverage from this description alone. `assets/` contains local template images. These should be captured from the local game setup where possible.
## `scripts/detect_and_click.py` ## Testing and checks
Standalone script (runs on `nik-gpu`, called once per sparkle-click attempt from the shell loop): screenshots the game window with `scrot`, template-matches `cafe_sparkle.png` using a color-masked `cv2.matchTemplate` (masks to yellow/white sparkle pixels so it doesn't match on background art), clicks the best match (offset-corrected — the template's anchor point isn't the click point), and reports its result on stdout/exit code (`MATCH x y score` / exit 0, or `NO_MATCH` / exit 1) so the caller shell loop can decide whether to keep clicking. Because the game only runs on `nik-gpu`, local macOS testing is limited.
`THRESHOLD = 0.97` and `OFFSET_X`/`OFFSET_Y` are the values most likely to need retuning if detection starts missing or mis-clicking — use the `screenshots/cafe/sparkle/` reference captures to recalibrate. When cropping or annotating these captures for calibration, use `./scratchpad` in the project root for the intermediate files, not `/tmp`. Before deploying, run static/syntax checks locally:
**Performance note — needs verification:** this script is invoked as a fresh process per click attempt, and Python + OpenCV cold-start has real overhead (commonly 200–500ms). If the sparkle target moves fast, confirm this hasn't caused missed detections in practice against the live game. If it has, consider a persistent process the shell loop talks to (pipe/socket) instead of a per-attempt cold start. ```
bash -n ba_dailies.sh
python3 -m py_compile ba_daily.py
python3 -m py_compile ba_auto/*.py
python3 -m py_compile ba_auto/tasks/*.py
```
On `nik-gpu`, run real integration tests against the live game.
Example:
```
ssh nik-gpu "~/ba_dailies.sh cafe"
```
When debugging image matching, write debug images to `./scratchpad`.
## Existing features
Current project state before Python-first refactor:
- mailbox claim exists
- cafe affection/income exists
- cafe uses OpenCV template matching for sparkle detection
- existing Python helper is `scripts/detect_and_click.py`
- existing Bash entry point is `ba_dailies.sh`
Migration goal:
- preserve existing behavior
- move feature logic from Bash into Python
- keep `ba_dailies.sh` as launcher only
- make detection logic reusable
- prepare for future features by copying the reference project's structure where appropriate
## Known cafe gaps to verify during migration
When migrating cafe logic, verify these manually against the actual current behavior:
- whether student rotation popups are handled
- whether rank-up popups are handled
- whether the view is zoomed/centered before sparkle detection
- whether detection still works when text overlaps the student
- whether one Python process can handle repeated sparkle detection faster than repeated cold starts
- whether both cafe rooms are handled consistently
- whether cafe income claim is robust against popup timing
Do not assume the old Bash implementation handles these correctly.
## Anti-patterns
Do not:
- add new `do_<feature>` Bash functions
- build a giant Bash automation script
- recreate reference logic as fixed coordinate Bash click chains
- skip reading the reference module before implementing a feature
- put OCR in Bash
- put OpenCV state machines in Bash
- make every feature shell out independently to `xdotool`
- duplicate common navigation in every task
- copy reference image assets blindly
- edit `~/repo/baas-reference/`
- implement event-specific features before the generic reusable machinery exists
## Feature implementation workflow
For every new feature:
1. Read the relevant `~/repo/baas-reference/module/...` file.
2. Summarize the upstream feature flow in notes or comments.
3. Add/update the reference mapping table.
4. Identify missing local driver primitives.
5. Implement or improve those primitives in `ba_auto/driver.py` or `ba_auto/detector.py`.
6. Implement the feature in `ba_auto/tasks/<feature>.py`.
7. Add CLI dispatch in `ba_daily.py`.
8. Keep `ba_dailies.sh` unchanged unless launcher behavior changes.
9. Run syntax checks.
10. Deploy to `nik-gpu`.
11. Test against the live game.
12. Update `plan.md` status.
## Priority when uncertain
When uncertain, prefer this order:
1. Preserve existing working behavior.
2. Follow the reference project's control flow.
3. Keep logic in Python.
4. Add reusable driver primitives instead of feature-specific hacks.
5. Use local screenshots/assets only when backend differences require it.
6. Avoid large rewrites that do not move the project closer to reference-driven Python architecture.
## User preference
The user wants this project to be as close to the original Blue Archive Auto Script architecture as practical, without recreating logic that already exists.
The user specifically does not want Claude Code to keep converting feature work into Bash.
Respect that preference.

874
plan.md
View File

@ -1,170 +1,786 @@
# ba-auto-daily implementation plan # ba-auto-daily implementation plan
Personal Blue Archive **JP** daily-automation script. Controls the game via `xdotool` Personal Blue Archive JP daily-automation project.
over a remote X11 session (Steam+Proton on `nik-gpu`), using OpenCV template/color
matching for detection. No OCR is set up yet (see Prerequisites).
Feature list and priority ordering below is derived from studying This project controls the PC/Steam/Proton Blue Archive client running on `nik-gpu` through local desktop automation:
`~/repo/baas-reference` (pur1fying/blue_archive_auto_script, GPL-3.0, read-only
reference — **not copied from**, reimplemented fresh in our own style). Its own - xdotool
default scheduler priority, complexity, and JP-relevance were used to rank the - scrot
backlog. See "Reference" under each feature for the baas file(s) that describe the - Python
mechanic, for research purposes only. - OpenCV
- later OCR when needed
Development happens on `nik-macbookair`.
The reference implementation lives at:
```
~/repo/baas-reference/
```
The reference project should be treated as the behavioral blueprint. This project should avoid recreating feature logic from scratch when the reference already implements it.
## Core project direction
This project is now Python-first.
The goal is not to grow a large Bash script.
The goal is to build a small local Python automation framework that adapts the reference project's Blue Archive logic to this user's unique PC/Steam/Proton environment.
`ba_dailies.sh` should only be a launcher.
Feature logic should live in Python.
## Target architecture
```
~/repo/ba-auto-daily/
├── ba_dailies.sh
├── ba_daily.py
├── ba_auto/
│ ├── __init__.py
│ ├── config.py
│ ├── driver.py
│ ├── detector.py
│ ├── navigation.py
│ ├── tasks/
│ │ ├── __init__.py
│ │ ├── mailbox.py
│ │ ├── cafe.py
│ │ ├── stamina.py
│ │ ├── group.py
│ │ ├── bounty.py
│ │ ├── commission.py
│ │ ├── arena.py
│ │ ├── shop_common.py
│ │ ├── shop_tactical.py
│ │ ├── lesson.py
│ │ └── ...
│ └── reference_notes/
│ └── mapping.md
├── assets/
├── screenshots/
├── scripts/
├── setup.sh
├── CLAUDE.md
└── plan.md
```
This is the intended direction. It does not have to be completed all at once.
## Project layout
| Path | What it is |
|---|---|
| `~/repo/ba-auto-daily/ba_dailies.sh` | Thin launcher only. It should call the Python entry point. Do not add new feature logic here. |
| `~/repo/ba-auto-daily/ba_daily.py` | Main Python CLI entry point. Dispatches tasks such as mailbox, cafe, stamina, group, etc. |
| `~/repo/ba-auto-daily/ba_auto/driver.py` | Local PC/Steam/Proton control backend. Wraps xdotool, scrot, waits, clicks, swipes, keypresses, screenshots, and window focus. |
| `~/repo/ba-auto-daily/ba_auto/detector.py` | OpenCV/template/color matching helpers. Existing `scripts/detect_and_click.py` logic should be migrated here. |
| `~/repo/ba-auto-daily/ba_auto/navigation.py` | Shared navigation helpers: home, menu, close popup, back, open feature screens. |
| `~/repo/ba-auto-daily/ba_auto/tasks/` | Feature implementations. Each task should adapt the relevant `baas-reference/module/...` logic where possible. |
| `~/repo/ba-auto-daily/ba_auto/reference_notes/mapping.md` | Reference mapping table: local feature → reference module → local implementation → driver gaps. |
| `~/repo/ba-auto-daily/assets/` | Locally captured template images, such as cafe sparkle. Do not blindly copy assets from the reference repo. |
| `~/repo/ba-auto-daily/screenshots/` | Human reference screenshots, mostly Moonlight/game captures, used for calibration and debugging. |
| `~/repo/ba-auto-daily/scripts/` | Transitional scripts. Long-term reusable Python logic should move into `ba_auto/`. |
| `~/repo/ba-auto-daily/setup.sh` | Bootstrap/deploy helper for `nik-gpu`. Should install/check dependencies and copy runtime files. |
| `~/repo/baas-reference/` | Read-only GPL-3.0 reference clone. Study and adapt. Never edit. |
## Runtime paths on nik-gpu
Preferred runtime layout:
| Path | What it is |
|---|---|
| `nik-gpu:~/ba_dailies.sh` | Thin launcher. |
| `nik-gpu:~/ba_daily.py` | Python CLI entry point. |
| `nik-gpu:~/ba_auto/` | Python package copied from this repo. |
| `nik-gpu:~/ba_assets/` | Runtime assets/templates. |
| `nik-gpu:~/.venvs/ba-auto-daily/` | Python virtual environment. |
Current older layout may include:
| Path | What it is |
|---|---|
| `nik-gpu:~/ba_scripts/detect_and_click.py` | Old standalone detector helper. Should eventually be replaced by `ba_auto/detector.py`. |
## Implementation strategy
For each feature:
1. Read the matching `~/repo/baas-reference/module/...` file.
2. Summarize the reference flow.
3. Identify reusable logic:
- state checks
- retry loops
- navigation sequence
- battle/sweep/shop rules
- detection method
- failure handling
4. Identify backend-specific calls that cannot be reused directly.
5. Implement missing generic primitives in `ba_auto/driver.py` or `ba_auto/detector.py`.
6. Implement the feature in `ba_auto/tasks/<feature>.py`.
7. Add CLI command dispatch in `ba_daily.py`.
8. Keep `ba_dailies.sh` unchanged unless launcher behavior changes.
9. Test syntax locally.
10. Deploy to `nik-gpu`.
11. Run against the live game.
12. Update this plan.
The intended result is not a Bash automation script.
The intended result is a Python automation framework using the reference repository as the behavioral blueprint.
## Reference mapping table
Maintain this table in `ba_auto/reference_notes/mapping.md`.
Initial seed:
| Local feature | Reference file | Reference functions/classes | Local file | Backend replacements | Status |
|---|---|---|---|---|---|
| Mailbox | Need to confirm in reference | Need to inspect | `ba_auto/tasks/mailbox.py` | tap/click via xdotool, screenshot via scrot | Existing Bash behavior; migrate to Python |
| Cafe | Need to confirm in reference | Need to inspect | `ba_auto/tasks/cafe.py` | template matching via OpenCV, click via xdotool | Existing Bash + helper behavior; migrate to Python |
| Stamina/AP | `module/collect_daily_free_power.py`, `module/collect_daily_task_power.py` | Need to inspect | `ba_auto/tasks/stamina.py` | color checks/clicks via local driver | Not started |
| Group/Club AP | `module/group.py` | Need to inspect | `ba_auto/tasks/group.py` | fixed click + state check via local driver | Not started |
| Bounty | `module/rewarded_task.py` | Need to inspect | `ba_auto/tasks/bounty.py` | sweep/color/OCR adaptation | Not started |
| Commissions | `module/clear_special_task_power.py` | Need to inspect | `ba_auto/tasks/commission.py` | sweep/color adaptation | Not started |
| Arena | `module/arena.py` | Need to inspect | `ba_auto/tasks/arena.py` | auto-fight + OCR + local driver | Not started |
| Common Shop | `module/shop/common_shop.py`, `module/shop/shop_utils.py` | Need to inspect | `ba_auto/tasks/shop_common.py` | OCR + tab navigation + local clicks | Not started |
| Tactical Shop | `module/shop/tactical_challenge_shop.py`, `module/shop/shop_utils.py` | Need to inspect | `ba_auto/tasks/shop_tactical.py` | OCR + tab navigation + local clicks | Not started |
| Lesson/Schedule | `module/lesson.py` | Need to inspect | `ba_auto/tasks/lesson.py` | OCR + template/portrait search + local driver | Not started |
Do not implement a feature without filling at least the relevant row.
## Status snapshot ## Status snapshot
| Feature | Status | | Feature | Current status | Target status |
|---|---| |---|---|---|
| Mailbox claim | ✅ Done (`do_mailbox`) | | Mailbox claim | Done in old Bash style | Migrate to `ba_auto/tasks/mailbox.py` |
| Cafe (pats + income) | ✅ Done (`do_cafe`) | | Cafe pats + income | Done in old Bash + standalone Python detector style | Migrate to `ba_auto/tasks/cafe.py` and `ba_auto/detector.py` |
| Everything else below | Not started | | Shared driver | Partial/implicit in scripts | Build `ba_auto/driver.py` |
| Python CLI | Missing | Build `ba_daily.py` |
| Reference mapping | Missing | Build `ba_auto/reference_notes/mapping.md` |
| Everything else | Not started | Implement reference-first in Python |
## Prerequisites (cross-cutting, unblock multiple tiers) ## Migration phase
- **OCR.** Not set up. Needed for currency/ticket-count readouts, region/tab name Before adding new game features, migrate the existing working implementation.
search, and a few other read-a-number moments. When we get there, evaluate
Tesseract vs PaddleOCR in the existing `~/.venvs/ba-auto-daily` venv. Until then,
stick to features that are pure color-check / template-match.
- **Auto-fight primitive.** A shared "start fight → max speed + auto-mode →
detect win/lose" routine that Arena, and any future story/raid automation, all
need. Build once, early, before Tier 3 combat features.
Reference: `module/main_story.py` (`auto_fight`/`enter_battle`).
--- ### Phase 1: Python skeleton
## Tier 1 — High priority, no OCR needed Create:
### Stamina/AP sweep (free + task-menu power) ```
Claim the daily free 10 AP purchase and the daily task-menu AP/pyroxene rewards. ba_daily.py
Pure color-check state detection, fixed clicks. High value: AP is capped and this ba_auto/__init__.py
is free currency/materials left on the table every day. ba_auto/config.py
Reference: `module/collect_daily_free_power.py`, `module/collect_daily_task_power.py`. ba_auto/driver.py
ba_auto/detector.py
ba_auto/navigation.py
ba_auto/tasks/__init__.py
ba_auto/tasks/mailbox.py
ba_auto/tasks/cafe.py
ba_auto/reference_notes/mapping.md
```
### Club/Group AP claim ### Phase 2: Launcher
Claim 10 AP from the club/dorm menu. Trivial fixed click + color check.
Reference: `module/group.py` (29 lines).
### Momo Talk (chat app conversations) Change `ba_dailies.sh` into a thin launcher:
Auto-complete unread chat conversations, including relationship-rank-up story
beats, claim pyroxene. No OCR — pure pixel-column scanning state machine
(reply bubble / "enter story" prompt detection by color). Runs on its own ~3h
cadence like cafe. Moderate build effort but no new dependencies.
Reference: `module/momo_talk.py` (183 lines, `getConversationState`).
--- ```bash
#!/usr/bin/env bash
set -euo pipefail
VENV_PYTHON="${VENV_PYTHON:-$HOME/.venvs/ba-auto-daily/bin/python3}"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$VENV_PYTHON" "$SCRIPT_DIR/ba_daily.py" "$@"
```
## Tier 2 — Medium priority, sweep-style AP consumption Keep compatibility with:
These three share the same shape: buy tickets (optional), sweep N times per ```
sub-area, detected mostly by color (a multi-pixel "sss/star-rank" signature, not ./ba_dailies.sh
OCR). Minor OCR only for optional coin-balance readouts — can hardcode/skip that ./ba_dailies.sh mailbox
part initially and just sweep until out of stamina/tickets. ./ba_dailies.sh cafe
```
### Commissions ("Special Task" / 委託) ### Phase 3: Driver extraction
Two sub-dungeons (Base Defense = EXP, Item Retrieval = credits).
Reference: `module/clear_special_task_power.py` (148 lines).
### Scrimmage (学院交流会) Move shell interactions into `ba_auto/driver.py`.
Three academies, ticket purchase available on JP.
Reference: `module/scrimmage.py` (166 lines).
### Bounty ("Rewarded Task" / 悬赏通缉) Driver primitives should include:
Three sub-areas, ticket purchase (0–12). Needs a little OCR for bounty-coin
balance if we want auto-refresh; otherwise skippable.
Reference: `module/rewarded_task.py` (204 lines).
### Normal/Hard story AP sweep (re-clear already-cleared stages) ```
Sweep already-cleared main-story stages by region/mission count to burn AP. focus_game()
Needs OCR for current-region-number + a fuzzy swipe-search over mission click(x, y)
buttons — more OCR-dependent than the three above, so ranked after them. double_click(x, y)
Reference: `module/explore_tasks/sweep_task.py`, `module/explore_tasks/task_utils.py`. keypress(key)
screenshot(path=None)
swipe(...)
wait(seconds)
wait_until(...)
```
--- ### Phase 4: Detector extraction
## Tier 3 — Needs real OCR investment Move `scripts/detect_and_click.py` logic into `ba_auto/detector.py`.
Do these once OCR is actually set up. Detector primitives should include:
### Battle Pass claim (JP-only feature upstream — relevant to us) ```
Claim pass mission points + tier rewards. Simple template claim-flow; OCR only load_template(...)
needed for the optional level/points stat readout (can stub that part). match_template(...)
Reference: `module/collect_pass_reward.py` (146 lines). find_best_match(...)
find_and_click_template(...)
color_mask(...)
debug_write_match(...)
```
### Common Shop + Tactical (Arena) Shop The old script may remain as a compatibility wrapper temporarily, but the reusable logic should live under `ba_auto/`.
Auto-buy configured items, auto-refresh when affordable. OCR needed for
currency balances and shop-tab name search.
Reference: `module/shop/common_shop.py`, `module/shop/tactical_challenge_shop.py`,
`module/shop/shop_utils.py`.
### Arena (PvP ladder) ### Phase 5: Mailbox migration
Fight until out of tickets or rank 1, claim season + daily rewards. Needs the
Tier-1-prerequisite auto-fight primitive, plus OCR for ticket count and
self/opponent level. High value (decays twice daily, at 06:00/20:00 reset) but
gated on OCR + auto-fight both being ready.
Reference: `module/arena.py` (173 lines).
--- Move mailbox logic from Bash to:
## Tier 4 — Heavy investment, lower ROI (defer) ```
ba_auto/tasks/mailbox.py
```
### Lesson / Schedule (affection farming via classes) The CLI should call it through Python.
Structurally the most complex non-crafting feature: fuzzy OCR region-name
matching, per-region student-portrait templates, multi-page swipe search, and a
bespoke isometric-grid pixel-sampling helper for locating a specific favorite
student. A stripped-down "just pick the highest-affection lesson in a fixed
region" version would be much cheaper than full favorite-student targeting —
consider that subset first if we ever pick this up.
Reference: `module/lesson.py` (615 lines).
### Crafting (自动制造) ### Phase 6: Cafe migration
The single most complex feature in the reference project (1000+ lines): 3-phase
material selection, priority-list + rarity-tier config, quantity-stepper OCR,
filter/sort UI manipulation. Large one-time build cost for recurring but
non-urgent value (crafting mats don't decay the way AP/tickets do).
Reference: `module/create.py` (1028 lines).
### Main story / Normal-Hard story "push" (auto-clear next uncleared stage) Move cafe logic from Bash to:
Distinct from the Tier-1/2 "sweep already-cleared stages" — this is unlocking
new content. baas ships ~920KB of hand-authored per-stage click/formation-swap
JSON scripts for "grid mode"; "simple mode" (plain auto-fight, no positioning)
is far cheaper but doesn't work on every stage. Only worth building incrementally,
per-stage, on demand — not a batch feature.
Reference: `module/main_story.py`, `module/explore_tasks/explore_task.py`.
### Group Story / Mini Story (side-plot cleanup) ```
OCR-detect a "NEW" badge across a paged grid, clear each unread plot. Pure ba_auto/tasks/cafe.py
convenience, optional. ```
Reference: `module/group_story.py`, `module/mini_story.py`.
### Event content (Activities: story/mission/challenge push) During migration, verify:
Shape mirrors Tier-2 sweep work, but the content is 100% ephemeral — a new
JSON/script per event, expires when the event ends. Recommendation: only build
the generic "event AP sweep" (reuses Tier-1/2 sweep logic almost as-is) as a
standing capability; don't try to pre-build specific named events. Note also
that per baas's own docs, current JP/Global events have mostly moved away from
grid-walking challenges, so "challenge push" content increasingly needs manual
play anyway — low automation ROI there specifically.
Reference: `module/activities/activity_utils.py`, `module/sweep_activity.py`.
--- - both rooms still work
- sparkle detection still works
- cafe income claim still works
- rank-up popups are handled or explicitly documented as not handled
- student rotation popups are handled or explicitly documented as not handled
- view centering/zoom state is robust
- repeated detection does not suffer from Python cold-start delay
## Skip list (not relevant to this project) ### Phase 7: setup.sh update
Update `setup.sh` so it deploys:
```
ba_dailies.sh
ba_daily.py
ba_auto/
assets/
```
to the expected runtime paths on `nik-gpu`.
## Prerequisites
### OCR
Not set up yet.
Needed for:
- currency readouts
- ticket counts
- region/tab name matching
- some shop logic
- some lesson/schedule logic
- arena ticket/rank/level checks
- bounty coin balance if auto-refresh is implemented
Candidates:
- Tesseract
- PaddleOCR
Do not add OCR until a feature needs it.
### Auto-fight primitive
Needed for:
- Arena
- future main story push
- some battle automation
Reference:
```
~/repo/baas-reference/module/main_story.py
```
Look for:
```
auto_fight
enter_battle
```
Target local module may be:
```
ba_auto/tasks/battle.py
```
or:
```
ba_auto/battle.py
```
This should become a reusable primitive, not arena-specific code.
## High priority backlog
### 1. Migration to Python-first
This is the highest priority.
Do this before adding new features.
Goal:
- Bash launcher only
- Python CLI
- Python driver
- Python detector
- mailbox migrated
- cafe migrated
- reference mapping started
Reference:
- Current local implementation
- Existing `ba_dailies.sh`
- Existing `scripts/detect_and_click.py`
### 2. Stamina/AP sweep
Claim:
- daily free AP purchase
- daily task-menu AP/pyroxene rewards
Likely mostly color/state detection and fixed clicks.
Reference:
```
~/repo/baas-reference/module/collect_daily_free_power.py
~/repo/baas-reference/module/collect_daily_task_power.py
```
Local target: `ba_auto/tasks/stamina.py`
OCR: Probably not needed for first version.
### 3. Club/Group AP claim
Claim AP from club/group.
Reference:
```
~/repo/baas-reference/module/group.py
```
Local target: `ba_auto/tasks/group.py`
OCR: Not expected.
### 4. Normal/Hard story AP sweep
Sweep already-cleared main story stages to burn AP.
Reference:
```
~/repo/baas-reference/module/explore_tasks/sweep_task.py
~/repo/baas-reference/module/explore_tasks/task_utils.py
```
Local target: `ba_auto/tasks/story_sweep.py`
OCR: Likely needed for current region/stage detection unless using fixed configured targets.
Suggested first version:
- user-configured fixed stage
- no region search
- no dynamic OCR
- sweep configured mission only
Later version:
- fuzzy stage/region selection
- OCR-assisted navigation
### 5. Bounty
Three sub-areas and sweep availability.
Reference:
```
~/repo/baas-reference/module/rewarded_task.py
```
Local target: `ba_auto/tasks/bounty.py`
OCR: Optional for coin balance/refresh logic. Can skip refresh for first version.
### 6. Commissions
Two sub-dungeons:
- Base Defense
- Item Retrieval
Reference:
```
~/repo/baas-reference/module/clear_special_task_power.py
```
Local target: `ba_auto/tasks/commission.py`
OCR: Probably avoidable for first version if configured fixed sweep target is used.
### 7. Arena
Fight until out of tickets or until configured stopping condition.
Reference:
```
~/repo/baas-reference/module/arena.py
```
Local target: `ba_auto/tasks/arena.py`
Needs:
- auto-fight primitive
- OCR or visual detection for ticket/rank state
- careful safety limits
Arena is high-value but more risky than fixed claim tasks.
### 8. Common Shop + Tactical Shop
Auto-buy configured items.
Reference:
```
~/repo/baas-reference/module/shop/common_shop.py
~/repo/baas-reference/module/shop/tactical_challenge_shop.py
~/repo/baas-reference/module/shop/shop_utils.py
```
Local targets:
```
ba_auto/tasks/shop_common.py
ba_auto/tasks/shop_tactical.py
```
Needs:
- OCR for currency balances
- shop tab detection
- configured buy list
- safe purchase confirmation logic
Start with a no-refresh, fixed configured buy list.
### 9. Lesson / Schedule
Affection farming via classes.
Reference:
```
~/repo/baas-reference/module/lesson.py
```
Local target: `ba_auto/tasks/lesson.py`
Complexity: High
Needs:
- OCR for region/area names
- multi-page swipe search
- student detection/portrait matching if targeting specific students
- isometric grid location logic if following reference fully
Suggested first version:
- pick a fixed region
- select available/highest visible lesson
- avoid favorite-student targeting at first
## Low priority backlog
### Scrimmage
Reference:
```
~/repo/baas-reference/module/scrimmage.py
```
Local target: `ba_auto/tasks/scrimmage.py`
Similar shape to Bounty/Commissions.
### Crafting
Reference:
```
~/repo/baas-reference/module/create.py
```
Local target: `ba_auto/tasks/crafting.py`
Very complex. Contains:
- material selection
- priority lists
- rarity tiers
- stepper/quantity UI
- filtering/sorting
- OCR-like decision points
Do not start until the framework and OCR are mature.
### Battle Pass claim
Reference:
```
~/repo/baas-reference/module/collect_pass_reward.py
```
Local target: `ba_auto/tasks/battle_pass.py`
Should be simpler than crafting.
OCR only needed for optional stats.
### Momo Talk
Reference:
```
~/repo/baas-reference/module/momo_talk.py
```
Local target: `ba_auto/tasks/momo_talk.py`
Potentially useful because it runs on a different cadence from daily reset.
Likely no OCR. Mostly state scanning and click flow.
### Main story push
This means clearing new uncleared stages, not sweeping already-cleared stages.
Reference:
```
~/repo/baas-reference/module/main_story.py
~/repo/baas-reference/module/explore_tasks/explore_task.py
```
Low priority because full grid-mode support requires lots of per-stage scripting.
A simple auto-fight-only mode can be added later.
### Group Story / Mini Story
Reference:
```
~/repo/baas-reference/module/group_story.py
~/repo/baas-reference/module/mini_story.py
```
Convenience only.
### Event content
Reference:
```
~/repo/baas-reference/module/activities/activity_utils.py
~/repo/baas-reference/module/sweep_activity.py
```
Do not prebuild specific event scripts.
Build only generic event AP sweep if it can reuse story sweep logic.
Event-specific content expires and is not worth hardcoding unless the user asks for a specific live event.
## Skip list
| Feature | Why skip | | Feature | Why skip |
|---|---| |---|---|
| Total Assault (raid) | Stubbed/dead code upstream too (`return True` before any real logic) and disabled by default; low value to chase. | | Total Assault / Raid | Low value and risky to automate. Reference support may be limited/stubbed. |
| Joint Firing Drill (综合战术测试) | Hard-disabled for JP server in baas's own code (`if self.server == "JP": return True`) despite being in their README — it's a no-op on JP regardless. | | Joint Firing Drill | Not worth prioritizing for JP if reference has server-specific limitations. |
| De-clothes localization toggle | CN-only. | | De-clothes localization toggle | CN-only / irrelevant. |
| Restart / refresh-uiautomator2 | Android/ADB housekeeping for baas's mobile control backend; not applicable to our PC/Steam/Proton setup. | | Restart / refresh-uiautomator2 | Android/ADB backend maintenance, not applicable to PC/Steam/Proton. |
| Auto-unfriend | Disabled by default upstream too; low value, mildly risky (deletes friends). | | Auto-unfriend | Risky, low value, destructive. |
| Daily minigame dispatcher | Event-specific and changes every campaign — not a stable feature to pre-build. Handle ad hoc if/when a specific live minigame is worth automating. | | Daily minigame dispatcher | Event-specific and unstable. Handle ad hoc only. |
--- ## Automation cadence notes
## Notes on cadence Some tasks decay on different schedules.
Several features are time-decaying rather than "any time" farmable, which should | Feature | Suggested cadence |
inform how often the script runs (cron/scheduler), not just build order: |---|---|
- Cafe income + Momo Talk: effectively continuous, ~3h internal cadence. | Cafe income / affection | Every few hours |
- Arena: resets twice daily (06:00, 20:00 UTC). | Momo Talk | Every few hours |
- Everything else: once-daily, JP/Global reset at 20:00 UTC (03:00 UTC+8). | Arena | Around reset windows / twice daily if implemented |
| Mailbox | Daily or with default run |
| AP/stamina/task rewards | Daily |
| Group AP | Daily |
| Bounty/Commissions/Scrimmage | Daily |
| Shops | Daily, after reset |
| Lesson/Schedule | Daily |
Scheduling should be handled outside the feature logic.
Feature code should perform one safe run and exit.
## Safety and robustness rules
Every task should have:
- maximum retry count
- timeout where appropriate
- safe failure mode
- clear stdout logging
- no infinite click loops
- no unbounded spending
- config guard for purchases
- dry-run or debug mode when useful
For purchases:
- default to conservative behavior
- avoid refresh loops until OCR/currency detection is reliable
- require explicit configured item list
- avoid buying unknown items
For battle features:
- require clear stop conditions
- avoid continuing blindly after unexpected state
- prefer returning failure over clicking randomly
## Configuration direction
Future config may live in `ba_auto/config.py` or `config.yaml`.
Possible config values:
```
server = JP
game_window_name = BlueArchive
display = :0
asset_dir = ~/ba_assets
screenshot_dir = ./scratchpad
cafe_max_clicks_per_room
story_sweep_target
shop_buy_list
arena_stop_condition
ocr_enabled
debug_enabled
```
Keep config explicit. Do not bury user-specific settings deep inside task logic.
## Debugging conventions
Use `./scratchpad` for:
- temporary screenshots
- cropped templates
- annotated match images
- OCR debug output
- one-off notes
Do not use `/tmp` or `/private/tmp` unless unavoidable.
When detector behavior changes, save debug outputs with clear names, for example:
```
scratchpad/cafe_match_2026-07-05_001.png
scratchpad/shop_ocr_debug_001.png
```
## Local validation commands
On `nik-macbookair`:
```
bash -n ba_dailies.sh
python3 -m py_compile ba_daily.py
python3 -m py_compile ba_auto/*.py
python3 -m py_compile ba_auto/tasks/*.py
```
On `nik-gpu`:
```
~/ba_dailies.sh mailbox
~/ba_dailies.sh cafe
```
After migration:
```
~/ba_dailies.sh
```
should run the default daily sequence.
## Near-term recommended task order
1. Rewrite `ba_dailies.sh` as a thin launcher.
2. Add `ba_daily.py`.
3. Add `ba_auto/driver.py`.
4. Add `ba_auto/detector.py`.
5. Add `ba_auto/navigation.py`.
6. Move mailbox logic to `ba_auto/tasks/mailbox.py`.
7. Move cafe logic to `ba_auto/tasks/cafe.py`.
8. Update `setup.sh`.
9. Add `ba_auto/reference_notes/mapping.md`.
10. Verify existing mailbox and cafe still work.
11. Implement stamina/AP.
12. Implement group/club AP.
13. Implement fixed-target sweep features.
14. Add OCR only when needed.
15. Attempt Arena/Shop/Lesson after the framework is stable.
## Claude Code guidance summary
When Claude Code works on this repo, it should follow this rule:
> Reference first.
> Python first.
> Driver primitives before feature hacks.
> Bash launcher only.
Do not turn this project into a Bash recreation of Blue Archive Auto Script.

48
setup.sh Executable file
View File

@ -0,0 +1,48 @@
#!/bin/bash
# Bootstrap a fresh clone of this repo so ba_dailies.sh can actually run.
#
# Run this ON nik-gpu, from the repo root (e.g. after `git clone` there) --
# not from macOS, since it installs into nik-gpu-local paths.
#
# It does NOT touch the game or take any screenshots; it only installs the
# venv and copies files to the fixed paths ba_dailies.sh/detect_and_click.py
# expect (outside the repo, per the two-machine deploy convention in
# CLAUDE.md).
set -e
VENV_DIR="$HOME/.venvs/ba-auto-daily"
SCRIPTS_DIR="$HOME/ba_scripts"
ASSETS_DIR="$HOME/ba_assets"
echo "== Checking host tools =="
missing=0
for cmd in xdotool scrot; do
if ! command -v "$cmd" >/dev/null 2>&1; then
echo "MISSING: $cmd (install with your package manager, e.g. sudo apt install $cmd)"
missing=1
fi
done
if [ "$missing" = 1 ]; then
echo "Install the missing tool(s) above, then re-run this script."
exit 1
fi
echo "xdotool, scrot: OK"
echo "== Setting up Python venv at $VENV_DIR =="
if [ ! -x "$VENV_DIR/bin/python3" ]; then
python3 -m venv "$VENV_DIR"
fi
"$VENV_DIR/bin/pip" install --quiet --upgrade pip
"$VENV_DIR/bin/pip" install --quiet opencv-python-headless numpy
"$VENV_DIR/bin/python3" -c "import cv2, numpy; print('opencv', cv2.__version__, '/ numpy', numpy.__version__)"
echo "== Deploying scripts + assets to fixed paths =="
mkdir -p "$SCRIPTS_DIR" "$ASSETS_DIR"
cp scripts/detect_and_click.py "$SCRIPTS_DIR/detect_and_click.py"
cp assets/cafe_sparkle.png "$ASSETS_DIR/cafe_sparkle.png"
cp ba_dailies.sh "$HOME/ba_dailies.sh"
chmod +x "$HOME/ba_dailies.sh"
echo "== Done =="
echo "Run with: ~/ba_dailies.sh [mailbox|cafe]"
echo "(requires the game already running, window titled 'BlueArchive')"