diff --git a/CLAUDE.md b/CLAUDE.md index 40dacfb..fdd2473 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,59 +1,504 @@ # CLAUDE.md -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. +This file provides guidance to Claude Code when working with code in this repository. -## What this is +## What this project is -Automation for daily chores in the mobile game Blue Archive: claiming mailbox rewards and farming/collecting cafe affection income. It drives the actual game client via simulated mouse clicks and keypresses (`xdotool`) against fixed screen coordinates, plus one template-matching detector for a dynamic (moving) UI element. +This repository is a personal Blue Archive JP daily-automation project. + +It controls the real Blue Archive PC/Steam/Proton client running on a Linux machine named `nik-gpu`. The development machine is a MacBook named `nik-macbookair`. + +The automation backend is local desktop control: + +- `xdotool` for mouse/keyboard/window control +- `scrot` for screenshots +- Python/OpenCV for image matching and color/template detection +- OCR later, when needed +- no Android emulator control +- no ADB +- no uiautomator2 + +The reference project is located at: + +``` +~/repo/baas-reference/ +``` + +That repository contains the full Blue Archive Auto Script implementation. It should be treated as the behavioral reference for this project. + +## Core architecture decision + +This project must be Python-first. + +**Do not implement new Blue Archive automation logic in `ba_dailies.sh`.** + +`ba_dailies.sh` is only a thin runtime/deployment launcher for convenience on `nik-gpu`. + +All feature logic must live in Python. + +The intended structure is: + +``` +ba-auto-daily/ +├── ba_dailies.sh +├── ba_daily.py +├── ba_auto/ +│ ├── __init__.py +│ ├── driver.py +│ ├── detector.py +│ ├── navigation.py +│ ├── config.py +│ ├── tasks/ +│ │ ├── __init__.py +│ │ ├── mailbox.py +│ │ ├── cafe.py +│ │ ├── stamina.py +│ │ ├── group.py +│ │ └── ... +│ └── reference_notes/ +│ └── mapping.md +├── assets/ +├── screenshots/ +├── scripts/ +├── setup.sh +├── plan.md +└── CLAUDE.md +``` + +The exact layout can evolve, but the architectural rule should not change: + +- Bash launches Python. +- Python owns automation logic. +- The reference repository guides feature behavior. +- Local driver primitives adapt that behavior to the PC/Steam/Proton setup. + +## Reference-first rule + +Before implementing any new feature, inspect the matching reference implementation in: + +``` +~/repo/baas-reference/module/ +``` + +Do not start by inventing a Bash click sequence. + +For every feature, first identify: + +1. Which reference file implements it. +2. Which class/function contains the main control flow. +3. What the reference uses for state detection. +4. What the reference uses for retries/failure handling. +5. Which parts depend on Android/uiautomator2 and must be replaced. +6. Which parts can be ported directly as Python control flow. +7. Which local driver primitives are missing. + +Then implement the feature in Python under: + +``` +ba_auto/tasks/ +``` + +## Reference repository usage + +The reference repository is **read-only**. + +### Allowed + +- read reference modules +- inspect control flow +- reuse architecture ideas +- reuse retry/state-machine structure +- reuse task decomposition ideas +- reuse constants/config concepts when appropriate +- write local notes describing how a reference module maps to this project + +### Not allowed + +- edit files in `~/repo/baas-reference/` +- reimplement a reference feature from scratch in Bash +- create a local solution that ignores the reference flow when a reference implementation already exists ## Two-machine architecture -This repo is developed on macOS (`nik-macbookair`) but the code only runs on a separate Linux box, `nik-gpu`, where the Blue Archive client and its X display actually live (`DISPLAY=:0`, GDM `XAUTHORITY` under `/run/user/1000`). There is no local way to execute or test these scripts — they must be deployed to `nik-gpu` to run for real. +Development happens on: `nik-macbookair` -- `ba_dailies.sh` and `scripts/detect_and_click.py` are pushed to `nik-gpu` with `scp`/`rsync` (e.g. `scp ba_dailies.sh nik-gpu:~/ba_dailies.sh`, `scp scripts/detect_and_click.py nik-gpu:~/ba_scripts/detect_and_click.py`) and invoked there over `ssh`. -- `assets/cafe_sparkle.png` (the template image) is likewise copied to `nik-gpu:~/ba_assets/cafe_sparkle.png`. -- `detect_and_click.py` runs a Python venv on `nik-gpu` at `~/.venvs/ba-auto-daily/bin/python3` (needs `opencv-python`/`numpy`); `ba_dailies.sh` invokes it via that fixed path (`VENV_PYTHON` in the script). -- Screenshot -> detect -> click for the cafe sparkle happens entirely on `nik-gpu` in one local pipeline (see the docstring in `scripts/detect_and_click.py`) rather than round-tripping images over SSH, because the sparkle target moves fast enough that a multi-hop pipeline would miss the click. -- `screenshots/` holds reference captures (taken on `nik-gpu`, pulled back for inspection) used to work out coordinates and template thresholds when coordinates drift after a game UI update — they are not test fixtures consumed by any script. +Runtime happens on: `nik-gpu` -Because there's no local execution path, "testing a change" means syntax-checking locally and then deploying to `nik-gpu` and running it against the live game there: -```bash -bash -n ba_dailies.sh -python3 -m py_compile scripts/detect_and_click.py +The Blue Archive client and X display live on `nik-gpu`. + +Assume: (`DISPLAY=:0`, GDM `XAUTHORITY` under `/run/user/1000`) + +The game runs under Steam/Proton on the Linux desktop. + +There is no reliable local execution path on macOS. Anything that interacts with the game must be deployed to `nik-gpu`. + +## Deployment model + +During iteration, files are pushed from `nik-macbookair` to `nik-gpu`. + +Typical paths on `nik-gpu`: + +``` +~/ba_dailies.sh +~/ba_daily.py +~/ba_auto/ +~/ba_assets/ +~/.venvs/ba-auto-daily/ ``` -## Dependencies on `nik-gpu` +The current setup may still contain older paths such as: -These are host-level prerequisites, not managed by this repo — confirm they're present before assuming a failure is a code bug: -- `xdotool` (window focus, clicks, keypresses) -- `scrot` (screenshot capture for the detector) -- A venv at `~/.venvs/ba-auto-daily/` with `opencv-python` and `numpy` installed, Python binary at `~/.venvs/ba-auto-daily/bin/python3` +``` +~/ba_scripts/detect_and_click.py +``` + +When refactoring, prefer consolidating Python code into `ba_auto/`. + +`setup.sh` should bootstrap or update the runtime layout on `nik-gpu`. + +For a fresh checkout on `nik-gpu`, run from the repo root: + +``` +./setup.sh +``` + +After initial setup, individual changes may be pushed with `scp` or `rsync`. + +## Runtime dependencies on nik-gpu + +These are host-level dependencies. Confirm they exist before assuming a bug is in the project code. + +Required now: + +- `xdotool` +- `scrot` +- `python3` + +Required Python packages: + +- `opencv-python` or `opencv-python-headless` +- `numpy` + +Expected venv: + +``` +~/.venvs/ba-auto-daily/bin/python3 +``` Quick check: -```bash + +``` ssh nik-gpu "which xdotool scrot && ~/.venvs/ba-auto-daily/bin/python3 -c 'import cv2, numpy; print(cv2.__version__)'" ``` + +Future dependency: OCR engine, likely Tesseract or PaddleOCR. + +Do not introduce OCR casually. Add it only when implementing a feature that actually needs OCR. + +## Bash policy + +`ba_dailies.sh` should be a thin launcher only. + +Preferred shape: + +```bash +#!/usr/bin/env bash +set -euo pipefail +VENV_PYTHON="${VENV_PYTHON:-$HOME/.venvs/ba-auto-daily/bin/python3}" +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +exec "$VENV_PYTHON" "$SCRIPT_DIR/ba_daily.py" "$@" +``` + +Acceptable Bash responsibilities: + +- choose Python binary +- set environment variables +- call Python entry point +- provide compatibility with old command names +- fail early if the Python entry point is missing + +Not acceptable in Bash: + +- feature state machines +- long click sequences +- OpenCV logic +- OCR logic +- retry loops for game states +- feature-specific navigation +- shop/sweep/battle logic +- new `do_` game automation functions + +If an existing Bash function exists, migrate it to Python rather than extending it. + +## Python entry point + +The intended Python CLI entry point is: + +``` +ba_daily.py +``` + +It should support commands such as: + +``` +./ba_dailies.sh +./ba_dailies.sh mailbox +./ba_dailies.sh cafe +./ba_dailies.sh stamina +./ba_dailies.sh group +``` + +No argument should run the default daily flow. + +Example default flow: + +1. focus game +2. mailbox +3. cafe +4. future daily tasks + +The CLI should dispatch into task modules under `ba_auto/tasks/`. + +## Driver layer + +Create and maintain a driver layer in: + +``` +ba_auto/driver.py +``` + +The driver layer should wrap the PC/Steam/Proton backend. + +It should provide reusable primitives such as: + +``` +focus_game() +screenshot() +click(x, y) +double_click(x, y) +drag/swipe(start, end, duration) +keypress(key) +sleep/wait +wait_until(...) +color_at(...) +region_average_color(...) +template_match(...) +find_and_click_template(...) +``` + +Feature modules should not directly shell out to `xdotool` or `scrot` unless a driver primitive is missing and being added. + +Prefer: + +```python +driver.click(x, y) +``` + +over: + +```python +subprocess.run(["xdotool", "click", ...]) +``` + +This keeps the project close to the reference architecture while replacing only the backend-control layer. + +## Detector layer + +Image matching and color matching should live in `ba_auto/detector.py` or in clearly named helper classes/functions. + +Existing logic from `scripts/detect_and_click.py` should be migrated into reusable Python functions. + +The detector should support: + +- screenshot input +- template matching +- threshold tuning +- masked matching +- click-offset handling +- debug image output to `./scratchpad` + +Avoid one Python cold start per click attempt where possible. Prefer long-running Python task logic that can take repeated screenshots and click repeatedly from one process. + +## Navigation layer + +Common navigation should live in `ba_auto/navigation.py`. + +Use this for shared flows such as: + +- returning home +- opening main menu +- opening mailbox +- opening cafe +- opening shop +- opening lesson/schedule +- closing popups +- generic back/escape handling + +Do not duplicate navigation click sequences inside every task if they can be shared. + +## Task modules + +Each feature should have a task module: `ba_auto/tasks/.py` + +Example: + +``` +ba_auto/tasks/mailbox.py +ba_auto/tasks/cafe.py +ba_auto/tasks/group.py +ba_auto/tasks/stamina.py +``` + +Each task module should expose a clear function such as: + +``` +run(driver, config) +``` + +or: + +``` +run_mailbox(driver, config) +``` + +Keep task files feature-focused. + +## Reference mapping notes + +Maintain a mapping file at: + +``` +ba_auto/reference_notes/mapping.md +``` + +Before or during implementation of a feature, update the mapping. + +Use this format: + +| Local feature | Reference file | Reference functions/classes | Local file | Backend replacements | Status | +|---|---|---|---|---|---| +| Cafe | `module/cafe.py` or relevant file | `...` | `ba_auto/tasks/cafe.py` | uiautomator2 tap -> xdotool click, screenshot -> scrot/OpenCV | In progress | + ## Working conventions -Use `./scratchpad` (create if missing) in the project root for temporary/intermediate files — e.g. cropped calibration images from `screenshots/cafe/sparkle/`, one-off debug output. Never write to `/tmp` or `/private/tmp`. +Use `./scratchpad` for temporary or intermediate files. -## `ba_dailies.sh` +Examples: -Entry point, run on `nik-gpu` as `./ba_dailies.sh [mailbox|cafe]`: -- No argument: focuses the game window, runs mailbox claim, then the full cafe routine. -- `mailbox`: focus + claim mailbox only. -- `cafe`: focus + cafe routine only (both rooms + income claim). +- cropped calibration images +- debug screenshots +- annotated match results +- temporary investigation notes -All interaction points (icon/button coordinates, max click-attempts per cafe room) are top-of-file constants — when the in-game UI shifts or the window resolution changes, update the coordinates there rather than inline in the functions. `focus_game` finds the window via `xdotool search --name "BlueArchive"` and hard-fails if the game isn't running. +Do not use `/tmp` or `/private/tmp` unless there is a strong reason. -The cafe routine (`do_cafe`) alternates between two cafe rooms; for each room it repeatedly calls into `detect_and_click.py` to find and click affection sparkles (up to `CAFE_MAX_CLICKS_PER_ROOM` times) before moving on, then claims cafe income at the end. +`screenshots/` contains human reference captures. They are useful for calibration and documentation, but they are not necessarily automated test fixtures. -**Known gap — needs verification:** the manual cafe flow this automates includes a few conditional steps not obviously covered above: closing a popup that only appears if a student is "rotated," zooming out/centering the view before detection starts, and closing a rank-up popup that can appear after a successful click. Confirm these are actually handled somewhere in `do_cafe` (or decide they're unnecessary in practice) rather than assuming coverage from this description alone. +`assets/` contains local template images. These should be captured from the local game setup where possible. -## `scripts/detect_and_click.py` +## Testing and checks -Standalone script (runs on `nik-gpu`, called once per sparkle-click attempt from the shell loop): screenshots the game window with `scrot`, template-matches `cafe_sparkle.png` using a color-masked `cv2.matchTemplate` (masks to yellow/white sparkle pixels so it doesn't match on background art), clicks the best match (offset-corrected — the template's anchor point isn't the click point), and reports its result on stdout/exit code (`MATCH x y score` / exit 0, or `NO_MATCH` / exit 1) so the caller shell loop can decide whether to keep clicking. +Because the game only runs on `nik-gpu`, local macOS testing is limited. -`THRESHOLD = 0.97` and `OFFSET_X`/`OFFSET_Y` are the values most likely to need retuning if detection starts missing or mis-clicking — use the `screenshots/cafe/sparkle/` reference captures to recalibrate. When cropping or annotating these captures for calibration, use `./scratchpad` in the project root for the intermediate files, not `/tmp`. +Before deploying, run static/syntax checks locally: -**Performance note — needs verification:** this script is invoked as a fresh process per click attempt, and Python + OpenCV cold-start has real overhead (commonly 200–500ms). If the sparkle target moves fast, confirm this hasn't caused missed detections in practice against the live game. If it has, consider a persistent process the shell loop talks to (pipe/socket) instead of a per-attempt cold start. \ No newline at end of file +``` +bash -n ba_dailies.sh +python3 -m py_compile ba_daily.py +python3 -m py_compile ba_auto/*.py +python3 -m py_compile ba_auto/tasks/*.py +``` + +On `nik-gpu`, run real integration tests against the live game. + +Example: + +``` +ssh nik-gpu "~/ba_dailies.sh cafe" +``` + +When debugging image matching, write debug images to `./scratchpad`. + +## Existing features + +Current project state before Python-first refactor: + +- mailbox claim exists +- cafe affection/income exists +- cafe uses OpenCV template matching for sparkle detection +- existing Python helper is `scripts/detect_and_click.py` +- existing Bash entry point is `ba_dailies.sh` + +Migration goal: + +- preserve existing behavior +- move feature logic from Bash into Python +- keep `ba_dailies.sh` as launcher only +- make detection logic reusable +- prepare for future features by copying the reference project's structure where appropriate + +## Known cafe gaps to verify during migration + +When migrating cafe logic, verify these manually against the actual current behavior: + +- whether student rotation popups are handled +- whether rank-up popups are handled +- whether the view is zoomed/centered before sparkle detection +- whether detection still works when text overlaps the student +- whether one Python process can handle repeated sparkle detection faster than repeated cold starts +- whether both cafe rooms are handled consistently +- whether cafe income claim is robust against popup timing + +Do not assume the old Bash implementation handles these correctly. + +## Anti-patterns + +Do not: + +- add new `do_` Bash functions +- build a giant Bash automation script +- recreate reference logic as fixed coordinate Bash click chains +- skip reading the reference module before implementing a feature +- put OCR in Bash +- put OpenCV state machines in Bash +- make every feature shell out independently to `xdotool` +- duplicate common navigation in every task +- copy reference image assets blindly +- edit `~/repo/baas-reference/` +- implement event-specific features before the generic reusable machinery exists + +## Feature implementation workflow + +For every new feature: + +1. Read the relevant `~/repo/baas-reference/module/...` file. +2. Summarize the upstream feature flow in notes or comments. +3. Add/update the reference mapping table. +4. Identify missing local driver primitives. +5. Implement or improve those primitives in `ba_auto/driver.py` or `ba_auto/detector.py`. +6. Implement the feature in `ba_auto/tasks/.py`. +7. Add CLI dispatch in `ba_daily.py`. +8. Keep `ba_dailies.sh` unchanged unless launcher behavior changes. +9. Run syntax checks. +10. Deploy to `nik-gpu`. +11. Test against the live game. +12. Update `plan.md` status. + +## Priority when uncertain + +When uncertain, prefer this order: + +1. Preserve existing working behavior. +2. Follow the reference project's control flow. +3. Keep logic in Python. +4. Add reusable driver primitives instead of feature-specific hacks. +5. Use local screenshots/assets only when backend differences require it. +6. Avoid large rewrites that do not move the project closer to reference-driven Python architecture. + +## User preference + +The user wants this project to be as close to the original Blue Archive Auto Script architecture as practical, without recreating logic that already exists. + +The user specifically does not want Claude Code to keep converting feature work into Bash. + +Respect that preference. diff --git a/plan.md b/plan.md index 249a539..bcfa345 100644 --- a/plan.md +++ b/plan.md @@ -1,170 +1,786 @@ # ba-auto-daily implementation plan -Personal Blue Archive **JP** daily-automation script. Controls the game via `xdotool` -over a remote X11 session (Steam+Proton on `nik-gpu`), using OpenCV template/color -matching for detection. No OCR is set up yet (see Prerequisites). +Personal Blue Archive JP daily-automation project. -Feature list and priority ordering below is derived from studying -`~/repo/baas-reference` (pur1fying/blue_archive_auto_script, GPL-3.0, read-only -reference — **not copied from**, reimplemented fresh in our own style). Its own -default scheduler priority, complexity, and JP-relevance were used to rank the -backlog. See "Reference" under each feature for the baas file(s) that describe the -mechanic, for research purposes only. +This project controls the PC/Steam/Proton Blue Archive client running on `nik-gpu` through local desktop automation: + +- xdotool +- scrot +- Python +- OpenCV +- later OCR when needed + +Development happens on `nik-macbookair`. + +The reference implementation lives at: + +``` +~/repo/baas-reference/ +``` + +The reference project should be treated as the behavioral blueprint. This project should avoid recreating feature logic from scratch when the reference already implements it. + +## Core project direction + +This project is now Python-first. + +The goal is not to grow a large Bash script. + +The goal is to build a small local Python automation framework that adapts the reference project's Blue Archive logic to this user's unique PC/Steam/Proton environment. + +`ba_dailies.sh` should only be a launcher. + +Feature logic should live in Python. + +## Target architecture + +``` +~/repo/ba-auto-daily/ +├── ba_dailies.sh +├── ba_daily.py +├── ba_auto/ +│ ├── __init__.py +│ ├── config.py +│ ├── driver.py +│ ├── detector.py +│ ├── navigation.py +│ ├── tasks/ +│ │ ├── __init__.py +│ │ ├── mailbox.py +│ │ ├── cafe.py +│ │ ├── stamina.py +│ │ ├── group.py +│ │ ├── bounty.py +│ │ ├── commission.py +│ │ ├── arena.py +│ │ ├── shop_common.py +│ │ ├── shop_tactical.py +│ │ ├── lesson.py +│ │ └── ... +│ └── reference_notes/ +│ └── mapping.md +├── assets/ +├── screenshots/ +├── scripts/ +├── setup.sh +├── CLAUDE.md +└── plan.md +``` + +This is the intended direction. It does not have to be completed all at once. + +## Project layout + +| Path | What it is | +|---|---| +| `~/repo/ba-auto-daily/ba_dailies.sh` | Thin launcher only. It should call the Python entry point. Do not add new feature logic here. | +| `~/repo/ba-auto-daily/ba_daily.py` | Main Python CLI entry point. Dispatches tasks such as mailbox, cafe, stamina, group, etc. | +| `~/repo/ba-auto-daily/ba_auto/driver.py` | Local PC/Steam/Proton control backend. Wraps xdotool, scrot, waits, clicks, swipes, keypresses, screenshots, and window focus. | +| `~/repo/ba-auto-daily/ba_auto/detector.py` | OpenCV/template/color matching helpers. Existing `scripts/detect_and_click.py` logic should be migrated here. | +| `~/repo/ba-auto-daily/ba_auto/navigation.py` | Shared navigation helpers: home, menu, close popup, back, open feature screens. | +| `~/repo/ba-auto-daily/ba_auto/tasks/` | Feature implementations. Each task should adapt the relevant `baas-reference/module/...` logic where possible. | +| `~/repo/ba-auto-daily/ba_auto/reference_notes/mapping.md` | Reference mapping table: local feature → reference module → local implementation → driver gaps. | +| `~/repo/ba-auto-daily/assets/` | Locally captured template images, such as cafe sparkle. Do not blindly copy assets from the reference repo. | +| `~/repo/ba-auto-daily/screenshots/` | Human reference screenshots, mostly Moonlight/game captures, used for calibration and debugging. | +| `~/repo/ba-auto-daily/scripts/` | Transitional scripts. Long-term reusable Python logic should move into `ba_auto/`. | +| `~/repo/ba-auto-daily/setup.sh` | Bootstrap/deploy helper for `nik-gpu`. Should install/check dependencies and copy runtime files. | +| `~/repo/baas-reference/` | Read-only GPL-3.0 reference clone. Study and adapt. Never edit. | + +## Runtime paths on nik-gpu + +Preferred runtime layout: + +| Path | What it is | +|---|---| +| `nik-gpu:~/ba_dailies.sh` | Thin launcher. | +| `nik-gpu:~/ba_daily.py` | Python CLI entry point. | +| `nik-gpu:~/ba_auto/` | Python package copied from this repo. | +| `nik-gpu:~/ba_assets/` | Runtime assets/templates. | +| `nik-gpu:~/.venvs/ba-auto-daily/` | Python virtual environment. | + +Current older layout may include: + +| Path | What it is | +|---|---| +| `nik-gpu:~/ba_scripts/detect_and_click.py` | Old standalone detector helper. Should eventually be replaced by `ba_auto/detector.py`. | + +## Implementation strategy + +For each feature: + +1. Read the matching `~/repo/baas-reference/module/...` file. +2. Summarize the reference flow. +3. Identify reusable logic: + - state checks + - retry loops + - navigation sequence + - battle/sweep/shop rules + - detection method + - failure handling +4. Identify backend-specific calls that cannot be reused directly. +5. Implement missing generic primitives in `ba_auto/driver.py` or `ba_auto/detector.py`. +6. Implement the feature in `ba_auto/tasks/.py`. +7. Add CLI command dispatch in `ba_daily.py`. +8. Keep `ba_dailies.sh` unchanged unless launcher behavior changes. +9. Test syntax locally. +10. Deploy to `nik-gpu`. +11. Run against the live game. +12. Update this plan. + +The intended result is not a Bash automation script. + +The intended result is a Python automation framework using the reference repository as the behavioral blueprint. + +## Reference mapping table + +Maintain this table in `ba_auto/reference_notes/mapping.md`. + +Initial seed: + +| Local feature | Reference file | Reference functions/classes | Local file | Backend replacements | Status | +|---|---|---|---|---|---| +| Mailbox | Need to confirm in reference | Need to inspect | `ba_auto/tasks/mailbox.py` | tap/click via xdotool, screenshot via scrot | Existing Bash behavior; migrate to Python | +| Cafe | Need to confirm in reference | Need to inspect | `ba_auto/tasks/cafe.py` | template matching via OpenCV, click via xdotool | Existing Bash + helper behavior; migrate to Python | +| Stamina/AP | `module/collect_daily_free_power.py`, `module/collect_daily_task_power.py` | Need to inspect | `ba_auto/tasks/stamina.py` | color checks/clicks via local driver | Not started | +| Group/Club AP | `module/group.py` | Need to inspect | `ba_auto/tasks/group.py` | fixed click + state check via local driver | Not started | +| Bounty | `module/rewarded_task.py` | Need to inspect | `ba_auto/tasks/bounty.py` | sweep/color/OCR adaptation | Not started | +| Commissions | `module/clear_special_task_power.py` | Need to inspect | `ba_auto/tasks/commission.py` | sweep/color adaptation | Not started | +| Arena | `module/arena.py` | Need to inspect | `ba_auto/tasks/arena.py` | auto-fight + OCR + local driver | Not started | +| Common Shop | `module/shop/common_shop.py`, `module/shop/shop_utils.py` | Need to inspect | `ba_auto/tasks/shop_common.py` | OCR + tab navigation + local clicks | Not started | +| Tactical Shop | `module/shop/tactical_challenge_shop.py`, `module/shop/shop_utils.py` | Need to inspect | `ba_auto/tasks/shop_tactical.py` | OCR + tab navigation + local clicks | Not started | +| Lesson/Schedule | `module/lesson.py` | Need to inspect | `ba_auto/tasks/lesson.py` | OCR + template/portrait search + local driver | Not started | + +Do not implement a feature without filling at least the relevant row. ## Status snapshot -| Feature | Status | -|---|---| -| Mailbox claim | ✅ Done (`do_mailbox`) | -| Cafe (pats + income) | ✅ Done (`do_cafe`) | -| Everything else below | Not started | +| Feature | Current status | Target status | +|---|---|---| +| Mailbox claim | Done in old Bash style | Migrate to `ba_auto/tasks/mailbox.py` | +| Cafe pats + income | Done in old Bash + standalone Python detector style | Migrate to `ba_auto/tasks/cafe.py` and `ba_auto/detector.py` | +| Shared driver | Partial/implicit in scripts | Build `ba_auto/driver.py` | +| Python CLI | Missing | Build `ba_daily.py` | +| Reference mapping | Missing | Build `ba_auto/reference_notes/mapping.md` | +| Everything else | Not started | Implement reference-first in Python | -## Prerequisites (cross-cutting, unblock multiple tiers) +## Migration phase -- **OCR.** Not set up. Needed for currency/ticket-count readouts, region/tab name - search, and a few other read-a-number moments. When we get there, evaluate - Tesseract vs PaddleOCR in the existing `~/.venvs/ba-auto-daily` venv. Until then, - stick to features that are pure color-check / template-match. -- **Auto-fight primitive.** A shared "start fight → max speed + auto-mode → - detect win/lose" routine that Arena, and any future story/raid automation, all - need. Build once, early, before Tier 3 combat features. - Reference: `module/main_story.py` (`auto_fight`/`enter_battle`). +Before adding new game features, migrate the existing working implementation. ---- +### Phase 1: Python skeleton -## Tier 1 — High priority, no OCR needed +Create: -### Stamina/AP sweep (free + task-menu power) -Claim the daily free 10 AP purchase and the daily task-menu AP/pyroxene rewards. -Pure color-check state detection, fixed clicks. High value: AP is capped and this -is free currency/materials left on the table every day. -Reference: `module/collect_daily_free_power.py`, `module/collect_daily_task_power.py`. +``` +ba_daily.py +ba_auto/__init__.py +ba_auto/config.py +ba_auto/driver.py +ba_auto/detector.py +ba_auto/navigation.py +ba_auto/tasks/__init__.py +ba_auto/tasks/mailbox.py +ba_auto/tasks/cafe.py +ba_auto/reference_notes/mapping.md +``` -### Club/Group AP claim -Claim 10 AP from the club/dorm menu. Trivial fixed click + color check. -Reference: `module/group.py` (29 lines). +### Phase 2: Launcher -### Momo Talk (chat app conversations) -Auto-complete unread chat conversations, including relationship-rank-up story -beats, claim pyroxene. No OCR — pure pixel-column scanning state machine -(reply bubble / "enter story" prompt detection by color). Runs on its own ~3h -cadence like cafe. Moderate build effort but no new dependencies. -Reference: `module/momo_talk.py` (183 lines, `getConversationState`). +Change `ba_dailies.sh` into a thin launcher: ---- +```bash +#!/usr/bin/env bash +set -euo pipefail +VENV_PYTHON="${VENV_PYTHON:-$HOME/.venvs/ba-auto-daily/bin/python3}" +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +exec "$VENV_PYTHON" "$SCRIPT_DIR/ba_daily.py" "$@" +``` -## Tier 2 — Medium priority, sweep-style AP consumption +Keep compatibility with: -These three share the same shape: buy tickets (optional), sweep N times per -sub-area, detected mostly by color (a multi-pixel "sss/star-rank" signature, not -OCR). Minor OCR only for optional coin-balance readouts — can hardcode/skip that -part initially and just sweep until out of stamina/tickets. +``` +./ba_dailies.sh +./ba_dailies.sh mailbox +./ba_dailies.sh cafe +``` -### Commissions ("Special Task" / 委託) -Two sub-dungeons (Base Defense = EXP, Item Retrieval = credits). -Reference: `module/clear_special_task_power.py` (148 lines). +### Phase 3: Driver extraction -### Scrimmage (学院交流会) -Three academies, ticket purchase available on JP. -Reference: `module/scrimmage.py` (166 lines). +Move shell interactions into `ba_auto/driver.py`. -### Bounty ("Rewarded Task" / 悬赏通缉) -Three sub-areas, ticket purchase (0–12). Needs a little OCR for bounty-coin -balance if we want auto-refresh; otherwise skippable. -Reference: `module/rewarded_task.py` (204 lines). +Driver primitives should include: -### Normal/Hard story AP sweep (re-clear already-cleared stages) -Sweep already-cleared main-story stages by region/mission count to burn AP. -Needs OCR for current-region-number + a fuzzy swipe-search over mission -buttons — more OCR-dependent than the three above, so ranked after them. -Reference: `module/explore_tasks/sweep_task.py`, `module/explore_tasks/task_utils.py`. +``` +focus_game() +click(x, y) +double_click(x, y) +keypress(key) +screenshot(path=None) +swipe(...) +wait(seconds) +wait_until(...) +``` ---- +### Phase 4: Detector extraction -## Tier 3 — Needs real OCR investment +Move `scripts/detect_and_click.py` logic into `ba_auto/detector.py`. -Do these once OCR is actually set up. +Detector primitives should include: -### Battle Pass claim (JP-only feature upstream — relevant to us) -Claim pass mission points + tier rewards. Simple template claim-flow; OCR only -needed for the optional level/points stat readout (can stub that part). -Reference: `module/collect_pass_reward.py` (146 lines). +``` +load_template(...) +match_template(...) +find_best_match(...) +find_and_click_template(...) +color_mask(...) +debug_write_match(...) +``` -### Common Shop + Tactical (Arena) Shop -Auto-buy configured items, auto-refresh when affordable. OCR needed for -currency balances and shop-tab name search. -Reference: `module/shop/common_shop.py`, `module/shop/tactical_challenge_shop.py`, -`module/shop/shop_utils.py`. +The old script may remain as a compatibility wrapper temporarily, but the reusable logic should live under `ba_auto/`. -### Arena (PvP ladder) -Fight until out of tickets or rank 1, claim season + daily rewards. Needs the -Tier-1-prerequisite auto-fight primitive, plus OCR for ticket count and -self/opponent level. High value (decays twice daily, at 06:00/20:00 reset) but -gated on OCR + auto-fight both being ready. -Reference: `module/arena.py` (173 lines). +### Phase 5: Mailbox migration ---- +Move mailbox logic from Bash to: -## Tier 4 — Heavy investment, lower ROI (defer) +``` +ba_auto/tasks/mailbox.py +``` -### Lesson / Schedule (affection farming via classes) -Structurally the most complex non-crafting feature: fuzzy OCR region-name -matching, per-region student-portrait templates, multi-page swipe search, and a -bespoke isometric-grid pixel-sampling helper for locating a specific favorite -student. A stripped-down "just pick the highest-affection lesson in a fixed -region" version would be much cheaper than full favorite-student targeting — -consider that subset first if we ever pick this up. -Reference: `module/lesson.py` (615 lines). +The CLI should call it through Python. -### Crafting (自动制造) -The single most complex feature in the reference project (1000+ lines): 3-phase -material selection, priority-list + rarity-tier config, quantity-stepper OCR, -filter/sort UI manipulation. Large one-time build cost for recurring but -non-urgent value (crafting mats don't decay the way AP/tickets do). -Reference: `module/create.py` (1028 lines). +### Phase 6: Cafe migration -### Main story / Normal-Hard story "push" (auto-clear next uncleared stage) -Distinct from the Tier-1/2 "sweep already-cleared stages" — this is unlocking -new content. baas ships ~920KB of hand-authored per-stage click/formation-swap -JSON scripts for "grid mode"; "simple mode" (plain auto-fight, no positioning) -is far cheaper but doesn't work on every stage. Only worth building incrementally, -per-stage, on demand — not a batch feature. -Reference: `module/main_story.py`, `module/explore_tasks/explore_task.py`. +Move cafe logic from Bash to: -### Group Story / Mini Story (side-plot cleanup) -OCR-detect a "NEW" badge across a paged grid, clear each unread plot. Pure -convenience, optional. -Reference: `module/group_story.py`, `module/mini_story.py`. +``` +ba_auto/tasks/cafe.py +``` -### Event content (Activities: story/mission/challenge push) -Shape mirrors Tier-2 sweep work, but the content is 100% ephemeral — a new -JSON/script per event, expires when the event ends. Recommendation: only build -the generic "event AP sweep" (reuses Tier-1/2 sweep logic almost as-is) as a -standing capability; don't try to pre-build specific named events. Note also -that per baas's own docs, current JP/Global events have mostly moved away from -grid-walking challenges, so "challenge push" content increasingly needs manual -play anyway — low automation ROI there specifically. -Reference: `module/activities/activity_utils.py`, `module/sweep_activity.py`. +During migration, verify: ---- +- both rooms still work +- sparkle detection still works +- cafe income claim still works +- rank-up popups are handled or explicitly documented as not handled +- student rotation popups are handled or explicitly documented as not handled +- view centering/zoom state is robust +- repeated detection does not suffer from Python cold-start delay -## Skip list (not relevant to this project) +### Phase 7: setup.sh update + +Update `setup.sh` so it deploys: + +``` +ba_dailies.sh +ba_daily.py +ba_auto/ +assets/ +``` + +to the expected runtime paths on `nik-gpu`. + +## Prerequisites + +### OCR + +Not set up yet. + +Needed for: + +- currency readouts +- ticket counts +- region/tab name matching +- some shop logic +- some lesson/schedule logic +- arena ticket/rank/level checks +- bounty coin balance if auto-refresh is implemented + +Candidates: + +- Tesseract +- PaddleOCR + +Do not add OCR until a feature needs it. + +### Auto-fight primitive + +Needed for: + +- Arena +- future main story push +- some battle automation + +Reference: + +``` +~/repo/baas-reference/module/main_story.py +``` + +Look for: + +``` +auto_fight +enter_battle +``` + +Target local module may be: + +``` +ba_auto/tasks/battle.py +``` + +or: + +``` +ba_auto/battle.py +``` + +This should become a reusable primitive, not arena-specific code. + +## High priority backlog + +### 1. Migration to Python-first + +This is the highest priority. + +Do this before adding new features. + +Goal: + +- Bash launcher only +- Python CLI +- Python driver +- Python detector +- mailbox migrated +- cafe migrated +- reference mapping started + +Reference: + +- Current local implementation +- Existing `ba_dailies.sh` +- Existing `scripts/detect_and_click.py` + +### 2. Stamina/AP sweep + +Claim: + +- daily free AP purchase +- daily task-menu AP/pyroxene rewards + +Likely mostly color/state detection and fixed clicks. + +Reference: + +``` +~/repo/baas-reference/module/collect_daily_free_power.py +~/repo/baas-reference/module/collect_daily_task_power.py +``` + +Local target: `ba_auto/tasks/stamina.py` + +OCR: Probably not needed for first version. + +### 3. Club/Group AP claim + +Claim AP from club/group. + +Reference: + +``` +~/repo/baas-reference/module/group.py +``` + +Local target: `ba_auto/tasks/group.py` + +OCR: Not expected. + +### 4. Normal/Hard story AP sweep + +Sweep already-cleared main story stages to burn AP. + +Reference: + +``` +~/repo/baas-reference/module/explore_tasks/sweep_task.py +~/repo/baas-reference/module/explore_tasks/task_utils.py +``` + +Local target: `ba_auto/tasks/story_sweep.py` + +OCR: Likely needed for current region/stage detection unless using fixed configured targets. + +Suggested first version: + +- user-configured fixed stage +- no region search +- no dynamic OCR +- sweep configured mission only + +Later version: + +- fuzzy stage/region selection +- OCR-assisted navigation + +### 5. Bounty + +Three sub-areas and sweep availability. + +Reference: + +``` +~/repo/baas-reference/module/rewarded_task.py +``` + +Local target: `ba_auto/tasks/bounty.py` + +OCR: Optional for coin balance/refresh logic. Can skip refresh for first version. + +### 6. Commissions + +Two sub-dungeons: + +- Base Defense +- Item Retrieval + +Reference: + +``` +~/repo/baas-reference/module/clear_special_task_power.py +``` + +Local target: `ba_auto/tasks/commission.py` + +OCR: Probably avoidable for first version if configured fixed sweep target is used. + +### 7. Arena + +Fight until out of tickets or until configured stopping condition. + +Reference: + +``` +~/repo/baas-reference/module/arena.py +``` + +Local target: `ba_auto/tasks/arena.py` + +Needs: + +- auto-fight primitive +- OCR or visual detection for ticket/rank state +- careful safety limits + +Arena is high-value but more risky than fixed claim tasks. + +### 8. Common Shop + Tactical Shop + +Auto-buy configured items. + +Reference: + +``` +~/repo/baas-reference/module/shop/common_shop.py +~/repo/baas-reference/module/shop/tactical_challenge_shop.py +~/repo/baas-reference/module/shop/shop_utils.py +``` + +Local targets: + +``` +ba_auto/tasks/shop_common.py +ba_auto/tasks/shop_tactical.py +``` + +Needs: + +- OCR for currency balances +- shop tab detection +- configured buy list +- safe purchase confirmation logic + +Start with a no-refresh, fixed configured buy list. + +### 9. Lesson / Schedule + +Affection farming via classes. + +Reference: + +``` +~/repo/baas-reference/module/lesson.py +``` + +Local target: `ba_auto/tasks/lesson.py` + +Complexity: High + +Needs: + +- OCR for region/area names +- multi-page swipe search +- student detection/portrait matching if targeting specific students +- isometric grid location logic if following reference fully + +Suggested first version: + +- pick a fixed region +- select available/highest visible lesson +- avoid favorite-student targeting at first + +## Low priority backlog + +### Scrimmage + +Reference: + +``` +~/repo/baas-reference/module/scrimmage.py +``` + +Local target: `ba_auto/tasks/scrimmage.py` + +Similar shape to Bounty/Commissions. + +### Crafting + +Reference: + +``` +~/repo/baas-reference/module/create.py +``` + +Local target: `ba_auto/tasks/crafting.py` + +Very complex. Contains: + +- material selection +- priority lists +- rarity tiers +- stepper/quantity UI +- filtering/sorting +- OCR-like decision points + +Do not start until the framework and OCR are mature. + +### Battle Pass claim + +Reference: + +``` +~/repo/baas-reference/module/collect_pass_reward.py +``` + +Local target: `ba_auto/tasks/battle_pass.py` + +Should be simpler than crafting. + +OCR only needed for optional stats. + +### Momo Talk + +Reference: + +``` +~/repo/baas-reference/module/momo_talk.py +``` + +Local target: `ba_auto/tasks/momo_talk.py` + +Potentially useful because it runs on a different cadence from daily reset. + +Likely no OCR. Mostly state scanning and click flow. + +### Main story push + +This means clearing new uncleared stages, not sweeping already-cleared stages. + +Reference: + +``` +~/repo/baas-reference/module/main_story.py +~/repo/baas-reference/module/explore_tasks/explore_task.py +``` + +Low priority because full grid-mode support requires lots of per-stage scripting. + +A simple auto-fight-only mode can be added later. + +### Group Story / Mini Story + +Reference: + +``` +~/repo/baas-reference/module/group_story.py +~/repo/baas-reference/module/mini_story.py +``` + +Convenience only. + +### Event content + +Reference: + +``` +~/repo/baas-reference/module/activities/activity_utils.py +~/repo/baas-reference/module/sweep_activity.py +``` + +Do not prebuild specific event scripts. + +Build only generic event AP sweep if it can reuse story sweep logic. + +Event-specific content expires and is not worth hardcoding unless the user asks for a specific live event. + +## Skip list | Feature | Why skip | |---|---| -| Total Assault (raid) | Stubbed/dead code upstream too (`return True` before any real logic) and disabled by default; low value to chase. | -| Joint Firing Drill (综合战术测试) | Hard-disabled for JP server in baas's own code (`if self.server == "JP": return True`) despite being in their README — it's a no-op on JP regardless. | -| De-clothes localization toggle | CN-only. | -| Restart / refresh-uiautomator2 | Android/ADB housekeeping for baas's mobile control backend; not applicable to our PC/Steam/Proton setup. | -| Auto-unfriend | Disabled by default upstream too; low value, mildly risky (deletes friends). | -| Daily minigame dispatcher | Event-specific and changes every campaign — not a stable feature to pre-build. Handle ad hoc if/when a specific live minigame is worth automating. | +| Total Assault / Raid | Low value and risky to automate. Reference support may be limited/stubbed. | +| Joint Firing Drill | Not worth prioritizing for JP if reference has server-specific limitations. | +| De-clothes localization toggle | CN-only / irrelevant. | +| Restart / refresh-uiautomator2 | Android/ADB backend maintenance, not applicable to PC/Steam/Proton. | +| Auto-unfriend | Risky, low value, destructive. | +| Daily minigame dispatcher | Event-specific and unstable. Handle ad hoc only. | ---- +## Automation cadence notes -## Notes on cadence +Some tasks decay on different schedules. -Several features are time-decaying rather than "any time" farmable, which should -inform how often the script runs (cron/scheduler), not just build order: -- Cafe income + Momo Talk: effectively continuous, ~3h internal cadence. -- Arena: resets twice daily (06:00, 20:00 UTC). -- Everything else: once-daily, JP/Global reset at 20:00 UTC (03:00 UTC+8). +| Feature | Suggested cadence | +|---|---| +| Cafe income / affection | Every few hours | +| Momo Talk | Every few hours | +| Arena | Around reset windows / twice daily if implemented | +| Mailbox | Daily or with default run | +| AP/stamina/task rewards | Daily | +| Group AP | Daily | +| Bounty/Commissions/Scrimmage | Daily | +| Shops | Daily, after reset | +| Lesson/Schedule | Daily | + +Scheduling should be handled outside the feature logic. + +Feature code should perform one safe run and exit. + +## Safety and robustness rules + +Every task should have: + +- maximum retry count +- timeout where appropriate +- safe failure mode +- clear stdout logging +- no infinite click loops +- no unbounded spending +- config guard for purchases +- dry-run or debug mode when useful + +For purchases: + +- default to conservative behavior +- avoid refresh loops until OCR/currency detection is reliable +- require explicit configured item list +- avoid buying unknown items + +For battle features: + +- require clear stop conditions +- avoid continuing blindly after unexpected state +- prefer returning failure over clicking randomly + +## Configuration direction + +Future config may live in `ba_auto/config.py` or `config.yaml`. + +Possible config values: + +``` +server = JP +game_window_name = BlueArchive +display = :0 +asset_dir = ~/ba_assets +screenshot_dir = ./scratchpad +cafe_max_clicks_per_room +story_sweep_target +shop_buy_list +arena_stop_condition +ocr_enabled +debug_enabled +``` + +Keep config explicit. Do not bury user-specific settings deep inside task logic. + +## Debugging conventions + +Use `./scratchpad` for: + +- temporary screenshots +- cropped templates +- annotated match images +- OCR debug output +- one-off notes + +Do not use `/tmp` or `/private/tmp` unless unavoidable. + +When detector behavior changes, save debug outputs with clear names, for example: + +``` +scratchpad/cafe_match_2026-07-05_001.png +scratchpad/shop_ocr_debug_001.png +``` + +## Local validation commands + +On `nik-macbookair`: + +``` +bash -n ba_dailies.sh +python3 -m py_compile ba_daily.py +python3 -m py_compile ba_auto/*.py +python3 -m py_compile ba_auto/tasks/*.py +``` + +On `nik-gpu`: + +``` +~/ba_dailies.sh mailbox +~/ba_dailies.sh cafe +``` + +After migration: + +``` +~/ba_dailies.sh +``` + +should run the default daily sequence. + +## Near-term recommended task order + +1. Rewrite `ba_dailies.sh` as a thin launcher. +2. Add `ba_daily.py`. +3. Add `ba_auto/driver.py`. +4. Add `ba_auto/detector.py`. +5. Add `ba_auto/navigation.py`. +6. Move mailbox logic to `ba_auto/tasks/mailbox.py`. +7. Move cafe logic to `ba_auto/tasks/cafe.py`. +8. Update `setup.sh`. +9. Add `ba_auto/reference_notes/mapping.md`. +10. Verify existing mailbox and cafe still work. +11. Implement stamina/AP. +12. Implement group/club AP. +13. Implement fixed-target sweep features. +14. Add OCR only when needed. +15. Attempt Arena/Shop/Lesson after the framework is stable. + +## Claude Code guidance summary + +When Claude Code works on this repo, it should follow this rule: + +> Reference first. +> Python first. +> Driver primitives before feature hacks. +> Bash launcher only. + +Do not turn this project into a Bash recreation of Blue Archive Auto Script. diff --git a/setup.sh b/setup.sh new file mode 100755 index 0000000..f41aaa1 --- /dev/null +++ b/setup.sh @@ -0,0 +1,48 @@ +#!/bin/bash +# Bootstrap a fresh clone of this repo so ba_dailies.sh can actually run. +# +# Run this ON nik-gpu, from the repo root (e.g. after `git clone` there) -- +# not from macOS, since it installs into nik-gpu-local paths. +# +# It does NOT touch the game or take any screenshots; it only installs the +# venv and copies files to the fixed paths ba_dailies.sh/detect_and_click.py +# expect (outside the repo, per the two-machine deploy convention in +# CLAUDE.md). +set -e + +VENV_DIR="$HOME/.venvs/ba-auto-daily" +SCRIPTS_DIR="$HOME/ba_scripts" +ASSETS_DIR="$HOME/ba_assets" + +echo "== Checking host tools ==" +missing=0 +for cmd in xdotool scrot; do + if ! command -v "$cmd" >/dev/null 2>&1; then + echo "MISSING: $cmd (install with your package manager, e.g. sudo apt install $cmd)" + missing=1 + fi +done +if [ "$missing" = 1 ]; then + echo "Install the missing tool(s) above, then re-run this script." + exit 1 +fi +echo "xdotool, scrot: OK" + +echo "== Setting up Python venv at $VENV_DIR ==" +if [ ! -x "$VENV_DIR/bin/python3" ]; then + python3 -m venv "$VENV_DIR" +fi +"$VENV_DIR/bin/pip" install --quiet --upgrade pip +"$VENV_DIR/bin/pip" install --quiet opencv-python-headless numpy +"$VENV_DIR/bin/python3" -c "import cv2, numpy; print('opencv', cv2.__version__, '/ numpy', numpy.__version__)" + +echo "== Deploying scripts + assets to fixed paths ==" +mkdir -p "$SCRIPTS_DIR" "$ASSETS_DIR" +cp scripts/detect_and_click.py "$SCRIPTS_DIR/detect_and_click.py" +cp assets/cafe_sparkle.png "$ASSETS_DIR/cafe_sparkle.png" +cp ba_dailies.sh "$HOME/ba_dailies.sh" +chmod +x "$HOME/ba_dailies.sh" + +echo "== Done ==" +echo "Run with: ~/ba_dailies.sh [mailbox|cafe]" +echo "(requires the game already running, window titled 'BlueArchive')"