ba-auto-daily/plan.md

ba-auto-daily implementation plan

Personal Blue Archive JP daily-automation project.

This project controls the PC/Steam/Proton Blue Archive client running on nik-gpu through local desktop automation:

• xdotool
• scrot
• Python
• OpenCV
• later OCR when needed

Development happens on nik-macbookair.

The reference implementation lives at:

~/repo/baas-reference/

The reference project should be treated as the behavioral blueprint. This project should avoid recreating feature logic from scratch when the reference already implements it.

Core project direction

This project is now Python-first.

The goal is not to grow a large Bash script.

The goal is to build a small local Python automation framework that adapts the reference project's Blue Archive logic to this user's unique PC/Steam/Proton environment.

ba_dailies.sh should only be a launcher.

Feature logic should live in Python.

Target architecture

~/repo/ba-auto-daily/
├── ba_dailies.sh
├── ba_daily.py
├── ba_auto/
│   ├── __init__.py
│   ├── config.py
│   ├── driver.py
│   ├── detector.py
│   ├── navigation.py
│   ├── tasks/
│   │   ├── __init__.py
│   │   ├── mailbox.py
│   │   ├── cafe.py
│   │   ├── stamina.py
│   │   ├── group.py
│   │   ├── bounty.py
│   │   ├── commission.py
│   │   ├── arena.py
│   │   ├── shop_common.py
│   │   ├── shop_tactical.py
│   │   ├── lesson.py
│   │   └── ...
│   └── reference_notes/
│       └── mapping.md
├── assets/
├── screenshots/
├── scripts/
├── setup.sh
├── CLAUDE.md
└── plan.md

This is the intended direction. It does not have to be completed all at once.

Project layout

Path	What it is
~/repo/ba-auto-daily/ba_dailies.sh	Thin launcher only. It should call the Python entry point. Do not add new feature logic here.
~/repo/ba-auto-daily/ba_daily.py	Main Python CLI entry point. Dispatches tasks such as mailbox, cafe, stamina, group, etc.
~/repo/ba-auto-daily/ba_auto/driver.py	Local PC/Steam/Proton control backend. Wraps xdotool, scrot, waits, clicks, swipes, keypresses, screenshots, and window focus.
~/repo/ba-auto-daily/ba_auto/detector.py	OpenCV/template/color matching helpers. Existing scripts/detect_and_click.py logic should be migrated here.
~/repo/ba-auto-daily/ba_auto/navigation.py	Shared navigation helpers: home, menu, close popup, back, open feature screens.
~/repo/ba-auto-daily/ba_auto/tasks/	Feature implementations. Each task should adapt the relevant baas-reference/module/... logic where possible.
~/repo/ba-auto-daily/ba_auto/reference_notes/mapping.md	Reference mapping table: local feature → reference module → local implementation → driver gaps.
~/repo/ba-auto-daily/assets/	Locally captured template images, such as cafe sparkle. Do not blindly copy assets from the reference repo.
~/repo/ba-auto-daily/screenshots/	Human reference screenshots, mostly Moonlight/game captures, used for calibration and debugging.
~/repo/ba-auto-daily/scripts/	Transitional scripts. Long-term reusable Python logic should move into ba_auto/.
~/repo/ba-auto-daily/setup.sh	Bootstrap/deploy helper for nik-gpu. Should install/check dependencies and copy runtime files.
~/repo/baas-reference/	Read-only GPL-3.0 reference clone. Study and adapt. Never edit.

Runtime paths on nik-gpu

Preferred runtime layout:

Path	What it is
nik-gpu:~/ba_dailies.sh	Thin launcher.
nik-gpu:~/ba_daily.py	Python CLI entry point.
nik-gpu:~/ba_auto/	Python package copied from this repo.
nik-gpu:~/ba_assets/	Runtime assets/templates.
nik-gpu:~/.venvs/ba-auto-daily/	Python virtual environment.

Current older layout may include:

Path	What it is
nik-gpu:~/ba_scripts/detect_and_click.py	Old standalone detector helper. Should eventually be replaced by ba_auto/detector.py.

Implementation strategy

For each feature:

1. Read the matching ~/repo/baas-reference/module/... file.
2. Summarize the reference flow.
3. Identify reusable logic:
   • state checks
   • retry loops
   • navigation sequence
   • battle/sweep/shop rules
   • detection method
   • failure handling
4. Identify backend-specific calls that cannot be reused directly.
5. Implement missing generic primitives in ba_auto/driver.py or ba_auto/detector.py.
6. Implement the feature in ba_auto/tasks/<feature>.py.
7. Add CLI command dispatch in ba_daily.py.
8. Keep ba_dailies.sh unchanged unless launcher behavior changes.
9. Test syntax locally.
10. Deploy to nik-gpu.
11. Run against the live game.
12. Update this plan.

The intended result is not a Bash automation script.

The intended result is a Python automation framework using the reference repository as the behavioral blueprint.

Reference mapping table

Maintain this table in ba_auto/reference_notes/mapping.md.

Initial seed:

Local feature	Reference file	Reference functions/classes	Local file	Backend replacements	Status
Mailbox	Need to confirm in reference	Need to inspect	ba_auto/tasks/mailbox.py	tap/click via xdotool, screenshot via scrot	Existing Bash behavior; migrate to Python
Cafe	Need to confirm in reference	Need to inspect	ba_auto/tasks/cafe.py	template matching via OpenCV, click via xdotool	Existing Bash + helper behavior; migrate to Python
Stamina/AP	module/collect_daily_free_power.py, module/collect_daily_task_power.py	Need to inspect	ba_auto/tasks/stamina.py	color checks/clicks via local driver	Not started
Group/Club AP	module/group.py	Need to inspect	ba_auto/tasks/group.py	fixed click + state check via local driver	Not started
Bounty	module/rewarded_task.py	Need to inspect	ba_auto/tasks/bounty.py	sweep/color/OCR adaptation	Not started
Commissions	module/clear_special_task_power.py	Need to inspect	ba_auto/tasks/commission.py	sweep/color adaptation	Not started
Arena	module/arena.py	Need to inspect	ba_auto/tasks/arena.py	auto-fight + OCR + local driver	Not started
Common Shop	module/shop/common_shop.py, module/shop/shop_utils.py	Need to inspect	ba_auto/tasks/shop_common.py	OCR + tab navigation + local clicks	Not started
Tactical Shop	module/shop/tactical_challenge_shop.py, module/shop/shop_utils.py	Need to inspect	ba_auto/tasks/shop_tactical.py	OCR + tab navigation + local clicks	Not started
Lesson/Schedule	module/lesson.py	Need to inspect	ba_auto/tasks/lesson.py	OCR + template/portrait search + local driver	Not started

Do not implement a feature without filling at least the relevant row.

Status snapshot

Feature	Current status	Target status
Mailbox claim	Done in old Bash style	Migrate to ba_auto/tasks/mailbox.py
Cafe pats + income	Done in old Bash + standalone Python detector style	Migrate to ba_auto/tasks/cafe.py and ba_auto/detector.py
Shared driver	Partial/implicit in scripts	Build ba_auto/driver.py
Python CLI	Missing	Build ba_daily.py
Reference mapping	Missing	Build ba_auto/reference_notes/mapping.md
Everything else	Not started	Implement reference-first in Python

Migration phase

Before adding new game features, migrate the existing working implementation.

Phase 1: Python skeleton

Create:

ba_daily.py
ba_auto/__init__.py
ba_auto/config.py
ba_auto/driver.py
ba_auto/detector.py
ba_auto/navigation.py
ba_auto/tasks/__init__.py
ba_auto/tasks/mailbox.py
ba_auto/tasks/cafe.py
ba_auto/reference_notes/mapping.md

Phase 2: Launcher

Change ba_dailies.sh into a thin launcher:

#!/usr/bin/env bash
set -euo pipefail
VENV_PYTHON="${VENV_PYTHON:-$HOME/.venvs/ba-auto-daily/bin/python3}"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$VENV_PYTHON" "$SCRIPT_DIR/ba_daily.py" "$@"

Keep compatibility with:

./ba_dailies.sh
./ba_dailies.sh mailbox
./ba_dailies.sh cafe

Phase 3: Driver extraction

Move shell interactions into ba_auto/driver.py.

Driver primitives should include:

focus_game()
click(x, y)
double_click(x, y)
keypress(key)
screenshot(path=None)
swipe(...)
wait(seconds)
wait_until(...)

Phase 4: Detector extraction

Move scripts/detect_and_click.py logic into ba_auto/detector.py.

Detector primitives should include:

load_template(...)
match_template(...)
find_best_match(...)
find_and_click_template(...)
color_mask(...)
debug_write_match(...)

The old script may remain as a compatibility wrapper temporarily, but the reusable logic should live under ba_auto/.

Phase 5: Mailbox migration

Move mailbox logic from Bash to:

ba_auto/tasks/mailbox.py

The CLI should call it through Python.

Phase 6: Cafe migration

Move cafe logic from Bash to:

ba_auto/tasks/cafe.py

During migration, verify:

• both rooms still work
• sparkle detection still works
• cafe income claim still works
• rank-up popups are handled or explicitly documented as not handled
• student rotation popups are handled or explicitly documented as not handled
• view centering/zoom state is robust
• repeated detection does not suffer from Python cold-start delay

Phase 7: setup.sh update

Update setup.sh so it deploys:

ba_dailies.sh
ba_daily.py
ba_auto/
assets/

to the expected runtime paths on nik-gpu.

Prerequisites

OCR

Not set up yet.

Needed for:

• currency readouts
• ticket counts
• region/tab name matching
• some shop logic
• some lesson/schedule logic
• arena ticket/rank/level checks
• bounty coin balance if auto-refresh is implemented

Candidates:

• Tesseract
• PaddleOCR

Do not add OCR until a feature needs it.

Auto-fight primitive

Needed for:

• Arena
• future main story push
• some battle automation

Reference:

~/repo/baas-reference/module/main_story.py

Look for:

auto_fight
enter_battle

Target local module may be:

ba_auto/tasks/battle.py

or:

ba_auto/battle.py

This should become a reusable primitive, not arena-specific code.

High priority backlog

1. Migration to Python-first

This is the highest priority.

Do this before adding new features.

Goal:

• Bash launcher only
• Python CLI
• Python driver
• Python detector
• mailbox migrated
• cafe migrated
• reference mapping started

Reference:

• Current local implementation
• Existing ba_dailies.sh
• Existing scripts/detect_and_click.py

2. Stamina/AP sweep

Claim:

• daily free AP purchase
• daily task-menu AP/pyroxene rewards

Likely mostly color/state detection and fixed clicks.

Reference:

~/repo/baas-reference/module/collect_daily_free_power.py
~/repo/baas-reference/module/collect_daily_task_power.py

Local target: ba_auto/tasks/stamina.py

OCR: Probably not needed for first version.

3. Club/Group AP claim

Claim AP from club/group.

Reference:

~/repo/baas-reference/module/group.py

Local target: ba_auto/tasks/group.py

OCR: Not expected.

4. Normal/Hard story AP sweep

Sweep already-cleared main story stages to burn AP.

Reference:

~/repo/baas-reference/module/explore_tasks/sweep_task.py
~/repo/baas-reference/module/explore_tasks/task_utils.py

Local target: ba_auto/tasks/story_sweep.py

OCR: Likely needed for current region/stage detection unless using fixed configured targets.

Suggested first version:

• user-configured fixed stage
• no region search
• no dynamic OCR
• sweep configured mission only

Later version:

• fuzzy stage/region selection
• OCR-assisted navigation

5. Bounty

Three sub-areas and sweep availability.

Reference:

~/repo/baas-reference/module/rewarded_task.py

Local target: ba_auto/tasks/bounty.py

OCR: Optional for coin balance/refresh logic. Can skip refresh for first version.

6. Commissions

Two sub-dungeons:

• Base Defense
• Item Retrieval

Reference:

~/repo/baas-reference/module/clear_special_task_power.py

Local target: ba_auto/tasks/commission.py

OCR: Probably avoidable for first version if configured fixed sweep target is used.

7. Arena

Fight until out of tickets or until configured stopping condition.

Reference:

~/repo/baas-reference/module/arena.py

Local target: ba_auto/tasks/arena.py

Needs:

• auto-fight primitive
• OCR or visual detection for ticket/rank state
• careful safety limits

Arena is high-value but more risky than fixed claim tasks.

8. Common Shop + Tactical Shop

Auto-buy configured items.

Reference:

~/repo/baas-reference/module/shop/common_shop.py
~/repo/baas-reference/module/shop/tactical_challenge_shop.py
~/repo/baas-reference/module/shop/shop_utils.py

Local targets:

ba_auto/tasks/shop_common.py
ba_auto/tasks/shop_tactical.py

Needs:

• OCR for currency balances
• shop tab detection
• configured buy list
• safe purchase confirmation logic

Start with a no-refresh, fixed configured buy list.

9. Lesson / Schedule

Affection farming via classes.

Reference:

~/repo/baas-reference/module/lesson.py

Local target: ba_auto/tasks/lesson.py

Complexity: High

Needs:

• OCR for region/area names
• multi-page swipe search
• student detection/portrait matching if targeting specific students
• isometric grid location logic if following reference fully

Suggested first version:

• pick a fixed region
• select available/highest visible lesson
• avoid favorite-student targeting at first

Low priority backlog

Scrimmage

Reference:

~/repo/baas-reference/module/scrimmage.py

Local target: ba_auto/tasks/scrimmage.py

Similar shape to Bounty/Commissions.

Crafting

Reference:

~/repo/baas-reference/module/create.py

Local target: ba_auto/tasks/crafting.py

Very complex. Contains:

• material selection
• priority lists
• rarity tiers
• stepper/quantity UI
• filtering/sorting
• OCR-like decision points

Do not start until the framework and OCR are mature.

Battle Pass claim

Reference:

~/repo/baas-reference/module/collect_pass_reward.py

Local target: ba_auto/tasks/battle_pass.py

Should be simpler than crafting.

OCR only needed for optional stats.

Momo Talk

Reference:

~/repo/baas-reference/module/momo_talk.py

Local target: ba_auto/tasks/momo_talk.py

Potentially useful because it runs on a different cadence from daily reset.

Likely no OCR. Mostly state scanning and click flow.

Main story push

This means clearing new uncleared stages, not sweeping already-cleared stages.

Reference:

~/repo/baas-reference/module/main_story.py
~/repo/baas-reference/module/explore_tasks/explore_task.py

Low priority because full grid-mode support requires lots of per-stage scripting.

A simple auto-fight-only mode can be added later.

Group Story / Mini Story

Reference:

~/repo/baas-reference/module/group_story.py
~/repo/baas-reference/module/mini_story.py

Convenience only.

Event content

Reference:

~/repo/baas-reference/module/activities/activity_utils.py
~/repo/baas-reference/module/sweep_activity.py

Do not prebuild specific event scripts.

Build only generic event AP sweep if it can reuse story sweep logic.

Event-specific content expires and is not worth hardcoding unless the user asks for a specific live event.

Skip list

Feature	Why skip
Total Assault / Raid	Low value and risky to automate. Reference support may be limited/stubbed.
Joint Firing Drill	Not worth prioritizing for JP if reference has server-specific limitations.
De-clothes localization toggle	CN-only / irrelevant.
Restart / refresh-uiautomator2	Android/ADB backend maintenance, not applicable to PC/Steam/Proton.
Auto-unfriend	Risky, low value, destructive.
Daily minigame dispatcher	Event-specific and unstable. Handle ad hoc only.

Automation cadence notes

Some tasks decay on different schedules.

Feature	Suggested cadence
Cafe income / affection	Every few hours
Momo Talk	Every few hours
Arena	Around reset windows / twice daily if implemented
Mailbox	Daily or with default run
AP/stamina/task rewards	Daily
Group AP	Daily
Bounty/Commissions/Scrimmage	Daily
Shops	Daily, after reset
Lesson/Schedule	Daily

Scheduling should be handled outside the feature logic.

Feature code should perform one safe run and exit.

Safety and robustness rules

Every task should have:

• maximum retry count
• timeout where appropriate
• safe failure mode
• clear stdout logging
• no infinite click loops
• no unbounded spending
• config guard for purchases
• dry-run or debug mode when useful

For purchases:

• default to conservative behavior
• avoid refresh loops until OCR/currency detection is reliable
• require explicit configured item list
• avoid buying unknown items

For battle features:

• require clear stop conditions
• avoid continuing blindly after unexpected state
• prefer returning failure over clicking randomly

Configuration direction

Future config may live in ba_auto/config.py or config.yaml.

Possible config values:

server = JP
game_window_name = BlueArchive
display = :0
asset_dir = ~/ba_assets
screenshot_dir = ./scratchpad
cafe_max_clicks_per_room
story_sweep_target
shop_buy_list
arena_stop_condition
ocr_enabled
debug_enabled

Keep config explicit. Do not bury user-specific settings deep inside task logic.

Debugging conventions

Use ./scratchpad for:

• temporary screenshots
• cropped templates
• annotated match images
• OCR debug output
• one-off notes

Do not use /tmp or /private/tmp unless unavoidable.

When detector behavior changes, save debug outputs with clear names, for example:

scratchpad/cafe_match_2026-07-05_001.png
scratchpad/shop_ocr_debug_001.png

Local validation commands

On nik-macbookair:

bash -n ba_dailies.sh
python3 -m py_compile ba_daily.py
python3 -m py_compile ba_auto/*.py
python3 -m py_compile ba_auto/tasks/*.py

On nik-gpu:

~/ba_dailies.sh mailbox
~/ba_dailies.sh cafe

After migration:

~/ba_dailies.sh

should run the default daily sequence.

Near-term recommended task order

1. Rewrite ba_dailies.sh as a thin launcher.
2. Add ba_daily.py.
3. Add ba_auto/driver.py.
4. Add ba_auto/detector.py.
5. Add ba_auto/navigation.py.
6. Move mailbox logic to ba_auto/tasks/mailbox.py.
7. Move cafe logic to ba_auto/tasks/cafe.py.
8. Update setup.sh.
9. Add ba_auto/reference_notes/mapping.md.
10. Verify existing mailbox and cafe still work.
11. Implement stamina/AP.
12. Implement group/club AP.
13. Implement fixed-target sweep features.
14. Add OCR only when needed.
15. Attempt Arena/Shop/Lesson after the framework is stable.

Claude Code guidance summary

When Claude Code works on this repo, it should follow this rule:

Reference first. Python first. Driver primitives before feature hacks. Bash launcher only.

Do not turn this project into a Bash recreation of Blue Archive Auto Script.