ba-auto-daily/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

What this is

Automation for daily chores in the mobile game Blue Archive: claiming mailbox rewards and farming/collecting cafe affection income. It drives the actual game client via simulated mouse clicks and keypresses (xdotool) against fixed screen coordinates, plus one template-matching detector for a dynamic (moving) UI element.

Two-machine architecture

This repo is developed on macOS (nik-macbookair) but the code only runs on a separate Linux box, nik-gpu, where the Blue Archive client and its X display actually live (DISPLAY=:0, GDM XAUTHORITY under /run/user/1000). There is no local way to execute or test these scripts — they must be deployed to nik-gpu to run for real.

• ba_dailies.sh and scripts/detect_and_click.py are pushed to nik-gpu with scp/rsync (e.g. scp ba_dailies.sh nik-gpu:~/ba_dailies.sh, scp scripts/detect_and_click.py nik-gpu:~/ba_scripts/detect_and_click.py) and invoked there over ssh.
• assets/cafe_sparkle.png (the template image) is likewise copied to nik-gpu:~/ba_assets/cafe_sparkle.png.
• detect_and_click.py runs a Python venv on nik-gpu at ~/.venvs/ba-auto-daily/bin/python3 (needs opencv-python/numpy); ba_dailies.sh invokes it via that fixed path (VENV_PYTHON in the script).
• Screenshot -> detect -> click for the cafe sparkle happens entirely on nik-gpu in one local pipeline (see the docstring in scripts/detect_and_click.py) rather than round-tripping images over SSH, because the sparkle target moves fast enough that a multi-hop pipeline would miss the click.
• screenshots/ holds reference captures (taken on nik-gpu, pulled back for inspection) used to work out coordinates and template thresholds when coordinates drift after a game UI update — they are not test fixtures consumed by any script.

Because there's no local execution path, "testing a change" means syntax-checking locally and then deploying to nik-gpu and running it against the live game there:

bash -n ba_dailies.sh
python3 -m py_compile scripts/detect_and_click.py

Dependencies on nik-gpu

These are host-level prerequisites, not managed by this repo — confirm they're present before assuming a failure is a code bug:

• xdotool (window focus, clicks, keypresses)
• scrot (screenshot capture for the detector)
• A venv at ~/.venvs/ba-auto-daily/ with opencv-python and numpy installed, Python binary at ~/.venvs/ba-auto-daily/bin/python3

Quick check:

ssh nik-gpu "which xdotool scrot && ~/.venvs/ba-auto-daily/bin/python3 -c 'import cv2, numpy; print(cv2.__version__)'"

Working conventions

Use ./scratchpad (create if missing) in the project root for temporary/intermediate files — e.g. cropped calibration images from screenshots/cafe/sparkle/, one-off debug output. Never write to /tmp or /private/tmp.

ba_dailies.sh

Entry point, run on nik-gpu as ./ba_dailies.sh [mailbox|cafe]:

• No argument: focuses the game window, runs mailbox claim, then the full cafe routine.
• mailbox: focus + claim mailbox only.
• cafe: focus + cafe routine only (both rooms + income claim).

All interaction points (icon/button coordinates, max click-attempts per cafe room) are top-of-file constants — when the in-game UI shifts or the window resolution changes, update the coordinates there rather than inline in the functions. focus_game finds the window via xdotool search --name "BlueArchive" and hard-fails if the game isn't running.

The cafe routine (do_cafe) alternates between two cafe rooms; for each room it repeatedly calls into detect_and_click.py to find and click affection sparkles (up to CAFE_MAX_CLICKS_PER_ROOM times) before moving on, then claims cafe income at the end.

Known gap — needs verification: the manual cafe flow this automates includes a few conditional steps not obviously covered above: closing a popup that only appears if a student is "rotated," zooming out/centering the view before detection starts, and closing a rank-up popup that can appear after a successful click. Confirm these are actually handled somewhere in do_cafe (or decide they're unnecessary in practice) rather than assuming coverage from this description alone.

scripts/detect_and_click.py

Standalone script (runs on nik-gpu, called once per sparkle-click attempt from the shell loop): screenshots the game window with scrot, template-matches cafe_sparkle.png using a color-masked cv2.matchTemplate (masks to yellow/white sparkle pixels so it doesn't match on background art), clicks the best match (offset-corrected — the template's anchor point isn't the click point), and reports its result on stdout/exit code (MATCH x y score / exit 0, or NO_MATCH / exit 1) so the caller shell loop can decide whether to keep clicking.

THRESHOLD = 0.97 and OFFSET_X/OFFSET_Y are the values most likely to need retuning if detection starts missing or mis-clicking — use the screenshots/cafe/sparkle/ reference captures to recalibrate. When cropping or annotating these captures for calibration, use ./scratchpad in the project root for the intermediate files, not /tmp.

Performance note — needs verification: this script is invoked as a fresh process per click attempt, and Python + OpenCV cold-start has real overhead (commonly 200–500ms). If the sparkle target moves fast, confirm this hasn't caused missed detections in practice against the live game. If it has, consider a persistent process the shell loop talks to (pipe/socket) instead of a per-attempt cold start.