fix: Ansible cleanup -- merge dual config, pin collections, align K3s version, fix bugs
Stage 4 of REFACTOR_PLAN.md. - Merge ansible/ansible.cfg into root ansible.cfg (single source of truth); the dual-config setup silently broke documented commands and lost host_key_checking=False when run from the ansible/ directory. - Add ansible/requirements.yml pinning community.general, ansible.posix, community.docker -- previously undocumented deps of the glances/watch-party roles that would fail a fresh bootstrap. - Align K3s version to v1.32.4+k3s1 across roles/k3s-server, roles/k3s-agent, and host_vars/gpu-node.yaml defaults (was skewed: .2 vs .4). This only changes what a *future* provision installs -- minisforum/debian are still live on v1.32.2+k3s1 until separately upgraded. - Fix kubeconfig fetch/replace path mismatch in k3s-server role: the `fetch` task (flat: true) writes to ~/.kube/config, but `replace` was targeting a /tmp/k3s-minisforum.yaml nothing creates -- would break a fresh rebuild. - gitea-runner: only remove /run/docker.sock when it's actually a directory (task name implied a check that wasn't there); tighten registration-token systemd unit from 0644 to 0600. - nvidia: stop unconditionally reporting `changed` (and restarting Docker) on every run for an idempotent runtime-configure command. - Gate the K3s join-token debug print and WireGuard client-config/QR display behind opt-in vars (k3s_show_token, wireguard_show_client_configs), default off -- both were printing real secrets to console on every run. - Parameterize the docker role for Debian and Ubuntu; homeassistant now depends on it (meta/main.yaml) instead of duplicating a Debian-only Docker install inline. - FQCN cleanup across wireguard, homeassistant, and ollama roles/handlers (bare module names -> ansible.builtin.*/community.general.*/ansible.posix.*), plus a few ansible-lint name-casing/idiom fixes. Handler renames verified against their `notify:` call sites so notifications still fire. - Update ansible/README.md and root README.md: add gpu-node/gpu_workstation (4th host, previously undocumented), docker/nvidia roles, setup-gpu-node.yaml, homeassistant.yaml, requirements.yml install step; correct the "Legacy" homeassistant label (it's the only thing serving ha.home.arpa); correct the Gitea ingress/backup-storage doc-drift; flag the Grafana/Loki static-PV binding drift discovered via live cluster check. Verified: all playbooks pass `ansible-playbook --syntax-check`, yamllint clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
57a78fd8e7
commit
5a00f5767b
27
README.md
27
README.md
@ -11,6 +11,7 @@ in Kubernetes manifests plus Helm values.
|
|||||||
| `minisforum` | `192.168.7.77` | K3s server, Traefik entrypoint, primary app node |
|
| `minisforum` | `192.168.7.77` | K3s server, Traefik entrypoint, primary app node |
|
||||||
| `debian` / `nik-debian` | `192.168.7.183` | K3s agent, NFS storage, secondary Pi-hole |
|
| `debian` / `nik-debian` | `192.168.7.183` | K3s agent, NFS storage, secondary Pi-hole |
|
||||||
| `mac-mini` | `192.168.7.96` | Standalone services such as Watch Party and Ollama |
|
| `mac-mini` | `192.168.7.96` | Standalone services such as Watch Party and Ollama |
|
||||||
|
| `gpu-node` / `nik-gpu` | `192.168.7.98` | K3s agent with NVIDIA GPU passthrough, spot-tainted; runs Ollama directly on the host |
|
||||||
|
|
||||||
The cluster uses Traefik instead of the bundled K3s ingress controller. Internal
|
The cluster uses Traefik instead of the bundled K3s ingress controller. Internal
|
||||||
services are published under `home.arpa` with certificates from an internal CA.
|
services are published under `home.arpa` with certificates from an internal CA.
|
||||||
@ -179,12 +180,14 @@ state that must live on known disks:
|
|||||||
| Location | Use |
|
| Location | Use |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| `/data/gitea` on `minisforum` | Gitea shared storage |
|
| `/data/gitea` on `minisforum` | Gitea shared storage |
|
||||||
| `/data/prometheus` on `minisforum` | Prometheus |
|
| `/data/prometheus` on `minisforum` | Prometheus (bound correctly) |
|
||||||
| `/data/grafana` on `minisforum` | Grafana |
|
| `/data/grafana` on `minisforum` | Static PV defined for Grafana, but **not currently bound** — see gotcha below |
|
||||||
| `/data/loki` on `minisforum` | Loki |
|
| `/data/loki` on `minisforum` | Static PV defined for Loki, but **not currently bound** — see gotcha below |
|
||||||
| `/mnt/storage` on `debian` | NFS media library and backups |
|
| `/mnt/storage` on `debian` | NFS media library |
|
||||||
|
| `/home/nik/backups` on `debian` | NFS export for Gitea's backup CronJob (separate from `/mnt/storage`) |
|
||||||
|
|
||||||
The Debian NFS server exports `/mnt/storage` to `192.168.7.77`.
|
The Debian NFS server exports both `/mnt/storage` and `/home/nik/backups` to
|
||||||
|
`192.168.7.77`.
|
||||||
|
|
||||||
## TLS and Trust
|
## TLS and Trust
|
||||||
|
|
||||||
@ -200,8 +203,18 @@ mobileconfig profile. The `ca-sync` CronJob updates those files from the
|
|||||||
|
|
||||||
- Argo CD Applications mostly set `prune: false`; removing resources from Git may
|
- Argo CD Applications mostly set `prune: false`; removing resources from Git may
|
||||||
require manual cleanup.
|
require manual cleanup.
|
||||||
- Gitea uses a manual public `IngressRoute`; the chart ingress is disabled in
|
- Gitea uses a manual public `IngressRoute`; `values/gitea.yaml` has no
|
||||||
`values/gitea.yaml`.
|
`ingress:` key at all, so the chart's own ingress is off by chart default,
|
||||||
|
not an explicit setting.
|
||||||
|
- Grafana and Loki's static hostPath PVs (`grafana-pv`, `loki-pv` in
|
||||||
|
`manifests/monitoring/monitoring-pvs.yaml`) are currently unbound — their
|
||||||
|
Helm-managed PVCs got dynamically provisioned via the `local-path`
|
||||||
|
StorageClass instead (confirmed live via `kubectl get pv/pvc -n
|
||||||
|
monitoring`), unlike Prometheus which binds `prometheus-pv` correctly. Data
|
||||||
|
is not lost, just not on the disk the docs/manifest imply — needs a
|
||||||
|
deliberate decision (bind properly with a data migration, or drop the
|
||||||
|
orphaned static PVs and document reality) before relying on `/data/grafana`
|
||||||
|
or `/data/loki` for backups/DR.
|
||||||
- Gitea `ROOT_URL` changes can require deleting the generated inline config
|
- Gitea `ROOT_URL` changes can require deleting the generated inline config
|
||||||
secret before reconciling.
|
secret before reconciling.
|
||||||
- Pi-hole does not provide wildcard DNS here; add each new internal hostname to
|
- Pi-hole does not provide wildcard DNS here; add each new internal hostname to
|
||||||
|
|||||||
@ -5,3 +5,5 @@
|
|||||||
inventory = ansible/inventory.yaml
|
inventory = ansible/inventory.yaml
|
||||||
roles_path = ansible/roles
|
roles_path = ansible/roles
|
||||||
host_key_checking = False
|
host_key_checking = False
|
||||||
|
inject_facts_as_vars = False
|
||||||
|
deprecation_warnings = False
|
||||||
|
|||||||
@ -6,16 +6,28 @@ cluster.
|
|||||||
|
|
||||||
## Inventory
|
## Inventory
|
||||||
|
|
||||||
`inventory.yaml` defines three groups:
|
`inventory.yaml` defines four groups:
|
||||||
|
|
||||||
| Group | Host | Purpose |
|
| Group | Host | Purpose |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| `k3s_server` | `minisforum` | K3s server at `192.168.7.77` |
|
| `k3s_server` | `minisforum` | K3s server at `192.168.7.77` |
|
||||||
| `k3s_agents` | `debian` | K3s agent and NFS storage at `192.168.7.183` |
|
| `k3s_agents` | `debian` | K3s agent and NFS storage at `192.168.7.183` |
|
||||||
| `mac_mini` | `mac-mini` | Docker/Ollama host at `192.168.7.96` |
|
| `mac_mini` | `mac-mini` | Docker/Ollama host at `192.168.7.96` |
|
||||||
|
| `gpu_workstation` | `gpu-node` | K3s agent with NVIDIA GPU passthrough at `192.168.7.98` (spot-tainted) |
|
||||||
|
|
||||||
All hosts use the `nik` user and the SSH key configured in `inventory.yaml`.
|
All hosts use the `nik` user and the SSH key configured in `inventory.yaml`.
|
||||||
|
|
||||||
|
## Collections
|
||||||
|
|
||||||
|
Install the third-party collections this repo's roles depend on before
|
||||||
|
running any playbook:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ansible-galaxy collection install -r ansible/requirements.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
(`community.general`, `ansible.posix`, `community.docker`.)
|
||||||
|
|
||||||
## Common Playbooks
|
## Common Playbooks
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@ -23,6 +35,7 @@ ansible-playbook -i ansible/inventory.yaml ansible/playbooks/bootstrap-minisforu
|
|||||||
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-k3s.yaml -K
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-k3s.yaml -K
|
||||||
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-nfs-debian.yaml -K
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-nfs-debian.yaml -K
|
||||||
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/join-debian-agent.yaml -K
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/join-debian-agent.yaml -K
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-gpu-node.yaml -K
|
||||||
```
|
```
|
||||||
|
|
||||||
Additional services:
|
Additional services:
|
||||||
@ -35,6 +48,7 @@ ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-glances-debia
|
|||||||
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-ollama.yaml -K
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-ollama.yaml -K
|
||||||
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/deploy-watch-party.yaml
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/deploy-watch-party.yaml
|
||||||
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/wireguard.yaml -K
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/wireguard.yaml -K
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/homeassistant.yaml -K
|
||||||
```
|
```
|
||||||
|
|
||||||
## Roles
|
## Roles
|
||||||
@ -42,21 +56,24 @@ ansible-playbook -i ansible/inventory.yaml ansible/playbooks/wireguard.yaml -K
|
|||||||
| Role | Responsibility |
|
| Role | Responsibility |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| `common` | Packages, user setup, firewall, base data directories |
|
| `common` | Packages, user setup, firewall, base data directories |
|
||||||
|
| `docker` | Docker CE install (Debian and Ubuntu); depended on by `homeassistant` |
|
||||||
|
| `nvidia` | NVIDIA driver, CUDA toolkit, and containerd/Docker GPU runtime config |
|
||||||
| `k3s-server` | K3s server install, kubeconfig fetch, Helm install, primary node label |
|
| `k3s-server` | K3s server install, kubeconfig fetch, Helm install, primary node label |
|
||||||
| `k3s-agent` | K3s agent join and storage node label |
|
| `k3s-agent` | K3s agent join and storage/GPU node label |
|
||||||
| `nfs-server` | Export `/mnt/storage` from Debian to the K3s server |
|
| `nfs-server` | Export `/mnt/storage` from Debian to the K3s server |
|
||||||
| `monitoring` | Host directories and ownership for Prometheus/Loki |
|
| `monitoring` | Host directories and ownership for Prometheus/Loki |
|
||||||
| `gitea-runner` | Gitea Actions runner systemd service |
|
| `gitea-runner` | Gitea Actions runner systemd service |
|
||||||
| `glances` | Host-level Glances service |
|
| `glances` | Host-level Glances service |
|
||||||
| `ollama` | Ollama service on the Mac Mini |
|
| `ollama` | Ollama service on the Mac Mini and GPU node (branches on OS) |
|
||||||
| `watch-party` | Watch Party Docker Compose deployment on the Mac Mini |
|
| `watch-party` | Watch Party Docker Compose deployment on the Mac Mini |
|
||||||
| `wireguard` | WireGuard server configuration |
|
| `wireguard` | WireGuard server configuration |
|
||||||
| `homeassistant` | Legacy standalone Home Assistant deployment |
|
| `homeassistant` | Standalone Home Assistant deployment (Docker Compose + systemd on `minisforum`) — this is the **only** thing serving `ha.home.arpa`, not legacy/dead |
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
|
|
||||||
- K3s version is set in `roles/k3s-server/defaults/main.yaml` and
|
- K3s version is defined in three places and must be kept in sync:
|
||||||
`roles/k3s-agent/defaults/main.yaml`.
|
`roles/k3s-server/defaults/main.yaml`, `roles/k3s-agent/defaults/main.yaml`,
|
||||||
|
and the override in `host_vars/gpu-node.yaml`.
|
||||||
- `setup-gitea-runner.yaml` reads `GITEA_RUNNER_TOKEN` from the local
|
- `setup-gitea-runner.yaml` reads `GITEA_RUNNER_TOKEN` from the local
|
||||||
environment.
|
environment.
|
||||||
- The K3s role disables bundled Traefik because Traefik is managed by Argo CD.
|
- The K3s role disables bundled Traefik because Traefik is managed by Argo CD.
|
||||||
@ -64,3 +81,7 @@ ansible-playbook -i ansible/inventory.yaml ansible/playbooks/wireguard.yaml -K
|
|||||||
mount that export directly.
|
mount that export directly.
|
||||||
- Keep host automation idempotent where practical. These playbooks are meant to
|
- Keep host automation idempotent where practical. These playbooks are meant to
|
||||||
be rerunnable during rebuilds.
|
be rerunnable during rebuilds.
|
||||||
|
- To see the real K3s join token (needed once, to populate
|
||||||
|
`vault_k3s_node_token`), pass `-e k3s_show_token=true` to `setup-k3s.yaml`;
|
||||||
|
it's suppressed by default. Same pattern for WireGuard client configs via
|
||||||
|
`-e wireguard_show_client_configs=true` on `wireguard.yaml`.
|
||||||
|
|||||||
@ -1,4 +0,0 @@
|
|||||||
[defaults]
|
|
||||||
inventory = inventory.yaml
|
|
||||||
inject_facts_as_vars = False
|
|
||||||
deprecation_warnings = False
|
|
||||||
@ -32,6 +32,8 @@ k3s_server_url: "https://192.168.7.77:6443"
|
|||||||
k3s_node_token: "{{ vault_k3s_node_token }}"
|
k3s_node_token: "{{ vault_k3s_node_token }}"
|
||||||
|
|
||||||
# Check current cluster version with: k3s --version on minisforum
|
# Check current cluster version with: k3s --version on minisforum
|
||||||
|
# Kept in sync with roles/k3s-server and roles/k3s-agent defaults — all three
|
||||||
|
# must match; see ansible/README.md "K3s version" note.
|
||||||
k3s_version: "v1.32.4+k3s1"
|
k3s_version: "v1.32.4+k3s1"
|
||||||
|
|
||||||
k3s_node_labels:
|
k3s_node_labels:
|
||||||
|
|||||||
@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
- name: Deploy Home Assistant on Minisforum
|
- name: Deploy Home Assistant on Minisforum
|
||||||
hosts: minisforum
|
hosts: minisforum
|
||||||
become: yes
|
become: true
|
||||||
roles:
|
roles:
|
||||||
- homeassistant
|
- homeassistant
|
||||||
@ -1,6 +1,6 @@
|
|||||||
# Config for: gpu-node workstation full setup
|
# Config for: gpu-node workstation full setup
|
||||||
# Applied by: ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-gpu-node.yaml
|
# Applied by: ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-gpu-node.yaml
|
||||||
- name: gpu-node setup
|
- name: GPU node setup
|
||||||
hosts: gpu_workstation
|
hosts: gpu_workstation
|
||||||
become: true
|
become: true
|
||||||
|
|
||||||
|
|||||||
6
ansible/requirements.yml
Normal file
6
ansible/requirements.yml
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
# Config for: Ansible Galaxy collections
|
||||||
|
# Applied by: ansible-galaxy collection install -r ansible/requirements.yml
|
||||||
|
collections:
|
||||||
|
- name: community.general
|
||||||
|
- name: ansible.posix
|
||||||
|
- name: community.docker
|
||||||
4
ansible/roles/docker/defaults/main.yaml
Normal file
4
ansible/roles/docker/defaults/main.yaml
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
---
|
||||||
|
# Part of role: docker
|
||||||
|
# Description: Default vars so this role doesn't depend on `common` having run first in the same play.
|
||||||
|
username: nik
|
||||||
@ -1,12 +1,26 @@
|
|||||||
---
|
---
|
||||||
# Part of role: docker
|
# Part of role: docker
|
||||||
# Called by: ansible/playbooks/setup-gpu-node.yaml
|
# Called by: ansible/playbooks/setup-gpu-node.yaml, ansible/roles/homeassistant (meta dependency)
|
||||||
# Description: Installs Docker CE on Ubuntu, adds user to docker group.
|
# Description: Installs Docker CE, adds user to docker group. Works on both Debian and Ubuntu.
|
||||||
|
|
||||||
|
- name: Install Docker prerequisites
|
||||||
|
ansible.builtin.apt:
|
||||||
|
name:
|
||||||
|
- ca-certificates
|
||||||
|
- curl
|
||||||
|
state: present
|
||||||
|
update_cache: true
|
||||||
|
|
||||||
|
- name: Create apt keyrings directory
|
||||||
|
ansible.builtin.file:
|
||||||
|
path: /usr/share/keyrings
|
||||||
|
state: directory
|
||||||
|
mode: "0755"
|
||||||
|
|
||||||
- name: Add Docker GPG key
|
- name: Add Docker GPG key
|
||||||
ansible.builtin.shell:
|
ansible.builtin.shell:
|
||||||
cmd: >
|
cmd: >
|
||||||
curl -fsSL https://download.docker.com/linux/ubuntu/gpg |
|
curl -fsSL https://download.docker.com/linux/{{ ansible_facts['distribution'] | lower }}/gpg |
|
||||||
gpg --dearmor -o /usr/share/keyrings/docker.gpg
|
gpg --dearmor -o /usr/share/keyrings/docker.gpg
|
||||||
creates: /usr/share/keyrings/docker.gpg
|
creates: /usr/share/keyrings/docker.gpg
|
||||||
|
|
||||||
@ -14,12 +28,12 @@
|
|||||||
ansible.builtin.apt_repository:
|
ansible.builtin.apt_repository:
|
||||||
repo: >
|
repo: >
|
||||||
deb [arch=amd64 signed-by=/usr/share/keyrings/docker.gpg]
|
deb [arch=amd64 signed-by=/usr/share/keyrings/docker.gpg]
|
||||||
https://download.docker.com/linux/ubuntu
|
https://download.docker.com/linux/{{ ansible_facts['distribution'] | lower }}
|
||||||
{{ ansible_facts['distribution_release'] }} stable
|
{{ ansible_facts['distribution_release'] }} stable
|
||||||
filename: docker
|
filename: docker
|
||||||
state: present
|
state: present
|
||||||
|
|
||||||
- name: Install Docker CE
|
- name: Install Docker CE and Compose plugin
|
||||||
ansible.builtin.apt:
|
ansible.builtin.apt:
|
||||||
name:
|
name:
|
||||||
- docker-ce
|
- docker-ce
|
||||||
|
|||||||
@ -74,7 +74,7 @@
|
|||||||
|
|
||||||
[Install]
|
[Install]
|
||||||
WantedBy=multi-user.target
|
WantedBy=multi-user.target
|
||||||
mode: "0644"
|
mode: "0600"
|
||||||
become: true
|
become: true
|
||||||
notify: Restart act_runner
|
notify: Restart act_runner
|
||||||
|
|
||||||
@ -86,11 +86,18 @@
|
|||||||
become: false
|
become: false
|
||||||
changed_when: false
|
changed_when: false
|
||||||
|
|
||||||
|
- name: Check docker.sock type
|
||||||
|
ansible.builtin.stat:
|
||||||
|
path: /run/docker.sock
|
||||||
|
register: docker_sock_stat
|
||||||
|
become: true
|
||||||
|
|
||||||
- name: Remove docker.sock if it is a directory
|
- name: Remove docker.sock if it is a directory
|
||||||
ansible.builtin.file:
|
ansible.builtin.file:
|
||||||
path: /run/docker.sock
|
path: /run/docker.sock
|
||||||
state: absent
|
state: absent
|
||||||
become: true
|
become: true
|
||||||
|
when: docker_sock_stat.stat.exists and docker_sock_stat.stat.isdir
|
||||||
|
|
||||||
- name: Enable and start Docker
|
- name: Enable and start Docker
|
||||||
ansible.builtin.systemd:
|
ansible.builtin.systemd:
|
||||||
|
|||||||
@ -1,4 +1,4 @@
|
|||||||
---
|
---
|
||||||
- name: reload systemd
|
- name: Reload systemd
|
||||||
systemd:
|
ansible.builtin.systemd:
|
||||||
daemon_reload: yes
|
daemon_reload: true
|
||||||
|
|||||||
3
ansible/roles/homeassistant/meta/main.yaml
Normal file
3
ansible/roles/homeassistant/meta/main.yaml
Normal file
@ -0,0 +1,3 @@
|
|||||||
|
---
|
||||||
|
dependencies:
|
||||||
|
- role: docker
|
||||||
@ -1,55 +1,11 @@
|
|||||||
---
|
---
|
||||||
- name: Install Docker prerequisites
|
# Part of role: homeassistant
|
||||||
apt:
|
# Called by: ansible/playbooks/homeassistant.yaml
|
||||||
name:
|
# Description: Deploys the standalone Home Assistant Docker Compose stack. Docker itself is
|
||||||
- ca-certificates
|
# installed by the `docker` role (meta dependency) instead of being duplicated here.
|
||||||
- curl
|
|
||||||
state: present
|
|
||||||
update_cache: yes
|
|
||||||
|
|
||||||
- name: Create apt keyrings directory
|
|
||||||
file:
|
|
||||||
path: /etc/apt/keyrings
|
|
||||||
state: directory
|
|
||||||
mode: "0755"
|
|
||||||
|
|
||||||
- name: Download Docker GPG key
|
|
||||||
get_url:
|
|
||||||
url: https://download.docker.com/linux/debian/gpg
|
|
||||||
dest: /etc/apt/keyrings/docker.asc
|
|
||||||
mode: "0644"
|
|
||||||
|
|
||||||
- name: Add Docker apt repository
|
|
||||||
apt_repository:
|
|
||||||
repo: "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian trixie stable"
|
|
||||||
state: present
|
|
||||||
filename: docker
|
|
||||||
|
|
||||||
- name: Install Docker CE and Compose plugin
|
|
||||||
apt:
|
|
||||||
name:
|
|
||||||
- docker-ce
|
|
||||||
- docker-ce-cli
|
|
||||||
- containerd.io
|
|
||||||
- docker-buildx-plugin
|
|
||||||
- docker-compose-plugin
|
|
||||||
state: present
|
|
||||||
update_cache: yes
|
|
||||||
|
|
||||||
- name: Enable and start Docker
|
|
||||||
systemd:
|
|
||||||
name: docker
|
|
||||||
enabled: yes
|
|
||||||
state: started
|
|
||||||
|
|
||||||
- name: Add nik to docker group
|
|
||||||
user:
|
|
||||||
name: nik
|
|
||||||
groups: docker
|
|
||||||
append: yes
|
|
||||||
|
|
||||||
- name: Create HA config directory
|
- name: Create HA config directory
|
||||||
file:
|
ansible.builtin.file:
|
||||||
path: /home/nik/homeassistant/config
|
path: /home/nik/homeassistant/config
|
||||||
state: directory
|
state: directory
|
||||||
owner: nik
|
owner: nik
|
||||||
@ -57,7 +13,7 @@
|
|||||||
mode: "0755"
|
mode: "0755"
|
||||||
|
|
||||||
- name: Deploy docker-compose.yaml
|
- name: Deploy docker-compose.yaml
|
||||||
template:
|
ansible.builtin.template:
|
||||||
src: docker-compose.yaml.j2
|
src: docker-compose.yaml.j2
|
||||||
dest: /home/nik/homeassistant/docker-compose.yaml
|
dest: /home/nik/homeassistant/docker-compose.yaml
|
||||||
owner: nik
|
owner: nik
|
||||||
@ -65,24 +21,24 @@
|
|||||||
mode: "0644"
|
mode: "0644"
|
||||||
|
|
||||||
- name: Deploy systemd unit
|
- name: Deploy systemd unit
|
||||||
template:
|
ansible.builtin.template:
|
||||||
src: homeassistant.service.j2
|
src: homeassistant.service.j2
|
||||||
dest: /etc/systemd/system/homeassistant.service
|
dest: /etc/systemd/system/homeassistant.service
|
||||||
mode: "0644"
|
mode: "0644"
|
||||||
notify: reload systemd
|
notify: Reload systemd
|
||||||
|
|
||||||
- name: Deploy base configuration.yaml
|
- name: Deploy base configuration.yaml
|
||||||
template:
|
ansible.builtin.template:
|
||||||
src: configuration.yaml.j2
|
src: configuration.yaml.j2
|
||||||
dest: /home/nik/homeassistant/config/configuration.yaml
|
dest: /home/nik/homeassistant/config/configuration.yaml
|
||||||
owner: nik
|
owner: nik
|
||||||
group: nik
|
group: nik
|
||||||
mode: "0644"
|
mode: "0644"
|
||||||
force: no
|
force: false
|
||||||
|
|
||||||
- name: Enable and start homeassistant
|
- name: Enable and start homeassistant
|
||||||
systemd:
|
ansible.builtin.systemd:
|
||||||
name: homeassistant
|
name: homeassistant
|
||||||
enabled: yes
|
enabled: true
|
||||||
state: started
|
state: started
|
||||||
daemon_reload: yes
|
daemon_reload: true
|
||||||
|
|||||||
@ -3,6 +3,6 @@
|
|||||||
# Called by: ansible/playbooks/join-debian-agent.yaml
|
# Called by: ansible/playbooks/join-debian-agent.yaml
|
||||||
# Description: Default variables for the k3s-agent role including version, server URL, and join token.
|
# Description: Default variables for the k3s-agent role including version, server URL, and join token.
|
||||||
|
|
||||||
k3s_version: v1.32.2+k3s1
|
k3s_version: v1.32.4+k3s1
|
||||||
k3s_server_url: https://192.168.7.77:6443
|
k3s_server_url: https://192.168.7.77:6443
|
||||||
k3s_node_token: ""
|
k3s_node_token: ""
|
||||||
|
|||||||
@ -3,7 +3,7 @@
|
|||||||
# Called by: ansible/playbooks/setup-k3s.yaml
|
# Called by: ansible/playbooks/setup-k3s.yaml
|
||||||
# Description: Default variables for the k3s-server role including version, IP, and server configuration.
|
# Description: Default variables for the k3s-server role including version, IP, and server configuration.
|
||||||
|
|
||||||
k3s_version: v1.32.2+k3s1
|
k3s_version: v1.32.4+k3s1
|
||||||
k3s_server_ip: 192.168.7.77
|
k3s_server_ip: 192.168.7.77
|
||||||
|
|
||||||
k3s_server_config:
|
k3s_server_config:
|
||||||
|
|||||||
@ -46,6 +46,7 @@
|
|||||||
- name: Print node token
|
- name: Print node token
|
||||||
ansible.builtin.debug:
|
ansible.builtin.debug:
|
||||||
msg: "K3s node token: {{ k3s_node_token }}"
|
msg: "K3s node token: {{ k3s_node_token }}"
|
||||||
|
when: k3s_show_token | default(false)
|
||||||
|
|
||||||
- name: Fetch kubeconfig to workstation
|
- name: Fetch kubeconfig to workstation
|
||||||
ansible.builtin.fetch:
|
ansible.builtin.fetch:
|
||||||
@ -55,7 +56,7 @@
|
|||||||
|
|
||||||
- name: Fix kubeconfig server address
|
- name: Fix kubeconfig server address
|
||||||
ansible.builtin.replace:
|
ansible.builtin.replace:
|
||||||
path: /tmp/k3s-minisforum.yaml
|
path: "{{ lookup('env', 'HOME') }}/.kube/config"
|
||||||
regexp: 'https://127\.0\.0\.1:6443'
|
regexp: 'https://127\.0\.0\.1:6443'
|
||||||
replace: "https://{{ k3s_server_ip }}:6443"
|
replace: "https://{{ k3s_server_ip }}:6443"
|
||||||
delegate_to: localhost
|
delegate_to: localhost
|
||||||
@ -67,6 +68,6 @@
|
|||||||
creates: /usr/local/bin/helm
|
creates: /usr/local/bin/helm
|
||||||
|
|
||||||
- name: Label server node as primary
|
- name: Label server node as primary
|
||||||
ansible.builtin.shell:
|
ansible.builtin.command:
|
||||||
cmd: k3s kubectl label node minisforum node-role=primary --overwrite
|
cmd: k3s kubectl label node minisforum node-role=primary --overwrite
|
||||||
changed_when: false
|
changed_when: false
|
||||||
|
|||||||
@ -2,7 +2,7 @@
|
|||||||
# Part of role: nvidia
|
# Part of role: nvidia
|
||||||
# Called by: ansible/playbooks/setup-gpu-node.yaml
|
# Called by: ansible/playbooks/setup-gpu-node.yaml
|
||||||
# Description: Restarts Docker after nvidia-container-toolkit runtime configuration.
|
# Description: Restarts Docker after nvidia-container-toolkit runtime configuration.
|
||||||
- name: restart docker
|
- name: Restart docker
|
||||||
ansible.builtin.systemd:
|
ansible.builtin.systemd:
|
||||||
name: docker
|
name: docker
|
||||||
state: restarted
|
state: restarted
|
||||||
|
|||||||
@ -55,11 +55,19 @@
|
|||||||
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
|
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
|
||||||
mode: "0644"
|
mode: "0644"
|
||||||
|
|
||||||
|
- name: Check if Docker already has the NVIDIA runtime configured
|
||||||
|
ansible.builtin.command:
|
||||||
|
cmd: grep -q '"nvidia"' /etc/docker/daemon.json
|
||||||
|
register: nvidia_docker_runtime_check
|
||||||
|
changed_when: false
|
||||||
|
failed_when: false
|
||||||
|
|
||||||
- name: Configure Docker runtime for NVIDIA
|
- name: Configure Docker runtime for NVIDIA
|
||||||
ansible.builtin.command:
|
ansible.builtin.command:
|
||||||
cmd: nvidia-ctk runtime configure --runtime=docker
|
cmd: nvidia-ctk runtime configure --runtime=docker
|
||||||
|
when: nvidia_docker_runtime_check.rc != 0
|
||||||
changed_when: true
|
changed_when: true
|
||||||
notify: restart docker
|
notify: Restart docker
|
||||||
|
|
||||||
- name: Reboot if driver was just installed
|
- name: Reboot if driver was just installed
|
||||||
ansible.builtin.reboot:
|
ansible.builtin.reboot:
|
||||||
|
|||||||
@ -3,12 +3,13 @@
|
|||||||
# Called by: ansible/playbooks/setup-ollama.yaml
|
# Called by: ansible/playbooks/setup-ollama.yaml
|
||||||
# ansible/playbooks/setup-gpu-node.yaml
|
# ansible/playbooks/setup-gpu-node.yaml
|
||||||
# Description: Handlers for the ollama role. Restarts ollama on config changes.
|
# Description: Handlers for the ollama role. Restarts ollama on config changes.
|
||||||
- name: restart ollama
|
- name: Restart ollama
|
||||||
become: true
|
become: true
|
||||||
command: launchctl kickstart -k system/com.ollama.ollama
|
ansible.builtin.command: launchctl kickstart -k system/com.ollama.ollama
|
||||||
|
changed_when: true
|
||||||
when: ansible_facts['system'] == 'Darwin'
|
when: ansible_facts['system'] == 'Darwin'
|
||||||
|
|
||||||
- name: restart ollama linux
|
- name: Restart ollama linux
|
||||||
ansible.builtin.systemd:
|
ansible.builtin.systemd:
|
||||||
name: ollama
|
name: ollama
|
||||||
state: restarted
|
state: restarted
|
||||||
|
|||||||
@ -13,19 +13,19 @@
|
|||||||
when: ansible_facts['system'] == 'Darwin'
|
when: ansible_facts['system'] == 'Darwin'
|
||||||
|
|
||||||
- name: Deploy ollama launchd plist
|
- name: Deploy ollama launchd plist
|
||||||
template:
|
ansible.builtin.template:
|
||||||
src: ollama.plist.j2
|
src: ollama.plist.j2
|
||||||
dest: /Library/LaunchDaemons/com.ollama.ollama.plist
|
dest: /Library/LaunchDaemons/com.ollama.ollama.plist
|
||||||
owner: root
|
owner: root
|
||||||
group: wheel
|
group: wheel
|
||||||
mode: "0644"
|
mode: "0644"
|
||||||
become: true
|
become: true
|
||||||
notify: restart ollama
|
notify: Restart ollama
|
||||||
when: ansible_facts['system'] == 'Darwin'
|
when: ansible_facts['system'] == 'Darwin'
|
||||||
|
|
||||||
- name: Load ollama launchd service
|
- name: Load ollama launchd service
|
||||||
become: true
|
become: true
|
||||||
command: launchctl load -w /Library/LaunchDaemons/com.ollama.ollama.plist
|
ansible.builtin.command: launchctl load -w /Library/LaunchDaemons/com.ollama.ollama.plist
|
||||||
args:
|
args:
|
||||||
creates: /var/run/ollama.pid
|
creates: /var/run/ollama.pid
|
||||||
ignore_errors: true
|
ignore_errors: true
|
||||||
@ -54,7 +54,7 @@
|
|||||||
group: root
|
group: root
|
||||||
mode: "0644"
|
mode: "0644"
|
||||||
become: true
|
become: true
|
||||||
notify: restart ollama linux
|
notify: Restart ollama linux
|
||||||
when: ansible_facts['system'] == 'Linux'
|
when: ansible_facts['system'] == 'Linux'
|
||||||
|
|
||||||
- name: Enable and start ollama service
|
- name: Enable and start ollama service
|
||||||
@ -68,7 +68,7 @@
|
|||||||
|
|
||||||
# ── shared ─────────────────────────────────────────────────────────────────────
|
# ── shared ─────────────────────────────────────────────────────────────────────
|
||||||
- name: Wait for ollama to be ready
|
- name: Wait for ollama to be ready
|
||||||
uri:
|
ansible.builtin.uri:
|
||||||
url: "http://localhost:{{ ollama_port }}"
|
url: "http://localhost:{{ ollama_port }}"
|
||||||
status_code: 200
|
status_code: 200
|
||||||
register: result
|
register: result
|
||||||
@ -77,13 +77,13 @@
|
|||||||
delay: 3
|
delay: 3
|
||||||
|
|
||||||
- name: Check installed ollama models
|
- name: Check installed ollama models
|
||||||
uri:
|
ansible.builtin.uri:
|
||||||
url: "http://localhost:{{ ollama_port }}/api/tags"
|
url: "http://localhost:{{ ollama_port }}/api/tags"
|
||||||
return_content: true
|
return_content: true
|
||||||
register: ollama_tags
|
register: ollama_tags
|
||||||
|
|
||||||
- name: Pull ollama models
|
- name: Pull ollama models
|
||||||
command: >
|
ansible.builtin.command: >
|
||||||
{{ '/opt/homebrew/bin/ollama' if ansible_facts['system'] == 'Darwin' else '/usr/local/bin/ollama' }}
|
{{ '/opt/homebrew/bin/ollama' if ansible_facts['system'] == 'Darwin' else '/usr/local/bin/ollama' }}
|
||||||
pull {{ item }}
|
pull {{ item }}
|
||||||
loop: "{{ ollama_models }}"
|
loop: "{{ ollama_models }}"
|
||||||
|
|||||||
@ -8,8 +8,8 @@
|
|||||||
repo: "{{ watch_party_repo }}"
|
repo: "{{ watch_party_repo }}"
|
||||||
dest: "{{ watch_party_dir }}"
|
dest: "{{ watch_party_dir }}"
|
||||||
version: main
|
version: main
|
||||||
update: yes
|
update: true
|
||||||
accept_hostkey: yes
|
accept_hostkey: true
|
||||||
environment:
|
environment:
|
||||||
GIT_SSL_NO_VERIFY: "true"
|
GIT_SSL_NO_VERIFY: "true"
|
||||||
|
|
||||||
|
|||||||
@ -1,5 +1,5 @@
|
|||||||
---
|
---
|
||||||
- name: Restart wg0
|
- name: Restart wg0
|
||||||
systemd:
|
ansible.builtin.systemd:
|
||||||
name: wg-quick@wg0
|
name: wg-quick@wg0
|
||||||
state: restarted
|
state: restarted
|
||||||
@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
- name: Install WireGuard and tools
|
- name: Install WireGuard and tools
|
||||||
apt:
|
ansible.builtin.apt:
|
||||||
name:
|
name:
|
||||||
- wireguard
|
- wireguard
|
||||||
- wireguard-tools
|
- wireguard-tools
|
||||||
@ -9,13 +9,13 @@
|
|||||||
update_cache: true
|
update_cache: true
|
||||||
|
|
||||||
- name: Allow WireGuard port through UFW
|
- name: Allow WireGuard port through UFW
|
||||||
ufw:
|
community.general.ufw:
|
||||||
rule: allow
|
rule: allow
|
||||||
port: "51820"
|
port: "51820"
|
||||||
proto: udp
|
proto: udp
|
||||||
|
|
||||||
- name: Enable IP forwarding
|
- name: Enable IP forwarding
|
||||||
sysctl:
|
ansible.posix.sysctl:
|
||||||
name: net.ipv4.ip_forward
|
name: net.ipv4.ip_forward
|
||||||
value: "1"
|
value: "1"
|
||||||
sysctl_set: true
|
sysctl_set: true
|
||||||
@ -23,7 +23,7 @@
|
|||||||
reload: true
|
reload: true
|
||||||
|
|
||||||
- name: Create WireGuard config directory
|
- name: Create WireGuard config directory
|
||||||
file:
|
ansible.builtin.file:
|
||||||
path: /etc/wireguard
|
path: /etc/wireguard
|
||||||
state: directory
|
state: directory
|
||||||
mode: "0700"
|
mode: "0700"
|
||||||
@ -32,88 +32,88 @@
|
|||||||
|
|
||||||
# --- Server keypair ---
|
# --- Server keypair ---
|
||||||
- name: Check if server private key exists
|
- name: Check if server private key exists
|
||||||
stat:
|
ansible.builtin.stat:
|
||||||
path: /etc/wireguard/server.key
|
path: /etc/wireguard/server.key
|
||||||
register: server_key_stat
|
register: server_key_stat
|
||||||
|
|
||||||
- name: Generate server private key
|
- name: Generate server private key
|
||||||
shell: wg genkey > /etc/wireguard/server.key
|
ansible.builtin.shell: wg genkey > /etc/wireguard/server.key
|
||||||
when: not server_key_stat.stat.exists
|
when: not server_key_stat.stat.exists
|
||||||
|
|
||||||
- name: Set permissions on server private key
|
- name: Set permissions on server private key
|
||||||
file:
|
ansible.builtin.file:
|
||||||
path: /etc/wireguard/server.key
|
path: /etc/wireguard/server.key
|
||||||
mode: "0600"
|
mode: "0600"
|
||||||
owner: root
|
owner: root
|
||||||
group: root
|
group: root
|
||||||
|
|
||||||
- name: Read server private key
|
- name: Read server private key
|
||||||
slurp:
|
ansible.builtin.slurp:
|
||||||
src: /etc/wireguard/server.key
|
src: /etc/wireguard/server.key
|
||||||
register: server_private_key
|
register: server_private_key
|
||||||
|
|
||||||
- name: Derive server public key
|
- name: Derive server public key
|
||||||
shell: wg pubkey < /etc/wireguard/server.key
|
ansible.builtin.shell: wg pubkey < /etc/wireguard/server.key
|
||||||
register: server_public_key
|
register: server_public_key
|
||||||
changed_when: false
|
changed_when: false
|
||||||
|
|
||||||
# --- Phone keypair ---
|
# --- Phone keypair ---
|
||||||
- name: Check if phone private key exists
|
- name: Check if phone private key exists
|
||||||
stat:
|
ansible.builtin.stat:
|
||||||
path: /etc/wireguard/phone.key
|
path: /etc/wireguard/phone.key
|
||||||
register: phone_key_stat
|
register: phone_key_stat
|
||||||
|
|
||||||
- name: Generate phone private key
|
- name: Generate phone private key
|
||||||
shell: wg genkey > /etc/wireguard/phone.key
|
ansible.builtin.shell: wg genkey > /etc/wireguard/phone.key
|
||||||
when: not phone_key_stat.stat.exists
|
when: not phone_key_stat.stat.exists
|
||||||
|
|
||||||
- name: Set permissions on phone private key
|
- name: Set permissions on phone private key
|
||||||
file:
|
ansible.builtin.file:
|
||||||
path: /etc/wireguard/phone.key
|
path: /etc/wireguard/phone.key
|
||||||
mode: "0600"
|
mode: "0600"
|
||||||
owner: root
|
owner: root
|
||||||
group: root
|
group: root
|
||||||
|
|
||||||
- name: Read phone private key
|
- name: Read phone private key
|
||||||
slurp:
|
ansible.builtin.slurp:
|
||||||
src: /etc/wireguard/phone.key
|
src: /etc/wireguard/phone.key
|
||||||
register: phone_private_key
|
register: phone_private_key
|
||||||
|
|
||||||
- name: Derive phone public key
|
- name: Derive phone public key
|
||||||
shell: wg pubkey < /etc/wireguard/phone.key
|
ansible.builtin.shell: wg pubkey < /etc/wireguard/phone.key
|
||||||
register: phone_public_key
|
register: phone_public_key
|
||||||
changed_when: false
|
changed_when: false
|
||||||
|
|
||||||
# --- Mac keypair ---
|
# --- Mac keypair ---
|
||||||
- name: Check if mac private key exists
|
- name: Check if mac private key exists
|
||||||
stat:
|
ansible.builtin.stat:
|
||||||
path: /etc/wireguard/mac.key
|
path: /etc/wireguard/mac.key
|
||||||
register: mac_key_stat
|
register: mac_key_stat
|
||||||
|
|
||||||
- name: Generate mac private key
|
- name: Generate mac private key
|
||||||
shell: wg genkey > /etc/wireguard/mac.key
|
ansible.builtin.shell: wg genkey > /etc/wireguard/mac.key
|
||||||
when: not mac_key_stat.stat.exists
|
when: not mac_key_stat.stat.exists
|
||||||
|
|
||||||
- name: Set permissions on mac private key
|
- name: Set permissions on mac private key
|
||||||
file:
|
ansible.builtin.file:
|
||||||
path: /etc/wireguard/mac.key
|
path: /etc/wireguard/mac.key
|
||||||
mode: "0600"
|
mode: "0600"
|
||||||
owner: root
|
owner: root
|
||||||
group: root
|
group: root
|
||||||
|
|
||||||
- name: Read mac private key
|
- name: Read mac private key
|
||||||
slurp:
|
ansible.builtin.slurp:
|
||||||
src: /etc/wireguard/mac.key
|
src: /etc/wireguard/mac.key
|
||||||
register: mac_private_key
|
register: mac_private_key
|
||||||
|
|
||||||
- name: Derive mac public key
|
- name: Derive mac public key
|
||||||
shell: wg pubkey < /etc/wireguard/mac.key
|
ansible.builtin.shell: wg pubkey < /etc/wireguard/mac.key
|
||||||
register: mac_public_key
|
register: mac_public_key
|
||||||
changed_when: false
|
changed_when: false
|
||||||
|
|
||||||
# --- Server config ---
|
# --- Server config ---
|
||||||
- name: Write wg0.conf
|
- name: Write wg0.conf
|
||||||
template:
|
ansible.builtin.template:
|
||||||
src: wg0.conf.j2
|
src: wg0.conf.j2
|
||||||
dest: /etc/wireguard/wg0.conf
|
dest: /etc/wireguard/wg0.conf
|
||||||
mode: "0600"
|
mode: "0600"
|
||||||
@ -123,14 +123,14 @@
|
|||||||
|
|
||||||
# --- Service ---
|
# --- Service ---
|
||||||
- name: Enable and start wg-quick@wg0
|
- name: Enable and start wg-quick@wg0
|
||||||
systemd:
|
ansible.builtin.systemd:
|
||||||
name: wg-quick@wg0
|
name: wg-quick@wg0
|
||||||
enabled: true
|
enabled: true
|
||||||
state: started
|
state: started
|
||||||
|
|
||||||
# --- Phone client config + QR ---
|
# --- Phone client config + QR ---
|
||||||
- name: Write phone client config
|
- name: Write phone client config
|
||||||
copy:
|
ansible.builtin.copy:
|
||||||
dest: /etc/wireguard/phone-client.conf
|
dest: /etc/wireguard/phone-client.conf
|
||||||
mode: "0600"
|
mode: "0600"
|
||||||
owner: root
|
owner: root
|
||||||
@ -149,7 +149,7 @@
|
|||||||
|
|
||||||
# --- Mac client config ---
|
# --- Mac client config ---
|
||||||
- name: Write mac client config
|
- name: Write mac client config
|
||||||
copy:
|
ansible.builtin.copy:
|
||||||
dest: /etc/wireguard/mac-client.conf
|
dest: /etc/wireguard/mac-client.conf
|
||||||
mode: "0600"
|
mode: "0600"
|
||||||
owner: root
|
owner: root
|
||||||
@ -167,19 +167,23 @@
|
|||||||
PersistentKeepalive = 25
|
PersistentKeepalive = 25
|
||||||
|
|
||||||
- name: Display mac client config
|
- name: Display mac client config
|
||||||
shell: cat /etc/wireguard/mac-client.conf
|
ansible.builtin.shell: cat /etc/wireguard/mac-client.conf
|
||||||
register: mac_conf
|
register: mac_conf
|
||||||
changed_when: false
|
changed_when: false
|
||||||
|
when: wireguard_show_client_configs | default(false)
|
||||||
|
|
||||||
- name: Show mac client config
|
- name: Show mac client config
|
||||||
debug:
|
ansible.builtin.debug:
|
||||||
msg: "{{ mac_conf.stdout_lines }}"
|
msg: "{{ mac_conf.stdout_lines }}"
|
||||||
|
when: wireguard_show_client_configs | default(false)
|
||||||
|
|
||||||
- name: Generate QR code for phone
|
- name: Generate QR code for phone
|
||||||
shell: qrencode -t ansiutf8 < /etc/wireguard/phone-client.conf
|
ansible.builtin.shell: qrencode -t ansiutf8 < /etc/wireguard/phone-client.conf
|
||||||
register: phone_qr
|
register: phone_qr
|
||||||
changed_when: false
|
changed_when: false
|
||||||
|
when: wireguard_show_client_configs | default(false)
|
||||||
|
|
||||||
- name: Display phone QR code
|
- name: Display phone QR code
|
||||||
debug:
|
ansible.builtin.debug:
|
||||||
msg: "{{ phone_qr.stdout_lines }}"
|
msg: "{{ phone_qr.stdout_lines }}"
|
||||||
|
when: wireguard_show_client_configs | default(false)
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user