"""ButtonTask demo server — voice classify + press log + cancel.""" from __future__ import annotations import asyncio import base64 import json import logging import os import re import shutil import subprocess import tempfile import uuid from collections import deque from datetime import datetime, timezone from pathlib import Path from typing import Any, Deque, Dict, List, Optional import httpx from fastapi import FastAPI, File, Form, Header, HTTPException, Request, UploadFile from fastapi.responses import HTMLResponse, JSONResponse, PlainTextResponse from fastapi.staticfiles import StaticFiles from fastapi.templating import Jinja2Templates from pydantic_settings import BaseSettings, SettingsConfigDict logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s") log = logging.getLogger("demo-server") UPLOAD_DIR = Path(os.environ.get("UPLOAD_DIR", "/tmp/buttontask-uploads")) UPLOAD_DIR.mkdir(parents=True, exist_ok=True) DATA_DIR = Path(os.environ.get("DATA_DIR", "/tmp/buttontask-data")) DATA_DIR.mkdir(parents=True, exist_ok=True) CATEGORIES_FILE = DATA_DIR / "categories.json" DEFAULT_CATEGORIES = [ "Клининг", "АХО", "Охрана", "Канцелярия", "Техническая служба", ] SILENCE_MARKERS = ( "", "(тишина)", "тишина", "...", ".", "-", "silence", "[silence]", "(silence)", ) def _load_categories() -> List[str]: if CATEGORIES_FILE.is_file(): try: data = json.loads(CATEGORIES_FILE.read_text(encoding="utf-8")) cats = [str(c).strip() for c in (data.get("categories") or []) if str(c).strip()] if cats: return cats except (OSError, json.JSONDecodeError, TypeError): pass return list(DEFAULT_CATEGORIES) def _save_categories(cats: List[str]) -> List[str]: cleaned = [] seen = set() for c in cats: name = str(c).strip() if not name: continue key = name.lower() if key in seen: continue seen.add(key) cleaned.append(name) if not cleaned: cleaned = list(DEFAULT_CATEGORIES) CATEGORIES_FILE.write_text( json.dumps({"categories": cleaned}, ensure_ascii=False, indent=2) + "\n", encoding="utf-8", ) return cleaned def _classify_system_prompt(cats: List[str]) -> str: listed = ", ".join(cats) return f"""Ты классификатор заявок с голосовой кнопки в здании. По тексту распознанной речи выбери ОДНУ категорию строго из списка: {listed} Учти, что текст приходит из STT и может содержать опечатки/созвучия (например «скрипки»/«скрытки» вместо «скрепки», «клиненг» вместо «клининг»). Восстанавливай смысл по контексту офисной заявки. Если речь пустая, неразборчивая, это шум/тишина, или ни одна категория не подходит — ответь ровно: UNMATCHED Примеры: - "пролили кофе", "уберите мусор" → Клининг (если есть в списке) - "закончились скрепки", "принесите скрепки", "кончилась бумага" → Канцелярия / АХО (если есть) - "подозрительный человек" → Охрана (если есть) Ответь ТОЛЬКО названием категории из списка или UNMATCHED, без пояснений.""" # Whisper often mishears short office words on noisy mics. _TRANSCRIPT_FIXES = ( (re.compile(r"\bскрипки\b", re.I), "скрепки"), (re.compile(r"\bскрытки\b", re.I), "скрепки"), (re.compile(r"\bскритки\b", re.I), "скрепки"), (re.compile(r"\bскрепке\b", re.I), "скрепки"), (re.compile(r"\bклиненг\b", re.I), "клининг"), (re.compile(r"\bклинингг\b", re.I), "клининг"), ) def _fix_transcript(text: str) -> str: out = (text or "").strip() for rx, repl in _TRANSCRIPT_FIXES: out = rx.sub(repl, out) return out.strip() # Bias Whisper toward office Russian vocabulary (reduces «скрипки» etc.). STT_PROMPT = ( "Голосовая заявка с панели в здании на русском. " "Возможные темы: клининг, уборка, скрепки, бумага, канцелярия, АХО, охрана, " "техническая служба, принесите, закончились, пролили, уберите." ) class Settings(BaseSettings): model_config = SettingsConfigDict(env_file=".env", case_sensitive=False, extra="ignore") api_key: str = "97098109-3188-496b-b075-0e8d83c2bef5" openrouter_api_key: str = "" openrouter_base_url: str = "https://openrouter.ai/api/v1" # gpt-4o-mini-transcribe usually beats vanilla Whisper on noisy short RU clips stt_model: str = "openai/gpt-4o-mini-transcribe" classify_model: str = "openai/gpt-4o-mini" socks5_proxy: str = "" mock_ai: bool = False host: str = "0.0.0.0" port: int = 5000 # Profile denoise via post-roll silence tail. Off by default — raw audio STT better here. denoise: bool = False denoise_nf: float = -18.0 denoise_prop: float = 0.75 settings = Settings() # In-memory event log and active voice calls _events: Deque[Dict[str, Any]] = deque(maxlen=200) _active: Dict[str, Dict[str, Any]] = {} # request_id -> call _lock = asyncio.Lock() _categories: List[str] = _load_categories() app = FastAPI(title="ButtonTask Demo Server", version="1.0.0") templates = Jinja2Templates(directory=str(Path(__file__).resolve().parent.parent / "templates")) static_dir = Path(__file__).resolve().parent.parent / "static" if static_dir.is_dir(): app.mount("/static", StaticFiles(directory=str(static_dir)), name="static") app.mount("/media", StaticFiles(directory=str(UPLOAD_DIR)), name="media") def _now_iso() -> str: return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") def _audio_url(filename: str) -> str: return f"/media/{filename}" def _push_event(kind: str, **payload: Any) -> Dict[str, Any]: ev = {"id": str(uuid.uuid4()), "ts": _now_iso(), "kind": kind, **payload} _events.appendleft(ev) log.info("event %s %s", kind, {k: v for k, v in payload.items() if k not in ("headers",)}) return ev def _is_silence(text: str) -> bool: t = (text or "").strip().lower() if t in SILENCE_MARKERS: return True # very short / non-letter noise letters = re.sub(r"[^\wа-яё]", "", t, flags=re.I) return len(letters) < 2 def _normalize_category(raw: str, cats: List[str]) -> Optional[str]: cleaned = (raw or "").strip().strip('"').strip("'") if not cleaned or cleaned.upper() == "UNMATCHED": return None for cat in cats: if cat.lower() == cleaned.lower(): return cat for cat in cats: if cat.lower() in cleaned.lower() or cleaned.lower() in cat.lower(): return cat return None def _mock_transcript(path: Path) -> str: name = path.name.lower() if "silent" in name or "тишин" in name: return "" if "coffee" in name or "кофе" in name: return "пролили кофе" if "paper" in name or "бумаг" in name: return "кончилась бумага" return "пролили кофе в коридоре" def _mock_classify(text: str, cats: List[str]) -> Optional[str]: if _is_silence(text): return None low = (text or "").lower() rules = [ (("кофе", "мусор", "гряз", "убор", "пролил"), "Клининг"), (("бумаг", "ручк", "картридж", "канцеляр"), "Канцелярия"), (("охран", "дверь", "подозрит"), "Охрана"), (("розетк", "кран", "сломан", "не работает"), "Техническая служба"), (("ахо", "хозяйств"), "АХО"), ] for words, cat in rules: if any(w in low for w in words) and cat in cats: return cat # first category containing substring match from text tokens — else unmatched return None def _extract_api_key(request: Request, header_key: Optional[str] = None) -> Optional[str]: """Prefer X-API-Key header; fall back to Authorization / query (debug).""" if header_key: return header_key.strip() or None # Case-insensitive header lookup (proxies / Qt variants) for name, val in request.headers.items(): if name.lower() == "x-api-key" and val: return val.strip() if name.lower() == "authorization" and val.lower().startswith("bearer "): return val[7:].strip() q = request.query_params.get("api_key") return q.strip() if q else None def _check_api_key(x_api_key: Optional[str]) -> None: expected = (settings.api_key or "").strip() got = (x_api_key or "").strip() if not got or got != expected: log.warning( "API auth failed: header_present=%s got_len=%s expected_len=%s got_prefix=%r", bool(got), len(got), len(expected), (got[:12] + "…") if got else "", ) raise HTTPException(status_code=401, detail="Invalid or missing X-API-Key") def _httpx_client() -> httpx.AsyncClient: kwargs: Dict[str, Any] = {"timeout": 120.0} proxy = (settings.socks5_proxy or "").strip() if proxy: kwargs["proxy"] = proxy return httpx.AsyncClient(**kwargs) def _write_wav_mono_s16(path: Path, audio: Any, rate: int) -> None: import wave import numpy as np pcm = np.clip(audio, -32768, 32767).astype(np.int16) with wave.open(str(path), "wb") as out: out.setnchannels(1) out.setsampwidth(2) out.setframerate(rate) out.writeframes(pcm.tobytes()) def _ffmpeg_to_mono16k(src: Path, dst: Path) -> bool: ffmpeg = shutil.which("ffmpeg") if not ffmpeg: return False # Light band-limit only — no blind denoise/gain (that hurt clarity). af = "highpass=f=80,lowpass=f=7500" try: subprocess.run( [ ffmpeg, "-y", "-i", str(src), "-af", af, "-ac", "1", "-ar", "16000", "-sample_fmt", "s16", str(dst), ], check=True, capture_output=True, timeout=90, ) return dst.is_file() and dst.stat().st_size > 44 except Exception as exc: log.warning("ffmpeg convert failed: %s", exc) return False def _preprocess_audio(src: Path, noise_tail_ms: int = 0) -> Path: """Convert to 16 kHz mono. Optional: trim/use silence tail for profile denoise.""" mid = src.with_name(f"{src.stem}_ff.wav") dst = src.with_name(f"{src.stem}_clean.wav") if not _ffmpeg_to_mono16k(src, mid): if src.suffix.lower() == ".wav": mid = src else: return src tail_ms = max(0, int(noise_tail_ms or 0)) # Default path: convert only (best STT on this hardware). Optionally trim tail. if not settings.denoise: try: import wave import numpy as np if tail_ms > 0: with wave.open(str(mid), "rb") as w: if w.getsampwidth() == 2: rate = w.getframerate() nch = w.getnchannels() frames = w.readframes(w.getnframes()) audio = np.frombuffer(frames, dtype=np.int16).astype(np.float32) if nch > 1: audio = audio.reshape(-1, nch).mean(axis=1) cut = int(rate * (tail_ms / 1000.0)) if cut > 0 and audio.size > cut + int(rate * 0.2): _write_wav_mono_s16(dst, audio[:-cut], rate) if mid != src: try: mid.unlink(missing_ok=True) except OSError: pass log.info("audio preprocess: convert+trim tail=%sms → %s", tail_ms, dst.name) return dst except Exception as exc: log.warning("tail trim failed: %s", exc) if mid != dst: try: if mid != src: mid.replace(dst) else: import shutil as _sh _sh.copy2(mid, dst) except OSError: return mid log.info("audio preprocess: convert only → %s", dst.name) return dst # denoise=true: subtract noise profile from post-roll tail, then trim try: import numpy as np import noisereduce as nr import wave with wave.open(str(mid), "rb") as w: if w.getsampwidth() != 2: mid.replace(dst) return dst rate = w.getframerate() nch = w.getnchannels() frames = w.readframes(w.getnframes()) audio = np.frombuffer(frames, dtype=np.int16).astype(np.float32) if nch > 1: audio = audio.reshape(-1, nch).mean(axis=1) tail_samples = int(rate * (tail_ms / 1000.0)) min_speech = int(rate * 0.2) if tail_samples < int(rate * 0.15) or audio.size < (tail_samples + min_speech): log.info("audio preprocess: no usable tail, convert only") mid.replace(dst) return dst speech = audio[:-tail_samples] noise = audio[-tail_samples:] prop = max(0.0, min(0.95, float(settings.denoise_prop))) reduced = nr.reduce_noise( y=speech, sr=rate, y_noise=noise, stationary=True, prop_decrease=prop, ) peak = float(np.max(np.abs(reduced))) if reduced.size else 0.0 if peak > 200: gain = min(6.0, (0.7 * 32767.0) / peak) reduced = reduced * gain _write_wav_mono_s16(dst, reduced, rate) try: if mid != src: mid.unlink(missing_ok=True) except OSError: pass log.info( "audio preprocess: profile denoise ok → %s (tail=%sms prop=%s)", dst.name, tail_ms, prop, ) return dst if dst.is_file() else src except Exception as exc: log.warning("profile denoise failed: %s", exc) try: if mid.exists(): mid.replace(dst) return dst except OSError: pass return mid if mid.exists() else src def _to_wav(src: Path) -> Path: """Ensure mono 16-bit PCM WAV (already cleaned if *_clean.wav).""" if src.name.endswith("_clean.wav"): return src return _preprocess_audio(src, noise_tail_ms=0) async def _stt_openrouter(audio_path: Path) -> str: if settings.mock_ai or not settings.openrouter_api_key: return _fix_transcript(_mock_transcript(audio_path)) wav = await asyncio.to_thread(_to_wav, audio_path) url = f"{settings.openrouter_base_url.rstrip('/')}/audio/transcriptions" headers = { "Authorization": f"Bearer {settings.openrouter_api_key}", "HTTP-Referer": "https://buttontask.local", "X-Title": "ButtonTask Demo", } async with _httpx_client() as client: with wav.open("rb") as f: files = {"file": (wav.name, f, "audio/wav")} data = { "model": settings.stt_model, "language": "ru", "prompt": STT_PROMPT, "temperature": "0", } resp = await client.post(url, headers=headers, files=files, data=data) if resp.status_code >= 400: # Fallback: chat completions with base64 audio (some OpenRouter models) text = await _stt_via_chat(wav) if text: return _fix_transcript(text) raise HTTPException(status_code=502, detail=f"STT failed: {resp.status_code} {resp.text[:300]}") payload = resp.json() return _fix_transcript((payload.get("text") or "").strip()) async def _stt_via_chat(audio_path: Path) -> str: """Fallback STT through chat completions with input_audio.""" raw = audio_path.read_bytes() b64 = base64.b64encode(raw).decode("ascii") fmt = "wav" if audio_path.suffix.lower() == ".wav" else audio_path.suffix.lstrip(".") or "wav" url = f"{settings.openrouter_base_url.rstrip('/')}/chat/completions" headers = { "Authorization": f"Bearer {settings.openrouter_api_key}", "Content-Type": "application/json", "HTTP-Referer": "https://buttontask.local", "X-Title": "ButtonTask Demo", } body = { "model": settings.stt_model, "messages": [ { "role": "user", "content": [ {"type": "text", "text": "Transcribe this audio to Russian text. Reply with only the transcript."}, {"type": "input_audio", "input_audio": {"data": b64, "format": fmt}}, ], } ], } try: async with _httpx_client() as client: resp = await client.post(url, headers=headers, json=body) if resp.status_code >= 400: log.warning("chat STT failed: %s %s", resp.status_code, resp.text[:200]) return "" data = resp.json() return (data["choices"][0]["message"]["content"] or "").strip() except Exception as exc: log.warning("chat STT exception: %s", exc) return "" async def _classify_openrouter(text: str) -> Optional[str]: cats = list(_categories) if settings.mock_ai or not settings.openrouter_api_key: return _mock_classify(text, cats) url = f"{settings.openrouter_base_url.rstrip('/')}/chat/completions" headers = { "Authorization": f"Bearer {settings.openrouter_api_key}", "Content-Type": "application/json", "HTTP-Referer": "https://buttontask.local", "X-Title": "ButtonTask Demo", } body = { "model": settings.classify_model, "temperature": 0, "messages": [ {"role": "system", "content": _classify_system_prompt(cats)}, {"role": "user", "content": text or "(тишина)"}, ], } async with _httpx_client() as client: resp = await client.post(url, headers=headers, json=body) if resp.status_code >= 400: raise HTTPException(status_code=502, detail=f"Classify failed: {resp.status_code} {resp.text[:300]}") data = resp.json() raw = (data["choices"][0]["message"]["content"] or "").strip() return _normalize_category(raw, cats) # ── Routes ────────────────────────────────────────────────────────────── @app.get("/", response_class=HTMLResponse) async def index(request: Request): return templates.TemplateResponse( "index.html", { "request": request, "events": list(_events)[:50], "active": list(_active.values()), "categories": list(_categories), "mock_ai": settings.mock_ai or not settings.openrouter_api_key, }, ) @app.get("/api/events") async def api_events(limit: int = 50): return { "events": list(_events)[: max(1, min(limit, 200))], "active": list(_active.values()), "categories": list(_categories), } @app.get("/api/categories") async def get_categories(): return {"categories": list(_categories)} @app.put("/api/categories") async def put_categories(request: Request): data = await request.json() cats = data.get("categories") if not isinstance(cats, list): raise HTTPException(status_code=400, detail="categories must be an array of strings") global _categories _categories = _save_categories(cats) _push_event("categories_updated", categories=list(_categories)) return {"ok": True, "categories": list(_categories)} @app.post("/api/v1/classify-audio") async def classify_audio( request: Request, audio: UploadFile = File(...), source_id: str = Form(...), noise_tail_ms: int = Form(0), x_api_key: Optional[str] = Header(default=None, alias="X-API-Key"), ): _check_api_key(_extract_api_key(request, x_api_key)) if not audio.filename and not audio.content_type: raise HTTPException(status_code=400, detail="No audio file provided") request_id = str(uuid.uuid4()) suffix = Path(audio.filename or "audio.wav").suffix or ".wav" if suffix.lower() not in (".wav", ".webm", ".ogg", ".mp3", ".m4a", ".flac", ".pcm", ".raw"): suffix = ".wav" audio_name = f"{request_id}{suffix}" dest = UPLOAD_DIR / audio_name content = await audio.read() if not content: raise HTTPException(status_code=400, detail="Empty audio file") dest.write_bytes(content) clean = await asyncio.to_thread(_preprocess_audio, dest, int(noise_tail_ms or 0)) audio_url = _audio_url(clean.name) try: transcript = await _stt_openrouter(clean) if _is_silence(transcript): category = None reason = "silence" else: category = await _classify_openrouter(transcript) reason = "ok" if category else "unmatched" except HTTPException: _push_event( "voice_error", source_id=source_id, request_id=request_id, error="STT/classify failed", ip=request.client.host if request.client else "", audio_url=audio_url, ) raise except Exception as exc: log.exception("classify failed") _push_event( "voice_error", source_id=source_id, request_id=request_id, error=str(exc), ip=request.client.host if request.client else "", audio_url=audio_url, ) return JSONResponse( status_code=500, content={"status": "error", "message": str(exc)}, ) ts = _now_iso() ip = request.client.host if request.client else "" # Warning: silence / no matching category — button shows warning and does not latch if category is None: message = "Тишина" if reason == "silence" else "Не распознано" _push_event( "voice_warning", source_id=source_id, request_id=request_id, category=message, request_text=transcript or "", reason=reason, ip=ip, audio_url=audio_url, ) return { "status": "warning", "request_id": request_id, "category": message, "message": message, "timestamp": ts, "source_id": source_id, "request_text": transcript or "", "needs_cancel": False, "reason": reason, "audio_url": audio_url, } call = { "request_id": request_id, "source_id": source_id, "category": category, "request_text": transcript, "timestamp": ts, "status": "active", "ip": ip, "audio_url": audio_url, } async with _lock: _active[request_id] = call _push_event( "voice", source_id=source_id, request_id=request_id, category=category, request_text=transcript, ip=ip, audio_url=audio_url, ) return { "status": "success", "request_id": request_id, "category": category, "timestamp": ts, "source_id": source_id, "request_text": transcript, "needs_cancel": True, "audio_url": audio_url, } @app.post("/api/v1/cancel") async def cancel_call( request: Request, x_api_key: Optional[str] = Header(default=None, alias="X-API-Key"), ): _check_api_key(_extract_api_key(request, x_api_key)) body: Dict[str, Any] = {} ctype = (request.headers.get("content-type") or "").lower() if "application/json" in ctype: body = await request.json() elif "application/x-www-form-urlencoded" in ctype or "multipart/form-data" in ctype: form = await request.form() body = {k: form.get(k) for k in form.keys()} else: raw = (await request.body()).decode("utf-8", errors="replace").strip() if raw: try: body = json.loads(raw) except json.JSONDecodeError: body = {} request_id = (body.get("request_id") or "").strip() source_id = (body.get("source_id") or "").strip() cancelled = None async with _lock: if request_id and request_id in _active: cancelled = _active.pop(request_id) elif source_id: for rid, call in list(_active.items()): if call.get("source_id") == source_id: cancelled = _active.pop(rid) break if not cancelled: _push_event("cancel_miss", request_id=request_id, source_id=source_id) return {"status": "ok", "cancelled": False, "message": "no active call"} cancelled["status"] = "cancelled" _push_event( "cancel", request_id=cancelled.get("request_id"), source_id=cancelled.get("source_id"), category=cancelled.get("category"), ) return {"status": "ok", "cancelled": True, "request_id": cancelled.get("request_id")} def _log_press(request: Request, path: str, method: str, qs: str, body: str = "") -> Dict[str, Any]: return _push_event( "press", path=path, method=method, query=qs, body=body[:500] if body else "", ip=request.client.host if request.client else "", ) @app.api_route("/api/click/btn", methods=["GET", "POST"]) async def click_btn(request: Request): qs = str(request.url.query) body = "" if request.method == "POST": body = (await request.body()).decode("utf-8", errors="replace") bid = request.query_params.get("bid", "") action = "reset" if bid.endswith("*00") else ("trigger" if bid.endswith("*01") else "unknown") _log_press(request, "/api/click/btn", request.method, qs, body) _push_event("latch", bid=bid, action=action, ip=request.client.host if request.client else "") return PlainTextResponse("OK") @app.get("/ok") async def ok_endpoint(request: Request): _log_press(request, "/ok", "GET", "") return PlainTextResponse("OK\n") @app.get("/fail") async def fail_endpoint(request: Request): _log_press(request, "/fail", "GET", "") return PlainTextResponse("FAIL\n", status_code=500) @app.get("/cleaning") async def cleaning(request: Request, result: str = "ok"): _log_press(request, "/cleaning", "GET", f"result={result}") return {"success": result, "service": "cleaning", "message": f"cleaning:{result}"} @app.get("/healthz") async def healthz(): return {"ok": True, "mock_ai": settings.mock_ai or not settings.openrouter_api_key} @app.get("/echo") async def echo(request: Request): _log_press(request, "/echo", "GET", str(request.url.query)) return {"query": dict(request.query_params), "headers": dict(request.headers)}