Tool crosses into game memory
- Control
- Screen-space capture only; no process-memory access or injection
- Residual risk
- Game policy can still change and should be reviewed.
Complete
Computer Vision / Desktop App2025
OWCoach
Produced real-time coaching suggestions from screen pixels alone-without process-memory access, injection, or live network dependence.
- Role
- Computer vision and desktop application engineer
- Environment
- Windows desktop · Local inference
- Ownership
- Capture · Calibration · Template matching · Rules engine · Overlay · Packaging
owcoach-vision5 system boundaries
Game Screen
Screen Capture
Template Matcher
Counter Rules
0.87template-match confidence threshold
0game-memory reads
0network calls during play
Localcounter-rules execution
01 / Interactive system
Move through the system layer by layer
owcoach-visionWindows desktop · Local inference · Read-only
Complete
System map
Boundaries and responsibilities
Public or untrustedAuthenticated boundaryInternal-only dependency
Game Screen selected.
02 / Context
The problem behind the system
A useful coaching overlay needs to understand the visible match state without crossing into game-process inspection or intrusive client behavior.
OWCoach treats terms-of-service safety as an architectural constraint: it reads the screen, evaluates a local ruleset, and renders a click-through recommendation surface.
Read the full project overview
A real-time Overwatch coaching overlay that reads the game screen - not its memory - to recommend hero counters as a match unfolds. A Python detection engine captures the scoreboard/killcam region, matches hero portraits with template matching, infers the enemy composition, and renders click-through suggestions through an Overwolf overlay. Packaged as a standalone Windows desktop app.
SAFE BY DESIGN - The whole thing is deliberately screen-space only. It never reads or writes game process memory, never injects into the client, and makes zero network calls during play - so it is not a cheat and stays inside the game's terms of service by construction. The counter logic is a local ruleset; nothing about the player's session is uploaded.
ENGINEERING - Auto-calibration locates the relevant HUD region across resolutions, template matching runs above a 0.87 confidence threshold to avoid false positives, and the overlay stays click-through so it never intercepts input. The build ships as a PyInstaller executable with a desktop shortcut and icon, plus a small stats-fetcher that pulls aggregate hero data offline.
03 / Decisions
The trade-offs that shaped the build
DecisionScreen-space only
Use what the player can already see.
- Reason
- Avoid process inspection, injection, and hidden game-state access.
- Trade-off
- Visual detection is less reliable than direct internal data.
- Revisit when
- Never toward memory access; only improve the visual pipeline.
DecisionLocal rules during play
Keep the match path independent from external services.
- Reason
- Latency, privacy, and availability all improve when recommendations stay local.
- Trade-off
- Rules need an offline update mechanism.
- Revisit when
- Only for clearly disclosed, non-match-time data refresh.
DecisionConfidence before recommendation
Prefer no answer to a confident-looking wrong answer.
- Reason
- Bad detections undermine the entire coaching layer.
- Trade-off
- Some match states produce no suggestion.
- Revisit when
- As calibration and template quality improve.
04 / Security
Threats, controls, and what remains
No control is presented as total risk elimination.
Live session data leaves the device
- Control
- Local rules and zero network calls during play
- Residual risk
- Offline data refresh still needs an explicit update path.
False-positive hero detection
- Control
- Calibration plus a 0.87 confidence threshold
- Residual risk
- UI changes and visual overlap can still reduce accuracy.
05 / Implementation
How the decisions appear in the build
owlive/detect.py
01# Reads the SCREEN locally - never game memory, never the network. TOS-safe.02import mss, numpy as np03 04# Screen-space template matching only: no process-memory reads, no injection.05HERO_ICONS = load_templates("hero_names.json")06 07with mss.mss() as sct:08 region = calibrate(sct) # auto-locate the killcam / scoreboard09 while running:10 frame = np.asarray(sct.grab(region))11 matches = match_templates(frame, HERO_ICONS, threshold=0.87)12 13 enemy = [m.hero for m in matches if m.team == "enemy"]14 counters = suggest_counters(enemy) # local ruleset - zero API calls15 overlay.render(counters) # click-through Overwolf overlayRepresentative / sanitized excerpt
Screen-space only - never touches game memory, makes zero network calls. TOS-safe by construction.
PythonOpenCVmssNumPyOverwolfPyInstaller
06 / Operations
The unhappy path is part of the design
What if calibration is wrong?
The matcher sees the wrong pixels and confidence falls.
- Signal
- Expected hero candidates disappear or stay below threshold.
- Response
- Re-run local calibration for the current resolution and UI layout.
What if no hero clears the threshold?
The pipeline withholds the affected recommendation.
- Signal
- The low-confidence state remains visible in diagnostics.
- Response
- Keep the overlay quiet and wait for a clearer frame.
07 / Evidence
Proof, source, and inspectable outcomes
08 / Results
What the project demonstrates
Recommendations use visible pixels rather than hidden process state
The match-time path runs locally without network calls
Low-confidence input fails quietly instead of inventing certainty
- Safety constraints produced a cleaner architecture.
- Calibration and thresholds keep uncertainty explicit.
- Broader resolution testing
- Authorized visual regression fixtures
- Clearer offline data-update provenance
- Real-time computer vision
- Desktop application design
- Constraint-driven engineering