Purpose
Desktop automation usually breaks at the exact moment operators need clarity, especially when a workflow mixes visual recognition, recovery logic, and human review.
ai automation · 2026
Local Computer Use Agent
A supervised Windows desktop automation agent with OCR, template matching, structured logs, local/OpenAI routing, and an operator-facing GUI.
activePython + Tkinter + OCRDesktop operator interface
My role
Designer and engineer
Status
active
Methods
- Python
- Tkinter
- OCR
- Template matching
- Structured runtime logs
- Local/OpenAI routing
- Operator review gates
Outputs
- Desktop operator interface
- Structured action logs
- Recovery and review workflow
Result
A working local automation system that keeps recognition, action, and recovery visible to the operator.
Implementation
How it works
The agent captures screens, runs OCR and template logic, classifies the visible state, and executes bounded actions while keeping the operator in the loop through logs, dry-run modes, browser support, and supervised fallbacks.
Built out of the ce_automator workspace, this system has to survive messy real Windows state, not ideal screenshots. OCR, template matching, logs, and recovery paths stay visible in the dossier because supervised control is the point of the build.