Local Computer Use Agent

A supervised Windows desktop automation agent with OCR, template matching, structured logs, local/OpenAI routing, and an operator-facing GUI.

Problem

Desktop automation usually breaks at the exact moment operators need clarity, especially when a workflow mixes visual recognition, recovery logic, and human review.

Why it matters

This project shows a real operator-first automation philosophy: observable state, recovery ladders, logging, and controlled execution instead of blind automation theater.

System description

The agent captures screens, runs OCR and template logic, classifies the visible state, and executes bounded actions while keeping the operator in the loop through logs, dry-run modes, browser support, and supervised fallbacks.

Tools / methods

PythonTkinterOCRTemplate matchingStructured runtime logsLocal/OpenAI routingOperator review gates

Constraints

Needs to recover from visual ambiguity without pretending certainty.
Has to stay supervised and auditable.
Must tolerate real Windows UI drift and timing issues.

Workspace source

ce_automator/README.md

Windows desktop computer-use agent with OCR, template matching, local/OpenAI model routing, structured logs, and a Tkinter operator GUI.