AI-Powered Computer Control for Windows

Novel UI Improvements for Recursive Control

Novel UI Improvements for Recursive Control

Date: October 2, 2025

Overview

We’ve added interactive, user-friendly UI enhancements that make Recursive Control more powerful, transparent, and easier to troubleshoot. These improvements focus on giving users visibility into what’s happening and making the system more engaging.


🎁 New Features

1. Chat Export System πŸ“€

Export your conversations in multiple formats for debugging, sharing, or documentation.

Features:

  • Export to JSON: Machine-readable format with timestamps
  • Export to Markdown: Human-readable format for documentation
  • Debug Export: Includes chat + plugin usage logs for troubleshooting
  • Copy to Clipboard: Quick copy for pasting elsewhere

Access:

File Menu β†’ Export Chat β†’ [Choose Format]

Formats:

JSON Export:

{
  "ExportTime": "2025-10-02 21:30:45",
  "MessageCount": 15,
  "Messages": [
    {
      "Timestamp": "2025-10-02T21:25:10",
      "Author": "You",
      "Content": "Open Chrome"
    },
    {
      "Timestamp": "2025-10-02T21:25:12",
      "Author": "AI",
      "Content": "Chrome has been opened successfully"
    }
  ]
}

Markdown Export:

# Chat Export - 2025-10-02 21:30:45

**Total Messages:** 15

---

## You
*2025-10-02T21:25:10*

Open Chrome

---

## AI
*2025-10-02T21:25:12*

Chrome has been opened successfully

---

Debug Export (with Tool Calls):

# Debugging Chat Export
**Export Time:** 2025-10-02 21:30:45
**Total Messages:** 15

## Chat Messages

### You - 2025-10-02T21:25:10

Open Chrome


### AI - 2025-10-02T21:25:12

Chrome has been opened successfully


---

## Plugin Usage Log

[21:25:10] WindowSelectionPlugin.ListWindowHandles [21:25:11] ExecuteCommand: chrome.exe [21:25:12] WindowSelectionPlugin.ForegroundSelect (12345678)

Use Cases:

  • Debugging: Export with tool calls to diagnose issues
  • Documentation: Share workflows in markdown
  • Analysis: Parse JSON exports programmatically
  • Support: Send debug logs to support team
  • Training: Create tutorials from actual interactions

2. Execution Visualizer 🎯

Real-time visual display of step-by-step execution progress.

Features:

  • Step-by-step display: See each action as it happens
  • Status icons: ⏳ Pending, βš™οΈ In Progress, βœ… Completed, ❌ Failed
  • Progress bar: Overall completion percentage
  • Color-coded steps: Visual feedback for status
  • Auto-scroll: Follows current step automatically

Visual Layout:

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Execution Progress                   β”‚
β”‚ Status: Step 3/10: Clicking element β”‚
β”‚ β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘ 30%                β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ #1 βœ… Take screenshot               β”‚
β”‚ #2 βœ… Find window handle            β”‚
β”‚ #3 βš™οΈ Click element (in progress)  β”‚
β”‚ #4 ⏳ Verify action                 β”‚
β”‚ #5 ⏳ Continue workflow             β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Color Scheme:

  • White/Gray: Pending (not started)
  • Light Blue: In Progress (currently executing)
  • Light Green: Completed (success)
  • Light Red: Failed (error occurred)
  • Light Gray: Skipped (intentionally skipped)

Benefits:

  • Transparency: See exactly what the AI is doing
  • Confidence: Visual feedback builds trust
  • Debugging: Identify where failures occur
  • Learning: Understand AI’s problem-solving approach
  • Engagement: Interactive feel vs black box

3. Activity Monitor πŸ“Š

Real-time system status and activity logging.

Features:

  • Status Indicators: AI, ONNX, Browser states
  • Activity Log: Color-coded event stream
  • Export Capability: Save logs for analysis
  • Auto-scroll: Always shows latest activity
  • Level Filtering: Debug, Info, Success, Warning, Error

Visual Layout:

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ πŸ€– AI: Processing (Blue)            β”‚
β”‚ πŸ‘οΈ ONNX: Ready (Green)             β”‚
β”‚ 🌐 Browser: Active - Chrome (Green)β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ [21:30:45] ℹ️ System: Started task β”‚
β”‚ [21:30:46] βœ… ONNX: Screenshot OK  β”‚
β”‚ [21:30:47] ℹ️ Planner: Step 1/10   β”‚
β”‚ [21:30:48] ⚠️ Warning: Slow resp.  β”‚
β”‚ [21:30:49] βœ… Success: Task done    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Icon Legend:

  • πŸ” Debug: Detailed diagnostic info
  • ℹ️ Info: General information
  • βœ… Success: Positive outcome
  • ⚠️ Warning: Potential issue
  • ❌ Error: Failure or problem

Benefits:

  • Awareness: Know system state at a glance
  • Monitoring: Watch AI activity in real-time
  • Diagnostics: Track down performance issues
  • Documentation: Export for issue reports
  • Transparency: No hidden operations

🎨 UI Philosophy

Interactive & Transparent

Users should see what’s happening, not guess. Every action should have visual feedback.

Informative, Not Overwhelming

Show important information clearly, hide details until needed. Progressive disclosure.

Engaging Experience

Computer control should feel interactive and responsive, not robotic.

Debugging-Friendly

When things go wrong, users should have the tools to understand why.


πŸ“‹ Implementation Details

ChatExporter Class

Location: FlowVision/lib/Classes/ChatExporter.cs

Methods:

// Export to JSON format
ChatExporter.ExportToJson(chatHistory);

// Export to Markdown format
ChatExporter.ExportToMarkdown(chatHistory);

// Export with plugin logs for debugging
ChatExporter.ExportWithToolCalls(chatHistory);

// Quick copy to clipboard
ChatExporter.CopyToClipboard(chatHistory);

Features:

  • Save file dialog with format-appropriate defaults
  • Automatic filename with timestamp
  • Error handling with user feedback
  • Includes plugin usage logs in debug export

ExecutionVisualizer Component

Location: FlowVision/lib/Classes/UI/ExecutionVisualizer.cs

Usage:

var visualizer = new ExecutionVisualizer();

// Start execution
visualizer.StartExecution(totalSteps: 10);

// Add steps
visualizer.AddStep("Take screenshot");
visualizer.AddStep("Click button");

// Update step status
visualizer.UpdateStep(0, StepStatus.InProgress);
visualizer.UpdateStep(0, StepStatus.Completed, "Screenshot captured");

// Complete
visualizer.CompleteExecution(success: true);

Features:

  • Fluent API for easy integration
  • Real-time visual updates
  • Auto-scrolling to current step
  • Color-coded status indicators
  • Progress bar for overall completion

ActivityMonitor Component

Location: FlowVision/lib/Classes/UI/ActivityMonitor.cs

Usage:

var monitor = new ActivityMonitor();

// Update system status
monitor.UpdateAIStatus("Processing", Color.Blue);
monitor.UpdateONNXStatus("Ready", Color.Green);
monitor.UpdateBrowserStatus("Active - Chrome", Color.Green);

// Log activities
monitor.LogActivity("System", "Task started", ActivityLevel.Info);
monitor.LogActivity("ONNX", "Screenshot captured", ActivityLevel.Success);
monitor.LogActivity("Planner", "Step 1/10", ActivityLevel.Info);
monitor.LogActivity("Network", "Slow response", ActivityLevel.Warning);
monitor.LogActivity("Task", "Completed successfully", ActivityLevel.Success);

// Export log
monitor.ExportLog();

Features:

  • Thread-safe updates
  • Color-coded by severity
  • Icon-based visual language
  • Timestamp for each entry
  • Export capability

πŸš€ Usage Examples

Example 1: Debugging a Failed Task

Scenario: User reports β€œAI clicked wrong button”

Steps:

  1. File β†’ Export Chat β†’ Export Debug Log
  2. Open exported file
  3. See exact sequence of actions
  4. Find tool calls that executed
  5. Identify incorrect window handle or coordinates
  6. Fix and retest

Export Shows:

### AI - 21:30:47

Clicking element at coordinates [300, 250]


## Plugin Usage Log

[21:30:47] MousePlugin.ClickOnWindow(12345678, [300, 250, 500, 310], true, 1) [21:30:47] Result: Clicked successfully

Analysis: Wrong window handle! Should have been 87654321 (different Chrome window).


Example 2: Monitoring Complex Workflow

Scenario: 15-step automation task

Execution Visualizer Shows:

βœ… Step 1/15: Screenshot captured
βœ… Step 2/15: Window found (Chrome)
βœ… Step 3/15: Brought to foreground
βš™οΈ Step 4/15: Typing search query (IN PROGRESS)
⏳ Step 5/15: Press Enter (PENDING)
⏳ Step 6/15: Wait for results (PENDING)
...

Activity Monitor Shows:

[21:30:45] ℹ️ System: Starting 15-step workflow
[21:30:46] βœ… ONNX: Screenshot captured (640x480)
[21:30:47] ℹ️ Planner: Step 4/15 - Type query
[21:30:48] βš™οΈ Keyboard: SendKeyToWindow(12345678, "Python tutorials")

Benefits:

  • User sees progress in real-time
  • Confidence that system is working
  • Can identify if step is taking too long
  • Visual confirmation of each action

Example 3: Sharing Workflow

Scenario: User wants to document their automation

Steps:

  1. Complete automation task
  2. File β†’ Export Chat β†’ Export to Markdown
  3. Share markdown file
  4. Others can see exact conversation and results

Result: Clean, readable documentation of the workflow.


πŸ’‘ Novel Features

What Makes These Improvements Unique?

1. Debug Export with Tool Calls

Novel: Most chat apps only export conversations. We export the actual tool calls that were executed, making debugging trivial.

Impact: Support teams can see exactly what the AI did, not just what it said.

2. Real-Time Execution Visualization

Novel: Not just a β€œloading” spinnerβ€”users see each step with status, icon, and color.

Impact: Builds trust and understanding. Users learn how the AI solves problems.

3. Activity Monitor Integration

Novel: System status + activity log in one place with color-coded severity.

Impact: Power users can monitor system health, casual users see reassuring status indicators.

4. Multi-Format Export

Novel: One feature, four export formats (JSON, Markdown, Debug, Clipboard) for different use cases.

Impact: Flexibility for developers (JSON), documentation writers (Markdown), support (Debug), and quick sharing (Clipboard).


🎯 Future Enhancements

Potential Additions

1. Element Highlighting:

  • Overlay on screenshots showing where AI will click
  • Visual confirmation before execution
  • Red outline = target, Green = success

2. Timeline View:

  • Horizontal timeline of all steps
  • Click to see details of each step
  • Duration visualization

3. Interactive Step Editing:

  • Pause execution
  • Modify next step
  • Resume with changes

4. Voice Feedback:

  • Optional audio cues for step completion
  • β€œStep 5 complete” announcement
  • Accessibility feature

5. Analytics Dashboard:

  • Success rate over time
  • Most used features
  • Average steps per task
  • Performance metrics

6. Collaboration Features:

  • Share workflows with team
  • Import exported workflows
  • Template library

πŸ“Š Metrics

Before UI Improvements:

  • Visibility: Low (black box behavior)
  • Debugging: Hard (no logs, no exports)
  • Engagement: Passive (waiting for results)
  • Trust: Uncertain (can’t see what’s happening)

After UI Improvements:

  • Visibility: High (see every step)
  • Debugging: Easy (export with tool calls)
  • Engagement: Active (watch progress real-time)
  • Trust: Strong (transparency builds confidence)

πŸ”§ Developer Guide

Adding to Your UI

Execution Visualizer:

// In your form
private ExecutionVisualizer visualizer;

void InitializeVisualizer()
{
    visualizer = new ExecutionVisualizer
    {
        Dock = DockStyle.Right,
        Width = 400
    };
    this.Controls.Add(visualizer);
}

// During execution
visualizer.StartExecution(steps.Count);
foreach (var step in steps)
{
    visualizer.AddStep(step.Description);
}

Activity Monitor:

// In your form
private ActivityMonitor monitor;

void InitializeMonitor()
{
    monitor = new ActivityMonitor
    {
        Dock = DockStyle.Right,
        Width = 300
    };
    this.Controls.Add(monitor);
}

// Log activities
monitor.LogActivity("AI", "Task started", ActivityLevel.Info);

βœ… Testing Checklist

Chat Export

  • JSON export creates valid JSON file
  • Markdown export is readable
  • Debug export includes plugin logs
  • Clipboard copy works
  • Timestamps are correct
  • Large chats export without errors

Execution Visualizer

  • Steps appear in correct order
  • Status updates work (Pending β†’ InProgress β†’ Completed)
  • Progress bar updates correctly
  • Auto-scroll follows current step
  • Colors change based on status
  • Failed steps show in red

Activity Monitor

  • Status indicators update correctly
  • Activity log shows timestamped entries
  • Color coding works for all levels
  • Export log creates valid file
  • Thread-safe (no UI freezing)
  • Icons display correctly

πŸ“ User Documentation

Quick Start: Exporting Chat

  1. Click File menu
  2. Select Export Chat
  3. Choose format:
    • JSON: For developers/programmers
    • Markdown: For documentation
    • Debug Log: For troubleshooting
    • Clipboard: For quick sharing
  4. Select save location
  5. Done! File is saved

Quick Start: Monitoring Execution

  1. Enable Multi-Agent Mode (for step-by-step execution)
  2. Start a task
  3. Watch the execution visualizer on the right
  4. See each step complete with checkmarks
  5. Progress bar shows overall completion

Quick Start: Activity Monitoring

  1. Open Activity Monitor panel
  2. Watch real-time status updates
  3. See color-coded activity log
  4. Export log if needed for troubleshooting

πŸŽ‰ Impact Summary

What We Achieved:

  1. Transparency: Users can see exactly what’s happening
  2. Debugability: Easy to export and analyze
  3. Engagement: Interactive, visual feedback
  4. Trust: Builds confidence through visibility
  5. Professionalism: Polished, modern UI experience

User Benefits:

  • βœ… Never wonder β€œis it working?”
  • βœ… Debug issues yourself before asking for help
  • βœ… Share workflows easily
  • βœ… Learn how AI solves problems
  • βœ… Feel in control, not helpless

Developer Benefits:

  • βœ… Easy to diagnose user issues
  • βœ… Export format works with existing tools
  • βœ… Clean component architecture
  • βœ… Extensible for future features
  • βœ… Well-documented APIs

πŸš€ Build Status

βœ… All UI components compile successfully
βœ… Chat export integrated into File menu
βœ… Execution visualizer ready to use
βœ… Activity monitor ready to use
βœ… No breaking changes
βœ… Backward compatible

These UI improvements transform Recursive Control from a functional tool into an engaging, transparent, and user-friendly platform. The focus on visibility, debugging, and interactivity makes it a joy to use! 🎨✨