Skip to content
UGConic
All work
PERSONAL AI · VOICE-FIRST INTERACTION

An AI Assistant That Can Listen, Remember, Research, and Act.

Vesper is a voice-first personal AI assistant built around persistent memory, live web research, and real-world actions that require explicit user confirmation.

Combining local and cloud AI models with voice services and personal productivity integrations, it brings conversation and practical action into one desktop experience.

Vesper research view with the core orb and source cards
The problem

AI should do more than answer questions.

Vesper explores a more useful approach: a voice-first assistant that remembers relevant context, gathers information, and performs selected actions through controlled workflows.

  • 1

    Chat interfaces aren't always convenient for hands-free interaction.

  • 2

    Personal context is easily lost between conversations.

  • 3

    Research means searching many pages and assembling the findings yourself.

  • 4

    Messaging, email, and calendar actions involve repetitive manual steps.

  • 5

    Unrestricted AI control over real accounts is risky when a model misunderstands or invents details.

The solution

A personal AI workspace with four connected capabilities

Vesper runs locally on your machine, while selected requests use external services when required.

  • Voice conversation

    Speak or type; Vesper responds in text and spoken audio.

  • Persistent memory

    Durable personal facts and history stored locally in SQLite.

  • Live web research

    Source cards and a synthesized, cited plain-English summary.

  • Confirmed actions

    WhatsApp, Gmail, Calendar, and website actions — only after explicit confirmation.

On privacy: local memory and conversation history are stored in SQLite, but the application does send specific information to external services when using cloud AI, voice processing, Google integrations, or web research. Not all data stays on the machine under every circumstance.

Voice-first interaction

Speak, and it listens

Not every language is equally supported, and transcription is not error-free.

  • Speak or type — your choice.
  • The browser captures audio and runs speech-to-text.
  • English and Urdu language hints for transcription.
  • The response streams into the interface as it's generated.
  • Text-to-speech converts responses into spoken audio.
  • Switch between available voice options.
  • Voice activity detection helps sense when you've finished.
  • Barge-in lets you interrupt while it's speaking.
Persistent memory

Context that carries forward

Vesper stores selected durable personal facts and conversation history in a local SQLite database — for example a stable preference or a professional detail.

  1. 1

    The assistant identifies information that may be worth remembering.

  2. 2

    A dedicated process evaluates potential durable facts.

  3. 3

    Duplicate or low-value facts are filtered out.

  4. 4

    Stored facts are provided as context for future responses.

  5. 5

    You can inspect and delete remembered facts in the interface.

Memory is selective — not perfect recall of everything ever said.

Hybrid AI

Two AI brains. One adaptive assistant.

Intelligent routing avoids spending cloud requests on simple greetings, uses the more capable model for selected responses, and keeps local inference for internal utilities — with the local model also serving as a fallback.

Local AI

Gemma 3 4B via Ollama

  • Runs on your local machine and GPU
  • Greetings, simple conversation, internal utilities
  • Fallback when the cloud model is unavailable or quota is exhausted

Cloud AI

NVIDIA Nemotron via OpenRouter

  • Handles more demanding responses when available
  • Subject to a limited free daily request allowance
  • Falls back to local on quota, rate limits, or service errors

Request → routing → local or cloud model → response → voice playback & history. Model names are implementation details, not a guarantee that free access or availability continues. Not all inference is local; cloud usage is not unlimited.

Research engine

From a question to a research briefing

The documented setup uses Wikipedia and DuckDuckGo without a search API key; Tavily support is present but optional and not configured here.

  1. 01

    Ask

    Pose a research question by voice or text.

  2. 02

    Search

    Vesper searches available web sources.

  3. 03

    Rank

    Sources are ranked using configured trust signals.

  4. 04

    Retrieve

    Selected pages are fetched and cleaned into readable text.

  5. 05

    Note

    Short findings are created for individual sources.

  6. 06

    Synthesize

    Findings become a plain-English summary.

  7. 07

    Cite

    Source cards and citations help you explore further.

Research quality depends on the available sources, the retrieval process, and the model’s output — it is not guaranteed to be accurate or exhaustive.

Real-world actions

AI drafts the action. You approve the outcome.

The AI helps write the wording, but deterministic application code manages recipient selection, draft review, and execution. Nothing is sent or scheduled without explicit confirmation.

  1. 01

    Request

    You ask Vesper to message, email, or schedule something.

  2. 02

    Interpret

    The AI helps interpret intent and draft wording.

  3. 03

    Resolve details

    Deterministic code resolves recipients and required details.

  4. 04

    Prepare draft

    An editable draft or event is prepared.

  5. 05

    Review

    You review and can correct or cancel.

  6. 06

    Approve

    Nothing is sent or created without explicit confirmation.

  7. 07

    Execute

    The application performs the action after approval.

  • WhatsApp

    • Resolves recipients from available contacts and handles some name variations
    • Lets you choose among multiple matches
    • Shows an editable draft; sends only after confirmation
    • Node.js bridge via whatsapp-web.js — automates WhatsApp Web, not the official API

  • Gmail

    • Reads and searches email
    • Drafts replies and new messages
    • Uses Google's official Gmail API
    • Shows the message for confirmation; supports thread replies

  • Google Calendar

    • Checks for scheduling conflicts
    • Builds event details from natural language
    • Reads back details and waits for confirmation
    • Uses Google's official Calendar API

  • Website & browser

    • Opens known websites and certain search URLs from voice or text

Read operations and action operations carry different risk levels. The parsing logic was tested independently, while live account actions are exercised manually by the developer — not every real-account workflow has comprehensive automated end-to-end testing.

Architecture

Voice or text to a controlled, streamed response

Input reaches the FastAPI backend, which routes intent, calls local or cloud AI plus the relevant tools, runs controlled action workflows, and streams the response back with voice and local persistence.

Local FastAPI backend

Runs on the user's machine; routes intent and coordinates models, tools, and persistence.

  1. Voice / TextWeb Audio + SSE
  2. FastAPIBackend + routing
  3. RoutingIntent + context
  4. Local / Cloud AIOllama · OpenRouter
  5. ToolsResearch & actions
  6. Confirmed ActionsHuman approval
  7. SQLiteMemory & history

Gmail, Calendar, and WhatsApp are separate integration branches — not every request invokes every integration. SMS is not part of Vesper; email, research, and actions are distinct paths.

Backend

Python · FastAPI · Uvicorn

Frontend

HTML/CSS/JS · Three.js orb · Web Audio + MediaRecorder · Server-Sent Events

AI

NVIDIA Nemotron (OpenRouter) · Gemma 3 4B (Ollama) · OpenAI-compatible client

Voice

ElevenLabs Scribe (STT) · ElevenLabs TTS · browser fallback

Integrations

Gmail API · Calendar API · WhatsApp bridge (whatsapp-web.js) · Wikipedia · DuckDuckGo · optional Tavily

Storage

Local SQLite · local config & auth tokens

Interface & design

Warm, personal, voice-first

Vesper's interface deliberately avoids the usual blue-and-purple AI look. Inside the product, a warm espresso palette takes over — while the UGConic site around it stays on brand.

  • Espresso-toned backgrounds
  • Amber and coral accents
  • Sage green for memory
  • Central animated orb (listening / thinking / speaking)
  • Live transcript & history
  • Editable memory panel
  • Research result cards
  • Editable action drafts
  • English / Urdu toggle
  • Voice selection
Local-first & privacy

Designed around a personal machine

The backend runs on the local machine.
Local model inference is provided by Ollama.
Memory and conversation history are stored in SQLite.
Google OAuth tokens and settings are stored locally.
A desktop launcher starts the local services and opens Vesper in its own window.

External-service paths

  • Cloud model requests go to OpenRouter when selected.
  • Voice processing uses ElevenLabs in the documented configuration.
  • Gmail and Calendar operations communicate with Google.
  • Research requests retrieve external web content.
  • WhatsApp automation connects to the logged-in WhatsApp Web session.
The safety architecture

Human approval is part of the product

AI-assisted
  • Understands the request
  • Drafts the wording
  • Helps interpret details
  • Provides context and suggestions
Application-controlled
  • Resolves the destination
  • Validates required details
  • Presents the draft or event
  • Waits for confirmation
  • Executes only after approval

This reduces specific risks from incorrect model-generated actions; it is not a complete security guarantee.

Status & limitations

A working personal prototype, honestly stated

Developed initially for a hackathon concept and continued as a personal project. These are real engineering constraints and areas for future improvement.

  • Runs on the developer's local machine, not as a publicly deployed SaaS.
  • Gmail, Calendar, and WhatsApp connect to the developer's own accounts.
  • Cloud model usage is limited by the available free quota.
  • The local model is less capable than the cloud model for some complex tasks.
  • WhatsApp automation uses whatsapp-web.js and may break when WhatsApp Web changes.
  • Speech recognition can mishear names or topics.
  • Live action workflows have not all had comprehensive automated end-to-end testing.

No public self-service signup, multi-tenant SaaS, paying customers, uptime guarantees, or security certifications are claimed — and no perfect memory, transcription, or research accuracy.

Project journey

How Vesper evolved

3 October 2026 — hackathon concept

Development began for a personal-AI hackathon concept.

8 October 2026 — continued independently

Work continued so real personal integrations could be explored without a public-demo requirement, evolving around practical constraints like regional account access and limited free cloud usage.

The resulting architecture combines local inference (Ollama), cloud inference (OpenRouter), voice (ElevenLabs), official Google APIs for Gmail and Calendar, a separate Node.js WhatsApp bridge, and local persistence with explicit confirmation for real-world actions. Claude Code was used as an AI coding assistant during development; it is not part of the runtime product.

More work

Related work

  • LeadPilot AI lead response and booking cover
    AI Automationinternal2026

    LeadPilot — AI Lead Response & Booking

    Engages inbound leads, qualifies prospects, books appointments, and automates follow-ups through a configurable chat widget and dashboard.

    View project
  • LinkedIn Post Studio content automation cover
    AI Automationinternal2026

    LinkedIn Post Studio

    Turns fresh AI news into personalized LinkedIn drafts and branded visuals, ready for human-reviewed publishing.

    View project

Build AI that does more than chat.

UGConic explores and builds practical AI products that connect natural interaction, useful context, and carefully controlled automation.