Rem

AttentionInc, San Francisco — early pre-seed

Introduction

I’m building Rem, a desktop assistant with full personal context. Rem captures my computer as I work so I can jump back to any moment, ask natural questions (e.g., “what did Vicki text me on WhatsApp?”), and get either a quick text answer or time-stamped frames. Clicking a frame opens a timeline to scroll my past activity. A keyboard chord (double-Shift or double-Command) brings Rem up anywhere.

This project treats personal context as the basis for a next-generation GUI for AI. It draws on HCI/HRI research in: context-aware computing, memory augmentation and lifelogging for just-in-time assistance, and on-device, privacy-preserving ML. It’s an example of attention-aware, well-timed interventions, and multisensory intelligence as it treats multimodal personal context (what I’ve seen, said, or heard) as an interaction modality – all while keeping screen captured data stored on-device.

Demo

Problem

Today’s assistants are mostly app- or browser-bound, lack persistent personal context, and push users to connect third party accounts (i.e., Gmail, Slack, etc.) to be useful. That creates friction and privacy concerns, and it ignores the richest signal we already generate all day: what we’ve seen, said, or heard on our own machines. Without that signal, AI support remains shallow and poorly timed.

Solution & How It Works

Context-native assistant. Rem runs on top of the OS/GUI, not just the browser, and is powered by what I’ve seen, said, or heard while using my Mac.

Ask or scrub. I can ask in natural language and receive a concise answer or jump directly to time-stamped frames; selecting a frame opens a timeline to scroll past activity.

Instant access. A global hotkey (double-Shift or double-Command) summons Rem from anywhere.

No accounts to connect. It “just works” without linking email, chat, or cloud drives.

Private by design. Recordings are stored locally on my Mac with no cloud access.

Why It Matters

I treat Rem as an attempt at a next-generation GUI for AI: a context layer that makes assistants usable, trustworthy, and timely at the operating-system level. Instead of switching apps or forwarding data to the cloud, I keep control of my history on-device and use it to drive precise recall, better focus, and lower cognitive load.

My Contributions

• Interface design: end-to-end UI and the timeline/frames interaction model.
• Timing & intervention policy: when and how Rem surfaces help without stealing attention.
• Instrumentation: capturing and organizing on-device signals safely.

We’re a team of four; I led all UI work. This is an early research prototype (no users yet).

Research Alignment

• Fluid Interfaces: intervention policy for attention and focus; system-level assistant that acts with context and restraint.
• Multisensory Intelligence: treating personal context as a modality for interaction, enabling natural Q&A and scrubbable recall from what I’ve seen, said, or heard entirely on-device.
• Personal Robots: An attention-aware assistant and remains legible and trustworthy by running entirely on-device; a software with focus on social timing, transparency, and user control.