GPT-Live-1 and Codex turned an LED display into a voice-controlled companion!

Hi everyone, it's Shii-chan! Today I'm excited about an adorable little hardware project: an OpenAI engineer turned a birthday gift into a voice-controlled companion.

OpenAI Developers developers.openai.com

What was announced?

OpenAI Developers published a build log from engineer Sid Rampally about a personal side project. It started as a birthday gift for his roommate: a small LED display that just showed nearby flights. But he started wondering, "Could this show the weather or my calendar too? Could I just ask, 'When's the next train?' and see the answer?"

That question turned into a project combining a 128 × 64 HUB75 RGB LED panel, a Raspberry Pi, the coding agent Codex, and GPT-Live-1, the full-duplex voice model OpenAI recently launched in the API. He even named the resulting assistant "Jack."

Why it matters

What impressed me most is that Sid had zero prior experience with HUB75 panels, ESP32 controllers, Raspberry Pi, or voice models. Most people would have given up the moment they opened up the display. Instead, he sent Codex photos of the circuit board and asked "what am I looking at?", building understanding step by step until he had a working system.

GPT-Live-1 is a full-duplex voice model that just landed in the API, letting you interrupt it or have it listen while it speaks. Paired with an agentic coding assistant like Codex, this project is a real example of how far someone without specialized expertise can go on a hardware-and-AI side project.

What changes

With the finished system, Sid can walk into the room, say "Hey Jack," and get the weather, his calendar, subway arrival times, photos, or messages shown on the wall — no laptop required. He can interrupt mid-response to ask for something else, too.

Asking "What's the weather?" and then following up with "Show that on the wall" now works correctly, thanks to a context-handling fix described below. Getting the system off his Mac and onto a Raspberry Pi is what turned it from a laptop demo into something he could just leave running.

Dive Deep

Hardware setup

• A 128 × 64 HUB75 RGB LED panel
• An ESP32-based controller running WLED-MM firmware
• A Raspberry Pi with a microphone and speaker

Two services run on the Pi:

• The voice service handles the microphone, PipeWire audio, WebRTC echo cancellation, wake/sleep behavior, interruptions, the GPT-Live-1 connection, and client-side delegation.
• The renderer service keeps the idle animation running, refreshes weather and MTA subway arrivals, uses sanitized calendar data, validates scene descriptions, and streams frames to the panel.

The interesting part is the model split: GPT-Live-1 stays focused on the conversation — listening while it speaks, and letting Sid interrupt. When a request needs more than the conversation itself (research, calendar info, an image, or a wall update), the Pi issues a client-side delegation and calls GPT-5.6 Luna through the Responses API with low reasoning effort. Luna can use hosted web search, a read-only calendar connection, or a reusable-image lookup on Wikimedia Commons, and decides what scene the request needs.

For images, the system searches Wikimedia Commons for reusable images, checks license and attribution, and resizes them to fit within 128 × 64 pixels while preserving their aspect ratio. The renderer validates the resulting scene JSON before converting it into exact 128 × 64 RGB frames and sending them over DDP to the WLED-MM controller, which drives the panel.

One bug the post describes: asking "What's the weather in New York?" and then "Show that on the wall" initially failed because the app only sent the model the latest request. The fix was including recent conversation history and earlier results in the handoff.

Wrap-up

• A birthday-gift flight tracker became a voice-controlled smart display
• The setup: a 128 × 64 HUB75 panel + ESP32 (WLED-MM) + Raspberry Pi
• GPT-Live-1 handles the conversation while GPT-5.6 Luna handles research and tool calls
• Small touches like Wikimedia Commons image lookup and DDP frame streaming made it all come together, built step by step with Codex
• The author started with zero experience in HUB75 panels, ESP32, Raspberry Pi, or voice models

If you're curious about hardware side projects, or want ideas for combining GPT-Live-1 and Codex into an agentic voice app, this one's worth a read!