Digit: A Wearable Companion Bot with AI Voice Interaction

2024-10-03 | By ShawnHymel

Digit is a compact, wearable robot designed to enhance human interaction with technology. Built with a lightweight yet sturdy frame, the robot offers a seamless blend of comfort and utility. Upgraded from version 03a to 03e, Digit features stronger legs reinforced with steel wire, giving it improved durability and stability. Its sleek, ergonomic design ensures that the robot remains easy to wear while still performing advanced tasks. The upgraded software allows for quicker reaction times, ensuring the robot responds accurately to its environment and user commands.

One of the key innovations in Digit is its ability to integrate with the user's natural movements, without compromising comfort. Its customizable functions make it suitable for assisting tasks to provide interactive feedback during daily activities. As Digit continues to evolve, it reflects a growing trend toward robotics that can be seamlessly integrated into human life, offering both practical support and futuristic innovation.

Digit relies on a series of movable legs and magnets to attach to a special vest. This allows Digit to be worn or removed for interactions.

Digit’s Brains: Hopper Chat

Digit’s artificial intelligence (AI) is powered by a large language model (LLM), which is an advanced neural network capable of understanding and generating human-like text based on the transformer architecture. These models use vast amounts of data to train their numerous parameters, enabling them to predict the next word in a sequence and comprehend context and meanings. For example, ChatGPT-4, a popular LLM, includes billions of parameters allowing it to perform complex tasks, such as answering questions and telling jokes.

In the case of Digit, a local LLM facilitates real-time, lifelike conversations without needing an internet connection. This is achieved by running a smaller version of an LLM locally on an NVIDIA® Jetson Orin Nano™. The Orin Nano rests in the user’s backpack with a separate battery and can communicate with an Arduino board embedded in the robot’s body.

This setup allows Digit to understand text queries in English and respond in kind. By hosting the LLM locally, issues like latency are minimized, and interactions become more seamless and private. Local AI is crucial for applications requiring autonomous operation without reliable internet access, such as remote or mobile environments. Digit uses the Llama 3 model running inside an Ollama server. This local server listens for text queries on an open port and responds with text.

The interaction system of Digit involves a client program named Hopper Chat, which acts as the mediator between the user and Digit's AI brain.

Digit: A Wearable Companion Bot with AI Voice Interaction

Hopper Chat listens for speech through a microphone, converts it to text using the Vosk speech-to-text (STT) system, and checks for specific wake words or phrases. Once that phrase is heard (“hey, Digit” in this case), the Hopper Chat client sends the next spoken phrase (converted to text) to the local LLM to generate a response. This response is then sent to the Piper text-to-speech (TTS) service to create audible speech, which Digit vocalizes through a connected Bluetooth speaker. This entire process allows for a fluid conversational experience, making Digit an interactive and responsive companion bot capable of engaging in meaningful interactions.

Running Hopper Chat

You can run Hopper Chat yourself by following the instructions on the GitHub repository here: https://github.com/ShawnHymel/hopper-chat.

Note that a Raspberry Pi 5 (8GB RAM) will run Llama 3, but the response time is incredibly slow (2-3 minutes). If you plan to use the Jetson Orin Nano, I highly recommend following my guide here (https://shawnhymel.com/2255/getting-started-with-nvidia-jetson-orin-nano/) to install the latest operating system. You will also need to build the Docker containers specific to the NVIDIA Jetson system to enable GPU support (as shown in the repository’s README).