Embodied Interaction with Large Language Models: A Spatio-Physical Approach for Engaging Non-technical Users
Despite their growing capabilities, large language models (LLMs) remain unfamiliar to many non-technical users. The true potential of LLMs can be unlocked by providing appropriate contextual information and prompts, which are not easily accessible to general users. To help users explore LLM capabilities in engaging ways, we need new opportunities that go beyond traditional text-based interfaces. We propose a novel human-LLM interaction that combines verbal input with physical movements. Our design enables users to provide movement instructions corresponding to verbal inputs, allowing the system to demonstrate LLM capabilities through physical interactions rather than technical prompts. This approach leverages the inherent intuitiveness of spatial manipulation while maintaining the powerful language understanding capabilities of LLMs, creating an accessible entry point for novice users without requiring technical expertise. We evaluated this approach through two complementary studies: an exploratory investigation in a museum and a comparative analysis against a traditional chatbot interface. Results reveals important insights about the role of physical interfaces in human-LLM interaction. While physical interaction does not necessarily improve perceived ease of use compared to text-based interfaces, it significantly enhances user engagement and creates novel, enjoyable experiences. These findings suggest that physical interfaces may serve complementary roles to traditional text-based interfaces.