Giant language mannequin (LLM)-based laptop use brokers execute person instructions by interacting with out there UI parts, however little is understood about how customers wish to work together with these brokers or what design components matter for his or her person expertise (UX). We performed a two-phase examine to map the UX design area for laptop use brokers. In Section 1, we reviewed current methods to develop a taxonomy of UX concerns, then refined it by means of interviews with eight UX and AI practitioners. The ensuing taxonomy included classes corresponding to person prompts, explainability, person management, and customers’ psychological fashions, with corresponding subcategories and instance design options. In Section 2, we ran a Wizard-of-Oz examine with 20 members, the place a researcher acted as a web-based laptop use agent and probed person reactions throughout regular, error-prone and dangerous execution. We used the findings to validate the taxonomy from Section 1 and deepen our perceive of the design area by figuring out the connections between design areas and divergence in person wants and eventualities. Our taxonomy and empirical insights present a map for builders to think about totally different points of person expertise in laptop use agent design and to situate their designs inside customers’ numerous wants and eventualities.
- † Carnegie Mellon College
