Selected Projects
A selection of research and engineering projects focused on human-AI interaction, natural language interfaces, and multimodal systems.
-
Natural Language Interaction for Editing Network Graphs
Designed and implemented a natural language interaction system to efficiently edit network graph visualizations. Compared GUI-based, natural language, and textual command-based interaction methods, showing significant efficiency improvements with NLP-driven techniques.
Skills: User-centered design, experimental design, front-end and back-end development, data visualization.
-
Multi-Modal Interaction System for Human-AI Collaboration
Developed interactive features for Vision-Language Models (VLMs) to integrate visual and textual data seamlessly. Reduced cognitive bottlenecks and enabled users to provide actionable feedback on object relationships and segmentation tasks.
Skills: Prototyping, usability testing, front-end and back-end development, multimodal interaction design.
-
Understanding Trade-offs in Human-in-the-Loop Feedback Collection: An Empirical Analysis of User Input Methods
Human-in-the-loop methods leverage human feedback to enhance machine learning and artificial intelligence. Manual review of outputs can correct errors, identify model weaknesses, or expand labels to broaden model capabilities.
This paper presents a user study evaluating changes in user performance over time and explores the trade-off between feedback quality and human effort. We compare four interactive input methods for reviewing and correcting outcomes in object detection and activity recognition in videos. Our findings indicate that while some complex input methods, such as free-text, require more time, the quality and impact of their feedback on model accuracy often surpass those of simpler methods that require less effort.