New vocabulary frequently emerges during collaborative human interactions, posing challenges for fixed-vocabulary speech and handwriting recognizers. This dissertation introduces SHACER, a system that dynamically learns out-of-vocabulary terms by integrating redundantly spoken and handwritten input. Analyses of multiple interaction contexts show that most handwritten terms are also spoken, are topic-specific, and are valuable for recall and retrieval. SHACER aligns speech and handwriting using phonetic and articulatory features to refine pronunciations and orthographic forms, enabling dynamic vocabulary acquisition. In a meeting-based application, SHACER significantly reduced recognition errors for handwritten abbreviations, demonstrating the value of multimodal redundancy for adaptive language understanding.
