TY  - THES
N2  - Neural language models typically employ a categorical approach to prediction and training, leading to several well-known computational and numerical limitations. These limitations are particularly evident in applied settings where language models are employed as means for communication. From speller systems employed as assistive technology to texting applications on smartphones, all language models revolve around category-based prediction. Research shows that neural-category approaches to language modeling are questionable for predicting low-frequency words that are essential for user personalization. It is also challenging to adapt these architectures to a changing vocabulary due to the initially learned vocabulary constraints, which limit predictions of relevant categories (i.e., words) a user can type. Recently, such categorical models were shown to be relatively complex with long inference times, which may be detrimental for user engagement. In this thesis, I reevaluate neural-category approaches and propose an alternative: continuous output prediction.
DO  - 10.6083/f1881m64w
DO  - DOI
AB  - Neural language models typically employ a categorical approach to prediction and training, leading to several well-known computational and numerical limitations. These limitations are particularly evident in applied settings where language models are employed as means for communication. From speller systems employed as assistive technology to texting applications on smartphones, all language models revolve around category-based prediction. Research shows that neural-category approaches to language modeling are questionable for predicting low-frequency words that are essential for user personalization. It is also challenging to adapt these architectures to a changing vocabulary due to the initially learned vocabulary constraints, which limit predictions of relevant categories (i.e., words) a user can type. Recently, such categorical models were shown to be relatively complex with long inference times, which may be detrimental for user engagement. In this thesis, I reevaluate neural-category approaches and propose an alternative: continuous output prediction.
T1  - Overcoming limitations of categorical language modeling
DA  - 2020
AU  - Dudy, Shiran
L1  - https://digitalcollections.ohsu.edu/record/8748/files/Dudy.Shiran.2020.pdf
PB  - Oregon Health and Science University
PY  - 2020
ID  - 8748
L4  - https://digitalcollections.ohsu.edu/record/8748/files/Dudy.Shiran.2020.pdf
KW  - augmentative and alternative communication
KW  - language models
TI  - Overcoming limitations of categorical language modeling
Y1  - 2020
L2  - https://digitalcollections.ohsu.edu/record/8748/files/Dudy.Shiran.2020.pdf
LK  - https://digitalcollections.ohsu.edu/record/8748/files/Dudy.Shiran.2020.pdf
UR  - https://digitalcollections.ohsu.edu/record/8748/files/Dudy.Shiran.2020.pdf
ER  -