Transcription of language samples is labor intensive and data on intra- and inter-annotator reliability is lacking. Mean Length of Utterance in Morphemes (MLUM) can be calculated from transcripts and used to evaluate intra-annotator reliability. We developed a computational method to determine transcriber consistency across a large speech corpus.