Background In part 1 of my project, I built a unigram language model: it estimates the probability of each word in a text simply based on the fraction of times the word appears in that text. The text used to train the unigram model is the book “A Game of Thrones” by George R. R. Martin (called train). …
Background Language modeling — that is, predicting the probability of a word in a sentence — is a fundamental task in natural language processing. It is used in many NLP applications such as autocomplete, spelling correction, or text generation. Currently, language models based on neural networks, especially transformers, are the state of the art: they predict very accurately a …
