n-gram-language-models for mathematics and statistics
Background In part 1 of my project, I built a unigram language model: it estimates the probability of each word in a text simply based on the fraction of times the word appears in that text. The text used to train the unigram model is the book “A Game of Thrones” by George R. R. Martin (called train). …
Background Language modeling — that is, predicting the probability of a word in a sentence — is a fundamental task in natural language processing. It is used in many NLP applications such as autocomplete, spelling correction, or text generation. Currently, language models based on neural networks, especially transformers, are the state of the art: they predict very accurately a …
Background In previous parts of my project, I built different n-gram models to predict the probability of each word in a given text. This probability is estimated using an n-gram — a sequence of words of length n — which contains the word. The below formula shows how the probability of the word “dream” is estimated …
