Year: 2020

N-gram language models – Part 1

Background Language modeling — that is, predicting the probability of a word in a sentence — is a fundamental task in natural language processing. It is used in many NLP applications such as autocomplete, spelling correction, or text generation.   Currently, language models based on neural networks, especially transformers, are the state of the art: they predict very accurately a …

N-gram language models – Part 1 Read More »

Gaussian samples – Part (1)

Background Gaussian sampling — that is, generating samples from a Gaussian distribution — plays an important role in many cutting-edge fields of data science, such as Gaussian process, variational autoencoder, or generative adversarial network. As a result, you often see functions like tf.random.normal in their tutorials. But, deep down, how does computer know how to generate Gaussian samples? This series …

Gaussian samples – Part (1) Read More »

Duality theorems

Introduction Optimization shows up everywhere in machine learning, from the ubiquitous gradient descent to quadratic programming in SVM, to expectation-maximization algorithm in Gaussian mixture models. However, one aspect of optimization that always puzzled me is duality: what on earth are a primal form and dual form of an optimization problem, and what good do they serve? Therefore, in this project, I will: Go …

Duality theorems Read More »