Every now and then I will come across a tensor manipulation method in Pytorch or some piece of code in Pytorch and think “I wish I had known about this earlier”. Then after one or two days complete...
This article acts as a repository of the concepts I learned about any topics in Statistics. I plan to use this as a reference for ideas/concepts/formulas in Statistics. Essentially this is a look-u...
Latent variable models defines a distribution over observations $x$ by using a latent variable $z$. There are $5$ separate distributions associated with a LVM. $p(x,z)$ : Joint distribution of ...
Evidence lower bound, commonly known as ELBO is one of the fundamental concepts to understand variational inference and also probabilistic decision theory. I am going to assume that readers are fam...
Image taken from MIT Introduction to Deep Learning 6.S191: Lecture 4 What are Latent Variables ? In simplest term, the variables to which we have no access to or in other words “unobserved” var...
The Basic Idea Jensen’s inequality requires prerequisite knowledge about the definition of convex functions. I will assume the readers are somewhat familiar with convexitiy of functions. Jensen’s...
There are two ways to introduce KL divergence, one is with the notion of “distance between distributions” and another is with information theory. The latter one is more in line with what is really...
Mutual information is one of the most important concepts in machine learning. It is closely related to concepts like Kullback–Leibler (KL) divergence, cross-entropy etc. This is such a fundamental...