Click to pin this point
Decoding 900 latent points for each model — takes about 10–15 seconds
An interactive visualization of how VAEs learn a structured, continuous latent space and why KL divergence term is important.
Start exploring ↓A standard autoencoder compresses an image into a small latent vector and then reconstructs it. A VAE adds a probabilistic twist: instead of encoding to a single fixed point, it encodes each input as a Gaussian distribution — defined by a mean (μ) and a variance (σ²) — and then samples a latent vector z from that distribution before decoding. This small change has profound consequences for the structure of the learned space.
The sampling step uses the reparameterization trick: instead of sampling directly (which blocks gradients), we compute z = μ + σ · ε where ε is random noise from N(0, I). This lets gradients flow through μ and σ, making the entire network trainable with standard backpropagation. The VAE is trained to minimize two terms simultaneously:
This VAE was trained on Fashion-MNIST with a 2-dimensional latent space, so we can visualize it directly. Every point (z₁, z₂) in the grid below is a possible latent vector. The background image is generated by decoding a 30×30 grid of points covering z₁, z₂ ∈ [−3, 3]. The colored dots show where 1,000 real test images land after being encoded.
Each color = one of 10 Fashion-MNIST categories. Click a label to hide/show it on the map.
The background is the decoder's output at every grid point in latent space.
Similar clothing items cluster nearby because the encoder maps similar inputs to nearby regions.
The colored dots are real test images projected into the same 2D space by the encoder. Notice how they align with the background image content!
Hover anywhere on the map to decode that latent point. Notice how similar items occupy nearby regions — boots cluster together, as do bags, T-shirts, and trousers.
Click once to pin point A (purple marker), then click again to pin point B (pink). The interpolation panel will activate so you can drag a slider to morph between the two items.
The KL divergence term is what separates a VAE from a plain autoencoder. It constrains the encoder to map every input to a distribution close to a standard normal N(0, I). This seemingly simple constraint has a dramatic effect on the structure of the learned latent space.
KL divergence acts as a regularizer on the latent space. By pushing all distributions toward N(0, I), it forces the encoder to use overlapping, compact regions — the decoder must then learn smooth, consistent mappings across the entire space. The cost is slightly worse reconstruction accuracy (the encoder can't perfectly memorize each input's exact location), but the benefit is a continuous, navigable latent space where every point generates something meaningful. This trade-off is precisely what makes VAEs useful as generative models rather than just compression tools.
Both models use the identical encoder and decoder architectures, trained on the same Fashion-MNIST data for the same number of epochs. The only difference is whether the KL divergence term is included in the loss. Hover over both maps and compare what you see.
Try this: Hover over the same (z₁, z₂) coordinate on both maps simultaneously. On the VAE side you'll see a recognizable clothing item; on the AE side the same coordinate often falls in a gap and produces a blurry, meaningless output — because the AE was never trained to decode that region.
A key test of any latent space is: can you smoothly interpolate between two points and get meaningful images at every step? Because the VAE's KL term enforces continuity, you can pick any two clothing items and walk a straight path between them — the decoder produces a plausible image throughout.
The VAE decoder handles this gracefully because the KL term enforces a continuous prior. Every point between A and B lies in a region close to N(0, I), so the decoder encountered similar inputs during training. Try interpolating between very different categories — a sneaker and a bag — and watch the morphing happen step by step. A plain autoencoder cannot do this reliably because the path between two points crosses uninformative dead zones.
The interpolation controls are embedded directly in the Latent Space Explorer section above — scroll up, pin two points, and the slider activates automatically.