Optimization Landscape

Preset Gallery

A curated set of presets illustrating canonical behaviors in gradient-based optimization.

0. Isotropic quadratic (geometry) 1. GD on quadratic (converges) 2. Anisotropic quadratic (slow) 3. Mixture of Gaussians (local minima) 4. Rosenbrock (banana valley) 5. Noise effects

Tip: in each preset, hover to use the probe (−∇f + Hessian). Pan/zoom to connect geometry with dynamics.

Optimization Landscape — Curated Presets

Load a preset, then review the concise notes below.

Preset

0. Isotropic quadratic — geometry

Objective: Quadratic (Isotropic) Particles: none (geometry only) Overlays: vector field, probe, contours

Setup

Hover to reveal ∇f arrow and Hessian ellipse. Discuss eigenvalues.
Enable 3D surface briefly to connect level sets with the bowl shape.

Interpretation

For a quadratic, level sets are circles and the vector field points directly to the minimizer.
The Hessian is constant; the ellipse is identical everywhere (up to visualization scaling).

1. Particles on quadratic — everything converges

Objective: Quadratic (Isotropic) h: moderate σ: 0 Stats: on

Key idea

Spawn a small cloud. Run. Trajectories contract to the minimizer.
Point to the statistics: \(f(x_t)\) decreases (nearly) monotonically.

Suggested experiment

Increase h until you see overshoot/instability; then reduce it back.

2. Anisotropic quadratic — conditioning slows convergence

Objective: Quadratic (Anisotropic) Anisotropy α: large h: small σ: 0

Interpretation

The valley is narrow: curvature is large in one direction, small in the other.
Stable step sizes are constrained by the largest eigenvalue; progress along the flat direction becomes slow.

Suggested experiment

Try doubling h: observe divergence or zig-zagging.
Reduce α with the anisotropy slider and compare.

3. Mixture of Gaussians — local minima / basins

Objective: Mixture of Gaussians (−log density) h: moderate σ: 0 Markers: component centers

Setup

Spawn particles near the middle. Run: trajectories split into different basins.
Reseed the mixture; discuss how nonconvex structure changes.

Interpretation

GD is deterministic: initial conditions decide which basin you reach.
Basins correspond to mixture components (approximately), but the true minima are not simply the centers.

4. Rosenbrock — curved valley

Objective: Rosenbrock (scaled) h: small σ: 0 Goal: illustrate curved optimization paths

Key idea

Follow a particle from off-valley: it is pulled into the valley, then progresses slowly along it.
Use the probe to show high curvature across the valley and low curvature along it.

Suggested experiment

Turn on the 3D surface briefly to emphasize the banana-shaped valley.

5. Noise — exploration vs. convergence

Objective: Mixture of Gaussians (−log density) h: moderate σ: > 0 Goal: illustrate stochasticity

Setup

With small noise, trajectories jitter; some particles can cross shallow barriers.
Increase σ: trajectories become more diffuse; convergence quality degrades.

Interpretation

Noise can help exploration but prevents exact convergence.
This is a toy analogue of stochastic gradients and Langevin-style samplers.

Presets configure objective, overlays, and particle initialization. You can still adjust any slider after loading.

Gradient Descent (with Noise)

Particles

Overlays

Mathematical definition

Statistics over time