Finding the Perfect Temperature: Gradient Descent

When forging a 440Hz tuning fork, the annealing process is everything. Heat the steel too quickly and the crystalline structure becomes chaotic, producing muddy overtones. Cool it too slowly and you waste hours. The master metallurgist learns to feel the gradient—how the metal’s hardness changes with each degree of temperature—and follows it down to the sweet spot where the steel rings pure and true.

Gradient descent works the same way, but in mathematical space. Given any function with hills and valleys, the algorithm takes steps proportional to the negative gradient, always moving downhill toward a minimum.

function gradientDescent(func, gradient, start, rate = 0.01, steps = 1000) {
  let x = start;
  for (let i = 0; i < steps; i++) {
    x -= rate * gradient(x);
  }
  return x;
}

// Find minimum of f(x) = (x-3)² + 5
const result = gradientDescent(
  x => (x-3)**2 + 5,
  x => 2*(x-3),
  0
);
console.log(result); // ~3.0

Erlang brings a different perspective—treating optimization as a conversation between processes. Each iteration becomes a message, each parameter adjustment a supervised restart:

-module(gradient_descent).
-export([optimize/0]).

descent(X, _Rate, 0) -> X;
descent(X, Rate, Steps) ->
    Gradient = 2 * (X - 3),
    NewX = X - Rate * Gradient,
    descent(NewX, Rate, Steps - 1).

optimize() ->
    Result = descent(0, 0.01, 1000),
    io:format("Minimum at: ~p~n", [Result]).

The learning rate is your hammer’s weight—too heavy and you overshoot the mark, too light and progress crawls. In metallurgy, this intuition comes from years of practice. In computing, we often start bold and decay the rate as we approach the target, just like controlled cooling in the forge.