Hyperparameter Search Tool

Grid search exhaustively tries every combination of the specified hyperparameter values. Random search samples uniformly at random from the search space for a fixed number of trials. For high-dimensional spaces, random search finds good configurations faster because it explores more diverse values of each dimension per trial.

Bayesian optimization builds a probabilistic surrogate model (typically a Gaussian process) of the objective function. After each trial it updates the model and uses an acquisition function — such as Expected Improvement (EI) — to select the next point most likely to improve on the best result so far. This is more sample-efficient than random search for expensive evaluations.

The horizontal axis is log10(learning rate) and the vertical axis is batch size. Each dot is one trial. Grid trials form a regular lattice; random trials are scattered uniformly; Bayesian trials cluster progressively toward promising regions based on the EI acquisition function.

The 2D space (LR × batch size) is divided into a 10×10 grid of cells. Coverage is the percentage of cells that contain at least one trial point. Grid search always achieves 100% on its own axes. Random and Bayesian coverage depends on the number of trials relative to the grid dimension.

Yes. Learning rates typically span several orders of magnitude (e.g., 1e-5 to 1e-1). Sampling uniformly on a log scale gives equal probability to each order of magnitude, which means random search explores the space proportionally rather than being dominated by large values.