Inference-time compute

Topic

Inference-time compute refers to the computational resources allocated to a machine learning model, particularly large language models, during the generation of an output or response. Unlike training-time compute, scaling inference-time compute allows models to perform multi-step reasoning, search, and verification processes (such as chain-of-thought) to improve accuracy and problem-solving capabilities on the fly.