FLOP Around and Find Out: LLM Training Workload Size Estimation With Power Monitoring

AbstractAs the capabilities of frontier AI systems rapidly advance, there is growing interest in a multilateral agreement between nations to mitigate the risks posed by future, powerful systems. Any such agreement will require a quantifiable definition of what counts as a "frontier" model. Existing legislation, like the EU AI Act and California's SB 53, uses the number of floating-point operations (FLOPs) in the training run as the threshold for which models count as frontier, and proposals for ...