Batch Run Costs
Record of batch inference runs: data size, model, API cost and how well each did
About This Project
Batch-Run-Costs provides a standardized method for recording and auditing batch machine learning inference jobs. It tracks essential metadata including model identifiers, routing methods, token consumption, and financial costs, enabling users to maintain a historical record of job performance and resource expenditure over time.
The project utilizes a JSONL-based data structure to store individual run records, which are managed through associated CLI tools for logging and searching. This approach allows developers to evaluate the efficacy of different models and inference routes, providing a clear reference for cost-benefit analysis and model selection in future tasks.