Batched Model Runs

Design a batching service that accepts individual requests and efficiently groups them into batches of up to 100 for processing by a model where all requests in a batch take the same time to complete.