AI Inference Request Batching: coding question

Available question text

Given a list of AI inference requests with arrival times and token counts, implement a batching strategy that groups requests to maximize throughput while respecting constraints on batch size, token limits, and per-request SLA wait times.

Continue with questions that share topics or the same question type.

Count Zero Request Servers
Coding
Implement a Data Loader with Readability, Resuming, and Batching (with Tests)
Coding
Implement Batch Request Processing for Thing Objects
Coding
Implement Node and Function classes with generic type inference (get_return_type)
Coding

Sources · 1 reference

These records explain where the question appears. A company tag or candidate report does not guarantee a future interview question.

Hello Interview · 1 record

Microsoft
AI Inference Request Batching
Reported Apr 1, 2026 · 1 source report
SENIOR
Source snapshot: Nov 12, 2026
View source

AI Inference Request Batching

Available question text

Related questions

AI Inference Request Batching

Available question text

Related questions