AI Engineer interview questionsCoding — the call that has to degrade round

Purpose

Sets the subject, and closes off prompt engineering as a place a candidate can spend the hour.

Purpose

Makes the tooling and the lookups explicitly allowed, and warns that the environment will change so a candidate does not spend minutes doubting their own code.

Purpose

Deliberately open. What they reach for first is the read — a candidate who starts by validating the model's output has the right instinct; one who starts by adding a try/catch around everything has a different one. Once they have something, switch the stub to return the fenced JSON, then the schema-valid refusal, in that order.

Signals to score

• Validates the model's output against a schema at the boundary rather than trusting its shape
• Treats a parse failure as an expected outcome with a policy, not as an exception to log
• Handles the response that is wrapped in a code fence without reaching for a regular expression that assumes one
• Notices the schema-valid refusal — every field present, the content an apology — and says schema validation cannot catch it
• Proposes a check on the content itself for that case, and keeps it cheap
• Does not retry with a byte-identical prompt at temperature 0 and expect a different result
• If it re-asks, appends what was wrong to the next attempt, and bounds the number of attempts
• Keeps the failure path returning the same type as the success path, rather than null or a thrown error the caller must know about
• Puts the model call behind a boundary the rest of the code does not see through
• Writes or extends a test for each failure as it appears, rather than testing by re-running by hand

Follow-up questions

• Run it against the stub now. What does your code do with that?
• Every field is there and the summary says the model cannot help. Which of your checks catches that?
• You are retrying. What is different about the second attempt?
• What does the caller of this function have to know about how it can fail?
• If I changed the model tomorrow, which of these checks would I have to revisit?

Purpose

Introduces a constraint from outside the code, which is where most production timeout mistakes come from.

Purpose

Three separate reads in one question: the number, what a timeout actually cancels, and whether their retry policy makes the tail worse. The second is the one most candidates have never thought about.

Signals to score

• Leaves room in the budget for everything else in the handler rather than setting the timeout at 10 seconds
• Says what fraction of calls the chosen timeout kills, using the profile rather than guessing
• Knows that abandoning the request does not stop the generation, and that the tokens are still billed
• Distinguishes which errors are worth retrying — a 429 or a 503 — from which are not, and never retries a 400
• Reads Retry-After when the stub sends one instead of using its own backoff
• Notices that retrying a timeout inside the same request budget makes the tail worse, not better
• Adds jitter, or explains why synchronised retries across instances are the failure they are avoiding
• Raises idempotency if the surrounding code writes anything, or asks whether it does
• Reaches for streaming as a way to change what the user experiences rather than what the p99 is
• Separates the timeout on first token from the timeout on the whole response, if streaming

Follow-up questions

• Your handler has 10 seconds. How much of it is this call allowed to have, and what has the rest?
• You time out at 8 seconds. What is the provider doing at second 9?
• The stub returns 429 with Retry-After: 30. What does your code do?
• You retry once on timeout. What just happened to your p99?
• Does this function write anything? Does the retry write it twice?

Purpose

Rules out waiting it out, and rules out treating it as an incident rather than as a condition the code is expected to have an answer for.

Purpose

The section the round exists for. A model call is the one dependency in most products that is expected to fail regularly, and the difference between a candidate who has run one and a candidate who has not is entirely in what they think the page does meanwhile.

Signals to score

• The page still renders — the failure is scoped to the part of it that needed the model
• Names the isolation mechanism concretely, rather than saying the error is handled
• Puts something useful in the space: the raw ticket, the previous summary with the time it was made, or an honest empty state
• Labels a stale value as stale rather than presenting it as current
• Never surfaces a provider error string, a status code or a stack trace to a customer
• Stops calling after a run of failures instead of sending every request into a dead provider
• Says what the breaker costs when it is wrong, and how it closes again
• Knows the queue of waiting requests is its own outage — their own latency dies before the provider comes back
• Separates what the customer sees from what is logged and alerted on, and says who is paged
• Distinguishes a feature that may degrade from one that may not, and asks which this is

Follow-up questions

• Which part of the page is allowed to fail, and how does the rest of it not?
• Show me what is in that space. What does the customer read?
• You have 40,000 of these a day. How many requests do you send to a provider you know is down?
• Your breaker opens by mistake on a slow afternoon. What did that cost?
• What do you want to be told about this, and when — a page at 3am or a number in the morning?

Purpose

Puts the partial output in front of a user before asking anything, so the answer cannot be purely server-side.

Purpose

The question is epistemic rather than mechanical, and it catches an assumption almost everyone carries: that a stream ending means the response finished.

Signals to score

• Says plainly that a closed connection and a finished response are indistinguishable without something that says which
• Requires a terminal event or a finish reason from the stream, and treats its absence as a failure
• Does not persist or cache a truncated answer as a completed one
• Marks the partial result as partial wherever it is stored, if it is stored at all
• Distinguishes the stream ending from the model stopping at a length limit, which is a different problem with the same symptom
• Tells the user the answer was cut off rather than letting it trail away
• Prefers re-asking over resuming, or justifies resumption with what it would cost to build
• Notices that this interacts with anything that reuses answers, and says so
• Asks what the consumer is — a browser, a queue, another service — before deciding what it needs to know
• Keeps the completed-or-not decision in one place rather than in every consumer

Follow-up questions

• The connection closed. Does that mean the answer finished?
• Where does those six hundred tokens end up, and is it marked as anything?
• The model stopped because it hit its length limit instead. Same symptom — how would you tell?
• Would you resume it or ask again? What does resuming cost you to build?
• Does anything else in this system reuse stored answers?