AI Engineer — LLM Systems & Evaluation

What this session covers

• RAG
• Evaluation
• Guardrails
• Latency
• Cost

Questions you'll face

1. 1. You are asked to ship a retrieval-augmented assistant over 200,000 internal documents. How do you approach the first two weeks?

   ~3 min

   Checks the candidate starts with evaluation and data, not model choice.

2. 2. Users say the assistant is 'sometimes wrong'. How do you turn that into something you can measure and improve?

   ~3 min

   Tests evaluation discipline over vibes.

3. 3. What guardrails do you put around an LLM feature that can trigger actions in other systems?

   ~3 min

   Production safety and least-privilege thinking.

4. 4. When is fine-tuning the right investment compared with better retrieval or prompting?

   ~3 min

   Assesses cost-aware judgement.

How the time is spent

1. 1

   Warm-up & context · 4 min

   Interviewer intro, your background, how the session runs.

2. 2

   Core questions · 24 min

   Role-specific questions with live follow-ups based on your answers.

3. 3

   Deep dive · 8 min

   One topic explored to the edge of your experience.

4. 4

   Your questions & wrap · 4 min

   Close the loop and hand over to feedback generation.