Describe a situation where you had to evaluate an AI system that you built.

In my last internship I built a small classifier that routed support tickets into the right queue. For evaluation, I didn't just look at the overall accuracy, because some mistakes only delayed a ticket while others sent urgent issues to the wrong team. I created a labeled set from recent real tickets with help from a teammate, kept part of it separate from the examples I was using during development, and looked specifically at the urgent-ticket cases. The first version did okay overall but missed too many tickets that mentioned outages in less direct language. I updated the preprocessing and added a few targeted rules before the model step, then reran the same evaluation set and saw that urgent-ticket misses dropped a lot without hurting the rest. That gave my lead enough confidence to use it in a limited internal workflow first.

During a campus hackathon, I helped build a small app that used an AI model to suggest study plans based on a student’s classes and deadlines. To evaluate it, I asked a handful of students to try it with real schedules and then compared the suggestions against a simple checklist we made with a tutor, like whether the plan spread work out instead of stacking everything at the end. I found that the model sounded confident even when it missed obvious exam dates, so I started logging the inputs that led to bad suggestions and realized our prompts were too vague. After I tightened the questions we sent to the model and added a basic check for calendar conflicts, the recommendations became much more reliable. What I learned was that even for a small project, getting feedback from the actual people who would use it mattered more than just seeing whether the demo looked impressive.