Design an API Gateway for LLM Providers

Design a component that powers LLM API calls across multiple providers, supporting per-organisation policies and the ability to collect and track quality feedback metrics based on model and provider performance.