AI

Agent Cost Optimizer

Hard
Agents

Implement cost optimization for agent systems:

Cost Factors:

  1. Model selection (GPT-4 vs GPT-3.5)
  2. Token usage (input/output)
  3. Tool call costs
  4. Infrastructure

Methods:

  1. select_model(complexity, budget): Route to cheapest viable model
    • Simple tasks → cheaper model
    • Complex tasks → expensive model
    • Return None if budget insufficient
  2. estimate_cost(model, in_tokens, out_tokens): Predict cost
  3. track_usage(model, in_tokens, out_tokens): Record spend
  4. optimize_memory_usage(messages, target): Compress to save tokens
  5. get_cost_report(): Breakdown by component

Cost Formula: cost = (input_tokens * input_price + output_tokens * output_price) / 1000

Examples

Example 1:
Input: opt = CostOptimizer(budget=1.0); model = opt.select_model('simple', 1.0); model
Output: 'gpt-3.5'
Explanation: Simple task routed to cheaper model

Starter Code

class CostOptimizer:
    """
    Optimize agent execution costs through model routing and token management.
    """
    
    def __init__(self, budget):
        self.budget = budget
        self.spent = 0
        self.model_costs = {
            'gpt-4': {'input': 0.03, 'output': 0.06},  # per 1K tokens
            'gpt-3.5': {'input': 0.0015, 'output': 0.002}
        }
        self.usage_history = []
    
    def select_model(self, task_complexity, remaining_budget):
        """
        Select appropriate model based on task and budget.
        Returns model name or None if budget insufficient.
        """
        # Your implementation here
        pass
    
    def estimate_cost(self, model, input_tokens, expected_output_tokens):
        """Estimate cost for operation"""
        # Your implementation here
        pass
    
    def track_usage(self, model, input_tokens, output_tokens):
        """Track actual usage and deduct from budget"""
        # Your implementation here
        pass
    
    def optimize_memory_usage(self, messages, target_tokens):
        """
        Optimize memory to fit target token count.
        Uses compression, summarization.
        """
        # Your implementation here
        pass
    
    def get_cost_report(self):
        """Generate cost breakdown report"""
        # Your implementation here
        pass
Lines: 1Characters: 0
Ready
The AI Interview - Master AI/ML Interviews