Dec 12 2024

Kaggle/Google Day 3 - Agents

  • Kaggle Day 3 - Agents (also see this link)

  • Google’s AI Evaluation Quality and Explainability - increase creative of generated responses, get 3 to 5 responses. Then evaluate each against one another.

    • Takes the instruction, context and a variable number of corresponding generated responses, and returns the pointwise evaluation metrics for each of the provided metrics. For this example the metrics are Q & A related, however the full list can be found on the website here
    • Google’s Gen AI evaluation service can help you with the following tasks:
      • Model selection: Choose the best pre-trained model for your task based on benchmark results and its performance on your specific data.
      • Generation settings: Tweak model parameters (like temperature) to optimize output for your needs.
      • Prompt engineering: Craft effective prompts and prompt templates to guide the model towards your preferred behavior and responses.
      • Improve and safeguard fine-tuning: Fine-tune a model to improve performance for your use case, while avoiding biases or undesirable behaviors.
      • RAG optimization: Select the most effective Retrieval Augmented Generation (RAG) architecture to enhance performance for your application.
      • Migration: Continuously assess and improve the performance of your AI solution by migrating to newer models when they provide a clear advantage for your specific use case.
      • Translation: Assess the quality of your model’s translations.