Vertex AI LLM Evaluation Services

We offer a comprehensive set of notebooks that demonstrate how to use Vertex AI LLM Evaluation Services in conjunction with other Vertex AI services. Additionally, we have provided notebooks that delve into the theory behind evaluation metrics.

Theory notebooks: - Metrics for Classification - Metrics for Summarization - Metrics for Text Generation - Metrics for Q&A

Requirements

To run the walkthrough and demonstration in the notebook you'll need access to a Google Cloud project with the Vertex AI API enabled.

Getting Help

If you have any questions or find any problems, please report through GitHub issues.