evals-skills
Evaluate skills for AI coding agents to help build LLM evaluations.
- Type
- Agent Skill
- Open source
- Yes
- GitHub Stars
- ★ 1.4k
- Source
- skill-github
- Repository
- github.com/hamelsmu/evals-skills
Overview
These skills are designed to guide AI coding agents and help users build LLM evaluations. They can help avoid common mistakes and recommend other skills to resolve issues. Suitable for engineers and product managers, especially those new to evaluation. After installing the plugin via Claude Code or Skills CLI, use the command `/evals-skills:<skill-name>` to invoke various skills.
Capabilities
- ▪Audit evaluation pipelines and identify priority issues
- ▪Guide users to review traces and categorize failures
- ▪Generate diverse synthetic test inputs
- ▪Design evaluators using LLMs as judges
- ▪Calibrate LLM judges against human labels
- ▪Evaluate retrieval and generation quality in RAG pipelines
Use cases
Setup
Run the following commands in Claude Code: "/plugin marketplace add hamelsmu/evals-skills" and "/plugin install evals-skills@hamelsmu-evals-skills"; or use npx: "npx skills add https://github.com/hamelsmu/evals-skills"
This information was compiled by AI from public sources and may contain inaccuracies — please refer to the source.
FAQ
How do I get started with these skills?
Start with the `eval-audit` skill and follow the steps in the README to install and run it.
Who are these skills for?
Suitable for engineers and product managers, especially those just getting started with evaluation.
Related skills
mantis
Provides security review capabilities for AI coding agents, automatically discovering, reproducing, and fixing vulnerabilities.
sast-skills
Transform your AI coding assistant into a SAST scanner to automatically detect vulnerabilities in code.
Lightswind-UI-Library
AI-native CLI-first React component library with MCP Server support.
scientific-agent-skills
Transform any AI agent into a scientific assistant with 147 ready-to-use research skills.
susi_alexa_skill
A skill that enables question-and-answer interactions between Alexa and Susi AI.
claude-context
Provides code search capabilities for Claude Code, turning the entire codebase into context.