RAGScore

RAGScore is an open-source tool designed for developers, data scientists, and researchers to effortlessly generate question-answer (QA) datasets and robustly evaluate retrieval-augmented generation (RAG) systems. It operates either locally or in the cloud, supports both CLI and Python API workflows, and seamlessly integrates with major LLM providers (Ollama, OpenAI, Anthropic, etc.). The tool delivers detailed multi-metric evaluations—accuracy, correctness, faithfulness, relevance, conciseness, and completeness—along with diagnostics for failure analysis, making it suitable for RAG debugging, audit, and quality assurance in domains like healthcare, legal, and research.