mcpwatch

← All servers

Grade B

IA-QA Eval & RAG — 38 tools

io.github.JcJamet/ia-qa-eval · v1.0.1 · score 89/100 · checked 2026-11-22

Automated, read-only signal from public data, not an audit or an accusation. Maintainers can dispute any finding. How grading works

LLM & RAG evals: hallucination, eval contracts, semantic tests, tokens & model info. Free, no key.

Source: https://github.com/ia-qa-com/mcp

Findings

SeverityRuleDetail
LowMW005Remote endpoint lists 39 tool(s) to anyone, no credentials needed.
MediumMW1022 tool(s) can execute commands or delete data (run_eval_contract, generate_eval_yaml). Require confirmation before the agent calls them.

What was checked

Public tools: get_testing_guidelines, find_tool, hallucination_check, run_eval_contract, run_semantic_tests, generate_ci_workflow, needle_haystack_generate, generate_eval_yaml, metamorphic_check, test_skill, skill_interview, llm_output_validator, compare_responses, analyze_responses, prompt_test_suite, llm_format_check, consistency_check, llm_json_schema_check, response_quality_score, conversation_analyze, cot_analyzer, ab_test_report, similarity_score, run_vlm_test_suite, embedding_similarity, rag_relevance_rank, bm25_score, build_rag_prompt, rerank_evaluate, count_tokens

Maintainer? Add the badge

Grades refresh daily. Think a finding is wrong? Dispute this grade.

[![mcpwatch](https://www.mcpwatch.io/badge/io-github-jcjamet-ia-qa-eval-d80c3458.svg)](https://www.mcpwatch.io/s/io-github-jcjamet-ia-qa-eval-d80c3458)