IA-QA Eval & RAG — 38 tools
io.github.JcJamet/ia-qa-eval · v1.0.1 · score 89/100 · checked 2026-11-22
Automated, read-only signal from public data, not an audit or an accusation. Maintainers can dispute any finding. How grading works
LLM & RAG evals: hallucination, eval contracts, semantic tests, tokens & model info. Free, no key.
Source: https://github.com/ia-qa-com/mcp
Findings
| Severity | Rule | Detail |
|---|---|---|
| Low | MW005 | Remote endpoint lists 39 tool(s) to anyone, no credentials needed. |
| Medium | MW102 | 2 tool(s) can execute commands or delete data (run_eval_contract, generate_eval_yaml). Require confirmation before the agent calls them. |
What was checked
- https://www.ia-qa.com/mcp/eval: answers without auth
Public tools: get_testing_guidelines, find_tool, hallucination_check, run_eval_contract, run_semantic_tests, generate_ci_workflow, needle_haystack_generate, generate_eval_yaml, metamorphic_check, test_skill, skill_interview, llm_output_validator, compare_responses, analyze_responses, prompt_test_suite, llm_format_check, consistency_check, llm_json_schema_check, response_quality_score, conversation_analyze, cot_analyzer, ab_test_report, similarity_score, run_vlm_test_suite, embedding_similarity, rag_relevance_rank, bm25_score, build_rag_prompt, rerank_evaluate, count_tokens
Maintainer? Add the badge
Grades refresh daily. Think a finding is wrong? Dispute this grade.
[](https://www.mcpwatch.io/s/io-github-jcjamet-ia-qa-eval-d80c3458)