# IA-QA Eval & RAG — 38 tools (io.github.JcJamet/ia-qa-eval): tool descriptions scored

- tools/list taken: 2026-11-02 (UTC)
- From: [Official MCP Registry](https://registry.modelcontextprotocol.io/v0/servers/io.github.JcJamet%2Fia-qa-eval/versions/latest)
- Connected to: https://www.ia-qa.com/mcp/eval
- Repository: https://github.com/ia-qa-com/mcp
- Scoring rules: v2
- Tools: 38
- Average: 62.9 / 100 (Cloudy)
- Near-identical pairs: 0

This page shows how IA-QA Eval & RAG — 38 tools (io.github.JcJamet/ia-qa-eval)'s tool descriptions scored, the issues found and the names of its tools and parameters. The descriptions are the server's own: read them where the copy came from.

Forecall asked the URL this server lists in the official MCP Registry for its tools/list once, on the date above, without calling any tool. The server may have changed since.

## Across the server

- Minor: 38 tools. Above 25, models start to tell tools apart less well.
- Minor: The server's instructions are 2125 characters. Claude Code reads only the first 2,048 by default (Codex 0.154.0 reads them whole). Put what the server is for first.

## Tools

- `conversation_analyze`: 39 / 100
  - Major: Says nothing about when to use it, or when not to.
  - Major: Arguments without a description: messages[].role, messages[].content.
  - Minor: String arguments with no constraint or example: messages[].content.
- `context_window_check`: 49 / 100
  - Major: Says nothing about when to use it, or when not to.
  - Minor: Arguments without a description: messages[].role, messages[].content.
  - Minor: String arguments with no constraint or example: messages[].role, messages[].content.
- `build_rag_prompt`: 50 / 100
  - Major: Says nothing about when to use it, or when not to.
  - Minor: Arguments without a description: chunks[].text, chunks[].source, chunks[].score.
  - Minor: String arguments with no constraint or example: chunks[].text, chunks[].source.
- `similarity_score`: 51 / 100
  - Major: Says nothing about when to use it, or when not to.
  - Minor: Arguments without a description: batch[].reference, batch[].hypothesis.
  - Minor: String arguments with no constraint or example: batch[].reference, batch[].hypothesis.
  - Minor: No `required` list, so every argument is optional.
- `run_eval_contract`: 52 / 100
  - Major: Says nothing about when to use it, or when not to.
  - Major: Arguments without a description: api_keys.groq, api_keys.openai, api_keys.anthropic, api_keys.google, api_keys.hf, api_keys.deepseek, api_keys.xai, overrides.model, overrides.provider, overrides.temperature, overrides.system_prompt.
  - Minor: String arguments with no constraint or example: api_keys.groq, api_keys.openai, api_keys.anthropic, api_keys.google, api_keys.hf, api_keys.deepseek, api_keys.xai, overrides.model, overrides.provider, overrides.system_prompt.
  - Minor: No `required` list, so every argument is optional.
- `run_vlm_test_suite`: 52 / 100
  - Major: Says nothing about when to use it, or when not to.
  - Major: Arguments without a description: image_base64, system_prompt, model, test_cases, test_cases[].id, test_cases[].question, test_cases[].assertion_type.
  - Minor: String arguments with no constraint or example: image_base64, system_prompt, test_cases[].id, test_cases[].question.
- `rag_relevance_rank`: 53 / 100
  - Major: Says nothing about when to use it, or when not to.
  - Minor: String arguments with no constraint or example: query.
- `cot_analyzer`: 55 / 100
  - Major: Says nothing about when to use it, or when not to.
- `embedding_similarity`: 55 / 100
  - Major: Says nothing about when to use it, or when not to.
  - Minor: Arguments without a description: batch[].text_a, batch[].text_b.
  - Minor: String arguments with no constraint or example: batch[].text_a, batch[].text_b.
  - Minor: No `required` list, so every argument is optional.
- `rerank_evaluate`: 55 / 100
  - Minor: Arguments without a description: query, passages[].text, passages[].id.
  - Minor: String arguments with no constraint or example: query, passages[].text, passages[].id.
  - Minor: The meaning of the boolean argument passages[].relevant is unclear.
- `model_info`: 58 / 100
  - Major: Says nothing about when to use it, or when not to.
- `prompt_test_suite`: 58 / 100
  - Major: Says nothing about when to use it, or when not to.
  - Minor: String arguments with no constraint or example: must_include, must_not_include.
  - Minor: The description mentions none of its 3 or more arguments.
- `run_semantic_tests`: 58 / 100
  - Major: Says nothing about when to use it, or when not to.
  - Minor: The description mentions none of its 3 or more arguments.
- `bm25_score`: 60 / 100
  - Major: Says nothing about when to use it, or when not to.
  - Minor: String arguments with no constraint or example: query.
- `llm_json_schema_check`: 60 / 100
  - Major: Says nothing about when to use it, or when not to.
- `estimate_llm_cost`: 61 / 100
  - Major: Says nothing about when to use it, or when not to.
- `token_budget_calculator`: 62 / 100
  - Major: Says nothing about when to use it, or when not to.
- `consistency_check`: 63 / 100
  - Major: Says nothing about when to use it, or when not to.
- `count_tokens`: 63 / 100
  - Major: Says nothing about when to use it, or when not to.
- `hallucination_check`: 63 / 100
  - Minor: Arguments without a description: cases[].id, cases[].answer, cases[].context.
  - Minor: String arguments with no constraint or example: cases[].id, cases[].answer, cases[].context.
  - Minor: No `required` list, so every argument is optional.
- `metamorphic_check`: 63 / 100
  - Minor: Arguments without a description: variants[].id, thresholds.case, thresholds.typo, thresholds.paraphrase, thresholds.reorder, thresholds.translation, thresholds.specialization.
  - Minor: String arguments with no constraint or example: base.question, variants[].id, variants[].question.
- `response_quality_score`: 65 / 100
- `test_skill`: 65 / 100
  - Minor: Arguments without a description: scenarios[].prompt, scenarios[].should_trigger, other_skills[].name, other_skills[].description.
  - Minor: String arguments with no constraint or example: scenarios[].prompt, other_skills[].name, other_skills[].description.
  - Minor: The meaning of the boolean argument scenarios[].should_trigger is unclear.
- `llm_generate`: 66 / 100
  - Major: Says nothing about when to use it, or when not to.
- `generate_ci_workflow`: 67 / 100
  - Major: Says nothing about when to use it, or when not to.
  - Minor: Arguments without a description: workflow_name.
  - Minor: String arguments with no constraint or example: cron, workflow_name, node_version.
  - Minor: No `required` list, so every argument is optional.
- `generate_eval_yaml`: 67 / 100
- `get_testing_guidelines`: 67 / 100
  - Major: Says nothing about when to use it, or when not to.
  - Minor: No `required` list, so every argument is optional.
- `llm_format_check`: 67 / 100
  - Major: Says nothing about when to use it, or when not to.
- `compare_models`: 70 / 100
  - Major: Says nothing about when to use it, or when not to.
- `compare_responses`: 70 / 100
  - Major: Says nothing about when to use it, or when not to.
- `llm_output_validator`: 71 / 100
  - Minor: The description mentions none of its 3 or more arguments.
- `ab_test_report`: 75 / 100
  - Minor: The description mentions none of its 3 or more arguments.
- `extract_json_from_text`: 75 / 100
- `find_tool`: 75 / 100
- `analyze_responses`: 77 / 100
  - Minor: String arguments with no constraint or example: responses[].text.
- `list_llm_models`: 77 / 100
  - Minor: No `required` list, so every argument is optional.
- `needle_haystack_generate`: 77 / 100
- `rate_tool`: 78 / 100

Is your server listed here and you would rather it were not? Email support@forecall.dev, and we will take it off within 7 days.

[All scored servers](https://forecall.dev/en/servers)