Forecall
Failure recordsDocs
All scored servers

SubwayInfo NYC: tool descriptions scored

tools/list taken
(UTC)
From
smithery.ai/servers/subwayinfo
Homepage
subwayinfo.nyc
Scoring rules
v2

Scores, issues and names only

This page shows how SubwayInfo NYC's tool descriptions scored, the issues found and the names of its tools and parameters. The descriptions are the server's own: read them where the copy came from.

This copy comes from Smithery's registry, which keeps each tool's name, description and input schema, but no output schema, annotations or instructions. Points for return values and for marking destructive tools can come out lower than the server's own tools/list would give.

A server with a public URL can be scored as it is today: score from a URL

servers/subwayinfo
Average score
38.3 / 100Rain
Tools
23
Near-identical descriptions
1

Across the server

Tools whose descriptions are nearly the same

1 of 1 pairs shown, highest description similarity first.
ToolToolDescription similarityName similarity
bus_list_alertsferry_list_alerts0.700.50

Tools

ferry_list_alerts

26 / 100Rain

Purpose
6 / 20
When to use
0 / 20
Arguments
15 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
5 / 10
  • Major
    The description only restates the name (9 words).restates_name
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    Its description is nearly the same as bus_list_alerts's (similarity 0.70); only the names and schemas tell them apart. Say in this description when to use bus_list_alerts instead.confusable_pair
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

mta_get_line_status

26 / 100Rain

Purpose
6 / 20
When to use
0 / 20
Arguments
15 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
5 / 10
  • Major
    The description only restates the name (8 words).restates_name
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

mta_search_stations

26 / 100Rain

Purpose
6 / 20
When to use
0 / 20
Arguments
15 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
5 / 10
  • Major
    The description only restates the name (7 words).restates_name
  • Major
    The description is too short (7 words).too_short
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

bike_get_availability_summary

32 / 100Rain

Purpose
17 / 20
When to use
0 / 20
Arguments
15 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
0 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

mta_list_alerts

32 / 100Rain

Purpose
17 / 20
When to use
0 / 20
Arguments
15 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
0 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    The description mentions none of its 3 or more arguments.params_not_in_description
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

mta_plan_trip

33 / 100Rain

Purpose
13 / 20
When to use
0 / 20
Arguments
15 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
5 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

bus_list_alerts

35 / 100Rain

Purpose
13 / 20
When to use
0 / 20
Arguments
17 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
5 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    Its description is nearly the same as ferry_list_alerts's (similarity 0.70); only the names and schemas tell them apart. Say in this description when to use ferry_list_alerts instead.confusable_pair
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

bus_get_route_info

37 / 100Rain

Purpose
17 / 20
When to use
0 / 20
Arguments
15 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
5 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

ferry_get_routes

37 / 100Rain

Purpose
17 / 20
When to use
0 / 20
Arguments
15 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
5 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

mta_get_station_info

37 / 100Rain

Purpose
17 / 20
When to use
0 / 20
Arguments
15 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
5 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

rail_get_station_info

37 / 100Rain

Purpose
17 / 20
When to use
0 / 20
Arguments
15 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
5 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

mta_get_planned_work

40 / 100Cloudy

Purpose
20 / 20
When to use
0 / 20
Arguments
15 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
5 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

rail_list_alerts

40 / 100Cloudy

Purpose
17 / 20
When to use
0 / 20
Arguments
18 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
5 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

bike_get_station_status

42 / 100Cloudy

Purpose
17 / 20
When to use
0 / 20
Arguments
15 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
10 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

bus_search_stops

42 / 100Cloudy

Purpose
17 / 20
When to use
5 / 20
Arguments
15 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
5 / 10
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

ferry_search_landings

42 / 100Cloudy

Purpose
17 / 20
When to use
5 / 20
Arguments
15 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
5 / 10
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

bike_search_stations

44 / 100Cloudy

Purpose
17 / 20
When to use
0 / 20
Arguments
15 / 25
Return value
7 / 15
Constraints and side effects
0 / 10
Examples
5 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

bus_get_arrivals

44 / 100Cloudy

Purpose
17 / 20
When to use
0 / 20
Arguments
17 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
10 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    The description mentions none of its 3 or more arguments.params_not_in_description
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

ferry_get_arrivals

44 / 100Cloudy

Purpose
17 / 20
When to use
0 / 20
Arguments
17 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
10 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    The description mentions none of its 3 or more arguments.params_not_in_description
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

mta_get_arrivals

44 / 100Cloudy

Purpose
17 / 20
When to use
0 / 20
Arguments
17 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
10 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    The description mentions none of its 3 or more arguments.params_not_in_description
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

rail_get_departures

44 / 100Cloudy

Purpose
17 / 20
When to use
0 / 20
Arguments
17 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
10 / 10
  • Major
    Says nothing about when to use it, or when not to.no_usage_context
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    The description mentions none of its 3 or more arguments.params_not_in_description
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

rail_search_stations

44 / 100Cloudy

Purpose
17 / 20
When to use
5 / 20
Arguments
17 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
5 / 10
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

transit_ask

52 / 100Cloudy

Purpose
20 / 20
When to use
5 / 20
Arguments
17 / 25
Return value
0 / 15
Constraints and side effects
0 / 10
Examples
10 / 10
  • Minor
    No `required` list, so every argument is optional.no_required
  • Minor
    Says nothing about what it returns.no_return_info
  • Minor
    No constraints, side effects, auth or rate limits described, and no annotations.no_constraints

What static scoring cannot tell

This score reads only the text of each description and schema: it says what is written, not what a model will do. Short tools whose names already say what they do, such as browser_close, score low here even when models use them well, and in our measurements current models chose the right tool in one step even from one-line descriptions, as long as the names and schemas differed. What the text cannot show, such as what happens over several steps and whether an added sentence helps or misleads, is what an evaluation measures, on the Pro and Team plans.

Is your server listed here and you would rather it were not? Email support@forecall.dev, and we will take it off within 7 days.

All scored servers