Skip to content
StrikeUp Help Centre
Release notes

Review and improve answer quality

When a visitor tells you a reply was wrong, the cause can sit in four places: the knowledge base, a skill, the toolkit that calls your API, or your guardrails. This guide takes one bad answer from the Feedback page, follows it through the AI action that wrote it and the tool run behind it, and tells you where to make the fix. Every page you use here only reads, so nothing you open changes what visitors see.

Before you start

Section titled “Before you start”
You needWhere it lives
Feedback switched on for your widget, so visitors can rate answers“Enable feedback” under ConfigureUser Interface; off by default
At least one thumbs-down from the last 14 daysAnalyticsFeedback
The Admin role, to open a conversation turn and a tool runAsk an Admin to change your role under SettingsUsers
Access to change skills, the knowledge base, toolkits or guardrails, for the fixConfigure and Security

Availability: every workspace, with no plan or feature flag. All signed-in users can read the lists; the conversation turn dialog and tool run pages need the Admin role.

To see whether there is a trend first, look at the Negative Feedback by Reason chart on DashboardPerformance. It tells you which reason visitors pick most often.

Steps

Section titled “Steps”

  1. Find a bad answer. Open AnalyticsFeedback, set the vote filter to Negative only and select a row. The detail panel shows the visitor’s reason and comment and, under AI Response, the AI action that wrote the answer, with its Chat Session ID and the time. Note the time.

    Step 1 on the Feedback page, filtered to "Negative only". The selected second row, with its reason missing_context highlighted, is a thumbs down on the answer "Your workspace will move to annual billing on 1 October. You will receive a prorated invoice today." from 14 September. The detail panel on the right shows the "Negative" badge and, under "AI Response", the AI action behind the answer: Anthropic, claude-sonnet-4-5, 3.32s, HTTP 200 and the "Chat Session ID" 8268684a-6ca1-bfc8-11e1-f0e239918baf, which identifies the conversation.Step 1 on the Feedback page, filtered to "Negative only". The selected second row, with its reason missing_context highlighted, is a thumbs down on the answer "Your workspace will move to annual billing on 1 October. You will receive a prorated invoice today." from 14 September. The detail panel on the right shows the "Negative" badge and, under "AI Response", the AI action behind the answer: Anthropic, claude-sonnet-4-5, 3.32s, HTTP 200 and the "Chat Session ID" 8268684a-6ca1-bfc8-11e1-f0e239918baf, which identifies the conversation.
    Start from a negative rating; the detail panel names the model, the time and the chat session of the answer.
    What this screenshot shows
    Screen
    Feedback (/admin/feedback)
    Highlighted
    cell “”, list-row-2, selected. shows the rating and its AI action in the detail panel
    Also visible
    Vote filter Negative only; Detail panel: Negative, AI Response with session 8268684a…
    Purpose
    step 1, find a bad answer

  2. Find the conversation turn. Open AnalyticsAI actions, select the Activity tab and choose a Period that covers the rating. Find the turn that started just before the time you noted. A red ”… failed” badge means some attempts failed and were retried.

    Step 2 on the "Activity" tab of AI actions. The "Conversation turns" table lists the last seven days of turns. The turn from 14 September at 16:10, stage support, model claude-sonnet-4-5, 4 calls with a red "1 failed" badge, about 38.6 s provider time and $0.07 cost, is the conversation from the feedback. The highlighted "View" button at the end of its row opens every call of that turn.Step 2 on the "Activity" tab of AI actions. The "Conversation turns" table lists the last seven days of turns. The turn from 14 September at 16:10, stage support, model claude-sonnet-4-5, 4 calls with a red "1 failed" badge, about 38.6 s provider time and $0.07 cost, is the conversation from the feedback. The highlighted "View" button at the end of its row opens every call of that turn.
    Find the same moment on the Activity tab and open the turn.
    What this screenshot shows
    Screen
    Activity (/admin/ai-logs)
    Highlighted
    button “View”, table-row-2-actions, enabled. opens the Conversation turn dialog
    Also visible
    Turn 14/09 16:10:04, 4 calls, 1 failed
    Purpose
    step 2, locate the turn

  3. Open the turn and pick the call that answered. Select View. The Conversation turn dialog lists every call in order; the last call usually wrote the answer the visitor rated. Select View call on its final attempt.

    Step 3: the "Conversation turn" dialog for 14 September, 16:10:04, cost $0.07. It lists three support calls in order. The first had a failed attempt with HTTP 529 and a successful retry; the second succeeded at once; the third and last call, at 16:10:42 with claude-sonnet-4-5, wrote the answer the customer rated. The highlighted "View call" button of that last call opens its AI action page.Step 3: the "Conversation turn" dialog for 14 September, 16:10:04, cost $0.07. It lists three support calls in order. The first had a failed attempt with HTTP 529 and a successful retry; the second succeeded at once; the third and last call, at 16:10:42 with claude-sonnet-4-5, wrote the answer the customer rated. The highlighted "View call" button of that last call opens its AI action page.
    The last call of a turn usually writes the answer; open it with View call.
    What this screenshot shows
    Screen
    Conversation turn (/admin/ai-logs)
    Highlighted
    button “View call”, dialog-last-call, enabled. opens /admin/ai-logs/<call id>
    Also visible
    Three support calls, first with 529 then success
    Purpose
    step 3, pick the call that answered

  4. Compare what the AI was given with what it said. On the AI action’s page, read Knowledge Base Context, Injected Skills and API Execution Results, then the answer under Response Data. If the right facts were there and the answer left them out, the fix is in the skill or the article. If the facts were missing, the knowledge base has a gap. Note the name of any tool that ran.

    Step 4 on the AI action page of that call. "Knowledge Base Context" shows that Billing FAQ.pdf (relevance 71%) was given to the AI, saying a plan change is prorated with a credit for the unused period. "Injected Skills" lists Plan changes. The highlighted "API Execution Results" section shows that the tool changeSubscriptionPlan ran successfully just before the answer. The answer itself did not mention the credit, which explains the customer's complaint.Step 4 on the AI action page of that call. "Knowledge Base Context" shows that Billing FAQ.pdf (relevance 71%) was given to the AI, saying a plan change is prorated with a credit for the unused period. "Injected Skills" lists Plan changes. The highlighted "API Execution Results" section shows that the tool changeSubscriptionPlan ran successfully just before the answer. The answer itself did not mention the credit, which explains the customer's complaint.
    Compare what the AI was given with what it answered; here the knowledge was there but the answer left the credit out.
    What this screenshot shows
    Screen
    Log Details (/admin/ai-logs/:id)
    Highlighted
    “”, api-execution-results-section, changeSubscriptionPlan succeeded. lists tool results the AI received
    Also visible
    Knowledge Base Context: Billing FAQ.pdf 71%; Injected Skills: Plan changes
    Purpose
    step 4, inspect inputs and tool results

  5. Find the tool run. Open AnalyticsTool runs, type the tool name into Search by tool name, endpoint, or status code... and select Search. The run from the same moment is the API call behind the answer.

    Step 5 on the Tool runs page. The highlighted search box "Search by tool name, endpoint, or status code..." contains changeSubscriptionPlan and the list shows the one matching run: a green 200 for the Demo Billing API tool changeSubscriptionPlan, PUT to the subscription's plan endpoint, at 14 September 16:10:41, seconds before the rated answer. Selecting the row opens the run with the exact request and response.Step 5 on the Tool runs page. The highlighted search box "Search by tool name, endpoint, or status code..." contains changeSubscriptionPlan and the list shows the one matching run: a green 200 for the Demo Billing API tool changeSubscriptionPlan, PUT to the subscription's plan endpoint, at 14 September 16:10:41, seconds before the rated answer. Selecting the row opens the run with the exact request and response.
    Search the tool runs by the tool name from the AI action to find the exact API call.
    What this screenshot shows
    Screen
    Tool runs (/admin/tool-execution-logs)
    Highlighted
    textbox “Search by tool name, endpoint, or status code...”, filter-card-left, changeSubscriptionPlan. filters runs by tool name
    Also visible
    One run: changeSubscriptionPlan PUT 200
    Purpose
    step 5, find the tool run

  6. Check the API call. Open the run. Compare Request Data with what your API expects, and read Response Data. If your API returned the right data, the problem is in the answer, not the toolkit. If the AI sent wrong values, or your API returned an error, the fix is in the toolkit.

    Step 6 on the tool run page. "Request Data" shows what the assistant sent to the plan endpoint of subscription sub_7Q2KD9: plan growth, billingPeriod annual. The highlighted "Response Data" shows what the API returned: the same subscription with effectiveFrom 2026-10-01 and proratedInvoice INV-2026-0533. The API did its job; the gap was in the answer, so the fix belongs in the skill or knowledge base, not in the toolkit.Step 6 on the tool run page. "Request Data" shows what the assistant sent to the plan endpoint of subscription sub_7Q2KD9: plan growth, billingPeriod annual. The highlighted "Response Data" shows what the API returned: the same subscription with effectiveFrom 2026-10-01 and proratedInvoice INV-2026-0533. The API did its job; the gap was in the answer, so the fix belongs in the skill or knowledge base, not in the toolkit.
    The request and response tell you whether the API or the answer was at fault.
    What this screenshot shows
    Screen
    Tool Execution Log Details (/admin/tool-execution-logs/:id)
    Highlighted
    “”, response-data, 200 with proratedInvoice. shows the API's answer as stored (masked by your rules)
    Also visible
    Request: sub_7Q2KD9, growth, annual
    Purpose
    step 6, check the API call

  7. Check what your guardrails blocked (optional). Open AnalyticsViolations. Ordinary questions in this list mean your company policy is too strict, and visitors get a refusal instead of help. Violations do not record the conversation, so compare the time instead.

    Step 7 on the Violations page ("Policy Violations"). The table lists the messages guardrails blocked, with the highlighted "Violation Reasons" column showing red badges such as data access, prompt injection, unsafe request: tax evasion and company policy violation for a cover-letter request. If ordinary questions show up here, the company policy is too strict; if harmful ones are missing, tighten it on the Security Guardrails page.Step 7 on the Violations page ("Policy Violations"). The table lists the messages guardrails blocked, with the highlighted "Violation Reasons" column showing red badges such as data access, prompt injection, unsafe request: tax evasion and company policy violation for a cover-letter request. If ordinary questions show up here, the company policy is too strict; if harmful ones are missing, tighten it on the Security Guardrails page.
    Check the Violations page too; blocked everyday questions point to a guardrails policy that is too strict.
    What this screenshot shows
    Screen
    Violations (/admin/policy-violations)
    Highlighted
    “”, table-header-reasons, ten rows. shows why each message was blocked
    Also visible
    Ten blocked messages with reasons, blocking response, version
    Purpose
    step 7, check what was blocked

  8. Fix the cause. Use the table in What happens next to decide where the change belongs, make it, and ask the same question in your widget to check the new answer.

Record the fix. Once the change is in, go back to the rating on AnalyticsFeedback. Set Triage Status to Actioned, note what you changed in Internal Notes and select Save Triage. Saving the triage needs the Admin role.

What happens next

Section titled “What happens next”

The trace tells you where the problem sits. Make the change in that place:

What you foundWhere the fix belongs
The right passage was in Knowledge Base Context but the answer ignored or garbled itThe skill that handles this kind of question, under ConfigureSkills
No passage, or only weak matches with a low RelevanceAn article in ConfigureKnowledge Base that answers the question
The expected skill is missing from Injected SkillsThe skill’s set-up under ConfigureSkills
The tool run shows wrong values in the request, or many Rejected before sending runsThe parameter descriptions and examples of the tool in ConfigureToolkits
The tool run shows Authorization (401/403)The toolkit’s credentials, or the customer’s authorisation
The tool run shows 5xx errors or timeoutsYour own API
Ordinary questions appear on ViolationsYour company policy under SecurityGuardrails

A change applies to new messages only. Earlier answers, ratings and logs stay as they were, so the old thumbs-down remains in the list. To check the fix, ask the same question in your widget and open the new AI action on AnalyticsAI actions. Over the following days, the Negative Feedback by Reason chart on DashboardPerformance should show fewer complaints of that kind.

A rating does not change the assistant on its own: nothing improves until you make the change.

If something goes wrong

Section titled “If something goes wrong”
SymptomWhat to check
The feedback list is emptyFeedback is off by default; switch on “Enable feedback” for the widget under ConfigureUser Interface. The list reaches back at most 14 days.
The detail panel shows only the answer text, not the AI actionNo matching AI action was found for that answer. Use the time of the rating to find the turn on Activity.
This turn is no longer available. when opening the turnYou need the Admin role. Without it, open the calls from the AI actions tab instead.
The turn is not on ActivityChoose a longer period; the tab reaches back at most 30 days.
Tool execution log not found when opening a tool runOpening tool runs needs the Admin role.
The search on Tool runs finds nothingSearch uses tool names, addresses and status codes, not message text. Copy the tool name exactly from the AI action.
Section titled “Related”