Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 23, 2026

Plan for AI Question Minimum Length Validation

  • Explore the codebase to understand the AI question validation flow
  • Add minimum length constant (10 characters) to constants.ts
  • Implement validation in AskAIOverlay.tsx to check question length before adding
  • Update selectedQuestions logic to filter out questions shorter than 10 characters
  • Add user-friendly error message string to constants.ts
  • Show validation feedback when questions are too short
  • Simplify validation logic based on code review feedback
  • Optimize validation using Array.some() for performance
  • Add validation during evaluation (after questions are added to criteria)
  • Fix lf function usage in validation file
  • Add inline validation with red border and error message for custom question inputs
  • Optimize validation code to avoid redundant function calls
  • Pass MinAIQuestionLength to lf at string definition time
  • Fix type error in classList usage
  • Fix bundling error by moving Misc namespace to top of file
  • Show validation always when text exists (not just when checked)
  • Run security checks
  • Final validation complete

Summary

This PR adds a minimum length validation of 10 characters for AI questions in the teachertool at three points:

  1. Inline validation in input fields: Shows red border and validation message when text is too short
  2. Before adding to checklist: Prevents users from adding short questions via the Ask AI dialog
  3. During evaluation: Validates questions when they are evaluated, showing the same friendly error message

Now questions shorter than 10 characters will:

  • Show a red border (2px) around the text input and display "Question must be at least 10 characters" inline whenever text exists (regardless of checkbox state)
  • Show an error toast when trying to add them to the checklist
  • Display "Question must be at least 10 characters" error when evaluated instead of "Unable to reach the AI service"

The error string is now properly localized at definition time by passing the minimum length value to the lf function. All type errors and bundling issues have been resolved.

Original prompt

This section details on the original issue you should resolve

<issue_title>AI service cannot be reached when entering simple inputs</issue_title>
<issue_description>Describe the bug
When using the Ask AI feature on the MakeCode Arcade Code Evaluation page, single-character inputs such as “j” or “g” consistently trigger an error.

Steps to reproduce the behavior:
1.Navigate to https://arcade.makecode.com/beta--eval.
2.In Checklist section, click "Ask AI".
3.In the Custom tab, enter a single-character question (e.g., “j” or “g”).
4.Click Add Selected, then click Evaluate.
5.Observe the system’s response.

Actual behavior
The system immediately displays:“Unable to evaluate”, “Unable to reach the AI service”.

Expected behavior
If the input is too short, the system should display a friendly validation message (e.g., “Please enter a longer question”), not a service-reachability error.

Screenshots
Image

Additional context
1.OS: Windows 11 24H2 & Mac mini M2, Tahoe 26.2
2.Browser: Microsoft Edge, Chrome, Firefox and Safari
3.Arcade version : 2.1.28
4.Microsoft MakeCode version: 12.2.18</issue_description>

<agent_instructions>make ai questions in the teachertool have a minimum length of 10 characters before letting them through, because naything less than that will just get the default "must be asking about computer science education" type response anyways</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@abchatra @jwunderl worth fixing? @jwunderl Huh, 2 characters works, 1 does not. I'm more inclined towards adding a minimum length for the question in the first place, anything less than a few words is just going to end up like this anyways / waste user & server time

Image</comment_new>


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Fix AI service reachability for single-character inputs Add 10-character minimum length validation for AI questions Jan 23, 2026
Copilot AI requested a review from jwunderl January 23, 2026 00:09
@jwunderl
Copy link
Member

@copilot this should also apply after the question has been added to the criteria

Copy link
Contributor Author

Copilot AI commented Jan 23, 2026

@copilot this should also apply after the question has been added to the criteria

Added validation during evaluation in commit 47a063e. Questions shorter than 10 characters will now show "Question must be at least 10 characters" error when evaluated, preventing the "Unable to reach the AI service" message.

Copilot AI changed the title Add 10-character minimum length validation for AI questions Add 10-character minimum length validation for AI questions at input and evaluation Jan 23, 2026
@jwunderl
Copy link
Member

@copilot Give it a warning that turns the text input red unless > 10 characters still, currently it just says "value Required" when empty

Copy link
Contributor Author

Copilot AI commented Jan 23, 2026

@copilot Give it a warning that turns the text input red unless > 10 characters still, currently it just says "value Required" when empty

Added inline validation in commits c16721a and 7e8a14d. Custom question text inputs now:

  • Show a red 2px border when checked and text is < 10 characters
  • Display validation message below the input: "Value Required" (empty) or "Question must be at least 10 characters" (too short)
  • Optimized to avoid redundant function calls during render

Copilot AI changed the title Add 10-character minimum length validation for AI questions at input and evaluation Add 10-character minimum length validation for AI questions with inline visual feedback Jan 23, 2026
export const UnableToReachAI = lf("Unable to reach the AI service");
export const UnexpectedError = lf("An unexpected error occurred");
export const Dismiss = lf("Dismiss");
export const QuestionTooShort = lf("Question must be at least {0} characters");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot this should be lf("Question must be at least {0} characters", Misc.MinAIQuestionLength);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit dfe5b60. Changed QuestionTooShort definition to pass the value to lf at definition time: lf("Question must be at least {0} characters", Misc.MinAIQuestionLength). Updated all usages to reference Strings.QuestionTooShort directly without runtime parameter substitution.

Copilot AI requested a review from jwunderl January 23, 2026 19:38
<Textarea
id={`ask-ai-custom-text-${p.id}`}
className={css["textarea"]}
className={classList(css["textarea"], hasError && css["textarea-error"])}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
className={classList(css["textarea"], hasError && css["textarea-error"])}
className={classList(css["textarea"], hasError ? css["textarea-error"] : undefined)}

@copilot has a type error as written

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit 506034f. Changed to use ternary operator: hasError ? css["textarea-error"] : undefined to avoid type error from boolean false value.

Copilot AI requested a review from jwunderl January 23, 2026 21:19
namespace Misc {
export const LearnMoreLink = "https://makecode.microbit.org/teachertool"; // TODO: Replace with golink or aka.ms link
export const ExperienceId = "code-eval";
export const MinAIQuestionLength = 10;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot this appears to have a bundling error and crashes on use before defined; move misc namespace to top of file to work around

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit a7375aa. Moved Misc namespace to the top of the file before Strings namespace to resolve the bundling error.

const inputs = plan.checks[0] as pxt.blocks.AiQuestionValidatorCheck;
logDebug(`Asking question: '${inputs.question}' on project with shareId: '${inputs.shareId}'`);
const response = await askCopilotQuestionAsync(inputs.shareId, inputs.question);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot this is a good warning on unable to evaluate, but it would be good if the 'ask ai' input itself was marked as needing attention (red / warning icon) until enough text was entered

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit a7375aa. The 'ask ai' input now shows validation immediately when text exists but is too short. The red border and validation message ("Question must be at least 10 characters") display as soon as the user types, regardless of checkbox state, providing clear visual feedback about why the question cannot be added or evaluated.

{customPrompts.length > 0 && (
<div className={css["custom-list"]}>
{customPrompts.map((p, index) => (
{customPrompts.map((p, index) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot there's no indication of why 'add selected' won't work right now; indicate on the question itself that it needs more text to it to be added

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit a7375aa. Validation messages now appear immediately below each question input when text is too short, clearly indicating why 'add selected' won't work. Users see either "Value Required" (empty) or "Question must be at least 10 characters" (1-9 chars) with a red border around the input.

Copilot AI requested a review from jwunderl January 23, 2026 21:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AI service cannot be reached when entering simple inputs

2 participants