A multi-turn chatbot application intermittently produces responses that are grammatically correct and on-topic but contradict earlier turns in the conversation, creating a confusing user experience. CoherenceEvaluator measures exactly this: whether the flow of ideas across a multi-turn conversation is logically consistent and non-contradictory without requiring an external knowledge base. RelevanceEvaluator (option A) measures whether responses are topically on-point but often requires a reference context or knowledge base. SimilarityEvaluator (option B) requires a reference answer for comparison. QAEvaluator (option C) is a composite evaluator for question-answering tasks that requires a ground-truth context document. CoherenceEvaluator is the only option that works purely from the conversation history itself with no external knowledge base, perfectly matching the stated constraint and the multi-turn chatbot use case.

Microsoft Learn Reference Topic: Evaluate conversational AI applications in Microsoft Foundry – CoherenceEvaluator for multi-turn chatbots

For deployment failures when containers do not start, review deployment logs in Azure ML Studio or use the CLI get-logs command to surface dependency installation errors, missing files, or scoring script import failures. For authentication errors such as 401 or 403 responses, check that client applications are using valid authentication keys or bearer tokens, as mismatched or expired credentials cause auth failures. For performance degradation and slow responses, review compute instance scaling configuration, since a surge in traffic saturates a fixed number of replicas causing queuing and high latency without autoscaling. For internal server errors such as HTTP 500, these typically originate in the scoring script, so inspect the init and run functions for unhandled exceptions and verify the environment has all required packages installed.

Microsoft Learn Reference Topic: Troubleshoot online endpoint deployment failures – Azure Machine Learning

In Microsoft Foundry ' s built-in evaluation framework, each evaluator measures a specific dimension of language quality. Fluency evaluates the grammatical correctness, sentence structure, and overall linguistic smoothness of generated text. A high fluency score means the response reads naturally, with proper grammar, punctuation, and sentence construction — regardless of factual accuracy or relevance to the topic. Coherence (option A) measures whether ideas flow logically from one sentence to the next — it is about logical structure, not grammar. Textual Similarity (option B) measures how closely the generated text matches a reference text using metrics like ROUGE or BLEU — it is a comparison metric, not a standalone quality dimension. Groundedness (option C) measures whether claims are supported by the provided context — it is a factual fidelity metric. For grammatical correctness specifically, Fluency is the correct evaluator.

Microsoft Learn Reference Topic: Built-in AI evaluators in Microsoft Foundry – Fluency, Coherence, Groundedness, and Similarity