Design and implement effective prompting strategies...

Q: In an LLM-based agent, what is the full agent loop (name and briefly describe each stage), and what is the prompting goal within that loop?

The agent loop is: Perceive : receive the user request or an observation. Reason : interpret the situation and determine the next step. Act : answer, call a tool, or perform another action. Observe : inspect the result of that action. Repeat : continue, correct the plan, or terminate. Prompting goal within the loop : make the reasoning and action stages more reliable (e.g., better planning, correct tool use, and correction after errors), not merely produce fluent text.

Q: When evaluating whether a prompt is “good” for agentic behaviour, what criterion should you use, and what common but insufficient criterion should you avoid?

Use effect on behaviour as the evaluation criterion: whether the prompt produces reliable agent behaviour (e.g., a correct plan, a valid tool call, or successful error recovery and replanning). Avoid judging a prompt only by whether it produces fluent text ; fluency alone does not indicate reliable reasoning/actions in an agent loop.

Q: For agentic prompting, what distinct functions do system prompts , few-shot prompting , and chain-of-thought/step-by-step instructions serve, and how do they work together?

System prompts : establish the overall policy for the agent (how it should generally behave). Few-shot prompting : provides examples that demonstrate the desired pattern of behaviour in context. Chain-of-thought / step-by-step instructions : encourage planning before action to improve multi-step task performance. They often work together by setting the policy (system), showing the pattern (few-shot), and inducing explicit planning that supports tool use, inspection of results, and correction (step-by-step).

Q: You are designing a prompt for an LLM-based agent (not a single-turn answer). What elements should the prompt specify to support agentic behaviour?

A prompt should specify: Role : what the agent is (its function/identity for the task). Goal : what outcome it must achieve. Constraints : limits and rules it must follow. Tools : what tools are available and that the agent may call them. Output format : how responses/actions should be structured. These details help the model plan actions, use tools appropriately, and produce consistent outputs.

Q: Beyond “giving step-by-step reasoning,” what agent capabilities should combining system prompts, few-shot examples, and step-by-step instructions help elicit?

Combining these methods should elicit: Planning : forming a sensible plan before acting. Tool use : choosing to call a tool as an action when appropriate. Error recovery : detecting that an action/tool result is wrong or unhelpful and correcting course. Replanning : updating the plan after observing results, then continuing or terminating appropriately. The combination is meant to improve multi-step reliability across the agent loop, not just produce an initial plan.

Q: What is not required by this learning objective about tools/memory, and what is required regarding how the LLM uses available components?

Not required : building every external tool or memory component. Required : prompting the LLM so it can use the available components appropriately (e.g., selecting tools when needed and acting coherently in the agent loop). So the focus is on prompt design and its behavioural effects, not implementing the external infrastructure itself.

Q: To demonstrate mastery (e.g., in an exam) of system prompts, few-shot prompting, and chain-of-thought prompting for agents, what should you do besides defining each term?

You should show how each strategy changes the agent’s behaviour and connect the prompt to an observable outcome . Acceptable outcomes include: a correct plan a valid tool call successful recovery after failure (including correcting the plan and continuing appropriately) Definitions alone are insufficient; the evidence is in agent performance.

Q: In prompting for agentic behaviour, what are the three core prompting strategies (system prompt, chain-of-thought prompt, few-shot prompt), and what does each one control?

System prompt A high-priority instruction that defines the agent’s overall policy and identity for the whole interaction (role, objective, limitations, tools, decision rules, safety constraints, required response format). Chain-of-thought prompt Instructs the model to reason step by step (“think before you act”), encouraging decomposition and sequential reasoning. Few-shot prompt Provides multiple input → desired output examples that demonstrate the exact behaviour to imitate (including tool selection, output structure, error handling) via in-context learning.

Q: What makes a system prompt “high-priority,” and what kinds of constraints and specifications can it define for an agent?

A system prompt is high-priority because it sets how the agent should behave throughout the interaction (its general policy). It can specify: Role/objective (e.g., “teaching assistant” or “research assistant”). Limitations and safety constraints . Available tools and decision rules (e.g., when tool use is allowed). Planning requirement (e.g., must plan before acting). Required response format (e.g., tool calls must be returned in a specified JSON format).

In an LLM-based agent, what is the full agent loop (name and briefly describe each stage), and what is the prompting goal within that loop?

The agent loop is:

Perceive: receive the user request or an observation.
Reason: interpret the situation and determine the next step.
Act: answer, call a tool, or perform another action.
Observe: inspect the result of that action.
Repeat: continue, correct the plan, or terminate.

Prompting goal within the loop: make the reasoning and action stages more reliable (e.g., better planning, correct tool use, and correction after errors), not merely produce fluent text.

When evaluating whether a prompt is “good” for agentic behaviour, what criterion should you use, and what common but insufficient criterion should you avoid?

Use effect on behaviour as the evaluation criterion: whether the prompt produces reliable agent behaviour (e.g., a correct plan, a valid tool call, or successful error recovery and replanning). Avoid judging a prompt only by whether it produces fluent text; fluency alone does not indicate reliable reasoning/actions in an agent loop.

For agentic prompting, what distinct functions do system prompts, few-shot prompting, and chain-of-thought/step-by-step instructions serve, and how do they work together?

System prompts: establish the overall policy for the agent (how it should generally behave).
Few-shot prompting: provides examples that demonstrate the desired pattern of behaviour in context.
Chain-of-thought / step-by-step instructions: encourage planning before action to improve multi-step task performance.

They often work together by setting the policy (system), showing the pattern (few-shot), and inducing explicit planning that supports tool use, inspection of results, and correction (step-by-step).

You are designing a prompt for an LLM-based agent (not a single-turn answer). What elements should the prompt specify to support agentic behaviour?

A prompt should specify:

Role: what the agent is (its function/identity for the task).
Goal: what outcome it must achieve.
Constraints: limits and rules it must follow.
Tools: what tools are available and that the agent may call them.
Output format: how responses/actions should be structured.

These details help the model plan actions, use tools appropriately, and produce consistent outputs.

Beyond “giving step-by-step reasoning,” what agent capabilities should combining system prompts, few-shot examples, and step-by-step instructions help elicit?

Combining these methods should elicit:

Planning: forming a sensible plan before acting.
Tool use: choosing to call a tool as an action when appropriate.
Error recovery: detecting that an action/tool result is wrong or unhelpful and correcting course.
Replanning: updating the plan after observing results, then continuing or terminating appropriately.

The combination is meant to improve multi-step reliability across the agent loop, not just produce an initial plan.

What is not required by this learning objective about tools/memory, and what is required regarding how the LLM uses available components?

Not required: building every external tool or memory component.
Required: prompting the LLM so it can use the available components appropriately (e.g., selecting tools when needed and acting coherently in the agent loop).

So the focus is on prompt design and its behavioural effects, not implementing the external infrastructure itself.

To demonstrate mastery (e.g., in an exam) of system prompts, few-shot prompting, and chain-of-thought prompting for agents, what should you do besides defining each term?

You should show how each strategy changes the agent’s behaviour and connect the prompt to an observable outcome. Acceptable outcomes include:

a correct plan
a valid tool call
successful recovery after failure (including correcting the plan and continuing appropriately)

Definitions alone are insufficient; the evidence is in agent performance.

In prompting for agentic behaviour, what are the three core prompting strategies (system prompt, chain-of-thought prompt, few-shot prompt), and what does each one control?

System prompt
- A high-priority instruction that defines the agent’s overall policy and identity for the whole interaction (role, objective, limitations, tools, decision rules, safety constraints, required response format).
Chain-of-thought prompt
- Instructs the model to reason step by step (“think before you act”), encouraging decomposition and sequential reasoning.
Few-shot prompt
- Provides multiple input → desired output examples that demonstrate the exact behaviour to imitate (including tool selection, output structure, error handling) via in-context learning.

What makes a system prompt “high-priority,” and what kinds of constraints and specifications can it define for an agent?

A system prompt is high-priority because it sets how the agent should behave throughout the interaction (its general policy). It can specify:

Role/objective (e.g., “teaching assistant” or “research assistant”).
Limitations and safety constraints.
Available tools and decision rules (e.g., when tool use is allowed).
Planning requirement (e.g., must plan before acting).
Required response format (e.g., tool calls must be returned in a specified JSON format).

When is a chain-of-thought instruction especially useful for agentic tasks, and what “simplest form of planning” does it represent?

A chain-of-thought instruction is especially useful when a task has dependencies, where later actions depend on earlier steps (e.g., searching for papers before categorizing them, or reading a file before calculating its average). It represents the simplest form of planning: “think before you act”, i.e., reason through the problem step by step before answering or taking an action.

What is a few-shot prompt in this context, and what must its examples demonstrate to elicit reliable agentic behaviour?

A few-shot prompt contains several examples of inputs and desired outputs that the model follows as a temporary pattern. This is in-context learning, meaning the model adapts in the current context without updating its model weights. To elicit agentic behaviour, examples should demonstrate the exact behaviour expected, including:

Correct tool selection
Output structure
Error handling

Why can prompting fail when the system prompt, few-shot examples, and planning instruction are inconsistent? Give the two specific inconsistency cases described.

Prompting can fail because the model receives conflicting signals about what to do. Two described cases are:

Format conflict: the system prompt requests JSON, but the few-shot examples contain prose, weakening the format instruction.
Planning conflict: the prompt asks for planning, but the examples show immediate answers, so the examples may weaken the planning instruction.

A “combined use” of the three strategies is recommended for agentic behaviour. What does combined use achieve that each strategy alone does not?

Combined use works because each component controls a different part of behaviour, and together they reinforce a full agent loop:

System prompt: defines the agent’s behaviour, identity, and constraints.
Few-shot prompting: demonstrates the desired input-output pattern (including tool calls and follow-up actions).
Chain-of-thought instruction: encourages the reasoning/decomposition process needed to execute the behaviour.

When aligned, these signals help the model move from a vague request to a structured perceive, reason, act, and observe loop.

Integration reasoning: How does combining system prompt + step-by-step (chain-of-thought) instruction + few-shot examples increase the likelihood of agentic behaviour for a research assistant?

Combining them aligns multiple control signals:

System prompt establishes the role and constraints (what the agent is and must obey).
Step-by-step instruction makes the agent decompose the task before acting (planning/“think before you act”).
Few-shot examples show what a correct plan, tool call, and follow-up action look like.

Because these signals are aligned, the model is more likely to perform a structured loop: perceive, reason, act, and observe.

In prompting an LLM for agentic behaviour, what does planning mean, and why is it considered “one of the most agentic capabilities”?

Planning means decomposing a complex task into manageable subtasks and deciding their order.

It is “one of the most agentic capabilities” because the system is deciding about future actions (what to do next and in what sequence), rather than merely responding to the present input.
A planning-oriented prompt shifts the model from immediately producing an output (e.g., starting to write) to first constructing an actionable plan that guides subsequent steps.

For prompting a multi-step task, what prompt structure should you use to force explicit task decomposition and controlled execution?

Use a prompt that enforces a plan-before-act workflow:

State the final goal clearly.
Ask the model to identify the required subtasks.
Ask it to determine dependencies between subtasks.
Require it to select the next action only after forming the plan.
Require it to inspect each result before continuing.
Allow it to revise the plan when an action fails.

This structure elicits agentic behaviour by turning generation into sequenced action selection with checks.

What are the three levels in the planning hierarchy, and what is each level responsible for?

Planning can operate at three explicit levels:

Strategic planning
- Defines the overall goal (e.g., writing a literature review).
Tactical planning
- Divides the goal into subtasks (e.g., searching, categorizing, drafting).
Operational planning
- Specifies the immediate action (e.g., using particular search terms in an academic database).

Why should a prompt make planning levels and dependencies explicit on complex tasks, and what kinds of planning failures can happen if it does not?

A complex-task prompt should make planning levels and dependencies explicit because otherwise the model may produce a plausible plan with hidden dependency errors.

Examples of dependency errors include:
- Trying to identify trends before collecting enough papers.
- Writing conclusions before gathering evidence.
The lecture emphasizes that LLMs are inconsistent planners and may omit critical steps or create circular dependencies, so explicit prompting reduces these risks.

What is replanning in agentic prompting, and what specific instruction can you add to trigger replanning after each action?

Replanning is the ability to change the original plan when new information shows that it is failing.

A useful instruction is: “After every action, compare the result with the required information. If the result is empty, incorrect, or insufficient, explain the problem briefly and choose an alternative action.”
This makes the model inspect outcomes, diagnose failure (empty/incorrect/insufficient), and then select a new action instead of persisting.

Apply replanning to this situation: Goal = find academic papers about media bias detection. Initial action = search for “media bias.” Observation = search returns irrelevant results. What revised action and next action should the plan choose?

The plan should replan by narrowing the query and then continuing with downstream work:

Revised action: use a narrower query such as “automated news media bias classification”.
New observation: relevant papers are found.
Next action: categorize the papers by methodology.

This demonstrates agentic behaviour: observe failure → change action → proceed once results are relevant.

What prompt details specifically support planning and reduce false “task completion” when an LLM is acting agentically?

Prompt details that support planning include:

Specify whether the model should plan once or revise the plan iteratively.
Tell it what counts as a successful result.
Define what it should do when a tool returns no result.
Require it to distinguish completed subtasks from pending subtasks.
Prevent it from claiming completion without checking the evidence.

Together, these constraints push the model to track progress and verify outputs rather than “finishing” prematurely.

When prompting an agent to complete a complex task (e.g., a literature review), how does instructing it to plan, inspect results, and replan after failure elicit agentic behaviour?

For complex goals, task decomposition is needed because the goal cannot usually be completed reliably in one action.

In a literature review, subtasks have dependencies:
- Scope must be defined before searching.
- Evidence must be collected before themes are identified.
- Drafting should follow categorization.
By requiring the model to plan, inspect results, and replan after failure, the prompt converts language generation into adaptive action selection, which is the targeted agentic behaviour.

When prompting an LLM to use external tools agentically, what tool information must the prompt include so the model can decide when to call a tool and generate correct arguments?

The prompt should describe each tool so the model can coordinate tool use (decide when to search/run code/retrieve info) and fill arguments correctly:

Purpose
- What the tool is for (what problem it solves).
Input parameters
- What parameters are required.
Parameter types
- What kinds of values each parameter expects.
Limitations
- Constraints the tool operates under.
When to use vs. not use
- Situations where a tool call is appropriate or inappropriate.

What is “structured output” in tool prompting, and why must a prompt make the tool-call format unambiguous and parseable?

Structured output is machine-readable output with a fixed format (often JSON or a function call). The prompt must make this format unambiguous and parseable because an agent program may fail when the format is invalid, even if the model’s underlying idea or reasoning was correct. The key exam focus is not exact syntax, but producing output that can be reliably parsed.

In a generic JSON tool-call structure, what are the required top-level fields and what does each represent?

A generic tool call can be represented as:

{ "tool": "tool_name", "arguments": {...} }

Meaning:

"tool"
- Identifies which function/tool to call (the tool’s name).
"arguments"
- Contains the required parameter values passed to that tool.

For a file-analysis agent with tools read_file(filepath) and calculate_average(numbers), what constraints and sequencing must the prompt enforce to elicit reliable multi-tool behavior?

The prompt should enforce both constraints and order:

Tool-use constraints
- Use readfile(filepath) only for files inside /safe/data/.
- Use calculateaverage(numbers) for arithmetic.
Required sequence
- First read the file.
- Then inspect the contents.
- Then call the calculator using the observed contents.
Tool-call correctness
- Every tool call must contain the tool name and valid arguments.

What error-recovery policy should be explicitly prompted for tool-using agents, and what failures does it prevent?

A strong tool-use prompt should include explicit error recovery rules:

Never treat an error message as a successful result.
Classify the cause: invalid argument, missing information, unavailable tool, or permission failure.
Try a justified alternative when one exists (e.g., modify arguments or select another tool).
Do not invent data after a failed tool call.
Stop and report the limitation when no safe alternative exists.

This prevents repeating the same failed action, inventing a successful result, or continuing with missing data.

How should a system prompt integrate tool use, structured output, and recovery policy to elicit agentic behaviour (as illustrated by the file-reading example)?

In the system prompt, specify:

Tool descriptions
- Purpose, valid arguments, limitations, and when to use each tool.
Output format
- Define a structured, parseable tool-call format so the agent program can reliably execute calls.
Recovery policy
- Rules for inspecting tool errors, diagnosing likely causes, and choosing safe alternatives or stopping.
Observation → next action coupling
- In multi-tool work, instruct the agent to use each observation (e.g., file contents or an error) to determine the next action (e.g., then calculate, or report/adjust on permission error).

You are implementing a prompt for an agentic LLM workflow. What is the recommended end-to-end prompt implementation workflow, from initial definition through revision?

Define the agent’s goal and allowed actions.
- Clarify what the agent is trying to achieve and what it is permitted to do.
Write the system prompt specifying:
- role, constraints, tools, planning rules, and output format.
Add a small set of consistent few-shot examples.
Add a step-by-step planning or reasoning instruction.
Test the prompt on normal and difficult tasks.
Record observable behaviours:
- tool choices, output validity, task completion, recovery behaviour.
Revise the prompt based on observed failures.

When evaluating whether a prompt elicits agentic behaviour, what should evaluation measure beyond “the final answer,” and why?

Evaluation should measure the entire agent process, not just a fluent final response, because LLM behaviour is variable and prompt-sensitive and a prompt that works once may fail on other inputs (especially with long-context tracking, structured output, or dependent actions). Useful measures include whether the agent:

Follows all instructions
Chooses the correct tool
Returns valid JSON
Remembers relevant context (context tracking)
Adapts after tool failure (error recovery)
Avoids hallucinating unsupported information (hallucination rate)

Distinguish weak evaluation vs strong evaluation for an agent prompt, including what each tends to conclude.

Weak evaluation
- Meaning: inspect one final answer and decide whether it “sounds good.”
- Typical conclusion: “The prompt works.”
Strong evaluation
- Meaning: test multiple tasks and measure: planning, tool selection, structured-output validity, error recovery, context tracking, and hallucination rate.
- Typical conclusion: quantified performance, e.g. “The prompt produced valid JSON in 19 of 20 trials, selected the correct tool in 9 of 10 scenarios, and recovered from 4 of 5 simulated failures.”

You want few-shot examples to improve an agent prompt. Which types of cases should the examples represent, and when should you include failure cases?

Few-shot examples should represent:

Normal cases
- Typical inputs the agent should handle reliably.
Ambiguous cases
- Inputs where interpretation could vary, so the agent learns the intended behaviour.
Failure cases (when relevant)
- Include these when error recovery is important, so the agent practices what to do when things go wrong.

In a prompt experiment comparing two file-analysis prompts, what differs between “Version A” and “Version B,” and what test set should you run to compare them?

Version A
- Minimal instruction: “Read the file and calculate the average.”
Version B
- Defines: role, tool descriptions, action order, JSON format, error handling, and a few examples.
Test set to run on both versions:
- valid files
- missing files
- malformed contents
- restricted paths

In the same file-analysis prompt experiment, what outcome measures should you compare across prompt versions to detect stronger agentic reliability (not just fluency)?

Compare observable reliability outcomes, including:

Number of correct tool sequences
- Whether the agent uses tools in an appropriate order to complete the task.
Valid outputs
- Whether the output meets the required structured-output constraints (e.g., validity).
Unsupported claims
- How often the agent makes claims not supported by available information (hallucination-like behaviour).

To determine whether an observed improvement is due to the prompt (rather than the model or random variation), what additional comparisons/controls does the practical exercise recommend running?

Recommended experiments include:

Compare models on the same multi-step problem.
Vary temperature and observe changes in behaviour.
Test context sensitivity.
Measure structured-output reliability (e.g., repeated structured-output requests).
Track token cost.

These help attribute performance changes to the prompt vs the model vs random variation.

Give examples of practical capability tests for evaluating an agent prompt’s behaviour across tasks (not just one trial).

Practical capability tests include:

Complex instruction-following trials
Tool-selection scenarios
Repeated JSON requests
Simulated tool failures
Long-context recall
Factual questions

These probe planning, tool selection, structured-output validity, recovery, context tracking, and hallucination risk.

Why is prompting described as a way to augment an LLM agent’s reasoning capability, and what kinds of agent failures does weak reasoning cause?

Reasoning is a fundamental bottleneck for agent performance, so prompting aims to improve how the model reasons about actions.
When the model cannot reason about which tool to use, the agent may call the wrong tool.
When it cannot plan a multi-step strategy, it may take locally sensible actions that do not achieve the overall goal.
When it hallucinates, it may act on false premises.

What limitations can still remain even with good prompting for LLM agents, and what additional measures are still necessary?

Prompting does not remove model limitations. Even with good prompts, LLM agents may still:
- Hallucinate.
- Lose information in long contexts.
- Respond differently to small prompt changes.
- Produce plausible-looking but flawed plans.
Good prompting reduces risk, but evaluation and external safeguards remain necessary.

When explaining “how prompting elicits agentic behaviour,” how should you integrate system prompts, few-shot examples, step-by-step reasoning, and tool/recovery instructions into a single reasoning-action loop explanation (rather than isolated tricks)?

System instructions establish the agent’s rules/policy.
Few-shot examples demonstrate the desired pattern, making that policy concrete.
Step-by-step reasoning channels the LLM toward planning before action, reducing premature actions.
Tool and recovery instructions connect the model’s reasoning to the environment:
- Tool and output rules make actions executable.
- Observations and error rules enable correction after acting.
Together, this explains how prompt design can turn a general language model into a more capable but still fallible agent.

You observe an LLM agent that “takes locally sensible actions that do not achieve the overall goal.” Which prompt element in the causal chain most directly targets this failure mode, and why?

Step-by-step planning most directly targets this problem.
Why: the failure is described as lacking a multi-step strategy, which leads to actions that seem reasonable in the moment but miss the overall goal.
Adding step-by-step planning is meant to reduce premature action by forcing the agent to plan before acting.

When justifying a prompt design for an agent, what kind of explanation is considered strong, and what kind is considered weak?

Strong justification
- Links each prompt element to an expected agent behaviour and explains the mechanism.
- Example mechanisms supported: a planning instruction should reduce premature action; a tool example should make the correct function and argument structure more likely.
Weak justification
- Makes unsupported claims like a prompt “improves reasoning” without evidence or without describing how it changes behaviour.

You are designing a prompt for an agent that uses tools. What prompt components must you specify to avoid underspecified tool use and underspecified outputs?

Tools, parameters, and restrictions
- Explicitly list the available tools, their parameters, and any restrictions so the agent’s tool calls are valid and constrained.
Valid output format/structure
- State the required output format so the agent produces an output that can be checked and used reliably.
Few-shot examples demonstrating tool calls
- Provide examples that show correct tool calls (including argument structure) and what to do when recovery is needed.

What instructions should a prompt include to support agentic behaviour after a tool call returns an observation, and why?

Observation inspection + replanning instruction
- The prompt should specify how the agent should inspect observations (tool results) and when/how to replan.
Reason this matters
- It prevents the agent from “completing” the task without integrating tool results, and supports recovery when observations indicate failure or mismatch.

What are the “common pitfalls” to avoid when designing a prompt for an agent (list them all)?

Confusing prompt levels
- Confusing a system prompt with a single user request.
Internal inconsistency
- Giving examples that contradict the written instructions.
Unanchored step-by-step
- Asking for step-by-step reasoning but not specifying what action should follow.
Missing specs
- Omitting tool parameters or valid output structure.
No verification
- Allowing the agent to claim success without checking tool results.
Weak testing
- Testing only one easy example.
Measuring the wrong things
- Measuring only answer quality while ignoring tool calls and recovery.
Overtrusting model size/fluency
- Assuming a larger or more fluent model automatically makes a reliable agent.

You are asked to propose how you would evaluate a designed agent prompt. What testing guidance should you include, and what should you measure besides answer quality?

Test both normal and failure cases
- Explain how you would test the prompt on normal cases and failure cases (not just one easy example).
Measure behaviour beyond final answers
- Do not measure only answer quality; also evaluate tool calls and recovery behaviour.

Use the exam “reasoning pattern” to explain a prompting intervention: what are the four links in the causal chain you should write, in order?

The required causal chain is:

Prompt feature: identify the instruction or example you added (what the prompt contains).
Intended model behaviour: explain what the model should do differently because of that feature.
Agent consequence: explain how that behavioural change affects planning, tool use, recovery, or task completion.
Evaluation: state how you would measure whether the change happened.

This structure forces you to move from prompt wording to observable agent behaviour and then to a testable measurement plan.

A few-shot example shows a failed search followed by a narrower query. Using the causal-chain pattern, explain why this should improve agentic performance and how you would test it.

Prompt feature: a few-shot example that includes a failed search and then a narrower revised query.
Intended model behaviour: when the agent observes an empty result, it should revise the query rather than stopping and completing the task.
Agent consequence: this should improve replanning and recovery, increasing the chance of completing the task successfully after a failure.
Evaluation: simulate failed searches and measure whether the agent selects a justified alternative (i.e., performs query revision and continues) rather than ending the interaction.

If an exam question asks for implementation vs evaluation of a prompting strategy for agentic behaviour, what should your answer include in each case?

If asked for implementation:
- Include a realistic prompt structure (not just definitions).
- Explain how that wording should improve the agent loop (prompt design → observable agent behaviour).
If asked for evaluation:
- Measure the complete interaction, including reasoning, actions, observations, and recovery.
- Use repeated tests and relevant metrics as part of an evaluation plan.

Why does a high-quality exam answer need to “move from prompt design to observable agent behaviour,” and what extra elements should be added to make that link credible?

The exam expects you to justify why specific prompt wording should change the agent loop, not merely describe prompt components. To make the link credible, include:

An explanation of the expected effect on agent behaviour
- Tie prompt features to changes in planning, tool use, recovery, or task completion.
An evaluation plan with repeated tests and relevant metrics
- States how the behavioural change would be measured.
A discussion of limitations and possible failure modes
- Shows awareness of where the prompt may not work or may break down.

Design and implement effective prompting strategies (chain-of-thought, few-shot, system prompts) to elicit agentic behaviour from language models

49 important questions on Design and implement effective prompting strategies (chain-of-thought, few-shot, system prompts) to elicit agentic behaviour from language models