For prompt design vs fine-tuning, I would first identify the failure. If the model has the right information but follows the wrong instructions, I would improve the prompt and examples. If it lacks current business facts, I would improve retrieval. I would consider fine-tuning only when a supported model, training data, and evaluation justify it.

These approaches can work together. They are different ways to change the information and behavior behind an answer, not a fixed ladder where every project must eventually reach fine-tuning.

Prompt design vs fine-tuning vs RAG: the decision table

Observed problem	First intervention I would try	What I would measure
The answer uses the wrong tone or format	Clear instructions and examples	Format and instruction compliance
The answer lacks a current policy or private fact	A maintained knowledge base or retrieval	Source retrieval and factual correctness
The workflow repeats a specialized behavior across many cases	Prompt baseline, then a supported fine-tuning option if warranted	Held-out task accuracy, latency, and operating cost
The agent must change a record or schedule a task	A tool or Action with defined permissions	Correct tool use and verified outcome
The answer is correct but too slow or expensive	Prompt length, output length, model choice, and routing	Cost and latency per successful task

Start with a prompt I can evaluate

I would give the model a job, the information it can use, a clear output format, and a rule for missing information. I would include a few examples where the expected response is unambiguous.

Role: Help customers understand the supplied delivery policy.
Sources: Use only the attached policy for delivery promises.
Workflow: Ask for the destination if it is missing. Find the relevant policy section. Answer briefly and identify the source.
Boundary: Do not invent delivery dates or change an order. If the policy does not answer the question, say what is missing and route to support.
Output: Answer, source, and next step.

This is an illustrative starting prompt. I would test it against actual customer questions before deployment, including questions the policy cannot answer.

Anthropic's prompting overview emphasizes success criteria and empirical evaluation. OpenAI's prompting guide describes instructions, examples, and relevant context.

Use retrieval when the missing piece is knowledge

A model cannot reliably infer my current return policy from general training. I would supply the approved policy and retrieve the section relevant to the question.

Retrieval-augmented generation, or RAG, brings selected source material into the request. It does not retrain the model's weights.

I would test retrieval separately from the answer. If the relevant section never reached the model, rewriting the answer instructions may hide the real problem.

Our knowledge base guide explains source preparation and testing. The Pickaxe upload walkthrough shows the product steps, and Knowledge Base documentation covers the current interface.

When I would consider fine-tuning

I would consider it when the task has a recurring behavior that a good prompt still handles inconsistently, and I have enough reviewed examples to train and evaluate a supported model.

I would keep a separate held-out set that is not part of training. Otherwise, a successful score may tell me that the model learned the examples rather than that it handles new work.

Availability is provider-specific. I would check which models can be trained, the platform's current access rules, expected support lifetime, and how I will serve the result.

For example, OpenAI's current optimization guidance says its fine-tuning platform is winding down and is closed to new users. I would not plan a new project around an old tutorial's assumption that anyone can start a training job.

Fine-tuning also does not remove the need for current sources or Actions. A trained response style is different from knowing today's order status or having permission to update it.

A worked example: a support agent

This is a hypothetical example. Suppose an agent answers delivery questions but fails in three ways: it invents a delivery window, writes a long answer, and promises to change an order.

1. Invented delivery window: attach the current delivery policy and verify the relevant section is retrieved.
2. Long answer: specify the short format and show a few examples.
3. Unauthorized change: define the boundary and provide an approved escalation path instead of implying an action occurred.

I would run the original and revised versions on the same questions. If the prompt and sources solve the failures, a training project adds little value.

If a repeated behavior still fails after that baseline, I would compare other models and supported training options. Our LLM selection guide explains evaluating those tradeoffs.

How I would test prompt design vs fine-tuning

I would evaluate the same task distribution and score the failures that matter to the user. A formatting error and an invented policy promise should not necessarily have equal weight.

• Routine questions with a clear source answer.
• Questions with missing information.
• Questions whose answer changed in a recent source revision.
• Requests outside the agent's scope.
• Adversarial requests to ignore the instructions.
• Tool errors and ambiguous tool results, if the workflow uses Actions.

I would record the model, prompt version, source version, output, cost, and review outcome. Then I would repeat the set after each meaningful change.

OpenAI's accuracy guide distinguishes context optimization from behavior optimization. That distinction helps me choose the intervention instead of changing everything at once.

Using these approaches in Pickaxe

In Pickaxe, I would start with the agent's instructions, connect the approved Knowledge Base sources, and choose a model that supports the needed workflow. I would add Actions only when the agent needs to do something beyond producing an answer.

The semantic prompt design guide offers a framework for structuring the conversation. I would still verify the results with a concrete evaluation, because a framework cannot guarantee accuracy.

I would not assume that a no-code builder exposes provider fine-tuning. I would check model and deployment support before designing that route.

Frequently asked questions

Is fine-tuning better than prompt engineering?

It can help with some repeated behaviors, but I would decide from a prompt baseline and held-out tests. It is not automatically the best fix for missing facts, unclear instructions, or a failed tool connection.

Is uploading documents the same as training?

No. A knowledge base supplies selected context at request time. Fine-tuning changes model weights through a training process.

Can I combine RAG and fine-tuning?

Yes, where the chosen model and deployment support it. I would keep current facts in maintained sources and evaluate the trained behavior separately.

What should I try first?

I would collect real failures, write success criteria, and test clear instructions with relevant sources. That gives me evidence for whether model changes or training are necessary.