Prompting from the mechanism: document continuation and in-context examples

A model continues the text it is given in the most likely way, so a prompt is not a command to a person but the beginning of a document. Instructions make one kind of continuation more likely, and a few finished examples in the same format make the next answer's format almost certain.

In Daveth Pro (AI Engineering)

What this page answers

What is the best way to write a prompt?

Write the start of the document you want continued: say the task and format, and add a few finished examples, since continuing a pattern is what models do best. Read it in “A prompt is the start of a document”

Why do examples in a prompt work so well?

Because the model's job is to continue a pattern, a few examples in the same format make the next answer's format almost certain. Read it in the lesson

A prompt is the start of a document

The model's job is to continue the text it's given in the most likely way. So a prompt isn't a command to a person; it's the beginning of a document, and the answer is whatever would most plausibly come next. A bare review, pasted on its own, could plausibly be followed by many things: a reply, another review, a comment. A review preceded by “Classify the sentiment… reply with one word” makes one kind of continuation far more likely. And a few finished examples in the same format make the next answer's format almost certain, because continuing a pattern is the thing these models are best at.

Predict first. Asked to pull the date out of 20 sentences and write it as YYYY-MM-DD, which way of asking gets the most exactly right?

  • just the sentence
  • clear instructions
  • instructions plus three worked examples

Answer. A tie between clear instructions and instructions + 3 examples: just the input 0%, clear instructions 100%, instructions + 3 examples 100%. Asked bare, the model mostly just talked about the sentence, because nothing told it a date in a particular format was wanted.

Three ways to ask, measured

Two tasks I wrote for this lesson: 40 short product reviews, half positive and half negative, some deliberately tricky (“It's fine I guess, but I wouldn't buy it again”); and 20 sentences each containing one date in a different format. Each was asked three ways. How to read the chart: for each task and way of asking, the solid bar is the share answered correctly, and the outlined bar the share answered in exactly the requested format (one word, or a bare date), which is what you'd need to use the answers in a spreadsheet.

Drawing: Accuracy and format compliance for three prompting styles on two tasks.

Clear instructions did most of the work on the reviews (0% → 95% correct) and transformed the format (0% → 100% one-word answers). On dates, the jump was 0% → 100% with instructions and 100% with examples. The bare prompts weren't answered wrongly so much as answered as something else: a comment on the review, a sentence about the date. The model did its job, continuing the text plausibly; the prompt just didn't make the wanted answer the plausible one. Here's what the same input got each way:

Inputjust the inputclear instructionsinstructions + 3 examples
Absolutely loved it, would buy again.I'm sorry, but I am not able to understand or respond to that message as it appears to be in another language or dialect. Could you pleasepositivepositive
The meeting moved to March 5th, 2024.I'm sorry, but I don't have any specific information about the meeting you're referring to or its date of March 5th, 22024-03-05March 2024-03-05