Build with purpose: 3 questions before you add AI

A lot of AI features start as a sentence like "add AI to the support inbox" or "put a model on the dashboard". The sentence is almost never about AI. It is about a job that is slow, or annoying, or wrong too often, and AI is the tool that happens to be in the headlines. Start from the tool and you build the wrong thing quickly. Start from the job and you sometimes build something with a model in it, and sometimes you do not.

So before I open an editor, I ask 3 questions. They are quick to answer, and skipping them is how a small feature turns into weeks of tuning prompts for a problem a lookup table would have solved.

01. What is the task?

Write the task as one sentence a user would recognise, with an input, an output and someone who acts on the result. "Add AI to analytics" is not a task. "Tell me which articles real people read to the end, so I know what to write next" is. It names a user (me), an input (visits), an output (a list of articles) and the decision it feeds (what to write next).

That sentence also tells you what done looks like. If you cannot write it, you are not ready to choose a tool, AI or otherwise.

If you cannot say what the output is and who uses it, no model will say it for you.

02. Could a rule work?

With the task written down, ask whether a rule in plain language would do it. If it would, write the rule. Rules are fast, cheap to run, easy to explain, and you can test them with 2 examples and a reason.

For the reading question, the rule is short: an article counts as read when a real browser has had it on screen for 30 seconds and scrolled half way. That is exactly what this site counts today. No model, no tuning, and when someone asks how a read is defined, the answer is one line.

Most security checks work the same way. When I built Preflight, a launch check for apps built fast with AI, the scanner came down to 14 patterns. A Stripe live key has a known shape. A Supabase table without row level security is missing one line of SQL. Patterns find those reliably. The part that needs judgement, deciding whether a finding is real in this particular codebase, is where the agent comes in.

A model earns its place where the input is open ended and the list of rules would never end: questions typed in people's own words, messy documents, drafts and summaries. Even then, rules sit around it. Dilly, the assistant on this site, answers in plain language, but only from a written knowledge base, it keeps only the last 10 messages, and it caps each one at 2,000 characters. The model handles the language. The rules handle what must not go wrong.

• A rule fits formats, thresholds, lookups, permissions, and anything you can check with an if.
• A model fits open language, fuzzy matching, drafting and summarising, where the rule list would never end.
• Most real features need both: a model inside rules that decide what it may see, what it may say, and what happens when it is wrong.

03. How will we test it?

Decide how you will know it works before you build it, and write that down as something that can fail. For a rule, that is a test with one input that should pass and one that should not. For a model, it is a fixed set of real inputs with the answers you expect, run again every time the prompt or the model changes.

Then break it on purpose. Change the code so it is wrong and run the check again. If the check still passes, it was never checking anything. I do this for every check I add, and it is the habit I packaged as Proof: the agent has to show the command and its real output, and every new check is broken once to prove it can fail.

Testing also finds the gaps you did not ask about. When I ran Preflight on this site, it flagged that Dilly has no rate limit, so one script could call the model as often as it liked. A rule, once more, is the fix, and it is next on my list.

If you cannot describe the test at all, you have learned something useful: the task is still too vague. Go back to question 1.

Understand the problem. Then choose the tools.

None of this is against AI. Most of what I build has a model in it somewhere. The order is the point: the task first, then the simplest thing that does it, then the test that proves it. The tool is the last decision, not the first.

Proof is free on my Releases page, and Preflight is there for the launch checks. Both came out of asking these 3 questions about my own work.