Tech With Tam
All resources

AI Setup Audit

  • custom instructions
  • projects
  • knowledge files
  • context
  • skills

What it is

Audit the instructions, memory, and files your ChatGPT or Claude setup runs on, then get the fixes ready to paste.

Who it’s for

Founders, consultants, and operators who use ChatGPT or Claude projects, custom GPTs, or Cowork for repeated work

What it helps you do

Reviews your instructions, memory, and knowledge files together, explains what is getting in the way, and drafts the replacements.

Why it’s useful

When AI keeps ignoring what you told it, the cause is usually the setup, not the prompt.

What you’ll get

  • New instructions ready to paste, with one clear line per file
  • A checklist of files to rename, merge, split, or delete
  • Test messages that show whether the fix worked
AI Setup Audit

When ChatGPT or Claude keeps ignoring what you told it, I usually find the problem in the setup, not the prompt.

Your setup is everything the AI reads before you type: your custom instructions, what it remembers about you, the instructions on a project or custom GPT, and the files you uploaded. It usually grows one piece at a time. A rule here, a file there, a memory saved months ago. Then the pieces start to disagree.

Here's a made-up example. A consultant's Claude project keeps writing stiff, formal proposals and quoting last year's prices. Her instructions say "be professional" twice. Memory says she prefers a formal tone. Her 2023 brand guide is still uploaded next to the current one. Her old prices are pasted into the instructions, and the current price list is a file nothing mentions.

Claude isn't ignoring her. It's following the old version of her, because that's what the setup says.

AI Setup Audit is a skill that reviews the whole setup at once and hands back the fixes.

What it checks

Three ideas do most of the work.

IdeaWhat it means for your setup
Context loadEverything in your instructions is read in every chat, so every line has to earn its place. "You are a world-class expert" doesn't.
Context pointerA line that tells the AI which file to open, and when: "Before quoting a price, open pricing.md." "Refer to my files" doesn't count.
Single source of truthSay each thing once, in one place. A price in your instructions, your memory, and a file will eventually disagree.

It also checks your files. It looks for names that don't say what's inside, old versions sitting next to new ones, one big document covering five topics, and scanned PDFs the AI can't read well.

One detail changes by app. ChatGPT searches your project files, so a file no instruction points to can go unfound. A small Claude project reads every file in every chat, so one outdated file can affect every answer.

What you get back

  • A short explanation of what's causing your problem
  • A numbered fix list, in the order to do it
  • New instructions ready to paste, with one line per file
  • A checklist of files to rename, merge, split, or delete
  • Memory changes for you to approve one at a time
  • A few test messages, so you can see whether the fix worked

Each fix comes with the reason behind it. You repair your setup and learn how a good one works, so the next project starts cleaner. See a full example audit on GitHub.

Try it on one project

Download the skill from GitHub. In Claude, upload the ZIP under Customize, then Skills. Uploaded skills work in Cowork too. In ChatGPT, attach the ZIP to a ChatGPT Work chat and ask it to use the skill. The GitHub page has the step-by-step for each app, including a command-line install for coding tools.

Then ask:

Audit my AI setup for this project. Here's what keeps going wrong: [describe it]. Ask me for anything you can't see.

Paste your instructions and list your files when it asks. In Cowork and ChatGPT Work it can read your folder itself, and it asks before changing anything.

It can't change your settings, so in a regular chat you paste the new text yourself. It only sees what you share, so a file it can't open gets judged by its name and your description. Numbers it suggests, like a 150-word limit, are starting guesses for you to adjust.

The approach adapts Matt Pocock's writing-for-agents skill, which applies the same ideas to instructions written for coding agents.

Explore more from the library