Agentic AI
8 Lessons From Building a Product With an AI Agent
What building Overcoat with an AI coding agent taught me about briefing, checking, and owning the work.
Websites and web apps are built for everyone, so they rarely fit any one person exactly. An ad covers half the page. A layout doesn’t match the way you work. A video player could work better. Each fix needs its own workaround, and most of them don’t stick.
What I wanted was one place to add my own CSS, JavaScript, and HTML to any site, with rules that beat the site’s own styles. So I built it with an AI coding agent.
That became Overcoat. I designed it, made the product decisions, and tested it on the sites I use every day. The agent wrote most of the code and tested it as we went. Overcoat is free, open source under the MIT license, and published under Generative AI Lab. The code is in the Overcoat GitHub repository.
Last week I wrote about the 10 AI agent terms I wish I’d known before building my first agent. That agent answers leads for one of my businesses. This time the agent helped build a product, and the lessons were different.
The hard part wasn’t getting the agent to write code. It was deciding what to build, which technical choices to trust, and whether the result actually worked. The first commit and a finished version were less than an hour apart, so the code was never the bottleneck. Here are 8 lessons from building it.
1. Start with the problem, not the code
Don’t start with a file structure or a framework. Start with what the product should do. For Overcoat, that meant: put my own styles and scripts on any site, choose which sites they run on, and turn them on and off quickly from the toolbar.
That became a short brief. The agent turned it into a plan: a background script, an options page for editing rules, a toolbar popup for quick toggles, and a small library for matching sites. I reviewed the plan and changed what didn’t fit before any code was written.
Why it matters: an agent is good at turning a clear goal into a structure. Making the goal clear and approving the plan is your job.
2. Question every technical choice
The first decision that mattered was how to add CSS to a page so it actually wins. Many sites use !important rules, and some block inline styles entirely.
The usual approach is to add a <style> tag to the page. That tag has to compete with the site’s own !important rules, and strict security policies can block it. The agent proposed Chrome’s insertCSS API instead, which adds the CSS as a user stylesheet. I asked why, and what we would lose. A user stylesheet overrides the site’s !important rules, isn’t blocked by the page’s security policy, and applies before the page paints, so the original design never flashes first. The reasons held up, so the choice stayed.
Why it matters: you don’t need to know every API. You do need to understand each choice well enough to explain it to someone else.
3. Make the agent prove its work
Ask the agent to test everything it builds, not just write it. For Overcoat, it launched a browser in the background, loaded the extension, opened test pages, and checked that each rule changed the page the way it should. When something failed, it showed a screenshot, fixed the issue, and ran the test again.
Passing tests have limits. They only prove the checks the agent wrote, and the agent decides what “correct” looks like. If that idea is wrong, the test still passes. That is why the next lesson matters.
Why it matters: an agent that runs and checks its own code is far more useful than one that only writes it. Ask for proof, then check the proof.
4. Test it live, with a human in the loop
Automated tests only go so far. I installed Overcoat in my own Chrome and used it on the sites I visit every day. That is where the problems the tests missed showed up: a switch that looked off when it was on, and a button label that was hard to read. Some features, like running your own JavaScript, could not be tested in the agent’s environment at all, so those were tested by hand.
Each problem went back to the agent as specific feedback: what I did, what I expected, and what happened. The agent fixed it and re-ran its tests, then I checked again on the live site. The feedback also became standing rules for the agent, so the same kind of mistake didn’t come back.
Why it matters: the agent writes the code, but you are the first real user. Your feedback is what turns working code into a product people can use.
5. Know what the agent can’t do
The agent was fast at writing and testing code, but it had clear limits. It couldn’t see the product the way a person does, so the switch that looked off passed its tests. It couldn’t run every feature in its environment, so the JavaScript features needed a human. And it couldn’t decide what mattered: the name, the license, and who publishes the product were still my calls.
The agent also changed where the time went. Most of it went to reviewing plans, checking claims, and testing, not writing code. Changes were cheap to make, so it was easy to try one, look at it on a real site, and keep it or drop it.
Why it matters: knowing where the agent stops tells you where to spend your own time.
6. Use the agent for the work around the code
Shipping an extension is more than code. The Chrome Web Store asks for a description, screenshots at exact sizes, promo images, a privacy policy, and a written reason for every permission.
I wrote the brief for each piece, and the agent built the screenshots and promo images and drafted the store text. Before writing “no user data is collected,” the agent checked the code for any network calls. There were none, so the privacy answers were short and true.
Why it matters: the last part of a launch is mostly forms, images, and policies. An agent handles that well, as long as you make it check the facts first.
7. Permissions are product decisions
Chrome treats access to all sites as a sensitive permission, and Overcoat needs it. The alternative was access only when you click the extension, but then your rules wouldn’t run on their own, which is the whole point.
No network calls means nothing leaves your browser. It doesn’t mean Overcoat can’t see pages. It can read and change any site you visit, and any JavaScript you add runs with that same access. That is why the code is public, and why you should only add scripts you understand.
The first version keeps full access. A later version will add an option that asks for each site when you save a rule for it.
Why it matters: every permission is a trade-off between what the product can do and how much users have to trust it. Decide that on purpose.
8. Publish the code
Overcoat is published under the MIT license. Anyone can read it, use it, change it, and build on it. For a tool that runs on the sites you visit, people should be able to see exactly what it does.
Why it matters: when an agent writes much of the code, open source lets other developers review it. It doesn’t prove the code is safe, but it means anyone can check.
Advice for developers starting out with agents
Treat the agent like a fast teammate who needs a clear brief and honest feedback. Describe the problem, approve the plan, ask it to prove its work, and test the product yourself on real sites. The agent can write the code. The decisions about names, permissions, and trade-offs stay with you.
The code is in the Overcoat GitHub repository under the MIT license. Open an issue with a bug, an idea, or a rule you’d like to share.
Next, we plan to add the option to describe a change in plain English and have Overcoat write the JS, HTML or CSS for you.
For updates, follow Generative AI Lab. You can also connect with me on LinkedIn or see how I work with AI teams on marketing.