Your chat widget is a free AI for anyone

A chat widget on a public page has to answer strangers; that's its job. Behind it sits a model you pay for by the token, on your API key. So everyone who can reach the page can spend your money, and the page has no way of knowing whether the person typing is a customer, a student with homework, or a script.

Most of the time nothing happens. When something does, it tends to be one of four things.

As a free general-purpose AI

The dentist's appointment bot will write an essay, debug code or plan a trip if you ask it nicely. Each answer is a full model call on the dentist's bill. It rarely looks like abuse; it looks like a lot of long, oddly engaged conversations.

As someone else's backend

The widget calls an endpoint on your server. Someone finds that endpoint in the page's network traffic, calls it directly, and wraps it as a free API for other people. Your server can't easily tell those calls from your own page's.

As a target for scripts

A script opens thousands of conversations, sometimes to run up your bill on purpose, sometimes just because it can. Because model providers set rate limits for your whole account, a flood on the widget can also push every real visitor into errors: the cost isn't just money, it's the widget going dark for everyone.

As a way into what's behind it

Visitors try to get the bot to recite its instructions, dump the documents it answers from, or say something you'd never want screenshotted with your name on it.

• Only allowing your own website. Browsers enforce that; a script calling your endpoint directly just claims to be your site.
• Limits per IP address. Rented residential proxies give a script thousands of addresses, each looking like someone's home internet.
• Limits per conversation. The script starts a new conversation.
• A token issued when the page loads. The script loads the page first, in a headless browser.
• A monthly message cap from your widget vendor. It caps the bill, but when the cap is hit the widget stops working for real customers too, so the attacker still wins.
• A CAPTCHA before chatting. It works a little and costs a lot of real conversations, which is why few sales widgets use one.

You can't make a public widget impossible to abuse. You can make abuse expensive, visible and short-lived:

• Keep the API key on your server. Never in the page, never in the widget's code.
• Cap every answer's length and every conversation's turns. A support answer rarely needs more than a few hundred words.
• Budget per visitor, not per message. Track what each visitor has cost you and slow them down when it's out of line with a real customer.
• Check that a message is on topic before the model sees it. A small, cheap check can turn away "write my essay" for a fraction of what answering it would cost.
• Tie each request to a short-lived session that only your page can create, so lifting the endpoint stops working once the session expires.
• Pass an anonymous visitor ID to the model provider. Anthropic's API has a metadata field for an end-user ID, and OpenAI's has a similar one, so abuse can be traced to one visitor instead of your whole account.
• Watch spend per visitor, and get an alert when one visitor or one pattern starts costing more than your real customers do.