Why demos lie
Every vendor demo we've sat through — and we've sat through a lot — uses a scripted account list, a pre-warmed mailbox, and a message that a human obviously edited. That's not a bad thing; it's a demo. The problem is that buyers use the demo as their evaluation.
The 12 questions below are designed to move you from 'the demo was impressive' to 'I know exactly how this thing will behave in my environment in month three.'
The 12 questions, in order of importance
We put them in order because if a vendor fumbles any of the first four, we don't ask the rest. Life is short.
- 1. When your agent doesn't know the answer to a prospect reply, what happens?
- 2. Who owns the sending domain and mailboxes — us or you?
- 3. Show us your last three deliverability incidents and how you handled them.
- 4. What's the raw sample size behind the case study on your homepage?
- 5. What data sources power enrichment and how often are they refreshed?
- 6. What percentage of your customers renew after month 12?
- 7. Where does the agent write back to Salesforce/HubSpot — object, field, timing?
- 8. What does 'personalized' mean here — show us three real examples from three real customers?
- 9. What is the annual cost at 5,000 sends/month, all-in, including data?
- 10. Who at your company will I be talking to in month four?
- 11. What does off-boarding look like if we leave — do we keep the messaging IP?
- 12. Show us the compliance stance: SOC 2, GDPR, CAN-SPAM enforcement, opt-out handling.
The four that matter most
Question 1 exposes autonomy. A great vendor answers 'the agent classifies the reply against 14 intents and either drafts a follow-up, escalates to a human via Slack, or marks the contact based on your rules.' A bad vendor says 'we alert your team.' That's a copilot in a trench coat.
Question 2 exposes deliverability ownership. If you own the domain and mailboxes, you own the reputation risk. Most sophisticated buyers we know prefer to own them; most first-time buyers should not.
Question 3 is the killshot. If a vendor won't tell you about their last three deliverability incidents, either they don't have them (unlikely) or they hide them (much more likely). The good vendors will tell you the story and what they changed.
Question 6 is renewal rate. Ask for it as a number. If they dodge, you have your answer.
How to run the pilot
Never pilot an AI SDR on your best accounts. Give it a tier-two segment, a fresh subdomain (never your primary), and a hard cap on daily sends for the first two weeks. Measure three things: reply rate, unsubscribe rate, and meetings booked with a real ICP fit.
Run the pilot for 8 weeks minimum. Anything shorter and you're measuring novelty, not sustained performance.
What to negotiate before signing
Every AI SDR contract we've seen has three clauses worth pushing on: exit terms (what happens to your data and messaging library), price caps on second-year renewal (default is 'discretion of vendor' — replace with 5%), and a 30-day out if you miss a mutually agreed performance floor in the first 90 days.
The last one is the only real accountability lever you have. Every vendor will tell you they don't do it. About a third will do it anyway if you push.