Why reply rate lies
AI SDRs are good at getting replies. Too good. A well-tuned agent can hit 6–8% reply rates by asking questions designed to provoke a response ('Are you the right person for this?') rather than to start a real conversation.
Those replies rarely turn into meetings. And the meetings that do get booked from them rarely turn into opportunities. Track reply rate as a diagnostic, not a KPI.
The three metrics that matter
Qualified meetings booked. Not all meetings — meetings where the prospect matches your ICP and has budget/authority/need. This is the number your CRO should care about.
Pipeline sourced per dollar spent. Total pipeline value attributed to the AI SDR divided by all-in cost (license + data + services + your team's time). Should exceed 3x within 6 months.
Unsubscribe-adjusted reply rate. Reply rate minus 3x unsubscribe rate. If you're getting a 6% reply rate but a 1.2% unsubscribe rate, your true 'good reply' rate is closer to 2%.
The metrics vendors show you
Vendors love showing you sends, opens, and reply rates. All three are trivially juiced. Sends are a function of budget. Opens are polluted by Apple Mail Privacy Protection and bot pre-fetching. Reply rate is what we just discussed.
Ask instead: what percentage of meetings booked convert to opportunity? That number cuts through everything.
How to instrument this correctly
In Salesforce or HubSpot, add a source field that tags every lead touched by the AI SDR. Track meetings, opportunities, and closed-won separately. Report weekly. Compare to your baseline human SDR cohort using the same window.
Do not trust the vendor's dashboard for reporting. Their numbers will always be more flattering than yours.