The First 10 Questions I’d Ask a CX AI Vendor
AI is everywhere in customer experience right now. Every vendor promises they’re going to save you money, boost efficiency, and make your customers fall in love with your brand.
For CX leaders in online retail who haven’t evaluated or purchased AI solutions before, this is overwhelming, and the stakes for choosing the wrong solution are high.
If I were in your shoes evaluating vendors, here are the first 10 questions I’d be asking.
1. Will you integrate with our tech stack?
If it doesn’t integrate, it doesn’t matter.
There’s a number of AI solutions hyperfocused on the typical DTC online retail tech stack: Shopify, Zendesk, Gorgias, Yotpo, Klaviyo, Loop Returns. If your brand falls into this profile, great! These solutions are made just for you!
But brands outside this profile may be looking at a different set of candidates. That’s not to say there’s fewer of them - there’s still a ton - but your options may be different.
2. Do you have a case study for a brand like ours?
Having a case study that shows results is one thing; having a case study for a brand similar to yours is another. Different industries have different types of inquiries, leading to varying automation success rates.
A CPG brand with mostly transactional inquiries may have a 70% resolution rate, while a consumer electronics brand with mostly troubleshooting inquiries may only have a 25% resolution rate, even though both are using the same AI product.
3. What communication channels are supported?
Your customers probably aren’t just reaching out in one place. Some AI platforms handle it all: email, chat, text, phone, social, even reviews and warranty claims. Others are single-channel.
If your customers mostly email and an AI solution’s best feature is a chatbot, that’s not a fit. Make sure they can show up where your customers actually are.
4. Can your solution DO things?
It’s not 2018 anymore. It’s not even 2022 anymore. Customers don’t want an AI that just repeats what they already read on the website. They want to get things done.
Cancel my order. Send me a refund. Shoot me a proactive update if my shipment’s delayed. Show me a product that would work for me.
For Co-Pilots - Tell my team member what they would score on a QA if they sent this email right now. Help my team write more empathetic replies when they’re exhausted from working a backlog.
If the AI can’t take action, it’s just a shinier help center, and you don’t want to pay for that.
5. What formats does your solution read?
I recently spoke to a customer service manager who thought she had everything she needed to easily launch AI. She went through the evaluation process, she made her decision, she signed the contract, and then when she handed over the content to her proprietary knowledge base, the vendor said our AI can’t read that. Before she could launch, she had to convert her content.
Not all AI solutions can read the same things. If you have your help center online, your internal knowledge base in Trello, your discount codes on a google sheet, and your product instructions in a PDF user guide, make sure your solution can read all of those things.
6. How do you measure quality and success?
AI vendors love to share accuracy rates and resolution rates, but what do these metrics mean exactly?
If your customer abandoned an AI email thread out of frustration, that may be a closed ticket, but it’s not a resolution. It’s not what you want your customers experiencing, and it’s not what you want your AI vendor to call a win.
Press the AI vendors to give you the math behind the numbers. What counts as a resolution? How do you know when the AI actually got it right? How are you measuring accuracy? Do you measure customer satisfaction? How do I know these customers are happy with this experience?
7. How do you handle data protection?
What data is stored? Where’s the data stored? How long is it kept? Are you compliant with GDPR, CCPA, all that fun stuff? Be open about how you’re planning to use the tool. This is doubly-important for global brands where information is shared across borders because different countries have different laws.
Don’t assume the vendor has got it locked down. Specific data laws and regulations aren't necessarily the type of information a sales rep would know off the top of their head. I’ve watched conversations get to the end before anyone realizes this solution isn’t going to work because of a regulation.
8. What time and resources do you need from me for implementation?
When I worked at a startup online retail brand, I can’t tell you how often I would hear from a potential partner “Just have your developer…” or “Just have your IT team….” Nope. We ain’t got those.
Vendors may assume you have teams on your end that can assist, or that you have complete documentation on all your processes, or that there’s bandwidth to take on this project right now, but that may not be the case.
By asking for specifics up front, everyone will be aligned.
9. What does life look like after launch?
What happens after the sale can be one of the most significant differentiators between AI solutions.
One solution may come with an account rep who meets with you every week, takes you under their wing, recommends personalized solutions for your instance, and turns you into an AI wizard. They’re easily reachable on Slack, and are happy to help with any questions.
Another solution may offer an account rep, if you’re willing to pay a premium price. No Slack access though. You have to submit a ticket to their engineers if you need something done.
A third solution may hand off a sea of articles and video tutorials. You’re expected to learn and figure things out on your own. If you need more help, here’s a list of agencies we trust.
None of these post-sale experiences are necessarily bad or wrong, but some will work for you more than others.
10. How long does it take a brand to achieve success?
In all the excitement about the potential results that AI can bring, it’s easy to jump to the part (or for your executive leadership to jump to the part) where you’re designing beautiful digital experiences that solve customer problems, and forget about all the real work that goes into getting there.
Training. Testing. Tweaking. Optimizing.
This takes time and effort, and for everyone to have realistic expectations going in, you should ask how long it has taken for brands to achieve their results.
Wrapping It Up
There are a lot of AI vendors out there right now, and the demos are getting even more cool. But if you don’t ask the hard questions, you risk ending up with a tool that doesn’t fit your business, doesn’t integrate with your systems, and frustrates your customers.
These 10 questions will cut through the fluff and give you a clear picture of whether a vendor is going to meet your business needs. And if you want a second set of eyes, AI Evaluation is exactly the kind of thing I help brands with everyday!
