10 Questions to Ask a Security Vendor Before You Agree to a POC

Most vendor qualification happens too late. These questions decide whether a vendor earns a spot in your bake-off — before you commit any time.

Running a security POC is expensive. Not in software costs — in time. Coordinating test environments, briefing your team, attending demos, scoring evidence, managing vendor communications. A three-vendor EDR evaluation can run 6–8 weeks of real engineering hours if you're not careful. The vendor qualification call — the conversation before you agree to anything — is where you protect that time. Done well, it eliminates one of your three shortlisted vendors before a single test environment is spun up. Most security teams don't do this call properly. They let the vendor run it as a discovery call, which means the vendor learns about your environment and you learn about their pitch. That's the wrong direction. Here are ten questions that put you in control of the conversation and tell you what you actually need to know.

Questions about capability

1. What does your Linux detection look like compared to Windows? This is the single most revealing technical question for EDR evaluations. The honest answer from a vendor with strong Linux coverage is specific: "Our agent on Linux uses eBPF rather than a kernel module, detection logic is equivalent across OS types, and we can show you comparable MITRE coverage data." The evasive answer — "We fully support Linux" — tells you everything. Follow up with: "Can you share your MITRE ATT&CK coverage matrix for Linux specifically?"
2. What's your mean time to detect in a production environment — not a lab? Vendors quote lab numbers. Labs are quiet, controlled, and nothing like your environment. Ask specifically about MTTD in a production environment for a customer similar to yours in size and stack. If they can't give you a number — or if the number comes with significant caveats — that's signal.
3. How does your autonomous response work, and what are the conditions that trigger it? You want to understand the logic, not just the feature. Does it trigger on confirmation of malicious activity or on suspicion? What's the false positive rate on autonomous containment? Can you tune the threshold? A vendor who can walk you through the decision tree is a vendor who understands their product.

Questions about fit 4. What percentage of your customers look like us? Size, industry, OS mix, cloud footprint. A vendor whose customer base is 80% Fortune 500 enterprises will have built their product, their support model, and their implementation process for that customer. If you're a 300-person SaaS company, find out before you invest in a POC. 5. What does your typical deployment look like for an environment our size? Timeline, resources required on your side, and what "done" means. You want to know: does this require professional services, or can one SE deploy and configure within a day? The answer shapes your POC plan. 6. What integrations with our stack are native versus custom? List your SIEM, SOAR, ticketing tool, identity provider and cloud platforms, and ask specifically which are native integrations and which require custom development. A vendor who says "we integrate with everything" is saying nothing. You want: "Splunk is native, full process tree enrichment, bidirectional action available. ServiceNow is via our REST API, no native connector."

Questions about the POC itself 7. What do you need from us to run a meaningful POC? This question does two things. It tells you the real cost of the evaluation in terms of your team's time and infrastructure. And it tells you whether the vendor knows what a meaningful POC looks like. A good answer includes: test endpoint requirements, network access, a proposed task list, and a timeline. A vague answer — "just give us access and we'll set it up" — suggests the POC will be vendor-led and optimised for their strengths. 8. Can we run our own test scenarios rather than yours? You want to run the same five tasks across every contender. If a vendor is resistant to this — if they prefer to demonstrate their own scenarios — that's worth noting. The best contenders will welcome your criteria because they're confident they can meet them.

Questions about commercial reality 9. What does pricing actually look like for our size, and what's in the base versus add-ons? Not "what's your pricing model" — that gets you a pricing page recitation. You want: at our scale, what would we pay annually for a base deployment, and what capabilities are extra? Managed threat hunting, identity protection, cloud modules — what's the true cost of the product you actually want? A vendor who won't give you a range on a qualification call is a vendor who will make you sit through six more meetings before you see a number. 10. What does the minimum commitment look like, and what happens if the POC doesn't meet our requirements? This is the exit question. You want to understand: if this vendor wins the bake-off and we proceed to contract, what are we committing to? 12 months? 3 years? Is there a performance clause if the MTTD benchmarks from the POC aren't sustained in production? A confident vendor will have a clear answer and won't be defensive about the question. If this question creates friction, that's worth weighing.

What to do with the answers After three qualification calls, you'll have a clear sense of which vendor is worth your full bake-off investment. The one who gave you specific numbers, knew their product's limitations, had a clear POC plan, and was transparent about pricing. The other two — or at least one of them — will have been more evasive, more sales-led, more resistant to your own test scenarios. That vendor doesn't earn a spot in the bake-off. You've just saved weeks.

Running your bake-off on pmpa Once you've qualified your shortlist, pmpa gives you one workspace to run the full evaluation — criteria, POC tasks, evidence, scoring. Contenders work from the same brief, blind to each other. You score as evidence arrives. Free for security teams. → picari.io May the best contender win.