How the tests are run
This page is the contract. Every benchmark links here, states which version of the method it used, and lists anything it did differently. If a vendor thinks a test is unfair, this is the page to argue with; if a reader wants to rerun one, this is the recipe.
Conflict of interest: I build LeadLayer, a B2B contact database, and Crawlfox, a Search and Scrape API for AI. Both compete with vendors tested here. Both are tested with the same procedure, and their results are published whether they win or lose. I take no money from any vendor tested and run no affiliate links to them.
1. Email accuracy
Question. If you export 1,000 contacts from a vendor today, how many of the emails are deliverable?
- Sample. 1,000 contacts per vendor, drawn on a stated date from a paid plan, using the vendor's own filters: 10 industries × 4 seniority bands (IC, manager, director, VP+), 25 contacts per cell, US-based companies with 11–1,000 employees. The filter set is saved and published with the test.
- Validators. Each email is checked by three independent validators (currently ZeroBounce, NeverBounce, MillionVerifier) within 48 hours of export. A result is valid only if at least two of three say deliverable; catch-all if two of three say catch-all; otherwise invalid. Validator disagreement is reported as its own number.
- Reported. Valid %, catch-all %, invalid %, disagreement %, and the breakdown by industry and seniority. Confidence interval at 95% for the headline rate (±3 points at n=1,000).
- Not measured. Real send deliverability, phone numbers, or whether the person still holds the title. Those are separate tests.
- Rotation. One vendor per month. Each vendor is retested no more than once per quarter so a single bad export doesn't define it.
2. Technographic hit rate
Question. When a provider says a domain runs a technology, how often is that true, and how much does it miss?
- Panel. A fixed set of 100 domains, published, stratified across company size and industry, frozen for 12 months so results are comparable month to month.
- Ground truth. For a fixed list of 20 technologies (CRM, marketing automation, analytics, chat, payments, hosting and others), each domain is hand-verified on the test date from the live site, DNS and public job posts. Two reviewers; disagreements resolved by a third check and logged.
- Providers. BuiltWith, Wappalyzer, Crawlfox, and any vendor technographic field that can be exported. Same day, same panel.
- Reported. Precision (of the claims made, how many were right), recall (of the true installs, how many were found), and F1 per provider and per technology. False positives are listed by name.
- Not measured. Technologies outside the 20, or installs the ground-truth process can't see (internal tools).
3. Pricing log
- Public pricing pages of tracked vendors are captured on the first business day of each month and whenever a change is reported. Each entry has a dated screenshot, the previous and new values, and a one-line note.
- Where a vendor's page shows no numbers (Apollo's did in October 2026), the entry says so and quotes the sales-provided figures only if a dated written quote exists.
4. Coverage
- A published list of 500 companies (from public registries, not from any vendor). For each vendor: how many are present, and the fill rate for employee count, industry, HQ, and at least one verified email.
What every published test includes
- The date of data collection and the date of publication
- The method version and any deviations
- The raw CSV with one row per contact or domain, under CC BY 4.0
- The exact filters or panel used, so the test can be repeated
- A section titled "What would change this result", written before the numbers are final
- Any vendor response received before publication, verbatim
Corrections
Errors are corrected on the page with a dated note at the top. The CSV is versioned; superseded versions stay downloadable. Vendors can request a rerun by email with a specific objection; reruns are published whether or not they change the result.
Changelog
- v1 · 31 Oct 2026 Initial method.