Vendor Evaluation

Vendor Evaluation

Pick the right tool or service, negotiate fair terms, and avoid the lock-in traps. Stack-agnostic. Applies to SaaS, infrastructure providers, agencies, and any external dependency.

---

When to use

• Selecting a tool or vendor for a new need
• Evaluating alternatives to a current vendor
• Build vs buy analysis
• Renewal coming up: should we stay or switch?
• Running a formal RFP or RFI
• Comparing finalists in a vendor selection
• Negotiating a contract
• Assessing vendor risk (financial, security, dependency)

When NOT to use

• General cost reduction (use cost-optimization)
• Specific contract legal terms (those go to legal)
• Performance issues with an existing vendor (try fixing before switching)
• Hiring an agency for a one-off project (lighter framework needed)

---

Required inputs

• The need (what problem are you solving, what would success look like)
• Constraints (budget, timeline, integration requirements)
• Stakeholders (users, IT, security, finance, legal)
• Existing context (what's already used, what's been tried)
• Compliance requirements

---

The framework: 5 phases

A structured vendor evaluation. Skip phases at your peril.

Phase 1: Define the need

Before looking at vendors, define what you actually need.

• What problem are you solving?
• What's the user / use case?
• What does "success" look like in 6 months? In 2 years?
• What's the budget (range, not just ceiling)?
• What's the timeline?
• Are there must-have integrations or constraints?

The temptation: skip this and start demoing. Vendors are happy to show off; you end up choosing what looks shiny rather than what fits.

Phase 2: Build vs buy

Before evaluating vendors, decide whether you should build instead.

Build when:

• It's core to the business (differentiating)
• The need is so specific no vendor matches
• The economics work at your scale
• You have the team to maintain it
• Vendor lock-in would be unacceptable

Buy when:

• It's table stakes (not differentiating)
• The need is well-served by existing products
• The economics favor it
• The team should focus elsewhere
• The vendor's specialization beats your generalism

Most teams over-build. The rule of thumb: buy unless there's a strong reason to build. Then question even that strong reason.

Phase 3: Generate the shortlist

Cast a wider net than feels comfortable, then narrow.

Sources:

• Internal team's existing knowledge
• Industry analyst reports (Gartner, Forrester, etc.)
• Peer recommendations (other companies similar to yours)
• Reviews (G2, Capterra; with caveats about review quality)
• Adjacent vendors you already use (often have the feature you need)
• Open-source alternatives

Cast wide first. Aim for 5-8 candidates. Then narrow to 2-4 finalists for deep evaluation.

Phase 4: Score the finalists

Use a scorecard. Without one, you'll be swayed by demo theatrics or who has the friendliest sales rep.

Scorecard dimensions (weight by your situation):

Functional fit (40%): Does it do what you need? Edge cases handled? UX quality. Workflow fit.

Technical fit (15%): Integration with your stack. API quality and completeness. Data export and portability. Performance at your scale. Self-hosted, hybrid, or SaaS-only.

Operational fit (10%): Onboarding effort. Training and adoption. Documentation quality. Support quality (test by submitting a ticket). SLAs.

Security and compliance (10%): SOC 2, ISO 27001, HIPAA, etc., as applicable. Data residency. Encryption at rest and in transit. Access controls and audit logs. Penetration test results (ask). Subprocessors.

Vendor health (10%): Years in business. Funding and runway (or revenue if private). Customer base size and similar customers. Public references. Roadmap visibility.

Cost (10%): License or subscription cost. Implementation and onboarding cost. Training cost. Integration cost. Opportunity cost (in-house resource time). Switching cost (in case of failure).

Lock-in risk (5%): Data export quality. Standard formats vs proprietary. Migration paths to alternatives. Open standards alignment. Contract escape clauses.

Score each finalist 1-5 on each dimension. Multiply by weight. Sum.

The score isn't gospel. It surfaces the tradeoffs.

Phase 5: Negotiate

Most enterprise contracts are negotiable. Most aren't negotiated.

What's negotiable:

• Price (multi-year, volume, prepayment, end-of-quarter timing)
• Terms (payment schedule, renewal terms)
• Success commitments (training, onboarding, support)
• SLAs (uptime, response time, credits)
• Termination clauses (auto-renewal, notice period, data export)
• Liability caps and indemnity (legal will care about these)
• Subprocessors and data handling (security/legal cares)

Common negotiation moves:

• Multi-year discount (3-year for a 15-30% discount is common)
• Volume tiers (commit to higher usage for a per-unit discount)
• Annual prepayment for a discount
• Free or discounted onboarding
• Pilot pricing (trial period at reduced rate)
• "Most favored customer" clauses (if your size warrants)

What to avoid:

• Multi-year lock with no escape clause for material breach
• Auto-renewal with short notice window (under 60 days; want longer)
• "All right, no further negotiation" stance from the start
• Signing without legal review

---

Workflow

Step 1: Define the need

Write a one-page brief: what we need, why, success criteria, constraints, stakeholders.

Step 2: Build vs buy

Honestly answer the build/buy question. Document the rationale.

Step 3: Generate shortlist

Wider net first, narrowed via desk research:

• Read summaries
• Skim reviews
• Look at customer logos
• Skim docs
• Skim API specs

Eliminate obvious misfits. Land on 2-4 finalists.

Step 4: Run demos and trials

For each finalist:

• Demo with the use case (don't take their default demo; bring yours)
• Trial period if possible
• Reference calls with similar customers
• Pilot with real data if feasible

Don't be charmed by the polished demo. Try it with your real workflow.

Step 5: Run security and compliance review

Critical for any vendor handling sensitive data:

• Request SOC 2 / ISO 27001 reports
• Review their security questionnaire (most have one ready)
• Verify data handling matches your requirements
• Identify subprocessors

This can take weeks for enterprise vendors. Start early.