July 16, 2026
Updated: July 16, 2026
A 25-question evidence checklist for evaluating penetration testing providers
Mohammed Khalil
Choose a penetration testing company by evaluating the assigned people, scope fit, safety controls, and evidence quality. Ask every shortlisted provider the same 25 questions and request proportionate proof instead of accepting brand claims. This guide is for service buyers not job candidates. If you still need a shortlist, begin with the broader penetration testing vendors guide, then return here to evaluate each provider consistently.
Check the assigned testers, delivery model, relevant experience, manual depth, scope, authorization, safety, data handling, reporting, remediation, retesting, and terms. Request a sanitized report, team evidence, safeguards, and exact deliverables. Apply hard gates first, then compare providers with the same scorecard. Certifications and brand recognition do not replace engagement-specific evidence.
Before contacting providers, document the objective, assets and environments, user roles, trust boundaries, sensitive data, third parties, constraints, deliverables, compliance driver, deadline, and retesting need. Comparable inputs produce comparable proposals.
1. Define the business objective. State the decision, risk, assurance need, or change the test must inform.
2. Ask every shortlisted provider the same core questions. Consistency makes answers comparable.
3. Probe vague language. Ask who, how, when, which assets, and what evidence supports the claim.
4. Request proportionate evidence. Protect other clients’ confidentiality while verifying capability.
5. Apply hard gates, then score. Authorization, safety, data handling, and material transparency failures can disqualify a provider regardless of points.
6. Complete reference, legal, scope, and contract checks. The scorecard supports professional judgment; it does not replace it.
The DeepStrike PENTEST Vendor Evidence Scorecard is an original editorial decision aid not an industry standard, certification, or guarantee. Rate every question from 0 to 5. Within each PENTEST dimension, average the question scores. Weighted points = (dimension average ÷ 5) × dimension weight. Example: People and Proof scores of 4, 3, 4, 5, and 4 average 4.0; (4 ÷ 5) × 15 = 12 weighted points out of 15.
Table 1. PENTEST question groups and dimension weights.
| PENTEST dimension | Questions | Weight |
|---|---|---|
| P — People and Proof | 1–5 | 15 |
| E — Engagement Fit and Scope | 6–9 | 15 |
| N — Safety, Authorization, and Data | 10–13 | 15 |
| T — Methodology and Technical Depth | 14–18 | 20 |
| E — Evidence, Reporting, and Communication | 19–21 | 15 |
| S — Support, Remediation, and Retesting | 22–23 | 10 |
| T — Terms, Transparency, and Total Cost | 24–25 | 10 |
| Total | 25 questions | 100 |
Suggested interpretation:
• 85–100: Strong fit, subject to reference, legal, scope, and contract verification.
• 70–84: Potential fit, but material clarifications or conditions remain.
• 55–69: High-risk or incomplete proposal; require evidence and corrective commitments before selection.
• Below 55: Do not shortlist without a substantial change in evidence or scope.
These ranges are editorial guidance, not universal procurement rules. A high total cannot cure a serious failure involving authorization, safety, data handling, undisclosed access by third parties, material misrepresentation, or undefined deliverables.
A provider that fails a material authorization, safety, data-protection, transparency, or deliverable gate should not be rescued by a high total score.
Table 2. Minimum qualification gates to clear before weighted scoring.
| Mandatory gate | Pass evidence | Do not proceed when |
|---|---|---|
| Written authorization and ROE | Approved scope, asset ownership, third-party permission | No verified authority or boundaries |
| Production safety | Stop conditions, contacts, rate/window controls | No accountable emergency process |
| Data and report handling | Least privilege, secure transfer, retention/deletion, incident terms | Undefined protection for credentials, evidence, or reports |
| Team and workforce transparency | Assigned lead, role mix, third-party disclosure, access governance | Concealed subcontracted or crowdsourced access |
| Deliverables and retesting | Comparable sample, outputs, limitations, escalation, retests | Vague outputs or guaranteed outcomes |
Figure 1. The DeepStrike PENTEST Vendor Evidence Scorecard compares providers across seven evidence-based dimensions while keeping authorization, safety, and data handling as hard gates.
Why it matters: The people shown during sales may not be the people testing your systems. Assigned-team quality, availability, and fit affect depth, judgment, and communication.
Strong-answer signals: Named roles or profiles, relevant recent work, clear senior oversight, and a process for approving substitutions.
Red flags: Only founder biographies, vague “global expert” claims, or refusal to discuss the delivery team.
Evidence to request: Sanitized tester profiles, engagement-relevant experience summaries, role allocation, and the quality-assurance (QA) reviewer’s role.
Useful follow-up: What happens if an assigned tester becomes unavailable, and who approves the replacement?
Why it matters: No workforce model is inherently good or bad. The key issues are disclosure, screening, least-privilege access, accountability, continuity, confidentiality, and supervision.
Strong-answer signals: A transparent model, documented onboarding and screening, need-to-know access, named accountability, secure offboarding, and consistent QA.
Red flags: Concealed subcontracting, uncontrolled researcher access, unclear data locations, or “our platform handles it” without governance detail.
Evidence to request: Workforce-model statement, subcontractor list or notification process, access-control summary, confidentiality requirements, and QA workflow.
Useful follow-up: Which people or entities can access our credentials, systems, evidence, and report at each stage?
Why it matters: Generic years of experience do not prove fit for a multi-tenant API, identity-heavy cloud environment, mobile application, industrial network, or fragile production system.
Strong-answer signals: Relevant examples expressed without client secrets, recognition of your trust boundaries and failure modes, and honest identification of specialist gaps.
Red flags: Industry labels used as a proxy for technical experience or claims that one methodology covers every architecture equally.
Evidence to request: Sanitized comparable-scope summaries, public research, training, or references that can be verified with permission.
Useful follow-up: Which part of our environment presents the greatest coverage risk, and how would you address it?
Why it matters: Individual certification, organizational accreditation, framework familiarity, and practical experience are different signals. None proves the quality of this specific engagement by itself.
Strong-answer signals: The provider explains relevance and limits, verifies current status, and links credentials to assigned roles and test objectives. CREST’s procurement guidance likewise treats accreditation and professional certification as due-diligence inputs, not substitutes for engagement fit.
Red flags: Logo walls, expired credentials, universally mandatory claims, or implying that OSCP, CREST, CHECK, PCI qualifications, or another credential guarantees outcomes.
Evidence to request: Verifiable credential records, accreditation scope, expiry or renewal status, and the assigned person’s role.
Useful follow-up: What capability required by our engagement is not demonstrated by these credentials?
Why it matters: Buyers need proof, but a trustworthy provider should not share another customer’s report, live finding, credentials, or internal architecture.
Strong-answer signals: Sanitized samples, documented peer review, reproducibility checks, severity calibration, secure evidence handling, public research, controlled references, and continuing training.
Red flags: Real client artifacts without consent, unverifiable claims, or quality measured only by the number of findings.
Evidence to request: Sanitized sample report, QA checklist, review responsibilities, correction process, and permitted references or public work.
Useful follow-up: Show how a high-impact conclusion moves from tester observation to validated, reviewed report finding.
Figure 2. A defensible vendor decision moves from a standardized question to evidence, verification, hard-gate review, and documented scoring.
Why it matters: A useful scope connects the business decision to assets, trust boundaries, user roles, attacker perspectives, data flows, and expected evidence not just an asset count.
Strong-answer signals: The provider asks what matters, maps objectives to test perspectives and coverage, and explains tradeoffs and exclusions.
Red flags: A fixed package before meaningful discovery or a scope based only on IPs, pages, or endpoint counts.
Evidence to request: Proposed objective-to-coverage mapping, assumptions, role matrix, architecture inputs, and exclusions. Use the separate penetration testing scope guide for detailed planning.
Useful follow-up: Which objective would be least supported by the proposed scope, and why?
Why it matters: Missing roles, credentials, API definitions, cloud access, or architecture context can create blind spots and wasted tester time.
Strong-answer signals: A dependency list tied to coverage, a secure exchange process, fallback options, and explicit limitations if inputs arrive late or not at all.
Red flags: “We need nothing” for a complex authenticated scope, or dependencies revealed only after testing starts.
Evidence to request: Pre-engagement checklist, responsibility matrix, secure-transfer method, access dates, and limitation language.
Useful follow-up: Which three inputs would most improve depth, and what happens if we cannot provide them?
Why it matters: Ambiguous boundaries create coverage disputes and operational or legal risk. Scope says what may be tested; ROE says how.
Strong-answer signals: Version-controlled scope, explicit ownership, environment labels, time zones, exclusions, named approvers, and a written change process.
Red flags: Scope spread across emails, verbal additions, or testing before the approved documents are final.
Evidence to request: Scope template, change-request process, ROE reference, and deliverable mapping. See the separate penetration testing statement of work guide for work-structure detail.
Useful follow-up: Who can authorize a scope change, and how will it affect effort, dates, and deliverables?
Why it matters: Black-box, gray-box, white-box, external, internal, authenticated, red-team, point-in-time, and recurring models answer different questions.
Strong-answer signals: A recommendation tied to the objective, available access, assurance need, threat model, and constraints, with acknowledged tradeoffs.
Red flags: One model presented as universally superior or recurring testing sold without explaining what changes between cycles.
Evidence to request: Option comparison, proposed perspective, assumptions, expected coverage, and reasons alternatives were rejected.
Useful follow-up: What would we learn and fail to learn under your recommended model?
Why it matters: NIST defines ROE as detailed testing constraints established before a test that authorize defined activities. Testing should not begin without authority from the relevant asset owner.
Strong-answer signals: Written targets, permitted and prohibited activities, dates, contacts, escalation, restrictions, evidence rules, and signatures or approvals.
Red flags: Willingness to “start discovery” first, customer email treated as sufficient for third-party systems, or no authority chain.
Evidence to request: Redacted authorization and ROE templates, approval workflow, and responsibility map. NIST SP 800-115 and the NIST ROE definition provide useful planning context.
Useful follow-up: Who must approve testing when the buyer, operator, host, and asset owner are different entities?
Why it matters: Authorized tests can still cause disruption if traffic, fragile services, or proof activities are poorly controlled.
Strong-answer signals: Agreed windows, rate and concurrency boundaries, prohibited activities, monitoring coordination, safe evidence collection, named contacts, and clear stop/resume authority.
Red flags: “We have never caused downtime,” undefined emergency contacts, or availability testing implied without explicit approval.
Evidence to request: Production-safety plan, stop-condition examples, escalation tree, source-IP list, and incident-handling process.
Useful follow-up: Give an example of a condition that would make the tester stop immediately and how the client would be notified.
Why it matters: A buyer cannot automatically authorize testing against infrastructure, tenants, or data it does not own or control. Provider rules also change.
Strong-answer signals: Asset-owner verification, separation of customer and provider layers, current-policy review, written permission where required, and excluded shared infrastructure.
Red flags: “It is in our environment, so it is authorized,” or reliance on outdated cloud-policy summaries.
Evidence to request: Ownership matrix, permission records, policy-review date, and cloud/SaaS exclusions. Check the current official policies for AWS, Microsoft online assets, and Google Cloud at engagement time.
Useful follow-up: Which proposed targets require permission or policy validation beyond our own authorization?
Why it matters: Pentest artifacts can contain privileged access, personal data, architecture details, and exploitable evidence.
Strong-answer signals: Data minimization, least privilege, appropriate encryption, access logging, approved locations, retention periods, secure delivery, deletion or return, backup treatment, and incident response.
Red flags: Indefinite retention, shared credentials, consumer file links, vague “industry-standard security,” or no answer for backups and AI services.
Evidence to request: Data-flow summary, retention schedule, deletion attestation process, access roles, incident-notification process, and report-delivery method.
Useful follow-up: After deletion, what copies can remain in backups, logs, ticketing systems, or third-party platforms, and for how long?
Why it matters: References support coverage and repeatability, but a checklist cannot understand your architecture, business process, or attacker incentives.
Strong-answer signals: A documented internal methodology informed by relevant references, with risk-based tailoring and traceable limitations. Depending on scope, useful sources include NIST SP 800-115, the OWASP Web Security Testing Guide, OWASP ASVS, the OWASP API Security Project, OWASP MASTG, PTES, and CREST guidance.
Red flags: Framework names without a coverage map, or “100% compliant with OWASP” used as a quality guarantee.
Evidence to request: Methodology summary, coverage matrix, tailoring rationale, and limitation process. The penetration testing methodology guide provides deeper context.
Useful follow-up: Which reference areas are relevant to our scope, and which are intentionally excluded?
Why it matters: Automation supports discovery and repeatability; expert analysis is needed for context, authorization boundaries, business logic, abuse paths, and chained weaknesses.
Strong-answer signals: A task-based explanation of automation, manual validation, false-positive handling, and tester time. No arbitrary percentage is required.
Red flags: Scanner output sold as a penetration test, a fixed “manual” percentage with no activity detail, or unvalidated tool findings.
Evidence to request: Activity breakdown, validation workflow, sample finding lineage, and assigned tester effort. See vulnerability assessment vs penetration testing for the distinction.
Useful follow-up: Which important risks in our environment are least likely to be found by your automated tools?
Why it matters: High-impact application and cloud risk often appears in the relationships among roles, objects, workflows, trust boundaries, and individually modest weaknesses.
Strong-answer signals: Role and object matrices, workflow mapping, abuse-case development, controlled attack-path validation, and attention to tenant separation and compensating controls.
Red flags: Coverage framed mainly around known-vulnerability databases, generic OWASP labels, or endpoint enumeration without role testing.
Evidence to request: Sanitized coverage examples, role/test matrix, attack-chain reporting example, and limitations.
Useful follow-up: How would you demonstrate authorization coverage across multiple roles without exposing production customer data?
Why it matters: A test is bounded by time, access, and scope. It cannot prove that vulnerabilities are absent, so traceable coverage and honest residual limitations matter.
Strong-answer signals: Coverage records, reproducibility checks, affected-asset mapping, independent review of high-impact findings, severity calibration, and explicit untested or blocked areas.
Red flags: “Zero findings means secure,” suppressed limitations, volume-based quality measures, or critical claims without review.
Evidence to request: Coverage log example, validation checklist, QA responsibilities, false-positive correction process, and limitations section.
Useful follow-up: How will the final report distinguish tested-and-not-observed, not tested, inaccessible, and out of scope?
Why it matters: AI can assist research, analysis, and drafting, but it can also expose customer data, introduce errors, or obscure accountability.
Strong-answer signals: Disclosed use cases, approved models and data locations, controls on customer inputs, retention settings, human validation, secure coding boundaries, and a named accountable reviewer.
Red flags: Sensitive data sent to unspecified third-party models, AI-generated findings without tester validation, or claims that AI replaces accountable expertise.
Evidence to request: AI-use policy, data-flow and retention summary, opt-out or approval process, human-review controls, and third-party list.
Useful follow-up: Could any credentials, source code, logs, screenshots, or report content leave the agreed processing boundary through an AI service?
Why it matters: A sample report reveals whether the provider turns observations into clear, validated, stakeholder-appropriate evidence.
Strong-answer signals: Defined executive and technical outputs, objective and scope clarity, affected assets, validated evidence, business impact, severity rationale, remediation, limitations, and optional readouts where contracted.
Red flags: Raw scanner output, a sample unrelated to the proposed service, generic remediation, or deliverables left until project close.
Evidence to request: Sanitized comparable sample, deliverable list, format, secure-delivery method, review stages, and acceptance criteria.
Useful follow-up: Which deliverable is intended for engineers, executives, auditors, and risk owners respectively?
Why it matters: Common Vulnerability Scoring System (CVSS) data can support consistency, but a numeric vector alone does not decide business priority.
Strong-answer signals: Transparent technical scoring, contextual adjustments, documented assumptions, chain-level analysis, affected data and assets, compensating controls, and risk-owner input where appropriate.
Red flags: CVSS treated as the whole risk decision, severity inflated for impact, or undocumented changes after draft review.
Evidence to request: Severity rubric, sample rationale, chain-handling approach, dispute process, and residual-risk field.
Useful follow-up: Show how the same technical weakness could receive different business priority in two different asset contexts.
Why it matters: A final report may arrive too late for an actively exploitable, high-impact issue. Routine communication also resolves blockers before they consume the test window.
Strong-answer signals: Named contacts, agreed cadence, secure channels, urgent-notification process, acknowledgment and escalation path, draft review, technical readout, executive readout, and final sign-off.
Red flags: Critical issues held for the final report, sensitive findings sent through unapproved channels, or no backup contact.
Evidence to request: Communication plan, severity-trigger examples, escalation tree, secure-channel details, and draft/final workflow.
Useful follow-up: What triggers immediate notification, who decides, and what minimum evidence is sent before full validation is complete?
Why it matters: Engineers may need clarification, prioritization, or design discussion to fix a finding safely without creating a different weakness.
Strong-answer signals: Contracted finding walkthroughs, reasonable clarification, developer sessions, prioritization support, ticket comments, or remediation review with boundaries and named owners.
Red flags: “The report is self-explanatory,” sales promises absent from the proposal, or support billed unexpectedly.
Evidence to request: Included hours or activities, support window, communication channel, excluded architecture work, and additional-rate terms.
Useful follow-up: What happens when our team believes the proposed fix is impractical or creates a new risk?
Why it matters: “Retesting included” can mean one check, one round, a time-limited window, or a separate assessment. Ambiguity creates closure disputes.
Strong-answer signals: Eligible findings, window, rounds, evidence standard, new-code and regression boundaries, scope-change treatment, closure states, updated outputs, and extra-charge rules.
Red flags: Unlimited or free retesting assumed but not written, retest status without evidence, or new vulnerabilities silently excluded without explanation.
Evidence to request: Retest terms, status taxonomy, sample validation letter or updated report, scheduling rules, and price for additional rounds.
Useful follow-up: If a fix changes the vulnerable workflow substantially, what validation remains included and what becomes new scope?
Why it matters: Two prices are not comparable when they cover different assets, roles, tester effort, meetings, reports, or retesting.
Strong-answer signals: Transparent assumptions, tester roles and effort, deliverables, meetings, retesting, travel, after-hours work, buyer dependencies, validity period, and change-control rates.
Red flags: A low headline price with essential work treated as extras, undisclosed junior delivery, or fees that cannot be traced to scope.
Evidence to request: Itemized proposal, effort model, inclusions/exclusions, dependency dates, change-order process, and cancellation or rescheduling terms.
Useful follow-up: Which three assumptions would most likely change this quote?
Why it matters: The contract must support the actual authorization, access model, data flow, deliverables, and risk allocation. This requires qualified legal and procurement review.
Strong-answer signals: Clear authorization, confidentiality, subcontractor disclosure, data handling, incident responsibility, intellectual-property treatment, relevant insurance evidence, liability allocation, acceptance, cancellation, and dispute processes.
Red flags: Testing authority implied rather than documented, hidden subcontracting, inconsistent data terms, guarantees of compliance or security, or refusal to define deliverables and retesting.
Evidence to request: Proposed agreement set, data-processing terms where applicable, subcontractor schedule, insurance certificate where relevant, and responsibility matrix.
Useful follow-up: Which contract term would prevent testing or alter delivery if it remains unresolved? This section is educational, not legal advice; involve legal, procurement, security, privacy, compliance, and asset owners.
Vendor quality is partly visible in discovery. A thoughtful provider should ask questions that change scope, safety, effort, or evidence not merely collect a domain and deadline.
A provider that confidently fixes scope and promises depth without asking meaningful environmental questions may be quoting a standardized package rather than your actual risk. PTES’s pre-engagement guidance similarly emphasizes goals, scope, third parties, communications, evidence handling, and permission before testing.
Ask how authenticated roles, authorization, business logic, file handling, payments, and workflow abuse will be covered, with a tailored map to relevant OWASP WSTG or ASVS areas.
Ask how endpoint inventory, tokens, object-level authorization, tenant separation, REST or GraphQL behavior, rate constraints, and undocumented dependencies will be tested.
Ask which accounts, subscriptions, projects, identity paths, containers, Kubernetes, serverless services, and trust relationships are in scope, then confirm current provider policies.
Ask whether testing covers client behavior, local storage, platform controls, backend APIs, builds, devices, and relevant OWASP MASTG coverage.
Ask how asset ownership, legacy systems, sensitive services, segmentation, directory services, test accounts, and production constraints will be controlled.
Ask about objectives, targets, permitted techniques, legal and HR approval, payload safety, deconfliction, detection goals, evidence boundaries, and stop conditions.
Ask what is human-led, automated, or crowdsourced and how continuity, cadence, access, integrations, retesting, deliverables, and pricing units work.
Ask how the provider defines model and application boundaries, data exposure, connected tools, agent permissions, third parties, safety constraints, and evidence handling.
A sample report is useful only if it resembles the proposed engagement and is sanitized without removing the structure needed for evaluation. Check whether it clearly shows:
A weak sample may reveal scanner output presented as a final report, generic descriptions unrelated to the environment, no affected-asset mapping, no business context, no limitations, no evidence of validation, vague remediation, or a large volume of low-value observations used to create the appearance of depth. Never ask a provider to disclose a real client report without permission.
Headline price is not a valid comparison when proposals describe different work. Normalize each quote against the same baseline:
Record unresolved assumptions beside the price. A lower quote may represent excellent efficiency or materially less coverage; a higher quote may represent more effort or simply a different commercial model. Ask the vendor to explain the difference. Use the separate penetration testing cost guide for budgeting context, not as a quality threshold.
Compliance requests should be normalized too. PCI DSS v4.0.1 contains explicit penetration-testing requirements for applicable payment environments, but applicability and validation responsibility must be confirmed through the current PCI SSC resource hub and the entity managing compliance. SOC 2, HIPAA, ISO/IEC 27001, customer assurance, and cyber-insurance requests vary. A penetration test can support a wider assurance program; it does not prove overall compliance.
Preference criteria can be scored. The following issues should be resolved as hard gates:
Clarify once and request evidence. If a material authorization, safety, transparency, or data-handling failure remains unresolved, do not shortlist the provider even if its weighted score is high.
Tailor asset names, environments, and assurance requirements before sending. For a complete procurement workflow, use the penetration testing RFP guide.
Figure 3. Hard authorization, safety, transparency, and data-handling gates should be resolved before a provider advances to weighted comparison.
☐ Objective, assets, roles, environments, constraints, and evidence needs are defined.
☐ Scope and effort are comparable across proposals.
☐ Assigned testers and workforce model are reviewed.
☐ Relevant experience and qualifications are verified.
☐ Methodology, manual depth, automation, AI use, and limitations are explained.
☐ Written authorization, ROE, third-party permission, and stop conditions are approved.
☐ Data, credential, evidence, report, retention, deletion, and incident terms are reviewed.
☐ Sanitized sample report and QA process are assessed.
☐ Critical-finding escalation and stakeholder communication are defined.
☐ Remediation support and retesting terms are written.
☐ Price is normalized against effort, deliverables, exclusions, and change terms.
☐ Contract, references or equivalent evidence, and hard gates are cleared.
☐ PENTEST scores and any category floors are documented.
☐ Final decision rationale and residual concerns are recorded.
Ask who will test, whether experience fits, and how scope, authorization, manual depth, safety, data, reporting, remediation, retesting, price, and contracts work. Request evidence and use the same questions for every provider.
Ask the provider to map automated and manual work to scope and effort. Raw scanner output, unvalidated findings, or no plan for business logic and authorization testing are warning signs.
Yes. Request a sanitized comparable report and review scope, limitations, methodology, affected assets, validated evidence, impact, severity rationale, remediation, escalation, and retesting.
Define eligible findings, window, rounds, evidence, scheduling, closure statuses, accepted or unresolved risk, updated deliverables, changed-code treatment, and charges.
Normalize scope, complexity, tester effort, manual work, constraints, reports, escalation, remediation, retesting, workforce, data handling, exclusions, and changes. Compare the contracted work not the headline price.
No. Credentials support due diligence but do not guarantee the assigned team, test depth, compliance result, future security, or breach prevention. A penetration test is one bounded assurance input.
Choose a penetration testing company by verifying the assigned people, technical depth, authorization, safety, evidence, reporting, remediation, retesting, and commercial terms. Use the PENTEST scorecard for consistent comparison, but treat it as a decision aid not a guarantee. Written authorization, agreed ROE, asset-owner permission, secure data handling, and professional judgment remain essential.
Once your objectives, scope, safety requirements, and evidence needs are defined, DeepStrike can discuss an authorized penetration testing engagement across the service areas currently listed on its website. Confirm the final scope, rules of engagement, deliverables, retesting terms, and contractual requirements before testing begins.
Mohammed Khalil is a Cybersecurity Architect at DeepStrike specializing in advanced penetration testing and offensive security. His certifications include CISSP, OSCP, and OSWE. His work focuses on application security, API security, cloud security, identity exposure, attack-path validation, and remediation-focused security testing.
Stay secure with DeepStrike penetration testing services. Reach out for a quote or customized technical proposal today
Contact Us
