Printable pilot resource
Define the baseline, record quality and workflow evidence, track failures and remediation, and make a written go, revise, or stop decision.
Define the comparison baseline, unit of analysis, reviewer instructions, severity scale, minimum sample, decision thresholds, and stop conditions in writing. Use representative approved scenarios and the same review standard for the current workflow and the pilot workflow. This scorecard supports evaluation; it does not determine legal sufficiency, CJIS compliance, or whether a product is appropriate for an agency.
Read the pilot planning guideComplete one cover sheet for the pilot and identify any changes to scope, reviewers, or measurement rules as they occur.
Use the same start and end points for the current process and the pilot. Generation speed alone does not measure the work required to produce a supervisor-ready draft.
Elapsed time from the agreed starting event to a supervisor-ready draft, including waiting, review, corrections, and export steps.
Record as: Median minutes; also note the range and outliers
Time the officer actively spends reviewing, correcting, adding, deleting, or reorganizing draft language before it is ready for supervisor review.
Record as: Median minutes and percentage of total time
Drafts returned for factual, clarity, completeness, attribution, policy, formatting, or other corrections. Record the reason for every return.
Record as: Count and percentage of reviewed drafts, by reason
Have trained reviewers compare each pilot draft with the approved source set and the agency's normal review standard. Count findings consistently and retain examples for adjudication.
Statements in the draft that are not supported by the officer-provided facts or another approved, identified source. Do not combine these with style preferences.
Record as: Findings per draft and percentage of drafts affected
Relevant supplied facts absent from the draft when the agency's review standard calls for their inclusion. Record why the omission matters to documentation quality.
Record as: Findings per draft, severity, and percentage affected
Language assigned to the wrong person, event, document, policy, instruction, or other source, including an incorrect or untraceable source reference.
Record as: Count by error type, severity, and source category
Separate system reliability from output quality, and collect structured feedback alongside open comments. A favorable comment does not offset an unresolved material finding.
Attempts that could not complete because of a product, model-provider, identity, network, export, or integration issue. Record duration, impact, recovery, and recurrence.
Record as: Failure rate, unavailable minutes, and affected workflows
Officer and reviewer ratings for usefulness, clarity of prompts, review burden, trust calibration, training sufficiency, and workflow fit, plus categorized comments.
Record as: Rating distribution, response count, and recurring themes
Record each finding separately. Include enough context for a reviewer who did not observe the original workflow to reproduce or adjudicate it.
Finding types
Unsupported statement · Material omission · Attribution/source error · Supervisor return · System failure · User-reported issue
Written decision gate
Compare the results with the thresholds approved before the pilot. Document exceptions and unresolved findings; do not treat an overall average as a substitute for reviewing high-severity failures.
Go
Approved scope may proceed subject to the conditions below.
Revise and re-test
Remediation and a defined re-test are required before expansion.
Stop
The pilot will not proceed in its current form.
Results apply only to the tested scenarios, users, product version, configuration, sources, deployment, and review process. Reassess material changes and validate agency requirements with the appropriate operational, security, privacy, procurement, and legal reviewers.
LeoPen resource · Updated September 1, 2026
Bring your workflows, representative scenarios, review standard, and decision thresholds to a focused planning session.
Schedule a working session