Ninety minutes. A sales engineer with a clean demo account, a fictional city, and a registration that goes through on the first click every single time. Everyone nods. Six months later your staff is doing manual workarounds for a scholarship discount that will not stack with a sibling discount.
The demo was not a lie. It just was not a test. Nobody made it one.
Run your scenarios, not theirs
Send every finalist the same four scenarios one week ahead, in writing, with your real fee table and your real facility list. Tell them they will drive, live, with your staff watching the screen. No slides, no video, no "we can show that offline."
Then hold them to it. A vendor who cannot do it in a controlled demo will not do it in March with 400 families online.
The four scenarios
- Sibling registration with a scholarship. One parent, three kids, two different programs, a fee assistance award applied to two of them and not the third. Does the discount stack with the sibling rate? What does the receipt say? What does the revenue report say the program actually earned?
- The double booked field. Put a league block and a rental request on the same field, same hour. Does the system stop it, warn about it, or cheerfully take the money twice? Then have staff resolve it and see who gets notified.
- The mid season refund. A kid breaks an arm in week 4 of an 8 week program. Prorate it, refund partially to the original card, credit the rest to the household account, and produce the audit trail your finance office will want.
- The failed payment. A recurring membership charge declines. What happens automatically, what does the family see, how many days until access is cut, and how does the front desk retry it without asking for the card number over the phone?
Score it while you are watching
Build a simple rubric and fill it in during the session, not after. Three columns per scenario: completed without a workaround, number of screens the staff member had to touch, and did the money end up right. Every evaluator scores independently before anyone talks. The conversation after that is worth more than the demo itself.
Weight the scenarios against your own scoring criteria, and notice how much of your published rubric actually measures software behavior versus proposal writing.
The reference calls are the real product review
Two published RFPs show what a serious reference requirement looks like. Rock Island, Illinois asked for three clients of similar size currently live on the system. Redmond, Washington went further in its registration and facility booking RFP, requiring the installation date and the deployed version for each reference, and reserving the right to contact references without notice to the vendor.
Those two details do the work. Similar size stops you from hearing about an agency ten times your scale. Installation date and version stop you from hearing about software you are not being sold.
Questions that surface problems
- What did go live week actually look like, and what broke?
- What did you pay in year one that was not in the proposal?
- What does your staff still do in a spreadsheet?
- How long does a support ticket take, and what happened the last time you had an outage during registration?
- Would you buy it again? If not, what would you buy?
If a vendor cannot survive four scenarios and five phone calls, it will not survive your season. RecreationHQ hands out its scenario list before the call, which is a habit we would recommend regardless of who you buy from. Start with the free cost recovery audit if you want a number to evaluate against.
Find your leaks
See what your organization could recover.
A 20-minute audit, your numbers, no obligation.
Get my cost-recovery audit →Or get one practical playbook like this by email, every week.