The product demo is the most polished version of any vendor’s story. The interface is clean, the scenarios are optimized for the features that look best, and the presenter knows every question that is likely to come up and has prepared answers for each. A demo tells you what a downtime solution is designed to do under ideal conditions. It does not tell you how the integration holds up six months after implementation when the EHR is upgraded. It does not tell you how responsive the support team is at 2 AM during an actual outage. It does not tell you whether the organization that purchased the solution three years ago would purchase it again.
Reference calls do. But only if they are structured to produce honest, specific information rather than the validated testimonials that vendors curate and prepare their reference customers to deliver.
The healthcare organizations that make the best vendor selection decisions treat reference calls as an investigation rather than a formality. They ask different questions than the vendor expects, they listen for what is not said as much as what is said, and they use the patterns across multiple references to build a picture of the vendor that the demo never provides.
Why Standard Reference Calls Produce Useless Information
Most reference calls in healthcare IT procurement follow a predictable and largely uninformative pattern. The evaluating organization contacts the references provided by the vendor, asks whether they are happy with the solution, and receives confirmation that they are. The call ends in 15 minutes with no information that was not already implied by the vendor’s willingness to provide the reference in the first place.
This outcome is almost guaranteed by how the call is structured. Vendor-curated references are selected because they will say positive things. They have been contacted in advance by the vendor, know what the call is about, and have implicitly agreed to serve as advocates. Asking them open-ended questions about their satisfaction produces positive responses that tell the evaluating organization nothing it did not already know.
The solution is not to distrust references but to ask questions that require specific, detailed responses rather than general endorsements. Specific, detailed responses are harder to curate and harder to deliver without revealing something honest, even from a customer who intends to be entirely positive.
The Questions That Produce Genuinely Useful Reference Information
The questions that produce the most useful reference information share a common characteristic: they ask about specific experiences rather than overall assessments. The specific questions that consistently reveal the most about a downtime vendor include:
Walk me through the last actual downtime event you used the system for. Not a drill, an actual event. What worked exactly as you expected, what surprised you, and what did not work as well as you had hoped?
This question is the single most revealing question in any downtime vendor reference call. It requires the reference to describe a real event in specific terms, which reveals gaps that general satisfaction questions never surface. A reference that struggles to answer this question because they have not had an actual event may reveal that the system has never been tested under real conditions. A reference that answers it enthusiastically with specific details about what held up and what did not is providing genuinely useful evaluation input.
Describe your support experience when something went wrong. Not the sales process, not the implementation, the moment when you had a problem and needed help. Who did you contact, how quickly did you hear back, and what was the resolution process?
Support quality is one of the most significant differentiators between downtime vendors and one of the hardest to evaluate from a demo. References who have experienced real support interactions can describe them specifically in ways that reveal whether the vendor treats post-sale support as a priority or as a cost center.
How current is your forms library right now, and who maintains it? How does it stay updated when your workflows change?
This question targets one of the most common sources of downtime solution degradation over time. A forms library that is not actively maintained drifts from current clinical workflows, reducing the solution’s effectiveness during actual events. References who describe a clear, functional forms maintenance process indicate a vendor that takes ongoing program quality seriously. References who are uncertain about their forms library currency are revealing a gap that the evaluating organization will likely inherit.
What did the HL7 integration look like after your last EHR upgrade? Did the vendor support the transition proactively, or did you discover issues during an event?
EHR upgrades are a recurring source of integration failures in downtime solutions, and how the vendor handles them is a reliable indicator of their long-term support quality. References who describe proactive vendor engagement during EHR transitions indicate a different kind of vendor relationship than those who describe discovering integration problems during an actual outage.
If you were selecting a downtime solution today with everything you know now, would you choose this vendor again, and is there anything about the selection you would do differently?
This question is more revealing than it appears. A reference who answers it with unqualified enthusiasm is providing one data point. A reference who pauses, qualifies their answer, or describes what they would do differently is providing much more useful input, even if their overall assessment remains positive.
Finding References the Vendor Did Not Provide
The most valuable references in any vendor evaluation are the ones the vendor did not offer. Vendor-curated references are selected because they are positive. References that the evaluating organization finds independently are self-selected for a different reason: they have something to say, which may be positive, negative, or nuanced in ways that vendor-curated references never are.
Sources for independent references include professional associations such as HIMSS and state hospital associations where peer networking can identify organizations using specific vendors, LinkedIn searches for healthcare IT professionals at organizations known to use the vendor, and direct outreach to organizations in the vendor’s published case studies or testimonials, who were curated as positive references but may have more to say than the vendor’s marketing material reveals.
When an independent reference is identified, ask the same specific questions described above. The responses from an independent reference are more reliable precisely because they have not been prepared, and the comparison between curated and independent references often reveals the most honest picture of the vendor’s real performance.
What to Listen for That References Will Not Say Directly
The most useful information from reference calls is often not in the direct answers but in the patterns of what references say, how they say it, and what they consistently avoid saying.
References who are enthusiastic in general terms but become vague or brief when describing specific experiences may be curated advocates who have less direct experience with the solution than their enthusiasm implies. A reference who has only used the solution during planned maintenance windows and has never experienced an actual unplanned outage is providing a fundamentally different kind of validation than one who has managed a 12-hour ransomware-driven event using the system.
References who volunteer criticisms without being asked are often the most valuable, because they are demonstrating a comfort with the vendor relationship that comes from genuine confidence in the vendor’s ability to address problems. A reference who says something like the implementation took longer than expected but the vendor was transparent throughout and the final product was exactly what we needed is telling you something more useful than one who says everything was perfect.
References who describe a vendor that proactively identified problems, communicated clearly, and resolved issues without requiring the customer to escalate repeatedly are describing a fundamentally different vendor than one where every resolution required multiple contacts and escalation to senior staff.
Using Reference Patterns to Make the Decision
No single reference call produces a complete picture. The pattern across four to six references, including both vendor-curated and independently identified contacts, is what produces an actionable evaluation. Patterns worth identifying include whether multiple references describe the same strengths, which confirms those strengths are consistent rather than situational, whether multiple references describe the same gaps or frustrations, which suggests structural issues rather than individual experiences, and whether the references who have been using the solution longest are the most or least enthusiastic, which reveals whether the vendor relationship and product quality hold up over time.
dbtech is confident in what reference calls reveal and actively encourages prospective customers to speak with current clients, including those who have managed actual unplanned outage events with the system. To request reference contacts or discuss the evaluation process, contact our team or request a demo.