Chapter 13 Part III — Data, Standards, and Interoperability
Analytics, Visualization, Quality Improvement, and Decision Support
How informatics turns data into defensible measures, useful visualizations, and decisions without confusing correlation with causation or dashboards with improvement.
Chapter Orientation
Analytics can create an illusion of certainty because numbers appear precise. A rate reported as 8.7% looks objective even when the denominator is poorly defined, the data are incomplete, and the population has changed. Informatics professionals must therefore understand not only how metrics are calculated but what claim the metric can legitimately support.
This chapter treats analytics as a reasoning discipline. The central question is not “What does the dashboard say?” but “What evidence does this measure provide, under what assumptions, for which decision?”
Learning Objectives
By the end of this chapter, you should be able to:
- Construct and critique basic rates, proportions, denominators, and benchmarks.
- Distinguish descriptive, diagnostic, predictive, and prescriptive analytics.
- Explain why association does not establish causation.
- Design visualizations that match the decision and reduce misinterpretation.
- Apply root-cause analysis and FMEA appropriately.
- Evaluate dashboards and decision-support outputs for actionability and unintended effects.
Lesson 13.1 — Measures Are Arguments Encoded as Numbers
-
Every metric has a numerator, denominator, time frame, population, and definition. If any of these are unclear, the metric is not yet interpretable. “Fall rate” could mean falls per resident-day, residents with a fall, or claims containing a fall diagnosis.
-
Denominators define who had the opportunity to experience the event. A catheter-associated infection measure requires an exposure denominator different from a general patient census. An inappropriate denominator can make performance appear better or worse without any clinical change.
-
Counts, proportions, rates, and ratios answer different questions. Ten pressure injuries may be alarming in a 20-bed unit and less informative in a 1,000-bed network. Normalization allows comparison, but only when populations are sufficiently comparable.
-
Benchmarks require contextual similarity. Comparing a rehabilitation hospital with a low-acuity ambulatory clinic would create a technically valid arithmetic comparison and a clinically meaningless performance conclusion.
-
Missingness can bias metrics. If documentation is more complete on one unit than another, the unit with better reporting may appear to have worse outcomes. Measurement systems can reward under-detection if data capture is ignored.
Clinical Example
Clinical Example — The “High” Fall Rate
A memory-care unit has more falls per 1,000 resident-days than other units. Before labeling performance poor, analysts should examine case mix, mobility goals, injury severity, observation practices, documentation completeness, and whether units use the same definitions.
NI-BC Connection: Data Management and Healthcare Technology — analytics, reporting, quality measures.
Retrieval Checkpoint
Retrieval Checkpoint
- Which components must be known before interpreting a rate?
- Why is denominator selection a clinical decision?
- When can benchmarking become misleading?
- How can better documentation make performance appear worse?
Lesson 13.2 — Analytics Types Describe Different Questions
-
Descriptive analytics ask what happened. Counts, averages, trends, and distributions summarize observed data. They are foundational because more advanced analysis cannot compensate for misunderstanding basic patterns.
-
Diagnostic analytics ask what might explain what happened. Stratification, drill-downs, comparisons, and relationship analysis can identify plausible drivers. They generate explanations to investigate rather than proving causality automatically.
-
Predictive analytics estimate what may happen. Statistical and machine-learning models use patterns in existing data to estimate outcomes or risk. Prediction can be useful even without full causal understanding, but its performance depends on population, data quality, and deployment context.
-
Prescriptive analytics recommend actions under defined objectives and constraints. This may involve optimization, decision rules, or simulated tradeoffs. A recommendation adds normative assumptions: which outcome matters, how risks are weighted, and what constraints are acceptable.
-
Analytics maturity is not a ladder where “predictive” is always better. A poorly defined predictive model is less useful than a clear descriptive measure that answers the actual operational question.
AI in Practice
AI in Practice — Ask AI to Label the Claim
When reviewing an analysis, ask an LLM to classify each statement as descriptive observation, hypothesis, prediction, causal claim, or recommendation. Then verify the classification yourself. This is useful for finding places where a report quietly jumps from “associated with” to “caused by.”
NI-BC Connection: Data Management and Healthcare Technology — analytics and decision support.
Retrieval Checkpoint
Retrieval Checkpoint
- How do descriptive and diagnostic analytics differ?
- What additional assumptions appear when analytics become prescriptive?
- Why is predictive analysis not automatically more valuable?
- Which type of claim requires the most caution about causality?
Lesson 13.3 — Correlation Is Useful, but Causation Requires Stronger Reasoning
-
An association tells you that variables move together, not why. Staffing level may be associated with falls because staffing influences supervision, because high-risk units receive more staffing, or because another factor affects both.
-
Confounding can create or distort relationships. A confounder is related to both the apparent exposure and outcome. Risk adjustment attempts to account for relevant differences, but it cannot guarantee that all important confounders are measured.
-
Temporal order matters. A factor cannot plausibly cause an outcome if it occurs afterward, yet poorly timestamped healthcare data can obscure sequencing.
-
Natural experiments, controlled studies, interrupted time-series analysis, and other designs can strengthen causal inference. Informatics teams should match the method to the decision rather than treating dashboard trends as proof.
-
Operational decisions can still be made under uncertainty. The correct response is not paralysis. Teams should state the strength of evidence, identify competing explanations, choose proportionate action, and monitor what happens next.
Informatics in Practice
Informatics in Practice — Replace “The Dashboard Proves”
Prefer language such as: “The increase coincided with…,” “The data are consistent with…,” “This analysis cannot distinguish between…,” or “We should test whether….” Precision about uncertainty improves decision quality.
NI-BC Connection: Foundations and Data/Technology — evidence-based practice, analysis, quality improvement.
Retrieval Checkpoint
Retrieval Checkpoint
- Why does association not establish mechanism?
- What is a confounder?
- Why is temporal order important?
- How can teams act responsibly when evidence is incomplete?
Lesson 13.4 — Visualization Should Reduce Cognitive Work
-
The chart type should match the question. Line charts reveal change over time; bars compare categories; scatterplots examine relationships; distributions reveal variation. Choosing a chart because it looks sophisticated can hide the actual pattern.
-
Axes and scales can change interpretation. Truncated axes may exaggerate small differences, while overly broad scales can hide meaningful change. The design should make the magnitude transparent rather than emotionally dramatic.
-
Color should encode meaning sparingly. Too many colors increase search burden and create accessibility problems. Use emphasis to direct attention to the decision-relevant pattern, not to decorate the dashboard.
-
Tables are sometimes better than charts. When users need exact values or only a few comparisons, a well-designed table can outperform a visualization.
-
Clinical dashboards should support action. A display should answer: What needs attention? Compared with what? Over what period? Who owns the response? How can the user investigate further? A collection of metrics without decision pathways is surveillance, not management.
Figure
Figure 13.1 — Match the Visualization to the Question
Production brief: Matrix: trend → line; category comparison → bar; distribution → histogram/box plot; relationship → scatter; exact small set → table. Include one “avoid” example such as a 3-D pie chart.
Alt text: Decision matrix showing common analytic questions and the visual form most suited to each.
NI-BC Connection: Data Management and Healthcare Technology — data visualization and reporting.
Retrieval Checkpoint
Retrieval Checkpoint
- Why should chart selection begin with the question?
- How can axis choices mislead?
- When may a table be preferable to a chart?
- What makes a clinical dashboard actionable?
Lesson 13.5 — Root-Cause Analysis and FMEA Look in Different Directions
-
Root-cause analysis examines an event that has already occurred. The goal is to understand contributing system factors and identify interventions that reduce recurrence. A strong RCA avoids stopping at “human error” when workflow, design, staffing, communication, training, or technology shaped the event.
-
FMEA is prospective. Failure Mode and Effects Analysis asks how a process could fail before harm occurs, estimates risk, and prioritizes mitigation. It is especially useful during new implementations or redesigns.
-
Both methods depend on process understanding. If the team maps an idealized workflow instead of actual practice, the analysis will miss workarounds and informal dependencies.
-
Risk scoring supports prioritization but should not replace judgment. Numeric severity/probability/detectability scores can create false precision. Clinically catastrophic but rare events may deserve attention even if a formula ranks them lower.
-
Corrective actions should change the system where possible. Reminders and retraining are weaker controls when a design can eliminate the error path, automate a reliable check, or reduce ambiguity.
AI in Practice
AI in Practice — Expand Failure Modes
Use an LLM to challenge an FMEA team with edge cases and alternative failure pathways after the team has mapped the process. Do not let AI assign final risk scores or replace frontline expertise; its value is breadth of hypothesis generation.
NI-BC Connection: Data Management and Healthcare Technology — RCA, FMEA, quality improvement.
Retrieval Checkpoint
Retrieval Checkpoint
- How do RCA and FMEA differ in time orientation?
- Why is “human error” usually an incomplete root cause?
- What is the limitation of risk scoring?
- Why are system-level controls often stronger than reminders?
Lesson 13.6 — Decision Support Must Connect Insight to Action
-
A prediction without an action pathway can add burden without benefit. If a model identifies high deterioration risk but staffing and escalation pathways cannot respond, the system may generate anxiety and alerts rather than better outcomes.
-
Thresholds encode tradeoffs. Lowering a threshold may identify more true cases but also generate more false positives. The correct threshold depends on consequences, prevalence, resource capacity, and whether missed cases or unnecessary interventions are more costly.
-
Performance should be evaluated after deployment. A model that performed well in retrospective validation may behave differently after workflow, population, or documentation patterns change.
-
Decision support should preserve clinician understanding. Users need enough explanation to know what the system is recommending, what information contributed, and what action is expected. Explanation needs differ by risk and use case.
-
Analytics and CDS can change the system they measure. Once clinicians know a metric or prediction drives action, documentation and behavior may adapt. Monitoring should look for gaming, workarounds, alert fatigue, and unintended effects.
Clinical Example
Clinical Example — The High-Risk List Nobody Can Work
A predictive model produces a daily list of 180 patients at high risk for readmission. Care management can contact only 35. The model may be statistically accurate, but the implementation has not connected prediction to operational capacity. A more useful design might optimize for the number of patients the team can actually serve.
NI-BC Connection: Data Management and Healthcare Technology — decision support, analytics, system performance.
Retrieval Checkpoint
Retrieval Checkpoint
- Why can accurate prediction fail operationally?
- What tradeoff changes when a decision threshold moves?
- Why is post-deployment evaluation necessary?
- How can a metric change the behavior it measures?
Chapter Case Study — The Readmission Dashboard
A health system launches a readmission-risk dashboard. The model identifies patients using EHR and claims data. Care managers complain that the highest-risk list is dominated by patients already receiving intensive services. Meanwhile, several patients with unstable housing and medication-access barriers are not flagged because social-risk data are inconsistently documented.
The dashboard displays a red risk score but does not show which factors contribute. Leadership wants the care-management team evaluated partly on the percentage of red patients contacted within 24 hours.
Analyze the case
- Which data-quality problems could affect the risk scores?
- What operational-capacity issue is present?
- How might the dashboard create inequitable effects?
- What additional explanation should users receive?
- What unintended behaviors might the performance metric create?
- Which measures would you use to evaluate the intervention after deployment?
Chapter Synthesis
- Measures are defined constructs, not self-explanatory facts.
- Analytics types answer different questions and support different levels of inference.
- Causal claims require stronger evidence than association.
- Visualization should reduce cognitive work and support decisions.
- RCA examines past events; FMEA anticipates future failure.
- Decision support creates value only when predictions connect to feasible action and ongoing evaluation.
Key Terminology
- Numerator
- Count of events or cases meeting the measure condition.
- Denominator
- Population or opportunity set against which the numerator is interpreted.
- Benchmark
- Reference value used for comparison.
- Descriptive analytics
- Analysis summarizing what happened.
- Predictive analytics
- Analysis estimating future or unknown outcomes from observed patterns.
- Confounding
- Distortion of an apparent relationship by another related factor.
- Risk adjustment
- Statistical approach intended to account for relevant differences among compared populations.
- RCA
- Root-cause analysis, a retrospective examination of contributing factors to an event.
- FMEA
- Failure Mode and Effects Analysis, a prospective method for identifying potential failures.
- Decision threshold
- Cutoff at which a score or probability triggers classification or action.
NI-BC Chapter Mapping
| Domain | Blueprint area | Lessons | Depth |
|---|---|---|---|
| III. Data/Technology | Analysis, reporting, visualization | 13.1–13.4 | Applied |
| III. Data/Technology | RCA and FMEA | 13.5 | Applied |
| III. Data/Technology | Decision support/analytics | 13.2, 13.6 | Applied |
| I. Foundations | Evidence-based practice | 13.3 | Reinforced |
Chapter Quiz
Answer each question, then select “Check answer” to reveal feedback. For Select All That Apply items, choose every correct option before checking. Expand “Why?” after checking to read the rationale.
A report states that 12 residents fell. What additional information is most necessary before comparing performance across facilities?
Why?
Raw counts are not comparable without a denominator, time interval, and consistent case definition that establish exposure and meaning.
Which analytic type primarily asks, “What happened?”
Why?
Descriptive analytics summarize what occurred; predictive analytics estimate what may occur, while prescriptive approaches address what action to take.
Which factors can make a benchmark comparison misleading?Select all that apply
Why?
Case mix, definitions, documentation completeness, and observation periods can distort comparisons. Appropriate risk adjustment is intended to improve—not undermine—comparability.
A staffing level is correlated with falls, but high-risk units also receive more staff. What concept should be considered?
Why?
High-risk units may receive more staff and also have more falls; the relationship between staffing and falls is therefore potentially confounded by underlying patient risk.
Which visualization is generally best for showing a metric’s trend across 24 months?
Why?
A line chart is generally appropriate for displaying change and pattern over an ordered time series such as 24 months.
Which method is most appropriate for prospectively identifying how a new medication-scanning workflow could fail?
Why?
FMEA prospectively identifies possible failure modes, causes, and effects before or during implementation; RCA is primarily retrospective after an event.
A clinically useful dashboard should help users understand:Select all that apply
Why?
A useful dashboard makes the signal, comparator, period, accountable actor, and path to investigation understandable rather than merely displaying numbers.
A model is accurate but flags 200 patients daily when staff can intervene on only 30. What is the primary implementation problem?
Why?
A model that generates far more actionable cases than available staff can manage has an operational-capacity mismatch even if its predictive performance is strong.
Why should a predictive model be monitored after deployment?
Why?
Data distributions, populations, workflows, and clinical practice change over time, so predeployment performance may not persist.
Which statement reflects appropriate causal language for an observational dashboard trend?
Why?
An observational trend can support an association or temporal coincidence, but causal claims require consideration of competing explanations and stronger design/evidence. —
References and Further Reading
- Agency for Healthcare Research and Quality. (2022). Strategies and Approaches for Investigating Patient Safety Events. AHRQ PSNet. https://psnet.ahrq.gov/primer/strategies-and-approaches-investigating-patient-safety-events
- Institute for Healthcare Improvement. Model for Improvement. https://www.ihi.org/library/model-for-improvement
- Agency for Healthcare Research and Quality. Health IT Evaluation Toolkit and Evaluation Measures Quick Reference Guides. https://digital.ahrq.gov/health-it-evaluation-toolkit
- Agency for Healthcare Research and Quality. (2022). Obtaining and Using Data in Practice: A Handbook for Health IT Advisors and Practice Facilitators. https://www.ahrq.gov/evidencenow/tools/healthit-advisor-handbook.html
- American Nurses Credentialing Center. (2025). Informatics Nursing Board Certification Examination: Test Content Outline. https://www.nursingworld.org/globalassets/informatics-tco_08292025-for-webposting.pdf