The Real Reasons Why Digital Projects Fail
MIT Sloan Management Review ·
Danae Diaz/Ikon Images It’s a familiar scenario: An organization implements digital tools intended to improve a particular process but fails to obtain the desired result. Employees who are meant to use them complain that the tools don’t solve the problem or that they add inefficiencies. Managers who investigate problems with the tools chalk the failure […]
Danae Diaz/Ikon Images
It’s a familiar scenario: An organization implements digital tools intended to improve a particular process but fails to obtain the desired result. Employees who are meant to use them complain that the tools don’t solve the problem or that they add inefficiencies. Managers who investigate problems with the tools chalk the failure up to poor adoption but don’t look more closely at what caused the outcome.
When those leaders do seek to analyze failed digital investments, they make a common error: They evaluate data quality and system usability as a single variable. However, those factors operate through entirely different mechanisms, and they fail in different ways; improving one does not improve the other.
The cost of that error is specific and recurring. When a digital investment underdelivers, organizations almost universally attribute the failure to implementation problems, insufficient training, or change management breakdowns. These explanations are structurally inevitable when leaders don’t distinguish between a data quality failure and a system design failure. But without that distinction, the postmortem cannot identify the real cause. The result is a corrective investment that addresses the wrong problem, followed eventually by another underperforming system, followed by another misdiagnosis.
Empirical research that I and others conducted across one of the most cognitively demanding decision environments available for study found that data usability and system usability affect cognitive load through entirely different pathways. 1 They aren’t two dimensions of the same problem. They are two distinct problems that require distinct diagnosis and distinct investment. The findings are directly actionable for any organization managing knowledge workers whose performance depends on digital systems.
Along with Kalle Lyytinen and David Aron at Case Western Reserve University and Michael R. Cauley at Vanderbilt University Medical Center, we surveyed 564 practicing physicians across 32 medical specialties to examine how data quality, system design, and information overload jointly shape doctors’ cognitive load when using electronic health records (EHRs) in high-stakes decision-making. We chose clinical medicine because it’s the highest-stakes, most extensively documented context for knowledge work under cognitive pressure and time constraints. The cognitive mechanisms that the research reveals are not unique to medicine, though.
EHR data usability, which encompasses the quality, completeness, and clinical relevance of patient information, increases germane cognitive load. Germane load is the productive cognitive effort associated with deep reasoning and meaningful engagement with complex information. When knowledge workers encounter high-quality, well-organized, contextually relevant data, they engage more deeply with it. That deeper engagement is the mechanism through which good judgments are made. It is not a symptom of overload or fatigue. It is the condition that produces decision quality. Better data makes physicians think harder about what matters. That’s the investment paying off.
We hypothesized that EHR systems that are highly usable — that is, those whose interface design, navigation structure, and workflow alignment support user interaction — reduce extraneous cognitive load. Extraneous load is unproductive cognitive effort generated by poor design: excessive navigation steps, misaligned workflows, alert fatigue, visual clutter, and documentation requirements that consume mental capacity without contributing to the decision at hand. Better system design eliminates that waste and redirects cognitive capacity toward the reasoning that produces accurate judgments. The problem it solves isn’t worker well-being. It addresses decision quality degradation caused by avoidable structural friction.
The research confirmed both effects with statistical precision. Data usability demonstrated a strong direct positive effect on cognitive load, with a standardized path coefficient of 0.597. System usability partially — and negatively — mediated that relationship, with an indirect effect of negative 0.571. Information overload mediated both pathways.
Consider what happens when an organization invests heavily in data quality without a proportional investment in system design. Data usability improves. Germane cognitive engagement increases. Knowledge workers are able to reason more deeply about better information. But if the system requires more navigation, more clicks, more workflow friction to access that information, extraneous cognitive load increases simultaneously. The total cognitive burden on the user goes up even as the quality of the underlying data improves. Decision throughput decreases. Users report fatigue and frustration. Leadership concludes that the digital transformation underperformed, without understanding the structural design problem: The investment improved data quality but didn’t include commensurate investment in reducing the friction of accessing and acting on that data.
The reverse failure is equally common. Organizations focus on system usability improvements, simplifying interfaces, reducing clicks, and redesigning dashboards without addressing underlying data quality. Extraneous load decreases. Navigation is easier. But if the data is incomplete, inconsistent, or poorly organized, the germane cognitive engagement that produces good decisions is not triggered. The system is easier to use. The decisions aren’t better.
Both failure modes are predictable once the distinction between them is visible, but neither can be seen when leaders evaluate digital system performance as a single variable.
Consider a financial services firm that invests heavily in a new data platform, consolidating market intelligence, portfolio analytics, and risk signals into a single source of truth. Data quality improves measurably. But the interface through which analysts access that data, built by a different vendor on a different timeline, requires eight navigation steps to surface a complete company profile and generates alerts at a threshold calibrated for compliance, not decision-making. Analyst productivity stalls. Senior talent starts leaving. The postmortem team concludes that the platform was poorly adopted and launches a training program.
The data investment was sound. The system design was not. The postmortem examined neither independently, so it fixed nothing. Had the two levers been assessed separately from the outset, the failure would have been locatable, correctable, and cheap to fix relative to what the misdiagnosis cost.
The finding that higher data usability reduced perceived information overload was not about limiting data volume. It was about improving signal quality: making sure the information in front of a decision maker was relevant, reliable, and organized. Acting on this insight requires attending to the following four governance practices.
Eliminate redundancy. When the same data appears in multiple places within the same system, users stop reasoning from content and start verifying consistency. That verification work produces no decision value. Eliminating redundancy removes it and increases trust in what remains.
Show your data’s lineage. When a decision maker cannot quickly assess whether a data point is current, how it was collected, or how much confidence it warrants, they spend mental capacity on source verification rather than the decision itself. Embedding simple indicators of reliability — when a measurement was taken, by what process, and with what confidence level — eliminates that overhead and lets judgment begin sooner.
Mandate the fields that matter most. Missing information at a critical decision point doesn’t just slow things down; it breaks the workflow entirely and forces improvisation that shouldn’t be necessary. Structuring data capture requirements around the fields most critical to core decisions ensures that gaps appear by exception, not by default.
Mind your alert thresholds. Alert fatigue — the learned tendency to dismiss notifications because most don’t require action — isn’t a user behavior problem. It’s a governance failure. Every alert threshold is a decision about where to direct a decision maker’s attention. When those thresholds are set too low, attention is trained away from the system entirely. Resetting them so alerts fire only when action is genuinely required is among the highest-return cognitive design improvements available, and it requires no new technology.
Organizations investing heavily in AI-enabled decision support systems across industries are generating the same postmortem pattern at scale. Leaders evaluate AI performance primarily through model accuracy, adoption rates, and efficiency gains. Those metrics are necessary. They are not sufficient.
AI systems that improve the quality and relevance of information surfaced to decision makers are engaging the germane load lever. They are making the data better. That is valuable. But if the interface through which users interact with AI outputs is poorly designed, if recommendations arrive without sufficient context, if the system requires significant navigation to understand the basis for a recommendation, or if it generates notifications users have learned to dismiss, the extraneous load the interface imposes will erode or eliminate the germane load benefit the AI delivers.
Making an AI investment that improves recommendation quality without making an equivalent investment in the interface design required to act on those recommendations will produce the same misdiagnosis as every prior generation of digital underperformance: The tools will be blamed, the training questioned, the workforce examined, and the structural design problem left in place.
Leaders should look for three qualities in any AI system in their portfolio. First, the data it surfaces must be worth the depth of reasoning it demands from the people using it. Second, the interface design should reduce the structural friction of accessing and acting on that data. And third, the information that is surfaced, when it is surfaced, and any accompanying signals as to its urgency, should be designed to direct judgment toward what matters, not scatter attention.
The questions that belong in leadership reviews of digital system performance extend beyond standard metrics of uptime, accuracy, and user adoption. They are questions about cognitive design.
Does the system reduce the effort required to find relevant information, or does it increase it? Are the alerts the system generates ones our people act on or ones they have learned to dismiss? Is the cognitive engagement this system demands producing better decisions, or is it creating structural friction that the design should be eliminating? What signals are being surfaced at each decision moment, and are they the right ones for that moment? Are there signs of decision quality degradation — rising error rates on complex decisions, increased escalation volumes, or shortened tenure among experienced decision makers — that point to a poorly designed cognitive environment rather than a talent or motivation problem?
Alert configurations, interface logic, workflow design, and data governance thresholds are cognitive design decisions with direct performance consequences; they are not IT configuration choices. Leaders who want to improve the returns on their digital investments must gain visibility into these decisions, not cede them entirely to implementation teams, vendors, and IT administrators.