In twenty years of the Internet age, big companies have come to know more about their customers, their suppliers, and their own operations than they ever did, or could. Information pouring in from sensors and points of sale enables businesses to know how to sell things to shoppers before they know they want them, fix machines before they break, reorder stock before it runs out. “Big data,” which McKinsey trumpeted in March as “the next frontier for innovation, competition, and productivity,” is in fact the previous frontier with a new name.
“Big data” is the most recent manifestation of the knowledge-management revolution that began in earnest in 1991, when Thomas Stewart’s Fortune article “Brainpower” brought the concept of KM to the attention of American corporate elites. Large companies, Stewart explained, have built up immense intellectual capital in the form of knowhow and data about everything from the appetites of customers to the names of whom on the fourth floor to buy roses for when a report has to be typed up today. However, that capital, being informal, is seldom exploited or even recognized as such by most firms.
It was the promise of KM to mobilize this unrealized competitive advantage. Yet the daily business pages make plain that top management still often doesn’t know all it needs to know to thrive in the new knowledge economy. The reason is hinted at in the very first sentence of a new McKinsey report on data mining, which will set off new waves of productivity growth and innovation “as long as the right policies and enablers are in place.”
Which, in a surprising number of companies, they aren’t. Plenty of firms fail to learn from their mistakes, or even from their successes. New markets are opening faster than the minds that need to understand them. Computers themselves, touted as tools to make the complex comprehensible, just as often make it confusing. The subprime-mortgage empires collapsed into rubble because the trade in arcane investment instruments proliferated in number and complexity to the point that not even the people who invented them quite knew how they were working. As post-crash memoirs and insider books are making embarrassingly clear, the firms responsible for business catastrophes from oil spills to financial meltdowns had piles of information at their disposal but brought very little knowledge about risks and no wisdom at all to their decision-making.
The most important thing that companies have learned in the past twenty years, in fact, is that managing knowledge requires knowing more about both knowledge and management than a lot of big firms seem to know.
Knowing the Known Knowns
KM today sits at the table with the other grownups. Any large company would be ashamed to admit that it lacks its own versions of lessons-learned systems, online communities of practice, “experts locators,” best-practice bibles, and knowledge-sharing workshops. Such tools have allowed a few firms to tap the flow of information the way that their industrial predecessors tapped the power of running water outside the factory to run their machines. Most, unfortunately, manage company knowledge more expensively than intensively.
Managing knowledge is hard to do well because managing knowledge is hard to do at all. Knowledge is at once a process, an outcome, and a raw material. Managing knowledge thus cuts across all the familiar institutional boundaries, which is why some firms base their KM efforts in their IT departments, some in HR, some in “business strategy” departments, some in new departments set up for the purpose.
Knowledge can be combined, internalized, and externalized, as well as forgotten. It can be learned by doing, by watching, by listening, or by stealing. It comes in many forms—tacit, explicit, cultural, and (the form apparently hardest to apply in a corporate setting) Plain As The Nose On Your Face. Different kinds of staff—those who do things for the company, those who think about how to do things, and those who think about which things are worth doing—need different kinds of knowledge to do their jobs well. A consultant who insists that there is a simple system to do all of these things across a very large organization has either fooled himself or is trying to fool his clients.
Perhaps sensibly, most firms shrink from attempting to manage knowledge at all, instead contenting themselves with managing information, which is knowledge in its most rudimentary form. This kind of KM is basically a housekeeping operation aimed at liberating staff at all levels from the tyranny of the stuffed email inbox, the overflowing meetings calendar, the War and Peace-length memos. Useful, certainly, but employing machines to manage a problem that the machines created in the first place is never likely to be transformational.
The more helpful electronic tools such as video chat, instant messaging, and online conferencing facilitate communications inside companies and can also be a medium of collaboration across departments, even national borders. These are undeniably useful in the hands of a globally dispersed workforce, who can now do from their desks what was once done around the water cooler and in the lunchroom when all the firm’s thinking heads worked in the same building.
Company intranets help people share what they know, but to share what the company knows, you need some kind of central information exchange. A much-cited model is Xerox’s pioneering Eureka system, which allowed techs to field-test and share the best ways to fix fiddly copiers. More complex problems demand more complex information, usually in the form of a central document repository. There, embalmed in bytes, are buried the firm’s best-practice reports, project analyses, case studies, and other institutional wisdom, organized in ways that makes them accessible to anyone regardless of department or time zone. Process-reengineering consultant Gene Bellinger writes, “With on-demand access to managed knowledge, every situation is addressed with the sum total of everything anyone in the organization has ever learned about a situation of a similar nature.” The problem is that the sum total of everything anyone learned about anything is usually a muddle. If you doubt it, Google “management.”
Also, a repository is no better than the questions asked of it, and people tend to seek only information that they perceive is relevant to them, because their notions of relevance are limited by their lack of information—the so-called relevance paradox. This doesn’t matter much, however, if people don’t ask questions in the first place. Left to themselves, people prefer to exploit the unofficial KM systems that every company has—namely, the guy down the hall or the secretary in the next room. Call it an archive, call it a repository, but it’s basically a library, and in the United States, at least, a manager will never get far asking workers to do anything that reminds them of homework.
Trust? Reciprocity? Cooperation?
KM’s promise depends on knowledge sharing—people within a firm sharing with each other, division sharing with division. Doing this relies on social networks that are built on norms of trust, reciprocity, and cooperation, as University of Toronto information expert Chun Wei Choo has been pointing out for years. However, these networks exist only in rudimentary form in most large firms, in which divisions compete more intensely with other divisions than all do with other firms, and where “market share” is measured in the CEO’s attention.
The cost of pursuing intracompany competition at the expense of pan-company collaboration was made plain by the GM-Toyota NUMMI project. This joint venture to manufacture vehicles to be sold under both brands achieved the hoped-for transfer of knowledge, insofar as GM gradually came to know how to make good cars efficiently. However, it took years for GM to apply what it learned in the NUMMI plant to its other divisions, and by then the company was headed for bankruptcy. The problem wasn’t a lack of trust, reciprocity, and cooperation between the participating firms—rather, it was a lack of trust, reciprocity, and cooperation between NUMMI and GM’s Detroit overseers, and between GM and its unions.
For these and other reasons, then, a management movement that promised to improve work has merely changed the way it is done. Harvard Business School management professor Amy Edmondson was one of several researchers who tapped data from Indian software developer Wipro to determine whether an organization’s captured (and codified) knowledge actually helps project teams perform better. The researchers found that a team’s use of an organization’s captured knowledge did enhance productivity, especially among workers who were inexperienced, based in faraway offices, or undertaking complex tasks. However, the researchers found that while the captured knowledge helped teams work better (meaning more efficiently), it did not necessarily help teams do better work.
Knowing What Isn’t Yet Known
One thing all companies do know is that knowing things counts in a competitive environment in which access to ideas matters as much as access to markets and raw materials. KM is arguably most valuable when it marshals company resources to create new and commercially useful knowledge by applying what the workforce already knows to novel possibilities. Unhappily, that is KM’s most valuable use, not its most common one.
One frequent error is the application of what is already known to the wrong kinds of possibilities, as is explained by Bob Lutz, the executive widely credited with making GM cars worth buying again. In his new book, Car Guys vs. Bean Counters: The Battle for the Soul of American Business, Lutz complains that GM executives in charge of product planning were interested in creating beautiful, profitable spreadsheets instead of beautiful, profitable products. Data-driven cost optimization was what they knew, so data-driven cost optimization was what they did, whether or not their customers really wanted a cost-optimized car.
The New Yorker reported recently on the early days of the Xerox Palo Alto Research Center. PARC was probably the most fecund idea factory ever, apart from Steve Jobs’ shower. In “Creation Myth: Xerox PARC, Apple and the truth about innovation,” writer Malcolm Gladwell makes several important points. One is that creativity flourishes when it is least constrained by pre-defined corporate ends. PARC was hardly managed at all in its heyday; as was once said by Thomas Edison (whose lab in Menlo Park was just a couple of miles up the road from Palo Alto), “Hell, there are no rules here—we’re trying to accomplish something.”
Edison also believed, with the true heart of the laboratory explorer, that just because a new invention doesn’t do what you planned it to do doesn’t mean it’s useless—you just have to figure out what that use is. Unfortunately, companies believe, with the true heart of the bureaucrat, that just because an invention does something interesting doesn’t mean it’s useful to the company. As Gladwell also notes, PARC’s parent company failed to exploit the commercial potential of such gadgets as the computer mouse and the graphic user interface not because Xerox executives were stupid—they were smart enough to set up PARC, after all—but because they ran a copier company and not a computer company.
Which brings us to a third point worth remembering: Exploiting new knowledge that points toward products outside the company’s business model requires creativity and risk-taking from the executive suite as well as from the lab. Adam Greenfield spent two years as a top designer at Nokia, the paper-manufacturer-power-generator-galoshes-maker turned cell-phone giant. The firm’s culture was rooted in the manufacture of mass quantities of a product for the lowest achievable cost, and it was very good at it. What it has lately proved not so good at is adapting that approach to a shifting cell-phone market that demands imagination rather than efficiency from a manufacturer.
During Greenfield’s time at Nokia, new projects had to be signed off on by people up the ladder vested in old knowledge careers. He praises Nokia’s engineers as people brilliant at optimizing a supply chain but not “neurocognitively equipped” to be similarly savvy about machines designed to sell fun and excitement rather than utility. Concluded a frustrated Greenfield, “I wouldn’t look for innovation from large organizations.”
Nokia is far from alone in failing to learn the right lessons from success. Because GM’s market share is so large, leaders for decades assumed they must be a success at making good cars, when in fact what they were good at was selling mediocre ones. The awkward truth is that while failure may teach a company how to succeed, success often teaches a company to fail, by misleading it into thinking that it knows more than it does.
Higher Intelligence, or Keeping Executives in the Know
“If only we knew what we know, we would be three times more effective tomorrow.” The thought is widely, if somewhat inaccurately, credited to Lew Platt, president, CEO, and chairman of Hewlett-Packard in the 1990s. The profusion of “we’s” obscures a useful insight about KM that becomes clearer if it is rephrased thus: If only the executives knew what the workforce knows, the company would be three times more effective tomorrow.
But they don’t, so it isn’t. Knowledge faces more obstacles on its way up the hierarchy than a salmon faces on its way upstream to lay eggs. Much of what is known by the workforce is not shared with their superiors, or it isn’t measurable, or it is ignored by the people empowered to put it to use.
To get an informative answer, an executive must ask informed questions. Many a wise company head acquired plenty of practical knowledge about the firm’s operations when they worked their way up the ladder. Most of today’s peripatetic senior executives are trained either as generalists or specialists in arts such as finance that teach little about life in the labs, the warehouses, or the personnel department. Their underlings, of course, know every clogged pipe in the company machine and have ideas about how to unclog them. But letting the boss know how to do things more efficiently risks giving her ideas about how she can do without you.
Eileen Shapiro, in her very good 1996 book Fad Surfing in the Boardroom, distilled the phenomenon of knowledge-hoarding into a maxim: What is observed or conceived is seldom disclosed. Knowledge that might be valuable to the company also is valuable inside the company. It is the currency by which workplace reputations, position, and perks are purchased, and the savvy knowledge miser often won’t share what she knows until it ceases to be valuable to her, which means until she is about to retire. (This protection of knowledge has an ancient institutional equivalent in the craftsman’s guild, organized less to perpetuate than to protect knowledge about how to do things to preserve its scarcity value on the skills market.) The only way to deal with the hoarders is to give them something else just as valuable, and smart firms offer incentives for knowledge-sharing or offer promotion and prestige in the form of appointment as a company mentor, teacher, or some other form of knowledge steward.
Just as often, the company’s own policies keep useful knowledge hidden from senior managers. Knowledge-sharing must be part of job design and performance-appraisal criteria if employees are to be expected to loosen their grip on the company’s real operating manual. Rewards systems also need to be aligned so that doing the smart thing for the company is not a dumb thing for the employees—all the employees. ”Analyzing organizational failures requires inquiry and openness, patience, and a tolerance for causal ambiguity,” writes Amy Edmondson in Harvard Business Review. “Yet managers typically admire and are rewarded for decisiveness, efficiency, and action—not thoughtful reflection.“
Then there is the sheer complexity of the modern globalized business corporation. The more layers through which information must pass, the more ways there are for it to become distorted or blocked. The result is what students of organizations have dubbed hierarchical incompetence, which can be summed up this way: The more the people at the top need to know, the less likely they are to know it. The challenge for many executives ends up being less how to manage knowledge than to manage without it.
An executive can never have intimate knowledge of a large organization; it must be inferred. Some such performance measures are so outdated that they still treat employees and their costs as expenses rather than assets, but alternatives are little more useful. In 2007, Deloitte Touche Tohmatsu and the Economist Intelligence Unit polled senior executives and board members around the world, who complained that nonfinancial performance measurements, while desirable, remain clumsy and unreliable.
If the feedback from the company to the executive suite is static-y, so is the feedback from inside the heads of the people inside it. Bosses are no more eager to hear bad news than anyone else; that makes their associates less eager to give it to them, even when the bad news is what they most need to hear. And like most people in authority, senior executives tend to discredit good information from rivals and critics and dismiss it from underlings on the wholly human and wholly foolish grounds that if their underlings had things to say worth listening to, they wouldn’t be underlings.
Even when senior managers have access to the kinds of information that would enhance their knowledge of company operations, they don’t always use it. In Analytics at Work: Smarter Decisions, Better Results,, Thomas Davenport, Jeanne Harris, and Robert Morison suggest ways to unleash the potential of what they call “rational, clear-headed analysis of dependable data.” That potential is real enough. In a recent study, MIT colleagues Erik Brynjolfsson and Andrew McAfee sought to learn what makes a company “good at IT.” They combined data from surveys, annual reports, and a composite measure of IT investments at 330 major U.S. companies and found that the more likely a firm was to rely on data-driven decision-making, the higher a firm’s productivity and profitability.
Alas, the authors add, ”our data show clearly that best practices are far from universal, even once they’re universally recognized as being ‘best.’” How is it possible that, a century after the heyday of “scientific” management and a half-century after the arrival of computers able to transform data into knowledge, decisions based on rational, clear-headed analysis are still so rare? McAfee and Brynjolfsson conclude that some bosses simply don’t believe in rational, clear-headed analysis. They rely on intuition and expertise rather than data—what they feel rather than what they know.
This apparent conviction that knowledge is useful for everyone in the company but the boss can be dismissed as hubris, and will be by a lot of executives who would rather plead guilty to arrogance than to stand trial for ignorance. Top executives as a class tend to be drawn toward the technical (how to do things) rather than the scientific (how to understand things). Is it possible that the persistent reliance on hunches and experience may be a necessary accommodation to the fact that they don’t actually know what they need to know to run a large business organization?
The Wise Guy in Every Crowd
Amy Edmondson argues that the CEO doesn’t know everything she needs to know about what the company needs to know, because no one in that position can. “The knowledge that each us has is narrower, if deeper, than ever,” she says. “So it’s OK to be ignorant, to need outside expertise.” That’s music to the ears of consultants everywhere, but is this outside expertise to be found outside the company? Or is it just outside the doors to the C-suite?
At the heart of KM has always lurked a subversive notion: If knowledge is a company’s most important asset, and if the people who work for it collectively possess a deeper knowledge of how the company works, then the people employed by it should be better placed to run it than the executives. Harnessing collective wisdom only needs some means to manage collectively. Doing so in conventional ways—polling, or subjecting decisions to what amounts to a plebiscite—are vulnerable to the same flaws of group decision-making that plague our national politics.
A new generation of collaboration tools in the form of so-called Enterprise 2.0 software may offer alternatives. Knowledge exchange and workflow tools used in conjunction with social-networking platforms (as blogs and wikis) create links between people within companies, or between companies and their partners or customers. By allowing employees to collaborate online, acting on information that is more timely and more relevant to company operations than that available to their employers, Enterprise 2.0 tools have the potential to (quoting Andrew McAfee) “render obsolete traditional notions of management and hierarchy.”
Might today’s new tools do more, and render obsolete traditional management and hierarchy themselves? Might they liberate a workforce to form networks of trust without a central system controlling their behavior? Might they transform a workforce into a companywide community of practice capable of taking up company management, with decisions about organizational structure, resource allocation and the rest made collectively and to some extent unconsciously?
Nature offers models of a sort in self-sustaining, self-regulating communities such as ant colonies or plant ecosystems that are governed by the collective “intelligence” of the various parts. In such a future, the company workforce would no longer be merely tools of the company. The company would become, in a way, the tool of the people who work for it. That wouldn’t be merely be reengineering. That would be revolution.
Not that senior managers need start piling up their briefcases and BlackBerries into barricades just yet. The tools at hand when KM was born twenty years ago also were thought to have the potential to revolutionize business practice. One is obliged to be skeptical of wonders that “liberate the workforce from the constraints of legacy communication and productivity tools like email.” It was email, remember, that was supposed to liberate the workforce from the telephone and the paper memo. Instead, it so enchained them that coping was one of the first tasks set for knowledge management. No wonder thinkers about IT are split, with one camp insisting that Enterprise 2.0 is truly a transformative technology and the other dismissing it as just an incremental evolution of existing tools.
Of course, it is not tools alone that make revolutions—it is people. The generation of employees now populating the workforce grew up with—indeed, helped invent—social networking and other media-based forms of information exchange. They live in the kind of open, self-directed virtual communities that exist otherwise only in the dreams of KM consultants and few visionary CEOs. For them, KM is not a project but a lifestyle. Thinking like a group, which to their parents means not really thinking at all, comes naturally to them.
It is a commonplace by now that this generation will require different management techniques to harness their knowledge to company ends. That, perhaps, underestimates the dimensions of the shift. As this new generation rises into senior management positions, we might find out after all that managing knowledge and managing companies are one and the same thing. 
The high failure rate among start-ups and small businesses is well-known. Chicago entrepreneur and columnist Jay Goltz (“a business speaker who actually runs a business“) is one of many to point out that most failed entrepreneurs don’t know why they failed, and therefore didn’t know what to fix.
The executives of large businesses are no different in failing to master the lessons their own operations might teach them. Their firms are not exactly too big to fail, but they are too big to fail quickly. Instead they fail in myriad ways that are small and almost invisible. Their sales meet targets but do not exploit their market as fully as they might have; their new hires are good but not the best.
By learning how and why such things are done in less-than-best ways, a company can learn how to do them better. The classic example is—or, rather, was—Toyota, whose vaunted continuous improvement was based on the belief that every small inefficiency on the production line was a failure to be learned from.
The goal, in short, is to learn from small failures so one can avoid the catastrophic ones. Most firms that try to do so use after-action reviews and other forms of postmortem meant to discover the black box in the wreckage. For two decades, Amy Edmondson has been studying the results of such investigations at enterprises as diverse as Big Pharma companies and NASA’s space-shuttle program. The results convince her that most organizations are probably failing to learn everything they might from failure.
Humans are “hard-wired to want others to think well of us,” she notes, and others will generally fail to think well of us when we are confessing a screwup. Nor is it easy to think well of oneself when confronting a failure. “Examining our failures in depth is emotionally unpleasant and can chip away at our self-esteem.”
And at many companies, where to assign blame is the first and sometimes the only question leaders want answered. Failure postmortems can easily turn into witch hunts. This is not helpful. Employees who are fearful of the consequences of failing will take every step to ensure they don’t. Easier to fudge the numbers, or to find someone else to blame.
Edmondson is among those who counsel that before anyone, or any firm, can learn from failure, admitting a failure must be separated from taking the blame for it. However, acknowledging failure, even forgiving it in others, is alien to the business mind. The culture conditions its leaders to believe that the best and maybe the only reliable motivator is fear of failure and its consequences. Making it OK for people to fail is therefore inconsistent with high performance standards.
The larger the project, the more is at risk, and the more the pressure is on decision-makers to get it right the first time. But getting it right the first time, while always to be hoped for, is unrealistic. The larger the project, the less likely it is that it will go right the first time. As Edmondson has put it, considering the inevitable small process failures to be bad is “misunderstanding how complex systems work.”
A big failure, such as a product rebranding or a company reorganization, is usually not one failure but a lot of little failures that occur in different departments or at different levels of the organization. That makes tracing effects back to causes difficult. Perhaps more damaging to morale, it makes pinning the blame on some hapless intern or department look dishonest.
Falling off a bike and skinning a knee will leave you wanting to do it right next time, and may suggest some reasons why you fell, but only by getting back on the bike will you learn how to do it right. Economist and Financial Times columnist Tim Harford is one of a number of management thinkers who have argued that the successful organization, rather than correct for failure, will seek it out through old-fashioned trial and error.
Every firm learns how to do things right through trial and error. Most do it haphazardly; a few plan and provide for it. Pilot projects, for instance, are usually designed to produce results that are flattering to the inventors. To do what such trial runs are supposed to—teach the firm whether a new product, process, or market works and how well—pilot projects should not exactly be designed to fail, but they should at least be designed to discover everything that could go wrong along with what might go right.
Harford concedes that making failure survivable is hard to do in systems that are tightly integrated. He likens the U.S. financial system to nuclear power plants in the sense that a modest failure in one part cascades through the system with catastrophic results. He has pointed to repackaged mortgages in the form of residential mortgage-backed securities and collateralized debt obligations as undertakings that, if they failed, would fail big. These financial safety systems, he wrote recently, merely offered “exciting new ways to blow things up.” —J.K.Jr.