Category: Learning Strategy

Posted on

When Institutional Knowledge Walks Out the Door

Organizations lose institutional knowledge every day. Loss of critical knowledge could be caused by retirement, turnover, or even worse, restructuring. The reason doesn’t change the result: critical knowledge disappears, and the people left behind scramble to recreate what was already known.

I’ve witnessed this happen recently in federal agencies and universities, as well as in the corporate world during my time in finance, back in the early 2000s. The universal scenario works like this: A subject matter expert leaves. Their replacement arrives weeks or months later to discover that the documentation is outdated, the processes aren’t written down, and/or the informal networks that made things work are no longer in place. Anyone who has been in the workforce for a length of time has seen it.

The Knowledge Management Gap

Most organizations treat knowledge management as a documentation problem. Create better SOPs. Maintain more detailed process guides. Build comprehensive wikis.

But documentation alone doesn’t capture institutional knowledge. It doesn’t capture the why behind decisions or explain which workarounds exist because of legitimate constraints versus which ones exist because “that’s how we’ve always done it.” It doesn’t preserve the relationships between systems, stakeholders, and strategic priorities.

Real knowledge management requires understanding that knowledge exists in three forms:

Explicit knowledge lives in documents, databases, and formal training materials. This is what most KM systems capture.

Tacit knowledge resides in people’s minds; it is the experience-based insights that experts utilize without conscious thought. This is harder to transfer but equally critical.

Embedded knowledge lives in processes, culture, and organizational relationships. This is nearly impossible to document, but devastating to lose.

What Actually Works

Organizations that maintain knowledge through transition do three things differently:

They build redundancy into expertise. Critical knowledge doesn’t live in one person’s head. Multiple people understand key systems, maintain important relationships, and can execute essential processes. This isn’t inefficient, it’s resilient. As a former university colleague once taught me, there should never be a single point of failure.

They create learning pathways, not just documentation. New team members don’t just read manuals. Instead, they work alongside experienced colleagues, participate in communities of practice, and have structured opportunities to ask questions and make sense of what they’re learning. (Something my current organization does well.)

They treat knowledge transfer as continuous work, not crisis response. Knowledge management isn’t something you do when someone gives notice. It’s built into regular operations—shadowing rotations, cross-training, documented decision-making processes, and accessible repositories that people actually use. It also doesn’t hurt to have a team focused on knowledge management. (Shout out to the KM team!)

The Real Cost of Lost Knowledge

When institutional knowledge leaves the organization, it’s not just efficiency that is lost; it’s also the expertise and experience that are lost. Organizations lose the ability to learn from past mistakes. They repeat failures that were already solved. Decisions are made without understanding the historical context.

In learning and development specifically, lost institutional knowledge means:

  • Training programs that don’t connect to organizational strategy because the people who understood both are gone
  • Assessment practices that measure the wrong things because no one remembers why current metrics were chosen
  • Technology implementations that ignore lessons from previous failures
  • Competency frameworks that drift from business needs because the connection isn’t documented

The cost shows up months later, in small inefficiencies and repeated mistakes that compound over time.

Building Resilient Knowledge Systems

Effective knowledge management isn’t about preventing people from leaving. It’s about building systems that can absorb transitions without losing critical capabilities.

That means:

  • Documenting decisions, not just processes. Capture why choices were made, what alternatives were considered, and what constraints influenced the outcome.
  • Creating apprenticeship opportunities. Pair experienced staff with newer team members on real projects, not just training exercises.
  • Building accessible knowledge repositories. Systems people actually use, organized around how work gets done, not how information is categorized.
  • Maintaining communities of practice. Regular opportunities for people to share insights, solve problems together, and build collective understanding.
  • Making knowledge sharing part of performance expectations. Reward people for developing others, not just for individual expertise.

What This Means for Learning Leaders

For those of us in learning and development, knowledge management is central, not peripheral, to our work. We’re responsible for building organizational capability. That includes the capability to preserve and transfer what we know.

During times of transition, that responsibility becomes more visible. The question isn’t whether knowledge will be lost. The question is whether we’ve built systems that can minimize that loss and accelerate recovery.

The organizations that navigate transitions successfully aren’t the ones that prevent change. They’re the ones who treat knowledge management as an essential and continuous infrastructure, worth the investment even when things are stable.

What knowledge management practices have you seen work (or fail spectacularly) during organizational transitions? The approaches that work in stable times may not work under pressure.

Posted on

Should This Course Be Virtual? Stop Converting Courses Blindly

In my current role, we are considering the necessity of virtual learning and how to implement it effectively. Is the question we should be asking: “Can this be delivered virtually?” OR “Should this be delivered virtually?” The difference boils down to virtual learning strategy. The first question assumes that technology determines training decisions. The second recognizes that strategic learning design should drive technology choices.

How many times have you seen conversion projects fail, or worse, just done poorly because teams skipped the strategic assessment phase? Does your organization focus on capabilities over learning effectiveness? If you measure technical feasibility instead of performance impact, you may end up with courses that technically work but don’t actually improve job performance.

Strategic Assessment Changes Everything

An effective virtual learning strategy starts with four questions that have nothing to do with technology:

  1. Does this content address an actual performance gap? The real problem could be workflow design, resource availability, or even policy. Virtual training (or any kind of training) won’t fix a problem that is not caused by a knowledge or skills gap.
  2. Will virtual delivery enhance or limit learning transfer? Context matters. Consider whether your content works better with self-paced reflection, live virtual interaction, or immediate hands-on application. Different formats serve different learning transfer needs.
  3. What’s the true cost comparison? Beyond platform fees and development time, consider instructor preparation, learner technology support, and the hidden costs of reduced engagement or learning effectiveness. (To name a few)
  4. How does this align with the organizational learning strategy? Random course conversions create inconsistent learner experiences. Strategic conversion builds systematic virtual learning capabilities.

Virtual learning effectiveness research:

https://trainingindustry.com/wiki/remote-learning/virtual-instructor-led-training-vilt

Weighing the Pros and Cons: Actual Training Versus Virtual Training – Brandon Hall Group

Design Decisions Drive Technology Needs

Once you know what learning outcomes you’re trying to achieve, technology decisions become much clearer. Need high-stakes skill practice? Look for platforms with robust simulation capabilities. Focusing on knowledge transfer? Simple video conferencing might be perfect. Using the technology-first (then adapting content to fit) approach could result in unnecessary constraints that could compromise learning effectiveness. The ultimate square peg-round hole problem.

Resource Allocation as Strategic Capability

A successful virtual learning initiative would treat conversion as an organizational capability, rather than focusing solely on individual course development. It should invest in instructor preparation, virtual content design standards, and quality assurance protocols that improve every subsequent virtual course. This perspective changes resource allocation decisions. Instead of evaluating each course conversion separately, organizations can build systematic virtual learning excellence that creates competitive advantages in talent development and operational efficiency.

Implementation Lessons from Government Training

Government training environments reveal what works under real constraints. Limited budgets, security requirements, global time zones, and mandatory compliance create conditions where strategic thinking isn’t optional. Organizations succeed by focusing on design principles before technology. They build instructor competency alongside content conversion. And they measure performance impact, not just engagement and satisfaction metrics. But most importantly, they recognize that virtual learning excellence requires different approaches than traditional training, not just digital versions of existing training.

Questions that Lead to Better Decisions

Instead of asking whether content can be delivered virtually, try these strategic questions:

  • What performance outcomes are we trying to achieve, and how will we measure success?
  • How does this conversion support broader organizational learning goals?
  • What instructor development and support systems need to be in place?
  • How will we ensure learning transfer to actual job performance?
  • What’s our plan for continuous improvement based on performance data?

These questions lead to conversion decisions that create lasting organizational value rather than short-term technology adoption. Because the goal isn’t to convert everything to virtual delivery. It’s to make strategic decisions about where virtual learning adds genuine value while building systematic capabilities that improve training effectiveness across the organization.

Posted on

The Feedback Loop Problem in Learning Design

Have you ever wondered if learning feedback systems are measuring the wrong things at the wrong time through the wrong channels? We ask about satisfaction after sessions end, assess knowledge weeks after content delivery (for instructor-led training), and evaluate application performance months later, when it’s more challenging to make improvements.

Meanwhile, learners send us immediate, actionable feedback signals constantly. The problem isn’t that we lack feedback from our learners, it’s that feedback loops are structured mroe like post-mortems than navigation systems.

What Direct Feedback Actually Looks Like

At six weeks into recovering from foot surgery, I’m learning what direct feedback systems look like. Physical limitations create immediate, unfiltered information about what’s working and what isn’t. Pain signals problems instantly. Energy levels indicate capacity in real-time. Range of motion shows progress directly. This feedback isn’t delayed or filtered through satisfaction surveys. It’s connected immediately to decisions that matter, and it informs the next action rather than evaluating the last one.

Learning design rarely operates this way. In my post about completion rates, I explored what we should measure beyond compliance metrics. But recovery is revealing a deeper issue: It’s not just what we measure, but how disconnected our feedback loops are from the learning process itself. (For an alternative explanation of feedback loops take a look at this post by Terry Heick.)

The Communication Delay Problem

Research on virtual instructor-led training effectiveness shows that systematic design approaches outperform technology features. But systematic design depends on feedback loops that can actually inform design decisions. Yet most learning feedback comes too late to improve the experience. We measure engagement after sessions end, assess comprehension days or weeks after content delivery, and track application months later. By the time we get usable information, the learning context has passed.

The rub – learners provide immediate feedback constantly. The hesitation before answering questions. How they rephrase concepts back to us. Whether they return to specific resources or bookmark particular sections. These are real-time signals about what is or isn’t working. The question here is: Have we designed to capture and respond?

Beyond Satisfaction: Performance-Connected Feedback

The indicators I identified in earlier posts (voluntary re-engagement, sophisticated questioning, cross-contextual application) are all direct performance signals. So how do we build systems that respond to them during learning? The feedback my foot is giving to me is immediate and connected to outcomes that matter. Learning feedback should also be connected this directly to learning outcomes. Let’s measure retention instead of reaction, skill transfer instead of satisfaction.

The Social Acceptability Filter

Sounds simple, so what’s stopping us? One barrier to direct learning feedback is social acceptability. Physical feedback doesn’t care about politeness. Pain tells you to stop regardless of social considerations. (Even though I was embarrassed to be stuck on the ground in public a half mile from my office when I broke my foot, I wasn’t going to stand up.) But in learning environments, social filters can make honest feedback feel risky.

People say they “got it” when they’re confused. They may rate sessions positively to avoid seeming difficult or nod along when they’re lost because questioning feels like admitting inadequacy. (I mean, we’ve all been there.) But what if we designed learning environments where direct feedback felt as natural as physical feedback? Where confusion was treated as valuable naviagation data rather than failure.

Building Learning Feedback Systems: Navigation Systems Instead of Evaluation Systems

Consider a feedback system that functions like navigation tools. Tools that help us adjust course while traveling, not evaluate the trip after you’ve arrived. (And let’s please not mention building the airplane while flying. Maybe I’ll share my thoughts on that another time.) Instead of post-session surveys, we could build in real-time comprehension checks that actually inform content delivery. Instead of delayed knowledge assessments, we could track application attempts that reveal gaps while there’s still time to address them. This isn’t about more technology or complex analytics. It’s about designing feedback loops that connect directly to learning decisions and respond quickly enough to improve the experience while it’s happening.

The goal isn’t perfect measurement. Don’t get me wrong. I still value the Kirkpatrick levels and they have their place. But we need to build feedback systems that actually improve learning while there’s still time to make adjustments. What direct learning signals have you noticed that current systems typically ignore? When you’ve experienced learning environments with good feedback loops, what made them different? I’d love to hear about the gaps you’ve observed and what signals you trust instead. Share your thoughts at the Coffee Corner or on LinkedIn.

Posted on

The Competency Mapping Revolution: Beyond Skills Lists

Many competency frameworks are simply skill inventories. We list what people should know, add some behavioral descriptors, and possibly add proficiency levels. Then we wonder why performance remains inconsistent across individuals who supposedly possess the same competencies. The problem isn’t that we’re measuring the wrong things; it’s that we’re thinking about competency development in the wrong way.

What We Get Wrong About Competencies

Traditional competency mapping treats expertise like a collection of discrete components. “Strategic thinking” may be defined as planning, analysis, and decision-making. “Communication” becomes writing, presenting, and listening. We create these intricate taxonomies that appear impressive in frameworks but often fail when people attempt to apply them.

Real competency isn’t additive. You don’t get strategic thinking by combining planning skills with analytical thinking. You get it by repeatedly navigating complex situations where multiple valid options exist, stakeholders have competing interests, and incomplete information forces judgment calls.

Confusion About Competencies

Part of the problem is that “competency mapping” is used to describe different things by different people. Some people use it to mean curriculum mapping, which involves aligning courses with job requirements or KSAs. Others think it means creating competency frameworks with lists and proficiency levels. These aren’t the same thing as mapping how expertise actually develops and transfers to performance.

When we confuse the tool (frameworks) or the process (curriculum alignment) with the real work (understanding how people develop expertise in context), we end up with solutions that look comprehensive but don’t actually improve performance.

Context Changes Everything

Here’s what’s missing from most competency conversations: context isn’t background noise, it’s the whole point. Strategic thinking in federal environments means navigating stakeholder politics, compliance requirements, and mission constraints simultaneously. That’s not the same skill as strategic thinking in a startup, where you can pivot quickly and break things.

The competencies that matter aren’t just what people can do – they’re what people can do within the specific constraints, pressures, and expectations of their actual work environment.

What Competency Mapping Actually Reveals

When you map competencies the way work actually happens, patterns emerge that skills lists miss entirely. You begin to see how expertise develops through repeated exposure to authentic challenges, rather than through training modules that teach components in isolation.

You also discover that the most valuable competencies are often the ones we never thought to name. The ability to recognize when standard procedures won’t work. The judgment to know which stakeholder concerns are worth addressing and which are distractions. The skill of translating complex requirements into actionable guidance for people who don’t live in your technical world.

The Questions That Matter

Instead of asking “What should people know how to do?” try asking:

What are the situations where good performers consistently succeed and others struggle? What makes those situations challenging? What do successful people notice or prioritize that others miss?

When people fail at tasks they theoretically have the skills for, what’s actually happening? What contextual factors are they not accounting for?

What does expertise look like when things don’t go according to plan?

These questions reveal competencies that cannot be captured in skills inventories. They point toward the kind of learning experiences that actually prepare people for the complexity of real work.

Moving Beyond Lists

The competency mapping revolution isn’t about better taxonomies or more sophisticated measurement tools. It’s about recognizing that competency development happens through authentic practice in realistic contexts, not through decomposed skills training.

This doesn’t mean throwing out frameworks entirely. It means using them as starting points for deeper conversations about what expertise actually looks like in practice, rather than as endpoints that define what people need to know.

The question isn’t whether learners can demonstrate strategic thinking on an assessment. It’s whether they can navigate the messy, ambiguous, politically complex situations where strategic thinking actually matters.

And that’s a very different kind of mapping problem.

Resources for Implementation

If you’re dealing with competency confusion in your organization, I’ve created three templates that distinguish between competency frameworks, curriculum mapping, and competency mapping approaches. Each addresses different organizational needs:

They’re designed to help teams understand which tool fits their actual needs and when to use each approach. The goal isn’t more sophisticated measurement – it’s clearer thinking about what we’re actually trying to develop.

Posted on

Beyond Completion Rates: What Tells Us Learning Happened

We celebrate high completion rates in our training programs. They feel like success, clean percentages that stakeholders understand and LMS systems track easily. However, do completion rates reveal anything meaningful about learning?

People click through entire modules without absorbing the content, just checking compliance boxes. Think about the last time you completed that annual mandatory training. Then ask your colleagues about the last time they finished theirs. In other instances, learners dive deep into the content, apply it immediately, but never officially “complete” the course because they got what they needed and moved on to solve real problems. Unless, of course, completion is required.

The gap between completion and learning remains. Are we measuring the right things?

Why We Love Completion Rates

Completion rates are seductive because they’re measurable. Your LMS generates clean reports with metrics that allow leadership to track progress, and everyone feels confident about their training investments. They solve the “how do we know people did the training” question. But completion rates measure compliance, not capability. They tell us someone reached the end of content, not whether they can do anything different because of it.

The actual evidence of learning is evident in job performance, as evidenced by someone making better decisions, solving problems they couldn’t handle before, or improving their work as a result of training. However, measuring performance change is complex, time-consuming, and often depends on factors beyond the training itself. So, we default to completion rates because they’re available, maybe not because they’re meaningful.

Signs That Learning Happened

If completion rates don’t tell us about learning, what does? Learning science points to several indicators that someone has genuinely absorbed and internalized new knowledge or skills:

Voluntary re-engagement with content. When people return to specific resources or bookmark particular sections, they’ve identified something valuable enough to revisit. This self-directed behavior suggests the content addressed real needs.

Evolution in questioning. Early in learning, people ask procedural questions: “How do I do this?” As understanding develops, questions become more sophisticated: “What if I did this differently?” or “How does this connect to that other concept?” The complexity of questions often reflects the depth of learning.

Cross-contextual application. Real learning transfers across situations. Someone who applies a conflict resolution technique from leadership training to a family situation has moved beyond surface-level memorization to genuine understanding.

Spontaneous knowledge-seeking. When learners voluntarily explore related resources or seek out additional learning opportunities, it indicates that the initial learning has created momentum for continued growth.

Teaching or explaining to others. Perhaps the strongest indicator of learning is when someone can introduce the concept to others or reference it naturally in conversations. This demonstrates both understanding and integration.

The Middle Ground: Better Indicators

These engagement patterns aren’t proof of learning either – they’re more like conditions that make learning transfer more likely. Someone who never engages deeply with content is unlikely to apply it. But deep engagement doesn’t guarantee application – it just creates better odds.

Return visits, time spent, and bookmarking behaviors are still analytics, not evidence. They tell us more about content quality and learner intent than completion rates do, but they’re not the same as watching someone solve problems they couldn’t solve before.

Think of engagement analytics as leading indicators rather than evidence of learning. They might predict whether learning transfer is possible, but they can’t tell us whether it actually happened. There’s more to explore about what these signals predict and what they miss entirely. More on that next week.

Questions I Ponder

What would happen if we designed learning experiences assuming most people won’t “complete” them traditionally? What if we optimized for immediate usefulness rather than comprehensive coverage?

How might we track performance improvements without creating burdensome measurement systems? And what would it mean to define training success based on application rather than completion?

I don’t have neat answers yet. However, I believe that our current metrics may be addressing the wrong problem.

Small Experiments Worth Trying

Instead of abandoning completion tracking entirely, what if we supplemented it with human-sized indicators?

Track what people actually use. Most LMS platforms already capture return visits and content engagement – we might just need to look at this data differently.

Ask simple application questions. Not immediately after training, but 30 days later: “Did you use this?” The answers might be revealing.

Notice skill demonstrations in real work. Look for evidence that people are doing things differently, rather than relying solely on assessment scores.

Pay attention to what people share. When someone references training content in a meeting or forwards a resource to a colleague, that’s application happening naturally.

The goal isn’t perfect measurement – it’s a better understanding of whether our learning experiences actually help people perform their jobs more effectively.

What patterns have you noticed between completion and actual learning? I invite you to share the gaps you’ve observed and what signals you trust instead. Converse with me at the Coffee Corner or on LinkedIn.

Proudly powered by WordPress
Verified by MonsterInsights