Observability Africa

One Person Cannot Lead and Debug: A Lean Incident Command Model

Abe Apithy
Observability Africa cover showing one overloaded incident responder splitting responsibilities across coordination, technical response, and communication.

When a serious incident begins, the strongest engineer is often asked to do everything. They diagnose the failure, approve risky changes, answer leadership questions, update customers, and keep notes. This can look efficient because the team is small. In practice, it creates a single point of cognitive failure.

One person cannot maintain the full incident picture while also debugging at depth. The more urgent the incident becomes, the more important it is to separate coordination from technical work.

A lean incident command model does not require a large organization or a complex tool. It needs clear authority, simple roles, a shared record, and a predictable communication rhythm.

Incident command is a coordination system

Google’s incident management guidance is based on the Incident Command System and emphasizes three needs: coordination, communication, and control. Its core roles are an incident commander, an operations lead, and a communications lead. The model can expand during a major event or contract when the incident is smaller.

The important idea is not the job title. It is the separation of responsibilities. The person directing the response should not also be the only person changing production.

The Google SRE incident response guidance states that the incident commander coordinates the response, the operations lead performs technical mitigation, and the communications lead manages updates and inquiries. Any role that has not been delegated remains with the incident commander, but delegation should happen as the response grows.

The minimum useful structure

A lean team can start with three responsibilities.

Incident commander

The incident commander owns the response process. They define the current objective, assign work, approve major direction changes, remove blockers, and decide when to escalate or hand off. They keep the team focused on customer impact and restoration.

The commander does not need to be the most senior person. They need enough context and authority to coordinate effectively. During an incident, role clarity matters more than the normal reporting structure.

Operations lead

The operations lead owns technical diagnosis and mitigation. They may work with several responders, but all production changes should be coordinated through this role. This reduces conflicting actions and makes rollback decisions clearer.

The operations lead should regularly report four facts:

Communications lead

The communications lead maintains a reliable information flow. They update internal stakeholders, customer-facing teams, and external status channels where appropriate. They also protect technical responders from repeated questions.

A short update should state the impact, current response, next checkpoint, and any action required from the audience. It should not speculate about a root cause that has not been confirmed.

What if only two people are available?

Two people can still separate command from technical work.

This is better than asking the technical responder to manage every conversation. If a third person joins, communications should be the first responsibility handed off. If only one person is available, they should still work in distinct modes: pause technical work at fixed intervals, record the current state, send an update, then return to diagnosis.

Use one shared incident record

Chat is useful for rapid coordination, but it is a poor source of truth. Important decisions are mixed with questions, automated alerts, and partial observations.

Create one incident record with:

The record should be short enough to maintain during pressure. Link to dashboards, traces, logs, tickets, and long chat transcripts instead of copying everything into it.

This approach aligns with the broader principle in NIST SP 800-61 Revision 3: incident response should be integrated into organizational risk management, with preparation, clear responsibilities, communication, response, and recovery treated as connected capabilities rather than isolated technical tasks.

Choose update intervals before the incident

Silence creates pressure. Stakeholders begin asking for updates, responders lose focus, and several versions of the incident story emerge.

Set a standard communication rhythm in advance. For example, a critical incident might require an update every 15 or 30 minutes, while a lower-severity incident may use a longer interval. The exact timing should reflect customer impact and organizational needs.

Every update should answer:

  1. What is the current impact?
  2. What has changed since the last update?
  3. What is the team doing now?
  4. When will the next update arrive?

Predictable updates reduce interruptions even when the technical situation has not changed.

Design for unreliable connectivity

Incident processes often assume stable internet access, always-available collaboration tools, and easy access to dashboards. That assumption is risky for distributed teams and environments where connectivity may degrade during a major event.

Keep an offline or low-bandwidth version of the incident process. It should include:

This is not an argument for building a separate process. It is a resilience layer for the same process.

Define severity through impact

Severity should describe impact and urgency, not how technically interesting the failure appears. A small component failure may deserve the highest severity if it blocks a critical customer journey. A widespread internal error may be lower severity if users are protected by graceful degradation.

A practical severity model can consider:

The incident commander should be allowed to raise severity quickly. Reducing severity should require evidence that impact and uncertainty have both decreased.

Practice the handoff

Long incidents create fatigue. A handoff should transfer more than a chat channel. The incoming commander or operations lead needs a concise operational picture.

Use a handoff checklist:

Practice handoffs during exercises. Google’s guidance recommends role-playing known incidents so responders become familiar with the structure before a real emergency.

A 30-minute implementation plan

  1. Choose names for the command, operations, and communications roles.
  2. Create a one-page incident record template.
  3. Define who may declare a major incident.
  4. Set default update intervals by severity.
  5. Document a two-person fallback model.
  6. Run a short simulation using a previous incident.
  7. Improve the process after each exercise and real event.

Structure protects technical focus

Incident command is not bureaucracy added to engineering. It is a way to protect attention when the organization can least afford confusion.

For lean teams supporting critical digital services across Africa, the model should remain simple, adaptable, and resilient to limited staffing or connectivity. Three clear responsibilities, one shared record, and regular updates can create meaningful control without expensive incident management software.

Observability Africa welcomes organizations interested in supporting independent incident-readiness education, exercises, and community resources while respecting editorial independence. Teams that want a structured review of their alerting, response process, and operational readiness can also request an independent observability audit.

Exit mobile version