Troubleshooting Mindset Series #011 What Changed Before the Problem Started?

When troubleshooting an incident, one of the most useful questions is often not:

“What is broken?”

but:

“What changed just before it broke?”

A system can run normally for months and suddenly develop a problem after a configuration change, software update, network modification, firewall policy change, or application deployment.

That timing can be an important clue.

Investigation

Step 1 — Establish the timeline

Identify:

• When did the problem first occur?
  
  
• Was the service working normally before that?
  
  
• Is the problem continuous or intermittent?
  
  
• When was the last known good state?
  
  

Step 2 — Look for changes

Check whether anything changed shortly before the incident:

• Network configuration
  
  
• Firewall policy
  
  
• VM configuration
  
  
• OS patches or updates
  
  
• Application deployment
  
  
• DNS changes
  
  
• Routing changes
  
  
• Hardware or infrastructure maintenance
  
  

Step 3 — Correlate the timing

A configuration change occurring five minutes before an incident is more interesting than a change made three months ago.

But correlation alone doesn't prove causation.

The change should still be validated against the symptoms and available evidence.

Step 4 — Compare before and after

If possible, compare the current configuration with the previous known-good configuration.

Sometimes the fastest path to the root cause is identifying what is different, rather than checking everything from scratch.

Common Mistake

One common troubleshooting mistake is starting with the component that looks suspicious.

For example:

“The network is slow, so the switch must be the problem.”

Instead, ask:

“What changed immediately before users started experiencing the problem?”

That question can dramatically narrow the investigation.

Lessons Learned

Not every change causes an incident.

But when an incident appears immediately after a change, that change deserves investigation.

Don't assume.
Don't blame.
Correlate the timeline with the evidence.

My Rule

The last known good state is one of the most valuable pieces of evidence in troubleshooting.

Knowing what changed between “working” and “broken” can reduce a huge troubleshooting scope into a much smaller one.

Discussion

When a production service suddenly develops a problem after months of stability, what do you check first — the current configuration, system logs, or recent changes?

#Sangfor #HCI #Troubleshooting #NetworkTroubleshooting #RootCauseAnalysis #TroubleshootingMindset

Thanks to share!

“What changed just before it broke?" must piece together with "And please don’t say ‘nothing.’"