A Cybersecurity Story

On March 26, University IT staff members' quick response and teamwork resolved an incident within the Stanford network. Follow the timeline of events below to see how the story unfolds.

11:31 a.m.

Several of Stanford's online services began experiencing intermittent outages. The unexplained disruption of multiple services prompted University IT to activate the Departmental Operations Center (DOC), which facilitates IT coordination throughout Stanford by way of a war room, a conference bridge, a procedures playbook, and well-defined roles.

12:16 p.m.

Members of the DOC were alerted and convened. Noting that widespread distributed denial of service attacks had been reported in cybersecurity mailing lists earlier that day, the DOC team was advised to search for a source generating a large volume of network connection attempts.

12:35 p.m.

Within 20 minutes, the University IT network team identified and immediately disconnected a host on Stanford's campus that was generating such traffic. Because of the quick response and collaboration, all network services were restored to normal within approximately one hour of when the outages first began.

Investigating the intrusion

The compromised system was found to be a Linux laptop computer. The University IT information security team examined the device and discovered that the attacker had gained root access through secure shell (SSH) connections initiated from overseas at approximately 10:30 that morning. Unbeknownst to the new owner of the laptop, the previous owner had permitted SSH connections from external sites via a configuration in the corresponding NetDB node. With direct access from remote locations, the attacker was able to successfully log in as root by guessing common passwords within hours of the computer being powered on.

Lessons learned

This story serves as a reminder of how important it is to protect every machine on our network from intrusion by applying best practices such as using strong passwords, integrating two-step authentication wherever possible, disabling direct root remote logins, enabling local firewalls, maintaining NetDB records, and regularly updating system software. These and other best practices are included in the new Minimum Security Standards website found at http://minsec.stanford.edu/.