The IT Operations Center’s Nonstop Work Keeps You Working
February 16, 2017
Most of us rely on technology to do our jobs, and without it, our ability to perform our work at Stanford would be greatly reduced or even stopped.
One University IT (UIT) team works 24 hours a day, 365 days a year to make sure Stanford’s technology systems, services, and network infrastructure stay up and running. Simply put, they work nonstop to reduce and eliminate any work stoppages, such as inoperable phone or paging services, network access issues, or server failures.
Led by Marvin Kirkendoll, the IT Operations Center (ITOC) consists of 16 onsite staff who constantly monitor the systems and services UIT provides to Stanford, its hospitals, and its remote locations, such as Washington and Beijing. The team acts as first responders for all major technical issues, and work quickly to resolve them. They handle about 2,000 incidents a month, 80 percent of which they are able to resolve within the team without escalation to other UIT teams, campus partners or vendors.
ITOC team members Yunhua Annie Liu, Harry Siva, and Kenny Chu in the ITOC Command Center.
“Communication is central to the learning and educational mission of the university,” said Yunhua Annie Liu, an ITOC specialist. “When the hospital trauma department has a paging issue around midnight, or the Stanford Ghana remote site has a network problem, they reach out to ITOC to fix it. ITOC keeps the critical lines of communication open across the whole campus and at remote sites worldwide.”
Located on the second floor of Forsythe Hall, these 13 men and three women work staggered shifts to provide round the clock support, which also includes performing system maintenance and testing.
What does ITOC success look like?
To students, faculty, and staff at Stanford, the work of ITOC is largely invisible, and that’s the way the team likes it.
“Success is defined when my team is able to proactively resolve incidents without escalation, and before users’ experience any impact,” said Kirkendoll.
To accomplish that, ITOC staff practice shared values: quick response time, strong collaboration with cross functional teams, and problem ownership.
ITOC team members Marty Dart (left) and Harry Siva monitoring UIT systems and services.
“The value that we provide is that our vast monitoring system allows us to identify and respond to problems quickly, 24x7,” said Marty Dart, an ITOC technical lead. “We've worked closely with other groups to establish best practices and ensure rapid recovery to problems.”
For end users, Dart said ITOC is “the first stop for accountability.”
“When a user calls in with a problem, we own it,” he said. “Regardless of which group ends up resolving the issue, users can still contact us to check status and escalate if needed.”
For other UIT teams and campus partners, ITOC provides an invaluable backup service and round-the-clock peace of mind.
“We allow UIT engineers and campus partners to focus on their core responsibilities and rest assured knowing issues will be monitored and handled,” said Kirkendoll, who has led the team since 2014 and been part of the team since its inception in 2005.
The School of Engineering IT team began leveraging ITOC in 2012 for after-hours monitoring and response. The partnership with ITOC allowed the school's small technical team to professionalize their emergency response procedures and reduce burden on their staff, said Alex Keller, Senior Systems Engineer at the School of Engineering.
"Effectively adding ITOC to our extended team, they could ensure that priority alerts received immediate attention but less critical situations didn’t result in an SoE IT staff member being unduly woken in the middle of the night," said Keller. "Over the years this partnership fostered a camaraderie between our respective teams and enhanced SoE’s visibility on the campus at large."
ITOC Manager, Marvin Kirkendoll.
Besides monitoring systems and resolving issues with them, ITOC also supports system upgrades and new service deployments. This year, ITOC is additionally focused on integrating incident and service alert notifications with UIT’s new service management system, ServiceNow, to be better connected with the rest of UIT’s technical teams.
That’s on top of ITOC’s routine work of serving as the primary support for enterprise voice, paging and operator services platform, operations owner of 24 critical services and Stanford’s Network Operations Center (NOC), and primary UIT coordinator for the university’s disaster recovery center.
Through it all, Kirkendoll hopes “a mindset of support excellence” is what shines through with every interaction his team members have with UIT colleagues, campus partners, and end users.
To learn more about ITOC, contact Marvin Kirkendoll at email@example.com.
To contact ITOC regarding to report an urgent technical issue or system outage, call 650-725-4357 anytime, any day, 24x7.
DISCLAIMER: UIT News is accurate on the publication date. We do not update information in past news items. We do make every effort to keep our service information pages up-to-date. Please search our service pages at uit.stanford.edu/search.
Google Team Drive is now available as part of Stanford’s G Suite service (formerly Google Apps). Team Drives are shared spaces where teams can store, search, and access files anywhere, from any device.