Automation and Service Reliability

Who We Are

What We Do

We are a team with 150+ years of collective experience managing and developing tools and solutions for Stanford University.  We are a modern, lean, self-governing engineering team and frequently switch hats between designing features, coding, testing for quality and performance. We value transparency and trust. We continue to create purpose in our work, seek individual mastery and encourage autonomy.

Our team is responsible for the ongoing support of numerous tools and integrations used across campus. These tools support various business functions. In order to successfully support many of these tools, we are required to intimately understand the underlying business processes and terminology behind the tools themselves e.g. network infrastructure and routing, collaboration and voice systems as well as phone number management.

Our team frequently gets pulled into projects requiring custom development. This development typically involves 3^rd party APIs and developing automation critical to the implementation or maintenance of an application or service, automating tasks in support of reliability, or developing tools where no software solution exists on the commercial market.

Communications, Networking and Voice Support

NetDB

NetDB is a Stanford University database application that stores network and node configuration information. It's generally used by local network administrators and University IT staff. Most commonly, NetDB is used to assign a new IP address to a computer or printer. The information in NetDB is loaded into two important network services at Stanford: DNS and DHCP.

• Read more about NetDB here.

NetDocs

NetDocs is an application that enables systems owners and delegates to review firewall projects and submit and remove firewall rules for those projects. We are currently serving as backup support for this tool.

Post Call Survey Support

A Post Call Survey takes place after normal call treatment. It is typically used to determine whether customers are satisfied with their call center experiences. This service allows us to configure a call flow that, after the agent disconnects from the caller, optionally sends the call to a Dialed Number configured for a Post Call Survey. We currently provide auxiliary support by sending call data to our 3rd party post call survey vendor.

Qualys Automation Tool

We have created an abstraction layer for Qualys that streamlines the functions in Qualys to make it easy for engineers to quickly display confirmed and potential vulnerabilties, rescan hosts, and/or get the information need so that they they can either be mitigated or marked false positive.

SNSR

Stanford Network Self-Registration (SNSR) is a tool which allows for user registration of devices onto the Stanford Network. Registration takes place via a web application SNSR followed by a client Stanford Network Registration Tool (SNRT). The product is available on both the wired and wireless networks. This product is a collaborative effort across University IT, including Networking, Desktop Systems, ISO, and Integration and Client Solutions. We are responsible for development, integration, deployment, and support of the tool into Stanford's environment.

• Additional Information
• For more help with SNSR, please submit a ServiceNow request.

SipDB

SipDB is a tool used as a central point for phone number and phone related data management, automated provisioning, and call reporting.  It acts as an integration point between order processing and VOIP systems, and it also acts as a reporting source for a variety of telephony related information. This tool is developed in Python and integrates with OrderIT for the purpose of provisioning phone numbers on campus. This tool also integrates with the Voicemail Password Reset tool.

• For more help with SIPdb, please submit a ServiceNow request.

The Data Collector

The Data Collector (TDC) provides metrics and trending information for voice services. In addition to providing historical counts of inbound and outbound call and pages per platform, we also correlate data from disparate systems on campus to help assess the business and understand adoption, growth, and compliance with security and other regulations. The Data Collector is a big data dataset consisting of 952 million call detail/messaging records growing by approximately 12.5 million records per month.

• For more help with The Data Collector, please submit a ServiceNow request.

UIPath

UIPath - We leverage UIPath to automate processes that can't be automated by way of APIs. UiPath enables us to streamline processes, uncover inefficiencies and provide insights, and allows us to leverage robots to do the repetitive work humans are forced to do because APIs do not exist to do that work.  Our RPA developers work with clients to understand all the repetitive steps performed by humans.  A robot framework is developed to repeat those same steps.  These can be customized to take inputs from spreadsheets or other data sources. Tedious swivel chair work can be done in some cases 500% faster and with much less errors. 

Voice Tools and Automation

Voice Tools and Automation - Our team serves as a resource to the many groups in IT Infrastructure responsible for delivering or supporting voice and collaboration services. In this space, there are a variety of tools such as DN Lookup, Dynamic Report Management, Paging Reports, SHC Network Location Search to name a few. Additionally we developed the "one click to join" functionality used on the video endpoints at Stanford. 

• For more help with voice automation, please submit a ServiceNow request.

Monitoring and Service Reliability

Our team develops and supports network monitoring and engineering tools which are continually adapted to meet the needs of the services and engineers who rely on them. These tools monitor over 4,000 devices (routers, switches, etc.) to produce trending data for over 250,000 interfaces. We leverage a suite of tools to gather and collect information from the devices on campus.

HP Network Automation

HP Network Automation - aka “Opsware” is software that automates the complete operational lifecycle of network devices from provisioning to policy-based change management, compliance, and security administration. We use this tool to provide configuration backup, mac address-table scraping, and make bulk changes. We also generate many reports from here.

Infosim

Infosim is a unified management solution for enterprise IT to discover/inventarize, monitor, track, ensure, and document service quality, including LAN/WAN, VPNs, storage, database, VoIP, and E2E service quality. We use Infosim to send major alarms to ITOC for Incident Management.

Statseeker

Statseeker network monitoring software is designed to monitor network performance by harvesting statistics from data that help engineers manage the network to make decisions and take action or identify critical issues. It provides network traffic stats for 250,000+ interfaces with ability to find top traffic ports quickly.

Grafana is an Open Source visualization platform used to query, visualize and alert on performance metrics collected from various servers and network devices.  Grafana, along with InfluxDB and Telegraf, allow us to better control access rights, as well as help us manage sharing visualizations among different teams and users. Grafana will ultimately replace the Cacti and Statseeker tools allowing the ability to create dynamic reports for the life of the device.  Grafana will also allow flexibility for end-user to set up alerts for their own devices via Slack, email or other means.  Currently we are running Adtran chassis reports, our linux servers performance (cpu, memory, processes, network bandwidth, uptime, etc) as well as PAN firewall historical data (about 5 years – 27,636,601 records).

In addition to the above Commercial Off The Shelf products, we have developed several tools that contribute to service reliability in Communication Services.

• Zzmon is used to tag events from many sources and allow one view of noteworthy events by time, service, device
• Alert subscription allows for individualized subscription to alerts from many sources
• Swatch is used to filter events in syslog/snmptrap stream and send to Zzmon and/or individual emails.
• Lookup Tool displays procedures/network status for network devices, servers, phones, etc.
• Autohowis provides up/down monitoring of network hosts, POS terminals, etc.
• Host/Switch Search  - find host in the network or hosts on a switch
• Bookstack/Infradocs  - This is a is a simple, open-source platform for organising and storing information. 

Metrics and Monitoring Dashboard are available on the IT Metrics dashboard. The Status and Metrics "traffic lights" on the University IT home page indicate the availability of key systems. In this program, gadgets are real-time graphical representations of metrics. They can be grouped into a dashboard,which is simply a container, and displayed on your browser screen. Integration and Client Solutions provides development and maintenance support for many of the monitoring gadgets made available on the campus dashboard.

• dashboard.stanford.edu
• For more help with Metrics and Monitoring, please submit a ServiceNow request.

Integration Development and Support

Tessitura Authentication Integration

We provide ongoing authentication support for Stanford Ticket Office Tessitura system.

Campus Maps API (LBRE)

Completed: We are developed a Campus Map API for LBRE to allow applications on campus to easily leverage their campus maps information database.

Cloudpath/SNSR Integration

Completed We developed and support the integration between SNSR and Cloudpath to deliver certificates to endpoints. 

Training, Mentoring, Professional Development and Evangelism

Our team is passionate about growing and developing our own skills as well as the skills and expertise of the Stanford community. As a result, members of our team have been known to teach technology classes or leading open labs for UIT's Tech Training division, participating in Communities of Practice, mentoring or being mentored in either the UIT Mentoring Program or the Stanford Manager Academy, attending the collaborative Unconference, and even teaching Stanford Continuing Studies courses. 

Certifications and Credentials

Some of the services we support require certifications. These certifications are maintained and current.    

More Information

For additional details about Integration and Client Solutions, contact Anne Pinkowski.