Client Technology Solutions and Consulting (CTSC) master service agreement (MSA) outlines obligations and commitments between the CTSC team and our clients. It is separate from any other agreement with University IT.
CTSC provides a handful of different technology services to the campus community. This MSA is mostly oriented around our Comprehensive Support agreement. Other facets of our service apply as indicated.
CTSC provides system administration covering nearly every aspect of deploying and managing a server (or group of servers) for the duration of its lifecycle. For a single monthly recurring charge, typically applied to each server that's included in the agreement, almost all supportive activities are covered.
Our comprehensive support agreements can cover systems around the clock (24/7) or only during business hours (M-F, 8-5). Either schedule can be applied to development or production servers, but we recommend 24/7 support for all mission-critical systems.
We can help you evaluate your technology needs, manage the risk categories of your data, understand how the cloud can successfully be used at Stanford, be made aware of all options for deployment, look at different Software-as-a-Service products, and successfully tackle large IT systems engineering projects. We use a time and materials hourly rate to recover this work.
CTSC manages central CrashPlan service, a product that automatically and constantly backs up user data to the cloud, including High Risk and PHI classifications. Today, across campus, there's over 3.5 petabytes of compressed and encrypted data securely stored in the cloud, from over 24,000 devices owned by nearly 13,000 user accounts.
For every comprehensive support engagement, we use a custom, established approach to guide our efforts throughout the lifecycle of our clients' solutions, from inspiration to ongoing operational work to ensure uptime and availability. We are a team of consultants that take our responsibility to educate and guide our users, protect Stanford data and advance the mission of the University seriously.
We maintain a list of common activities that are included in our comprehensive support agreements. Here are some of the highlights:
There are some activities that are not included in our comprehensive support agreements. These include:
While our comprehensive support agreements cover most expenses relating to the build and operational support of your system, there are some charges that might be required to fully protect and manage your IT resources.
If we build a solution in Amazon Web Services, Microsoft Azure or Google Cloud Platform, we will create a dedicated account where all associated costs within that account are charged against your PTA. This account is for the exclusive use of CTSC-managed resources. We'll happily create a second account for non-CTSC purposes and integrate that with Stanford's infrastructure.
Our comprehensive support agreements include management of backups to protect your data. CTSC staff will install and configure enterprise backup software to store copies of your files in the cloud, monitor the daily or weekly progress of backups and troubleshoot any issues that might come up. We will also perform test restores and complete restores as needed.
Cloud backup is a cost-effective way to ensure the continuity of your operations should a regional disaster strike or an accidental deletion interrupt your work.
CTSC uses enterprise-grade backup software that has been approved by Stanford Information Security Office to protect all risk classifications. Depending on your particular system, the software might be configured to encrypt data before it leaves your server and might be stored on encrypted cloud storage volumes. It will always be transmitted over an encrypted network connection, however.
As part of this process, your PTA will be charged a modest recurring monthly charge for the backup software.
Cloud deployments will utilize cloud native backup solutions and may not necessitate a separate backup license.
In addition to the software licenses to perform backups, our clients’ PTA will be charged each month for the associated cloud storage costs. This variable charge corresponds with the volume of data that’s stored in the cloud.
We select the best cloud vendor and storage tiers based on factors like the expected frequency of performing restores, the speed at which we might need to restore data, the overall volume of data and other client-centric considerations. Every storage provider that CTSC uses has already been approved for all risk classifications.
Depending on the volume of data and timing of restores, there will likely be a charge from your cloud provider for data transfer, expedited retrieval, I/O operations and possibly other activities.
In order to use Red Hat Enterprise Linux (RHEL), a support agreement must be purchased to provide access to patches and software updates. CTSC will help with the procurement of the agreement and apply the annual licenses, but any expenses are the responsibility of our clients. We recommend deploying CentOS as a free and open-sourced version of RHEL.
Operating systems that age into a limited, near-end-of-life or extended-support phase may require a for-fee subscription to receive software patches. The expenses relating to this subscription is the responsibility of our clients.
If the extended-support agreement is determined to be too limited to secure your particular solution, or fails to include the necessary remediation against vulnerabilities in applications or the operating system, CTSC will insist on upgrading or rebuilding your server to bring the system into compliance as a condition of continued support.
Clients are responsible for purchasing and managing necessary support contracts with vendors. CTSC does not manage or pay for support agreements with hardware vendors like Dell, or support agreements from cloud vendors like Amazon Web Services, either.
CTSC is committed to billing transparency and accuracy. At the earlier stages of your engagement with CTSC, we will discuss whether your service requires either an additional complex charge or whether you should expect a time and materials charge in addition to the flat monthly recurring charge. Many systems support agreements do not require either.
While most of CTSC's services are included in our comprehensive support agreements, there may be rare cases where we must apply a supplemental charge to support complex systems. We apply this charge to recover extra, ongoing administration overhead. An example of complex systems might be a server arranged as a failover cluster, or a system that requires additional management of attached fibre channel storage.
There may be occasions when a given project requires deeper research or a process of discovery, the deployment of a proof of concept or an initial beta system for demonstration and trial use. For these, an hourly time-and-materials charge will be discussed to recover work outside of our comprehensive support agreements. Typically, this will be billed at the higher standard University IT rate for “Design & Development” work. Billing at the less expensive “Service Deployment & Administration” would be at the discretion of CTSC where appropriate.
If your collaborators need to log into systems securely and you wish to provide them with a Stanford email address, we recommend a full-sponsored SUNet ID. Our clients are responsible for this modest monthly recurring charge.
As part of Stanford's Minimum Security requirements, all high risk systems must be administered using a PAW. CTSC's technologists have been provisioned a PAW, and regularly use this special laptop to undertake our administrative activities.
If our clients wish to perform administrator activities on their high-risk servers (usually defined as a root or elevated privilege actions), they must procure their own Privileged Administrative Workstations for their constituents. This can be requested using ServiceNow.
CTSC is a service center that uses a cost recovery model; this means we are supported entirely by the revenue we generate from our services. This includes paying for our tools and software, training, staff salary and benefits, as well as impacts our ability to grow our team. The rates and fees we charge our clients recover these expenses. Importantly, our annual budget is designed with the intention of breaking even; we are legally prohibited from profiting beyond 5% of our actual expenses.
When work commences on systems covered under our comprehensive support agreements, CTSC will apply the associated monthly recurring charge to your PTA to recover the expense of providing this service. This recurring charge will continue each month until the system is decommissioned or the agreement is terminated.
Our comprehensive support model typically covers virtually all support activities relating to your system. To balance out an uneven distribution of work, we expect our clients to commit to at least a one-year term of service.
We hope you find value in our support agreements. However, if that’s not the case and we fall short of your needs or expectations, we want to know as soon as possible for any course correction. We invite you to share your feedback so we can learn and improve. We’ll listen and adjust our practice to make sure you’re satisfied.
However, if for whatever reason, things go awry and we are unable to keep your trust, faith, and satisfaction, we will agree to transition your system’s support to another team and immediately stop billing.
Conversely, if CTSC's client relationship becomes problematic, we commit to seeking a productive, respectful, and healthy exchange to provide an opportunity to make improvements. Ultimately, CTSC reserves the right to terminate a client’s support agreement if we can not find common ground.
Normally, our business hours are from 8:00 AM to 5:00 PM, Monday through Friday. We follow the University's staff calendar that informs when we're out of the office for holidays.
Every December and January, Stanford closes the campus to regular activities for about two weeks between Christmas and New Years' Day. During Winter Closure, we defer regular requests for support until we return in January. Like the rest of University IT, we operate only a skeleton crew during this time, responding only to major outages.
We love chatting with our clients using Stanford Slack Enterprise Grid. This is especially true when setting up a system where we need to work collaboratively. We have shared our #tcg-external channel with the whole grid, so anyone can search, lurk and join to chat. We're online nearly all the workday (and then some).
CTSC typically creates new client-specific channels, just to engage privately with our colleagues. We will share that channel with our clients' workspaces.
Please submit any non-urgent support requests and incidents directly to our team by using our custom support link: https://ctsc.sh/help. This easy-to-remember URL resolves to Stanford ServiceNow, but is hard-wired to our assignment group.
Your ticket will be trackable and viewable by our entire team. If the ticket is best served by another group, we'll route your help ticket accordingly.
We will not participate in client-side support ticketing systems or support mailing lists.
If you have a 24/7 Comprehensive Support agreement, you can contact us off-hours via our support line at +1 (858) 888-9634.
If your system has a business-hours only comprehensive support agreement, we will undertake any work at the next available workday.
We commit to responding expeditiously to alerts, outages and incidents. However, like the rest of University IT, we can't commit to a resolution period.
Calling our off-hours urgent support telephone number, +1 (858) 888-9634, will give you the option to be connected to a manager or director to escalate an issue.
You can send our whole team an email by sending something to our Office 365 group address: team@tcg.sh.
CTSC supports systems that run modern versions of Red Hat-related distributions of Linux (including CentOS, Oracle Enterprise Linux and Amazon Linux 2), Debian-related Linux (including Ubuntu) and systems running Windows Server operating systems.
Security compliance at Stanford requires operating systems that are updatable with regular releases of software patches. For this reason, any OS that no longer enjoys mainstream support by the vendor or community of developers cannot be supported by CTSC.
Keeping your operating system and software up-to-date is a necessary and critical component of protecting Stanford’s digital assets. Even servers that exclusively hold fully publically-accessible data are important to secure, because scripted and automated attacks will exploit a vulnerability. Once compromised, cybercriminals are in a beneficial position to transit across our network to compromise other systems.
Security compliance at Stanford also requires CTSC to patch Linux systems no less frequently than every 90 days; Windows systems will be patched no less frequently than once a month. Be advised that these are minimum frequencies. Depending on the results from security-related efforts, actual patching and maintenance frequency might be greater.
There may be times when high-risk software vulnerabilities are discovered on your system. For this reason, urgent remediative activities may be required. While we will attempt to accommodate our clients’ need for uptime and consistent availability, our necessary emergency response to certain high-risk vulnerabilities might supersede this accommodation and could disrupt your business operations.
For any system with a business-hours only support agreement, regular operational maintenance will be performed exclusively between Monday and Friday, 8:00 AM through 5:00 PM (notwithstanding urgent security-related activities that might occur).
For systems that enjoy 24/7 support, three additional off-hours maintenance windows are available. These windows correspond with the published University IT-wide maintenance windows found at https://uit.stanford.edu/service/changemgt/maintenance-windows. All times are local.
During certain times of the year, University IT pauses most regular activities on systems that have the potential to disrupt important campus activities. Information about maintenance freezes is published here: https://uit.stanford.edu/service/changemgt/freeze; example periods include:
CTSC-managed systems that don’t have the potential to disrupt these activities might not be included in this freeze. For example, file or application servers used exclusively by a department or workgroup whose operation doesn’t affect central billing, academic activity are likely not affected by these freeze events.
CTSC will submit a ServiceNow Change Request prior to undertaking any potentially disruptive work.
Clients will identify the internal contact who will receive requests (by email) to approve or deny the request to commence work. An expeditious response is important. Without approval, work cannot be undertaken and may require rescheduling.
Our clients' business and/or technical contacts have the responsibility to communicate outages or disruptions to a workgroup or stakeholders.
We recognize security-related software, especially when configured to support the highest levels of data risk classification, can be frustrating to encounter. It may impede, thwart or delay normal work performed by our clients. For example, application binary whitelisting will prevent any unauthorized software modifications, while Privileged Access Workstations can be expensive..
Our clients, like everyone at Stanford, are expected to respect these data handling requirements listed in Chapter 6 of the Stanford Admin Guide and the associated Information Security Office and Privacy Office pages.
Clients must not tamper with, configure or remove software managed by CTSC for these purposes.
We will work diligently to minimize the impact of any required software on the performance of our client systems.
CTSC uses enterprise systems management tools to configure and administer systems at scale. We use automation when possible to minimize error. For this reason, we require software like BigFix, Puppet and Site 24x7 to be installed and configured to our specifications.
When inheriting a system supported previously by another team, we will evaluate whether we require the server to be rebuilt based on our standard configuration. At our discretion, we may limit our modifications only to installing and (re-)configuring management software to our specifications.
We ask every new client to provide at least one technical and one business contact. It can be the same person, but not a mailing address or group IMAP account.
This information is used for notifications for events like patching, change request approvals, our client mailing list, billing questions and other business-related purposes.
We ask that our clients keep us informed of any internal organization changes, new PTA billing accounts and other events that might affect the delivery of our service.