IT Spotlight: Systems Operations

iStock_000069490103_MediumInterested in getting to know your SysOps team? Check out our IT Spotlight on the team.

Be sure to check out SysOps Success Stories! >

Who is the Systems Operations Team?

The Systems Operations team, commonly referred to as SysOps, is not well known but plays a vital role in the daily readiness of all data systems supporting the campus technology. The SysOps team is a component of Information Technology’s Enterprise Infrastructure Systems (EIS) department. As part of EIS, the SysOps team is responsible not only for critical systems, but plays a critical role in the upkeep and reliability of the underlying infrastructure that holds up all the servers and applications located in the data center.

The Systems Operations team is currently comprised of the following members: Dwayne Trouille, David Pierce, Ken Gutierrez, Michael Young, and Jody Hudnall.

What does the Systems Operations Team do?

Every morning, each member of the SysOps team starts their day by inspecting all of their critical server environments, inspecting backup logs, inspecting SMTP SPAM logs, scanning resource usage in the virtual server environment, walking through the data center visually inspecting all the equipment for failures, and reporting on all findings. When problems are identified, the issue is documented and immediately addressed.

Throughout the remainder of the work day, the SysOps team will field numerous service incident tickets, to include phone calls on reported issues with systems. When not addressing problems, SysOps team members stay active by working on important projects in order to meet deadlines, as well as, researching ways of improving their systems feature sets or efficiencies. Each day is full of activity that requires a lot of their time and focus in front of their computer monitor. So while this team does not regularly get out and work face-to-face with other areas of IT or campus departments, they work vigorously from their office to ensure maximum uptime and reliability of all critical systems on campus that are located in the data center.

At the end of the work day, the SysOps team attempts to wrap-up all daily tasks, find an ending point for ongoing projects, update all their documentation from that day, and perform a final walkthrough of the data center to visually inspect all equipment for new failures.

When away from the office, all SysOps team members continue to receive monitoring alerts from critical systems. In the event a failure occurs, the responsible team member will remote into the data center and make repairs or come into the office should it be required. Many of the hardware based systems, such as hard disk arrays or servers, automatically report a failure to the vendor and a repair part is automatically dispatched. In addition, the responsible team member is in communications with the vendor to coordinate a timely dispatching of the part and/or repair engineer.  Even if the failure is outside the scope of all the SysOps team members’ ability, the team always responds to failures that impact the data center, such as, cooling system failures, electrical outages, backup generator failures, and backup battery UPS failures.

Where is the Systems Operations Team located?

The SysOps team is located in the Oak Street Parking Deck on the second floor. The location of this team was based on keeping them in close proximity to the VSU campus data center.  Locating the team next to the data center allows for rapid response times, regular visual monitoring and tighter door access security.

When is the Systems Operations Team called to action?

The SysOps team is responsible for the deployment of new servers in the VMware virtual server environment, as well as, making adjustments to the virtual environment when servers are not performing as expected. Anytime the underlying hardware is being upgraded, replaced or repaired, the SysOps team is the first responder and often performs the required tasks. Many of the campus applications used by students, faculty and staff are supported by the SysOps team; therefore, the team is often involved with projects starting with the researching of an application or service all the way through the testing and deployment processes. Finally, the SysOps team provides post deployment support of all file systems and databases and any associated applications or services.

Why is the Systems Operations Team so vital to campus technology?

The importance of SysOps to the operability of VSU’s campus technology cannot be truly appreciated until the full scope of their support role is understood. To summarize the scope it is important to understand that everything inside the data center whether physical or logical in nature is supported to some degree by the SysOps team. To this point, the SysOps team has defined not only a disaster recovery process for the data center, but has also documented all critical services running in the data center and developed standard operating procedures used in the proper shutdown and startup of these critical services.

Systems Operations Team responsibilities summary:

Systems Operations is responsible for managing the Valdosta State University data center to include but not limited to:

  • Data Center Facility Support and Protection
    • Secure Access
    • Electrical Power
    • Backup Power Generator
    • Battery Backup System
    • Air Conditioning
    • Fire Suppression System
  • Hardware Installation and Support
    • Computer Rack Placement/Maintenance
    • Server Equipment Installation
    • Storage Installation
    • Networking Equipment Installation
    • Daily Inspections of All Housed Equipment
    • Tier 1 Support for All Housed Equipment
    • Tier 2 Support for Some Equipment
    • Tier 3 Support for Most Equipment
  • Application/Service Installation and Support
    • Primary Support for the VMware Server Farm
    • Primary Support for the Cisco UCS Blade and Chassis System
    • Primary Support for all SAN Storage
    • Primary Support for Data Backup System
    • Primary Support for Linux Deployments, including all Linux based Web Services systems
    • Primary Support for Oracle Applications Databases,
    • Primary Support for Internal SMTP Services, including the Email Distribution List Service
    • Primary Support for some Academic and Business Systems/Applications

Systems Operations Summary

The SysOps team is no different than any of the other departments of Information Technology. All areas of IT are expected to perform above average and ensure that VSU’s technology is available 24/7.  What does separate the SysOps from the rest of IT is how vital their support for the data center is to all students, faculty, staff and other supporting IT departments.