Reactive Work: Reactive work will be defined as all work that is done as a reaction to a system event or user need. Examples would be handling system problems, hardware failures, requests for changes in authorizations, accounts, application settings. Work that has a business need to happen in a rapid fashion, either to alleviate a problem with an existing process or system, or a change in configuration. Reactive work, in general, will be work that can be completed in under half an hour, or the process for determining the method of remedying the problem can be determined within a half hour.
Proactive Work: Proactive work is all work that can be scheduled for some future time. Work that needs to be done to maintain processes and systems in good functional and secure condition. Proactive work will be scheduled and will be a lower priority than reactive work.
Change Management Procedure
All changes requests to supported systems will go through the HelpSU system, so that they can be tracked. Some system changes will be deemed to be proactive work, and will be tracked in a different projects queue for ease of maintenance of the reactive work queues.
Requests for change will be acknowledged by the Data Center group, and a timeline suggested for those changes deemed to be proactive work. Reactive work will be acknowledged within 4 business hours, and completed within one business day.
The Data Center Staff may need to wait for the next scheduled maintenance period to perform some change requests, due to system downtime requirements of the request.
Changes to monitoring, process, addition of new applications or machines, and modifications of configuration files will need to be approved by the Data Center staff or Data Center manager before they are put into place.
Scheduled Maintenance Procedure
The Data Center staff will set scheduled maintenance windows, and adhere to them for system changes which require system downtime. Care will be taken to limit the systems downtimes as much as possible. Care will also be taken to not make changes which require a downtime to systems outside of the scheduled maintenance time period.
For each scheduled maintenance period, if a maintenance is going to performed, an email will be sent to the mailing list of the affected system, system-announce or if it is just a particular application (for instance a restart of a system that is known to take a while to recover) system-application-announce. (All mailing lists referenced in this document are hosted on med.stanford.edu.) The announcement will contain the duration of the maintenance, and a general description of what is going to be done (including the brief description of any and all patches which are going to be installed).
Backup and Restore Procedure
Backups will be scheduled on a regular basis. Incremental backups will be done nightly. Full backups will be done approximately monthly. A copy of full backups will be stored in a different physical location within 2 days of the full backup.
Recoveries will be able to be done as needed, recently modified files (within the last 2 months) should be available within a normal reactive work schedule. Files deleted longer ago than 2 months may not be available as rapidly, but should be available within 3 business days.
The Data Center will maintain backups for the documented necessary retention period for various legal requirements, currently this appears to be 6 years. Data Center will maintain the necessary software to restore data from all archived tapes within this time period.
Physical Access Policies
Physical access to the data center will be limited to those with administrative need to enter the data center. Access will be via electronic badge. Any personnel not part of the data center staff will be accompanied by a data center staff member. Access to the data center can be arranged as needed during normal business hours. Emergency access after hours will be accommodated, the method for starting the process of gaining access after hours will be to call the Data Center phone line and leave a voice message with your name, contact phone number, and reason for needing access. A data center staff member will return all calls within a reasonable time for after hours emergencies.
Firewall Modification Requests
All requests to modify firewall rules must be formally submitted to the data center staff. Firewall modification requests must be within the general security models for the Data Center or they will not be completed. Requests for firewall modifications will be completed with 1 business day.
Appropriate Use Policie
Adhere to the Stanford Acceptable Use Policies
Hardware Requirement
All new machines going into the Data Center must be rack mountable, unless prior arrangements have been made to allow particular non-rack-mountable hardware into the Data Center. Existing machines which have a business need to be in the Data Center which are not rack mountable will be allowed, but there will be an expectation that these machines will be replaced within a reasonably short time period with more appropriate hardware or the functions that those non-rack-mountable machines to be relocated to other servers which are more appropriate for the Data Center.
All hosts will need to undergo a security scan or audit prior to being relocated into the Data Center. Also, while the Data Center provides increased network security, it is still necessary for hosts to take care of their own host-based security policies. Hosts in the Data Center will be regularly scanned for vulnerabilities and those reports provided to the administrator(s) of the hosts.
All machines and hardware that will move into the Data Center will need to be coordinated and scheduled with the Data Center staff. As we grow the number of machines in the Data Center, we will need to incrementally expand the infrastructure that supports the entire Data Center. Sometimes this may mean a small delay in the deployment of hardware into the Data Center until we have the appropriate infrastructure (including console, network, power, and rack space) for the hardware to be deployed
Security Policies
The School of Medicine Data Center is a consolidated server room intended to provide a 24x7 high availability, redundant, and secure environment for those School of Medicine machines which need a higher level of security than the rest of the School of Medicine network can provide. Intended uses include meeting HIPAA requirements for servers, meeting California code requirements for privacy, as well as other servers that need high availability and increased security. The Data Center design is intended to enable either the Data Center systems administration staff, or the systems administrators of the servers housed in the Data Center to be able to effectively manage their machines remotely and securely. Much of this is directly driven by the School of Medicine IRT Strategic plan.
Security Zones
The Data Center will have three distinct security zones, Semi-Secure, Secure, and Highly-Secure. The Semi-Secure zone will host machines that provide services to anything outside of the Data Center, these hosts will also be able to connect to anything outside of the Data Center (examples: Web Servers, Domain Controllers, External File Servers). In the Secure Zone will be machines that provide services only to other machines in the Data Center, these machines also will be able to make connections to machines outside the Data Center (examples: proxied Web Application Servers, Data Center internal SMTP servers, authentication servers). And in the Ultra-Secure zone we will house those machines that only accept connections from machines in the Secure zone, and they will not be able to make outgoing connections. As well as this segregation by security zone, there will also be group and function separation so that different groups will not be able to access each others servers unless specifically authorized.
Network Access
The security policy for the Semi-secured zone will only allow access to those hosts and ports that are explicitly stated to have a need to be accessed from outside of the Data Center, and only from specified locations (the world, Stanford, School of Medicine, or particular IP addresses for example). There will be some very limited access to machines in the Secured zone from non-authenticated external connections, but only those absolutely necessary for the functioning of the machines (Kerberos, for example).
All other remote network access to the Data Center will be VPN connections through the firewall. All VPN connections will be logged, and will only allow the VPN user access to those portions of the Data Center that they are authorized to be accessing.
Physical Access
The Data Center is intended as a limited physical access location for servers to reside . There will be remote network access to the servers via VPN for system administration access, and for other user access (researchers or programmers for example). Systems Administrators of machines which are housed in the Data Center will have access to tools to assist in the remote administration of their servers, however they should plan their servers as if they will only get physical access to them when it is necessary to perform hardware modifications or replacements. With this in mind, it is highly recommended that all servers be configured with administrative tools such as ssh servers, Windows Terminal Services, or Apple Remote Desktop in order to allow systems administrators to remotely maintain their servers. If you are unsure if you have these sorts of things set up for a server, a good test is to unplug the monitor or serial console (if either are present) and see if you can do all of your administration without needing to reconnect them.
For those instances when console access is absolutely necessary (machine is hung during boot, actions must be performed while the machine is in a non-networking state, or other similar situations) we will have a remote KVM over IP solution for which there is a windows client that will allow one to interact with the console of servers located in the Data Center. The same system will also enable an administrator to remotely control the outlets to which a device is connected. There are physical limitations on how many simultaneous users this system will support, and we will expect that this will be used as the least favorable way of connecting with machines.
Production Web Environment Policies:
Access Control Policies
Who should have access
What access should they have
When should they have access
What is the change process for altering who has access and when
All administrative and content-altering methods of accessing the production web environment will be done with uniquely identifiable and secured communication methods.
Security Policies
What technologies are considered safe
What we do to keep particular documents safe
Secure access
Security problem response process
Quality Assurance Policies
We will monitor the services with BB for local monitoring, we will also monitor some services with an external monitor like redalert or fresh
How do we let our customers see the content they are going to post before approving it
Tablespace Policies
Tablespace will be created for each application running on the IRT-APP servers as needed by the application.
Access tokens for the tablespaces will be solely maintained by the Data Center Staff.
Access to the tablespaces will only be granted in manners which maintain the password integrity of the databases. For example, on the app servers, the programmers will not know what the database connection looks like, because it will be in the config file not in their application. The same will be expected of any other application or process that needs access to the tablespaces.
Access to the databases tablespace will only be granted to machines which reside in the secure zone of the Data Center
To get tablespace assigned, put in a helpSU request along with what needs the access and for what business purpose.
