Built in the beginning of 1970's, the Computer Centre air-conditioning and cooling systems were designed to be modular and easily adaptable to the unpredictable future needs of computing. The infrastructure of LHC-computing that will be housed in the existing Computer Room with its five Computing farms and over 11000 PC's increases the requirements of cooling and air-conditioning power to a new level. The nominal thermal loads from the equipment rise from the current design maximum of 1MW to estimated maximum of 2MW in the future. This paper presents calculations and proposes solutions to meet the new nominal requirements. The air-conditioning system must also be able to cope with a situation of power cut in the main supply. A calculation of the temperature evolution during the power cut and a justified operation strategy for this scenario is also presented

Lindroos, J

Peón-Hernández, G

CERN Document Server

EUROPEAN ORGANIZATION FOR NUCLEAR RESEARCHORGANISATION EUROPÉENNE POUR LA RECHERCHE NUCLÉAIRECERN - ST DivisionPresented at the 4th ST WorkshopChamonix, France, 30 January - 2 February 2001CERN-ST-2001-00530 January 2001UPGRADING OF THE AIR-CONDITIONING OF THE COMPUTER ROOM IN THECOMPUTER CENTRE FOR THE LHC ERAJ. Lindroos and G. PeónAbstractBuilt in the beginning of 1970’s, the Computer Centre air-conditioning and cooling systemswere designed to be modular and easily adaptable to the unpredictable future needs ofcomputing. The infrastructure of LHC-computing that will be housed in the existingComputer Room with its five Computing farms and over 11000 PC’s increases therequirements of cooling and air-conditioning power to a new level. The nominal thermal loadsfrom the equipment rise from the current design maximum of 1MW to estimated maximum of2MW in the future. This paper presents calculations and proposes solutions to meet the newnominal requirements. The air-conditioning system must also be able to cope with a situationof power cut in the main supply. A calculation of the temperature evolution during the powercut and a justified operation strategy for this scenario is also presented.21 INTRODUCTIONThis paper presents the main modifications needed to the existing air-conditioning system of theComputer Room in the Computer Centre to maintain the present ambient conditions, when the heatdissipation from the equipment be 2MW. This is the announced heat load after year 2005, the start ofLHC computing. The temperature conditions in the Computer Room in steady state and power cutsituations are also presented with conclusions and propositions for further studies.1.1 HistoryThe construction of the Computer Centre was started end of 1960’s and the installation and start-up ofthe air-conditioning system was done in the beginning of 1970’s when the operation was moved frombuilding 510 to this new building 513. The requirements for air-conditioning were rigorous. Thesystem was to be designed and constructed in a manner that allowed any kind of data handlingequipment to be installed without needing major modification on the air-conditioning system. Thismeans for example that the air could be supplied both from the ceiling and from the false floor. Thefalse floor option has not been used up to this day but it can prove to be useful in the future. Freecooling was one of the innovative aspects of the system. Although it was not very much used at thattime, it has shown to be very energy efficient as the chillers can be stopped for long periods of time.The thermal loads in the old computer centre in building 510 were measured and a thermal loadtwice the measured one was used as the baseline for the design. In thirty years, there has not been anyproblem with the cooling power of the system. Some of the chiller's power has later been taken forother installations than the Computer Room. After the reception of the installation, the ComputerCentre air-conditioning was proposed as a reference for future installations on computer rooms.1.2 The Existing Air-conditioning SystemThe Computer Centre air-conditioning was originally designed to cover the 2’080 m2 Computer Roomand annexed office blocks. Today, the Computer Room is divided by a wall and only 1’440 m2 isoccupied by computers. The rest is occupied by office barracks (Barn area) built inside the hall.Altogether 10 stations placed on the roof level serve the Computer Room and the annexed officeblocks. The stations have functions for filtering, cooling (all stations), heating (two stations) andhumidification. A maximum of 8 stations is allowed to operate simultaneously, 2 being in redundancy.The current nominal cooling power is 1,3 MW for the Computer Room (1.05 MW for the equipmentand 0.25 MW from lighting and external loads) and 0.55 MW for the office blocks. The total coolingpower is therefore 1,85MW. With all the system overheads (fresh air cooling, - fan motors etc.), thenominal cooling requirements account for 2.4 MW. However, at present, only 4 or 5 stations operate atthe same time due to the low thermal load in the Computer Room (estimated load in the room 0.5MW).Figure 1: Cooling power distribution in the Computer CentreTwo out of three 1.2 MW chillers installed in parallel produce the chilled water needed by thestations. The third one is on redundancy.3The stations deliver air at constant temperature of 14°C to the common concrete plenum. Fromhere, the air goes to the Computer Room via steel ducts and supply air plenums. The exhaust air issucked from above the false ceiling. The 14 ambient air temperature probes situated in different zonesof the Computer Room control 119 motorized regulation dampers in the supply air ducts to keep thetemperature at a given set point value.Normally the ventilation operates on free-cooling from autumn to spring, however some usersneed cooling also during this period and for them there is a smaller chiller of 200kW dedicated for thispurpose.The three cooling towers located on the roof have been renewed in 1998. They serve thecondensers of the chillers and have a total capacity of 4,5MW.Figure 2: Scheme of the existing chilled water network1.3 Air-conditioning in the LHC EraTo meet the cooling requirements of the LHC computing, modifications to the existing system andsome new installations are needed. To cover the additional need for space for the equipment, the barnarea is transformed back to a computer room.To supply the needed air flow to the Computer Room, all annexed spaces must be disconnectedfrom the system. This way all the stations are available for the Computer Room and the maximum airflow can be obtained. Maximum nine stations out of the 10 would be operating simultaneously,leaving 1 station for redundancy.The existing chilled water capacity is not sufficient to cover the future needs. One chiller of1,2MW has to be added to supply the needed chilled water cooling to the stations and to cover theneeds of the other users also.The installed cooling towers however can provide enough cooling for the chillers in these newconditions.3x COOLING TOWER 1,5MW1,2MW 1,2MW 1,2MWOTHERUSERS10x AIR-CONDITIONING STATIONCHILLEDWATER30m34Figure 3: Scheme of the air conditioning system in the Computer RoomTable 1Comparison table between present and future installationValue Today LHC eraArea 1440m2 2000m2Volume 8640m3 12000m3Max. number of stations operational 8 (+2 in redundancy) 9 (+1 in redundancy)Air flow per station 85 000 m3/h 85 000 m3/hMaximum air flow 680 000m3/h 765 000m3/hFlow to Computer Room (inc. The Barn) 430 000m3/h 765 000m3/hFlow to office blocks 250 000m3/h -Fresh air flow 25 000m3/h 6 000m3/hMaximum heat dissipation from Eq. 1,05MW 2MWTotal dissipation in the Computer Room 1,3MW 2,25MWTotal cooling in nominal conditions 2,4MW 2,7MWTemperature difference 14/22°C (dT 8°C) 14/23°C (dT 9°C)Chilled water capacity 2,4MW 3,6MWCooling tower capacity 4,5MW 4,5MWElectrical power supply 1500kW 1800kWSecured power supply 22kW 500kW2 STEADY STATE CASE STUDYIn the steady state situation the total heat dissipation from the equipment is 2MW and the 9 air-conditioning stations are delivering 765000m3/h of air at temperature of 14°C.52.1 Presentation of the ModelIn order to estimate the temperature around the computers, a model of the Computer Room has beenmade. The computers are housed in 400 racks arranged in a repetitive manner. Taking profit of therepetitive pattern and to reduce computing time, only the simplest repetitive unit has been modeled. Itconsists of half a rack that contains four layers of computers. To better remove the heat generated inthe computers, the computers are equipped with fans. The air passage through the computers isrepresented in green in the left figure of Fig. 3. The four planes in the XZ plane of the right figure ofFig. 3, represent an hypothetical heat dissipating surface of the computers. The red arrows representthe air supply through a grill and the green surface represent the exhaust plane. All the four limitingplanes of the model have symmetry boundary conditions applied.Figure 4: Model geometry and boundary conditions2.2 ResultsThe most interesting results for our study is the temperature around the computer area. Fig. 4 showsthe temperature map on the YZ plane at the central point of a rack. It is observed that the highesttemperature at the entrance of the computer's ventilation system is about 29oC. The hottest computersare those further from the ventilation. Fig. 5 illustrates the air speed provoked by the ventilationsystem inside the computers.6Figure 5: Temperature map at the centre of a rackFigure 6: Air velocity provoked by the ventilation system inside the computers73 POWER CUT OPERATION STUDYIn case of a general power cut in CERN, the first diesel set starts to supply the network after a switchover time of 45 seconds. In this first phase only the safety equipment will be supplied. The second setwill start supplying other connected equipment after about 1.5 minutes. The air-conditioningequipment in the Computer Centre will be powered when the two diesel generator sets are inoperation.3.1 System Operation During the Power CutIn a power cut situation, the UPS keeps supplying the computing equipment in the Computer Room.The secured network can not power all the air conditioning equipment because the power of the dieselis limited. To keep the required electrical power as low as possible, only the fans and pumps will bepowered. This will require some 500kW of electrical power. Since there is normally a delay of 1.5minutes. (it could be in the worst case 3 minutes) before the start of the second diesel set, theComputer Room is without any cooling during this period and the temperature rises rapidly. After theswitch over time, the diesel starts to power the fans in the stations and the chilled water pumps. A30m3 chilled water tank is connected in series with the pumps and the stations. The warm airexchanges heat with the water circulating in the cooling coil and, since the chillers are not running, thetemperature in the tank rises and so does the air pulsed to the Computer Room.3.2 ResultsThe starting room temperature for the calculation is 22°C (set point). The calculations show that thetemperature will be about 25°C after 1.5 minutes without air-conditioning. If at this time the fans andpumps are powered, after 10 minutes the temperature will be 30°C and after 30 minutes, which isthought to be the worst case scenario, the temperature will be 42°C.In the calculation it has been considered that part of the heat is stored in the materials, thecomputers and the metallic racks.Figure 7: Temperature evolution with different switch over times4 CONCLUSIONSThe study shows that the increased thermal loads in the Computer Room from the LHC computingcould be handled with the existing system when the presented modifications, new installations andsystem readjusting are done. However, the system will run at its limit and the redundancy is reduced.The temperature distribution in the room is not satisfactory. The calculations show thattemperatures are too high in the occupied zone. Further studies are needed to find solutions on how todistribute the air in a better way.2025303540450 5 10 15 20 25 30time (min)T(oC)T (oC) no delayT (oC) 1.5 min delayT (oC) 3 min delay8The reduced cooling capacity during a power cut results in a maximum temperature that is abovethe acceptable temperature limit (less than 35°C) in the worst case scenario (30 minute power cut). Ifthe autonomy of the UPS is less than 20 minutes, the temperature will stay below the targettemperature and only the minimum of equipment needs to be powered by the secured supply. Thevalues given by the manufacturers on the acceptable speed of temperature rise however is exceeded(max. 5K/h).5 DISCUSSION AND FURTHER STEPSTo resolve the problem of air distribution in the Computer Room further calculations and studies areneeded. The problem comes from the fact that the supplied air does not penetrate deep enough in to theroom to mix the air in the occupied level. A study using jet type air diffusers should be done to seewhat would be the temperatures with this kind of system. One approach is to introduce part of the airfrom the false floor, which is a method largely used in computer rooms. This would be easy toimplement since some ducting exists already connecting the supply air plenum and the false floor.Once the UPS solution and autonomy time has been defined, it is possible to study further howto provide more cooling power in a case where the autonomy of the UPS exceeds the 20 minute criticalpoint. There exist at least two possibilities to add redundant cooling power. Firstly, one chiller couldbe connected to the secured power network. Secondly, an interconnection with another chilled waternetwork would add the volume of chilled water available. This connection could be done to thenetworks of the Booster/PS.When the use of the Computer Room and the equipment to be installed is better defined, morestudies and calculations will be performed to find the adequate design and operation strategy.REFERENCES[1] ASHRAE Handbooks[2] A. Funken, Les réseaux eléctriques de sécurité du CERN, CERN-ST-2000-036

Upgrading of the Air-conditioning of the Computer Room in the Computer Centre for the LHC era

Abstract

Similar works

Full text

Available Versions

CERN Document Server