With the growth of the Internet for personal use (e.g. Facebook, Amazon, Google, Gmail) and business purposes (i.e.file storage, web applications, collaboration and communication, VOIP) I thought it would be useful to talk about what actually powers all these things. I have a secondary reason for this too – when a non-technical person asks me what I do for a living, I have yet to come up with a short simple answer that actually explains it!
First of all, let me define what I mean by ‘Internet Infrastructure’. All the hardware and services required to make this web page appear in your browser, or an RSS feed download into your reader, or VOIP calls / emails get to your desktop. All the underlying technologies that are unseen, but ‘make the Internet go’.
I see Internet Infrastructure consisting of a ‘Top 5’ areas :
- Data Centres
- Network Connectivity
- Computer Equipment
- Storage Services
- Server Applications
A Data Centre is basically a specialist building that has the ability to power (and cool) massive amounts of computer equipment. Typically a Data Centre would also have a very large amount of network bandwidth to accommodate data transfer in and out of it. Data Centres are built as highly redundant and resilient facilities – at the base level – you would expect a Data Centre to have at least N+1 power (this likely comes as a local feed from the national electrical grid as ‘N’, and a backup generator for the ‘+1’).
The Data Centre is the home for Internet Infrastructure. It is the central point of aggregation and distribution of data and network services. These facilities tend to include:
– 24 x 7 Staffed Operations Centre (typically called a NOC, the staff monitor all activities of the Data Centre and ensure smooth operation as well as deal with equipment issues)
– Building Management System (the BMS normally monitors and alerts on temperature zones, power and cooling usage, outside temp., access control and CCTV)
– Secure Access Controls (i.e biometrics on all entry and DC floor doors)
– Fire Alarm and Suppression (ie. VESDA for detection and Inergen gas for suppression)
The unit of measurement for a Data Centre is space and power. How much space will the equipment require and how much power will it draw (which is effectively double that, as cooling a server takes about as much power as just having the device operating).
Possibly to most important foundation block of Internet Infrastructure is the Network. Without a network connection no data can pass between Data Centres, over the Internet, and ultimately onto your Desktop, Laptop or Mobile Handset. For the purpose of this post, let’s talk about the network infrastructure in a Data Centre, where data passed in to computer equipment, is processed and/or stored, and passed back out of the DC.
So you would expect at least N+1 network connectivity into a Data Centre in the form of at least 2 Fibre Cables from telecommunications providers on diverse rings. Therefore if one had service cut, the Data Centre’s network connection would not be affected. Some data centres (Hosting365’s is one) are Carrier Neutral – which means a number of carriers have a Point-Of-Presence in the facility, so the Data Centre is not affected by any commercial or technical issues of a single carrier.
Next you would expect redundant switch gear in the Data Centre in separate racks so again if the switch gear failed, the other set of it would simply take over and no service interruption would be experienced.
The unit of measurement for network connectivity is megabits per second and available megabits on the carrier connection. There may be 1 Gigabit available but the DC may only be using, and paying for, 100 megabits. The ability to meet peak demand is important though, so Data Centres will have a lot more connectivity available than is required for daily operations.
Now that the two basics of Internet Infrastructure are in place – the ability to power your equipment, and the ability to connect it to the Internet, the next thing is the computer hardware that uses this to process and store the applications and data.
By computer equipment, for this basic post, I really mean Servers. A Server is a more complex and high-end version of a desktop PC. An average server might consist of 2 power supplies (for redundancy), 8-12 RAM slots, anything from 2-10 hard drive bays and multiple processors (not just multi-core!).
Servers are housed in Racks in a DC which are typically 42u in height. (1U is 1-unit and a low-end server takes up just 1 of these units, other servers scale within these racks to multiple ‘U’). Racks are normally powered by 2 PDU (Power Distribution Units) which connect to (if available) multiple power supply units in the server.
A low-end installation may be only a single server, which is the simplest form of Internet Infrastructure. The server would be connected to the DC Power, the Network, an OS and other required applications installed on it. Then it is ready to ‘power and push’ data on the Internet. More complex deployments would include pools of servers, with different applications on each one, or clusters of pools for multiple clusters with dedicated application requirements.
The unit of measure for Servers is Processor Power and RAM. Although there is a lot more to selecting a server such as expandability, reliability, network ports, BUS speed, Cache size and speed. Personally I would like the unit of measure in Servers to change, I think for buyers and users it should be rated in ‘MIPS’ – which is ‘Millions of Instructions Per Second’ which is effectively all that matters, and how today’s Mainframe computers (IBM BlueGene is a high end Mainframe) are measured.
Data Storage is a huge part of Internet Infrastructure. All those emails accessible online, all the web pages on your favourite web site, all those photos on Facebook … are all stored on a hard drive in a DC somewhere. The basic level of storage is on-server storage, which means the hard drives in the computer server. This can cause not just performance and capacity issues, but also redundancy ones – local storage is inherently as prone to failure as the server it is in.
It is common to use specific storage devices – such as Direct Attached Storage (a dedicated and dumb storage appliance connected direct to your server), Network Attached Storage (a storage device that can be accessed by multiple machines over a network connection, and independent of the server itself) and Storage Area Networks, which are high-end, resilient and redundant set-ups that give high performance levels and are very scalable. A Storage Area Network may be shared among many services, applications, servers and customers.
The unit of measure in storage is gigabytes (getting to be more commonly terabytes now) and IO’s per second (input-output read/writes the device can perform per second).
The final piece of underlying Internet Infrastructure is the server applications themselves. In order for an web application to be delivered from a server, that server requires an Operation System (typically Windows or Linux), a Web Server application (like Apache or Microsoft IIS), and a Database (such as MySQL, MS-SQL or Oracle). There any many more variations here, but the basic web server has these 3 things. From here you can install blog software, an ecommerce site, your new web 2.0 application, or any Internet capable piece of software (more include – Instant Messaging Server, File Storage Server, Message Board)
More complex applications tend to have dedicated servers, or pools or servers, for specific things – like a cluster of Database Servers, or a pool of Web Server to serve those ‘www.’ page requests. These may also have more complex network setup such as dedicated routers, load balancing and firewall devices (for traffic management and security respectively).
This post is only scratching the surface and (hopefully!) providing a very basic overview of what Internet Infrastructure constitutes. The actual deliver of power and cooling infrastructure is a very technical field and has professional disciplines dedicated to it. Very much so also on the network side as well as computer equipment (including storage) and applications.