The challenges of developing telecom infrastructure have been a topicof keen discussion for the past decade ” increasingly complexapplications and higher bandwidth requirements coupled with shorterproduct lifecycles and smaller development teams have pushed thetelecom industry to the brink.
For as long as these challenges have been part of the industrydialogue, the commercial off-the-shelf (COTS) model has been touted asthe solution. If the problem is smaller development teams, buyingblades makes intuitive sense ” less development work should mean that asmaller team can successfully build a system from off-the-shelfcomponents. If the problem is meeting stringent time-to-marketrequirements, again, it follows that buying blades should acceleratethe overall product schedule as the blades are, in theory, fullydebugged and ready to deploy into the application.
The COTS model is built on the assumption that development isdifficult and integration is easy. Given this assumption, the COTSmodel makes sense ” outsource the development of blades, chassis, andsoftware components to a set of vendors that specialize in thecomponents, and do a “quick” integration of the selected pieces. Thismodel is appealing to system architects who like to choose from a widevariety of blade vendors and look for the best-of-breed on eachcomponent, and appeals to the executive sponsors who like the idea ofbuilding products without all those pesky development engineers.
However, the core assumption of COTS is wrong ” development isstraight-forward and predictable, while integration is messy,time-consuming, and highly unpredictable. This is the dirty littlesecret of COTS that has resulted in the delay or cancellation of manyprojects.
The point of integration is to ensure the various components worktogether and to resolve the incompatibilities. Until the integrationtask starts, it is not known whether any issues that surface will beminor and easily addressed or whether major issues will arise, therebyrequiring substantial redesign, workarounds, or even re-opening thecomponent selection process.
When building a project schedule, it is difficult to includeintegration activities given the huge range of possible durations. Forthis reason, many project managers include it in the schedule with ashort duration and either keep their fingers crossed that any issueswill be quickly and easily resolved ” or exclude integration from theschedule entirely. After all, if all of the components arestandards-compliant, there shouldn't be any issues, right?
Experienced integration engineers know that solving integrationproblems frequently involves getting detailed technical support fromthe designers of the components and often requires resolving differinginterpretations between the developers of various components in thesystem.
This challenge often exists even when the integration engineer is inthe same company as the development engineers, because the COTS modeldivides engineering teams in exactly the wrong place and separatesintegration engineers and development engineers into differentcompanies rather than housing them under one roof with a common set ofpriorities.
In this case, it can be impossible to get a quick answer from thevendor company, and the ability to share information may be hampered bycorporate IP protection policies, differing project priorities betweenthe two companies, or in a many cases, a desire to “protect” thedevelopment engineers from the customers and shield them behind layersof field engineers and customer support systems.
Finally, the COTS model often puts system integrators at crosspurposes with the blade vendors. During the integration process, it isoften the case that the best way to resolve an integration issue, or tomake the platform more consistent and “user friendly”, is to make achange to the underlying product, either to change some hardwareimplementation or add some software feature.
This is in direct conflict with the COTS model of aiming to sell asingle standard product to multiple customers ” because as soon ascustomization is required, many of the benefits of COTS start todisappear.
Mindful of these challenges, the largest telecom equipmentmanufacturers (TEMs) have embraced COTS in a very specific fashion thathighlights the challenges they see in COTS and provides an instructionmanual for smaller vendors (Figure 1 below).
The major TEMs have created common platform teams that buildapplication-ready platforms using blades sourced both from third-partyvendors and internal engineering groups.
|Figure1. Evolving TEM development approach|
These common platform teams shield the TEM application teams fromthe pains of COTS integration, and they update the common platform onregular intervals with releases that incorporate new blades and newsoftware functionality. These releases are put through a suite ofintegration testing that is intended to wring out any incompatibilitiesbetween blades, BIOS versions, and software components.
In addition, the common platform team often develops layers ofsoftware that provide services such as diagnostics, bladeconfiguration, firmware updating, and software distribution. Theseservices allow the application teams to quickly port new applicationsonto the platform, and the overall approach is to simplify the tasksfor the application teams, allowing them to focus on the needs of theapplication and not on the nuances of low level blade and shelf managerinteractions.
It is interesting to contrast this approach with the approach toCOTS attempted by many startup TEMs. Most telecom startups have somenew application in mind, and have a small group of developers with deepexpertise in that application and the end market.
Mindful of the cost and time-to-market impacts of developing aproprietary platform, they embrace open COTS platforms and buy into therhetoric suggesting that all the components will interoperate withoutissue and that the bulk of their efforts can focus on selecting thebest-of-breed hardware and software components, quickly integrate them,and then work on the application. Many of these companies assign oneperson as the “integration engineer”, or even worse, assume that theapplication developers can do it in their spare time.
The reality is often sadly different ” the integration effort soaksup more and more time and effort from the engineers working on it,diverting resources from the and the effort quickly expands to includetracking action items with multiple vendors and managing a scheduledependent on four or five separate vendors, each of whom announces aslip every month or two, leading to schedule juggling on a weeklybasis.
Learning from the largest equipment vendors, one solution is tobuild an internal platform team, staffed with strong architects who cansort out the intricacies of how the platform should work, and equippedwith strong vendor management skills. However, most small TEMs can'tafford a common platform team with 10 or 20 people, let alone the 100+found in the largest equipment manufacturers. Where does that leavethem?
These industry challenges aren't magically going to vanish ” therequirements to get complex applications into the market quickly withminimal development teams are real requirements and returning to themodel of developing the project completely in-house isn't a practicalsolution.
Fortunately, a solution is now available in the form of pre-integrated systemsthat include hardware, software, and reference applications.
These fully-integrated systems include a set of components that havebeen tested together and tuned to operate as a system, as well as a setof value-add software that includes platform management, highavailability middleware, unified management, and protocol stacks. Ablock diagram for a representative pre-integrated system is shownFigure 2 below.
|Figure2. FlexTCA Fully Integrated System|
Pre-integrated systems offer the best time-to-market available. As arule of thumb, applications built on pre-integrated systems tend toreach the market within a year, while systems built on COTS componentsoften 24 months and proprietary platforms as much as four years.
Pre-integrated systems have been called the third phase of telecominfrastructure development. Pre-integrated systems based on openstandards leverage the benefits of COTS, but offer a compellingsolution to the challenges posed by the COTS model. Pre-integratedsystems removes the risk of integration and schedule uncertainty thatplagues projects using the COTS model, while delivering on theaccelerated time-to-market and reduced development costs that pushedcustomers to the COTS model in the first place. Given these advantages,it looks like the third phase of telecom is here to stay.
Michael Coward, is the CTO and co-founder of ContinuousComputing. Mr. Coward specializes in system architecture and thedesign of highly available redundant platforms, including the creationof the company's Ethernet-HA architecture which replaces the PCI buswith redundant Ethernet links and allows for the creation of highlyscalable, distributed systems with superior redundancy and resiliency.Mr. Coward has long experience in the telecommunications industry. Heholds an M.S. in electrical engineering from the California Instituteof Technology.