From Global Knowledge Clickbait to Discourse on Cisco's UC Domination

Bopping around on Twitter Monday, I ran across this tweet by Global Knowledge:

How #Cisco Unified Communications Manager carved a dominant presence in the communications landscape http://t.co/ctMrRDfMkU #cucm
— Global Knowledge (@GlobalKnowledge) November 4, 2013

I figured I'd bite, but quickly determined that this whitepaper does NOT tell you how Cisco carved anything in anything.  It's just an overview of the CUCM solution.  This got me thinking it would be fun (Am I weird? Don't answer that.) to write a post that took into account all the things that made Cisco the leader in the Unified Communications space. So here goes my attempt at explaining the story.

Back in the day, you had the PBX.  And the PBX was a MONSTER.  It was complicated to manage, really expensive (both up-front and for recurring maintenance), and inflexible as all-get-out.  Initially, PBX services only augmented services provided by the PSTN, namely dial-tone.  Then came voicemail solutions, and ACD solutions, and ring-downs, and you get the idea..  By the time the mid-90's rolled around computer networks, or Local Area Networks (LANs),  were becoming more mainstream and leading the vendor pack was Cisco.  Cisco was founded in 1984 as a network vendor that sold routers, used for connecting disparate network segments (and sometimes disparate protocols) together, over various transportation mediums.  These networks could be local on the LAN, or remote via circuits provided by the telcos.  In, 1993, Cisco entered the LAN switching market by acquiring Crescendo Communications. Over the next few years, Cisco rounded out the LAN switching porfolio by acquiring Kalpana and Grand Junction Networks.  With this dominance in computer networking, Cisco began to earn a loyal IT following because their solutions JUST WORKED. 

In addition to the Crescendo acquistion, another interesting thing happened in 1993...the creation of the Cisco Certified Internetworking Expert (CCIE) certification.  Thus began Cisco's dominance in another area: mindshare in being THE certification for the networking industry.  Cisco certifications are valued even by Cisco competitors, such as Juniper and Brocade.  Most IT departments view Cisco certifications as a requirement for positions ranging from Help Desk Technicians to Network Engineers.  By creating their certification program early-on, they got the jump on other possible industry giants that might have wanted to create as reputable a certification portfolio.

During the mid-90's, the Internet Protocol (IP) was becoming the de facto standard, and with it more and more applications as well as traditionally isolated solutions were being ported to run over IP-based networks.  One of those solutions was the PBX.  Now in most companies the PBX was owned and operated by the facilities department.  Because the PBX was a huge capital investment, akin to things like real estate and furniture, it made sense for the onus of the PBX to be on facilities.  Keep that in the back of your mind for a bit, we'll come back to it.  The first IP-based PBX was Selsius System's CallManager.  Selsius was formed in 1997, out of another company called Intecom.  Cisco took note of what Selsius was doing with IP-based voice, and decided to make an offer for them.  They successfully beat out Nortel for the bid, and acquired Selsius in November of 1998.  Oh how things would have been different if Nortel had won that bid....  If you want to read more on some history here with Selsius, check out this old post by Mark Nelson, a Cisco employee.

With IP-based solutions, the IT department started to own more and more corporate infrastructure.  Network engineering jobs saw an increase in demand, which required people with Cisco certifications like the CCNA, CCNP, and CCIE.  Therefore, Cisco's network market share increased because these certified folks wanted to work on the same gear they studied for.  See what they did there?

After the acquisition of Selsius, Cisco spent two years getting the product ready for prime-time.  In 2000, Cisco released version 3.0 of Cisco CallManager, which was the first Cisco branded release of the product.  Remember the facilities department and their grasp on the PBX?  Well, CallManager was pitched to IT departments, specifically NETWORK ENGINEERS.  This effectively supplanted the facilities department's long-held PBX vendor relationships.  Well played, Cisco, well played.   This was really a perfect storm.  Cisco was the IT department's friend, because their stuff "just worked".  Ever hear the phrase "Nobody ever got fired for buying x", where x equals a big IT vendor.  Well, because the IT department was being handed more and more solutions, it made sense for them to take the least risky path.  Even with the bugs that plagued those early CallManager releases, Cisco's world class support was there to allow those customers to rest easily knowing that they were in good hands.  This path of least risk is further proven by this brand survey, which puts Cisco at the top of the heap.  The author specifically calls out risk aversion as one of the main reasons for Cisco's leadership.

A strong component of the Cisco strategy has always been to sell and deliver solutions through the use of the Value Added Reseller channel.  The channel acts as an extension to Cisco's sales and engineering staff, without incurring direct expenses associated with the relationships.  Cisco certifications also play heavy into the channel space, because in order to be able to sell certain Cisco products, the Cisco VAR (also referred to as Cisco Partner) must have engineering talent on staff for the products in question.  Cisco Partners often sell other technology solutions as well, including those made by Microsoft, HP, IBM, etc.  This means that oftentimes, the partner sales rep has relationships with accounts that Cisco's own sales team might not have.  Talk about hedging your bets.  This channel delivery model has without a doubt been one of the key components involved with Cisco's dominance in various markets, including Unified Communications.

Oh, there's one more thing that Cisco did to help solidify their position as a market leader. Ever watch an episode of 24 and noticed a Cisco IP Phone? What better way to subtly influence a society full of TV-watchers than to put your products in the middle of TV shows! Genius! This started back in 2005, which was still relatively early for Cisco's UC portfolio.  But it got the image of those phones out there, you know, saving the world and stuff.

Everybody got the recipe for UC success?

Build a brand on a reliable premise, such as the plumbing of your corporate IT infrastructure.  Create a "club" mentality with certifications that aren't that easy to obtain, and that are respected by IT departments.  Encourage people to get said certifications because they will get you network engineering jobs.  Buy a VoIP company.  Stir new and old product offerings until well mixed.  Leverage partners for extending sales and engineering teams. Sell to your certified network engineers in IT departments everywhere. Make tons of money.  Sounds easy, eh?  Thanks for reading!

Some Cisco UC Basics, part three

In the olden days (read: 3 years ago), Cisco's desktop IM/Softphone client combo was called Cisco Unified Personal Communicator (CUPC). The experience was less than stellar.  In addition to CUPC, Cisco had an attempt at a softphone on iOS and Android called Cisco Mobile.  Nowadays, the desktop and mobile device landscape for Cisco entirely centers around a client they call Jabber.  You might remember a few years ago that Cisco bought Jabber, Inc (the folks behind the XMPP protocol).  It's only been in the last couple of years that Cisco has visibly capitalized on this investment, which has basically entailed a major re-branding and consolidation of CUPC and Cisco Mobile.  Jabber works on Windows, Mac, Android, and iOS (no love for Linux, despite a ton of people asking for it, and the irony that all of the UC appliances run on top of Linux).  Do not be deceived, these Jabber clients are in no way the same code base.  A major shortcoming of Jabber to date has been a lack of feature parity between platforms.  That's getting better, but still has a ways to go until the features on Jabber for Mac are the same as Jabber for Android.  Another change in the move to Jabber is that Cisco switched from the presence protocol of choice being SIMPLE to it being XMPP.  That's a little ironic because Jonathan Rosenberg, the creator of SIMPLE actually works for Cisco.

The most exciting thing that Cisco has done in the last few years is purchase Tandberg. Which, as a side note, does give a clue to Cisco's primary innovation mechanism: acquisitions.  Tandberg had really perfected video conferencing through the firewall through the use of the H.460 protocol, which is H.323's solution for NAT traversal.  Basically Tandberg had these two appliances, called Video Communications Server (VCS), one sitting inside the firewall, one outside the firewall or in the DMZ.  The one internal is constantly connecting to the outside box over TCP asking if it has any calls for the internal endpoints.  If it does, then media and signaling ports are opened up by the internal box as requests to the external one, and magically the firewall doesn't have to have a bunch of pin holes added from the outside in (except when the outside VCS is sitting in the DMZ, where you'll need signaling and media ports opened from the outside to the DMZ).  Tandberg adapted this to work with SIP as well, so B2B video "just worked".  You would still have STUN, TURN, and ICE on the outside for endpoint purposes, but NAT traversal at the head-end works through this pairing of appliances inside and outside of the firewall.  Fast forward to now, and this coupling of appliances is going to be a major strategy for Cisco in providing VPN-less connectivity for the Jabber client, but also B2B audio and video calling.  If you're a burgeoning UC admin, I highly recommend getting familiar with the concept of SIP DNS SRV records and SIP URI's, because it's going to become more significant in future UC deployments, regardless of if Telepresence is in use or not.

The Tandberg acquisition happened around the same time that Cisco purchased another company you may or may not have heard of before: WebEx.  WebEx had created an industry all on its own around web conferencing, and with Cisco's acquisition, everyone stood around and scratched their head trying to figure out what Cisco's play was going to be.  So we waited.  And waited.  And then finally something interesting happened...Cisco created an integration option with their traditional conferencing platform, known as MeetingPlace, and WebEx for scheduling and sharing of web content.  MeetingPlace was a monolithic platform that required multiple nodes for audio, video, and web conferencing, as well as separate nodes for web-based scheduling.  MeetingPlace would integrate with CUCM via either H.323 or SIP.  So Cisco's integration between MeetingPlace and WebEx worked OK, but felt like an awkward blind date.  The integration was a little brittle, and the administrative interfaces didn't look anything alike, so it was a bit of a learning curve to juggle the two of them.  A little over a year ago, Cisco announced something that surprised a lot of people: Cisco's WebEx in an on-premise package called Cisco WebEx Meetings Server (CWMS).  This also marked a lack of announcement for another release of MeetingPlace, so the writing was on the wall that Cisco was going to sunset the MeetingPlace product in favor of WebEx.

CWMS is an exciting premise, because it allows for a customer to have a full copy of the WebEx platform hosted on their own servers.  The scalability ranges from 50 concurrent users all the way up to 2,000.  A downside to "on-prem WebEx" is that a lot of compute power is required to run even the basic 50 user configuration.  Also, the solution requires that VMware vCenter is deployed, which can add some complexity to the environment.  The full CWMS solution consists of an Internet Reverse Proxy (IRP) virtual machine, an Admin virtual machine, and for larger implementations (250+ users), a separate Media virtual machine.  For the smallest size installation, a single physical host can handle an instance of CWMS with an IRP and Admin VM.  To make it redundant, you add a second physical host.  As you scale up, more physical hosts are required because the processor and memory requirements grow quite a bit.  At the 800 user mark, the total number of cores required for a NON-REDUNDANT configuration is 80, and the amount of memory required is 116 GB.  At the 2,000 user mark, there's an additional virtual machine type for web conferencing only, and you are required to have 3 media virtual machines.  Total number of cores for a non-redundant configuration is 160, and total amount of RAM is 276 GB.  Ouch.

If you want to connect your video conferencing solution to WebEx, you might want to sit down.  WebEx was a proprietary platform when Cisco bought them.  So proprietary, in fact, that it took Cisco 5 years to announce that they were going to allow integration between their Telepresence portfolio and WebEx.  The integration was named WebEx Enabled Telepresence (WET), and requires a Tandberg Expressway/Control VCS pairing, DNS SRV, the Telepresence Management Suite (TMS) product, a Multipoint Control Unit (MCU), and some endpoints.  The downside to this solution is that you'll likely need to hold an IPO just to afford everything.  To setup a meeting, the conferencing is scheduled either through TMS or WebEx, and when it's time for the meeting all the local participants will connect to the MCU as well as the WebEx "cloud".  Everyone can see everyone else, however to those on-premise folks, the WebEx video will appear as a single feed and vice versa for WebEx participants.  The experience can be a little frustrating if you have a large number of people on WebEx video.  This solution DOES NOT work with the on-premise CWMS.  At least not yet.  

I hope this has been helpful to give you a snapshot of some current Cisco UC basics.  Let me know if there's anything you'd like to read about that has puzzled you, and I'll do my best to break it down.  Thanks for reading!

Some Cisco UC Basics, part Deux

Cisco's call control product is UC Manager, which used to be named CallManager, and gets referred to as Communications Manager or CUCM.  It's designed to be scalable, potentially up to 10,000 endpoints registered per node.  The per-node scalability is determined by the VMWare OVA template that gets applied in the initial build-out of the system.  Systems Integrators will choose the correct OVA by evaluating the current number of endpoints in the environment plus a target growth number, which normally is provided by the customer.  The reason not every Systems Integrator goes right to the 10,000 user OVA is that the system specifications for those virtual machines is weighted heavier, therefore available system resources on the physical host are reduced,  which reduces the overall capacity of the physical host to hold other virtual machines.

UC Manager nodes can be clustered together, and that involves a Cisco proprietary protocol that passes call and endpoint state between nodes within the cluster.  This proprietary protocol requires that every node maintain a connection to every other node, so the connections are fully-meshed.  This impacts bandwidth required between nodes in the cluster, as a single connection requires up to 1.5MB of bandwidth.  This is a big deal if you are planning to cluster nodes over the WAN, and have limited bandwidth to work with.  This protocol allows for the entire cluster to act as one big registrar for endpoints.  Cisco allows for up to 8 call processing nodes in a cluster, and they cap the number of phones per cluster at 40,000. 

In SIP terms, UC Manager is a B2BUA, however media never flows through, it's always between endpoints.  This isn't even an configuration option to change this.  This fact gives a little bit more predictability to the scalability considerations of the system.  Nowadays, SIP is definitely the protocol of choice for CallManager, both line-side to endpoints and trunk side to PSTN gateways, although the protocol used to the gateway can vary depending on what Cisco voice engineer you talk to.

The gateway is almost always a Cisco router.  Voice functionality is built into Cisco's networking OS (IOS), and is enabled by a "UC" license.  The gateway runs one of 4 possible protocols: SCCP (Skinny), MGCP, H.323, or SIP.  The protocol used is usually determined by what the customer is connecting to the gateway.  Analog lines, for instance, usually work better if SCCP or MGCP is used. PRI's are going to entail usage of either MGCP, H.323, or SIP.  If the customer is setting up a SIP trunk to a telco, SIP would be the protocol used from the gateway to CUCM.  MGCP relies on CallManager for call control and dial-plan decisions.  H.323 and SIP are effectively peer-to-peer protocols, meaning there are separate dial-plans and call routing decisions made on both CallManager and the gateway.  Another set of functions that the gateway can provide is transcoding, ad-hoc conferencing, and Media Termination Point (for media relaying and DTMF termination) services.  These services register to CUCM using SCCP, and can easily be co-located with the other protocols.

In the first part of this series, I mentioned that Cisco had chosen IBM Informix for the database component.  The way that CallManager passes configuration data between nodes is through a database Publisher/Subscriber model.  There are a few challenges to this approach in the CallManager space.  The first node installed becomes the Publisher server.  This role is locked in to the first installed node, and cannot be changed later without more or less backing up the Publisher, re-installing, and restoring the backup.  So if you find yourself with a downed Publisher and no backup, you'll have a headache in front of you, because the environment will need to be rebuilt.  The Subscriber nodes will continue functioning, however no configuration changes will be able to be made to the servers without that Publisher.  Cisco has not provided the capability to promote a Subscriber server to a Publisher.  Also, for sometimes unknown reasons (and quite a few known ones), database replication will break.  There's a list of CLI commands to go through when this happens, and that resolves the problem 80% of the time, and the remaining 20% of the time you will be on the phone with Cisco's Technical Assistance Center (TAC).

Stay tuned for part 3, where I'll talk about some of the other Cisco UC applications (client and server).

Some Cisco UC Basics, part 1

In part one of this series, I'm going to explain the overall Cisco UC landscape as it exists today (with a bit of historical information thrown in). 

The Cisco UC Product Suite consists of a slew of different appliance virtual machines, providing functions like traditional call control (dial-tone), voicemail, presence, E911 (specifically, providing more specific location information to the Public Safety Answering Point than just physical address), conferencing (think web-based content sharing and audio bridging), and telepresence (video conferencing).  All of those functions run separately, so there are no co-resident applications on a single virtual server instance.  On the physical host multiple virtual machines are allowed, however over-subscription of resources is not permitted.  For instance, if each virtual machine takes 2 processor cores and 4 GB of RAM, a physical host that has 2 quad core processors (8 total cores) and 32 GB of RAM would have a maximum of 4 virtual machines. Even though there was more RAM available, the physical host ran out of available cores, and thus would not be able to hold additional virtual machines.

All the individual services within the virtual machines run on top of a re-packaged Red Hat Enterprise Linux.   Even though this is the RHEL that a lot of companies know and love, the normal Linux bash shell is not accessible.  Cisco built a CLI wrapper that mimics some of the traditional IOS functionality and allows anadministrator to perform basic OS administrative functions.  This re-packaged RHEL is referred to as the Cisco Telephony OS.  Cisco uses an IBM Informix database for storing all configuration, call states, and Call Detail Records.  Depending on the application, you may or may not be able to directly access the database, and might be limited to an API for accessing and/or modifying relevant information.  The graphical administrative interfaces and web-based API's all run on top of Apache Tomcat.  

As far as scalability goes, Cisco is very rigid in their product system requirements.  This means you can't just go willy nilly and throw a product instance on just any old server or hypervisor.  The general rule of thumb is that their products should be on UCS servers, virtualized on VMWare 5.x, and if there is any SAN connected to the servers it should be connected via Fibre Channel.  However they will allow customers to go outside of these requirements to some degree.  You may use any brand server you wanted as long as the hardware specs meet the requirements; and you also may use iSCSI or NFS for storage. The caveat is that the level of support you would get from Cisco  would not be as good as it would be if you had stayed on UCS servers and fibre channel or direct attached storage. 

At first, customers didn't have the option to virtualize, so every appliance would run on bare metal machines that were OEM'ed from HP or IBM by Cisco.  Version 8.0 introduced the option for doing the hardware abstraction, and now virtualization is going to be the only way these appliances are supported in the upcoming 10.0 release.  There have been rumors that Cisco would open up the hypervisor requirement to use something else.  Personally, I'm hoping for some KVM support, because today VMWare is just another thing that customers have to purchase.

Thanks for reading, and stay tuned for part 2!