Big Data In Telco

Introduction

It is not the strongest that survives, nor the most intelligent, it is the one which most capable of adapting to change - Charles Darwin

Big Data has been the latest fashionable technology for quite some time now. While its applications and potential applications are no longer a doubtful, what still continues to perplex users is how relevant it can be to their business and more importantly how to conceptualize and build a Big Data stack that is suitable?

From a professional angle, I have been participating in regular engagements with Telco enterprises, trying to find out reasonable ways to push Big Data based solutions to them, and have observed certain key characteristics

ONE. In traditional Telecom the scope of solution is usually big, often complicated by diverse requirements set by the standard bodies and Govt. authorities. This conventionally prevented many players to enter the playing fields and operators were stuck with only a handful of well-known vendors. Growth of interest in Big Data and the technology to process the same transformed the playing fields, because it no longer remained a Telecom problem but became a Computer Science/IT problem like choosing the right Database or Operating System. The Playing Fields were made more flat making entry of several vendors large general purpose one to small niche ones alike

TWO. There is no standard for Big Data. This is a disadvantage as well as an advantage. Disadvantage because solutions need to depend largely on precedence and best practices, and often on experimentation before the right direction can be found. Advantage because it keeps the lines of innovation open and space for both Open Source and Proprietary solutions. This situation also leads to an environment where customers might have any strong keenness for the exact technology as long as some basic expectations are met

a) Cost Effective: In most cases because of lack of a clear and complete use cases, customers will want to reuse their existing hardware, software and human infrastructure. Cloud Hosting – pay as you use – with zero setup expenditure is an inviting option for most. However vendors interested in selling infrastructure often try to deter them by bringing up topics of security and performance. But those are really secondary concern as long as you are not running some time critical business or accidentally leaking key privacy information of your subscribers into the public.

b) Avoid Lock In: This is why Open Source based solutions keep winning. In the Big Data space, Open Source has continued to define the de facto standards and vendors adopting Open Source components in their solution automatically gain some advantage. By remaining compatible with the Open Source distribution in terms of the Interfaces, the Configuration and (though often overlooked) the Data Format, vendors offer an assurance that customers are not locked in and in case they find a strong reason, can always replace the core engines.

c) Mix, Match, Integrate: Customers have more or less agreed that there is no comprehensive Big Data solution and it is always pragmatic to mix and match the most popular and appropriate software for each category (eg. storage, computing, analytics, real time computation, reporting, visualization and so on). It is also unlikely that customers will throw away their previous and time proven investments in similar tools (like BI Tools, Analytics Workbench, Database etc.) and adopt a big bang change. More likely than not, they will think of integrating these tools with a compatible ‘Big Data Core’, where they could retain the same familiarity of older tools, and yet expanding their capability to ‘logically’ crunch larger amounts and variety of data

THREE. Telcos have large amount of data both from machine sources (KPIs from network probes, signaling data, transactional records and so on) and human sources (customer data, interaction data, behavioral information and so on). Some of this information is very unique. For example mobility information (while it is fairly possible to spoof your IP address, it is virtually impossible to anonymize the mobile cell area you are in), customer demographics (age, gender, occupation, country and so on because in most cases Govt. regulations require such information to be furnished to the Telecom operator for protecting both the consumer and the service provider), Financial indicators (Billing Data, Mobile Money etc.) and so on. The key problem to solve is how to use this information to promote consumption, increase retention and improve operational efficiency

Telco Perspective
The key problem Telcos face is the lack of a killer service like voice or messaging which can stand out in their Big Data offering. Big Data technologies thus remain invisible to the common consumer in terms of variety or newer services offered to him/her and spent instead to provide the conventional services with better experience. In other words there is less innovation in the actual service being offered but more on how it is offered.


Three key patterns can be observed

a)  Promote Consumption: The general approach is to increase consumption by users by providing them the right kind of service stack. It does not necessarily mean developing new services but finding out suitable offers which will gain acceptance and hence drive more consumption. Personalization and Contextualization are the key factors here. Recommendations (based on preference, mood, locality, usage), special tariff, partnership with OTT service providers are some ways Telcos can boost the consumption.

b)  Increase Retention: Unlike many businesses, Telcos mainly thrive on regular revenue assurance from its user base and hence retaining them becomes highly significant, especially when attracting a new user is costlier by several magnitudes in view of slow service innovation, customer inertia and slow onboard time due to physical barriers involved (eg. point of sale interactions, device interactions etc.). While customer churn is protected in some ways because of these constraints and improvement in Quality of Service guarantee, attractive tariff plans rewards/bonus etc. help to increase customer retention and any eventual churn possibilities.

c)  Improve Efficiency: Increasing efficiency mainly concerns about saving money. Its main purpose is to help to do more or at least the same things with lower expenditure. One of the key aspects of efficiency improvement is cutting down on the waste activities, and this is where Big Data has been able to be proven effective. It is now possible to identify with reasonable accuracy the potential buyers, exact pain areas, detailed behavior patterns and so on. Improving efficiency is reflected into activities such as network optimization (eg. re-distributing cell areas or BTS antennas based on traffic pattern), Farud Management (eg. detecting habitual churners), IT management (eg. commodity hardware vs. custom hardware) customer care (eg. 360 degree view of the customer to serve him/her better), precise marketing (eg. avoid selling products to irrelevant groups) and so on.

The State of Telco Big Data
The ultimate aim of any business is to grow the revenue and reduce costs. Big Data technologies have been quite effectively used to analyze data, identify waste in infrastructure and processes and remove them. So in a way they have proven to be useful in ‘Reducing Costs’. However when it comes to ‘Generate New Revenue’, they kind of fall short in terms of new service offering and depend almost entirely on promoting consumption of the existing ones.


In fact Telcos don’t have too many products which can be called their own. As a result Big Data application in most cases has been limited to realizing recommendation and personalization services. Such as personalization of Value Added Services – such as Ring Back Tone, Mobile News etc., Tariff Plan recommendation based on user’s voice, data or message consumption history or carrying out intelligent advertisement campaigns. The lack of native services, prompt Telcos to ‘resell’ or have ‘symbiosis’ with OTT services like instant messaging, entertainment, maps, advertisers etc.

Another key bottleneck Telcos face today is inadvertent ‘Data Leak’. While the operator can theoretically capture and analyze data happening through their channels, there is rarely any deep correlation and exploration of data generated at various sources. To give an example, most Telcos for example are also ISPs (Internet Service Providers) and can theoretically capture the consumer’s wireline Internet behavior and recommend matching services in the wireless domain. Imagine a user who exhibits a pattern to watch videos on his/her fixed broadband and may be enticed into a high speed 4G connection with a special discounted tariff plan or a trial pack knowing that he/she has a high chance of ending up subscribing.

As I have observed earlier in this post, Telco operators are in general having access to huge amount of data which may be correlated, analyzed and explored together to decode entire behavior patterns and profiles and perform useful predictions. These can be valuable assets to the operator and at the same time to third party service providers. Enterprises other than Telco can merge the intelligence gathered from Telco with their own intelligence and service logic to offer cross platform products. Typical examples being Telcos sharing mobility patterns, mobile internet data, consumption category, loyalty factors etc. which provide additional insights. This treatment of data as commodity is one of the key trends in Telco world for monetizing on Big Data technologies. Commoditization is a painful process as large amount of raw data needs to be crunched, cleansed, analyzed, anonymized, summarized into smaller and simpler forms for other businesses to consume.

Vendor Perspective
Traditional Telco vendors have the biggest advantage of deep knowledge of the Telco Data Science as well as the Telco operators and their business problems. Hence the key value from Telco Vendors can come by exploiting these aspects and not competing with pure Technology vendors who are in the business for a longer time and more mature. In fact Big Data is potentially emerging as a general purpose technology like OS, Database, Programming Language and so on and in the end there will be only few key vendors. However a thriving ecosystem is still possible when companies start to focus on the applications side, to utilize the Technologies and Tools to address the business needs.

  • Telco Vendors are no different. They can create values by providing out of the box tools to support the business of Telco Operators better for their specific day to day use cases. 

Some common cases may be:
  • Develop and Provide prediction models for customer churn, product recommendation, service recommendation, fraud detection, network optimization etc.
  • Share data and events as commodities to third party applications (OTT providers)
  • Cooperate with OTT providers to provide rich experience to the consumers, data comes as a tool
  • Offer innovative services and improve quality of operation by correlating multiple data sources fast
  • Enable real time decisions to provide instantaneous personalized services

......  and so on

In the following I present a rough architecture for a complete stack that can be used to build a Big Data Solution. It is not very much Telco specific and I believe can be replicated to other enterprises as well.
In several subsequent posts I will elaborate on this architecture and try to evaluate the pros and cons of various approaches in building each of these layers including more details on various technology choices, interfaces and integration options that may be available to us.













Comments

Popular Posts