Big Data Business Intelligence for Telecom and Communication Service Providers Training Course

Overview

Communications service providers (CSP) are facing pressure to reduce costs and maximize average revenue per user (ARPU), while ensuring an excellent customer experience, but data volumes keep growing. Global mobile data traffic will grow at a compound annual growth rate (CAGR) of 78 percent to 2016, reaching 10.8 exabytes per month.

Meanwhile, CSPs are generating large volumes of data, including call detail records (CDR), network data and customer data. Companies that fully exploit this data gain a competitive edge. According to a recent survey by The Economist Intelligence Unit, companies that use data-directed decision-making enjoy a 5-6% boost in productivity. Yet 53% of companies leverage only half of their valuable data, and one-fourth of respondents noted that vast quantities of useful data go untapped. The data volumes are so high that manual analysis is impossible, and most legacy software systems can’t keep up, resulting in valuable data being discarded or ignored.

With Big Data & Analytics’ high-speed, scalable big data software, CSPs can mine all their data for better decision making in less time. Different Big Data products and techniques provide an end-to-end software platform for collecting, preparing, analyzing and presenting insights from big data. Application areas include network performance monitoring, fraud detection, customer churn detection and credit risk analysis. Big Data & Analytics products scale to handle terabytes of data but implementation of such tools need new kind of cloud based database system like Hadoop or massive scale parallel computing processor ( KPU etc.)

This course work on Big Data BI for Telco covers all the emerging new areas in which CSPs are investing for productivity gain and opening up new business revenue stream. The course will provide a complete 360 degree over view of Big Data BI in Telco so that decision makers and managers can have a very wide and comprehensive overview of possibilities of Big Data BI in Telco for productivity and revenue gain.

Course objectives

Main objective of the course is to introduce new Big Data business intelligence techniques in 4 sectors of Telecom Business (Marketing/Sales, Network Operation, Financial operation and Customer Relation Management). Students will be introduced to following:

  • Introduction to Big Data-what is 4Vs (volume, velocity, variety and veracity) in Big Data- Generation, extraction and management from Telco perspective
  • How Big Data analytic differs from legacy data analytic
  • In-house justification of Big Data -Telco perspective
  • Introduction to Hadoop Ecosystem- familiarity with all Hadoop tools like Hive, Pig, SPARC –when and how they are used to solve Big Data problem
  • How Big Data is extracted to analyze for analytics tool-how Business Analysis’s can reduce their pain points of collection and analysis of data through integrated Hadoop dashboard approach
  • Basic introduction of Insight analytics, visualization analytics and predictive analytics for Telco
  • Customer Churn analytic and Big Data-how Big Data analytic can reduce customer churn and customer dissatisfaction in Telco-case studies
  • Network failure and service failure analytics from Network meta-data and IPDR
  • Financial analysis-fraud, wastage and ROI estimation from sales and operational data
  • Customer acquisition problem-Target marketing, customer segmentation and cross-sale from sales data
  • Introduction and summary of all Big Data analytic products and where they fit into Telco analytic space
  • Conclusion-how to take step-by-step approach to introduce Big Data Business Intelligence in your organization

Target Audience

  • Network operation, Financial Managers, CRM managers and top IT managers in Telco CIO office.
  • Business Analysts in Telco
  • CFO office managers/analysts
  • Operational managers
  • QA managers

Requirements

  • Should have basic knowledge of business operation and data systems in Telecom in their domain
  • Must have basic understanding of SQL/Oracle or relational database
  • Basic understanding of Statistics (in Excel levels)

Course Outline

Breakdown of topics on daily basis: (Each session is 2 hours)

Day-1: Session -1: Business Overview of Why Big Data Business Intelligence in Telco.

  • Case Studies from T-Mobile, Verizon etc.
  • Big Data adaptation rate in North American Telco & and how they are aligning their future business model and operation around Big Data BI
  • Broad Scale Application Area
  • Network and Service management
  • Customer Churn Management
  • Data Integration & Dashboard visualization
  • Fraud management
  • Business Rule generation
  • Customer profiling
  • Localized Ad pushing

Day-1: Session-2 : Introduction of Big Data-1

  • Main characteristics of Big Data-volume, variety, velocity and veracity. MPP architecture for volume.
  • Data Warehouses – static schema, slowly evolving dataset
  • MPP Databases like Greenplum, Exadata, Teradata, Netezza, Vertica etc.
  • Hadoop Based Solutions – no conditions on structure of dataset.
  • Typical pattern : HDFS, MapReduce (crunch), retrieve from HDFS
  • Batch- suited for analytical/non-interactive
  • Volume : CEP streaming data
  • Typical choices – CEP products (e.g. Infostreams, Apama, MarkLogic etc)
  • Less production ready – Storm/S4
  • NoSQL Databases – (columnar and key-value): Best suited as analytical adjunct to data warehouse/database

Day-1 : Session -3 : Introduction to Big Data-2

NoSQL solutions

  • KV Store – Keyspace, Flare, SchemaFree, RAMCloud, Oracle NoSQL Database (OnDB)
  • KV Store – Dynamo, Voldemort, Dynomite, SubRecord, Mo8onDb, DovetailDB
  • KV Store (Hierarchical) – GT.m, Cache
  • KV Store (Ordered) – TokyoTyrant, Lightcloud, NMDB, Luxio, MemcacheDB, Actord
  • KV Cache – Memcached, Repcached, Coherence, Infinispan, EXtremeScale, JBossCache, Velocity, Terracoqua
  • Tuple Store – Gigaspaces, Coord, Apache River
  • Object Database – ZopeDB, DB40, Shoal
  • Document Store – CouchDB, Cloudant, Couchbase, MongoDB, Jackrabbit, XML-Databases, ThruDB, CloudKit, Prsevere, Riak-Basho, Scalaris
  • Wide Columnar Store – BigTable, HBase, Apache Cassandra, Hypertable, KAI, OpenNeptune, Qbase, KDI

Varieties of Data: Introduction to Data Cleaning issue in Big Data

  • RDBMS – static structure/schema, doesn’t promote agile, exploratory environment.
  • NoSQL – semi structured, enough structure to store data without exact schema before storing data
  • Data cleaning issues

Day-1 : Session-4 : Big Data Introduction-3 : Hadoop

  • When to select Hadoop?
  • STRUCTURED – Enterprise data warehouses/databases can store massive data (at a cost) but impose structure (not good for active exploration)
  • SEMI STRUCTURED data – tough to do with traditional solutions (DW/DB)
  • Warehousing data = HUGE effort and static even after implementation
  • For variety & volume of data, crunched on commodity hardware – HADOOP
  • Commodity H/W needed to create a Hadoop Cluster

Introduction to Map Reduce /HDFS

  • MapReduce – distribute computing over multiple servers
  • HDFS – make data available locally for the computing process (with redundancy)
  • Data – can be unstructured/schema-less (unlike RDBMS)
  • Developer responsibility to make sense of data
  • Programming MapReduce = working with Java (pros/cons), manually loading data into HDFS

Day-2: Session-1.1: Spark : In Memory distributed database

  • What is “In memory” processing?
  • Spark SQL
  • Spark SDK
  • Spark API
  • RDD
  • Spark Lib
  • Hanna
  • How to migrate an existing Hadoop system to Spark

Day-2 Session -1.2: Storm -Real time processing in Big Data

  • Streams
  • Sprouts
  • Bolts
  • Topologies

Day-2: Session-2: Big Data Management System

  • Moving parts, compute nodes start/fail :ZooKeeper – For configuration/coordination/naming services
  • Complex pipeline/workflow: Oozie – manage workflow, dependencies, daisy chain
  • Deploy, configure, cluster management, upgrade etc (sys admin) :Ambari
  • In Cloud : Whirr
  • Evolving Big Data platform tools for tracking
  • ETL layer application issues

Day-2: Session-3: Predictive analytics in Business Intelligence -1: Fundamental Techniques & Machine learning based BI :

  • Introduction to Machine learning
  • Learning classification techniques
  • Bayesian Prediction-preparing training file
  • Markov random field
  • Supervised and unsupervised learning
  • Feature extraction
  • Support Vector Machine
  • Neural Network
  • Reinforcement learning
  • Big Data large variable problem -Random forest (RF)
  • Representation learning
  • Deep learning
  • Big Data Automation problem – Multi-model ensemble RF
  • Automation through Soft10-M
  • LDA and topic modeling
  • Agile learning
  • Agent based learning- Example from Telco operation
  • Distributed learning –Example from Telco operation
  • Introduction to Open source Tools for predictive analytics : R, Rapidminer, Mahut
  • More scalable Analytic-Apache Hama, Spark and CMU Graph lab

Day-2: Session-4 Predictive analytics eco-system-2: Common predictive analytic problems in Telecom

  • Insight analytic
  • Visualization analytic
  • Structured predictive analytic
  • Unstructured predictive analytic
  • Customer profiling
  • Recommendation Engine
  • Pattern detection
  • Rule/Scenario discovery –failure, fraud, optimization
  • Root cause discovery
  • Sentiment analysis
  • CRM analytic
  • Network analytic
  • Text Analytics
  • Technology assisted review
  • Fraud analytic
  • Real Time Analytic

Day-3 : Sesion-1 : Network Operation analytic- root cause analysis of network failures, service interruption from meta data, IPDR and CRM:

  • CPU Usage
  • Memory Usage
  • QoS Queue Usage
  • Device Temperature
  • Interface Error
  • IoS versions
  • Routing Events
  • Latency variations
  • Syslog analytics
  • Packet Loss
  • Load simulation
  • Topology inference
  • Performance Threshold
  • Device Traps
  • IPDR ( IP detailed record) collection and processing
  • Use of IPDR data for Subscriber Bandwidth consumption, Network interface utilization, modem status and diagnostic
  • HFC information

Day-3: Session-2: Tools for Network service failure analysis:

  • Network Summary Dashboard: monitor overall network deployments and track your organization’s key performance indicators
  • Peak Period Analysis Dashboard: understand the application and subscriber trends driving peak utilization, with location-specific granularity
  • Routing Efficiency Dashboard: control network costs and build business cases for capital projects with a complete understanding of interconnect and transit relationships
  • Real-Time Entertainment Dashboard: access metrics that matter, including video views, duration, and video quality of experience (QoE)
  • IPv6 Transition Dashboard: investigate the ongoing adoption of IPv6 on your network and gain insight into the applications and devices driving trends
  • Case-Study-1: The Alcatel-Lucent Big Network Analytics (BNA) Data Miner
  • Multi-dimensional mobile intelligence (m.IQ6)

Day-3 : Session 3: Big Data BI for Marketing/Sales –Understanding sales/marketing from Sales data: ( All of them will be shown with a live predictive analytic demo )

  • To identify highest velocity clients
  • To identify clients for a given products
  • To identify right set of products for a client ( Recommendation Engine)
  • Market segmentation technique
  • Cross-Sale and upsale technique
  • Client segmentation technique
  • Sales revenue forecasting technique

Day-3: Session 4: BI needed for Telco CFO office:

  • Overview of Business Analytics works needed in a CFO office
  • Risk analysis on new investment
  • Revenue, profit forecasting
  • New client acquisition forecasting
  • Loss forecasting
  • Fraud analytic on finances ( details next session )

Day-4 : Session-1: Fraud prevention BI from Big Data in Telco-Fraud analytic:

  • Bandwidth leakage / Bandwidth fraud
  • Vendor fraud/over charging for projects
  • Customer refund/claims frauds
  • Travel reimbursement frauds

Day-4 : Session-2: From Churning Prediction to Churn Prevention:

  • 3 Types of Churn : Active/Deliberate , Rotational/Incidental, Passive Involuntary
  • 3 classification of churned customers: Total, Hidden, Partial
  • Understanding CRM variables for churn
  • Customer behavior data collection
  • Customer perception data collection
  • Customer demographics data collection
  • Cleaning CRM Data
  • Unstructured CRM data ( customer call, tickets, emails) and their conversion to structured data for Churn analysis
  • Social Media CRM-new way to extract customer satisfaction index
  • Case Study-1 : T-Mobile USA: Churn Reduction by 50%

Day-4 : Session-3: How to use predictive analysis for root cause analysis of customer dis-satisfaction :

  • Case Study -1 : Linking dissatisfaction to issues – Accounting, Engineering failures like service interruption, poor bandwidth service
  • Case Study-2: Big Data QA dashboard to track customer satisfaction index from various parameters such as call escalations, criticality of issues, pending service interruption events etc.

Day-4: Session-4: Big Data Dashboard for quick accessibility of diverse data and display :

  • Integration of existing application platform with Big Data Dashboard
  • Big Data management
  • Case Study of Big Data Dashboard: Tableau and Pentaho
  • Use Big Data app to push location based Advertisement
  • Tracking system and management

Day-5 : Session-1: How to justify Big Data BI implementation within an organization:

  • Defining ROI for Big Data implementation
  • Case studies for saving Analyst Time for collection and preparation of Data –increase in productivity gain
  • Case studies of revenue gain from customer churn
  • Revenue gain from location based and other targeted Ad
  • An integrated spreadsheet approach to calculate approx. expense vs. Revenue gain/savings from Big Data implementation.

Day-5 : Session-2: Step by Step procedure to replace legacy data system to Big Data System:

  • Understanding practical Big Data Migration Roadmap
  • What are the important information needed before architecting a Big Data implementation
  • What are the different ways of calculating volume, velocity, variety and veracity of data
  • How to estimate data growth
  • Case studies in 2 Telco

Day-5: Session 3 & 4: Review of Big Data Vendors and review of their products. Q/A session:

  • AccentureAlcatel-Lucent
  • Amazon –A9
  • APTEAN (Formerly CDC Software)
  • Cisco Systems
  • Cloudera
  • Dell
  • EMC
  • GoodData Corporation
  • Guavus
  • Hitachi Data Systems
  • Hortonworks
  • Huawei
  • HP
  • IBM
  • Informatica
  • Intel
  • Jaspersoft
  • Microsoft
  • MongoDB (Formerly 10Gen)
  • MU Sigma
  • Netapp
  • Opera Solutions
  • Oracle
  • Pentaho
  • Platfora
  • Qliktech
  • Quantum
  • Rackspace
  • Revolution Analytics
  • Salesforce
  • SAP
  • SAS Institute
  • Sisense
  • Software AG/Terracotta
  • Soft10 Automation
  • Splunk
  • Sqrrl
  • Supermicro
  • Tableau Software
  • Teradata
  • Think Big Analytics
  • Tidemark Systems
  • VMware (Part of EMC)

Leave a Reply

Your email address will not be published. Required fields are marked *