Exponentially growing volumes of unstructured and semi-structured data, especially from new sources such as machines, sensors, logs, and social media are posing serious challenges to the traditional BI/DW tools which were not designed for such data sources and data types. Newer technologies like Hadoop, adequately handle bigger volumes of Big Data and batch processing, but they are not geared to handle high velocities of data which fall in the realm of complex event processing, real time analytics, streaming or fast data, and operational intelligence.
“Real-time Big Data isn’t just a process for storing petabytes or exabytes of data in a data warehouse,” says Michael Minelli, co-author of "Big Data, Big Analytics. “ It’s about the ability to make better decisions and take meaningful actions at the right time. It’s about detecting fraud while someone is swiping a credit card, or triggering an offer while a shopper is standing on a checkout line, or placing an ad on a website while someone is reading a specific article. It’s about combining and analyzing data so you can take the right action, at the right time, and at the right place.”
At KMS, we use Real time analytics, fast data technologies and operational intelligence to support the growing needs of analyzing streaming data or complex event processing using high velocity data ingestion technologies, analyzing them on the fly, and drawing insights by correlating parallel streams of fast data, or data from multiple sources at various latencies (from batch to real time) to reveal actionable information. Our Real time analytics and Operational intelligence (OI) services provide visibility into business processes, events, and operations as they are happening. These are enabled by special technologies that can handle machine data, sensor data, event streams, and other forms of streaming data and Big Data. Organizations can act on the information by immediately sending an alert to the appropriate manager, updating a management dashboard, offering an incentive to customers based on their present location, adjusting machinery or preventing fraud.
In the context of Smart Cities, Real Time analytics and OI are critical. The network of devices and sensors in Smart Cities generate huge volumes and velocities of semi-structured and unstructured data which needs to be analyzed as a part of complex event processing – public waste collection, flood water management, citizen services, mobility and transportation, predictive maintenance of assets, etc. Be it about providing real time transport navigation to citizens in their smartphones, or real time operational intelligence about the extent of waste collection from the public trash bins, or understanding citizens' challenges by analyzing complaints and feedback from social blogs - Smart City analytics inherently require real time analytics on semi-structured and unstructured data.
The KMS proprietary Big Data analytics framework is capable of Real Time analytics and providing operational intelligence via:
- Open architecture that can ingest data from various data sources like sensor controllers, smart phones, text data from blogs / forums, etc.
- High velocity data ingestion technologies with write intensive databases.
- Real time analytics and streaming data analysis (using technologies like Apache Storm).
- Big Data technologies and Advanced analytics solutions to ingest, store, analyze and draw insights from the streaming data in conjunction with other data sources as a part of complex event processing.
- Business intelligence technologies to present insights in the form of visual interfaces.
Our Real Time analytics services can be applied in diverse areas including:
Smart City Governance and citizen services
Intelligence and surveillance
Simulating and Monitoring toxic gas leakage and possible impact
Risk and Disaster Management
Location based analytics
Smart order routing
Transaction cost analysis
Pricing and analytics
Data warehouse augmentation
Supply chain analytics
Preventive and Predictive maintenance