Real Time Storm Project - Edureka.co

Real Time Storm Project - Edureka.co Real Time Storm Project Back Home Categories Online Courses Mock Interviews Webinars NEW Community Write for Us Categories Artificial Intelligence AI vs Machine Learning vs Deep LearningMachine Learning AlgorithmsArtificial Intelligence TutorialWhat is Deep LearningDeep Learning TutorialInstall TensorFlowDeep Learning with PythonBackpropagationTensorFlow TutorialConvolutional Neural Network TutorialVIEW ALL BI and Visualization What is TableauTableau TutorialTableau Interview QuestionsWhat is InformaticaInformatica Interview QuestionsPower BI TutorialPower BI Interview QuestionsOLTP vs OLAPQlikView TutorialAdvanced Excel Formulas TutorialVIEW ALL Big Data What is HadoopHadoop ArchitectureHadoop TutorialHadoop Interview QuestionsHadoop EcosystemData Science vs Big Data vs Data AnalyticsWhat is Big DataMapReduce TutorialPig TutorialSpark TutorialSpark Interview QuestionsBig Data TutorialHive TutorialVIEW ALL Blockchain Blockchain TutorialWhat is BlockchainHyperledger FabricWhat Is EthereumEthereum TutorialB lockchain ApplicationsSolidity TutorialBlockchain ProgrammingHow Blockchain WorksVIEW ALL Cloud Computing What is AWSAWS TutorialAWS CertificationAzure Interview QuestionsAzure TutorialWhat Is Cloud ComputingWhat Is SalesforceIoT TutorialSalesforce TutorialSalesforce Interview QuestionsVIEW ALL Cyber Security Cloud SecurityWhat is CryptographyNmap TutorialSQL Injection AttacksHow To Install Kali LinuxHow to become an Ethical Hacker?Footprinting in Ethical HackingNetwork Scanning for Ethical HackingARP SpoofingApplication SecurityVIEW ALL Data Science Python Pandas TutorialWhat is Machine LearningMachine Learning TutorialMachine Learning ProjectsMachine Learning Interview QuestionsWhat Is Data ScienceSAS TutorialR TutorialData Science ProjectsHow to become a data scientistData Science Interview QuestionsData Scientist SalaryVIEW ALL Data Warehousing and ETL What is Data WarehouseDimension Table in Data WarehousingData Warehousing Interview QuestionsData warehouse architectureTalend T utorialTalend ETL ToolTalend Interview QuestionsFact Table and its TypesInformatica TransformationsInformatica TutorialVIEW ALL Databases What is MySQLMySQL Data TypesSQL JoinsSQL Data TypesWhat is MongoDBMongoDB Interview QuestionsMySQL TutorialSQL Interview QuestionsSQL CommandsMySQL Interview QuestionsVIEW ALL DevOps What is DevOpsDevOps vs AgileDevOps ToolsDevOps TutorialHow To Become A DevOps EngineerDevOps Interview QuestionsWhat Is DockerDocker TutorialDocker Interview QuestionsWhat Is ChefWhat Is KubernetesKubernetes TutorialVIEW ALL Front End Web Development What is JavaScript â€" All You Need To Know About JavaScriptJavaScript TutorialJavaScript Interview QuestionsJavaScript FrameworksAngular TutorialAngular Interview QuestionsWhat is REST API?React TutorialReact vs AngularjQuery TutorialNode TutorialReact Interview QuestionsVIEW ALL Mobile Development Android TutorialAndroid Interview QuestionsAndroid ArchitectureAndroid SQLite DatabaseProgramming In this blog, we are goi ng to execute a real time storm project.ICD9Data.com provides disease codes ranging from 000-999. These codes are assigned to different diseases uniquely. It is available as a mobile app also.Let us consider any mobile app used by doctors is generating sensor data. This sensor data has information such as device id, device network, user, latitude, longitude, time, disease code etc. For our scenario, we will be considering only latitude, longitude, time and disease code.These data are processed using storm and trident topology to detect disease outbreak within any geographic location for disease code less than or equal to 322 at a particular time. If the count of a particular disease increases the threshold, in this case 2,000, then the system will send an alert message with the name of the country, disease code and time. To simplify things for this scenario, we will map every diagnosis event to the closest country.In this scenario, we will use disease codes between 315 and 322. Refe r the table below for diseases and their codes.Below is the flow of the project which is very self explanatory.Step 1: Sensor data is streamed by spout from the mobile app.Step 2: Spout sends only latitude, longitude, time and disease code as tuple.Step 3: The disease codes below or equal to 322 are filtered and they are sent forward.Step 4: Depending on the latitude and longitude, a country is assigned.Step 5: Hour is assigned to each tuple.Step 6: GroupBy key is done where key iscountry, dcode, time.Count is calculated after each tuple is processed.Step 7: Outbreak Trend State ensures that whatever is processed once should not be processed second time.Step 8: We have kept a threshold of 2,000, if the count crosses the threshold, we consider it as an outbreak and alert message is dispatched with the details of the outbreak.Download the storm project.Firstly, start the storm cluster, then run the storm project using below commands.Command: cd storm/Command: bin/storm jar /home/edure ka/SensorDataAnalysis/SDAnalysis/target/sensor-data-0.0.1-SNAPSHOT.jar in.edureka.trident.topology.DiseaseAlertTopologyThe threshold of 2,000 crossed in Turkey for disease code 320 and Disease Outbreak Alert was sent.This project is just to give you a practical view of storms trident topology concept. Here we have taken all the inputs randomly. You can put your business logic in the code according to your data and test it.Got a question for us? Mention them in the comments section and we will get back to you.Related Posts:Introduction to Real-Time Analytics with Apache StormLambda ArchitectureApache Storm Use CasesWhat is Apache Storm all about?Get started with Apache StormRecommended videos for you Is Hadoop A Necessity For Data Science? Watch Now Introduction to Big Data TDD and Pig Unit Watch Now Hadoop Cluster With High Availability Watch Now Hive Tutorial Understanding Hive In Depth Watch Now Streaming With Apache Spark and Scala Watch Now Big Data Processing With Apache Spark Watch Now Apache Spark For Faster Batch Processing Watch Now Is It The Right Time For Me To Learn Hadoop ? Find out. Watch Now MapReduce Design Patterns Application of Join Pattern Watch Now Hadoop for Java Professionals Watch Now What Is Hadoop All You Need To Know About Hadoop Watch Now Power of Python With BigData Watch Now Ways to Succeed with Hadoop in 2015 Watch Now MapReduce Tutorial All You Need To Know About MapReduce Watch Now Secure Your Hadoop Cluster With Kerberos Watch Now Apache Spark Will Replace Hadoop ! Know Why Watch Now Boost Your Data Career with Predictive Analytics! Learn How ? Watch Now Bulk Loading Into HBase With MapReduce Watch Now Big Data Processing with Spark and Scala Watch Now Advanced Security In Hadoop Cluster Watch NowRecommended blogs for you DynamoDB vs MongoDB: Which One Meets Your Business Needs Better? Read Article Hadoop YARN Tutorial Learn the Fundamentals of YARN Architecture Read Article Spark Tutorial: Real Time Cluster Computing Fra mework Read Article Hadoop Ecosystem: Hadoop Tools for Crunching Big Data Read Article Big Bucks for Big Data Professionals: A Hype or Hope? Read Article Machine Learning and Big Data: Is it the future? Read Article How to Run Hive Scripts? Read Article What Is Splunk? A Beginners Guide To Understanding Splunk Read Article Insights on HBase Architecture Read Article Hive Yarn Get Electrified By Spark Read Article How essential is Hadoop Training? Read Article Hive and Yarn Examples on Spark Read Article A Deep Dive Into Pig Read Article Most Important Scala Interview Questions to Prepare in 2020 Read Article Install Puppet Install Puppet in Four Simple Steps Read Article Big Data Analytics: Turning Insights into Action Read Article Install Hadoop: Setting up a Single Node Hadoop Cluster Read Article What are Kafka Streams and How are they implemented? Read Article Basics of HBase Read Article How to become a Hadoop Developer? Job Trends and Salary Read Article Comments 0 Comment s Trending Courses in Big Data Big Data Hadoop Certification Training158k Enrolled LearnersWeekend/WeekdayLive Class Reviews 5 (62950)