Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Leverage Apache Spark within a modern data engineering ecosystem. This hands-on guide will teach you how to write fully functional applications, follow industry best practices, and learn the rationale behind these decisions. With Apache Spark as the foundation, you will follow a step-by-step journey beginning with the basics of data ingestion, processing, and transformation, and ending up with an entire local data platform running Apache Spark, Apache Zeppelin, Apache Kafka, Redis, MySQL, Minio (S3), and Apache Airflow.

Data mining. --- Spark (Electronic resource : Apache Software Foundation) --- Algorithmic knowledge discovery --- Factual data analysis --- KDD (Information retrieval) --- Knowledge discovery in data --- Knowledge discovery in databases --- Mining, Data --- Database searching --- Apache Spark (Electronic resource : Apache Software Foundation)

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Take a journey toward discovering, learning, and using Apache Spark 3.0. In this book, you will gain expertise on the powerful and efficient distributed data processing engine inside of Apache Spark; its user-friendly, comprehensive, and flexible programming model for processing data in batch and streaming; and the scalable machine learning algorithms and practical utilities to build machine learning applications. Beginning Apache Spark 3 begins by explaining different ways of interacting with Apache Spark, such as Spark Concepts and Architecture, and Spark Unified Stack. Next, it offers an overview of Spark SQL before moving on to its advanced features. It covers tips and techniques for dealing with performance issues, followed by an overview of the structured streaming processing engine. It concludes with a demonstration of how to develop machine learning applications using Spark MLlib and how to manage the machine learning development lifecycle. This book is packed with practical examples and code snippets to help you master concepts and features immediately after they are covered in each section. After reading this book, you will have the knowledge required to build your own big data pipelines, applications, and machine learning applications. What You Will Learn Master the Spark unified data analytics engine and its various components Work in tandem to provide a scalable, fault tolerant and performant data processing engine Leverage the user-friendly and flexible programming model to perform simple to complex data analytics using dataframe and Spark SQL Develop machine learning applications using Spark MLlib Manage the machine learning development lifecycle using MLflow Who This Book Is For Data scientists, data engineers and software developers.

Spark (Electronic resource : Apache Software Foundation) --- Big data. --- Distributed databases. --- Distributed data bases --- Distributed database systems --- Databases --- Cyberinfrastructure --- Data sets, Large --- Large data sets --- Data sets --- Open source software. --- Machine learning. --- Learning, Machine --- Artificial intelligence --- Machine theory --- Free software (Open source software) --- Open code software --- Opensource software --- Computer software --- Apache Spark (Electronic resource : Apache Software Foundation)

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Integrate full-stack open-source fast data pipeline architecture and choose the correct technology—Spark, Mesos, Akka, Cassandra, and Kafka (SMACK)—in every layer. Fast data is becoming a requirement for many enterprises. So far, however, the focus has largely been on collecting, aggregating, and crunching large data sets in a timely manner. In many cases organizations need more than one paradigm to perform efficient analyses. Big Data SMACK explains each technology and, more importantly, how to integrate them. It provides detailed coverage of the practical benefits of these technologies and incorporates real-world examples. The book focuses on the problems and scenarios solved by the architecture, as well as the solutions provided by each technology. This book covers the five main concepts of data pipeline architecture and how to integrate, replace, and reinforce every layer: The engine: Apache Spark The container: Apache Mesos The model: Akka< The storage: Apache Cassandra The broker: Apache Kafka.

Computer science. --- Data structures (Computer science). --- Database management. --- Computer Science. --- Big Data. --- Database Management. --- Data Structures. --- Big data. --- Data sets, Large --- Large data sets --- Data base management --- Data services (Database management) --- Database management services --- DBMS (Computer science) --- Generalized data management systems --- Services, Database management --- Systems, Database management --- Systems, Generalized database management --- Electronic data processing --- Information structures (Computer science) --- Structures, Data (Computer science) --- Structures, Information (Computer science) --- File organization (Computer science) --- Abstract data types (Computer science) --- Informatics --- Science --- Spark (Electronic resource : Apache Software Foundation) --- Apache Mesos (Electronic resource) --- Akka (Electronic resource) --- Apache Cassandra. --- Apache Kafka. --- Cassandra (Electronic resource) --- Apache Spark (Electronic resource : Apache Software Foundation) --- Mesos (Electronic resource) --- Data sets --- Data structures (Computer scienc.

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Work with Apache Spark using Scala to deploy and set up single-node, multi-node, and high-availability clusters. This book discusses various components of Spark such as Spark Core, DataFrames, Datasets and SQL, Spark Streaming, Spark MLib, and R on Spark with the help of practical code snippets for each topic. Practical Apache Spark also covers the integration of Apache Spark with Kafka with examples. You’ll follow a learn-to-do-by-yourself approach to learning – learn the concepts, practice the code snippets in Scala, and complete the assignments given to get an overall exposure. On completion, you’ll have knowledge of the functional programming aspects of Scala, and hands-on expertise in various Spark components. You’ll also become familiar with machine learning algorithms with real-time usage. You will: Discover the functional programming features of Scala Understand the complete architecture of Spark and its components Integrate Apache Spark with Hive and Kafka Use Spark SQL, DataFrames, and Datasets to process data using traditional SQL queries Work with different machine learning concepts and libraries using Spark's MLlib packages.

Scala (Computer program language) --- Functional programming languages --- Object-oriented programming languages --- Multiparadigm programming (Computer science) --- Big data. --- Open source software. --- Computer programming. --- Computer science. --- Big Data. --- Open Source. --- Programming Languages, Compilers, Interpreters. --- Informatics --- Science --- Computers --- Electronic computer programming --- Electronic data processing --- Electronic digital computers --- Programming (Electronic computers) --- Coding theory --- Free software (Open source software) --- Open code software --- Opensource software --- Computer software --- Data sets, Large --- Large data sets --- Data sets --- Programming --- Spark (Electronic resource : Apache Software Foundation) --- SPARK (Electronic resource) --- Apache Spark (Electronic resource : Apache Software Foundation) --- Programming languages (Electronic computers). --- Computer languages --- Computer program languages --- Computer programming languages --- Machine language --- Languages, Artificial

Choose an application

• Reference Manager
• EndNote
• RefWorks (Direct export to RefWorks)

Smart cities operate under more resource-efficient management and economy than ordinary cities. As such, advanced business models have emerged around smart cities, which led to the creation of smart enterprises and organizations that depend on advanced technologies. This book includes 21 selected and peer-reviewed articles contributed in the wide spectrum of artificial intelligence applications to smart cities. Chapters refer to the following areas of interest: vehicular traffic prediction, social big data analysis, smart city management, driving and routing, localization, safety, health, and life quality.

Information technology industries --- spatio-temporal --- residual networks --- bus traffic flow prediction --- advance rate --- shield performance --- principal component analysis --- ANFIS-GA --- tunnel --- online learning --- extreme learning machine --- cyclic dynamics --- transfer learning --- knowledge preservation --- Feature Adaptive --- optimization --- Bacterial Foraging algorithm --- Swarm Intelligence algorithm --- Isolated Microgrid --- traffic surveillance video --- state analysis --- Grassmann manifold --- neural network --- machine-learning --- quality of life --- Better Life Index --- bagging --- ensemble learning --- pedestrian attributes --- surveillance image --- semantic attributes recognition --- multi-label learning --- large-scale database --- traffic congestion detection --- minimizing traffic congestion --- traffic prediction --- deep learning --- urban mobility --- ITS --- Vehicle-to-Infrastructure --- neural networks --- LSTM --- embeddings --- trajectories --- motion behavior --- smart tourism --- driver’s behavior detection --- texting and driving --- convolutional neural network --- smart car --- smart cities --- smart infotainment --- driver distraction --- cameras --- convolution --- detection --- image recognition --- DSS --- diabetes prediction --- homecare assistance information system --- muti-attribute analysis --- artificial training dataset --- machine learning --- big data --- data analysis --- sensors --- Internet of Things --- vehicular networks --- VDTN --- routing --- message scheduling --- traffic flow prediction --- wavenet --- TrafficWave --- RNN --- GRU --- SAEs --- risk assessment --- neural architecture search --- recurrent neural network --- automated driving vehicle --- decision support system --- artificial intelligence --- disaster management --- Smart city --- program management --- integrated model --- smart city --- intelligence transportation system --- computer vision --- potential pedestrian safety --- data mining --- healthcare --- Apache Spark --- disease detection --- symptoms detection --- Arabic language --- Saudi dialect --- Twitter --- high performance computing (HPC) --- spatial-temporal dependencies --- traffic periodicity --- graph convolutional network --- traffic speed prediction --- vehicular traffic --- surveillance video --- big data analysis --- autonomous driving --- life quality --- pattern recognition