Cloudera Administration Handbook

Author: Rohit Menon

Publisher: Packt Publishing Ltd

ISBN: 1783558970

Category: Computers

Page: 254

View: 8045

DOWNLOAD NOW »
An easy-to-follow Apache Hadoop administrator’s guide filled with practical screenshots and explanations for each step and configuration. This book is great for administrators interested in setting up and managing a large Hadoop cluster. If you are an administrator, or want to be an administrator, and you are ready to build and maintain a production-level cluster running CDH5, then this book is for you.

Expert Hadoop Administration

Managing, Tuning, and Securing Spark, YARN, and HDFS

Author: Sam R. Alapati

Publisher: Addison-Wesley Professional

ISBN: 0134703383

Category: Computers

Page: 848

View: 7203

DOWNLOAD NOW »
This is the eBook of the printed book and may not include any media, website access codes, or print supplements that may come packaged with the bound book. The Comprehensive, Up-to-Date Apache Hadoop Administration Handbook and Reference “Sam Alapati has worked with production Hadoop clusters for six years. His unique depth of experience has enabled him to write the go-to resource for all administrators looking to spec, size, expand, and secure production Hadoop clusters of any size.” —Paul Dix, Series Editor In Expert Hadoop® Administration, leading Hadoop administrator Sam R. Alapati brings together authoritative knowledge for creating, configuring, securing, managing, and optimizing production Hadoop clusters in any environment. Drawing on his experience with large-scale Hadoop administration, Alapati integrates action-oriented advice with carefully researched explanations of both problems and solutions. He covers an unmatched range of topics and offers an unparalleled collection of realistic examples. Alapati demystifies complex Hadoop environments, helping you understand exactly what happens behind the scenes when you administer your cluster. You’ll gain unprecedented insight as you walk through building clusters from scratch and configuring high availability, performance, security, encryption, and other key attributes. The high-value administration skills you learn here will be indispensable no matter what Hadoop distribution you use or what Hadoop applications you run. Understand Hadoop’s architecture from an administrator’s standpoint Create simple and fully distributed clusters Run MapReduce and Spark applications in a Hadoop cluster Manage and protect Hadoop data and high availability Work with HDFS commands, file permissions, and storage management Move data, and use YARN to allocate resources and schedule jobs Manage job workflows with Oozie and Hue Secure, monitor, log, and optimize Hadoop Benchmark and troubleshoot Hadoop

Hadoop: The Definitive Guide

Author: Tom White

Publisher: "O'Reilly Media, Inc."

ISBN: 1449338771

Category: Computers

Page: 688

View: 5392

DOWNLOAD NOW »
Ready to unlock the power of your data? With this comprehensive guide, you’ll learn how to build and maintain reliable, scalable, distributed systems with Apache Hadoop. This book is ideal for programmers looking to analyze datasets of any size, and for administrators who want to set up and run Hadoop clusters. You’ll find illuminating case studies that demonstrate how Hadoop is used to solve specific problems. This third edition covers recent changes to Hadoop, including material on the new MapReduce API, as well as MapReduce 2 and its more flexible execution model (YARN). Store large datasets with the Hadoop Distributed File System (HDFS) Run distributed computations with MapReduce Use Hadoop’s data and I/O building blocks for compression, data integrity, serialization (including Avro), and persistence Discover common pitfalls and advanced features for writing real-world MapReduce programs Design, build, and administer a dedicated Hadoop cluster—or run Hadoop in the cloud Load data from relational databases into HDFS, using Sqoop Perform large-scale data processing with the Pig query language Analyze datasets with Hive, Hadoop’s data warehousing system Take advantage of HBase for structured and semi-structured data, and ZooKeeper for building distributed systems

Learning Cloudera Impala

Author: Avkash Chauhan

Publisher: Packt Publishing Ltd

ISBN: 1783281286

Category: Computers

Page: 150

View: 5096

DOWNLOAD NOW »
This book is an easy-to-follow, step-by-step tutorial where each chapter takes your knowledge to the next level. The book covers practical knowledge with tips to implement this knowledge in real-world scenarios. A chapter with a real-life example is included to help you understand the concepts in full.Using Cloudera Impala is for those who really want to take advantage of their Hadoop cluster by processing extremely large amounts of raw data in Hadoop at real-time speed. Prior knowledge of Hadoop and some exposure to HIVE and MapReduce is expected.

Hadoop Operations

A Guide for Developers and Administrators

Author: Eric Sammer

Publisher: "O'Reilly Media, Inc."

ISBN: 144932729X

Category: Computers

Page: 298

View: 2895

DOWNLOAD NOW »
If you’ve been asked to maintain large and complex Hadoop clusters, this book is a must. Demand for operations-specific material has skyrocketed now that Hadoop is becoming the de facto standard for truly large-scale data processing in the data center. Eric Sammer, Principal Solution Architect at Cloudera, shows you the particulars of running Hadoop in production, from planning, installing, and configuring the system to providing ongoing maintenance. Rather than run through all possible scenarios, this pragmatic operations guide calls out what works, as demonstrated in critical deployments. Get a high-level overview of HDFS and MapReduce: why they exist and how they work Plan a Hadoop deployment, from hardware and OS selection to network requirements Learn setup and configuration details with a list of critical properties Manage resources by sharing a cluster across multiple groups Get a runbook of the most common cluster maintenance tasks Monitor Hadoop clusters—and learn troubleshooting with the help of real-world war stories Use basic tools and techniques to handle backup and catastrophic failure

Hbase Administration Cookbook

Author: Yifeng Jiang

Publisher: Packt Publishing Ltd

ISBN: 1849517150

Category: Computers

Page: 332

View: 3087

DOWNLOAD NOW »
As part of Packt's cookbook series, each recipe offers a practical, step-by-step solution to common problems found in HBase administration. This book is for HBase administrators, developers, and will even help Hadoop administrators. You are not required to have HBase experience, but are expected to have a basic understanding of Hadoop and MapReduce.

Hadoop 2.x Administration Cookbook

Author: Gurmukh Singh

Publisher: Packt Publishing Ltd

ISBN: 1787126870

Category: Computers

Page: 348

View: 1366

DOWNLOAD NOW »
Over 100 practical recipes to help you become an expert Hadoop administrator About This Book Become an expert Hadoop administrator and perform tasks to optimize your Hadoop Cluster Import and export data into Hive and use Oozie to manage workflow. Practical recipes will help you plan and secure your Hadoop cluster, and make it highly available Who This Book Is For If you are a system administrator with a basic understanding of Hadoop and you want to get into Hadoop administration, this book is for you. It's also ideal if you are a Hadoop administrator who wants a quick reference guide to all the Hadoop administration-related tasks and solutions to commonly occurring problems What You Will Learn Set up the Hadoop architecture to run a Hadoop cluster smoothly Maintain a Hadoop cluster on HDFS, YARN, and MapReduce Understand high availability with Zookeeper and Journal Node Configure Flume for data ingestion and Oozie to run various workflows Tune the Hadoop cluster for optimal performance Schedule jobs on a Hadoop cluster using the Fair and Capacity scheduler Secure your cluster and troubleshoot it for various common pain points In Detail Hadoop enables the distributed storage and processing of large datasets across clusters of computers. Learning how to administer Hadoop is crucial to exploit its unique features. With this book, you will be able to overcome common problems encountered in Hadoop administration. The book begins with laying the foundation by showing you the steps needed to set up a Hadoop cluster and its various nodes. You will get a better understanding of how to maintain Hadoop cluster, especially on the HDFS layer and using YARN and MapReduce. Further on, you will explore durability and high availability of a Hadoop cluster. You'll get a better understanding of the schedulers in Hadoop and how to configure and use them for your tasks. You will also get hands-on experience with the backup and recovery options and the performance tuning aspects of Hadoop. Finally, you will get a better understanding of troubleshooting, diagnostics, and best practices in Hadoop administration. By the end of this book, you will have a proper understanding of working with Hadoop clusters and will also be able to secure, encrypt it, and configure auditing for your Hadoop clusters. Style and approach This book contains short recipes that will help you run a Hadoop cluster efficiently. The recipes are solutions to real-life problems that administrators encounter while working with a Hadoop cluster

Oracle Big Data Handbook

Author: Tom Plunkett,Brian Macdonald,Bruce Nelson,Mark Hornick,Helen Sun,Keith Laker,Khader Mohiuddin,Debra Harding,Gokula Mishra,David Segleau,Robert Stackowiak

Publisher: McGraw Hill Professional

ISBN: 0071827269

Category: Computers

Page: 464

View: 9455

DOWNLOAD NOW »
"Cowritten by members of Oracle's big data team, [this book] provides complete coverage of Oracle's comprehensive, integrated set of products for acquiring, organizing, analyzing, and leveraging unstructured data. The book discusses the strategies and technologies essential for a successful big data implementation, including Apache Hadoop, Oracle Big Data Appliance, Oracle Big Data Connectors, Oracle NoSQL Database, Oracle Endeca, Oracle Advanced Analytics, and Oracle's open source R offerings"--Page 4 of cover.

Programming Hive

Author: Edward Capriolo,Dean Wampler,Jason Rutherglen

Publisher: "O'Reilly Media, Inc."

ISBN: 1449319335

Category: Computers

Page: 328

View: 5708

DOWNLOAD NOW »
Describes the features and functions of Apache Hive, the data infrastructure for Hadoop.

Kafka: The Definitive Guide

Real-Time Data and Stream Processing at Scale

Author: Neha Narkhede,Gwen Shapira,Todd Palino

Publisher: "O'Reilly Media, Inc."

ISBN: 1491936118

Category: Computers

Page: 322

View: 2305

DOWNLOAD NOW »
Every enterprise application creates data, whether it’s log messages, metrics, user activity, outgoing messages, or something else. And how to move all of this data becomes nearly as important as the data itself. If you’re an application architect, developer, or production engineer new to Apache Kafka, this practical guide shows you how to use this open source streaming platform to handle real-time data feeds. Engineers from Confluent and LinkedIn who are responsible for developing Kafka explain how to deploy production Kafka clusters, write reliable event-driven microservices, and build scalable stream-processing applications with this platform. Through detailed examples, you’ll learn Kafka’s design principles, reliability guarantees, key APIs, and architecture details, including the replication protocol, the controller, and the storage layer. Understand publish-subscribe messaging and how it fits in the big data ecosystem. Explore Kafka producers and consumers for writing and reading messages Understand Kafka patterns and use-case requirements to ensure reliable data delivery Get best practices for building data pipelines and applications with Kafka Manage Kafka in production, and learn to perform monitoring, tuning, and maintenance tasks Learn the most critical metrics among Kafka’s operational measurements Explore how Kafka’s stream delivery capabilities make it a perfect source for stream processing systems

Executing Data Quality Projects

Ten Steps to Quality Data and Trusted Information (TM)

Author: Danette McGilvray

Publisher: Elsevier

ISBN: 0080558399

Category: Computers

Page: 352

View: 560

DOWNLOAD NOW »
Information is currency. Recent studies show that data quality problems are costing businesses billions of dollars each year, with poor data linked to waste and inefficiency, damaged credibility among customers and suppliers, and an organizational inability to make sound decisions. In this important and timely new book, Danette McGilvray presents her “Ten Steps approach to information quality, a proven method for both understanding and creating information quality in the enterprise. Her trademarked approach—in which she has trained Fortune 500 clients and hundreds of workshop attendees—applies to all types of data and to all types of organizations. * Includes numerous templates, detailed examples, and practical advice for executing every step of the “Ten Steps approach. * Allows for quick reference with an easy-to-use format highlighting key concepts and definitions, important checkpoints, communication activities, and best practices. * A companion Web site includes links to numerous data quality resources, including many of the planning and information-gathering templates featured in the text, quick summaries of key ideas from the Ten Step methodology, and other tools and information available online.

HBase

The Definitive Guide

Author: Lars George

Publisher: "O'Reilly Media, Inc."

ISBN: 1449396100

Category: Computers

Page: 522

View: 2303

DOWNLOAD NOW »
If your organization is looking for a storage solution to accommodate a virtually endless amount of data, this book will show you how Apache HBase can fulfill your needs. As the open source implementation of Google's BigTable architecture, HBase scales to billions of rows and millions of columns, while ensuring that write and read performance remain constant.HBase: The Definitive Guideprovides the details you require, whether you simply want to evaluate this high-performance, non-relational database, or put it into practice right away. HBase's adoption rate is beginning to climb, and several IT executives are asking pointed questions about this high-capacity database. This is the only book available to give you meaningful answers. Learn how to distribute large datasets across an inexpensive cluster of commodity servers Develop HBase clients in many programming languages, including Java, Python, and Ruby Get details on HBase's primary storage system, HDFS—Hadoop’s distributed and replicated filesystem Learn how HBase's native interface to Hadoop’s MapReduce framework enables easy development and execution of batch jobs that can scan entire tables Discover the integration between HBase and other facets of the Apache Hadoop project

Hadoop Security

Protecting Your Big Data Platform

Author: Ben Spivey,Joey Echeverria

Publisher: "O'Reilly Media, Inc."

ISBN: 1491901349

Category: Computers

Page: 340

View: 3686

DOWNLOAD NOW »
As more corporations turn to Hadoop to store and process their most valuable data, the risk of a potential breach of those systems increases exponentially. This practical book not only shows Hadoop administrators and security architects how to protect Hadoop data from unauthorized access, it also shows how to limit the ability of an attacker to corrupt or modify data in the event of a security breach. Authors Ben Spivey and Joey Echeverria provide in-depth information about the security features available in Hadoop, and organize them according to common computer security concepts. You’ll also get real-world examples that demonstrate how you can apply these concepts to your use cases. Understand the challenges of securing distributed systems, particularly Hadoop Use best practices for preparing Hadoop cluster hardware as securely as possible Get an overview of the Kerberos network authentication protocol Delve into authorization and accounting principles as they apply to Hadoop Learn how to use mechanisms to protect data in a Hadoop cluster, both in transit and at rest Integrate Hadoop data ingest into enterprise-wide security architecture Ensure that security architecture reaches all the way to end-user access

Parallel R

Data Analysis in the Distributed World

Author: Q. Ethan McCallum,Stephen Weston

Publisher: "O'Reilly Media, Inc."

ISBN: 1449320333

Category: Computers

Page: 126

View: 1392

DOWNLOAD NOW »
It’s tough to argue with R as a high-quality, cross-platform, open source statistical software product—unless you’re in the business of crunching Big Data. This concise book introduces you to several strategies for using R to analyze large datasets, including three chapters on using R and Hadoop together. You’ll learn the basics of Snow, Multicore, Parallel, Segue, RHIPE, and Hadoop Streaming, including how to find them, how to use them, when they work well, and when they don’t. With these packages, you can overcome R’s single-threaded nature by spreading work across multiple CPUs, or offloading work to multiple machines to address R’s memory barrier. Snow: works well in a traditional cluster environment Multicore: popular for multiprocessor and multicore computers Parallel: part of the upcoming R 2.14.0 release R+Hadoop: provides low-level access to a popular form of cluster computing RHIPE: uses Hadoop’s power with R’s language and interactive shell Segue: lets you use Elastic MapReduce as a backend for lapply-style operations

Python Cookbook

Author: Alex Martelli,Anna Ravenscroft,David Ascher

Publisher: "O'Reilly Media, Inc."

ISBN: 9780596554743

Category: Computers

Page: 846

View: 7924

DOWNLOAD NOW »
Portable, powerful, and a breeze to use, Python is the popular open source object-oriented programming language used for both standalone programs and scripting applications. It is now being used by an increasing number of major organizations, including NASA and Google.Updated for Python 2.4, The Python Cookbook, 2nd Edition offers a wealth of useful code for all Python programmers, not just advanced practitioners. Like its predecessor, the new edition provides solutions to problems that Python programmers face everyday.It now includes over 200 recipes that range from simple tasks, such as working with dictionaries and list comprehensions, to complex tasks, such as monitoring a network and building a templating system. This revised version also includes new chapters on topics such as time, money, and metaprogramming.Here's a list of additional topics covered: Manipulating text Searching and sorting Working with files and the filesystem Object-oriented programming Dealing with threads and processes System administration Interacting with databases Creating user interfaces Network and web programming Processing XML Distributed programming Debugging and testing Another advantage of The Python Cookbook, 2nd Edition is its trio of authors--three well-known Python programming experts, who are highly visible on email lists and in newsgroups, and speak often at Python conferences.With scores of practical examples and pertinent background information, The Python Cookbook, 2nd Edition is the one source you need if you're looking to build efficient, flexible, scalable, and well-integrated systems.

Modern Big Data Processing with Hadoop

Expert techniques for architecting end-to-end big data solutions to get valuable insights

Author: V Naresh Kumar,Prashant Shindgikar

Publisher: Packt Publishing Ltd

ISBN: 1787128814

Category: Computers

Page: 394

View: 8561

DOWNLOAD NOW »
A comprehensive guide to design, build and execute effective Big Data strategies using Hadoop Key Features -Get an in-depth view of the Apache Hadoop ecosystem and an overview of the architectural patterns pertaining to the popular Big Data platform -Conquer different data processing and analytics challenges using a multitude of tools such as Apache Spark, Elasticsearch, Tableau and more -A comprehensive, step-by-step guide that will teach you everything you need to know, to be an expert Hadoop Architect Book Description The complex structure of data these days requires sophisticated solutions for data transformation, to make the information more accessible to the users.This book empowers you to build such solutions with relative ease with the help of Apache Hadoop, along with a host of other Big Data tools. This book will give you a complete understanding of the data lifecycle management with Hadoop, followed by modeling of structured and unstructured data in Hadoop. It will also show you how to design real-time streaming pipelines by leveraging tools such as Apache Spark, and build efficient enterprise search solutions using Elasticsearch. You will learn to build enterprise-grade analytics solutions on Hadoop, and how to visualize your data using tools such as Apache Superset. This book also covers techniques for deploying your Big Data solutions on the cloud Apache Ambari, as well as expert techniques for managing and administering your Hadoop cluster. By the end of this book, you will have all the knowledge you need to build expert Big Data systems. What you will learn Build an efficient enterprise Big Data strategy centered around Apache Hadoop Gain a thorough understanding of using Hadoop with various Big Data frameworks such as Apache Spark, Elasticsearch and more Set up and deploy your Big Data environment on premises or on the cloud with Apache Ambari Design effective streaming data pipelines and build your own enterprise search solutions Utilize the historical data to build your analytics solutions and visualize them using popular tools such as Apache Superset Plan, set up and administer your Hadoop cluster efficiently Who this book is for This book is for Big Data professionals who want to fast-track their career in the Hadoop industry and become an expert Big Data architect. Project managers and mainframe professionals looking forward to build a career in Big Data Hadoop will also find this book to be useful. Some understanding of Hadoop is required to get the best out of this book.

AWS Certified Solutions Architect Official Study Guide

Associate Exam

Author: Joe Baron,Hisham Baz,Tim Bixler,Biff Gaut,Kevin E. Kelly,Sean Senior,John Stamper

Publisher: John Wiley & Sons

ISBN: 1119139554

Category: Computers

Page: 504

View: 4562

DOWNLOAD NOW »
Validate your AWS skills. This is your opportunity to take the next step in your career by expanding and validating your skills on the AWS cloud. AWS has been the frontrunner in cloud computing products and services, and the AWS Certified Solutions Architect Official Study Guide for the Associate exam will get you fully prepared through expert content, and real-world knowledge, key exam essentials, chapter review questions, access to Sybex’s interactive online learning environment, and much more. This official study guide, written by AWS experts, covers exam concepts, and provides key review on exam topics, including: Mapping Multi-Tier Architectures to AWS Services, such as web/app servers, firewalls, caches and load balancers Understanding managed RDBMS through AWS RDS (MySQL, Oracle, SQL Server, Postgres, Aurora) Understanding Loose Coupling and Stateless Systems Comparing Different Consistency Models in AWS Services Understanding how AWS CloudFront can make your application more cost efficient, faster and secure Implementing Route tables, Access Control Lists, Firewalls, NAT, and DNS Applying AWS Security Features along with traditional Information and Application Security Using Compute, Networking, Storage, and Database AWS services Architecting Large Scale Distributed Systems Understanding of Elasticity and Scalability Concepts Understanding of Network Technologies Relating to AWS Deploying and Managing Services with tools such as CloudFormation, OpsWorks and Elastic Beanstalk. Learn from the AWS subject-matter experts, review with proven study tools, and apply real-world scenarios. If you are looking to take the AWS Certified Solutions Architect Associate exam, this guide is what you need for comprehensive content and robust study tools that will help you gain the edge on exam day and throughout your career.

Monitoring Hadoop

Author: Gurmukh Singh

Publisher: Packt Publishing Ltd

ISBN: 1783281561

Category: Computers

Page: 100

View: 7572

DOWNLOAD NOW »
This book is useful for Hadoop administrators who need to learn how to monitor and diagnose their clusters. Also, the book will prove useful for new users of the technology, as the language used is simple and easy to grasp.