Apache Spark and Scala training in Hyderabad
Apache spark:
Apache Spark is a lightning-fast cluster computing designed for fast computation. It is based on Hadoop MapReduce and it extends the MapReduce model to efficiently use it for more types of computations which includes interactive queries and stream processing. The main feature of Spark is its in-memory cluster computing that increases the processing speed of an application.
Apache Spark is a lightning-fast cluster computing designed for fast computation. It is based on Hadoop MapReduce and it extends the MapReduce model to efficiently use it for more types of computations which includes interactive queries and stream processing. The main feature of Spark is its in-memory cluster computing that increases the processing speed of an application.
Spark is designed to cover a wide range of workloads such as batch applications, iterative algorithms, interactive queries and streaming. Apart from supporting all these workload in a respective system, it reduces the management burden of maintaining separate tools.
Course Content:
- History of Big Data & Apache Spark
- Introduction to the Spark Shell and the training environment
- Intro to Spark DataFrames and Spark SQL
- Introduction to RDDs
- Lazy Evaluation
- Transformations and Actions
- Caching
- Using the Spark UIs
- Data Sources: reading from Parquet, S3, Cassandra, HDFS, and your local file system
- Spark's Architecture
- Programming with Accumulators and Broadcast variables
- Debugging and tuning Spark jobs using Spark's admin UIs
- Memory & Persistence
- Advanced programming with RDDs (understanding the shuffle phase, partitioning, etc.)
- Visualization: matplotlib, gg_plot, dashboards, exploration and visualization in notebooks
- Introduction to Spark Streaming
- Introduction to MLlib and GraphX
SCALA Programing
Scala is a general purpose programming language designed to express common programming patterns in a concise, elegant, and type-safe way. It smoothly integrates features of object- oriented and functional languages, enabling Java and other programmers to be more productive.
Course Contents:
- A brief history of the Java platform to date
- Distinguishing between the Java language and platform
- Pain points when using Java for software development
- Possible criteria for an improved version of Java
- How and why the Scala language was created
- Everything is an object
- Class declarations
- Data typing
- Operators and methods
- Pattern matching
- Functions
- Anonymous and nested functions
- Traits
- Built in types, literals and operators
- Testing for equality of state and reference
- Conditionals, simple matching and external iteration
- Working with lists, arrays, sets and maps
- Throwing and catching exceptions
- Adding annotations to your code
- Using standard Java libraries
- A minimal class declaration
- Understanding primary constructors
- Specifying alternative constructors
- Declaring and overriding methods
- Creating base classes and class hierarchies
- Creating traits and mixing them into classes
- How a Scala inheritance tree is linearized
- Advanced uses of for expressions
- Understanding function values and closures
- Using closures to create internal iterators
- Creating and using higher order functions
- Practical examples of higher order functions
- Currying and partially applied functions
- Creating your own Domain Specific Languages(DSL's)
- Using the match keyword to return a value
- Using case classes for pattern matching
- Adding pattern guards to match conditions
- Partially specifying matches with wildcards
- Deep matching using case constructors
- Matching against collections of items
- Using extractors instead of case classes
- Writing standard JUnit tests in Scala
- Conventional TDD using the ScalaTest tool
- Behavior Driven Development using ScalaTest
- Using functional concepts in TDD
- Working with XML literals in code
- Embedding XPath like expressions
- Using Pattern Matching to process XML data
- Serializing and deserializing to and from XML
- Issues with conventional approaches to multi-threading
- How an actor-based approach helps you write thread-safe code
- The Scala architecture for creating actor-based systems
- Different coding styles supported by the actor model