Explore

Introduction to Data Engineering

Approx. 10 hours to complete

Save Course

Go to Course

Course Summary

Learn the fundamentals of data engineering and how to design, build, and maintain data systems. This course covers the key concepts of data engineering and how to use different tools to manage and process large amounts of data.

Key Learning Points

Understand the basics of data engineering and how it fits into the larger data ecosystem
Learn how to design and build data pipelines using different tools and technologies
Explore data processing and storage techniques for both batch and streaming data

Job Positions & Salaries of people who have taken this course might have

- USA: $95,000 - $145,000
- India: ₹800,000 - ₹2,000,000
- Spain: €30,000 - €50,000
- USA: $95,000 - $145,000
- India: ₹800,000 - ₹2,000,000
- Spain: €30,000 - €50,000
- USA: $110,000 - $165,000
- India: ₹1,200,000 - ₹2,500,000
- Spain: €35,000 - €60,000
- USA: $95,000 - $145,000
- India: ₹800,000 - ₹2,000,000
- Spain: €30,000 - €50,000
- USA: $110,000 - $165,000
- India: ₹1,200,000 - ₹2,500,000
- Spain: €35,000 - €60,000
- USA: $120,000 - $180,000
- India: ₹1,500,000 - ₹3,000,000
- Spain: €40,000 - €70,000

Learning Outcomes

Design and build data pipelines using different tools and technologies
Implement batch and streaming data processing techniques
Understand the fundamentals of data storage and data warehousing

Prerequisites or good to have knowledge before taking this course

Basic knowledge of programming in Python or Java
Familiarity with SQL

Course Difficulty Level

Intermediate

Course Format

Online self-paced
Video lectures
Hands-on projects

Similar Courses

Introduction to Big Data
Data Warehousing for Business Intelligence
Advanced Data Structures in Java

Related Education Paths

Notable People in This Field

Hilary Mason
Wes McKinney

Related Books

Description

This course introduces you to the core concepts, processes, and tools you need to know in order to get a foundational knowledge of data engineering. You will gain an understanding of the modern data ecosystem and the role Data Engineers, Data Scientists, and Data Analysts play in this ecosystem.

The Data Engineering Ecosystem includes several different components. It includes disparate data types, formats, and sources of data. Data Pipelines gather data from multiple sources, transform it into analytics-ready data, and make it available to data consumers for analytics and decision-making. Data repositories, such as relational and non-relational databases, data warehouses, data marts, data lakes, and big data stores process and store this data. Data Integration Platforms combine disparate data into a unified view for the data consumers. You will learn about each of these components in this course. You will also learn about Big Data and the use of some of the Big Data processing tools. A typical Data Engineering lifecycle includes architecting data platforms, designing data stores, and gathering, importing, wrangling, querying, and analyzing data. It also includes performance monitoring and finetuning to ensure systems are performing at optimal levels. In this course, you will learn about the data engineering lifecycle. You will also learn about security, governance, and compliance. Data Engineering is recognized as one of the fastest-growing fields today. The career opportunities available in the field and the different paths you can take to enter this field are discussed in the course. The course also includes hands-on labs that guide you to create your IBM Cloud Lite account, provision a database instance, load data into the database instance, and perform some basic querying operations that help you understand your dataset.

Outline

What is Data Engineering?
Welcome to Introduction to Data Engineering
Modern Data Ecosystem
Key Players in the Data Ecosystem
What is Data Engineering?
Viewpoints: Defining Data Engineering
Viewpoints: Evolution of Data Engineering
Responsibilities and Skillsets of a Data Engineer
Viewpoints: Skills and Qualities to be a Data Engineer
A Day in the Life of a Data Engineer
Summary and Highlights
Summary and Highlights
Practice Quiz
Graded Quiz
Practice Quiz
Graded Quiz

The Data Engineering Ecosystem
Overview of the Data Engineering Ecosystem
Types of Data
Understanding Different Types of File Formats
Sources of Data
Languages for Data Professionals
Viewpoints: Working with Varied Data Sources and Types
Overview of Data Repositories
RDBMS
NoSQL
Data Warehouses, Data Marts, and Data Lakes
Viewpoints: Considerations for Choice of Data Repository
ETL, ELT, and Data Pipelines
Data Integration Platforms
Viewpoints: Tools, Databases, and Data Repositories of Choice
Foundations of Big Data
Big Data Processing Tools: Hadoop, HDFS, Hive, and Spark
Viewpoints: Impact of Big Data on Data Engineering
Summary and Highlights
Summary and Highlights
Summary and Highlights
Practice Quiz
Graded Quiz
Practice Quiz
Graded Quiz
Practice Quiz
Graded Quiz

Data Engineering Lifecycle
Architecting the Data Platform
Factors for Selecting and Designing Data Stores
Security
Viewpoints: Importance of Data Security
How to Gather and Import Data
Data Wrangling
Tools for Data Wrangling
Querying and Analyzing Data
Performance Tuning and Troubleshooting
Governance and Compliance
Summary and Highlights
Summary and Highlights
Summary and Highlights
Summary and Highlights
Optional: Overview of the DataOps Methodology
Practice Quiz
Graded Quiz
Practice Quiz
Graded Quiz
Practice Quiz
Graded Quiz
Practice Quiz
Graded Quiz

Career Opportunities and Data Engineering in Action
Career Opportunities in Data Engineering
Viewpoints: Get into Data Engineering
Data Engineering Learning Path
Viewpoints: What Do Employers Look for in a Data Engineer
Viewpoints: The Many Paths to Data Engineering
Viewpoints: Advice to Aspiring Data Engineers
Summary and Highlights
Congratulations and Next Steps
Practice Quiz
Graded Quiz
Final Quiz

Summary of User Reviews

Introduction to Data Engineering course on Coursera received positive reviews from many users. The course covers key aspects of data engineering in an easy-to-understand manner. Many users appreciated the practical approach taken by the instructor while teaching the course.

Key Aspect Users Liked About This Course

Practical approach to learning data engineering

Pros from User Reviews

Easy-to-understand explanations of complex concepts
Practical assignments that help to reinforce learning
Engaging instructor who keeps the course interesting
In-depth coverage of key data engineering topics
Well-structured course materials

Cons from User Reviews

Some users felt that the course was too short
Not suitable for those looking for advanced-level data engineering course
Limited interaction with other students
Some technical issues reported with the platform

Recommended for you

Predictive Analytics Final Project

Apply your predictive modelling acumen in a business case setting. The final project brings together the skills and knowledge acquired throughout the MicroMasters programme....

Save Course

Fundamental pharmaceutical selling skills

Sales planning*the adaptive selling*Detailing>>Features>>Benefits>>Value* Verbal&nonverbal communication*win-win closing I will share with you the best practices of my 28 years of experience in the pharmaceutical and healthcare field...

Save Course

Bayesian Statistics: From Concept to Data Analysis

This course introduces the Bayesian approach to statistics, starting with the concept of probability and moving to the analysis of data....

Save Course

ServiceNow Certified System Administrator (CSA): Quebec 2021

Prepare for ServiceNow CSA exam with the most comprehensive practice tests with detailed answers, screenshots and links Exam Purpose Exam Preparation...

Save Course