This course covers methods and practices to implement data engineering solutions by using Microsoft Fabric. Students will learn how to design and develop effective data loading patterns, data architectures, and orchestration processes. Objectives for this course include ingesting and transforming data and securing, managing, and monitoring data engineering solutions. This course is designed for experienced data professionals skilled at data integration and orchestration, such as those with the DP-203: Azure Data Engineer certification.
Course Outline
Lesson 1: Ingest Data with Dataflows Gen2 in Microsoft Fabric
- Introduction
- Understand Dataflows Gen2 in Microsoft Fabric
- Explore Dataflows Gen2 in Microsoft Fabric
- Integrate Dataflows Gen2 and Pipelines in Microsoft Fabric
- Exercise – Create and use a Dataflow Gen2 in Microsoft Fabric
- Module assessment
- Summary
Lesson 2: Orchestrate processes and data movement with Microsoft Fabric
- Introduction
- Understand pipelines
- Use the Copy Data activity
- Use pipeline templates
- Run and monitor pipelines
- Exercise – Ingest data with a pipeline
- Module assessment
- Summary
Lesson 3: Use Apache Spark in Microsoft Fabric
- Introduction
- Prepare to use Apache Spark
- Run Spark code
- Work with data in a Spark dataframe
- Work with data using Spark SQL
- Visualize data in a Spark notebook
- Exercise – Analyze data with Apache Spark
- Module assessment
- Summary
Lesson 4: Get started with Real-Time Intelligence in Microsoft Fabric
- Introduction
- What is real-time data analytics?
- Real-Time Intelligence in Microsoft Fabric
- Ingest and transform real-time data
- Store and query real-time data
- Visualize real-time data
- Automate actions
- Exercise – Explore Real-Time Intelligence in Fabric
- Module assessment
- Summary
Lesson 5: Use real-time eventstreams in Microsoft Fabric
- Introduction
- Components of eventstreams
- Eventstream sources and destinations
- Eventstream transformations
- Exercise – Ingest real-time data with Eventstream in Microsoft Fabric
- Module assessment
- Summary
Lesson 6: Work with real-time data in a Microsoft Fabric eventhouse
- Introduction
- Get started with an eventhouse
- Use KQL effectively
- Materialized views and stored functions
- Exercise – Work with data in a Microsoft Fabric eventhouse
- Module assessment
- Summary
Lesson 7: Implement a Lakehouse with Microsoft Fabric
Introduction to end-to-end analytics using Microsoft Fabric
- Introduction
- Explore end-to-end analytics with Microsoft Fabric
- Explore data teams and Microsoft Fabric
- Enable and use Microsoft Fabric
- Module assessment
- Summary
Lesson 8: Get started with lakehouses in Microsoft Fabric
- Introduction
- Explore the Microsoft Fabric lakehouse
- Work with Microsoft Fabric lakehouses
- Explore and transform data in a lakehouse
- Exercise – Create a Microsoft Fabric lakehouse
- Module assessment
- Summary
Lesson 9: Use Apache Spark in Microsoft Fabric
- Introduction
- Prepare to use Apache Spark
- Run Spark code
- Work with data in a Spark dataframe
- Work with data using Spark SQL
- Visualize data in a Spark notebook
- Exercise – Analyze data with Apache Spark
- Module assessment
- Summary
Lesson 10: Work with Delta Lake tables in Microsoft Fabric
- Introduction
- Understand Delta Lake
- Create delta tables
- Optimize delta tables
- Work with delta tables in Spark
- Use delta tables with streaming data
- Exercise – Use delta tables in Apache Spark
- Module assessment
- Summary
Lesson 11: Ingest Data with Dataflows Gen2 in Microsoft Fabric
- Introduction
- Understand Dataflows Gen2 in Microsoft Fabric
- Explore Dataflows Gen2 in Microsoft Fabric
- Integrate Dataflows Gen2 and Pipelines in Microsoft Fabric
- Exercise – Create and use a Dataflow Gen2 in Microsoft Fabric
- Module assessment
- Summary
Lesson 12: Orchestrate processes and data movement with Microsoft Fabric
- Introduction
- Understand pipelines
- Use the Copy Data activity
- Use pipeline templates
- Run and monitor pipelines
- Exercise – Ingest data with a pipeline
- Module assessment
- Summary
Lesson 13: Organize a Fabric lakehouse using medallion architecture design
- Introduction
- Describe medallion architecture
- Implement a medallion architecture in Fabric
- Query and report on data in your Fabric lakehouse
- Considerations for managing your lakehouse
- Exercise – Organize your Fabric lakehouse using a medallion architecture
- Module assessment
- Summary
Lesson 14: Implement Real-Time Intelligence with Microsoft Fabric
Get started with Real-Time Intelligence in Microsoft Fabric
- Introduction
- What is real-time data analytics?
- Real-Time Intelligence in Microsoft Fabric
- Ingest and transform real-time data
- Store and query real-time data
- Visualize real-time data
- Automate actions
- Exercise – Explore Real-Time Intelligence in Fabric
- Module assessment
- Summary
Lesson 15: Use real-time eventstreams in Microsoft Fabric
- Introduction
- Components of eventstreams
- Eventstream sources and destinations
- Eventstream transformations
- Exercise – Ingest real-time data with Eventstream in Microsoft Fabric
- Module assessment
- Summary
Lesson 16: Work with real-time data in a Microsoft Fabric eventhouse
- Introduction
- Get started with an eventhouse
- Use KQL effectively
- Materialized views and stored functions
- Exercise – Work with data in a Microsoft Fabric eventhouse
- Module assessment
- Summary
Lesson 17: Create Real-Time Dashboards with Microsoft Fabric
- Introduction
- Get started with real-time dashboards
- Advanced features
- Real-Time Dashboard best practices
- Exercise – Get started with real-time dashboards
- Module assessment
- Summary
Lesson 18: Implement a data warehouse with Microsoft Fabric
Introduction to end-to-end analytics using Microsoft Fabric
- Introduction
- Explore end-to-end analytics with Microsoft Fabric
- Explore data teams and Microsoft Fabric
- Enable and use Microsoft Fabric
- Module assessment
- Summary
Lesson 19: Get started with data warehouses in Microsoft Fabric
- Introduction
- Understand data warehouse fundamentals
- Understand data warehouses in Fabric
- Query and transform data
- Prepare data for analysis and reporting
- Secure and monitor your data warehouse
- Exercise – Analyze data in a data warehouse
- Module assessment
- Summary
Lesson 20: Load data into a Microsoft Fabric data warehouse
- Introduction
- Explore data load strategies
- Use data pipelines to load a warehouse
- Load data using T-SQL
- Load and transform data with Dataflow Gen2
- Exercise: Load data into a warehouse in Microsoft Fabric
- Module assessment
- Summary
Lesson 21: Query a data warehouse in Microsoft Fabric
- Introduction
- Query data
- Use the SQL query editor
- Explore the visual query editor
- Use client tools to query a warehouse
- Exercise: Query a data warehouse in Microsoft Fabric
- Module assessment
- Summary
Lesson 22: Monitor a Microsoft Fabric data warehouse
- Introduction
- Monitor capacity metrics
- Monitor current activity
- Monitor queries
- Exercise – Monitor a data warehouse in Microsoft Fabric
- Module assessment
- Summary
Lesson 23: Secure a Microsoft Fabric data warehouse
- Introduction
- Explore dynamic data masking
- Implement row-level security
- Implement column-level security
- Configure SQL granular permissions using T-SQL
- Exercise: Secure a warehouse in Microsoft Fabric
- Module assessment
- Summary
Lesson 24: Manage a Microsoft Fabric environment
Implement continuous integration and continuous delivery (CI/CD) in Microsoft
Fabric
- Introduction
- Understand Continuous Integration and Continuous Delivery (CI/CD)
- Implement version control and Git integration
- Implement deployment pipelines
- Automate CI/CD using Fabric APIs
- Exercise: Implement deployment pipelines in Microsoft Fabric
- Module assessment
- Summary
Lesson 25: Monitor activities in Microsoft Fabric
- Introduction
- Understand monitoring
- Use Microsoft Fabric Monitor Hub
- Take action with Microsoft Fabric Activator
- Exercise – Monitor Fabric activity in the Monitor hub
- Module assessment
- Summary
Lesson 26: Secure data access in Microsoft Fabric
- Introduction
- Understand the Fabric security model
- Configure workspace and item permissions
- Apply granular permissions
- Exercise: Secure data access in Microsoft Fabric
- Module assessment
- Summary
Lesson 27: Administer a Microsoft Fabric environment
- Introduction
- Understand the Fabric Architecture
- Understand the Fabric administrator role
- Manage Fabric security
- Govern data in Fabric
- Module assessment
- Summary