DP-3027 Implement a data engineering solution with Azure Databricks - ITEL
DP-3027 Implement a data engineering solution with Azure Databricks
ENQUIRE NOW!
Course Code:
DP-3027
Duration:
1 Day
Next Available Course Date:
Contact Us
Delivery Mode:
Instructor-led training (ILT)/Online Learning (OLL)
Fees:
$817.5

Other Available Course Dates:

*Note: Please enquire within for available Saturday schedules.

Learn how to harness the power of Apache Spark and powerful clusters running on the Azure Databricks platform to run large data engineering workloads in the cloud.

Course Outline

Lesson : Implement a data engineering solution with Azure Databricks

  • Perform incremental processing with spark structured streaming
    • Introduction
    • Set up real-time data sources for incremental processing
    • Optimize Delta Lake for incremental processing in Azure Databricks
    • Handle late data and out-of-order events in incremental processing
    • Monitoring and performance tuning strategies for incremental processing in Azure Databricks
    • Exercise – Real-time ingestion and processing with Delta Live Tables with Azure Databricks
  • Implement streaming architecture patterns with Delta Live Tables
    • Introduction
    • Event driven architectures with Delta Live tables
    • Ingest data with structured streaming
    • Maintain data consistency and reliability with structured streaming
    • Scale streaming workloads with Delta Live tables
    • Exercise – end-to-end streaming pipeline with Delta Live tables
  • Optimize performance with Spark and Delta Live Tables
    • Introduction
    • Optimize performance with Spark and Delta Live Tables
    • Perform cost-based optimization and query tuning
    • Use change data capture (CDC)
    • Use enhanced autoscaling
    • Implement observability and data quality metrics
    • Exercise – optimize data pipelines for better performance in Azure Databricks
  • Implement CI/CD workflows in Azure Databricks
    • Introduction
    • Implement version control and Git integration
    • Perform unit testing and integration testing
    • Manage and configure your environment
    • Implement rollback and roll-forward strategies
    • Exercise – Implement CI/CD workflows
  • Automate workloads with Azure Databricks Jobs
    • Introduction
    • Implement job scheduling and automation
    • Optimize workflows with parameters
    • Handle dependency management
    • Implement error handling and retry mechanisms
    • Explore best practices and guidelines
    • Exercise – Automate data ingestion and processing
  • Manage data privacy and governance with Azure Databricks
    • Introduction
    • Implement data encryption techniques in Azure Databricks
    • Manage access controls in Azure Databricks
    • Implement data masking and anonymization in Azure Databricks
    • Use compliance frameworks and secure data sharing in Azure Databricks
    • Use data lineage and metadata management
    • Implement governance automation in Azure Databricks
    • Exercise – Practice the implementation of Unity Catalog
  • Use SQL Warehouses in Azure Databricks
    • Introduction
    • Get started with SQL Warehouses
    • Create databases and tables
    • Create queries and dashboards
    • Exercise – Use a SQL Warehouse in Azure Databricks
  • Run Azure Databricks Notebooks with Azure Data Factory
    • Introduction
    • Understand Azure Databricks notebooks and pipelines
    • Create a linked service for Azure Databricks
    • Use a Notebook activity in a pipeline
    • Use parameters in a notebook
    • Exercise – Run an Azure Databricks Notebook with Azure Data Factory

Course Fee (inclusive of 9% GST)

$817.5 *Funding not available
Course Prerequisite
None
Important Notes
Course Completion Requirements:
  1. Attended at least 75% of the course
  2. Declared as competent during the assessments
Who Should Attend?
Data Engineers, Data Scientists, ELT Developers learn how to harness the power of Apache Spark and powerful clusters running on the Azure Databricks platform to run large data engineering workloads in the cloud.

Why ITEL?

  • Diverse Range of IT Courses

    We offer a diverse range of IT courses tailored to student needs. Our curriculum covers foundational to advanced topics, ensuring comprehensive learning. We stay updated with industry trends to deliver relevant courses.

  • Skilled & Experienced Instructors

    We are proud to have a team of highly skilled and experienced instructors. Our instructors are industry professionals with in-depth knowledge and expertise across various IT domains.

  • Practical & Hands-on Exercises

    Our courses feature hands-on exercises, projects, and simulations to build practical skills. Students gain confidence by applying knowledge to real-world scenarios.

Course Enquiry for DP-3027 Implement a data engineering solution with Azure Databricks

"*" indicates required fields

This field is for validation purposes and should be left unchanged.
I want to find out more about:*
Where did you hear of ITEL?*
*Note: If you chose Others, kindly provide more information in the Remarks/Comments/Questions box.
Consent*
Newsletter Subscription

Download Windows Server 2019 Administration (SF) Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Download Cloud Technology Solutions Architect Associate (SF) Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Download Security Analysis and Penetration Testing (SF) Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Download Information Systems Security (SF) [OLL] Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Download Microsoft Certified: Azure Fundamentals (AZ-900) Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Download Microsoft Certified Azure Network Engineer Associate (AZ-700) Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Download Cyber Threat Intelligence Analyst (SF) Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Download Cyber Forensics Investigator (SF) [OLL] Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Download Cyber Security Incident Handling and Response (SF) Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Download Certified Ethical Hacker (CEH) (SF) [OLL] Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Download Certified Ethical Hacker (CEH) (SF) Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Download (SCTP) Advanced Certificate in Cyber Security Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Download (SCTP) Higher Certificate in Network Administration Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Download Strategic Cyber Security Leadership (SF) Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Download Cisco Certified Network Associate (CCNA) (SF) Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Course Listing Download Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List

Cisco Certified Network Associate Download Course Brochure

This field is for validation purposes and should be left unchanged.
Sign Up For Our Mailing List