Mini Project Development for Azure Data Engineer

Learn All in One Azure Data Engineering services along with hands-on (requirement gathering to implementation)

------------------------------------------------------------------------------------------------------------------

Recently added sessions:

• Added code snippets for Databricks <-> SQL Server using JDBC

• Data Generator code added. Now you can generate your own datasets for this workshop.

• Copy data from Azure SQL to Azure SQL without ADF (kind of linked server)

• ADF Code promotion - manual activity

• Dynamically change DB connection in ADF (Parameterizing linked service)

• Visible comparison of Blob and Azure Data Lake Gen2 storage using data analytics through Azure Databricks.

• NY Taxi data files source link added in resources section

• Integrate Key Vault with Azure Databricks and extract secrets via Secret Scope

• Use Blob SDK in Azure Databricks to read/write files on blob like IO operations

• Connecting to SQL Server with pyodbc in Azure Databricks

• Datasets used in this course are now available to download. Visit Azure Data Factory section.

------------------------------------------------------------------------------------------------------------------

We will execute a small project. It will help you to learn:

1) Design concepts - How to design big data system. Terminologies of each components/layers.

2) Hands-on - We will be creating required Azure data services and will look various options available.

3) Tips and Tricks

4) Helps in DP-200 and DP-201 certification preparation.

Technologies you will learn:

Azure Data Factory V2, Databricks (using Standard cluster), Storage Services, SQL Synapse Analytics (DB part)

Course highlights:

1) Resources: Links to external resource for further reading

2) Datasets : Download and use dataset used in this course. Do more practice.

3) Architecture and Design : Start learning on how to deign the data load on cloud platform.