Home Create your First Azure Data Factory
Post
Cancel

Create your First Azure Data Factory

Reference

Bookmark from the original Notion lesson

Overview: Want to copy a file from one folder to another folder using copy activity in ADF.

  1. Create Storage Account and a Container

    Create Storage Account and a Container

  2. Create an Input Folder and place a file that we want to copy.

    Create an input folder and place a file

  3. Create a Data Factory

    Create a Data Factory

  4. Go to ADF and goto Author and Monitor

    Go to ADF and Author and Monitor

  5. Create a Linked Service: Since for our demo, our ADF needs to interact with an external resource which in our case is the storage account we have to create a linked service.

    Create a linked service

  6. Create a Dataset for source and sink

    1. Source Dataset

      Create source dataset

      Configure source dataset

      Source dataset file settings

    2. Sink Dataset

      Create sink dataset

  7. Create a Pipeline

    Create a pipeline

    Configure copy activity pipeline

    Pipeline copy activity settings

  8. Validate your pipeline. Incase we forget to configure anything in our pipeline then it will notify us.

  9. Publish our pipeline. Once we publish, all the pipelines, linked services, datasets and triggers will be published in our ADF.

  10. Incase we just want to check if our pipeline is running as intended for debugging purpose before publishing we can use the Debug mode.

Source

Based on my Notion lesson page: 3. Create your First Azure Data Factory.

This post is licensed under CC BY 4.0 by the author.