Using Data Flows

The process of mapping data flows refers to visually designed data transformations in the Azure Data Factory. Also, the process of data flow permits data engineers to develop data transformation logic without writing code. Thereby, the resulting data flow are executed as activities within Azure Data Factory pipelines that use scaled-out Apache Spark clusters. Some of the important features of data flow include –

• Firstly, data flow activities can be operationalized using existing Azure Data Factory scheduling, control, flow, and monitoring capabilities.
• Secondly, mapping data flow provides an entirely visual experience with no coding required.
• Thirdly, data flow runs on ADF-managed execution clusters for scaled-out data processing.
• Lastly, Azure Data Factory handles all the code translation, path optimization, and execution of your data flow jobs.

Steps to Create Data Flows

• Firstly, data flows are created from the factory resources pane like pipelines and datasets. 
• Secondly,  to create a data flow, select the plus sign next to Factory Resources, and then select Data Flow.
• Thirdly, the action takes you to the data flow canvas, where you can create your transformation logic.
• Then, select Add source to start configuring your source transformation. 

Elements of Data Flow

• Authoring data flows: Mapping data flow has a unique authoring canvas designed to make building transformation logic easy. 
• Graph: The graph displays the transformation stream and displays the lineage of source data as it flows into one or more sinks.
• Configuration Panel: The configuration panel shows the settings specific to the currently selected transformation. If no transformation is selected, it shows the data flow.
• Transformation settings: The first tab in each transformation’s configuration pane contains the settings specific to that transformation. 
• Optimize: The Optimize tab contains settings to configure partitioning schemes. 
• Inspect: The Inspect tab provides a view into the metadata of the data stream that is being transformed. Inspect is a read-only view of your metadata. It is not required to have the debug mode enabled to see metadata in the Inspect pane.
• Data preview: In case the debug mode is on, the Data Preview tab provides you with an interactive snapshot of the data at each transform.

For more visit: Available Transformations