If you have workflows on the Alteryx Analytics Cloud Platform and are curious to know how you can go beyond simply running your workflows on a schedule, try automating your data pipelines using Alteryx Analytics Cloud Plans.
What is a Plan?
A Plan on the Alteryx Analytics Cloud Platform enables you to execute workflows and other tasks by defining the sequence in which you would like them to be executed and orchestrated.
For example, when executing Designer Cloud workflow tasks, you can also execute downstream tasks such as HTTP calls or Slack alerts as part of the plan. Plans give you full control of your pipelines by allowing you to integrate with downstream systems and perform advanced actions while orchestrating your data pipelines, such as triggering webhook endpoints or calling external APIs.
Plans can be run manually or be configured to run on a schedule.
Plans are easy to build and use – they are created using an interface that looks and feels like Designer Cloud’s visual builder canvas.
Why Create a Plan?
Plans allow you to operationalize and automate your data pipelines on the Alteryx Analytics Cloud Platform so that you can achieve true analytics automation in the cloud.
For advanced automation, when creating a Plan, you have flexibility regarding task execution with advanced features like HTTP Tasks, Slack Tasks, Conditional Execution, Parallel Execution, and more.
Moreover, if you are running several different workflows, instead of running a workflow in its entirety every time, Plans allow you to configure specific steps within a workflow to run as tasks within a Plan — without the need to rely on APIs or external tooling.
Overall, Plans allow for powerful analytics automation. To illustrate the power of Plans, let’s go through a simple use case:
Plans in Practice
You are an analyst of a consumer-packaged-goods company that sells products to multiple retailers, and you want to create an automated stream of events to expedite your analytics workflows.
Using Designer Cloud, a data prep solution that is part of the Alteryx Analytics Cloud Platform, you prepare and clean data from multiple retailers. Then, with Plans, you orchestrate the workflow to run on a regular schedule, automatically ingesting and cleaning data from your retailers on a monthly basis.
When the automated Plan runs, you are notified on success or failure via Slack, Microsoft Teams, or a task triggered by an HTTP webhook, which allows for integration with third-party apps, such as using Twilio to send direct SMS notifications to your phone.
Plans can also take the data output from our Designer Cloud workflow and use it as an input dataset in a downstream machine learning prediction task. For example, we can take our retail data and use it to predict seasonal trends, the fastest growing markets, and much more with the power of Alteryx Machine Learning.
If using the full Alteryx Analytics Cloud Platform, we could use the data we prepared earlier in Designer Cloud to feed other apps such as Reporting or Alteryx Auto Insights to quickly and easily discover the insights we need to make informed decisions.
Let’s look at some of the tasks that are currently available in Plans (May, 2023):
Plan Tasks
Workflow Task
By adding a Workflow Task in a plan, you can:
· Orchestrate the run of a Designer Cloud workflow you have built
· Schedule the run of that workflow
· Use the output from the run to feed downstream tasks
Machine Learning Prediction Task
By adding a Machine Learning Prediction Task in a plan, you can:
· Run a machine learning prediction based on input data and output a table of predictions based on that data
· Connect a Designer Cloud Task output to feed your machine learning models
Slack Task
By adding a Slack Task in a plan, you can:
· Send a Slack message to a specific channel
· Send a Slack direct message
A simple use case is to send yourself a Slack message when a task in a Plan fails (or succeeds)
HTTP Task
By adding an HTTP task in a Plan, you can:
· Send a Webhook Notification
· Execute a Plan from another Plan
· Make a request to any external API (that you have access to). An HTTP Task use case is sending yourself a Microsoft Teams message when a Plan fails – see this blog post on how to setup an HTTP Task for Microsoft Teams
Other Features of Plans
Execution Anchors
As the name suggests, the anchors in a Plan task allows you to specify the status of a task execution as a condition for running a task. You can choose one of the following statuses as a condition to execute the subsequent tasks:
· On success: This is the default status; only on the successful execution of the task should the subsequent task execute
· On failure: Think of this as a fallback condition to execute a task when the previous one fails
· Always: This ignores the success or failure of the task in question and results in the execution of the subsequent task whenever the prior task is executed, regardless of outcome
As suggested before, you can send yourself a notification on failed executions by defining a condition and running an HTTP task on that node. Similarly, you can choose to notify project stakeholders by email on successful executions, or both on success and failure, depending on your use case and preferences.
Parallel Execution
Running tasks in parallel can save time and increase the efficiency of your Plans. For instance, trying to transform and move two files into two separate tables will be faster when the respective tasks are run in parallel rather than sequentially.
Do keep in mind that for tasks to run in parallel, there must be no dependencies between them.
Plans Design page vs. Plan Run page
Plans are built in the Plans Design Phase canvas. This allows you configure plans, add tasks, schedule your plan, and run your Plan.
When you proceed to run a Plan, you are brought to the Plan Run page, where you can view useful information on your historical Plan runs. The page provides helpful information such as when your plans started and finished. It also tells you the status of your plans, whether your plans failed or succeeded, and how long it took to get to that status. Importantly, from the Plan Run page, you can also click into each task and see the particular inputs and outputs, giving you intricate details on the health of your plans and allowing you to analyze each step of your orchestrated pipeline.
Restarting Plans and Retrying from Failure
If your plan fails, try restarting the plan by clicking the Retry From Failed button. Clicking this button runs the plan again beginning with the task that failed and all downstream tasks. Upstream tasks that have already run successfully are skipped, ensuring that you don’t waste resources re-running unnecessary steps.
If multiple tasks fail, then all the tasks that failed and their downstream tasks will be run again. If you would like to run the plan from the beginning, go back to the Plan Design Page and click “Run” again.
Scheduling Plans
After you have built your robust Plans pipelines, you can choose when to run your Plan by using our Scheduling feature. This feature allows users to:
· Pick the frequency at which to run the Plan
· Pick the time and day to run the Plan
What is in Store for Plans
With the Alteryx Analytics Cloud Platform expanding to support many apps, Plans will also be updated to be able to be used with these apps.
Conclusion
Plans extends the power of the Alteryx Analytics Cloud Platform by making it easy to schedule, automate, and orchestrate your data pipelines.
If you want to enhance your cloud analytics automation and orchestration capabilities, be sure to take a look at the capabilities offered by Alteryx Plans.
If you are curious to learn more about Plans, schedule a demo here.