Power Automate license enforcement - looking at it from a Dynamics 365 CE perspective

Mid May 2023, a warning popped up in Microsoft Message Center regarding "Non-Compliant Power Automate Flows" -  soon after the message however disappeared so this got probably missed by the majority of Dataverse and Dynamics 365 admins.

This however raised some concerns in the broader Microsoft community ( see Will Power Automate enforcement licensing kill your flows?  and Upcoming licensing enforcement in Power Automate explained ). 

To be honest I did not pay a lot of attention since the message apparently vanished in thin air and after consultation with Microsoft support they said that this message was sent prematurely. But then, end of October another warning popped up.

So it seems that Microsoft is finally cracking down on Power Automate flows which are not associated with a properly licensed user for premium connectors or Power Automate flows not directly linked to a Power App. When you built your own model-driven app on top of Dynamics 365 Customer Engagement (CE) which uses Power Automate flows, you will need to associate the Power Automate Flow with with your new app.

There is a PowerShell script to identify the flows which at risk to be turned off  across your tenant - see I have many environments - how can I get the flows that need my attention across tenants in the Power Automate Licensing FAQ - which uses the Get-AdminFlowAtRiskOfSuspension cmdlet

The Get-AdminFlowAtRiskOfSuspension cmdlet is part of a separate PowerShell module which you can install using Install-Module -Name Microsoft.PowerApps.Administration.PowerShell. It will run a scan of your environments and outline

Check out Associate flows with apps - Power Automate | Microsoft Learn on how you need to link up a flow with an app (see below screenshot on where to do this in the Power Automate flow detail screen). If you make this change on a flow which is a part of a solution, then the associations will be part of the solution file and can be transported cross environments.

Related articles/blog posts:

• Will Power Automate enforcement licensing kill your flows?
• Upcoming licensing enforcement in Power Automate explained
• Flows in app context - is there a clear definition of what it means?
• How do you license Power Automate flows when using them with Power Apps

Dynamics 365 and Power Platform monthly reading list April 2024

Copilots, AI and machine learning

• Artificial Intelligence 2023 Playlist by Stanford Online (YouTube) and Stanford CS229: Machine Learning | Spring 2022 Stanford Online (YouTube)
• Use Semantic Kernel with Lakehouse in Microsoft Fabric
• Fabric end-to-end use case: Analytics engineering part 1 - dbt with the Lakehouse by @s_debruyn
• From Microsoft to global brands, Dynamics 365 Copilot is helping transform customer experiences across service, sales and marketing
• Accelerate support ramp-ups, handoffs and wrap-ups with Copilot summarization in Dynamics 365 Customer Service
• Introducing new Copilot experiences to boost productivity and elevate customer experiences across the organization
• Transform your business with Microsoft AI (Microsoft Learning Path)
• Accelerating revenue with AI Powered Solutions: a CRO Guide (e-book)
• Microsoft Azure AI Fundamentals: Generative AI (Learning path)
• 10 AI terms everyone should know
• Revolutionizing marketing workflows with Copilot in Dynamics 365 Customer Insights
• Creating Custom Copilots: Azure AI Studio and Copilot Studio. When to use what?
• Build your Copilot testing strategy in Dynamics 365 Customer Service
• Learn Live: Microsoft Learn AI Skills Challenge
• The Art and Science of working with AI
• Microsoft Copilot in Azure extends capabilities to Azure SQL Database (Private preview)
• How to connect Copilot Studio to Dynamics 365 Customer Service (YouTube)
• Introducing Copilot pane in Power BI Desktop (preview)
• Enable Copilot case and conversation summaries 
• Trust Micrsoft Copilot with your Power Platform and Dynamics 365 data - Power CAT live (YouTube)
• Building Modern Enterprise Solutions: Microsoft Dataverse and Microsoft Copilot
• Microsoft Copilot scenario library - real use cases for HR,Finance, IT, Marketing and Sales
• Microsoft Copilot for Sales FAQ
• Day 19: Microsoft Copilot vs Microsoft Copilot for Microsoft 365 vs Copilot aka PVA
• Announcing Microsoft Copilot for Service
• Enable Copilot in Customer Service and custom apps (Dynamics 365)
• Microsoft Copilot for Service: Introducing new Copilot experiences to boost productivity and elevate customer experiences across the organization  and Copilot in Microsoft Dynamics 365 (YouTube)
• Generative AI for beginners (Microsoft)
• Large language models, explained with a minimum of math and jargon
• Using copilot with SharePoint knowledge to serve your customers
• Dynamics 365 Customer Insights - Deliver higher-impact experiences with better data quality and configuration with Copilot
• Information Assistant Industry accelerator on GitHub
• Announcing Microsoft Copilot for Service and Microsoft Copilot for Service product page
• Build your own copilots with Azure AI Studio (Microsoft Mechanics recording)
• Build GPT-automated customer support with Azure Communication Services

 

Technical topics (Configuration, customization and extensibility)

• Lakehouse vs Warehouse vs Datamart - the difference between the three Fabric objects
• Power Platform Well-Architected framework
• Creating service principals really easily using PAC CLI by @carldesouza
• The power of Prefer Headers in the  Dynamics CRM web API
• Analyze Dataverse tables from Microsoft Fabric
• Enterprise security with Power Platform (Microsoft white paper)
• Microsoft Fabric, explained for existing Synapse users
• Announcing new cloud governance guidance in the Microsoft Cloud Adoption Framework (CAF) for Azure
• PCF Learning Roadmap
• Mask sensitive data and prevent unauthorized access and Dataverse: about masking rules 
• Protect enterprise solutions with new Microsoft Power Platform security features
• Power Platform Wave 1 2024: Top 10 features you need to know! (YouTube)
• Add intelligence into your apps with Dataverse AI functions (YouTube)
• Leverage the Power Platform API to create a capacity report
• Associate activities with multiple related records (2023 Wave 2 Public preview)
• Power Platform tenant settings  (YouTube)
• Working with solution dependencies for objects made easier
• Improve solution import time
• https://aka.ms/azai - develop apps that use Azure AI services
• How to create a new Dynamics 365 SCM trial environment from the Power Platform Admin Center (PPAC) preview
• What's new in Playwright 1.32 (YouTube)
• Using Application Insights for Dataverse monitoring
• Modernize customer support with Copilot in Dynamics 365 Customer Service
• Announcing the Power Platform solutions with AI and Copilot learning path
• Sample library for Digital Contact Center Platform

Topics for Dynamics 365 Business Applications Platform consultants, project managers and power users

• Modernize Service Guided Tour
• Forrester TEI study shows 315% ROI when modernizing customer service with Microsoft Dynamics 365 Customer Service
• End of support for Project Service Automation (PSA 3.x)
• Model driven Apps (Power Platform)  visual glossary
• My annual letter: leading in a new era - annual letter from Satya Nadella, CEO at Microsoft
• Finding adequate metrics for  outer,inner and process quality in software development
• Customer story: Avanade puts people first, accelerates sales productivity with Microsoft Copilot for Sales
• Keeping up to date with changes to the Release Wave Plans
• A consultant's guide: overcoming the challenges of joining a Microsoft Dynamics 365 project mid-way through
• Accelerate Copilot extensibility and Power Apps creation with Microsoft Dataverse
• How Microsoft and generative AI are transforming financial services contact centers

Implementing Azure Synapse Link for Dataverse: gotchas and tips

Azure Synapse Link for Dataverse allows you to easily export data from a Dataverse (or Dynamics 365) instance to Azure Data Lake Storage Gen2 (ADLS) and/or Azure Synapse. Azure Synapse Link for Dataverse provides a continuous replication of standard and custom entities/tables to Azure Synapse and Azure Data Lake. 

I highly recommend you to view the awesome YouTube playlist Azure Synapse Link and Dataverse - better together from Scott Sewell (@Scottsewell) as an introduction.

This blog post provides a number of tips & tricks but is not an exhaustive list - it is highly recommended to go through the links in the Microsoft documentation listed in the reference section below. You can also take a look at the presentation I delivered at Techorama  in May 2023 which is available on Github - Azure Synapse Link for Dataverse from 0 to 100

1. Check the region of your Dataverse/Dynamics 365 instance

The configuration of Azure Synapse Link for Dataverse is done through the Power Platform maker portal but before you can get started you should first setup Azure Data Lake Storage Gen2 and Azure Synapse in your Azure subscription.  

It is however best that you first check in the configuration screen in which region your instance is located since the storage account and Synapse Workspace must be created in the same region as the Power Apps environment for which you want to enable Azure Synapse Link.  From the PPAC user interface it is currently not possible to create a Dataverse/Dynamics 365 instance in a specific region but this is possible with the PowerShell - see Creating a Dataverse instance in a specific Azure region using Power Apps Admin PowerShell module. 

If you need to move a Dataverse or Dynamics 365 instance to a different Azure region, you can open a Microsoft support tickets. Based on recent experience this specific type of Microsoft support request is handled fairly quickly (within 1-2 business days).

Azure Data Lake Storage is a set of capabilities, built on Azure Blob Storage. When you create a storage account and check the "enable hierarchical namespace" checkbox on the advanced tab, you create an Azure Data Lake Storage Gen2.

2. Make sure all prerequisites are in place before enabling Azure Synapse Link

Definitely make sure that all security configuration outlined on Create an Azure Synapse Link for Dataverse with your Azure Synapse Workspace (Microsoft docs) are correctly setup. The exception messages which are shown in the Azure Synapse Link configuration pages aren't always very helpful.

3. Azure Synapse Link for Dataverse is a Lake Database

In the documentation from Microsoft (Understand lake database concepts) a lake database is defined as:

A lake database provides a relational metadata layer over one or more files in a data lake. You can create a lake database that includes definitions for tables, including column names and data types as well as relationships between primary and foreign key columns. The tables reference files in the data lake, enabling you to apply relational semantics to working with the data and querying it using SQL. However, the storage of the data files is decoupled from the database schema; enabling more flexibility than a relational database system typically offers.

The data is stored ADLS Gen2 in accordance with the Common Data Model (CDM) -the folders used conform to well-defined and standardized metadata structures (mapped 1:1 with Dataverse tables/entities). At the root you will see a metadata file (called model.json) which contains semantic information about all of the entity/table records, attributes and relationships between the tables/entities.

The way the files are being written depends on the Azure Synapse Link for Dataverse configuration - both the partitioning mode and in place vs append only mode can be configured - see Advanced Configuration Options in Azure Synapse Link 

4. Synapse Link for Dataverse uses passthrough authentication using ACLs in Azure Data Lake - no support for SQL authentication

Since all the the data for the tables in Azure Synapse Link for Dataverse are CSV files which are stored in Azure Data Lake Storage, this also means that security needs to be set at the level of the files in Azure Data Lake Storage Gen2. There is no support for SQL authentication in the Lake DB which is created by Azure Synapse Link for Dataverse.

References:

• Use the Common Data Model to optimize Azure Data Lake Storage Gen2 (Microsoft docs)
• Create an Azure Synapse Link for Dataverse with your Azure Synapse Workspace (Microsoft docs)
• Azure Synapse Link for Dataverse - understanding advanced configuration settings (Microsoft docs)
• Do more with data - from Data Export Service to Azure Synapse Link for Dataverse (Microsoft docs)
• Use Power BI to analyze the CDS data in Azure Data Lake Storage Gen2
• Synapse Link for Dataverse - Option Sets
• Dataverse metadata in Synapse
• Setting up Azure Synapse Link for Dynamics 365/Dataverse
• Power BI modeling guidance for Power Platform
• Azure Synapse Serverless SQL Pools cheat sheet
• Controlling cost with Azure Synapse Serverless Pools

Near real-time and snapshots in Azure Synapse Link for Dataverse

The Azure Synapse Link for Dataverse documentation contains a section about Access near real-time data and read-only snapshot data but it does not really explain why you want to use one or the other. 

When you open an Azure Synapse SQL Serverless LakeDB in SQL Server Management Studio you see a clear distinction between the two versions of the table data - whereas in Azure Synapse Studio there is no obvious distinction besides the name you will see the "account" table the "account_partitioned" view:

• Near real time data: external table for all the underlying CSV files exported by the Azure Synapse Lin for Dataverse sync engine. There is a soft SLA for the data to be present in these tables within 15 minutes
• Snapshot data/partitioned views: views on top of the near-real time data which are updated on an hourly interval.

In most scenarios, it best to do queries against these partitioned views since you will avoid read conflicts and you are sure that a full transaction has been written on the CSV files in Azure Data Lake storage. 

A typical exception that you might receive when doing queries directly against the "tables" is "https://`[datalakestoragegen2name].dfs.core.windows.net$$/[lakedbname]/[tablename/Snapshot/2023-05_1684234580/2023-05.csv" does not exist or you don't have file access rights)" but this also depends on your specific context. If you have a lot of create, updates or deletes on Dataverse tables this might happen more regularly. Even though, the partitioned views are update on an hourly basis - it might be that the Synapse Link engine is just refreshing the views at the same point that you perform a query, which will give you a similar exception but the changes that this occurs are more rare.

You can check the last sync timestamp and sync status in the Power Platform maker portal (see screenshot below)

For the moment, you will also have to manually check the monitoring page (which can be quite tedious if you have a lot of environments) but there is an item in the Microsoft release planner "Receive notifications about the state of Azure Synapse Link for Dataverse" which is apparently in public preview but I haven't seen it in for  environments (not in the https://make.powerapps.com and also not in  https://make.preview.powerapps.com/)  I have access to. 

It is also not easy to see if something went wrong with the refresh of the partitioned views - up until now the easiest way to find out is running a SQL query -  select name,create_date from sys.views order by create_date desc against the LakeDB.

Quick tip: prevent automatic deletion of an inactive Microsoft Dataverse environment

 A couple of months ago Microsoft activated Automatic deletion of inactive Microsoft Dataverse environments.  To avoid that an environment gets deleted, sign in to the Power Platform Admin Center (PPAC) and select the environment. On the environment page, select Trigger environment activity.

We don't talk about storage

When designing a solution using Dataverse or Dynamics 365 CE - Dataverse storage is rarely one of the hot topics and probably most of the times overlooked. In this post I will dive a little deeper into Dataverse storage architecture and why it is important to discuss this during a Dynamics 365 CE or Dataverse implementation 

Fundamentals of Dataverse storage architecture

Since April 2019, both Dataverse and Dynamics 365 CE (Online) use a tiered storage model. This means that different data types in Dataverse are stored in the most optimal storage type. Azure files and blobs are used for attachments, relational data (tables) is stored in Azure SQL, audit logs are stored in Azure Cosmos DB, search is powered by Azure Cognitive Services, etc .... 

From an end user perspective, this is completely transparent and administrators don't need to manage all of these underlying Azure components since Microsoft takes care of all of this.

Microsoft has however different pricing schema's for these underlying storage types and the price difference is quite significant. You can follow up on the storage capacity in the Power Platform Admin Center. Based on the number of licenses inside your tenant, you will get a specific capacity entitlement for database, file and logs. (See New Microsoft Dataverse storage capacity > verifying your new storage model for more details)

Understanding the cost impact

Database storage in Dataverse comes at a premium price tag. Usually you don't notice this unless you will need to buy additional storage (see  What Dataverse capacity is included with the Power Apps and Power Automate plans? for how much additional storage you get with additional licenses).

(1) Pricing based on the list price shown in my personal tenant - August 2023. Prices may vary depending on your licensing agreement. If you buy additional  eligible licenses you will also get additional storage allocated, it is also possible to use Power Platform Pay-as-you-go plans instead of buying license but the capacity pricing using PAYG is even higher.

You need to buy additional storage capacity if you are over the allocated storage capacity since storage capacity is being enforced - if you exceed storage capacity you will not be able to create a new environment (requires a minimum of 1 GB capacity) but also copy, restore and recovery operations will be blocked.

Archiving and data retention policies

Besides the impact on your budget of retaining all data in Dataverse (or Dynamics 365) forever - you however also need to consider the potential security and legal risk. Best practices dictate that data should only be kept as long as it is useful or as long as you are legally allowed to retain it - GDPR mandates to define specific data retention period's for personal data.

Defining a data retention policy helps businesses reduce legal risks, security threats and also reduce costs. Data retention policies contain the data retention period and the required actions to take when this period lapses. You should have a data retention period defined for each of entities/tables in use for your Dataverse environment.

In July 2023 finally released Dataverse long term data retention overview in public preview.  This feature allows you to create a view on Dataverse tables/entities for data that you need to retain for a longer period. The view is used as a selection criteria to define  which data is moved to long term archive storage in a Microsoft managed data lake. With this feature, you might be able to use out of the box functionality  instead of having to built your own custom solution.. This functionality is still in preview and should not be used on production - also keep in mind that no pricing details are available yet (they will be announced around the GA timeframe). In a upcoming post I will delve a little deeper into the archive/long term data retention functionality but you can already take a look at Early dive into Dataverse Long Term Retention

Related posts and references:

• Storage capacity management for Dynamics 365/Dataverse - how to track storage evolution with Power BI
• Inside Dataverse Data Storage -Power CAT Live YouTube video
• Cost optimization: older mails in Dataverse now stored in Dataverse file storage
• Storage capacity management for Dynamics 365/Dataverse - how to track storage evolution with Power BI

Reducing size of the SubscriptionTrackingDeletedObject in Dynamics 365 CE or Dataverse

When you delete a large amount of data from Dynamics 365 CE or Dataverse, you will notice that the total storage consumed does not decrease. This is because the SubscriptionTrackingDeletedObject table stores deleted records from the Dynamics 365 database. This table is used to track deleted records for replication and synchronization purposes and records are being kept by default for quite a large amount of time.

When recently encountering this problem, Microsoft support suggested us to reduce the number of days the records are kept in the table from the default 90 to 15 days - to do this  take the following steps:

• Install the latest version of the OrganizationSettingsEditor solution which you can download from https://github.com/seanmcne/OrgDbOrgSettings/releases   (The OrgDBOrgSettingsTool dates back to Dynamics CRM 2011 but is being maintained by Sean McNellis and is still quite relevant)

• Change the ExpireChangeTrackingInDays and ExpireSubscriptionsInDays from the default setting of 90 days to 15 days. In sandbox environments, you might be able to change this value to 0 but for production environments Microsoft recommended to keep it on a minimum of 15 days. The OrgDBSettings utility utilizes the solution configuration page to provide access to the editor (in classic mode) - see OrgDBOrgSettings - where to find it after installing? for details.

Afterwards - just keep patient until the lapsed number of days have been passed and then you should see a size in reduction for the SubscriptionTrackingDeletedObject

Dynamics 365 and Power Platform monthly reading list July 2023

2023 Release Wave 2

• Dynamics 365, Viva Sales (aka Microsoft Sales Copilot) and supply chain platform: 2023 release wave 2 plan - early access will be available on July 31th
• Microsoft Power Platform: 2023 release wave 2 plan
• Power Platform 2023 Release Wave 2 Highlights (YouTube)
• Dynamics 365 and Power Platform 2023 Release Wave 2 highlights (YouTube)
• Dynamics 365 Customer Service 2023 Release Wave 2 highlights (YouTube)

Technical topics (Configuration, customization and extensibility)

• ALM for Power Platform (March 2023 Washington user group recording)
• Analyse your solution dependencies in  Dataverse with VSCode "PowerApps Helper" extension built by @MMe2K (Michael Megel)
• Power Apps Copilot (preview) feature ... Friend or Foe? (YouTube) by @dchristian19
• Azure Cost Management Power BI report (Github)
• Codename Project Bose: Calculate Azure Cost of an enterprise by cost centers, divisions and projects
• KQL Visualizations (YouTube)
• How to use invitation id of Customer Voice Survey invitation in Power Automate Flow by @inogic
• Use the Customer 360 component in Dynamics 365 Customer Service by @D365Goddess
• Additional capabilities in timeline view with Dynamics CRM
• Optimize performance for Create and Update operations (Microsoft Docs) - interesting update for data migration scenario using CreateMultiple and UpdateMultiple operations
• Dataverse long term data retention is finally available in preview - no info available yet on whether retained Dataverse records (offloaded to Data Lake) will still be billed at the same price as Dataverse storage.
• The Dataverse Accelerator is live on AppSource: https://aka.ms/dvaccelerator
• Discover the biggest Power Platform Announcements from Microsoft Build 2023 (YouTube) by @LisaCrosbie
• Azure Synapse Link for Dataverse : exporting to Delta Lake
• Kusto Detective Agency - learn KQL by playing a detective game
• Power Apps Data Sources Ranked! Which is best? YouTube video by Shane Young (@ShanesCows)
• New card visual in Power BI and Create a sparkline in the new card visual
• Take Customer Service to new heights with cross-geo support for Copilot
• Announcing monthly (release) channel for model-driven apps - release channels provide more choice for adopting new features. It seems that Dynamics 365 and Power Platform will allow to get features activated on a monthly basis
• Announcing public preview of IP firewall in Dataverse - only available for managed environments
• Microsoft Copilot readiness hub
• The importance of Orderby when retrieving more than 50K records from Dataverse
• Start using Azure Cost Management

Topics for Dynamics 365 Business Applications Platform consultants, project managers and power users

• Introducing Microsoft Dynamics 365 copilot
• Copilot in Viva Sales
• Dynamics 365 Implementation Portal now available
• A report on Global State of Customer Service sophistication
• Reshaping the future of business with Microsoft Sales Copilot, Dynamics 365 Customer Insights and a new cloud migration program
• Introducing AIM from Microsoft: Future-proof your business in the cloud

Cost optimization: older mails in Dataverse now stored in Dataverse File Storage

Back in April 2019, Microsoft introduced a tiered storage architecture. This tiered storage architecture allows Microsoft to use the most appropriate storage type for different types of data used in Dynamics 365 and Dataverse. Last month, Microsoft started moving  the storage contents of the e-mail body (email.description) from more expensive Dataverse Database storage to cheaper Dataverse File Storage - for the official documentation check out Dataverse - Email activity tables -Email storage.

Based on the list prices I see in my personal tenant (see screenshot below) - this cuts the unit cost for additional capacity required for e-mails down from 37,4 €/GB/month to 1,87€/GB/month (Quite interesting how they still use the older name Common Data Service :-) )

Your pricing might defer based on the type of licensing agreement that you have.  If you had to buy additional storage capacity in the past, you might want to revise your storage usage and the required storage capacity add-ons.

You can check out the storage capacity in the Power Platform Admin Center - I added an example from a specific instance were Dataverse storage for e-mails (see ActivityPointerBase table) from 63 GB to 30GB.

A the same time, we saw a new storage file type appearing called Email which grew to 3,8 GB. This is a lot less than the storage saved in the database since the data in file storage (Azure Blob Storage behind the scenes) is actually compressed.

All in all, a great job from Microsoft which might save you a lot of money.  Don't hesitate to add a comments sharing your experience with this.... 

Azure Synapse Link for Dataverse playlist by Scott Sewell

A great starting point if you are new to Azure Synapse Link for Dataverse is Scott Sewell's YouTube playlist on Azure Synapse Link for Dataverse

Quick tip: use the new release planner for Dynamics 365 and Power Platform to track and planning upcoming changes

The new release planner for Dynamics 365 and Power Platform is awesome - check it out at https://aka.ms/releaseplanner . It allows you to track the features that are planned, coming soon or already available for trial. You have visibility across active release waves in a portal (built with Power pages) and you can personalize, filter and sort release plans when you sign in.

Quick tip: update to Dataverse developer plan

Great update from Microsoft on the Dataverse developer plan - check out the new video from Power CAT team on Dataverse Environments for everyone - New Developer Plan - Power CAT Live on YouTube.

Interesting updates in the new developer plan:

• 3 developer environments available per user
• Don't take up storage capacity with a maximum of 2GB data.
• Premium connectors included for testing without additional licenses
• Possible to play around with ALM to get a to know solution management better.
• Automatic cleanup of inactive environments after 90 days.

If you don't have access to the PPAC, you will need to use https://make.preview.powerapps.com/ for the moment which shows the option to create new environments from within the environment selector.

Dynamics 365 and Power Platform monthly reading list March 2023

 Power Platform and Dynamics 365 release 2023 wave 1

• Test engine to support model-driven apps (Public Preview May 2023)
• Make solution export source code repository friendly (Public Preview March 2023)
• Dynamics 365 Wave 1 2023: Top 10 features that you need to know! by @LisaCrosbie 
• Power Platform Wave 1 2023:  Top 10 features that you need to know! by @LisaCrosbie
• Virtual tables creation wizard now in Public Preview
• The new look for model driven Power Apps is available now in preview
• 2023 Release Wave 1 YouTube Playlist 

Technical topics (Configuration, customization and extensibility)

• Zero to Hero 2022 - Power Platform Governance and ALM by @thevictordantas
• How does Power Automate flow cope when a lot of actions are executed? by @timopertila
• Using Azure Key Vault to store secrets for use in environment variables and flows
• Introduction to Pipelines for Power Platform (YouTube)  by @rezadorrani and Build your first pipeline with Power Platform pipelines by @laskewitz
• Setup pipelines in Power Platform (Microsoft Learn)
• Signing up for a developer plan for Power Virtual Agents
• Power Platform build tools and Azure DevOps
• Pipelines in Power Platform is Generally Available (GA)
• Azure App Service Patterns and Features for the Azure Well-Architected Framework
• Adding a filter for lookup columns made easier with advanced lookup
• Every Azure Synapse Analytics Engineer Needs to Know this Essential Dataverse Feature - Synapse Link (YouTube)
• Power BI Dynamics 365 Sales templates with Scott Sewell
• Dataverse Environments for everyone - new developer plan - Power CAT Live by @topness
• Connections and connection references explained by @BergmannBene
• Model-driven apps: how to use the side pane to show custom html by @temmy_raharjo
• Enable managed environments in Power Platform Admin Center by @priyesh_wngman7

Topics for Dynamics 365 Business Applications Platform consultants, project managers and power users

• Sales forecasting increase accuracy with AI predictive forecasting by @inogic
• Add voice mail to your call queue management
• Microsoft and Futurum research: Discover insights to create experiences that customers love
• Guidance on Sales Hub and custom model driven apps
• Create a Digital Contact Center in seconds with the Customer Service free trial

Dynamics 365 and Power Platform monthly reading list December 2022

First post of 2023 - so let's close with last month's  suggested reading list. If you want to catch up on previous reading lists for 2022 - here is the list:

• Dynamics 365 and Power Platform monthly reading list September 2022
• Dynamics 365 and Power Platform monthly reading list July 2022 
• Dynamics 365 and Power Platform monthly reading list May 2022
• Dynamics 365 and Power Platform monthly reading list April 2022
• Dynamics 365 and Power Platform monthly reading list February 2022
• Dynamics 365 and Power Platform monthly reading list January 2022

 Technical topics (Configuration, customization and extensibility)

• Dataverse: simple advice on how to design your plug-in by @temmy_raharjo
• Match contacts and accounts by email domain with Power Automate
• Setting as a solution component in Power Apps by @inogic 
• Dataverse group teams with Azure AD dynamic user membership groups
• Introducing Test Engine: an open platform for automated testing of canvas apps
• Adaptive cards in Power Apps
• Adaptive card designer overview (Preview) in  Power Apps 
• Create a card with data from Dataverse (preview) 
• Build powerful Adaptive Card experiences for Teams using Cards for Power Apps by @michelcarlo
• Complete a Dataverse Task by an adaptive card in Teams
• Setup Azure DevOps projects for ALM accelerator for Power Platform
• Granular billing for Azure Data Factory
• 2022 Release Wave 2: Timeline maker enhancements
• Announcing two new white papers that dive deep into Power Pages architecture and security
• Streamline low code governance in your governance (Ignite 2022 recording BRK41)
• How to setup and connect to Dataverse environment in Postman
• A few notes on the Timeline: model-driven Power Apps form  tweaks
• Power Platform Capacity Monitoring by @TheCRMNinja
• Top 25 Power Apps Tips, Tricks and Best Practices for Makers by @rezadorrani

Topics for Dynamics 365 Business Applications Platform consultants, project managers and power users

• Enable AI generated conversation summary (Preview) by @D365Goddess
• E-mail validation (Preview) in Dynamics 365 Sales
• Utilizing the swarming feature in Teams (Preview) for intelligent collaboration within Dynamics 365
• Leverage low-code to do more with less at Microsoft Ignite 2022
• Live from Microsoft Ignite 2022: Introducing new AI, automation and collaboration capabilities for Dynamics 365
• Duplicate lead detection increases sellers' productivity
• The future of IT with Charles Lammana - Power CAT Live (YouTube)
• Power BI modeling guidance for Power Platform
• How we can correct the mistakes made with Agile (YouTube)
• Exploring sales usage reporting feature in Dynamics 365 CE sales analytics
• Unleash your Dynamics 365 data with Azure Synapse Analytics and Power BI (YouTube recording)
• Price points of Power Platform (2022 pricing and licensing) by @jukkan
• One Dynamics One Platform - Introduction (FastTrack TechTalks)
• Discover new ways to analyze your customer service data
• Customer Service voice support channel launches in the UK (November 2022)

Creating a Dataverse instance in a specific Azure region using Power Apps Admin PowerShell module

One of the prerequisites if you need to setup Azure Synapse Link for Dataverse is that the Dataverse/Dynamics 365 environment must be created in the same Azure region as the Azure Data Lake storage account. 

So, if you know that you will be using Azure Synapse Link for Dataverse functionality and you want to avoid opening a support ticket for moving your environment (see Quick tip: finding the Azure data center for your Dynamics 365/Online environment for more details), you will need to create the Dataverse instance in the correct region. 

Unfortunately, this is not possible using the standard user interface but luckily you can use the Power Apps admin PowerShell module (Microsoft.PowerApps.Administration.PowerShell)

I am not sure in which version the functionality was added to enable creation of a Dataverse environment in a specific Azure region was added so if you already installed this PowerShell previously I would recommend updating to the latest version using the update-module command.

When using the New-AdminPowerAppEnvironment command you can specify the location and region you need (for example -LocationName "europe" -RegionName "westeurope"). For a list of supported environment locations and regions you can run Get-AdminPowerAppEnvironmentLocations

References:

• Get started using the Power Apps admin module
• Provision Dynamics 365 CRM and Dataverse environments on specific region and datacenter

Speaking engagements in coming months

With all Covid bans lifted and summer holidays well over, the conference season kicks off. 

I will be speaking at a couple of events in the coming weeks and months:

• Dataminds evening session Upcoming in-person event on September 29th organized by dataMinds.be at Inetum-Realdolmen offices in Kontich together with Benni De Jagere. First session a little bit off the beaten track for data professionals: #dataviz for investors. Second session: #PowerBI roadmap and #AMA by Benni.
• Collabdays Belgium 2022. Free community-driven event in Brussels, Belgium. Focus is Microsoft 365 with some Power Platform and Azure sprinkled on top. I am particularly excited to be speaking at this conference which was born out of the SharePoint Saturday conferences which I helped organize many years ago. I will be delivering Dataverse Deep Dive: watch out for sharks.
• Cloudbrew 2022. A two-day conference focusing on all things Azure on November 18-19 in Mechelen Belgium. I will be delivering Using Python and Azure Cloud for trading and investing

Detailed Power Platform request usage information in Power Platform Admin Center in preview

I wrote a post in January 2022 on the changes in Dynamics 365 and Power Platform request limits and allocations mentioning that detailed reporting was not available at that point in time. 

In the meanwhile Microsoft has released Detailed Power Platform request usage information in the Power Platform admin center (Preview). For integration application users you need to look at the Non-licensed User report which shows request usage per day for non-licensed users and the total entitlement for the tenant.  Unfortunately for environments with a lot of integrations you might need to revert to some Excel skills or Power BI to make sense of the data (currently working in a tenant where we have 100K+ lines in the CSV export file).

For the moment there is no high usage enforcement for Power Platform request limits, but this might start at least six months after the reports have been made available. 

References:

• Power Platform Licensing FAQ - what add-ons are available to the Power Apps and Power Automate plans

Thoughts about the Dynamics 365 solution architect role

I having been working as a solution architect for over 15 years now and the last 7 years as a Dynamics 365 solution architect (from when we still referred to it as simply Dynamics CRM) and I really like it.  But if I tell people that I work as a Dynamics 365 solution architect (even people work in IT), I immediately get the idea that they don't have a clue about what the role entails or what I do (and don't get me started about recruiters).

In this post I will try to shed some light on why it is so hard to clearly define the role and responsibilities of a Dynamics 365 solution architect. I will also explain on what Microsoft thinks about the Dynamics 365 role and why what Microsoft prescribes is not necessarily what applies to the project that you will be working on as a solution architect.  I will also list a number of lessons learned and learning resources that I used in the last couple of years.

Context matters

How we approach architecture, defines what architecture, in effect, is, at least for our system. That is, no matter what we say architecture is (for), architecture is (an outcome of) what we do. (Source: What is Software Architecture and related concerns) 

The last couple of years, I mainly worked as a solution architect in a large team at Toyota Motor Europe (TME), so I adopted the way of working of TME and the standards and boundaries set for solution architects within Toyota (I will be writing a separate blog post on lessons learned from the Toyota way)

So depending on the customer your work for, the size of the Dynamics 365 project and the associated complexity, the number of consultants working on the project and the overall maturity of the project team as well as the project methodology (agile vs waterfall) the tasks you take on as a Dynamics 365 solution architect and your role and responsibilities might differ quite a lot. 

Whether you are full-time working for a single project (embedded architect) or supporting multiple teams (wandering/travelling architect) also makes a big difference.

Architecture is about significant design decisions

I think that if you look at the role of the Dynamics 365 solution architecture from a very minimalistic viewpoint, there are three key activities and deliverables: 

• The high level architecture visualization - remember a picture says more than a thousand words - definitely take a look at good models like C4 from Simon Brown
• Maintaining a list of architecture design decisions together with the project team (take a look at  Architectural Decision Records | adr.github.io for some more info)
• Coaching and mentoring the team to create an architecture awareness (understand design trade-offs, keep track of non-functional requirements, identify architecture risks and counter measures, understanding and handling technical debt)

A solution architect has to keep the long-term use of his solution in mind and implement scalability and adaptability into the solution for the future.

Success By Design and FastTrack

I learned a lot from the joint-design workshops and conversations with the Microsoft FastTrack solution architect assigned on some of my projects. For those of you unfamiliar with FastTrack for Dynamics 365, this is a Microsoft implementation support program for large Dynamics 365 implementations - this includes Customer Engagement (Sales, Customer Services, Field Services, Project Operations  and Marketing) as well as Dynamics 365 Finance, Commerce, Supply Chain Management and Human Resources.

The full FastTrack program is only available for a selected group of customers (threshold based on Dynamics 365 annual licensing revenue or internal approval by Microsoft account team)  and Microsoft partners (only partners with gold or silver status in Cloud Business Applications competency).  When you participate in the FastTrack program, you will be assigned a Microsoft FastTrack solution architect which you can use as a sounding board for design choices, receive guidance on best practices and how to plan for a successful roll-out.

Even if you can't join the Dynamics 365 FastTrack program it is still useful to take a look at the Success By Design resources - especially the Dynamics 365 Implementation Guide (recently revised) is a very extensive reference.

Also take a look at Dynamics 365 Fasttrack Architecture Insights - recordings created and shared by solution architects from the Dynamics 365 engineering team.

Microsoft about the Dynamics 365 solution architect

Two years ago I took a Microsoft certification exam targeted at solution architects in the Dynamics 365 space. I really enjoyed taking the exam MB-600: Microsoft Dynamics 365/Power Platform Solution Architect (now replaced by PL-600: Microsoft Power Platform Solution Architect) since it gives great insights on how Microsoft looks at the role of a Dynamics 365 solution architect. 

As a preparation for this exam, I followed the Architect solutions for Dynamics 365 and Microsoft Power Platform (screenshot below taken from this learning path) and together with the Microsoft FastTrack boot camp for Dynamics 365 CE solution architects, this shaped a lot of my thinking about the role of a Dynamics 365 solution architect. 

A key task of the solution architect is solution envisioning. Traditionally, a development-focused architect would start with custom development and low-level Azure services. A business applications focused solution architect will instead start with Dynamics 365 and Power Platform and then use third party components, custom development and Microsoft Azure to address any gaps.

In-depth knowledge of the products in the Microsoft BizApps stacks is therefore a must have and hands-on experience with one or more products helps you in becoming a better architect.  If you look at the videos from Gus Gonzalez on How do you become a solution architect and 3 rules for solution architects and consultants or listen to some of the episodes of the CRM MVP Podcast you will also get a feel about what other solution architects think about their role.

Microsoft seems to assume that a solution architect needs to be embedded into the team and does not make any mention of the notion of the travelling architect. The idea behind a travelling architect is that he/she will support multiple projects and provide support for teams who lack architectural skills by either coaching them, tackling specific architecture issues by him/herself, etc ... If you are working for a partner organization who is doing multiple smaller projects in parallel the role of a travelling architect makes more sense.

Working with other architects 

Depending on their scope of work, different types of architects work at different levels of abstraction - until now I have worked with enterprise architects, business architects and technical architects. In my current project, I work very closely with EA to make sure that the solution blueprint aligns with the set of architecture principles and principles defined by EA. 

A solution architect is not the same as an enterprise architect (although you might combine both roles) - an enterprise architect has a more strategic role and handles the initiatives that are required within a digital transformation at enterprise level to ensure that the business achieves their goals (business intent). It is however useful that you have a good understanding about enterprise architecture as a function and affinity with EA frameworks.

Get involved during the initial definition of a project/program

When I took a TOGAF training course years ago, I still remember an architect being called a "best effort architect" when he is not involved in the early stages of a program or project. If you are working on customer side and you are not involved during the initial business case creation, project or program initial scope definition, you are a "best effort architect" meaning that you miss a lot of the context and a large numbers of important decisions have already been made so you can only give it your best effort. 

Same goes for architects working at a consultancy or implementation partner, if you are not involved during the initial presales phase, you are constrained due to (implicit) design decisions which were already made during presales phase while answering the RFP or presenting demo cases.

Learn by doing

So if you are a Dynamics 365 solution consultant or technical consultant and you want to take the leap, there are a lot of resources to ease into the role but nothing beats learning by doing. Also keep in mind that as a solution architect lifelong learning is a prerequisite for success (but probably also for other roles)

If you are lucky, you might encounter someone within the company you work for who can coach you to grow into the role but also reach out to the broader Dynamics 365 community for questions or feedback.

References:

• 3 rules for solution architects and consultants by @gusgonzalez
• Dynamics 365 Fasttrack Architecture Insights - recordings created and shared by solution architects from the Dynamics 365 engineering team
• Fasttrack - Dynamics 365 Implementation Guide (pdf) - not being updated anymore - Microsoft only updates the web version at https://aka.ms/D365ImplementationGuide
• Common types of Software Architects
• Software architecture for a Post AI and Post Quantum World  (Grady Booch - YouTube)
• Handbook of software architecture (Grady Booch)
• The solution blueprint review as cornerstone of FastTrack Success by Design
• How architecture diagrams enable better conversations by @kzhen
• An architect is not an evangelist