Hidden gem : the business process catalog for Dynamics 365 apps

The business process catalog for Dynamics 365 apps is an Excel workbook that contains more than 700 processes grouped by end-to-end scenarios and business process areas for different products (Dynamics 365 SCM, Business Central, Customer Service etc ...). It can be used as a blueprint for requirement-gathering and implementation templates. 

Microsoft updates the catalog at least four times each year and the latest November 2024 release was recently made available. This latest releases includes:

• Azure DevOps templates that you can use to track your implementation
• Configuration deliverable downloads for Finance and Operation apps as well as Customer Engagement apps.
• AI generated process flow diagrams for level 3 processes to accelerate the discover process.

Download the latest version on https://aka.ms/businessprocesscatalog and explore AI generated flow diagrams at https://aka.ms/businessprocessflow

We don't talk about storage

When designing a solution using Dataverse or Dynamics 365 CE - Dataverse storage is rarely one of the hot topics and probably most of the times overlooked. In this post I will dive a little deeper into Dataverse storage architecture and why it is important to discuss this during a Dynamics 365 CE or Dataverse implementation 

Fundamentals of Dataverse storage architecture

Since April 2019, both Dataverse and Dynamics 365 CE (Online) use a tiered storage model. This means that different data types in Dataverse are stored in the most optimal storage type. Azure files and blobs are used for attachments, relational data (tables) is stored in Azure SQL, audit logs are stored in Azure Cosmos DB, search is powered by Azure Cognitive Services, etc .... 

From an end user perspective, this is completely transparent and administrators don't need to manage all of these underlying Azure components since Microsoft takes care of all of this.

Microsoft has however different pricing schema's for these underlying storage types and the price difference is quite significant. You can follow up on the storage capacity in the Power Platform Admin Center. Based on the number of licenses inside your tenant, you will get a specific capacity entitlement for database, file and logs. (See New Microsoft Dataverse storage capacity > verifying your new storage model for more details)

Understanding the cost impact

Database storage in Dataverse comes at a premium price tag. Usually you don't notice this unless you will need to buy additional storage (see  What Dataverse capacity is included with the Power Apps and Power Automate plans? for how much additional storage you get with additional licenses).

(1) Pricing based on the list price shown in my personal tenant - August 2023. Prices may vary depending on your licensing agreement. If you buy additional  eligible licenses you will also get additional storage allocated, it is also possible to use Power Platform Pay-as-you-go plans instead of buying license but the capacity pricing using PAYG is even higher.

You need to buy additional storage capacity if you are over the allocated storage capacity since storage capacity is being enforced - if you exceed storage capacity you will not be able to create a new environment (requires a minimum of 1 GB capacity) but also copy, restore and recovery operations will be blocked.

Archiving and data retention policies

Besides the impact on your budget of retaining all data in Dataverse (or Dynamics 365) forever - you however also need to consider the potential security and legal risk. Best practices dictate that data should only be kept as long as it is useful or as long as you are legally allowed to retain it - GDPR mandates to define specific data retention period's for personal data.

Defining a data retention policy helps businesses reduce legal risks, security threats and also reduce costs. Data retention policies contain the data retention period and the required actions to take when this period lapses. You should have a data retention period defined for each of entities/tables in use for your Dataverse environment.

In July 2023 finally released Dataverse long term data retention overview in public preview.  This feature allows you to create a view on Dataverse tables/entities for data that you need to retain for a longer period. The view is used as a selection criteria to define  which data is moved to long term archive storage in a Microsoft managed data lake. With this feature, you might be able to use out of the box functionality  instead of having to built your own custom solution.. This functionality is still in preview and should not be used on production - also keep in mind that no pricing details are available yet (they will be announced around the GA timeframe). In a upcoming post I will delve a little deeper into the archive/long term data retention functionality but you can already take a look at Early dive into Dataverse Long Term Retention

Related posts and references:

• Storage capacity management for Dynamics 365/Dataverse - how to track storage evolution with Power BI
• Inside Dataverse Data Storage -Power CAT Live YouTube video
• Cost optimization: older mails in Dataverse now stored in Dataverse file storage
• Storage capacity management for Dynamics 365/Dataverse - how to track storage evolution with Power BI

Why I blog and you might want to consider it as well

A couple of weeks ago I got the question why I write a blog since it would be better to do a podcast or publish videos on technical content. In this post, I will explain why it is still a good idea to start blogging.  I wrote my first blog post almost 18 years ago  and got inspired by Robert Scoble (Scobleizer) who worked as a technical evangelist at Microsoft at that time and had a very popular blog.  1483 blog posts and almost 2 million views of these blogs I still consider it a good idea, since  I think that a lot of good things have happened to me in my professional live because I decided to start my blog and share my thoughts on technology.  

Starting a blog is an easy starting point for you to be active in the technical community. Sharing is caring might sound lame but it is a good credo to live by as an IT professional/consultant.  In a professional context,  most persons  only know the people that they directly work with - colleagues or customers. 

Blogging (but also Twitter and technical communities) allows you to expand that and suddenly there are people all over the world that you can learn from and they can learn from you. Suddenly you will realize there are others who face similar challenges. By sharing your experiences and offering advice to support, you can build a community of readers who value your experience. By writing good high-quality content on a regular basis, you can establish yourself as an expert in your field which in turn will help you build your personal brand and increase your visibility. Starting a blog is a lot easier than creating a video on YouTube or doing a podcast.

The most underrated skill to have in IT is the ability to communicate efficiently. Blogging is levelling that skill up. In an era where people mostly rely on PowerPoints slides for presenting a plan or a vision, it is quite refreshing to see them back it up with a well-written, solid document afterwards. I usually cringe when seeing poorly written documents or e-mails so I think that good writing skills will set you apart. 

I think of a blog as a way to write notes to myself and you should too. Throughout your daily job, you are continuously solving problems: write about how you approached solving these problems and where you found helpful information. By writing your approach down, you will remember it better and it will be easier for you to find it later (and others will benefit of this as well). 

The more you explain a concept or a topic to other people, the better you will get at understanding it yourself. This might sound strange but it is true, teach to learn. When you need to teach something, you are required to understand it in a deep and comprehensive way. The process of writing, will help your own understanding and requires you to think critically and find creative ways to present the topic which further enhances your understanding and retention of the information.

Even when you are learning something new, you are probably taking notes. Blog these notes, taking these notes. Rewriting them will help you understand the content much better, while it helps others too. If you think that everything is already written, do it for your own reference. If you find it useful yourself to go back to your blog, then it is OK. Don't fuzz to much about getting little visitors to your blog in the beginning.

References:

• You should be writing every day. Here's Why (and how to do it)
• Why engineers should focus on writing

Detailed Power Platform request usage information in Power Platform Admin Center in preview

I wrote a post in January 2022 on the changes in Dynamics 365 and Power Platform request limits and allocations mentioning that detailed reporting was not available at that point in time. 

In the meanwhile Microsoft has released Detailed Power Platform request usage information in the Power Platform admin center (Preview). For integration application users you need to look at the Non-licensed User report which shows request usage per day for non-licensed users and the total entitlement for the tenant.  Unfortunately for environments with a lot of integrations you might need to revert to some Excel skills or Power BI to make sense of the data (currently working in a tenant where we have 100K+ lines in the CSV export file).

For the moment there is no high usage enforcement for Power Platform request limits, but this might start at least six months after the reports have been made available. 

References:

• Power Platform Licensing FAQ - what add-ons are available to the Power Apps and Power Automate plans

Thoughts about the Dynamics 365 solution architect role

I having been working as a solution architect for over 15 years now and the last 7 years as a Dynamics 365 solution architect (from when we still referred to it as simply Dynamics CRM) and I really like it.  But if I tell people that I work as a Dynamics 365 solution architect (even people work in IT), I immediately get the idea that they don't have a clue about what the role entails or what I do (and don't get me started about recruiters).

In this post I will try to shed some light on why it is so hard to clearly define the role and responsibilities of a Dynamics 365 solution architect. I will also explain on what Microsoft thinks about the Dynamics 365 role and why what Microsoft prescribes is not necessarily what applies to the project that you will be working on as a solution architect.  I will also list a number of lessons learned and learning resources that I used in the last couple of years.

Context matters

How we approach architecture, defines what architecture, in effect, is, at least for our system. That is, no matter what we say architecture is (for), architecture is (an outcome of) what we do. (Source: What is Software Architecture and related concerns) 

The last couple of years, I mainly worked as a solution architect in a large team at Toyota Motor Europe (TME), so I adopted the way of working of TME and the standards and boundaries set for solution architects within Toyota (I will be writing a separate blog post on lessons learned from the Toyota way)

So depending on the customer your work for, the size of the Dynamics 365 project and the associated complexity, the number of consultants working on the project and the overall maturity of the project team as well as the project methodology (agile vs waterfall) the tasks you take on as a Dynamics 365 solution architect and your role and responsibilities might differ quite a lot. 

Whether you are full-time working for a single project (embedded architect) or supporting multiple teams (wandering/travelling architect) also makes a big difference.

Architecture is about significant design decisions

I think that if you look at the role of the Dynamics 365 solution architecture from a very minimalistic viewpoint, there are three key activities and deliverables: 

• The high level architecture visualization - remember a picture says more than a thousand words - definitely take a look at good models like C4 from Simon Brown
• Maintaining a list of architecture design decisions together with the project team (take a look at  Architectural Decision Records | adr.github.io for some more info)
• Coaching and mentoring the team to create an architecture awareness (understand design trade-offs, keep track of non-functional requirements, identify architecture risks and counter measures, understanding and handling technical debt)

A solution architect has to keep the long-term use of his solution in mind and implement scalability and adaptability into the solution for the future.

Success By Design and FastTrack

I learned a lot from the joint-design workshops and conversations with the Microsoft FastTrack solution architect assigned on some of my projects. For those of you unfamiliar with FastTrack for Dynamics 365, this is a Microsoft implementation support program for large Dynamics 365 implementations - this includes Customer Engagement (Sales, Customer Services, Field Services, Project Operations  and Marketing) as well as Dynamics 365 Finance, Commerce, Supply Chain Management and Human Resources.

The full FastTrack program is only available for a selected group of customers (threshold based on Dynamics 365 annual licensing revenue or internal approval by Microsoft account team)  and Microsoft partners (only partners with gold or silver status in Cloud Business Applications competency).  When you participate in the FastTrack program, you will be assigned a Microsoft FastTrack solution architect which you can use as a sounding board for design choices, receive guidance on best practices and how to plan for a successful roll-out.

Even if you can't join the Dynamics 365 FastTrack program it is still useful to take a look at the Success By Design resources - especially the Dynamics 365 Implementation Guide (recently revised) is a very extensive reference.

Also take a look at Dynamics 365 Fasttrack Architecture Insights - recordings created and shared by solution architects from the Dynamics 365 engineering team.

Microsoft about the Dynamics 365 solution architect

Two years ago I took a Microsoft certification exam targeted at solution architects in the Dynamics 365 space. I really enjoyed taking the exam MB-600: Microsoft Dynamics 365/Power Platform Solution Architect (now replaced by PL-600: Microsoft Power Platform Solution Architect) since it gives great insights on how Microsoft looks at the role of a Dynamics 365 solution architect. 

As a preparation for this exam, I followed the Architect solutions for Dynamics 365 and Microsoft Power Platform (screenshot below taken from this learning path) and together with the Microsoft FastTrack boot camp for Dynamics 365 CE solution architects, this shaped a lot of my thinking about the role of a Dynamics 365 solution architect. 

A key task of the solution architect is solution envisioning. Traditionally, a development-focused architect would start with custom development and low-level Azure services. A business applications focused solution architect will instead start with Dynamics 365 and Power Platform and then use third party components, custom development and Microsoft Azure to address any gaps.

In-depth knowledge of the products in the Microsoft BizApps stacks is therefore a must have and hands-on experience with one or more products helps you in becoming a better architect.  If you look at the videos from Gus Gonzalez on How do you become a solution architect and 3 rules for solution architects and consultants or listen to some of the episodes of the CRM MVP Podcast you will also get a feel about what other solution architects think about their role.

Microsoft seems to assume that a solution architect needs to be embedded into the team and does not make any mention of the notion of the travelling architect. The idea behind a travelling architect is that he/she will support multiple projects and provide support for teams who lack architectural skills by either coaching them, tackling specific architecture issues by him/herself, etc ... If you are working for a partner organization who is doing multiple smaller projects in parallel the role of a travelling architect makes more sense.

Working with other architects 

Depending on their scope of work, different types of architects work at different levels of abstraction - until now I have worked with enterprise architects, business architects and technical architects. In my current project, I work very closely with EA to make sure that the solution blueprint aligns with the set of architecture principles and principles defined by EA. 

A solution architect is not the same as an enterprise architect (although you might combine both roles) - an enterprise architect has a more strategic role and handles the initiatives that are required within a digital transformation at enterprise level to ensure that the business achieves their goals (business intent). It is however useful that you have a good understanding about enterprise architecture as a function and affinity with EA frameworks.

Get involved during the initial definition of a project/program

When I took a TOGAF training course years ago, I still remember an architect being called a "best effort architect" when he is not involved in the early stages of a program or project. If you are working on customer side and you are not involved during the initial business case creation, project or program initial scope definition, you are a "best effort architect" meaning that you miss a lot of the context and a large numbers of important decisions have already been made so you can only give it your best effort. 

Same goes for architects working at a consultancy or implementation partner, if you are not involved during the initial presales phase, you are constrained due to (implicit) design decisions which were already made during presales phase while answering the RFP or presenting demo cases.

Learn by doing

So if you are a Dynamics 365 solution consultant or technical consultant and you want to take the leap, there are a lot of resources to ease into the role but nothing beats learning by doing. Also keep in mind that as a solution architect lifelong learning is a prerequisite for success (but probably also for other roles)

If you are lucky, you might encounter someone within the company you work for who can coach you to grow into the role but also reach out to the broader Dynamics 365 community for questions or feedback.

References:

• 3 rules for solution architects and consultants by @gusgonzalez
• Dynamics 365 Fasttrack Architecture Insights - recordings created and shared by solution architects from the Dynamics 365 engineering team
• Fasttrack - Dynamics 365 Implementation Guide (pdf) - not being updated anymore - Microsoft only updates the web version at https://aka.ms/D365ImplementationGuide
• Common types of Software Architects
• Software architecture for a Post AI and Post Quantum World  (Grady Booch - YouTube)
• Handbook of software architecture (Grady Booch)
• The solution blueprint review as cornerstone of FastTrack Success by Design
• How architecture diagrams enable better conversations by @kzhen
• An architect is not an evangelist

Introducing the PowerApps Checker PowerShell module to check PowerApps solution quality

Microsoft recently updated their tooling for checking the quality of  CDS/PowerApps/Dynamics 365 solution files by releasing the PowerApps checker PowerShell Module . With this new PowerShell module it is possible to receive a detailed report for your solutions identifying problems/issues taking into account best practices around performance, usage, upgrade readiness, supportability and maintainability. The static code analysis is based on rulesets which are provided by Microsoft and which are updated on a regular basis.

The main difference with the previously released tooling in the PowerApps maker portal (See Announcing general availibility of Solution Checker) is that you can now also do checks to validate both managed and unmanaged solutions (CRM2011 to current) so include on-premise solution validation.

The PowerShell module can be installed from https://www.powershellgallery.com/packages/Microsoft.PowerApps.Checker.PowerShell/1.0.2  and provides a number of commands that you can use to perform the checks.

Before you can use the Invoke-PowerAppsChecker PowerShell cmdlet ,  you need to create an Azure Active Directory (AAD) application in a tenant with PowerApps or Dynamics 365 licenses. Follow the steps on Get started using the Microsoft.PowerApps.Checker.PowerShell module to correctly do this using either PowerShell or manually.

The PowerAppsChecker cmdlets in interactive-mode (meaning you need to login everytime you run the checker) or using an application-based token.

Behind the scenes the PowerShell module will connect to the PowerApps checker Web API

The output of Invoke-PowerAppsChecker  is a zip file containing one or more reports in a standardized JSON format. The report format is based on static analysis results referred to as Static Analysis Results Interchange Format (SARIF).

By analyzing results and fixing indicated errors, you can learn how to write high-quality code and decrease the cost of fixing issues later.

Depending on the size & complexity of your project, I might be recommended to include it as part of your ALM strategy.

Other blog post:

• Check Dynamics 365 solution quality with the new PowerApps Solution Checker
• General availability of PowerApps solution checker makes  it easier to check Dynamics 365 solution quality

Microsoft cloud IT architecture resources available

Summary: Learn core cloud architecture concepts for Microsoft identity, security, networking, and hybrid. Review prescriptive recommendations for protecting files, identities, and devices when using Microsoft's cloud. Learn how to deploy a modern and secure desktop with Windows 10 and Office ProPlus.

These architecture tools and posters give you information about Microsoft cloud services, including Office 365, Windows 10, Azure Active Directory, Microsoft Intune, Microsoft Dynamics 365, and hybrid on-premises and cloud solutions. IT decision makers and architects can use these resources to determine the ideal solutions for their workloads and to make decisions about core infrastructure components such as identity and security.
https://docs.microsoft.com/en-us/office365/enterprise/microsoft-cloud-it-architecture-resources