Book review: beyond diversification - what every investor needs to know about asset allocation

I recently finished reading Beyond diversification from Sebastien Page.  Sebastien Page (Chief Investment Officer at T. Rowe Price) explains in Beyond diversification the different approaches to forecasting returns, risks and correlations across asset classes by combining academic research and practical hands-on examples. 

This book is most likely targeted at the sophisticated investor  and should not be the first book to pick up if you want to understand diversification but it provides great insights on how asset managers think about their portfolios. 

The book also extensively refers to a number of academic papers that Sebastien Page has written on asset allocation, risk measurement and return forecasting. It explores a number of dynamic asset allocation strategies, acknowledging that risk is time-varying and requires adaptive approaches. Sebastien Page also explains why the typical fixed weight asset allocation (60-40 portfolio) does not deliver a constant risk exposure.

Dynamics 365 and Power Platform monthly reading list September 2024

 2024 Release Wave 2

Release plans for 2024 release wave 2 are now publicly available - Release plans for Dynamics 365, Power Platform, and Cloud for Industry - Dynamics 365 | Microsoft Learn - from July 29th, early access will be available for activation - other key dates for 2024 release wave 2 are also available . Also check out the videos like Dynamics 365 Service 2024 Release Wave 2 or Dynamics 365 and Power Platform 2024 Release Wave 2 - these might also contain features/functionality which are not listed in the release plans.

• 2024 Wave 2: Dynamics 365 Customer Service (YouTube) by @d365goddess
• 2924
• Use the Power Platform Environment Settings App 
• Change history 2024 release wave 1 and change history 2024 release wave 2 -  there are frequent changes on features which are part of a release wave - follow up using the change history
• 2024 release wave 2 for Dynamics 365 Contact Center
• Managed identity support for Dataverse plugins

Copilots, AI and machine learning

• Document classification with Power Automate and Azure AI by @mattbdevaney
• Understanding Copilots at Microsoft (YouTube)
• Microsoft Copilot Studio implementation guide
• So you want to build your own copilot
• Use Dataverse timeline highlights powered by generative AI
• Microsoft Copilot for Finance (YouTube)
• Simplify IT management with Microsoft Copilot for Azure - save time and get answers fast
• Microsoft Copilot for Service on ServiceNow
• Building a multi-channel notification system with Azure Functions and Azure Communication Services - Part 1 #60daysofIA
• Azure AI Translator GitHub repo

Technical topics (Configuration, customization and extensibility)

• Unlocking the full potential of Dynamics Digital Contact Center Platform with Bring Your Own Carrier (BYOC)
• How Microsoft does Quality Assurance (QA)
• Why use managed solutions by @BergmannBene
• Filtering embedded Power BI reports
• Outbound dialer enhancements in Dynamics 365 Customer Service
• Setting up Postman environment to test the Dataverse APIs - quick tip
• Application Insights integration with model-driven Power Apps explored
• Power Platform Well Architected
• User Experience in model-driven apps - the good, the bad and the ugly (YouTube)
• Customer Service: Setup Copilot (YouTube)
• Power Platform Community Tools series - Bulk Data Updater (YouTube)
• Build your own agent using Microsoft Copilot Studio (GitHub Repo)
• The scarce resource of Dataverse storage capacity
• Demo: Build intelligent and scalable solutions with Power Apps and Power Platform (YouTube)
• Manage application users in the Power Platform Admin Center (PPAC)
• Dataverse: Azure Synapse Link vs Fabric Link (YouTube) 
• Adjust other Dynamics 365 copilots in Copilot Studio
• What is a Power Platform Solutions Architect? Action packed explanation! (YouTube)
• Building smarter bots with copilot studio (YouTube)
• Announcing General Availability of Block unmanaged customizations
• 6 stages of solution deployment by  @LinkeD365
• Everything you didn't know you needed to know about Power Platform Solutions
• Dynamics 365 Finance is gradually moving onto the same base platform as Dynamics 365 CE - useful shortcuts
• https://aka.ms/ERPTrialLicense
• https://aka.ms/ERPDevDeploy 
• https://aka.ms/ERPDevConfig 

Topics for Dynamics 365 Business Applications Platform consultants, project managers and power users

• Three ways AI is transforming service from the field to the contact center (Microsoft white paper)
• Capture and communicate value with the business value toolkit (Power Platform CoE toolkit)
• Introducing Microsoft Dynamics 365 Contact Center(YouTube)
• Eneco Mobility supercharges its customer car with generative AI by introducing Dynamics 365 Contact Center (Case study - Capgemini) - follow  Peter Ruiter (@pajruiter)  to learn more about copilots and Dynamics 365 Contact Center

Quick tip: Get-AzSubscription and Azure Cloud Shell

The Get-AzSubscription PowerShell cmdlet gets the subscription ID, subscription name and home tenant for subscriptions that the current account can access. You can also pass this values in as a parameter e.g. "Get-AzSubscription -SubscriptionId" to find the name of a subscription. 

The easiest way to execute this cmdlet (without having to install anything on your machine) is by using Azure Cloud Shell.  A handy feature of Azure Cloud Shell is predictive intellisense - use the RightArrow key to accept an inline suggestion.

Book tips: Gently easing into Agile and DevOps with the Phoenix project and the Unicorn project

If you are not familiar with Agile or DevOps, I highly recommend the books below:

• The Phoenix Project: a Novel about IT,DevOps and Helping Your Business Win (Gene Kim) is a book which illustrates how Agile principles can be applied to solve complex problems in IT. It is  written like a novel, which makes it both educational and entertaining. The main character, Bill Palmer, an IT manager is tasked to rescue a critical business project named Phoenix Project. Thebook is best suited for IT professionals, managers and executives. 
• The Unicorn Project: A Novel About Developers, Digital Disruption, and Thriving in the Age of Data (Gene Kim) was written 6 years later and takes us back to the same fictional company but this time we get the perspective from the people directly involved in development, testing and operations. 

If you have worked in consultancy or software development, you will have surely have a number of "aha moments" which will bring a smile to your face ...

Power Platform Managed Identity support for Dataverse Plugins (preview)

This is a long overdue feature which allows Dataverse plug-ins to connect to Azure resources (e.g. Azure Key Vault) using managed identities, eliminating the need for credential management and enhancing security.   It currently is available in public preview but GA is expected for November 2024

You will need to create a managed identity record in Dataverse and link it ot your specific plugin assembly - for more details take a look at Set up managed identity for Power Platform (Preview) and Power Platform managed identity overview (Preview) for more details.

In your plugin, you will need to the implement the IManagedIdentityService interface which allows your plugins to interact with managed identities. Using this interface, your plugin can request an access token for a specific Azure resource. I asked Copilot to generate the code for this below but did not check it yet ...

References:

• Announcing public preview of Power Platform Managed Identity support for Dataverse plugin-ins

Unmasking hidden issues in Dataverse: the surprising role of event expander operations in System job logs - down the rabbit hole we go

 As outlined in Large AsyncOperationBase increase in Dataverse/Dynamics 365 CE: the canary in the coalmine - an increase in the amount of storage consumed by system jobs (visible in the Power Platform Admin Center storage capacity report) is a tell-tale for a Dataverse or Dynamics CRM environment which has some hidden problems.

The AsyncOperationBase table keeps a record of all asynchronous jobs (Async plugins, async workflows, internal Microsoft jobs, etc ...) which are running in the background processing data in your environment. If you have a lot of failed or cancelled jobs, there might be an issue with plugins or workflows. If you see a lot of awaiting resources jobs, there might be a big data load happening on your environment (or maybe an infinite loop).

During a periodic review of the System jobs health status, we noticed that there were a lot more jobs in status "Waiting for resources" then we were used too (More than 200.000). We raised a Microsoft support ticket for this and we got an update on this (redacted version below):

"Microsoft is rolling out a new way of how audit logs are being written in station 4 (EMEA) in a deferred manner. Entities representing these deferred operations are created in the AsyncOperationBase table. 

These operations, while rolled up in the AsyncOperationBase table, execute outside of the Async Service and are not meant to be interpreted as additional backlog that the Async Service needs to process. These operations have no negative impact on System Job throughput. When the Audit operation has been fully processed outside of the Async Service, these operations will be removed from the AsyncOperationBase table. Event Expander Operation jobs are used as part of this new audit functionality, these are important jobs to ensure auditing is not lost. 

These jobs are however processed by a separate service, so they do not affect async throughput, etc. in any way.  Seeing a lot of these jobs (operation type 92 - event expander operation) is not an issue, as these are constantly churning in order to write audit history. If you have custom reporting in place to monitor system jobs - you should exclude AsyncOperationType 92"

Quick tip: Don't bother upgrading very old Raspberry Pi OS version with apt upgrade, just re-install

A couple of weeks ago, I uncovered an old Raspberry Pi 2 in a drawer and I wanted to try to upgrade the OS to a more recent version. The Raspberry PI still  had  Debian 8 (Jessie) installed.  After a number of failed attempts I just gave up and did a fresh install with Debian 11 (Bullseye)

PS To check the installed Raspberry PI OS version type "cat /etc/os-release" in the command prompt or "hostnamectl"

From boring to soaring: why leaders need to master story telling

"The most powerful person in the world is the storyteller - Steve Jobs"

I recently read the Harvard Business Review article : your strategy needs a story and the central idea resonated with me: "storytelling is crucial for business leaders because it effectively bridges the gap between strategy and action, making complex strategies engaging, memorable, and motivating for implementation1". Steve Job's quote also aligns with this view on the importance of storytelling in business and marketing

Story telling transforms the complexity of numbers and logic behind a business strategy into simplicity and action.  (Narrative and numbers: the values of stories  by Aswath Damodaran is a great book on this same topic, but with a different take on it)

93% of a company's employees do not understand its strategy and storytelling can be a solution to make sure that employees understand and implement the strategy and customers understand your value proposition. 

PS Although the quote "The most powerful person in the world is the storyteller" is often attributed to Steve Jobs, there's no definitive evidence that he actually said it.

Dynamics 365 Customer Service optimization resources webinar series from FastTrack team

 The Microsoft FastTrack team published a series of videos with best practices and guidance for Dynamics 365 Customer Service implementations in the last couple of months - here is an overview:

• Introduction to implementation resources, samples and tools for Dynamics 365 Customer Service | April 16, 2024  
• Implementation optimization resources for Dynamics 365 Customer - Channel Integration Framework (CIF) | May 13, 2024 
• Implementation optimization resources - D365 Customer Service (Samples and tools) | May 29, 2024 
• Implementation Optimization resources - Testing for D365 Customer Service | July 11, 2024  - 
• Implementation Optimization resources - Dynamics 365 Analytics for Service | July 17th, 2024  - 

How to improve your blog posts with Microsoft Copilot

 Here is a little trick that I found on how you can improve your blogposts with Microsoft Copilot. Use this two prompts:

1. Prompt 1: Read the blog post below and suggest 3 questions that a reader will have that are not answered in the article: (Add URL)
2. Prompt 2: Now answer the questions. Avoid bullet-points.

It is however to always review the answers before you just copy/paste them into your blog - the generated content might not be correct - this is referred to as Generative AI hallucinations 

Quick tip: disable meeting invite notifications in Teams activity feed.

There is a new (annoying) functionality which  show meeting invite notifications in your Microsoft Teams activity feed. To disable this go to your  Teams settings and open the 'Notifications and activity' menu on the left sside. At the bottom you will find the calendar notifications which you can change to 'Off'. See Manage notification settings on Microsoft support for more details

Quick note: Python date and time objects

In Python, a naive datetime object is one that does not contain any information about time zones or daylight saving time. This means it is unaware of the context in which it exists, such as whether it represents local time, UTC, or any other time zone. By default, the datetime object in Python is naive. You can make them timezone aware using the pytz library.

If you are working with pandas dataframes or series, you can also use the tz_localize method of the Pandas DateTimeIndex object.

References:

•  datetime - basic date and time (Python docs)
• pandas.Series.tz_localize

Quick Tip: Azure Synapse Link Troubleshooting Guide

 If you encounter issues when setting up Azure Synapse Link for Dataverse - the first place to check is the Azure Synapse Link Troubleshooting Guide - https://aka.ms/synapselinkTSG .

Related posts:

• Implementing Azure Synapse Link for Dataverse: gotchas and tips
• Near real-time and snapshots in Azure Synapse Link for Dataverse
• Azure Synapse Link for Dataverse playlist by Scott Sewell

Quick tip: cheap static website hosting in Azure Storage

Azure Storage's static website hosting feature provides a serverless solution for serving static content such as HTML and images, directly from the $web container. It's a cost-effective option since there's no charge for enabling the feature, with expenses arising only from storage use and operations. Additionally, it includes traffic metrics for easy monitoring of visitor statistics.

References:

• Static website hosting in Azure Storage
• Tutorial: host a static website using Azure Storage

Start CRM development tools from command prompt with Power Platform CLI

It is important to keep your Dynamics 365 development tools up to date, in the past I did this by using a PowerShell script provided by Microsoft. 

But now the Plugin Registration Tool (prt) and Configuration Migration Tool (cmt) are part of the Power Platform CLI - see Dataverse  development tools for more info - so you can launch the Plugin Registration Tool easily from command line using "pac tool prt"

When you install the Power Platform CLI with .NET tool (which requires .NET 6.0 to be installed) you need to use the same mechanism to keep the CLI up to date.

Power Automate license enforcement - looking at it from a Dynamics 365 CE perspective

Mid May 2023, a warning popped up in Microsoft Message Center regarding "Non-Compliant Power Automate Flows" -  soon after the message however disappeared so this got probably missed by the majority of Dataverse and Dynamics 365 admins.

This however raised some concerns in the broader Microsoft community ( see Will Power Automate enforcement licensing kill your flows?  and Upcoming licensing enforcement in Power Automate explained ). 

To be honest I did not pay a lot of attention since the message apparently vanished in thin air and after consultation with Microsoft support they said that this message was sent prematurely. But then, end of October another warning popped up.

So it seems that Microsoft is finally cracking down on Power Automate flows which are not associated with a properly licensed user for premium connectors or Power Automate flows not directly linked to a Power App. When you built your own model-driven app on top of Dynamics 365 Customer Engagement (CE) which uses Power Automate flows, you will need to associate the Power Automate Flow with with your new app.

There is a PowerShell script to identify the flows which at risk to be turned off  across your tenant - see I have many environments - how can I get the flows that need my attention across tenants in the Power Automate Licensing FAQ - which uses the Get-AdminFlowAtRiskOfSuspension cmdlet

The Get-AdminFlowAtRiskOfSuspension cmdlet is part of a separate PowerShell module which you can install using Install-Module -Name Microsoft.PowerApps.Administration.PowerShell. It will run a scan of your environments and outline

Check out Associate flows with apps - Power Automate | Microsoft Learn on how you need to link up a flow with an app (see below screenshot on where to do this in the Power Automate flow detail screen). If you make this change on a flow which is a part of a solution, then the associations will be part of the solution file and can be transported cross environments.

Related articles/blog posts:

• Will Power Automate enforcement licensing kill your flows?
• Upcoming licensing enforcement in Power Automate explained
• Flows in app context - is there a clear definition of what it means?
• How do you license Power Automate flows when using them with Power Apps

The ABC of AI: Retrieval-Augmented-Generation (RAG) and grounding

This is the first in a series of blog posts about more advanced generative AI and Large Language Model (LLM) concepts which I use as notes to myself (check out Why I blog and you might want to consider it as well)

Retrieval-augmented generation (RAG) is an AI framework that enhances the quality of responses generated by large language models (LLMs). LLMs are trained on a massive amount of data and understand statistical relationships between words but lack true comprehension of their meanings. So when faced with specific questions in a dynamic context so that is where RAG comes in.

RAG integrates information retrieval into LLM answers by using these steps:

1. User inputs prompt: when you ask a question, RAG uses your input prompt
2. RAG retrieves relevant information from an external knowledge base based on the user prompt
3. RAG combines this external content with your original promt creating a richer input for the LLM

RAG and grounding are related concepts in the context of enhancing LLMs. Grounding is the process of providing LLMs with information about a specific use-case. RAG is one of the techniques which is used for grounding (another technique is dense retrieval - see Dense X Retrieval: What retrieval granularity should we use  for more details). 

Microsoft Copilot offers a good example of grounding and RAG in use. The reason why Copilot is able to give more targetted responses, is because it uses grounding to improve the specificity of the prompt. 

Copilot uses Microsoft Graph, which can retrieve information about relationships between users, activities and organizational data (like info in Power Platform/Dataverse and/or Dynamics 365, info from e-mails, chats, documents and meetings) as part of the prompt grounding process. Microsoft Copilot will use the user prompt and additional info retrieved through Microsoft and then sends it to the LLM. For more details see Microsoft Copilot for Microsoft 365 overview

The most common systems to provide external data for RAG LLMs are vector databases and feature stores.

References:

• Implement Retrieval Augmented Generation (RAG) with Azure OpenAI Service (Microsoft Learn)
• RAGAs- How to evaluate RAG pipelines chatbot
• Intro to Retrieval-Augmented Generation (RAG) with Generative AI and OpenAI on Azure
• Azure Cosmos DB - Vector Database
• How vector search and semantic ranking improve your  GPT prompts (Microsoft Mechanics)
• The 5 types of LLM apps (YouTube)
• RAG application with Azure open AI & Azure Cognitive Search (French legal use case - Python notebook)
• Microsoft Copilot for Microsoft 365 overview
• Grounding language model with chunking-free in-context retrieval
• Grounding LLMs
• Langchain - use cases - Q&A with RAG
• Retrieval Augmented Generation (RAGs) for LLMs

Dynamics 365 and Power Platform monthly reading list April 2024

Copilots, AI and machine learning

• Artificial Intelligence 2023 Playlist by Stanford Online (YouTube) and Stanford CS229: Machine Learning | Spring 2022 Stanford Online (YouTube)
• Use Semantic Kernel with Lakehouse in Microsoft Fabric
• Fabric end-to-end use case: Analytics engineering part 1 - dbt with the Lakehouse by @s_debruyn
• From Microsoft to global brands, Dynamics 365 Copilot is helping transform customer experiences across service, sales and marketing
• Accelerate support ramp-ups, handoffs and wrap-ups with Copilot summarization in Dynamics 365 Customer Service
• Introducing new Copilot experiences to boost productivity and elevate customer experiences across the organization
• Transform your business with Microsoft AI (Microsoft Learning Path)
• Accelerating revenue with AI Powered Solutions: a CRO Guide (e-book)
• Microsoft Azure AI Fundamentals: Generative AI (Learning path)
• 10 AI terms everyone should know
• Revolutionizing marketing workflows with Copilot in Dynamics 365 Customer Insights
• Creating Custom Copilots: Azure AI Studio and Copilot Studio. When to use what?
• Build your Copilot testing strategy in Dynamics 365 Customer Service
• Learn Live: Microsoft Learn AI Skills Challenge
• The Art and Science of working with AI
• Microsoft Copilot in Azure extends capabilities to Azure SQL Database (Private preview)
• How to connect Copilot Studio to Dynamics 365 Customer Service (YouTube)
• Introducing Copilot pane in Power BI Desktop (preview)
• Enable Copilot case and conversation summaries 
• Trust Micrsoft Copilot with your Power Platform and Dynamics 365 data - Power CAT live (YouTube)
• Building Modern Enterprise Solutions: Microsoft Dataverse and Microsoft Copilot
• Microsoft Copilot scenario library - real use cases for HR,Finance, IT, Marketing and Sales
• Microsoft Copilot for Sales FAQ
• Day 19: Microsoft Copilot vs Microsoft Copilot for Microsoft 365 vs Copilot aka PVA
• Announcing Microsoft Copilot for Service
• Enable Copilot in Customer Service and custom apps (Dynamics 365)
• Microsoft Copilot for Service: Introducing new Copilot experiences to boost productivity and elevate customer experiences across the organization  and Copilot in Microsoft Dynamics 365 (YouTube)
• Generative AI for beginners (Microsoft)
• Large language models, explained with a minimum of math and jargon
• Using copilot with SharePoint knowledge to serve your customers
• Dynamics 365 Customer Insights - Deliver higher-impact experiences with better data quality and configuration with Copilot
• Information Assistant Industry accelerator on GitHub
• Announcing Microsoft Copilot for Service and Microsoft Copilot for Service product page
• Build your own copilots with Azure AI Studio (Microsoft Mechanics recording)
• Build GPT-automated customer support with Azure Communication Services

 

Technical topics (Configuration, customization and extensibility)

• Lakehouse vs Warehouse vs Datamart - the difference between the three Fabric objects
• Power Platform Well-Architected framework
• Creating service principals really easily using PAC CLI by @carldesouza
• The power of Prefer Headers in the  Dynamics CRM web API
• Analyze Dataverse tables from Microsoft Fabric
• Enterprise security with Power Platform (Microsoft white paper)
• Microsoft Fabric, explained for existing Synapse users
• Announcing new cloud governance guidance in the Microsoft Cloud Adoption Framework (CAF) for Azure
• PCF Learning Roadmap
• Mask sensitive data and prevent unauthorized access and Dataverse: about masking rules 
• Protect enterprise solutions with new Microsoft Power Platform security features
• Power Platform Wave 1 2024: Top 10 features you need to know! (YouTube)
• Add intelligence into your apps with Dataverse AI functions (YouTube)
• Leverage the Power Platform API to create a capacity report
• Associate activities with multiple related records (2023 Wave 2 Public preview)
• Power Platform tenant settings  (YouTube)
• Working with solution dependencies for objects made easier
• Improve solution import time
• https://aka.ms/azai - develop apps that use Azure AI services
• How to create a new Dynamics 365 SCM trial environment from the Power Platform Admin Center (PPAC) preview
• What's new in Playwright 1.32 (YouTube)
• Using Application Insights for Dataverse monitoring
• Modernize customer support with Copilot in Dynamics 365 Customer Service
• Announcing the Power Platform solutions with AI and Copilot learning path
• Sample library for Digital Contact Center Platform

Topics for Dynamics 365 Business Applications Platform consultants, project managers and power users

• Modernize Service Guided Tour
• Forrester TEI study shows 315% ROI when modernizing customer service with Microsoft Dynamics 365 Customer Service
• End of support for Project Service Automation (PSA 3.x)
• Model driven Apps (Power Platform)  visual glossary
• My annual letter: leading in a new era - annual letter from Satya Nadella, CEO at Microsoft
• Finding adequate metrics for  outer,inner and process quality in software development
• Customer story: Avanade puts people first, accelerates sales productivity with Microsoft Copilot for Sales
• Keeping up to date with changes to the Release Wave Plans
• A consultant's guide: overcoming the challenges of joining a Microsoft Dynamics 365 project mid-way through
• Accelerate Copilot extensibility and Power Apps creation with Microsoft Dataverse
• How Microsoft and generative AI are transforming financial services contact centers

SQL Server Integration Services Project template available for Visual Studio 2022

 Since end of 2022, there is also a SQL Server Integration Services  Project template available for Visual Studio 2022 which you can install from the Visual Studio Marketplace. You can install it from the direct download link here or you can search for it in the Visual Studio 2022 extension manager and install it from there.

Classic Azure Application Insights deprecated on February 29th 2024 - 7 days to go

 If you missed it - classic Azure Application Insights will be deprecated on February 29th 2024. If you missed the different notification e-mails, you can quite easily see the warning if you navigate to an Azure Application Insights resource in Azure Portal.

Migration is actually quite easy - you just click on the link provided and this will open up the menu depicted below which allows you to associate your Azure Application Insights resource to a Log Analytics Workspace. The good news is that there are no pricing changes when moving to the workspace-based model. 

As indicated in the migration window, this is  a one way operation so plan for it in advance - the points below might impact on how you will do the migration:

• You can link different Application Insight resources to a single Log Analytics workspace or you can make the split - in most case you want to consolidate it.
• Instrumentation keys do not change during the migration so you don't need to worry about this
• The export feature is not available on the Application Insights workspace-based resources - you need to look at diagnostic settings for exporting telemetry
• There might be some schema changes - important to consider when doing KQL queries - check out query data across Log Analytics workspaces, applications and resources in Azure Monitor
• Existing log data will not immediately move to the Log Analytics workspace - only new logs generated after the migration will be stored in the new log location.

Procreate 5 handleiding video overzicht

Lijst van ondersteunende video's bij het boek Procreate 5 Handleiding - tekenen op de ipad (Stefan de Groot, 2020)

• Hoofdstuk 2 - Gallery
• Hoofdstuk 3 - Canvas & brushes
• Hoofdstuk 4 - Teken, veeg, gum en vormen
• Hoofdstuk 5 - Layers
• Hoofdstuk 6 - Kleuren
• Hoofdstuk 7 - Gebaren
• Hoofdstuk 8 - Actions
• Hoofdstuk 9 - Adjustments
• Hoofdstuk 10 - Selecteren en transformeren
• Hoofdstuk 11 - Snelkoppelingen & Apple Pencil