Tuesday, April 28, 2020

What can you do with the Azure Cosmos DB free tier?

Beginning of March 2020, Microsoft announced the availability of a free tier of Azure Cosmos DB (see Azure Cosmos DB Free Tier is now available)

"When free tier is enabled on an Azure Cosmos DB account, you’ll get the first 400 RU/s and 5 GB of storage for free for the lifetime of the account. Additionally, when using shared throughput databases, you can create up to 25 containers that share 400 RU/s at the database level. There’s a maximum of one free tier account per Azure subscription and you must opt-in when creating the account."

But maybe you are wondering what you can actually do with 400 RU/s? Request Units per second (RU/s) represent the "cost" of a request in terms of CPU, memory and IO. In Azure Cosmos DB you can provision "performance" upfront by setting RU/s at database level, collection level or both. It is however also possible to create Azure Cosmos DB containers and databases in autoscale mode  Containers and databases configured in auto scale mode will automatically and instantly scale the provisioned throughput based on your application needs without impacting the availability, latency, throughput, or performance of the workload globally

I can highly recommend you to take a look at the Microsoft Ignite 2019 session - A developer's guide to Azure Cosmos DB, from onboarding to going live in production . The code samples from session are available on Github - https://github.com/deborahc/cosmos-perf-tips

References:

Monday, April 27, 2020

Things to watch out for when configuring the export of CDS data to Azure Data Lake

End October 2019 Microsoft announced general availability of the export to Azure Data Lake functionality  (previously called Project Athena) for CDS and Dynamics 365 (Sales, Customer Service and Field Service).



It is quite easy to setup by following the steps outlined in the announcement blog post  or the official documentation on exporting entity data to Azure Data Lake Storage Gen2 - there are however two important prerequisites:


  • The login that you use to configure the Export to Data Lake settings needs to be an owner of the Azure storage account. If you missed this step you will get an exception in the configuration wizard on the second screen.


References:

Wednesday, April 22, 2020

Working with multiple time series trading data from Quandl in Jupyter Notebooks

In the previous example - Using Euronext stock data from Quandl in Jupyter notebooks I downloaded a single dataset from Quandl. But it is also possible to download multiple datasets by passing in a list of Quandl codes.

In the example below, I downloaded the prices of a number of diversified holding companies which are traded on Euronext Brussels and compared the cumulative returns (not including dividend payments) using Jupyter Notebooks.


The Quandl Python API allows you to make a filtered time series call and request only specific
columns - in this example the 'Last' (Closing price) is retrieved by specifying the index 4. In a next
step I renamed the columns in the pandas dataframe to make it easier to work with the data
afterwards.



Take a look at the full python notebook at https://github.com/jorisp/tradingnotebooks/blob/master/Quandl_Belgian_Holdings-Shared.ipynb to see how this data can be used to visualize cumulative returns for these different stocks



 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
%matplotlib inline
import quandl
import matplotlib.pyplot as plt


quandl.ApiConfig.api_key = "<Your Key Here>"

#Retrieve Last price only for the 5 holdings (excluding mono holdings) trading on Euronext Brussels
#Data is available from February 2014 onwards - Ackermans Van Haren (ACKB), Brederode (BREB), Sofina (SOF), 
#GBL and Bois Sauvage (COMB )
data = quandl.get(['EURONEXT/ACKB.4','EURONEXT/BREB.4','EURONEXT/SOF.4','EURONEXT/GBLB.4','EURONEXT/COMB.4'])

#Rename column names 
data.rename(columns={'EURONEXT/ACKB - Last': 'ACKB', 'EURONEXT/BREB - Last': 'BREB','EURONEXT/SOF - Last':'SOF',
                     'EURONEXT/GBLB - Last':'GBLB','EURONEXT/COMB - Last':'COMB'},inplace=True)

Monday, April 20, 2020

Dynamics 365 and Power Platform monthly reading list March 2020

Dynamics 365 and Power Platform - 2020 Wave 1 Topics


Topics for Dynamics 365 Business Application Platform consultants, analysts, project managers and power users

Using Euronext stock data from Quandl in Jupyter notebooks

The last couple of weeks I have been learning about Python and how to use it for stock and derivative trading. One of the challenges is getting stock trading data for European stocks (without having to pay for it).  One of the first things I started with is using Jupyter notebooks to quickly visualize stock market information.

The easiest way to get started with Jupyter is using an all-in-one Python distribution - the one I used is Anaconda since it is easy to setup and it includes a number of interesting libraries I want to use in next steps.



I like to try out things hands-on but I did use a number of training resources to get up to speed:
To get trading data about European stocks I used QuandlQuandl is a marketplace for financial and economic data which is either freely available or requires a paid subscription. Data is contributed by multiple data publishers like World Bank, trading exchanges and investment research firms. Quandl provides REST API access to the available data sets but also has specific Python and R libraries. You first need to register to get an API key. A lot of European stocks are traded on Euronext and Quandl provides you access to Euronext data - https://www.quandl.com/data/EURONEXT-Euronext-Stock-Exchange

Install the quandl Python package using the Anaconda command prompt. It is best to setup virtual environments to manage separate package installation that you need for a particular project, isolating the packages in other environments but for simplicity I just installed in the base environment.

Next it is quite easy to retrieve stock data from Quandl - you first import the quandl package and next you call the quandl.get() method. By default, Quandl will retrieve the dataset into a pandas DataFrame. Since I specified no additional parameters, the entire timeseries dataset was retrieved - from February 2014 until now. Afterwards I used the plot command which uses the matplotlib library to display a graph of the closing prices.



For the full Jupyter notebook take a look at Github https://github.com/jorisp/tradingnotebooks/blob/master/Quandl_API_Euronext_ABI_Shared.ipynb


Thursday, April 02, 2020

Dynamics 365 and Power Platform monthly reading list February 2020

Dynamics 365 and Power Platform - 2020 Wave 1 Topics
Technical topics  (Configuration, customization and extensibility)

Sunday, March 29, 2020

Quick tip: Skype for Business and Microsoft Teams network assessment tool

To ensure that your network meets the requirements for using Skype for Business or Microsof Teams for audio and/or video calls, you can download and run the Skype for Business Network Assessment tool.




Sunday, March 22, 2020

Generating Azure Application Insight Key in Azure DevOps pipeline

If you want to generate an Application Insights key in your Azure DevOps pipeline - you can use the the PowerShell code snippet below in a Azure PowerShell task.


Look at Automate Azure Application Insight resources using PowerShell and New-AzureRmApplicationInsightsKeyApiKey for more details.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
Install-Module AzureRM.ApplicationInsights -force -verbose -scope CurrentUser
Import-Module AzureRM.ApplicationInsights

$resourcegroupname ='rg_func****'
$aicomponentname ='func****'
$permissions = @("ReadTelemetry", "WriteAnnotations")
$apikeydescription = 'testapikey'

New-AzureRmApplicationInsightsApiKey -ResourceGroupName $resourcegroupname 
-Name $aicomponentname -Description $apikeydescription -Permissions $permissions

Friday, March 20, 2020

Azure Application Insights for Dynamics 365 and Power Platform solution architects and consultants

A best practice which is quite often overlooked is enabling monitoring and logging capabilities in your applications or solutions. You should always monitor your applications/solutions whether they run in the cloud or on premise because you want to know when something fails (before the users start complaining) and also understand how users are working with your solutions.



Both Dynamics 365 and Power Platform have the option to enable integration with Azure Application Insights - but what is Application Insights?

Azure Application Insights  is a feature of Azure Monitor and it is basically an APM (Application Performance Management) tool. Application Insights provides standard integration with a lot of Azure components adding automatic monitoring capabilities but you can also extend it with custom monitoring so if you get an exception in your code that is unhandled Azure Application Insights will pick this up. You can use it for on-premise and cloud applications but the ease with which you can instrument your applications might differ.


You can integrate Application Insights into your model-driven apps (either Dynamics 365 first party apps which you extended or completely custom-model driven apps) by using the Javascript SDK (See Application Insights for web pages for more details). Unfortunately the old documentation on this topic has disappeared - https://azure.microsoft.com/en-us/documentation/articles/app-insights-sample-mscrm/ now does a redirect to the general document. 


Luckily @DanzMaverick wrote an interesting plugin for XrmToolBox which allows you to easily instrument your different model driven forms.  Be sure to update to the latest version though since it contains a number of fixes. If you want to keep up to date with updates on this interesting tool follow https://github.com/Power-Maverick/ApplicationInsightsManager




Using Application Insights Manager plugin you will be able to instrument a number of different model forms using just a number of clicks - behind the scenes it will wire up your forms with the required JavaScript web resources.



A recent update on the Power Platform also allows you to log telemetry for your Canvas Apps using Application Insights.

Dynamics 365 can also leverage Azure components like Azure web apps, web jobs or Azure functions which will also require monitoring and logging capabilities. Depending on the type of component you might get out of the box monitoring or you might need to use code based monitoring - most scenarios are described in the documentation - https://docs.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview

Things I learned when using Azure Application Insights in the last couple of months:

Tuesday, March 10, 2020

Enabling C# 7.1 in Visual Studio 2017 projects

Today, when I tried to compile one of the Visual Studio projects in our solution - I received the exception "Feature 'default literal' is not available in C# 7.0. Please use language version 7.1 or greater". (For more info see default literal in C# 7.0)



To enable C# 7.1 you need to following these steps:

  • Right click on your project and select Properties, next navigate to the build tab and click Advanced

  • Next you can either choose C#7.1 from the list or choose C# latest minor version (latest).

Monday, March 09, 2020

Dynamics 365 and Power Platform monthly reading list January 2020

Dynamics 365 and Power Platform - 2020 Wave 1 topics

Starting April 2020, new features in Dynamics 365 and Power Platform will be activated, but you can already try out some of the new features by enabling early access - for a full list of available features see 2020 release wave 1 features available for early access
Technical topics (Configuration, customization and extensibility)

Forms Pro Quick tip: switch between Microsoft Forms Pro and Microsoft Forms

Microsoft Forms Pro is an enterprise survey tool which leverages both Microsoft Forms and the Common Data Service (For more details see Getting started with Microsoft Forms Pro: customer feedback is the ultimate truth.)

Once you are assigned a Forms Pro license, it is not so obvious that you can still use Microsoft Forms (at least not for me) To switch between Microsoft Forms Pro and Microsoft Forms you should select your photo in the Office 365 header and then select Switch to Forms



Thursday, February 06, 2020

Quick tip: keep your Dynamics 365 development tools up to date using Nuget

I recently encountered an issue when I tried switching the view of the registered plugins & custom workflow activities in the Plugin Registration Tool.



When updating to the latest version using the PowerShell script listed on Download Dynamics 365 development tools from Nuget the error got resolved. Lessons learned: make sure that you keep your Dynamics 365 development tools up to date.

Monday, February 03, 2020

Lessons learned about Dynamics 365 solution layering - Part 1


This is the first in a series of blog posts on Dynamics 365 solutions and solution layering in which I will cover the basics about Dynamics 365 solutions and gradually delve deeper into the mysterious world of the inner workings of Dynamics 365 solutions.

In my opinion,  based on how managed solutions and solution layering work - I think that in the majority of cases you should only use a single managed solution to avoid issues down the line. Read on to see why...


This blog post assumes that you are using managed solutions to deploy to all environments and only have unmanaged solutions in your development environment.

Dynamics 365 solutions fundamentals

If you work with Dynamics 365/CDS solutions, you should be aware of the concept of solution layering. Solution layering occurs on import of solutions when a specific component is affected by a change within one or more solution. Solution layering will determine the kind of behavior that a user will see in Dynamics 365.

The figure above shows how layering works: at the bottom, you have the standard CDS solutions on top of which you might have some first party apps installed like Dynamics 365 Sales/Customer Service etc... If you deploy custom managed solutions, they will go on top based on the order in which they are installed. Direct customizations and unmanaged solutions are always at the top but there is no real layering of unmanaged solutions - they all end up directly modifying the base behavior.

Solution layers describe the dependency chain of component from the root solution introducing it, through each solution that extends or changes the component’s behavior. Solution layers are created through extension of an existing component (taking a dependency on it) or through creation of a new component or version of a solution. So it is very important to understand that layers should be considered on a per component basis.

In April 2019, Microsoft introduced a new user interface component which visualizes the different solution layers which are impacting the behavior of a specific component - View solution layers for a component. Below is an example of the solution layers of the case form which was quite an interesting example.


Going beyond the basics

But what happens if two or more solutions define solution components differently? This article  - Introduction to solutions - conflict resolution explains the two conflict resolution strategies - Merge and Top Wins. The article is also valid for online according to Microsoft support - even though it was written for Dynamics 365 on premise and last updated in December 2017.


When importing a managed solution with the overwrite customization option, it is possible to override the "top wins" conflict resolution  but this does not apply to components which apply a “merge strategy” - for more details see Understand how managed solutions are merged

So you need to be very careful when adding components which apply a merge strategy in multiple solutions with different publishers since the merge behavior might cause unexpected side effect such as creating a separate active layer on top which would block your customization to appear to the user.

An active layer can emerge in two scenarios - a direct customization was done in the instance but it seems that it is also possible that a managed active layer is created when using only managed solutions without direct customizations in the system. This happens when managed solutions in one system, with different publishers (the Microsoft solutions count as a different publisher) are trying to update the same component. It is a defense mechanism that Microsoft has built in when there are object inconsistencies across managed solutions.

The general recommendation is that you keep the number of managed solutions with common components (e.g. forms with a different form layout) to a minimum. This will avoid merging issues across the layers.

Single solution to rule them all?


Taking into account all of the above it seems that you would opt to use a single managed solution to deploy to other environments.

Single solution approach
Pros:

  • Easier to version control
  • Avoid solution layering issues
  • Less environments needed
  • Easier to promote across environments
Cons:

  • Longer duration of solution import
  • Long list of components


I do think that a single solution might make sense in the majority of cases but there are of course exceptions to this rule.  I would recommend you to take a look at the Solution Lifecycle Management: Dynamics 365 for Customer Engagement Apps whitepaper and especially the section on solution boundaries which outlines some of other valid scenarios for having multiple solutions e.g.  the scenario where you need to deploy to multiple regional production instances with small variations with a core solution approach and on top local solutions  or the scenario where you need to release subsets of applications on different cadence/timeline.


Friday, January 31, 2020

Dynamics 365 and Power Platform monthly reading list December 2019

Technical topics (Configuration, customization and extensibility)

Topics for Dynamics 365 Business Application Platform analysts, project managers and power users

Thursday, January 16, 2020

Dynamics 365 or Power Platform New Year's resolutions

We are already 3 weeks in 2020 but there is still time for New Year's solution so choose one of the 12  potential Dynamics 365/Power Platform New Year's resolution as proposed in Episode 69 of the CRM MVP Podcast hosted by @gusgonzalez2.



I will try to take at least 10 out of 12 in 2020 so what is yours?
  1. Write a valuable blog article every week - I probably want to go for 52 blog posts this year.
  2. Deliver at least 4 webinars or  community presentations in a year
  3. Register for the Dynamics Insider program and commit to test every new release and provide feedback to Microsoft
  4. Answer 60 questions on the forums (Microsoft Dynamics Forums, CRMUG Forums, etc...) 
  5. Attend a training from a Dynamics 365/Power Platform expert that you admire or who you consider to be a top expert in the domain
  6. Create a XrmToolBox plugin 
  7. Speak at a major conference
  8. Contribute 24 tips to @crmtipoftheday by sending an e-mail to jar@crmtipoftheday.com
  9. Learn something new about Power Platform/Dynamics 365 every month - you can learn a lot by dedicating 8 hours a month on a single topic
  10. Teach something new about Power Platform/Dynamics 365 every month e.g by delivering a lunch&learn session in your company
  11. Use "new" functionality in Power Platform/Dynamics 365 at least once every month e.g. use Flow/Power Automate instead of using workflows or use the new Admin portal and solution designer
  12. Release a free solution to the community  e.g. a PCF control 

Update on Dynamics 365/CDS request limits

End of August Microsoft announced an API based limitation which is based on users and the type of licenses they have - the latest documentation is available on http://aka.ms/platformlimits  as well as PowerApps and Microsoft Flow licensing FAQs for October 2019. I would recommend regularly checking these pages as they have been updated quite a few times in the last months.

During interactions with Microsoft the last couple of months, they explained that the allocated number of API calls within the different licenses are based upon internal telemetry on the current Dynamics 365 customer base. The claim is that 95% of customers fall within the standard allocated API limits. But if you are using a lot of integrations, you might need to  re-architect part of your solution.

Listed below are the key takeaways:
  • Users with a Dynamics 365 Enterprise Application license have 20.000 API requests allocated in a 24 hour window.
  • Technical/non-interactive/application users get allocated 100.000 API requests if at least one Dynamics 365 API license is available.
  • If a user exceeds the limits the admin for the tenant/environment will receive a notification - end users will not be blocked from using the app.
  • This new licensing went into effect for new customers who on boarded after October 2019. Existing customers have a transition period until October 2020 or when their licensing contract expires. Whichever is longer. For customers with an enterprise agreement this will be the end of their EA (in most cases I know these contracts are valid for 3 years), customers on a CSP contract typically have a yearly expiration date. Reach out to your licensing partner or Microsoft for more details. 
  • The currently available statistics in the Power Platform Admin Center are still quite rudimentary but are a good starting point to assess the impact on your environment 
  • It is possible to purchase additional blocks of 10,000 daily API requests for $50 per month (For details reach out to your licensing partner)
  • Batch requests (Executemultiple) only count as 1 API call, so you can wrap a 1.000 individual calls in one ExecuteMultiple call.
References:

Tuesday, January 14, 2020

My perspective on the PowerApps Build Tools for Azure DevOps

Mid July 2019 Microsoft released a preview of a set of PowerApps specific Azure DevOps Build tasks.  In the last months this tooling has been updated on a quite regular pace which indicates that  ALM (Application Lifecycle Management) for Power Platform (and Dynamics 365 Sales/Customer Service/etc..) is high on the priority list for Microsoft.


For those of you who are new to Azure DevOps, here is a small summary. Azure DevOps is a set of services hosted on Microsoft Azure cloud which support your full software development lifecycle  e.g. you can use Azure Boards for work tracking and backlogs, Azure Pipelines for  CI/CD, Azure Repos for source control, and much more.  Azure DevOps is successor to  Visual Studio Team Services (VSTS) and the best thing of all you can get started with it for free. (For more details see Pricing for Azure DevOps). You can start learning Azure DevOps by exploring the Azure DevOps Hands-On Labs

In the past most Dynamics 365 CE consultants largely relied on a BYOALM (Bring Your Own ALM) approach meaning that you need a combination of PowerShell script, SDK extensions, etc … to automate the build and release of Dynamics components. I even think that in the majority of cases there is no fully automated build and release process in place - meaning that a deployment relies on a number of (hopefully documented) manual steps.  In one of the projects I recently worked on - we have been using the excellent Dynamics 365 Build Tools for Azure DevOps from Wael Haemze so there are other extensions available for Azure DevOps as well.

After you have installed the PowerApps Build Tools you will see a whole set of build and release tasks that you can use in your build and release pipelines. To explore the possibilities you can start with the PowerApps Build tools for Azure DevOps Hands On Lab files which contains a walk through of the different scenarios like for example using the PowerApps Solution Checker (see reference section for more information on this)



If you compare the PowerApps Build Tools with the Dynamics 365 Build Tools, you will probably see that Dynamics 365 Build Tools currently still offers more capabilities but it does seem worthwhile to start exploring the newly released Microsoft tooling.  I recently also got feedback within the context of a Microsoft support case that they recommended to use the new PowerApps build tooling because they would not troubleshoot issues with other extensions on top of Azure DevOps in combination with Dynamics 365.

I  also expect more information to come available in the coming weeks as we are getting closer to the Dynamics 365 and Power Platform 2020 Release Wave 1 . In the meanwhile I will be sharing more information in some upcoming blog posts.


References:


Tuesday, December 31, 2019

Dynamics 365 Customer Engagement release version transparency

By now, you should be aware know that all Dynamics 365 Customer Engagement Online environments are on the same “major” version but it is also important to understand that Dynamics 365 CE receives continuous updates on a weekly release cadence. You probably already saw these updates being announced in Office 365 message center.



Each update is deployed as part of a Release Train, stopping at each ‘station’ along the way before rolling on to subsequent station. Not all regions would be on the same release as any time. Early station will have lower levels of number of customers impacted, ending with Europe and North America where majority of our customers are and then dedicated scale groups where customers with larger organisations are hosted. For a more in-depth session on this topic I can highly recommend the recording BRK2039 - Microsoft PowerApps and Dynamics 365: Modernizing the way we update

In November 2019  Microsoft updated their documentation and provides a lot more transparency on this release process - so you can now get a full overview of Released Versions of Dynamics 365 for Customer Engagement which outlines all previous releases as well as the upcoming releases in the different stations.

The version number mechanism for Dynamics 365 CE instances has also been updated so it is now easier to see which update has already been rolled out to a specific instance.


Dynamics 365 and Power Platform monthly reading list November 2019

Technical topics (Configuration, customization and extensibility)

Topics for Dynamics 365 Business Application Platform analysts, project managers and power users