Bots and Dynamics 365 Part 1 : Introduction and building a Q&A bot

For those of you who want to see a glimpse of the potential of bots in combination with Dynamics 365 check out the Microsoft Data Amp 2017 Keynote (Channel 9 recording) but also some actual reference cases such as

• Insurance goes bot – artificial intelligence deployed in customer contact (VHV Versicherungen)
• UPS paves the way for better service with faster development and artificial intelligence

Bots are officially a hype in Silicon Valley  and while stories like It begins – Bots are learning to chat in their own language might depict a bright future for bots due to the recent advances in machine learning and artificial intelligence, developing a good bot will still require a good design and up front thinking about how you want your users to interact with the bot.  The user experience of poorly designed chat bots will frustrate users trying to perform more complex tasks. And even when this will not become a headline such as Tay, the Microsoft that pranksters trained to spew racist comments you will still have disappointed a potential customer.  But companies which start early experimenting with chat bots, are able to leverage advances in natural language processing and artificial intelligence and a number of different bot building frameworks.  The important thing to realize is that building a bot is not so much about the underlying technology but about creating a memorable customer experience – a great example is Lemonade sets new world record which show how AI handles a filed insurance claim in under 3 seconds.

Luckily, building the underlying platform has become a lot easier and Microsoft has also made available a number of different tools to help you in designing a great experience. Microsoft not only provides the Bot Framework (publicly available since April 2016) which includes a Bot Builder SDK and Developer Portal but also launched the Azure Bot Service  in the November 2016. Azure Bot Service has been built on top of the Azure Bot Framework and provides a number of templates to get started with the creation of bots. In theory you can code your bot directly from within the built-in editor but you will probably switch over to Visual Studio afterwards as well as integrate with your source code repository by downloading the generated bot code to your local machine.

One of the basic templates which is available in Azure Bot Service to write your bot is the the Question&Answer template.  To get started you need to built up a knowledgebase with questions and answers, information which might already exists in content such as FAQs, documents,etc…  Microsoft QnA Maker  will be able to use these existing sources and provides both a web interface and REST API that trains AI to respond to user’s questions in a more natural, conversational way. I took the example of https://www.statefarm.com/customer-care/faqs/auto-insurance and used it as input for QnA Maker to build up a knowledge base. Within QnA Maker itself you can test it afterwards  to see how your bot reacts to specific questions.

The next step is creating a bot using the Azure Bot Service – next to the QnA template there are also other templates available – see Bot Service templates  for more details. You can start experimenting with the bot service framework at a very low cost, the free model allows you to test it out with 10000 messages/month and you will only pay for the consumption of the underlying Azure functions  or you can use the premium pricing model which provides an SLA as well as as access to premium channels. (See Azure Bot Service pricing for more details)

After the bot service has been created, you will still need to connect it to the QnA service that you published previously by filling in the QnAKnowledgebaseId and qnASubscriptionKey in the App Settings of your bot which you can retrieve from the QnA settings page.

User interact with your bot through channels, the Azure Bot service provides fourteen channels such as Cortana, Facebook Messenger, Kik but also embedded web chat.  Web chat is the channel which is enabled by default, and you can start quite quickly by just embedding an iframe in your web 

You can also use a DirectLine (not web chat) channel, and start from the source code available on https://github.com/Microsoft/BotFramework-WebChat  to build an embeddable web chat control which is both available as a React component and a self-contained control which is easily usable by non-React websites.

But if you really want to leverage the full potential of chat bots, you will need to integrate the chat bot channel into the core systems and data repositories of your organization such as Dynamics 365 for Customer Engagement and Service Management. From a Dynamics 365 perspective there are some interesting generic use cases for bots:

• Customer Service – tier one support, bots can simulate 24/7 customer service representatives and act as tier one support and escalate case for follow-up by actual customer service representatives. 
• Customer Service – appointment scheduling for servicing such as maintenance, repair, etc …
• Lead generation - an embedded chat bot in your website can answer simple product and pricing inquiries from customers and at the same time create leads.

In a next post I will delve a little deeper into how you can integrate your bot with Dynamics 365 but you can already explore this topic yourself by looking into a number of blog series on this topic such as the series from @NishantranaCRM :

• Getting started with Microsoft Bot Framework
• Using dialogs in Microsoft bot framework (creating lead in CRM)
• Using FormFlow in Microsoft Bot Framework.
• Using LUIS AI in Microsoft Bot Framework Part 1
• Using LUIS AI in Microsoft Bot Framework.- Part 2
• Publishing Bot to Azure and adding it to Skype

Other references:

• Microsoft Azure announces industry’s first cloud bot-as-a-service
• Azure Bot Service Overview (Bot Framework Preview documentation) 
• Conversational bots deep dive – What’s new with the General Availability of Azure Bot Service and Language Understanding
• Batches and batches of Bots (Coding4Fun Blog)
• Dynamics COE series on bot framework (some articles might not be applicable anymore since the Azure bot framework has undergone some significant changes in the last couple of months) is also a good starting point:
• Bot Framework: 01 – Hello World
• Bot Framework: 02 – Dialogs
• Bot Framework: 03 – Nested Dialogs
• Bot Framework: 04 – Attachments and cards
• Bot Framework: 05 – LUIS (Language Understanding Intelligent Service)
• Bot Framework: 06 – Deploying your bot

Musings and rants about solution architecture, EA and consulting

The last year I have been working as a solution architect for a large European company as part of the EA team and I must say, it  has been quite an interesting experience. The vision/ideas that the other architects within EA have about software architecture made me explore a little bit on how software architecture has been perceived by the industry. During in this exploration I stumbled on a number of interesting articles that I want to share:

• The developers vs enterprise architects showdown: you shall know us by our trail of diagrams
• Multiple perspectives on technical problems and solutions
• The role, skills and duties of a software architect

I also became a big fan of Eric Dietrich (@daedtech) writings – so check out some of his postings

• Consulting for your current employer: make your boss your first client
• Defining the corporate hierarchy
• Hacking your career as a non-developer
• Conference speaking isn’t good  for your career until you make it good
• Learning in a world where programming skills aren’t that important
• The decline of the enterprise architect

Other architects you might also follow on Twitter include  Martin Fowler (@martin fowler) (author of a number of excellent books you should read as well) , Michael Nygard (@mtnygard)  Matt Stine (@mstine) and Simon Brown (check out his video on  Visualize, Document and explore your Software architecture). Leave a comment if you think I missed a thought leader in the software architecture space on Twitter or other blog post which you consider must reads or just interesting.

Dynamics 365 monthly reading list November 2017

Technical topics (Configuration, customization and extensibility)

• Mobile offline functionality on a tablet
• Using EWS editor to fix Dynamics 365 App for Outlook issues
• Implementing field mapping using Xrm.WebApi
• Just some tips for working with virtual entities by @andz_88
• How to create test data with GUIDs in Microsoft Dynamics 365
• Managing Dynamics 365 updates (updated article November 2017)
• Virtual entities – Relationships by
• Upgrade Adxstudio Portals v7.x to xRM Portals Community Edition
• Dynamics CRM Tip of the Day 1032: be careful with those (Javascript) function names
• Fixing missing services site map after upgrade from Dynamics 365 -  8.2 to 9.0
• Fixing Dynamics  Outlook app deployment issues
• Common Data Integration Patterns – the aggregation pattern
• Dynamics 365 portals: Liquid with a Javascript framework – vue.js
• Loading dependent javascript files in Dynamics 365 v9
• Microsoft Dynamics 365 v9.0 clientside for devs besides XRM.WebApi by @a33ik
• Where did my instance go – I hadn’t seen this glitch yet -  great tip from @DavidYack
• Register a new webhook through plugin registration tool in Dynamics 365 
• Dynamics 365 v9 unified interface icons
• CodeNow for #XrmToolbox – more than just code now… let’s get your organization POA stats this time

Topics for Dynamics 365 business analysts, project managers, power users and end users

• 4 steps that keep Microsoft Dynamics 365 implementations from running off the rails
• Dynamics 365 – a few things partners need to know better
• New automation and visualization features for business process flows (public preview)
• Dynamics 365 User Group Australia – SA Water’s digital journey and the role of Microsoft Dynamics (Youtube recording 00:20:13)
• Microsoft and Adobe build on partnership with integration of Adobe Experience Manager and Dynamics 365
• Does your CRM data resemble the walking dead?
• Release Notes for Field Service and Project Service Automation Update Release 2
• What’s new in Voice of the Customer version 9.0.823.1 to 9.0.959.8 – improved scalability – it is now possible to sent out 50.000 surveys per day
• Configuring Custom Controls for views in Dynamics 365 CE 9.0
• Using PowerApps and Microsoft Flow to simplify Dynamics 365 activity management
• List of values: optionset vs lookup
• Reports enhancements in Dynamics 365
• How to change the Panel Header of a sub grid in Dynamics 365
• It’s time to integrate Dynamics CRM and SharePoint (better) – some interesting add-ons to look into. (Remark: I haven’t tested them out yet – so don’t consider this as an official endorsement)
• Deciphering Dynamics 365 App for Outlook requirements and available features

The Dynamics 365 CE SDK is dead, long live the new Developer Guide

From Dynamics 365 9.0 onwards, the SDK (Software Development Kit) is not available as a single SDK monolithic download – but it has been split up in different parts:

• Developer Guide for Dynamics 365 Customer Engagement  (still possible to export as PDF) – definitely take a look at  Xrm.WebApi (Client API reference)

• Dynamics 365 assemblies and some tools (e.g. solutionpackager, package deployer and plug-in registration tools) are now all available from NUGET – check out Microsoft.CrmSdk.CoreAssemblies on Nuget

• The Dynamics 365 Entity Reference is also available  (and updated) online

For more details check out What’s new for Customer Engagement developer documentation in version 9.0.

Quick tip: using Google Analytics in Blogger

Blogger provides some out of the box statistics on number of pageviews, traffic sources, etc… but some functionality is unfortunately missing such as the ability to export these statistics,user by time of day, session durations, etc … Fortunately it is also possible to use Google Analytics as well by following these simple steps:

• Create a web property ID at https://analytics.google.com
• Open your Blogger Settings > Other > Google Analytics and enter the web property ID

Dynamics 365 monthly reading list October 2017

The Dynamics 365 Spring release (or July 2017 Update v9.0) reached the GA (Generally Available) milestone in the first weekend of October (online only) – so the first “official” documentation became available as well.

• What’s new for developers in the July 2017 Update
• Important changes coming in Dynamics 365 Customer Engagement
• Dynamics 365 v9.0 is already available by @ramontebar
• Dynamics 365 9.0 introduced native autonumber support but configuration was only available through the SDK – it is amazing to see that two tools became available in the first week to work with this Autonumber attributes in Microsoft Dynamics 365 from or Easy Autonumber utility for Dynamics 365 July 2017 Update (v9)
• Dynamics 365 Customer Engagement General Availability Readme (Known issues)
• What’s new in the July 2017 update for Enterprise Edition
• Interact with data from external systems using the new virtual entities by @WizOfD365
• Microsoft Dynamics 365 CE Platform Update for Developers (Ignite 2017 video)
• What’s new in Dynamics 365
• The long road to unified interface in Dynamics 365

Technical topics (Configuration, customization and extensibility)

• Dynamics 365, Organization Insights and dotmailer
• Identify differences between Dynamics 365 (CRM) instances
• Scoring surveys based on answers with Voice of the Customer
• Part 1 – Build a referral tracking system with Microsoft Flow, Azure Functions and your CRM
• Part 2 – Build a referral tracking system with Microsoft Flow, Azure Functions and your CRM
• Tips and tricks for deploying and managing Dynamics 365 for Sales and Service – Ignite 2017 – tips from the Dynamics PFE team
• Return from the land of no return – disable Notes, Activities, Connections, Feedback, Queues and/or change ownership model from @Crmtipoftheday
• Limiting users to specific Dynamics 365 organizations
• How to use Task flows to help your mobile workforce manage micro processes in Dynamics 365 from @magnetismxrm
• How to improve as Dynamis 365 developer
• CRM Tip #993: Update before your maps break
• Set default price list on product as read-only in Dynamics CRM
• A canary in CRM
• Tip 987: Kill a few million async jobs
• How to change icons of custom entities in Dynamics 365 – if you use Dynamics 365 9.0 – you also have scalable vector graphics (svg) icon support . Also check out this post on Dynamics 365: Updating Entity Icons
• How to use Javascript for editable grids in Dynamics 365
• Ribbon customization
• Best practices for Microsoft Dynamics security settings: append vs append to
• Adding custom icons with tooltips for a column in Dynamics 365

Topics for Dynamics 365 business analysts, project managers, power users and end users

• How to configure relationship analytics cards in Dynamics 365
• You are about to start a Dynamics project – would you go Agile or Waterfall?
Cover the bases of a successful CRM deployment
CRM observation – why alternate keys are not published sometimes
• Manage the status of your team’s bookings with Universal Resource Scheduling
• Modernizing business process with Cloud and AI

Technorati Tags: CRM,microsoft,msdyn365,dynamics+365,dynamics+crm,cloud

Quick tip: Format JSON documents in Notepad++

Notepad++ does not have built-in support for formatting JSON documents, but you can use JSONViewer for this. It can be installed directly using the "Plugin Manager" in Notepad++. Just got to "Plugins > Plugin Manager > Show Plugin Manager > Available tab"

Use Chrome user profiles to simulate different Dynamics 365 identities

Great tip that I saw on Twitter from @jukkan and @georgedude – you don’t need to use private browsing in Google Chrome to login to Dynamics 365 with a separate identity, you can use Chrome user profiles instead.

Click the person icon

 in the top right corner to configure your chrome profiles.

Check out Share Chrome with others or add a profile for more details.

Technorati Tags: msdyn365,Dynamics+365,Dynamics365,crmtip,crm

Easy way to get the Dynamics CRM Realm ID for server-based SharePoint integration

Server-based SharePoint integration in Dynamics CRM actually uses the same techniques as high-trust SharePoint app (For more background read Security in SharePoint Apps – Part 1) that you can develop yourself. SharePoint apps (or add-ins) use Appprincipals (a separate identity for an app) to act on your behalf when interacting with SharePoint. SharePoint knows that it can trust the request from the AppPrincipal when it is correctly registered. So one of the steps in Configure server-based authentication with Microsoft Dynamics 365 (on-premises) and SharePoint on-premises  requires you to register the AppPrincipal. To be able to use Register-SPAppPrincipal, you will need the CRMRealmId.

But there is an easier way to get to the crmrealmid than described in the walkthrough (if you forgot to note it down as suggested by the article) – you can simply go to the Settings>Customizations>Developer Resources and use the instance reference ID.

Rise of the machines : machine learning and artificial intelligence in Dynamics 365 Part 1

A couple of months ago, I wrote a blog post Microsoft Azure Machine Learning – the power to predict which provided  a high level view  of Microsoft’s service offering around machine learning and artificial intelligence in the cloud. In the next series of posts I will delve a little deeper into a number of scenarios where we can leverage Microsoft AI offering (machine learning and Cognitive services)  together with Dynamics 365/CRM.

If you look at most legacy CRM solutions, they are nothing more than digital contact lists and notebooks where you can look up contact information, check personal details or see when someone on your team has last contacted a customer. They contain a huge amount of data, but without built-in machine learning or predictive analytics, defining priorities on which prospects to contact first for a potential deal is like searching for a needle in a haystack. The added value of a modern CRM system is, that it should guide and support your sales team so that they can focus on what they are meant to do – sell. By automating tedious tasks such as sending follow up emails and reminders, you will free up more time to focus on selling. If you are not sure of the current state of your CRM solution, just take a look at what your mobile phone experience.

Microsoft is betting big on AI (See One year later, Microsoft AI and Research grows to 8K people in massive bet on artificial intelligence) and Dynamics 365 is a platform which can benefit greatly from these investments. Although, the first built-in functionality is still quite rudimentary and some of it is still in preview – the potential seems enormous (also see How machine learning will change Dynamics 365 from Ben Hosking )

Figure 2 Dynamics 365 Customer Insights, relationship analytics and relationship assistant.  

The table below lists functionality/features within the Dynamics 365/CRM platform which uses machine learning and artificial intelligence.

	Functionality	Availability	Remarks
Sentiment analysis within Microsoft Social Engagement	Understand the public perception using sentiment analysis	GA	Native sentiment detection is be available in the 17 Core languages (English, Danish, Dutch, Finnish, French, German, Greek, Italian, Norwegian, Polish, Portuguese (Portugal), Russian, Spanish, Swedish, Turkish, Japanese, Chinese traditional) and Arabic
Social selling assistant	The Social Selling Assistant recommends personal actions for you to  leverage social networks (e.g. which posts to share etc …)	GA
Use Azure text analytics with Dynamics 365 (Preview)	Suggest similar cases	US region only – Preview	The rules you create can use either Cognitive Services Text Analytics or some features let you use the built-in similarities matching engine in Microsoft Dynamics 365. Notice that you can only create one similar records suggestions rule for each entity type.
Use Azure text analytics with Dynamics 365 (Preview)	Automatically suggest knowledge base articles	US region only – Preview	Uses Azure Cognitive Services Text analytics services
Use Azure text analytics with Dynamics 365 (Preview)	Topic analysis – identifies topics/categories for cases to allow to better manage and resolve cases	Currently not available anymore	Azure Cognitive Services Topic Detection API was removed, so this functionality is not available anymore. Expected to be updated to use the new Topic API.
Use Azure text analytics with Dynamics 365 (Preview)	Using Document suggestions , Dynamics 365 can presents a list of suggested documents to the user while the user works on an opportunity/quote etc …	GA	The Document Suggestions feature doesn't require a connection to the Azure Text Analytics service. If you choose not to use Azure Text Analytics, Document Suggestions will use the built-in keyword matching logic available in Microsoft Dynamics 365
Product recommendations (Preview)	Dynamics 365 uses the Cognitive Services recommendation service to build an advanced recommendation model for automatic cross-sell product recommendations that are based on historical transaction data.	US Region only – Preview will end February 2018 and feature will be removed
Lead scoring and contact and account data augmentation through Versium Predict	Versium predict can use CRM data to generate predictive models to score leads, to augment data on contacts or accounts, and to acquire lists of new prospective consumers or businesses from Versium's database.		Microsoft has an OEM arrangement with Versium Predict which allows Dynamics 365 customers to install and use Versium Predict.
Relationship assistant	Provide guidance and assistance on activities that you should perform based on interactions with customers	US Region only – preview	Relationship assistant is part of the bigger relationship insights functionality which includes e-mail engagement and auto capture.
Dynamics 365 Customer Insights	Azure-based SAAS service which enables organizations to bring together transactional, web and social customer data to visualize it in a Customer 360°. DCI also allows you apply advanced data transformation algorithms (KPI calculation and machine learning) on this data to generate role-specific insights.

(*) There currently are a number of changes ongoing where some of the functionality which was powered by Azure Text Analytics and Product Recommendation APIs, will need to use the newer Azure Cognitive Services APIs.


In the next blog posts I will delve a little deeper into some of the available built-in functionality as well as look at how you can extend Dynamics 365 to integrate with Azure Cognitive Services, Azure Machine Learning, etc ...

Interesting links:

• Microsoft Data Amp 2017 – Keynotes (Youtube videos) – inspiration about how artificial intelligence and advanced analytics can be leveraged.
• Microsoft Data Amp 2017  - Channel 9 videos
• Introducing  Cortana Intelligence Solutions

Improve Dynamics 365 and submit your ideas

Microsoft recently setup a new channel to submit your ideas to improve Dynamics 365 – go check it out on https://ideas.dynamics.com/ . Some development teams at Microsoft, definitely take the feedback formulated through this new channel to heart.  During one of the Dynamics 365 Spring 2017  Preview executive briefing sessions one of the presenters told us  “If you look at the new “Social pane” section you will notice that out of the top 10 ideas, 6 are currently addressed in the new Activity timelines in Dynamics 365”.  But also one of the things that we bumped into when doing a project with Dynamics CRM Online Portals – the fact that Plugin exception information needs to be surfaced in Dynamics CRM Online Portals was recently picked up the Portal team (thanks

So go ahead and submit your own ideas for improvement and don’t forget to vote for these ideas:

• Include stage and rank in plug-in trace log by @rappen
• Store milliseconds in plug-in trace log by @rappen
• Extend the IPluginExecutionContext with business related user data by @PhilipVerlinden
• Extend the IPluginExecutionContext and workflows with Azure Redis Cache by @PhilipVerlinden

Technorati Tags: CRM,microsoft,dynamics+365,msdyn365

Dynamics 365 monthly reading September 2017

Technical topics (Configuration, customization and extensibility)

• Everyday I’m Shuffling – an alternative approach to DevOps for Microsoft Dynamics 365 (Channel 9 00:54:20)
• Microsoft flow – Create record in Dynamics 365 from incoming emails in Gmail
• Dynamics 365 instance backup to Azure storage
• Debugging quick create form parameters
• Approaching storage limit in Dynamics 365 – watch out VOC logs
• Getting ready for Dynamics 365 9.0 : learn Microsoft Flow
• Investigate Plugin Statistics in Microsoft Dynamics 365
• Form script wars: revenge of the events
• Dynamics 365 Attachment management and Dynamics 365 Online Portals – do not mix
• What you need to know about the Dynamics 365 Outlook web app
• Speed up your Dynamics 365 javascript development!
• Retrieve metadata using name in Web API
• Replicate Dynamics 365 Online Data to SQL database using Data Export Service
• Dynamics 365 July 2017 Update: Customer Engagement Portals
• Dynamics 365 Monthly Update – August 2017 (Dynamics CRM PFE team)

Topics for Dynamics 365 business analysts, power users and end users

• The formula for winning right customers: wooing strategy plus CRM
• Are you ready for Dynamics 365 Changes?
• Tip #906: Pets are dangerous – take a critical approach towards your favorite/pet feature of Dynamics CRM/365
• Business rules in Dynamics 365
• #tip 974: Attachments versus documents in Dynamics 365 – don’t forget the user experience
• A quick fix for browser rendering issues in Dynamics 365/CRM
• Delivering CRM projects with business impact
• Getting started with the Dynamics 365 for Customer Engagement Data Connector (Microsoft Access)

Topics about Dynamics 365 Spring release (aka 9.0 release)
The Dynamics 365 Spring release (version number 9.x) is the next upcoming release of Dynamics 365. For the moment we only got news about the updates for the next version of Dynamics CRM/365 online so no news yet for on-premise deployments.

• Dynamics 365 v9.0 summary (July 2017 Update)
• All you need to know about Dynamics 365 V9.0 (For now)
• Dynamics 365 Spring 2017 Release: 5 takeaways from a Business Solutions MVP
• Virtual entities in Dynamics 365
• Accessing Business Process Flow as Entities – D365 July  2017 Update
• Dynamics 365 July 2017 update: virtual entity
• What is Microsoft Dynamics 365 Business Edition?
• Dynamics 365 – Business edition vs. the world
• Comparing the new Dynamics 365 for Sales Business Edition
• New enhancements coming to Dynamics 365

Technorati Tags: CRM,microsoft,msdyn365

Dynamics 365, Azure webjobs, Azure Service Bus queues and deadletters

When you need to integrate Dynamics 365 (online) with other applications/services in on premise or in the cloud there are a number of challenges. One of the most commonly used integration patterns is messaging since it allows for loose coupling and it provides a reliable way of integration since both applications don’t necessarily have the same responsiveness or uptime. One of the key extensibility points from an integration perspective that you can leverage is the fact that you can connect Microsoft Dynamics 365 (both on premise and online) with the Azure Service  (See Azure integration with Dynamics 365). In an asynchronous messaging pattern, you will however still need a message processing component.  Since you are already leveraging cloud components, it might make sense to also use Azure for this message processing component.

There a number of Azure components you might take a look at:

• Azure cloud services: use an Azure worker role for message processing (for some background on this read Should I use cloud services or something else?)
• Azure app services: use  Azure web jobs for message processing.
• Azure virtual machine hosting the processing components
• Azure functions  - similar to Azure web jobs since it was built on the same code base as webjobs and has a similar API.
• Azure logic apps

If you look at the two first options (worker roles and web jobs), you will notice that they offer similar functionality payload – ability to run code repeatedly. However, there are some differences between the two - primarily in terms of environment customization. Web Jobs are good for running small tasks (not very computation heavy) and useful in scenarios where you do not have the need to depend on host OS features. Worker Roles can be used to run computation heavy workloads and their host OS can be modified a bit (by installing dependency components in OS through startup tasks). Worker roles are costly (similar to running a dedicated VM) while web jobs do not carry any additional cost. In case you need extreme customization on OS environment, then you would simply run a windows service inside custom OS image in Azure VM.  So basically you are looking at a trade off between control (and full flexibility) and agility combined with ease of management.

Now, if you look specifically at Azure webjobs, you will see that they can be triggered by a schedule, messages in Azure storage queue or blobs added to Azure storage and http call by calling the Kudu Webjobs API. But given the fact that Dynamics 365/RM already provides integration  with Azure Service Bus, this is the one that you will typically use – for a good walkthrough take a look at  How to use Azure Service Bus with the WebJobs SDK .

One of the things to keep in mind though, is that when an Azure webjob encounters an error in processing a message, it will retry processing it a specified number of times.  The number of retries is configured through the JobHostConfiguration.Queues.MaxDequeueCount property – see Azure webjobs and JobHostConfiguration for more details. This mechanism has been been built to avoid that a queue-based application gets stuck in a loop receiving and aborting the same message it can not process – this is also referred to as poison message handling.  So if this count is exceeded, the message will be move to the deadletter queue (when it is configured).

But the Azure service bus queues also have a property called MaxDeliveryCount (See QueueDescription class) which is by default set to 10 – so if this setting is lower than the setting in the JobHostConfiguration, your message might be moved to the deadletter queue (DLQ) earlier – see Overview of service bus dead-letter queues for more details.
If you open the deadletter queue with Azure Service Bus Explorer you will notice that these messages have additional properties called DeadLetterReason and DeadLetterDescription – these might be helpful in determining what causes the messages to end up in the DLQ. You can also move a message to the dead letter queue yourself by calling the BrokeredMessage.DeadLetter method.

Lessons learned:  If you are building an integration using Azure Service Bus queue, don't forget to think about how to handle messages which are deadlettered.


Technorati Tags: azure,dynamics365,service+Bus,messaging,application+integration,development

Dynamics 365 monthly reading August 2017

Technical topics (Configuration, customization and extensibility)

• Preventing folder names with GUIDs in SharePoint (Dynamics CRM/Dynamics 365 for Enterprise)
• Important changes coming in Dynamics 365 Customer Engagement
• Dynamics 365 Monthly update – July 2017 (CRM in the field – information from Dynamics PFE team)
• Dynamics 365 Monthly update – June 2017 (CRM in the field – information from Dynamics PFE team)
• Change createdby/modifiedby field at runtime using Dynamics CRM plugin
• Running Dynamics 365 Configuration Data mover jobs in Azure Functions 
• Deprecation of Dynamics 365 for Outlook
• New in V9: what’s that with Xrm.Page being deprecated?
• Analyzing Customer Insights data with Power BI
• XrmToolbox: the tools you need to get the job done (Youtube 00:52:46)
• Update Dynamics CRM Client SDK version to avoid calls to ACS
• Using Server to Server (S2S) authentication to connect your application to Dynamics 365
• Best practices when writing Dynamics CRM plugins
Using ILMerge for Dynamics CRM Plugins
• How to edit Dynamics 365 apps with the App Designer and Site Designer
• Async is smooth, smooth is fast – moving to async calls in Dynamics 365
• Where to store configuration data in Dynamics 365/CRM but also read Using Azure Key Vault to store configuration data which provides a more secure way of storing sensitive configuration data
• How to use Dynamics CRM/365 optionset labels within Power BI
• Auditing automatically disabled with unmanaged solution deployment in Dynamics 365
• Tips on submitting a Dynamics 365 App to Microsoft AppSource
• How to become Microsoft Dynamics 365 Consultant
• Refreshing web resources and iframes – the supported way

Topics for business analysts, power users and end users

• You cannot stop Dynamics 365 updating but you can be prepared
• Coming soon : Dynamics 365 for Sales, Business edition, designed for small-to-medium sized business
• 6 steps to creating a killer sales culture
• Dynamics 365 July Update 2017: Unified interface & customer service hub
• Dynamics 365 User Interface Refresh
• The evolving expectation on customer experience
• Dynamics 365 – Relationship Insights
• Relationship Insights (Youtube 00:05:35)
• Video guide to chat & co-browsing for Dynamics 365
• Comparing the new Dynamics 365 for Sales Business Edition with Enterprise Edition
• Activity timelines in Dynamics 365

Voice of the Customer for Dynamics 365

In March 2015, Microsoft announced the acquisition of Mojo Surveys – a customer feedback solution which was built solely for Microsoft Dynamics CRM (both online and on-premise).  This product was re-engineered and rebranded to Voice of the Customer (VoC). VoC was initially made available in preview in January 2016 – Preview feature: Voice of the Customer surveys and became generally available in the December 2016 release (VoC solution version 8.3.x and higher). At the moment VoC is only available for Dynamics CRM/365 (online) and apparently there are no plans to make it available for integration with on premise Dynamics CRM/365.

Voice of the Customer adds rule-based, multi-channel feedback surveys integrated with workflows, campaigns and e-mails on top of Dynamics 365. These surveys can be used in a range of scenarios from service feedback to focus groups to sales scenarios. Voice of the Customer allows you design and publish surveys from within Dynamics 365, distribute them to your target audience. Respondents can take part in a survey in different ways, depending on how you sent the invitation out, whether you want anonymous or non-anonymous responses and whether you want to integrate data from CRM into the survey. Responses can be incorporated into a business process by creating response routing rules. Voice of the Customer is available for installation as a preferred solution from the Applications tab in the Dynamics 365 Administration center.

From Voice of the Customer version 9.0 onward, you immediately receive a confirmation screen for accepting the license terms and for activating VoC when configuring it from within the Applications tab.

In previous versions, you first had to trigger the installation and afterwards go to the solution and accept the licensing terms from within the solution screen.

In the previous version, you could also Retrigger response processing to troubleshoot when you did not see responses coming in see Where are my Voice of the Customer responses? for more details. It is unclear where this setting has gone in version 9.x and higher.

Before you install the solution it is important to realize that behind the scenes Voice of the Customer uses a number of different Azure components such as Azure blog storage, Azure web and worker roles, Azure Keyvault, Azure SQL  see Install the Voice of the Customer solution for more details about this. These components are subject to other privacy and compliancy governance policies – for more information see Azure Trust Center

It is quite easy to get started with VoC and the steps are thoroughly outlined in Create a Voice of the Customer survey to get customer feedback . In previous version, I noticed that sometimes the survey definition got corrupted so it is important to quite regularly click on “Preview survey”. If you get an error like “Isv code aborted the operation”, it is best to delete the last question that you added, save the survey, run the preview and re-create the survey question.

Other good walkthroughs you can take a look at are A Step by Step Guide to Create Your First Survey with Dynamics CRM 2016 Voice of the Customer as well as Voice of the Customer for Dynamics CRM 2016 (configuration screens have changed a little bit from 9.x onwards but these articles are still worth taking a look at). One of the things, I also liked about the surveys is the fact that they are also mobile-friendly

Managed metadata column values not visible in SharePoint view

I recently encountered a problem with views in a SharePoint site collection – the values of the managed metadata column were not visible anymore for most users in the list views. However  when the user edited the item or viewed the details of an item – they could see the value. Apparently this issue was caused as a side effect of migrating Windows users to claim users. For performance reasons, managed metadata (or taxonomy items) are stored at site collection level in a hidden list called TaxonomyHiddenList which is accessible at the root level of a site collection e.g. http://servername/sites/sitecollection1/lists/taxonomyhiddenlist.  The list by default has read access to all Authenticated users but when performing the claims migration the security was mixed up, by giving all authenticated users read permission again, the issue was resolved.

Technorati Tags: SharePoint,troubleshooting,taxonomy,managed+metadata

Free CRMUG Belux webinar – What’s new in Dynamics 365 Spring 2017 release

By now, you have probably all read about the upcoming Dynamics 365 spring 2017 release. If you want to learn more about it register for the free CRMUG Belux webinar – What’s new in Dynamics 365 Spring 2017 release . This June, Microsoft announced the next Dynamics 365 release, called the “Spring 2017” release wave. During a 3-day (!) long preview session, Microsoft’s product team took a deep dive into the new capabilities that are coming to the platform, such as:

• An updated user interface, providing a uniform experience for users across web, app and mobile,
• New features in Dynamics 365 Customer Insights that leverage data across and beyond Dynamics CRM and machine learning capabilities (Also check out Introduction to Dynamics 365 Customer Insights to get started)
• New social selling capabilities, a direct result of the LinkedIn acquisition
• New features around scalability and extensibility, -
• The release of the Business Edition, a sales and marketing app triggered towards small and medium businesses.

Your CRM-UG BeLux team has managed to bundle all these exciting announcements in a 1-hour lunch webinar for you on August 28th. Our presenters, Microsoft’s Philip Verlinden and Dieter De Cock, have even promised there will be time for Q&A.

Introduction to Dynamics 365 for Customer Insights (DCI)

Dynamics 365 for Customer Insights (DCI) was introduced in December 2016 and has recently received some interesting updates in the Dynamics 365 Spring update.  DCI is an Azure-based SAAS service which enables organizations to bring together transactional, web and social customer data to visualize it in a Customer 360°. DCI also allows you apply advanced data transformation algorithms (KPI calculation and machine learning) on this data to generate role-specific insights for different roles within an organization. 

The key differentiators within Dynamics 365 for Customer Insights are:

• It provides a scalable data infrastructure which can handle millions of interactions, entities and relationships between entities. Building such an infrastructure can be a daunting task and will consume a lot of upfront investment in  IT capacity. By providing a platform where you only pay for the data that you need to handle, you will be able to shift investment to resources and more value added tasks such as integrating the intelligence within your existing business processes
• Out of the box connector for Dynamics 365 for Customer Engagement (CRM) which allows you to rapidly get going starting from the existing effort that you put into building your CRM platform
• Ability to match customer identities in different systems using conflation algorithms and predictive matching – this allows to merge/cluster customer identities at scale.
• Built-in advanced machine learning applications such as segmentation – which identifies cohorts of users with the same characteristics or at the same stage of the customer journey – and built-in predictive scoring. Business users will be able to define the business outcome they want to predict (e.g. customer churn = customer for which status becomes inactive) and DCI will automatically create a machine learning model and validate it (behind the scene it will do feature selection, machine learning selection, training and validation,…). Prediction scores will be automatically generated for new entities or they will be recalculated for entities for which one of the signals/characteristics impacting the predictive score will change. This allows you to get up and running quickly with a machine learning solution without hiring data scientists.
• Extensible platform – ability to ingest data from multiple data source by building your own connectors and ability to expose DCI entities,metrics and predictions through Power BI, Power Apps or using the REST API

Customer Insights is leveraging the same components which are also used within the Cortana Intelligence Solutions but they are taking it a number of steps further by providing out of the box connectors, configurations screens, ingest and egress paths as well as built-in visualize layer. You however have less control on the inner workings of the Customer Insights solution since this is a SAAS solutions whereas most Cortana Intelligence Solutions combine a SAAS, IAAS and PAAS platform components.

One of the customer cases that you should definitely take a look at is Marston’s and Customer Insights - Marston, is a brewing company operating more than 1700 pubs in the UK and was one of the early customers that started using this application.  In order to create a personalized pub experience for their customers and achieve a more targeted marketing they wanted to bring together customer data from social media such as Facebook, Twitter, and TripAdvisor. They also wanted to look at Wi-Fi registration data, point of sale transactions, table bookings and survey results to understand their customers.  This required a level of data integration and analysis that goes beyond just CRM alone and that’s were Customer Insights comes in. Shown below is an example of dashboard within DCI, which is tailored towards a house manager of a Marston’s inn.

Word of advice: Customer Insights for Dynamics 365 is currently in preview – available functionality is subject to change and both pricing and product naming might change before the final release. From a licensing perspective, we know see two tiers as described on the Dynamics 365 Customer Insights pricing page. Pricing is based on the amount of data which needs to be processed as well as KPIs used, predictive match policies and predictive score models.

References:

• Dynamics 365 for Customer Insights – Getting Started
• Using connectors to simplify data ingestion at scale in Dynamics 365 for Customer Insights
• Predicting business outcomes with Customer Insights (Blog post)
• Predicting business outcomes with Customer Insights (Youtube 00:01:53)
• Dynamics 365 for Customer Insights: July 2017 refresh

Introducing Cortana Intelligence Solutions

It has been a couple of months since I last looked at Cortana Intelligence Solutions (CIS) in the Cortana solution gallery so I was pleasantly surprised by a number of new solution templates which have been added.  These solution templates target different problem domains in different industries and are leveraging the full breadth of the Microsoft data platform offering – some of the scenarios covered are:

• Predictive Maintenance for Aerospace - this Predictive Maintenance solution monitors aircraft and predicts the remaining useful life of aircraft engine components. Uses Azure HDInsight, Azure Machine Learning, Azure Event Hubs, etc … the different components are outlined in a solution diagram which is provided by the solution template as well.
• Campaign optimization with SQL Server 2016 - demonstrates how to build and deploy a machine learning model with SQL Server 2016 with R Services to recommend actions to maximize the purchase rate of leads targeted by a campaign.
• Vehicle telemetry analytics - this solution demonstrates how car dealerships, automobile manufacturers and insurance companies can use the capabilities of Cortana Intelligence to gain real-time and predictive insights on vehicle health and driving habits.

Cortana Intelligence Solutions is just one of the pieces in the Cortana Intelligence Suite next to Power BI, Cognitive Services, Microsoft Bot Framework and a whole lot of Azure cloud components such as Azure Machine Learning, Azure HDInsight, Azure SQL Data Warehouse, Azure Data Factory and many more.

Cortana Intelligence Suite provides both a platform and  process guidance (Team Data Science Process) to perform advanced analytics from start to finish. The main goal of Cortana Intelligence Solutions  is to reduce the complexity of deploying advanced data analytics solutions by providing Azure building blocks to operationalize your data science process. By providing these building blocks, Microsoft tries to enable companies to make smarter decisions at a rapid pace without having to worry about the complexity of designing, deploying and operating scalable data architectures.
I truly believe in the potential of data to be able to truly transform a business, a point which has been backed by the Capturing the $1.6 Trillion Data Dividend  white paper which showed that data leaders have a competitive advantages over their peers. These data leaders have a number of characteristics in common in the way that they invest in data:

• Leverage new data such as linking transactional data with customer behavior, sensor data, social data, mobile data eetc…
• Expanded use of analytic techniques – including more predictive analytics and big data processing techniques such as Mapreduce
• Use new metrics – e.g. new ways of looking at operations or measuring performance
• Include new users – expand the number and type of users who have access to the organization’s data and analytics

The interesting part of the Cortana Intelligence Suite is that it also describes the Team Data Science Process lifecycle which provides a recommended lifecycle which can be used to structure data science projects (see Data science for the rest of us for a good starting point of your new to the field of data science).

A key point that I can’t stress enough is the fact that you need a solution understanding of the business context. Data science projects are typically not IT-driven, business should be in the lead. Data science is a team effort where close cooperation with business is required to understand and identify the business problems. The most complex part of the data science process is formulating the  questions that define the business goals and that data science techniques can target. Most companies agree that managing customer churn is important, but how you define churn is different in each industry and also the impact of churn of specific customers might be different e.g in telco customers are influenced by both friends within the network and friends of friends (See Using social network analysis to predict churn which used data from a Belgian telco provider as well as the research paper Mining telecommunication networks to enhance customer lifetime predictions)

References:

• Cortana Intelligence Solutions (James Serra)
• Cortana Intelligence Suite Overview (Channel 9 video)
• Cortana Intelligence Suite Retail Industry Solution – How to guides