Dynamics 365 weekly reading list 11/23/2016

The big storm of announcements about Dynamics 365 has passed but still some interesting tidbits of information and new tools were released in the previous weeks:

• Dynamics 365 (CRM) scheduled workflows using Microsoft Flow
• Choose between Flow, Logic Apps, Functions and WebJobs in Azure
• Reflecting on Dynamics 365
• AngularJS demo solution for Dynamics CRM
• Getting started with Dynamics 365 PowerApps
• White Paper: Microsoft Dynamics CRM 2016 Service Pack 1 Performance Benchmark on Azure Infrastructure as a Service (Iaas)
• Dynamics 365 Site Map Designer
• Monitor your Dynamics 365 (online) instance with Organization Insights (Youtube – 00:02:20)
• Activity Management Enhancements in Dynamics 365 (v 8.2)
• PowerApps and Microsoft Flow: Is Microsoft putting traditional software developers out of business
• No, Common Data Service is not the new XRM
• Common Data Model (CDM) Entity Reference document
• Bot framework – solving business problems with the Microsoft Bot Framework – describing an interesting scenario using Microsoft Flow, Dynamics 365, Office 365 and other Azure components.
• Why my action is not available through the Web API
• 3 benefits of Knowledge Articles in Dynamics 365
• Microsoft Dynamics 365 Developer Toolkit (public beta 1)
• Calling Microsoft Flow from Power Query and Power BI
• Project Service Automation in Dynamics 365 – what are your options?
• Moving Dynamics CRM Business Process Flows with multiple branches
• What does Adobe Marketing mean for Dynamics
• Role of PowerApps, Flow and Common Data Model in Dynamics 365
• Creating a PowerApps Mobile Application with Dynamics CRM in 1 hour

Previous edition – Dynamics 365 weekly reading list 11/05/2016

Ignite 2016 slidedeck and video downloader

 

Microsoft Ignite is one of the biggest Microsoft conferences in the year with over 700 different sessions over the course of 5 days so it is quite impossible to attend all sessions. Luckily Microsoft provides a recording of most of the sessions on Channel 9 | Microsoft Ignite 2016 and for those of you who would like to get an offline copy of all the different sessions as well as the PowerPoint decks – take a look at  the Powershell script – Ignite 2016 slidedeck and video downloader  - if you want to download only the powerpoint decks you can use the command  -  .\Ignite2016Downloader.ps1 -NoVideos -DownloadFolder d:\Ignite2016 ,  this is already 12,5 GB.

Here’s my watchlist for the next couple of weeks:

• Business application platform roadmap, strategy and packaging(BRK2271)
• Evolve with Microsoft Dynamics CRM platform and online (BRK2161)
• Review CRM Implementations best practices (BRK3243)
• Integrate with Microsoft Exchange Future Strategies (BRK3308)
• Achieve productivity excellence with Microsoft Dynamics CRM and Office 365 (BRK3265)
• Optimize connectivity to Microsoft Dynamics CRM Online(BRK3310)
• Dive Into the Microsoft Common Data Model (BRK3315)
• Enable agile enterprise connectivity anywhere using Microsoft integration (BRK2308-TS)
• Dive into Microsoft Flow, create automated workflows between your favorite apps (BRK3311)
• Dive into PowerApps, building apps that mean business without writing code (BRK3326)
• Build cloud-first enterprise integration solutions with logic apps (BRK3342)
• Gather intelligence in Dynamics CRM with Power BI and Cortana Intelligence (BRK2163)
• Essentials of Application Lifecycle Manager for CRM (BRK2168)

Technorati Tags: Microsoft,Ignite,Flow,CDM,Msdyncrm,msdyn365

Dynamics 365 weekly reading list

 

Welcome to the first edition of the Dynamics 365 weekly reading list – with the official “release” of Dynamics 365, Microsoft took an important step in providing an integrated cloud platform which combines CRM and ERP. Follow this blog for your weekly reading list about the Dynamics 365 platform.

• Microsoft Dynamics 365 – Now generally available
• Dynamics 365 – What’s new – information about the upcoming December 2016 release
• Microsoft Dynamics 365 editions and licensing
• Microsoft Dynamics 365 home
• Dynamics 365 app for Outlook is coming soon!
• Helping customers accelerate their digital transformation with intelligent business applications
• Dynamics 365 – Purpose-built for you and your business (Youtube)
• Common Data Service is now generally available! – CDS encompasses the CDM (Common Data Model) which provides a mechanism to unlock data which is used in the different Dynamics 365 apps and which will allow for easier integration – see Announcing the general availability of the PowerApps Common Data Service. Although GA there are still some things which need to be clarified as outlined in CDM: New data model for the common good?
• Project Service Automation for Dynamics 365 (Youtube)
• Tap the power of Dynamics 365 directly from Microsoft Outlook (Youtube)
• Sell effectively with Microsoft Dynamics 365 (Youtube 00:02:17)
• Microsoft Dynamics 365 First Look Highlights (Youtube 00:02:03) and all other recordings – Microsoft Dynamics 365 First Look
• Highlights and reactions: Microsoft Dynamics 365
• Dynamics 365 -  Acquisitions that made the cut, or didn’t
• Dynamics 365 – getting into the flow of things
• Microsoft Dynamics 365 : the what, when and why

Technorati Tags: Dynamics365,DynamicsCRM,Microsoft

Fixing Visual Studio projects created with CRM Developer Toolkit

If you try to open a Visual Studio 2012 CRM plug-in or workflow activity project which was created with the CRM Developer Toolkit in Visual Studio 2013 or Visual Studio 2013 you will notice that it does not open and that it states – “This project is incompatible with the current edition of Visual Studio”. Unfortunately the CRM Developer Toolkit has not been updated to work with the newer versions of Visual Studio. There is however an easy fix for this -  open the csproj file of the Visual Studio project in Notepad++ – look for the ProjectTypeGuids line and simply delete it. This fixed it for me with Visual Studio 2015 – if you encounter other errors – take a look at How to remove dependencies to CRM Developer Toolkit

Tags van Technorati: msdyncrm,Dynamics+CRM,Visual+studio,Microsoft,CRM,development,c#,csharp

Creating a new Documents view in CRM to display SharePoint documents

Integration between SharePoint and Dynamics CRM allows you to link documents to Dynamics CRM entities but the documents will be stored in SharePoint instead of the CRM database (See Using SharePoint Online to store Dynamics CRM Online Documents for more information). This allows you to take advantages of the strenghts of both platforms but one of the things missing is how you can surface metadata/document properties from linked documents in Dynamics CRM.

In  Showing SharePoint document properties in Dynamics CRM views I  already explained how you can adapt the standard Associated Documents view to show extra document properties from documents stored in SharePoint document libraries. In this post, I will explain how you can create new views for the SharePointDocument entity.

If you just create a new view and try to make it work out of the box you will see that this not works. You will need to export your CRM solution. You will notice that the exported zip file contains three XML files.

• solution.xml : Contains the details of the solution package including publisher information and a listing of any missing or key dependencies
• [Content_Types].xml : standard Microsoft export file that identifies the content of the solution. The [Content_Types].xml file type is part of the Open Packaging Conventions (OPC) standard. For more information, see OPC: A New Standard For Packaging Your Data on the MSDN Web site.
• customizations.xml: this file contains all of the content and mark-up that you will need to change.

Now if you look at the SavedQuery XML from the custom view which I configured in the view editor (left side of the image below), you will notice that it is quite different from the Associated Documents view that I customized previously (See Showing SharePoint document properties in Dynamics CRM views )

For a description of the different XML nodes and attributes used you can take a look at the Saved Query XML Reference but the most notable differences in the Associated Documents grid are the following – and these are also the changes that you need to make in the XML for your custom view.

• SharePointDocument.RetrieveDocument – the official documentation only states that a string value is expected but not really what value is needed. For integration with SharePoint you need to specify SharePointDocument.RetrieveDocument
• … – defines a set of columns to return in a saved query, it seems that it is necessary to also take over a number of SharePoint specific columns.  (Reference: Saved query XML reference columnsetxml   )
• … (Reference: Saved query XML reference layoutxml )
• 0  -  0 is for a normal main application view,  2 is for an associated view (Reference: Saved query XML reference querytype )
• … – here you need to specify the additional SharePoint document metadata columns that you want to show in the grid

After you have made the changes to view as outlined above you will see that your custom view will also work correctly. I have added the full savedquery also in Sample SavedQuery download link. In a next post I will outline how you make it possible to modify SharePoint document properties from within the CRM interface.

Related posts:

• Using SharePoint Online to store Dynamics CRM Online Documents
• Showing SharePoint document properties in  Dynamics CRM views

Technorati Tags: SharePoint,Dynamics+CRM,CRM,Office365,Dynamics365

So you want to become a data scientist?

It has been over 4 years that the article Data Scientist: the sexiest job in the 21st century appeared and the interest in a data scientist career has probably increased if you take a look at Google search trends – see https://www.google.com/trends/explore?q=data%20science (although I’m quite puzzled by the high interest in Nigeria).

For those of you out there who have never heard of data science, check out Data science for the rest of us. It has also become easier to get started given the abundance of resources available on the web and MOOC platforms such as Coursera and Edx offer quite a few data science and machine learning courses. Listed below are a number of links to get you started. However keep in mind that if you want to make a profession of this, you will need to put a lot of effort in it – check out this excellent post Stop saying learning to code is easy and just replace “Programming” with “Data Science” and you will get the picture.

• How to become a data scientist (Quora)?
• 16 analytic disciplines compared to data science
• Are you ready for data science?
• Data science as a profession – time is now
• Data science – an introduction (Wikibook)
• Pandas ebook – Pandas is a python package which provides the fundamental high-level building blocks for doing practical, real world data analysis in Python.
• R for Machine Learning (MIT lecture notes)
• Data Science Cheat Sheet
• Brussels Data Science Community (Parleys video channel)
• This is why your data scientist sucks
• What makes a data scientist – finding unicorns in the midwest
• Data science and machine learning essentials (Edx MOOC)

Technorati Tags: datascience,machine+learning,data+scientist,data+science,R,python

Showing SharePoint document properties in Dynamics CRM views

As outlined in a previous blog post, Dynamics CRM provides out of the box integration with SharePoint – meaning that you can upload your documents into SharePoint directly from within the CRM user interface and link them to specific CRM records (See Using SharePoint Online to store Dynamics CRM Online documents for more details). This same integration scenario is also available for Dynamics CRM on-premise and SharePoint Server on-premise deployment.  From Dynamics CRM 2016 onwards it is also possible to use hybrid integration scenarios where as outlined in Setup SharePoint Integration with Dynamics CRM

The preferred method for integration is using Server-to-server integration (See Integrate Microsoft Dynamics CRM CRM with SharePoint) but there are some noticeable differences between the Dynamics CRM List component and server-based SharePoint integration as outlined in Important considerations for server-based SharePoint integration as well as SharePoint Integration Reloaded Part 1 from Scott Durow. From Dynamics CRM

 One of the great features which is enabled using server-based SharePoint integration is the fact that you are also able to show fields (or site columns) which are added in SharePoint within the Documents view in Dynamics CRM. Site columns or library columns (also referred to as fields or attributes) are used to organize documents so that people can find back the documents they need more easily. Document libraries in SharePoint are like real libraries where you can borrow books, a library has a system in place to search for the books by publication year, author, language, genre, age category, etc… In a SharePoint document library you would use site columns or library columns which would contain this information. This is also called metadata – “data (information) that provides information about other data”.  So if you enable document management on the Account entity in CRM using server-based-integration it would look like the screenshot below.  

 Dynamics CRM uses a number of entities to enable the integration with SharePoint :

• SharePoint site (schema: SharePointSite): represents a location record in Microsoft Dynamics CRM that points to a site collection or a site on a server that is running SharePoint Server (See SharePointSite entity messages and methods )
• SharePointDocumentLocation : represents a location record in Microsoft Dynamics CRM that points to a document folder on a server that is running SharePoint Server. You can associate the SharePoint document location records in Microsoft Dynamics CRM with the entity records for the entities for which the SharePoint Server integration or document management feature is enabled. (See SharePointDocumentLocation entity messages and methods)
• UserMapping : represents a custom claim mapping record in Microsoft Dynamics CRM to use a value other than the default value used by CRM Online for authenticating and authorizing CRM users in SharePoint (See UserMapping entity messages and methods)

Next to these three entities which are documented by Microsoft there is a 4th entity SharePointDocument which is actually quite an important entity since it is in the SharePointDocument entity that you can define additional SharePoint columns that you want to be displayed in the grid. Internally CRM will translate the FETCHXML query to a SharePoint query (CAML – Collaborative Application Markup Language ) – if you are interested to know how this is done you can use JustDecompile and take a look at the code in the Microsoft.CRM.ObjectModel.dll which apparently has the implementation code for the SharePoint integration.

If you look at the SharePoint Document entity in CRM and check out the list of fields you’ll see there are quite a few fields (such as Title, version, etc..) that you can display. These fields will display the corresponding values from columns in a SharePoint document library if you add them to the SharePoint Document Associated View. So for example you can quite easily make the title of a document in SharePoint appear in the CRM view.

This is great for the out of the box fields but if you have modified the library in SharePoint to add additional attributes you’ll notice they are not displayed. So let’s image that you have added a custom field called DocType (of type text) in a SharePoint document library as shown below and you want to show the values of this SharePoint column in Dynamics CRM (I suppose that you already setup with document management integration already setup). You will need to perform a number of steps to accomplish this: 

• Create a new solution in Dynamics CRM (you can also use the Customize the system option but this is not really the recommended way)
• Add the SharePointDocument entity and the Document Associated Grid to your CRM solution

• Navigate to the SharePoint document library that is used to store the documents linked to your CRM record and add a column of type text with name “DocType”.
• Add a field on the SharePointDocument entity with exactly the same display name “DocType” of type single line of text. Dynamics CRM will use the display name of the fields to do the mapping with the query which is send to SharePoint. All values returned from SharePoint will be of type text except for a date field in SharePoint. So you need to create fields of the corresponding type in CRM.
• Modify the “Document Associated Grid” view that you added to your CRM solution and add the newly created CRM field to this view. (This will only work with the “Document Associated Grid” view – to make this work with new views you will need some extra steps that I will outline in a next blog post)

Once you fill in the values of the SharePoint fields for the different documents you will notice that they appear in the CRM “Document Associated Grid” as well. This seems to work for the following types of SharePoint fields: text, number,date,choice, boolean,currency and person. I currently only tested the scenario in Dynamics CRM Online (8.1.0.452) in combination with SharePoint Online as well as for Dynamics CRM 2016 (Update 0.1) on premise in combination with SharePoint Server 2013 but I guess it should also work in other setups which are using server-based-integration.

References:

• SharePoint integration reloaded – Part 1
• SharePoint Integration reloaded – Part 2 – How authentication works and how CRM retrieves the SharePoint documents
• SharePoint integration reloaded – Part 3 – Code sample for a CRM workflow which allows to store the documents in a different location in SharePoint

Technorati Tags: SharePoint,Dynamics+CRM,CRM,Office365,Dynamics365

SQL Server Management Studio available as separate download

I just finished my first installation of SQL Server 2016 and I noticed that SQL Server Management Studio was available as a completely separate install option and not a part of the main installation. Microsoft decided to make SQL Server Management Studio (SSMS) available as a separate download . This also means that SSMS does not require a license anymore. The current release (13.0.15700.28) works with all supported versions of SQL Server (2008 to 2016) as well as SQL Azure. The rationale behind is that Microsoft will be able to update the tooling at a swifter pace than the releases of SQL Server itself.

Technorati Tags: SQL+Server,SSMS,Microsoft

Quick tip - using Azure Powershell with multiple subscriptions

If you have multiple Azure subscriptions linked to the account that you use to login – the operations in Azure PowerShell are executed against the “current subscription”. You can use the Get-AzureSubscription PowerShell cmdlet to see all of the subscriptions and also which is the “current” subscription. During a PowerShell session, you will need to use the Select-AzureSubscription cmdlet to choose which subscription data set is used by other Azure cmdlets with the following syntax "Select-AzureSubscription -name "NameOfYourSubscription".

Technorati Tags: Azure,Microsoft

Software boundaries and limits for Dynamics CRM 2015/2016/Online Part 1

One of the TechNet pages that I consulted quite often as SharePoint architect was the Software boundaries and limits for SharePoint 2013, so I expected the same page to exist for Dynamics CRM 2015/2016 and Dynamics CRM Online. Unfortunately, this is not the case. There is however some valuable insights hidden in a number of interesting white papers and web pages so it might be logical to combine them.

But let’s first take a look at why such a resource might be needed more than ever. I think that massive Dynamics CRM deployments – in number of users -  are still quite rare especially if you compare it with SharePoint.  This is however quite normal given that both products are covering completely different functional domains and where SharePoint might be used by everyone in a company, Dynamics CRM users are typically still found in customer facing departments (service management and sales & marketing) . This might be one of the reasons why there hasn’t been any need to compile such an extensive list of boundaries and limits for Dynamics CRM. But in an era where winning the customers heart and mind is getting harder and harder – it becomes imperative that companies transform from ‘product-focused’ to ‘customer-focused’ which will put CRM more at the center of your enterprise architecture and which might also trigger larger deployments of CRM both in terms of number of users, transactions and records.

But in the enterprise market we are currently already encountering at what you might describe as “High Volume” systems. A “High Volume” system might have one or more of the following characteristics:

• High volume data:  more than 1 mio records in the base tables (Contacts,Accounts, …) or more than 5000 activities/day or more than 1 mio/year
• High volume transactions
• High volume users/security:  more than 300 concurrent users or more than 1000 teams and business units

Typically these “High Volume” CRM systems take more effort and thought to setup and to keep them up and running smoothly and  unfortunately there is no high-volume switch available in Dynamics CRM to make this work auto-magically.

I also got feedback from another CRM architect stating that there are no real hard limits in Dynamics CRM for on-premise deployments but only scalability guidelines that you should follow. But still when you are doing a large CRM deployment chances are that you might call in Microsoft consultancy to assist you or audit your design they will point you to some specific boundaries and limits that you should adhere to. These guidelines and boundaries and are defined in the Microsoft Product Line Architecture for Dynamics CRM which unfortunately is not publicly available (See Introducing the Microsoft Product Line Architecture if you are unfamiliar with PLA) and although these are indeed no hard limits –if you don’t comply you will be asked to provide mitigating or corrective actions.

To get started you should definitely take a look at the Microsoft Dynamics CRM 2015 and Microsoft Dynamics CRM 2016 Performance and scalability documentation. This download contains the Dynamics CRM Scalable Customizations white paper which describes how SQL Server platform-level issues can occur that create slow Microsoft Dynamics CRM application performance or error messages returned to the end-user. It also provides information about how to optimize a custom implementation and enable better performance that results in better end-user experience. It also contains the Scalable security modeling with Microsoft Dynamics CRM 2015 white paper which describes how security modeling features in Microsoft Dynamics CRM 2015 and Microsoft Dynamics CRM 2016 related to authorization work at scale, the associated implications, and guidance on common and recommended usage patterns for modeling Microsoft Dynamics CRM 2015 and Microsoft Dynamics CRM 2016 security at scale, incorporating teams as appropriate.  Both white papers are updated versions of the papers which were initially written for CRM 2011 and CRM 2013 because in essence the core architecture as described in Microsoft Dynamics CRM Reference Architecture has not changed significantly in the later versions.

Given the focus of Microsoft on Dynamics CRM Online you might also be faced with a large deployment of Dynamics CRM Online and you have to keep in mind that Dynamics CRM Online does contain some hard limits. I will cover these in a next blog post.  But the Microsoft Dynamics CRM Online patterns & principles for solution builders white paper download provides already some solid guidance specifically about building solutions using Microsoft Dynamics CRM Online. Also keep in mind that Dynamics CRM Online deeply integrates into Office 365 (Yammer, SharePoint Online, Exchange Online and Office 365 Groups) and you might also need to the integrate some Microsoft Azure components so you should also have a solid grasp of these components as well as a solution architect.

Last but not least - it is a sad truth that in most deployments bad performance is not due to the number of users, transactions or records stored in Dynamics CRM but mainly caused by bad code – so also take a look at Best practices for developing with Microsoft Dynamics CRM.

References:

• Software requirements for Microsoft Dynamics CRM 2015
• Microsoft Dynamics CRM Server 2015 hardware requirements
• Introducing the Microsoft Product Line Architecture
• Microsoft Dynamics CRM Reference Architecture
• Microsoft Dynamics CRM 2015 and Microsoft Dynamics CRM 2016 Performance and scalability documentation

Technorati Tags: Dynamics+crm,CRM+online,CRM,architecture,solution+architecture,Office365,Microsoft

Using filled maps in Microsoft Power BI for provinces, regions and counties in European countries

A couple of weeks ago I wrote a blog post Problem with filled maps (choropleths) in Power BI for Belgian provinces in which I stated that there are some problems with using filled maps for international locations. Thanks to the excellent support of the Power BI team I managed to resolve the issue for the Belgian provinces.

If you ever want to use filled maps with Power BI it is important to use the correct data category and this can be quite confusing for Europe where we have a multitude of different ways of dividing up a country which do not always correspond to the way it is done in the U.S. The data categories for locations that you can use on a filled map are Continent,Country/Region, State or Province and County .

So when I switched the data category for my Belgian provinces to county – the filled map already looked at lot better but not completely since it shows the Grand Duchy of Luxembourg (https://en.wikipedia.org/wiki/Luxembourg)  and not the province Luxembourg (https://en.wikipedia.org/wiki/Luxembourg_(Belgium) ) on the map.

So I tried it out for a number of neighboring countries - the next table gives an example of what the different data category labels (in bold in the first row) correspond to for a number of European countries.

I tested it with Belgium, Germany, the Netherlands and France. I did not get it the regions to work correctly for  France but this might be because France has officially merged a number of regions since January 1st. France is also subdivided into a number of departments  and these were shown on the map as expected.

Power BI Data Category to use	State or Province	County	Remarks
Belgium	Flanders, Walloon region and Brussels Capital region	Antwerp, East-Flanders, Flemish Brabant, Limburg, West-Flanders, Hainaut, Liège, Luxembourg, Namur, Brabant-Walloon	Province Luxembourg not depicted correctly
Germany	The different bundesländer: Baden-Württemberg, Bavaria, Berlin, Brandenburg, Bremen, Hamburg, Hesse, Lower Saxony, Mecklenburg-Vorpommern,North Rhine-Westphalia Rhineland-Palatinate, Saarland, Saxony, Saxony-Anhalt, Schleswig-Holstein,Thuringia
France		The different departments as outlined on https://en.wikipedia.org/wiki/List_of_French_departments_by_population .	I removed the overseas departments to make it workable
Netherlands	The different provinces: Drenthe, Flevoland, Fryslân, Gelderland, Groningen, Limburg, North Brabant, North Holland, Overijssel, South Holland, Utrecht, Zeeland

Different German states with the GDP per capita.

French departments with population density.

Dutch provinces with population density

Technorati Tags: Power+BI,Microsoft,visualization,geocoding

Data science for the rest of us

A couple of weeks ago I followed an interesting webinar from Microsoft called Data Science for the rest of us. I have been interested in data science ever since I read the excellent book Doing Data Science: straight talk from the frontline from Cathy O’Neill and Rachel Schutt and articles like the Data Scientist: the sexiest job of the 21st century sparked this interest even more.

In this webinar Brendan Rohrer (@_brohrer_)  explains with a number of great examples some key ingredients or trade secrets of doing data science in easy to understand terms – here’s a quick recap (although I really recommend you to watch the video):

• Trade secret 1: Data is not the starting point (and you have to ask sharp questions): I really like the definition as formulated by  Jeff Leek (@jtleek) (taken from Data science done well looks easy, which is a big problem) Data science is the process of formulating a quantitative question that can be answered with data, collecting and cleaning the data, analyzing the data, and communicating the answer to the question to a relevant audience. So you first need a precise question and then you need to look for the right data or as indicated in the webinar relevant, connected, accurate and enough data. I’m not a data scientist but this really seems like the hardest part (or as phrased here For Big Data scientist, ‘janitor work’ is the key hurdle to insights )
• Trade secret 2: Turn your data in a picture – check out the example used in the seminar below. It is important to understand that people effortlessly recognize and classify objects among tens of thousands of  possibilities so visualization of your data can help you to make sense of the data (For an interesting scientific article on this topic – take a look at How does the brain solve visual object recognition? )

• Trade secret 3: Data science can only answer five questions: predict how much/how many [regression], which category does something belong to [classification], which groups exist in a dataset [clustering], is something weird [anomaly detection] and which action should you take[reinforcement learning].
• Trade secret 4: Machine learning is simple. This statement is a little aggerated – but the analogy of mastering a foreign language and mastering machine learning is indeed correct. You need to learn the lingo (everyone probably knows tables – either in Excel or a database, but data scientist will refer to these  rows of data in a table as data point or samples by data scients. The columns in your table typically describe a specific characteristic – well  data scientist will call this a feature.)
• Trade secret 5: there are a lot of right ways to solve a specific problem. If you look at the Machine learning algorithm cheat sheet for Microsoft Azure Machine Learning Studio you will notice that there a lot of different ways to solve a specific problem (with certain nuances such as the number of features available, or speed of calculating the model, …) but in most cases it apparently does not seem to matter that much.

To get an overview of other Microsoft webinars on similar topics check out Big Data and Advance Analytics: On-demand and upcoming live webinars
References links:

• What can data science do for me?
• Data exploration through visualization
• What types of questions can data science answer?
• Which algorithm family can answer my questions?
• Machine learning algorithm cheat sheet for Microsoft Azure Machine Learning Studio
• How to choose algorithms for Microsoft Azure Machine Learning

Technorati Tags: datascience,data+science,azure,analytics,azureml,microsoft,machine+learning

Resolving the Dynamics CRM minFreeThreads error

A while ago I booted up a new Hyper-V virtual machine with Dynamics CRM installed and I received the following error “The value for ‘minFreeThreads’ must be less than the thread pool limit of 400 threads.” when opening the Dynamics CRM web page.

Apparently the  Microsoft .NET Threadpools settings of the machine.config where changed based on the guidelines defined in Optimizing and maintaining a Microsoft Dynamics CRM 2011 Server Infrastructure.

Parameter	Value
maxWorkerThreads	100
maxIoThreads	100
maxconnection	12*n (where n is the number of CPUs)
minFreeThreads	88*n
minLocalRequestFreeThreads	76*n
minWorkerThreads	50 (manually add this parameter to the file

So if you change the number of processors assigned to the machine - you will also need to change the machine.config

Technorati Tags: dynamics+crm,administration,optimization

Problem with filled Maps (choropleths) in Power BI for Belgian provinces

Update 2016/02/06: Thanks to the Power BI team feedback – I managed to get this working correctly – check out Using filled maps in Microsoft Power BI for provinces, regions and counties in European countries for the explanation.

A couple of weeks ago I wanted to try out the new filled map functionality (also referred to as choropleth) in Power BI ( See Tutorial: Filled Maps (Choropleths) in Power BI) – I wanted to start with a very simple data set

Province	Dutch name	French name	Capital	Surface	Population
Antwerp	Antwerpen	Anvers	Antwerpen	2860	1813282
East-Flanders	Oost-Vlaanderen	Flandre orientale	Gent	2982	1477346
Flemish Brabant	Vlaams-Brabant	Brabant flamand	Leuven	2106	1114299
Limburg	Limburg	Limbourg	Hasselt	2414	860204
West-Flanders	West-Vlaanderen	Flandre occidentale	Brugge	3151	1178996
Hainaut	Henegouwen	Hainaut	Mons	3800	1335360
Liège	Luik	Liège	Liège	3844	1094791
Luxembourg	Luxemburg	Luxembourg	Arlon	4443	278748
Namur	Namen	Namur	Namur	3664	487145
Brabant-Walloon	Waals-Brabant	Brabant wallon	Wavre	1093	393700

Unfortunately I could not get the filled map to display correctly – I tried the province names in three different languages but nothing seemed to work.

According to Bing Maps Geographic Coverage – geocoding precision for Belgium should be fairly good. What are your experieces with this – do filled maps work correctly for provinces/regions outside of US? Leave a comment.

Technorati Tags: Power+BI,visualization,geocoding

Using Microsoft Power BI Desktop to build Dynamics CRM Online Reports Part 5 –Refreshing data and custom visuals

This is the fifth part in a series of blog posts about Power BI and Dynamics CRM Online – previous blog posts:

• Using Microsoft Power BI Desktop to build Dynamics CRM Online reports – Part 1 – Introduction
• Using Microsoft Power BI Desktop to build Dynamics CRM Online reports – Part 2 – Using option sets in reports
• Using Microsoft Power BI Desktop to build Dynamics CRM Online reports – Part 3 – Relationships and the map control
• Using Microsoft Power BI Desktop to build Dynamics CRM Online reports – Part 4 -  Sharing and collaborating

When you publish a Power BI report the data will not be automatically refresh (except for direct query data sources e.g. connectivity with SQL Server Analysis Services) – so you will need to define a data refresh schedule.

But before you can define the refresh schedule Power BI needs to be able access the Dynamics CRM Online OrganizationData.svc service, fortunately this service supports certain authentication capabilities found in the oAuth2 protocol. The OAuth 2.0 authorization framework - definition from the spec at Internet Engineering Task Force (IETF) enables a third-party application to obtain limited access to an HTTP service, either on behalf of a resource owner by orchestrating an approval interaction between the resource owner and the HTTP service, or by allowing the third-party application to obtain access on its own behalf. So oAuth is one of the industry standards around federated identity and it’s main goal is to eliminate the need to give system A your user name and password for accessing system B and it allows you to determine what system B can get from system A once it’s been allowed access. So in simple terms – oAuth allows Power BI to talk to Dynamics CRM Online using the access token that you got back when first authenticate using the screen below and in this way Power BI does not need to store the user name and password.

You have to make sure that the credentials for the different data sources are up to date before you can set up the refresh schedule so you have to specify the credentials and make sure that you use oAuth as authentication method.

In Power BI Standard edition you then have the option to schedule a daily or weekly refresh – for an hourly data refresh you will need to upgrade to Power BI Pro. The table below lists the different available refresh options and the required subscription of Power BI (Source: Data Refresh in Power BI).

Data Refresh	Power BI (free)	Power BI Pro
Datasets scheduled to refresh	Daily	Hourly
Streaming data in your dashboards and reports using Microsoft Power BI REST API or Microsoft Stream Analytics	10K rows/hour	1M rows/hour
Live data sources with full interactivity (Azure SQL Data Warehouse, Spark on HDInsight)	Not supported	Supported
On premise data sources requiring Power BI Personal Gateway and on-premise SQL Server Analysis Services requiring Analysis Services Connector	Not supported	Supported

In this second part we will explore how you can use custom visuals developed by third parties into Power BI (a feature introduced with the October 2015 Update – see Visualize your data, your way using custom visuals in Power BI for more details).  In this post I will not focus on how you can build your own custom visuals but here is some background information for those who want to get started. To help developers get started, Microsoft published the code for all their visualizations on GitHub Power BI Visuals as an open source project. The project contains over 20 visualization types, the framework to run them and the testing framework. The visuals are built using D3.js which is a JavaScript library for manipulating (html) documents based on data.  From the website:
D3 allows you to bind arbitrary data to a Document Object Model (DOM), and then apply data-driven transformations to the document. For example, you can use D3 to generate an HTML table from an array of numbers. Or, use the same data to create an interactive SVG bar chart with smooth transitions and interaction.

So how does this look like from a visualization designer perspective -  first you can take a look at the Power BI Visual Gallery for some example custom visuals. Here you will need to download your custom visual definition file – in this example I will use the TadPole Spark Grid Plus – when you click on the download link – you will see that it downloads a pbiviz file.

As a data source I will start from the  sales and marketing sample which you can download from the Power BI industry samples (Excel workbooks) . You can  import this Excel file within Power BI Desktop, and Power BI Desktrop will try to import the Power Query queries, Power Pivot models and Power View worksheets which you can later on refine using Power BI Desktop. (See Import Excel workbooks into Power BI Desktop for more details).

Next I will create a new report page using data from the sales fact table (Total Units and Sales $) per manufacturer and per year. Afterwards you will need to import the definition file for your custom visual by selecting File>Import> Power BI Custom Visual or clicking the three dots in the visualizations pane and selecting the pbviz file that you just download. Next you can apply your visualization to the report data.

As you see in the example below, it shows a spark line (for sales in units and dollars) with colored and thickened line segments. The black colored segments mean that the value has gone up since last period (desirable), and the red colored segments mean the value has gone down (undesirable). (This behavior is configurable using the properties of the visualization)

In a next post I will take a look at how you can embed Power BI reports in other web applications as well as Dynamics CRM.

Technorati Tags: Power+BI,Visualization,analytics,dynamics+crm,crm+online,msdyncrm