Thursday, January 19, 2023

Technical debt and Dynamics 365 solution architecture

Technical debt is an important concept to understand as a solution architect. As outlined in Thoughts about the Dynamics 365 solution architect role, a solution architect has to keep the long-term use of the solution in mind and implement scalability and adaptability into the solution for the future. This is where you will need to understand the concept of technical debt.  Software design and development is full of metaphors to help non-technical people understand specific concepts - technical debt is one of them. 



What is technical debt?

Technical debt is created by (un)conscious actions to take shortcuts in development or design of a software solution. These shortcuts are taken due to some short-term benefits such as faster delivery time, easier implement etc. ... but they have an impact on the longer term. 

Having technical debt is like having financial debt, where time spent on this "not-quite-right implementation" counts as interest on the debt and paying the interest on the loan (the debt) might slow down the project/program in the long term. 

You will see 3 types of technical debt in projects:
  • Strategic technical debt: technical debt caused by decisions made to shorten time to market, delay development or expenses, ...
  • Unavoidable technical debt: technical debt accumulated due to the fact that the platform/technology/product version might not be supported anymore, ...
  • Naïve technical debt: due to business immaturity (unclear or changing requirements, misalignment between product owner and business stakeholders) or technical immaturity (no coding standards, no peer reviews, no automated tests, no refactoring, no consideration for)

It is important to grasp that technical debt is not always an indicator of low quality (but it might be). Technical debt happens because even though we do the best we can - given what we know - we subsequently learn better ways to solve a problem or better ways appear due to technical improvements in the underlying platform or technology stack.  You probably heard the phrase "Ignorance is bliss" - well for technical debt this is not the case. The fact that nobody in a team has heard of technical debt should be treated as a red flag for software quality.

How to repay technical debt?

Regardless of the type of technical debt you accumulated, in the end you will always need to repay the technical debt to make sure that you don’t jeopardize your product evolution. If the cost to implement a certain change (be it a bug fix or adding an additional feature) keeps increasing over time, it might be an indicator that you have incurred technical debt on your project.

The first step is however making business aware of the existence of technical debt and the impact of it. I really like the ideas suggested in The Wall of Technical Debt on how to make technical debt visible. Building a regular habit of identifying technical debt and learning on how to negotiate with business on technical debt payoff is a good approach.

Keeping technical debt to a minimal is a team effort - make sure that delivering quality is engrained in the culture and way of working of the team.

How to identify technical debt in your Dynamics 365 solution architecture?

As mentioned previously technical debt is automatically accumulated as changes are happening on the technical platform underpinning your solution. So I highly recommend to follow up on the  important changes (deprecations) in Power Apps and Power Automate page and also on the Deprecation of Dynamics 365 apps page. 

By writing code, you are automatically accumulating some amount of technical debt but there is tooling available to help you identify technical debt in your code and solutions:

  • SonarQube - static code analysis tool which helps you identify bugs and code smells - works for multiple languages. Quite useful for .NET and Javascript development which are typically used in CRM development
  • Power Apps solution checker - specific Power Platform tooling to analyze Dataverse or Dynamics CRM solutions. Can be run from the maker portal,  Microsoft Power Platform Build tools for Azure DevOps or PowerShell
  • Checkmarx provides static application security testing (SAST and open source composition analysis (SCA). It helps you uncover security issues in your code base or in 3d party libraries/packages that you are using. If you are looking for alternatives, you might take a look at the vulnerability scanning tools listing from OWASP
Technical debt goes broader than purely code. Technical debt is a collection of issues which makes the overall solution hard to maintain, prevent easy implementation of updates, might impact performance and scalability. So lack of technical documentation, absence of a good application lifecycle management (ALM) process, lack of test automation/heavy reliance on manual testing, missing architectural design decision log, brittle integration, monitoring, etc ... are also things to take into account as part of a health assessment of a solution architecture.

References:

Tuesday, January 10, 2023

Interactive chart visualizations using Python and bqplot: visualizing S&P500 returns

A couple of months ago, I stumbled upon this interesting presentation Jupyter Notebooks: interactive visualization approaches. The presentation showed how you can use bqplot to build interactive visualizations. 

Bqplot contains a set of 2D plotting widgets built on top of the ipywidgets framework for Jupyter notebooks. The bqplot package aims to bring d3.js visualizations to Python while retaining the flexibility and ease of use of ipywidgets and was developed by the quantitative research team at Bloomberg. You can install bqplot using conda or pip. 



One of the examples built by the team that you can find on Github is a Jupyter notebook which shows US equity market performance (using the S&P 500 index) where you can select an interval on a time series chart - for the selected area you get the total return as well as a histogram of the daily returns.

References:

Monday, January 09, 2023

Why I blog and you might want to consider it as well

A couple of weeks ago I got the question why I write a blog since it would be better to do a podcast or publish videos on technical content. In this post, I will explain why it is still a good idea to start blogging.  I wrote my first blog post almost 18 years ago  and got inspired by Robert Scoble (Scobleizer) who worked as a technical evangelist at Microsoft at that time and had a very popular blog.  1483 blog posts and almost 2 million views of these blogs I still consider it a good idea, since  I think that a lot of good things have happened to me in my professional live because I decided to start my blog and share my thoughts on technology.  



Starting a blog is an easy starting point for you to be active in the technical community. Sharing is caring might sound lame but it is a good credo to live by as an IT professional/consultant.  In a professional context,  most persons  only know the people that they directly work with - colleagues or customers. 

Blogging (but also Twitter and technical communities) allows you to expand that and suddenly there are people all over the world that you can learn from and they can learn from you. Suddenly you will realize there are others who face similar challenges. By sharing your experiences and offering advice to support, you can build a community of readers who value your experience. By writing good high-quality content on a regular basis, you can establish yourself as an expert in your field which in turn will help you build your personal brand and increase your visibility. Starting a blog is a lot easier than creating a video on YouTube or doing a podcast.

The most underrated skill to have in IT is the ability to communicate efficiently. Blogging is levelling that skill up. In an era where people mostly rely on PowerPoints slides for presenting a plan or a vision, it is quite refreshing to see them back it up with a well-written, solid document afterwards. I usually cringe when seeing poorly written documents or e-mails so I think that good writing skills will set you apart. 

I think of a blog as a way to write notes to myself and you should too. Throughout your daily job, you are continuously solving problems: write about how you approached solving these problems and where you found helpful information. By writing your approach down, you will remember it better and it will be easier for you to find it later (and others will benefit of this as well). 


The more you explain a concept or a topic to other people, the better you will get at understanding it yourself. This might sound strange but it is true, teach to learn. When you need to teach something, you are required to understand it in a deep and comprehensive way. The process of writing, will help your own understanding and requires you to think critically and find creative ways to present the topic which further enhances your understanding and retention of the information.

Even when you are learning something new, you are probably taking notes. Blog these notes, taking these notes. Rewriting them will help you understand the content much better, while it helps others too. If you think that everything is already written, do it for your own reference. If you find it useful yourself to go back to your blog, then it is OK. Don't fuzz to much about getting little visitors to your blog in the beginning.

References:

Thursday, January 05, 2023

Dynamics 365 and Power Platform monthly reading list December 2022

First post of 2023 - so let's close with last month's  suggested reading list. If you want to catch up on previous reading lists for 2022 - here is the list:

 Technical topics (Configuration, customization and extensibility)

Topics for Dynamics 365 Business Applications Platform consultants, project managers and power users

Monday, January 02, 2023

Notes on deploying and troubleshooting a Streamlit app on Azure App Services

 A couple of weeks ago I was playing around with Streamlit and decided to deploy it on Azure a
using Azure App Services using the guidance from Deploying Streamlit Applications with Azure App Services . 

Streamlit is an open-source Python library that allows you to create interactive, data-driven web applications in just a few lines of Python code. It does not require you to have any JavaScript, html or CSS experience.  




The deployment using the steps outlined in the blog post went quite smooth but when I navigated to the website, I was greeted by an exception.

Since I haven't worked with Linux for over 20 years now, I feared to be in for a long and painful experience to get this resolved but it actually turned out to be easier then expected. 

First step, I took was looking at the Application Logs for the Azure Web App. Go to the Azure App Service > Diagnose and solve problems > Application Logs. 

When scrolling through the Application Logs

The exception log "TypeError: Descriptor cannot be created directly. Your  generated code is 
out of data and must be regenerated with protoc > 3.19.0. If you cannot immediately
regenerate your protos, some other possible workarounds are: 1. Downgrade the protobuf package to 3.20.x or lower" actually pointed me to a thread on the Streamlit forums - Issue with Protocol Buffers. After changing requirements.txt  to deploy a newer version of Streamlit (see Configure a Linux Python App for Azure App Service for more details on how the Azure App Service deployment engine automatically runs pip install.) all started working correctly again.