The Truth About AI Adoption

On March 5th, Arrow ECS hosted the pan-Baltic event “Microsoft Partner Connect,” featuring me as a guest speaker to discuss Peero’s insights on the challenges of AI adoption.  

I was honored to speak to top Microsoft partners in the Baltics, though finding a message relevant to their business growth was challenging. After some research, I discovered that despite investments in Copilot licenses and training, many organizations report low adoption rates and question Copilot value. This has led customers to reassess their investments and prompted Microsoft and its partners to consider how these trends may affect renewals and future opportunities. 

Based on my decade of experience in people engagement and behavior activation, strategic initiatives like AI Adoption are more closely aligned with HR than IT. Yet for successful outcomes, both functions must collaborate! In this article I will explore the essential role of IT in providing HR leaders with Copilot usage data to support informed analysis and decision-making.  

If you’re an HR leader or in charge of rolling out AI at your organization, ask for real time, detailed, and actionable data with context from your CIO! Here’s some advice on how to get it. 

Steps for Building Copilot Usage BI Report with Live Data 

The saying “one size doesn’t fit all” applies also to AI education and training, as individuals have varying backgrounds and maturity levels requiring tailored approach. Unfortunately, default Copilot usage data available to Microsoft365 administrators is limited therefore together with Peero’s senior data engineer Toms Vesperis we embraced the challenge to build better visibility!  

The first goal was to de-anonymize Copilot usage data, which isn’t accessible by default in Microsoft365 Admin Center > Reports > Usage due to security and compliance considerations. However, administrators can use Microsoft365 Purview Portal > Solutions > Audit to run queries on Copilot workloads. This process produces detailed Copilot usage reports by individual users. 

The second objective was to automate the extraction of Copilot usage data from the Microsoft365 Purview Portal. Previously, administrators could only execute manual audit queries within the portal for data covering the last seven days. We identified a solution on GitHub that enabled the development of an automated workflow. Utilizing Power Automate > Cloud Flows > M365 Copilot Reporter – Daily Coordinator, we streamlined data extraction. While the retrieved data can be directed to any database, we opted for Power Automate Premium and transferred the data to Dataverse, which will subsequently integrate with Power BI. 

The third objective was to extract data on Copilot Chat (free version) usage, since the existing GitHub solution only supported Copilot Business (paid version). This step was crucial, as many organizations prefer testing the basic version included in their Microsoft365 subscription before purchasing. We built custom automation: Power Automate > My Flows > M365 Copilot Free Users. 

The fourth objective was to develop a live Power BI dashboard intended for sharing with HR and business organization management initiatives. The process began by importing data from Dataverse and establishing a foundational semantic model using a standard star schema. Shortly thereafter, the initial dashboard featuring live Copilot usage data was successfully implemented! 

The fifth and final objective was to expand the dataset for greater context, since we currently only track usage data by user. Adding details like user roles and departments will enable analysis of adoption rates across functions – for example, identifying higher use in marketing than in legal – and help leadership tailor training to encourage AI adoption among specific groups. 

In summary, driving successful Copilot adoption requires a collaborative approach between HR and IT, leveraging enhanced data visibility to inform strategic decisions. By overcoming limitations in default reporting, automating data flows, and enriching datasets with meaningful context, you can unlock actionable insights to tailor AI training to diverse user needs.  

But how do you handle people who resist training or attend just to tick a box? In a forthcoming article, “The Truth About AI Adoption (Part 2 – HR)”, I will examine the complex human resources factors involved in the challenges of adopting AI. In summary, employee resistance to change is common, so it is crucial to address key prerequisites prior to implementing AI initiatives. 

Artūrs Lazdekalns,

CEO