It’s been a while since I’ve broken out PowerShell to solve a problem, but a scenario came up where I thought I could automate something I needed to do – look up a bunch of Microsoft 365 Tenant IDs based on domain names. Here’s how I tackled it:
First, I actually had a list of email addresses and just wanted the domain of each one. The list was in Excel so that’s easy enough – using the Text to Columns feature I selected the data, used the ‘Delimited’ option under Original data type then pressed Next:
Then on the next step, changed the Delimiters from the default ‘Tab’ to ‘Other’ and put the @ symbol on, and as you can see in the Data preview it takes the alias off the email address for the first column, and leaves the domain in the second:
Clicking ‘Finish’ gave me a column full of domains. From this, I created a header for each row (alias and domain):
And then in Excel went to File > Save As > and called the file ‘addresses’ while picking CSV from the dropdown:
Easy enough. From here, I knew I’d need to feed this data into PowerShell using the Import-CSV command, but first I wanted to work out what the one liner command was to get a M365 Tenant ID…. except I couldn’t find one. All the examples were how to find your own M365 Tenant ID after authenticating. I knew it was public and easily accessible since sites like https://whatismytenantid.com/ work great but only accept one domain at a time.
I ended up finding a Function written by Daniel Bradley which was fairly simple and using an API, with the core of it being this one line:
Alright, we should be able to put this all together. Set the $file variable as the imported CSV file, then for each domain record run the Invoke-RestMethod command using the current $record.
Except that didn’t work because I forgot the $record is the entire object and not just the domain membertype. To specify that, we just use $record.domain so the pure domain is used.
Except that didn’t work either and I don’t know why. Instead, I just made a new variable from the $record.domain and called that $newdomain, then referenced THAT in the Invoke-Restmethod line.
That did work, so I could then echo out the results of both the current $newdomain variable, and the newly looked up $result and again specifying the membertype of tenantid (as a bunch of other info gets looked up with that command).
I also then wanted to export this data back out to a new CSV, in this case one called ‘myfile.csv’. Again, I have to work around membertypes so just make a new variable containing the single tenantid line, and use the >> operator to create/append to a file:
Works perfectly and I end up with a CSV that has a column of domains, and a column of Tenant IDs. If a domain had no Tenant ID then that value will be blank.
I’m sure this could be written better, but for quick occasional tasks for yourself, you just need something that works.
The idea of taking a Microsoft exam can be quite daunting. Self doubt creeps in, and it’s easy to talk yourself out of putting yourself through a stressful situation that you could just avoid. But, taking a Microsoft exam and passing is a great feeling, and qualifies your understanding of the topic.
I’m hoping I can convince you – the ones out there who have wondered if they should try working towards a Microsoft Certification – to give it a shot.
Let’s work through the fundamentals. There’s Microsoft Exams, and Microsoft Certifications. Often there’s a 1 to 1 relationship – pass an exam, get a certification. Some certifications have prerequisites that you’re already holding other Microsoft Certifications. Exams normally have a code such as AZ-104 where the certification doesn’t. For a nice 1 pager of all the Credentials and exams, have a look at the Certifications Poster.
Screenshot of the Certifications poster, please use the link for the latest version!
For your first exam and certification, focus on choosing one that’s at the fundamentals level. These exams all end in 9xx, such as MS-900 for Microsoft 365 Fundamentals. A full list of these fundamentals I have listed on MSPortals.io with a few handy links on each.
You’ll also see a section called ‘Study resources‘. This will contain links such as the ‘Get trained‘ area, which jumps to the certification itself and lists ways to prepare and practise for the exam.
Preparing for the exam is usually an online self-paced course, and this is worth going through to understand the topics and areas that will be covered in the exam. Practice for the exam will take you to a set of multiple choice questions, which is a good test of your knowledge to see if you’re ready to book.
Microsoft exams need a 700/1000 score to pass – which is the equivalent of 70%. For 50 questions, you should be getting at least 35 right, but that’s still cutting it a bit close. You have the luxury of time, being able to look things up and check your answer as you go so I’d be aiming more for getting >45 out of the 50 right.
You can also run an Exam Sandbox, which is just running the actual exam software with unrelated questions just to get a feel of how it’ll be when you actually sit your exam.
Fundamental exams are shorter than the more in-depth exams, and last for roughly 65 minutes with 35-50 questions. Of that hour and a bit, there’s still 15-20 minutes expected of set up and wind down.
There is a cost associated with taking a Microsoft Exam, and on the 1st November 2024 these prices have just changed. The cost is region dependant, but ranges from $44USD to $99USD for the fundamentals.
The certification page will have an area to let you ‘Take the exam‘ which is where you schedule it. Sometimes you might be able to book it for the same day, other times you may need to look a few days or weeks forward to book in an available slot. You’ll have the option of testing the software and making sure everything works as a part of this. Exams used to be in-person only, but now you can do them remote.
On the day of the exam, log in ~15 minutes early and follow the instructions you were emailed – go through the tests again, and there’s a bit of an onboarding and verification process to go through. You may need to take photos of your identity and your work area to show you’re the right person taking the exam and don’t have access to any items that would be considered cheating. You’ll be on camera and open microphone the entire exam, and have a host in the background monitoring you.
Fundamental exams are NOT open book, but other exams are (for those, you can access content on learn.microsoft.com during the exam).
Once you’re in the exam, take your time. You’ll see how long you have to go, and mostly can go back to previous questions or skip questions to answer later (when this isn’t possible, you’ll be prompted – read all screens carefully!).
If you happen to fail, don’t be disheartened. You’ll see your score, how well you did in each area, and you can take the exam again. The first time you can take it again after waiting 24 hours. Further retake policies are available here. Plenty of people fail (including me!) and just treat it as more practise – taking the exam a second time is less stressful than the first as you’ve got a much better idea on what you’re in for. The questions the second time and beyond may not be exactly the same, normally you get a random subset from a larger pool of questions – but you’ll probably see a few that aren’t new.
If you pass well done! The panic of taking the exam should be over, you’ll get a congratulations email and can take the satisfaction of posting about your achievement on LinkedIn.
If you want to check your exam/certification status, log into your Learn Profile https://aka.ms/LearnProfile but don’t expect this to fully update immediately after the exam, some of the information can take a day or two to update.
I’ve also collected a lot of Microsoft exam and certification related links and created a ‘Training’ section on MSPortals.io: https://msportals.io/training?search=
If you have any questions or want any advice, drop a comment below. If you pass an exam, post it on LinkedIn and tell me about it so I can congratulate you! https://www.linkedin.com/in/adamfowlerit/
Word, Excel, PowerPoint, Outlook, OneNote, and Teams (unless you’re in the EU) are some of the apps that make up the Microsoft 365 Apps suite. We don’t call it Office 365 anymore, and they’ve been around for a very long time. Despite the name change, ‘Office’ is used across Microsoft documentation, the Essential Eight, Windows Registry settings etc so I will use also use it for the rest of this article.
Unsurprisingly, there’s both a lot of flexibility in configuration options for these apps, as well as many settings that have security considerations. As with my other blog posts of late, I wanted to have a look at the Center for Internet Security’s (CIS) Microsoft Intune for Office Benchmark 1.0 and pick my favourite 5 recommendations; ones that I think have a high impact, aren’t on by default, and/or ones you may not have considered.
As with other Intune benchmarks, you don’t have to use Microsoft Intune (you can use Group Policy/registry) but these options are natively supported via Intune. To create these policies via Intune from the Microsoft Intune admin center go to Apps > Policy > Policies for Office apps.
…so if you aren’t doing the above (or if you’re not sure) – go sort that out first before you worry about these extra ones!
Alright, let’s get on with my 5 picks:
#1 – 2.3.23.2 Ensure ‘Block signing into Office’ is set to ‘Enabled: Org ID only’
Official description of the setting: This policy setting controls whether users can provide credentials to Office using either their Microsoft Account or the user ID assigned by your organization for accessing Office 365. If you enable this policy setting, you can specify one of the following options: – If you select “Both IDs allowed”, users can sign in and access Office content by using either ID – If you select “Microsoft Account only”, users can sign in only by using their Microsoft Account. – If you select “Organization only”, users can sign in only by using the user ID assigned by your organization for accessing Office 365. – If you select “None allowed”, users cannot sign in by using either ID. If you disable or do not configure this policy setting, users can sign in by using either ID.
Note: This policy does not apply to licensing. A user can license their product using any applicable ID if they have a valid license associated with that account. Providing credentials for licensing purposes when that ID type has been disabled, however, will not affect the signed in state of Office.
This setting controls whether a consumer Microsoft Account can be used to sign into the Office suite. By default, both a work account and a Microsoft Account can be signed in, so changing it to Org ID only prevents that. This prevents a user either accidentally or wilfully saving and opening files from their personal OneDrive and anywhere else the Microsoft Account may have access to. You can imagine a user not realising they’ve been saving their last year of work on their personal unprotected OneDrive, or doing so because it made it easier to continue working on documents via their home computer. There should be no legitimate business need for this setting to be allowed, so change it.
#2 – 2.3.38.1.1 Ensure ‘Improve Proofing Tools’ is set to ‘Disabled’
This setting controls whether data learnt from Office Proofing Tools (such as spell check) is sent back to Microsoft. This option is enabled by default. It will include information such as additions to the dictionary (maybe you keep writing Project Phoenixx but that’s actually the ‘correct’ spelling’) or maybe your drivers license combination of letters and numbers, or credit card. Here’s the actual description of the setting:
This policy setting controls whether the Help Improve Proofing Tools feature sends usage data to Microsoft. The Help Improve Proofing Tools feature collects data about use of the Proofing Tools, such as additions to the custom dictionary, and sends it to Microsoft. After about six months, the feature stops sending data to Microsoft and deletes the data collection file from the user’s computer. If you enable this policy setting, this feature is enabled if users choose to participate in the Customer Experience Improvement Program (CEIP). If your organization has policies that govern the use of external resources such as the CEIP, allowing the use of the Help Improve Proofing Tools feature might cause them to violate these policies. If you disable this policy setting, the Help Improve Proofing Tools feature does not collect proofing tool usage information and transmit it to Microsoft. If you do not configure this policy setting, the behavior is the equivalent of setting the policy to “Enabled”.
Beyond this data going back to Microsoft, it’s also saving it on your computer in a secondary data collection file. Quite simply, it’s introducing extra risk in both a second location of data + sending off to Microsoft, with no direct immediate user benefit, and no obvious method of showing what data it’s transmitting so should be disabled. On this point, this isn’t questioning how much you trust Microsoft or not – you’re probably using their operating system, software, cloud storage, search results and AI – risk is risk and you reduce it wherever you can that makes sense, and this is one of those scenarios.
#3. Modern Office File Formats: 2.11.8.6.1 Ensure ‘Default file format’ is set to ‘Enabled: Word Document (.docx)’ 2.2.4.6.1 Ensure ‘Default file format’ is set to ‘Enabled: Excel Workbook (*.xlsx)’ 2.6.6.5.1 Ensure ‘Default file format’ is set to ‘Enabled: PowerPoint Presentation (*pptx)’
These are all the same but each application needs it’s own setting enabled. Worth noting is the same setting exists for Access – ideally you don’t have that anywhere, but if you do, change that setting too. It’s also actually two settings – enabling it, then setting the ‘Save x files as’ and choosing the above listed options, e.g. PowerPoint Presentation (*pptx).
Although this setting doesn’t block the older default Office document types (.doc, .xls, .ppt), it makes sure the default format for saving is the newer .docx, .xlsx, pptx. The older formats were the default up to Office 2003, and in Office 2007 onward is where the ‘x’ version (which is based on XML and if you rename any of these documents to .ZIP, you can check out what’s inside!) was introduced. Although I can’t find much officially around the differences, the general takes are that the newer format is less prone to corruption, more secure, better organised internally, and more open for other programs to be able to read the data inside.
Most companies will have the older file formats floating around still, but this setting works towards encouraging the new (and 16 years since release, it’s hard to still call it ‘new’!) file format.
Setting description from Word: This policy setting determines the default file format for saving files in Word.
If you enable this policy setting, you can set the default file format from among the following options:
– Word Document (*.docx): This option is the default configuration in Word. – Single Files Web Page (*.mht) – Web Page (*.htm; *.html) – Web Page, Filtered (*.htm, *.html) – Rich Text Format (*.rtf) – Plain Text (*.txt) – Word 6.0/95 (*.doc) – Word 6.0/95 – Chinese (Simplified) (*.doc) – Word 6.0/95 – Chinese (Traditional) (*.doc) – Word 6.0/95 – Japanese (*.doc) – Word 6.0/95 – Korean (*.doc) – Word 97-2002 and 6.0/95 – RTF – Word 5.1 for Macintosh (*.mcw) – Word 5.0 for Macintosh (*.mcw) – Word 2.x for Windows (*.doc) – Works 4.0 for Windows (*.wps) – WordPerfect 5.x for Windows (*.doc) – WordPerfect 5.1 for DOS (*.doc) – Word Macro-Enabled Document (*.docm) – Word Template (*.dotx) – Word Macro-Enabled Template (*.dotm) – Word 97 – 2003 Document (*.doc) – Word 97 – 2003 Template (*.dot) – Word XML Document (*.xml) – Strict Open XML Document (*.docx) – OpenDocument Text (*.odt)
Users can choose to save presentations or documents in a different file format than the default.
If you disable or do not configure this policy setting, Word saves new files in the Office Open XML format: Word files have a .docx extension. For users who run recent versions of Word, Microsoft offers the Microsoft Office Compatibility Pack, which enables them to open and save Office Open XML files. If some users in your organization cannot install the Compatibility Pack, or are running versions of Word older than Microsoft Office 2000 with Service Pack 3, they might not be able to access Office Open XML files.
This policy setting is often set in combination with the “Save As Open XML in Compatibility Mode” policy setting.
The 4 settings in Intune are below, and the Group Policy/Registry settings are here: WordAccessExcelPowerPoint
#4. 2.3.23.3 Ensure ‘Control Blogging’ is set to ‘Enabled: All Blogging Disabled’
I partly like this one because not many people know this is even a thing. Description: This policy setting controls whether users can compose and post blog entries from Word. If you enable this policy setting, you can choose from three options for controlling blogging:
* Enabled – Users may compose and post blog entries from Word to any available blog provider. This is the default configuration in Word.
* Only SharePoint blogs allowed – Users can only post blog entries to SharePoint sites.
* Disabled – The blogging feature in Word is disabled entirely.
If you disable or do not configure this policy setting, the behavior is the equivalent of setting the policy to Enabled-Enabled.
Word can send off contents of documents to certain blogging platforms via a direct connection from inside the application, and is enabled by default. Although the amount of your user base that would even consider this is quite low, all it takes is for one person to decide to do it, then publish the wrong document to a public site.
As usual, there’s usually no great reason to allow this at all, so disable it – even restricting to SharePoint sites doesn’t mean it’s restricted to the SharePoint sites you control.
Intune setting is Control Blogging, which you need to Enable and set to All blogging disabled, or Group Policy/Registry settings here.
5. 2.5.14.3.4 Ensure ‘Outlook Security Mode’ is set to ‘Enabled’
There’s an Outlook Security Mode? Sounds like something that should be enabled! Description: This policy setting controls which set of security settings are enforced in Outlook.
If you enable this policy setting, you can choose from four options for enforcing Outlook security settings:
* Outlook Default Security – This option is the default configuration in Outlook. Users can configure security themselves, and Outlook ignores any security-related settings configured in Group Policy.
* Use Security Form from ‘Outlook Security Settings’ Public Folder – Outlook uses the settings from the security form published in the designated public folder.
* Use Security Form from ‘Outlook 10 Security Settings’ Public Folder – Outlook uses the settings from the security form published in the designated public folder.
* Use Outlook Security Group Policy – Outlook uses security settings from Group Policy.
Important – You must enable this policy setting if you want to apply the other Outlook security policy settings mentioned in this guide.
If you disable or do not configure this policy setting, Outlook users can configure security for themselves, and Outlook ignores any security-related settings that are configured in Group Policy.
Note – In previous versions of Outlook, when security settings were published in a form in Exchange Server public folders, users who needed these settings required the HKEY_CURRENT_USER\Software\Policies\Microsoft\Security\CheckAdminSettings registry key to be set on their computers for the settings to apply. In Outlook, the CheckAdminSettings registry key is no longer used to determine users’ security settings. Instead, the Outlook Security Mode setting can be used to determine whether Outlook security should be controlled directly by Group Policy, by the security form from the Outlook Security Settings Public Folder, or by the settings on users’ own computers.
If you need to change any of those related settings from the default, you instead need to change this from ‘Microsoft recommended baseline’ to Manually configured, and ‘Use Outlook Security Group Policy’ – and then ensure all related policies are configured the way you want.
The CIS benchmark documentation also mentions: Note: This setting is essential for ensuring that the other Outlook security settings mentioned in this baseline are applied as suggested.
So, what all this means is the CIS benchmark overall has different configuration recommendations compared to the Microsoft recommended baseline, but in doing this option it’s worth assessing all the settings that the baseline would do!
Intune setting is ‘Outlook Security Mode’ and Group Policy/Registry settings here
I hope you found the above options interesting, and as always this is designed to grow awareness of what you need to consider in managing an environment, and always have that security mindset. These options are not set and forget either – you need frequent checks to make sure no gaps have been created either by reconfiguration or new settings coming in.
I’ve been diving into the Center for Internet Security’s (CIS) benchmarks lately – which are a set of benchmarks to use against different technologies (including Microsoft 365 and freely available for non-commercial use). They are a good set of checks to go through in a tenant to review configuration with a security focus; including how to remediate.
There is of course a lot more to it than reading a document and configuring items the way it says to; you need to understand what you’re changing, and what impact that may have to the business and it’s end users. For example; blocking the ability to share anonymous links from SharePoint/OneDrive is generally ‘a good idea’ security wise, but if your users are actually doing that you probably don’t want to just shut that off. You need to assess what’s being used and how, and have a strategy to get to a more secure point.
Anyway, I’ve picked my favourite 5 settings from their comprehensive list that I feel people could miss; I may have missed these myself when I used to be a Microsoft 365 administrator.
For PowerShell commands, if you’re not sure how to get to the right module (e.g. Exchange Online from my first example) then check out msshells.net which will show you how to install and connect.
The headings are quoted from CIS, but the rest of the material is my own:
1 – 3.1.1 Ensure Microsoft 365 audit log search is Enabled If you’re a Microsoft 365 focused admin, Azure and log search may not be a front of mind for you unless you go looking to solve a problem that arises.
This should be enabled in new tenants, but older ones may not have it. First check it’s status with the PowerShell command in Exchange Online:
This should have no user impact and just enables the ingestion of the Audit Logs.
To view these logs you can use PowerShell commands, but this is generally one I’d rather use a GUI for – go to the Microsoft Purview portal and the Audit section, and trigger a search. Without getting into too much detail, there’s two tiers of Audit – Standard and Premium. Read further information here.
2. Ensure modern authentication for SharePoint applications is required
This is another that many old tenants may have disabled. SharePoint has ‘legacy authentication’ similar to other services that are planning or already deprecated legacy auth – Exchange Online being the common one most people know about.
To check if you have this disabled, connect via PowerShell to SharePoint Online and run the command:
Get-SPOTenant | ft LegacyAuthProtocolsEnabled
True means it’s enabled, False means disabled – and we want it to be False. The command to enable it is:
Set-SPOTenant -LegacyAuthProtocolsEnabled $false
Entra ID’s Conditional Access should be configured to block all Legacy Auth requests also, but this is an extra layer to make sure SharePoint won’t work that way anyway (plus holes are poked through Conditional Access all the time!). There seems to be very little official public documentation about this option from Microsoft – I could find this example where they show how to set it to $true to avoid some login issues which is a bit concerning.
This is one that may have some user impact or application impact if systems are connecting to SharePoint Online in legacy ways. Users should be used to modern auth and match their experiences in other Microsoft 365 services – applications however would need to be redesigned or updated to accommodate this. You can search the Entra ID authentication logs for any attempts to connect to SharePoint Online using legacy authentication over as long as possible before changing this setting.
3. Ensure sign-in to shared mailboxes is blocked (Automated)
Shared Mailboxes are both a delight for having a central area for emails to go to, and multiple staff having access to them, but also a dismay in user expectations of being able to send as the account, and potentially log in as it. A reception desk or similar may have multiple people jumping in and out of the location, but they want to access the same contents without the time-taking task of logging in and out of the computer each time. Regardless – security wise each user should have a unique login, and all their actions performed under that login.
Although it can be a fight and go way beyond a technical issue – shared mailboxes should be disabled from logging in. A disabled from login shared mailbox can still send and receive emails; you’re only disabling the ability to log in using that account itself, and the mailbox can still be accessed as a delegate.
As this is a per account setting, you’ll need to check all shared mailboxes. As per Microsoft Learn, you can block a single account from the Microsoft 365 Admin Center and go to Users > Active Users, select a Shared Mailbox, and click the ‘Block Sign-In’ option:
… but this doesn’t really scale to check all Shared Mailboxes and change the setting. Instead, as per Microsoft Learn, we have to use Exchange Online PowerShell to find the shared mailboxes, then we can use Microsoft Graph PowerShell SDK to disable them. After connecting to Exchange Online and Microsoft Graph with the below scope:
Connect-Graph -Scopes User.ReadWrite.All
You can then run the one command to use Exchange Online to find all Shared mailboxes and then use Microsoft Graph to set the account to disabled:
Also note that Exchange Online has it’s own ‘AccountDisabled’ variable which you could set to true, but this blocks sign in to the mailbox, and not the entire account to any M365 service/Entra ID authentication.
User impact on this needs to be assessed by again checking the Entra ID logs against each Shared Mailbox, and working out how to set systems up to avoid the shared account login. There may be some user resistance to this, but one argument could be ‘what if someone sent a nasty email to your boss under the account and they thought it was you – you couldn’t prove it wasn’t easily if others are also using that same account’.
4. Ensure ‘Per-user MFA’ is disabled
This is the ‘old’ MFA before Conditional Access was around. This only suported the MFA methods of ‘Call to phone, Text message to phone, Notification through mobile app, and Verification code deom mobile app or hardware token’.
If you see any users enabled, then you should ensure Conditional Access is set up and ready to go, then change the users to disabled. You also shouldn’t be using this function at all when Conditional Access is enabled. A few warnings from Microsoft:
5. Ensure a dynamic group for guest users is created
I quite like this one. Yes, you can reasonably easily determine if an account is a guest account or not, but having an automated group means it’s easy to point other Conditional Access policies, or other monitoring, on what these accounts are doing in your tenant. It’s very little effort to create, doesn’t need maintenance, and can help in other scenarios when you want to review what guest accounts are around.
How to do this is well documented by Microsoft but you do need an Entra ID Plan 1 or Plan 2 license to create dynamic groups.
and after selecting a Group Name and choosing Memebership type: Dynamics user, click ‘add dynamic query’:
Use the filters Property = userType, Operator = Equals, Value = Guest, and click ‘Save’, then ‘Create’.
Note that it can take a minute for the group to initially populate. You can now use this group to block from certain things as an extra layer of protection against accidental permissions, or have extra Conditional Access policies that always require certain MFA methods.
If you don’t have Entra ID Plan 2 to be able to have guest reviews, you could use the membership of this group as a simple way to review the guest accounts in your tenant.
That’s my top 5 picks – check out the CIS Benchmark for Microsoft 365 yourself along with their other benchmarks as there’s a lot to learn and check through. It’s also not a one-time thing, settings change, the benchmark itself grows (currently at v3.1 at the time of writing), plus there’s more security to check beyond this!
I’ve recently purchased Copilot for Microsoft 365 to play with on my own tenant, and wanted to share my experience. No, I did not use Copilot or any other AI to write this post :) Some of the below may sound picky, but I’m trying to be clear around names and functions as I found a lot of it hard to correctly define as I went. You’re going to see the word ‘Copilot’ a lot – sorry.
First, I’ll attempt to clarify that I’m only looking at Copilot for Microsoft 365. What is “Microsoft Copilot” is a harder question to get your head around, because Bing Chat for Enterprise/Bing Chat Enterprise is now just called Copilot. Copilot for Microsoft 365 is Copilot integrated into the Microsoft 365 apps – so think of Copilot as the AI solution itself, and “Copilot for X ” as anything else as Copilot being integrated with, and able to use some of the data in it, as well as giving answers more contextual to that solution (but not limited to!).
I found this really good graphic of Microsoft Copilot but can’t find the original source!
Alright, so looking at the options below we have Copilot which is free, Copilot for Microsoft 365 which you pay for per user and integrates into Microsoft 365 apps… and there’s also Copilot Pro which gives you integration with some of the Microsoft 365 apps and a few extra base Copilot perks. Copilot Pro is for consumers and targeted at individuals, you can’t buy this against a business account.
In my own tenant of 1 active user, I purchased Copilot for Microsoft 365. I had to do this for a year because it was either that, or 3 years. My tenant is quite old, US based, and also has some trial/unique test licenses applied from Microsoft.
Note that despite the official diagram above’s title is “Microsoft 365 Copilot”, that was the name at launch and it is now “Microsoft Copilot for Microsoft 365”
The purchasing and assigning of Copilot for Microsoft 365 is somewhat of a non-event like most other licenses. I was able to buy a single license, as Microsoft reduced the 300 minimum requirement in December 2023. From the Microsoft 365 admin center, you have the following subservices available:
From the user side, I just noticed the Copilot options turn up as soon as I went to check them. The first thing I wanted to work out, was how do I interface with Copilot when it’s app agnostic? Here’s where the fun started;
https://copilot.microsoft.com, https://www.microsoft365.com/chat/, and https://bing.com/chat look pretty much the same, two have a ‘work’ option to make sure you’re processing data in your own tenant and not giving sensitive information back to Microsoft in other ways… and the third requires a work login to access – so what’s the difference?
Honestly I’m not sure. I tried working this out but failed – maybe there isn’t one? It will save what you searched, and that history will be visible across all three chat solutions:
I was going to use this example question of ‘how many emails did I get yesterday’ as my next point of frustration – when I have been testing this for the last few days, it’s incorrectly told me each time that I’ve received just 1 email. Then when asking it about another email I received in that yesterday timeframe, agreeing that also was received yesterday. When questioning it around the discrepancy, it stopped the conversation.
However, it looks like that’s already been resolved:
I am glad it’s now not giving an incorrect answer, but I did expect it to be able to actually answer this question with a correct number. We’ll move on to finding some information I know is in there:
I asked to see ALL the emails where I’ve passed a Microsoft exam. It found one from 2008, something else irrelevant to exams, and then 6 other emails for me to check out.
Note that I have a bunch of emails that should quite clearly be picked up, such as this one from February this year and I found by doing a search of the word ‘Exam’ on my mailbox:
How about certifications then? For some reason it did find a much newer email about a Certification renewal (the word ‘Exam’ wasn’t on that one). When I asked to ‘show me more emails’ it proceeded to just do what I asked out of context, and showed me 5 example emails from my mailbox unrelated to the previous query.
These experiences are frustrating – partly because I can see the amazing potential Copilot has (honestly I can!), but also how it can consistently miss the mark of my expectations around it. It could be that my expectations are too high – but if they are, then it’s Copilot’s job to set them correctly as part of it’s answer system. And no, putting a label saying ‘AI-generated content may be incorrect’ at the bottom of everything doesn’t quite cut it.
OK, how about asking Copilot for Microsoft 365 about files? Asking it what I accessed last is correct, and matches what I see on the ‘My content > All’ section of Microsoft 365. However, that fell over quickly when I asked it was folders were in the root of my OneDrive – it claimed it’d have to do a search for that (why don’t you just go search then?), as well as showing me an ini file I’d opened 6 days ago for Diablo IV – and for reference, that was the 7th last file I’d accessed. Asking it to search for the folders resulted in it telling me that now it had done that search but couldn’t list the folders. Taking it’s next suggestion, I asked it to list the contents of a folder called Work – I’d created it a few days ago and it has only ever had 1 file in it. The results came back incorrect again, claiming the presentation2 file which as per the first result, was in a folder called ‘Documents’ and not in a folder called ‘Work’.
OK, enough digging for data.
My other surprise on purchasing the Copilot for Microsoft 365 license was receiving a call at about 5am, which although I woke up to, did not answer in time. I called it back, heard a recording saying it was Microsoft, and assumed it was a scam call. Checking my emails later, I noticed that a case had been logged in my name at 1:06am called “Getting started with Copilot, we’re here to help.”
I logged onto the support area of the Microsoft 365 Admin portal, and yes there was a ticket under my name, with my mobile phone number (including the +61 area code for Australia) that had been logged for me. Yes, I have notes on this experience:
Although Microsoft Support can be used for both break/fix and advisory calls, it should not be used as a marketing tool to proactively ensure a customer is getting value from something they only just purchased. In other words, don’t shoehorn a solution to a problem you see, into a different system not designed for that.
Don’t list it as the customer doing the action themselves if you automate something on their behalf.
Don’t put down on my behalf that I’d like a phone call about the ticket you logged pretending to be me.
Have a look at the customer’s timezone and call them during business hours.
Who came up with the incident title? At least start with ‘Auto generated’ – you’ve used the title as a way to communicate to me when that’s very much not what the title of an incident is supposed to do.
After calling me and waking me up, don’t send an email asking me to respond, but if I don’t you’ll call back again, but claim to do so ‘again’ during business hours.
Don’t use the Status of ‘Feedback’ when it really isn’t – I’m probably not going to have feedback a few hours after enabling the service (but give me a few days to write up a blog post!).
Support also advised me that “A ticket is logged when copilot is purchased” and proceeded to give me a bunch of links about Copilot for Microsoft 365 anyway. Seems like that could have just been the email they sent without all the other noise.
There was one good link in there which was about Copilot prompts – worth a quick look but seriously, why isn’t this just linked at the top of the Copilot prompt area? There’s a lot of white space this link could go in.
I’ve had a few others claim similar experiences when enabling Copilot for Microsoft 365 including Microsoft MVP Karen Lopez:
Sure. Added info: I told them I was speaking at a Microsoft event and wasn’t available for a call that week. Asked then to try again mid-April. They stopped it all.
I know I’ve banged on about frustrations here, but my general point is to try and set realistic expectations around the current state of Copilot for Microsoft 365. It is not a magical answer to doing most your work for you. It is really good at writing responses for you as either starters, frameworks, or mostly done content to fine tune. It’s really good at summarising emails. It’s really good at responding to something you don’t want to spend time on – I was ‘invited’ to attend free LinkedIn Workshops to help me put content on that platform, and clicking reply brings up a great Copilot experience – auto answer type buttons depending on the response I want to give, an area to get Copilot to help me draft a response, or I can just start typing and Copilot stays out of the way.
Although I can’t think of many situations that a poem would be my response, it’s one of those options you have to try:
So yes, these sorts of functions are hugely valuable just for these sort of use cases on email. It also does a lot of great stuff in Word, PowerPoint, and Excel, along with Outlook as above – but those deserve their own posts. Copilot, and in turn Copilot for Microsoft 365 is going to get better at a hugely accelerated rate, and the items that are less focused on purely the language side of LLM and a bit more data based will be valuable. And, despite my criticisms above, I still think everyone should buy or at least try this to learn, get ready, and understand what is actually possible right now in our era of AI – just make sure your environment is ready for it with the right controls, processes, and security in place.
