As a software engineer based in regional Australia, my passion lies at the intersection of IAM (Identity and Access Management), cyber security, and Microsoft Entra. A few weeks ago, I received a link to the Microsoft Copilot Ninja Certificate Program. Initially, like many others, I dismissed it as just another Copilot tool. However, curiosity got the better of me, and I decided to explore its capabilities.
One thing lead to another, and now I am a Ninja. And yes, I can see you are jealous of this qualification.
The LLM Conundrum
The tech landscape is flooded with Large Language Models (LLMs), each claiming to be better than the next and can replace our jobs. However, not all LLMs live up to their promises. This proliferation of LLMs raises concerns about wasted computational resources and their impact on global energy consumption. But let’s steer away from that rabbit hole for now.
Before we begin with this opinion piece I think it is incredibly important to understand how LLM are actually operating under the scenes. Now I am no expert in this field. At university I did some Machine Leaning and Document Analysis courses which funny enough coincide just enough. As to not bore you, here is the quickest and simplest explanation.
Generative pre-trained transformer (GPT)
(The following is over simplified and generalised and would upset many Machine Learning Experts)
LLMs are trained on an immense amount of text data from sources like websites, books, research papers, and Reddit. These models learn to represent words as vectors, which you can think of as arrows on a 3D graph. Each arrow represents the “meaning” of a word, and words with similar meanings are close together in this vector space. Now, imagine that the 3D graph you envisioned earlier has not just three dimensions, but rather around 100,000 dimensions. Each dimension encodes a characteristic of the meaning. E.g. rude vs kind.
These generative transformers, trained on massive datasets, operate with an input/output framework for words. Given a context, they predict the next most likely word. In fact, they generate several potential words, but then select the most likely one based on a “temperature” parameter (which controls how creative the word choice can be).
Ultimately, these models keep predicting the next word until they reach the end of a sentence or response. It’s important to note that their process isn’t akin to human thinking; they operate by probabilistically predicting the most likely next word based on the current context.
Undoubtedly, there exists an abundance of Large Language Models (LLMs) that engage in tasks they are great at imitating but might not be great at actually processing. This is where we need to be careful when LLM meets cyber security.
Microsoft Copilot For Security's Hidden Potential
As I delved deeper into Copilot, I realized its potential impact on businesses—particularly smaller enterprises. While large corporations boast dedicated teams managing their identity ecosystems, smaller businesses often lack such resources. How do they keep up with Entra’s ever-evolving landscape? Where should they look for logs when faced with a security incident? And more importantly, what do those logs even mean?
How can we empower smaller business to improve their security posture? Well, I think LLMs are a good start. From personal experience they have a real ability to make anyone mediocre at anything. That's not to be rude by any means. Running the IT department in a small to medium business can be very hard. You wear multiple hats and need to be proficient at them all. So why do I think Copilot is a good fit here?
Capabilities
As you should now understand, each Copilot is just a wrapper around input data and the end user's input so when we think of Microsoft Copilot for Security which just need to know what they have integrated it into and why. There are four main use cases as of mid 2024:
1. Incident Summarization
Enhance your organization’s understanding of incidents and bolster internal communication by utilizing generative AI. This technology quickly transforms intricate security alerts into clear, actionable briefs. These summaries facilitate faster reactions and more efficient decision-making processes.
2. Impact Analysis
Employ AI-powered analytics to evaluate the potential repercussions of security incidents. This approach provides critical insights into the systems and data impacted, enabling you to prioritize your response strategies efficiently.
3. Reverse Engineering
Streamline your security processes by removing the necessity for manual malware reverse engineering. Empower all analysts with the ability to comprehend the tactics used by attackers. Convert intricate command line scripts into easily understandable natural language, providing lucid explanations of each action. Swiftly identify and associate indicators present in the script with corresponding elements within your infrastructure.
4. Guided Reposes for Incident Remediation
Obtain practical, sequential instructions for managing security incidents, encompassing steps for initial assessment, detailed inquiry, confinement, and corrective measures. Access to pertinent direct links for suggested procedures ensures a more rapid response.
Right Tool For The Job?
Let’s delve into the security interface which these users would use on a daily basis. Microsoft Entra typically displays list views of data with unintuitive names and meanings. Acronyms are often woven into the logs and property list views. However, bridging the gap between what a partially skilled user desires and what they actually receive requires a human translation layer.
LLMs excel at constructing highly customizable “user interfaces” by formatting information according to your specifications. But that’s only part of the challenge. While LLMs can generate convincing sentences based on input, can they provide reliable information from real-time datasets? Here lies the murkiness. No one can assert with 100% certainty that Copilot would “imagine” a hacking scenario when, in reality, the data indicates everything is secure.
To address this, we need an efficient mechanism to translate audit logs into LLM-readable insights. For instance, consider identifying risky logs in terms of Microsoft Entra ID logs. My suspicion is that Microsoft has an analytics engine under the hood which is actively processes your audit logs to find anomalies, leveraging LLMs as just a presentation layer. If this approach yields accuracy, precision, and reliability, then it becomes the right tool for the job—especially when paired with a robust analytics engine. I believe Microsoft have the data and money to make that true.
My biggest problem with this tool is that it integrates in with Microsoft Defender XDR, Microsoft Sentinel and Microsoft Intune which are all very expensive tools. Therefore, making it hard to get the best experiance for this tool unless you are willing to buy all the bells and whistles.
Implications On Security
It is hard to have a definitive answer. On one hand, lowering the bar to enter the IAM landscape is a good thing. However, one of the sharpest edges of this industry is that it take only one vulnerability to compromise the castle. Worse yet, there is so much knowledge that is built on top of very technical concepts to understand. Especially in Entra where any button can have a million dollar consequences. At the moment Microsoft is limiting the functionality of this Copilot to keep people safe.
Wrapping Up
Thanks for reading through this opinion piece. It's pretty light-hearted but I hope you can see what I mean about LLMs mixing with security. As always if you need some help navigating this technology you can contact-us.