Before Dense and MoE: What Actually Happens When You Talk to an AI?

Before discussing different LLM architectures, it is useful to understand what happens during something that now feels completely ordinary: asking a chatbot a question.

Imagine an employee types:

“Summarise this document and tell me the three issues I should discuss with my manager.”

To us, this is a sentence with meaning. To a Large Language Model, however, the journey is slightly different.

1. You provide the input

The interaction starts with a prompt.

The prompt can be a simple question, but in an organisational environment it may contain much more: instructions, previous conversation, retrieved documents, corporate policies, customer information or data coming from other systems.

This distinction matters for Security teams because what reaches the model may be considerably more than what the employee sees in the chatbot window.

2. The text becomes tokens

The model does not process sentences exactly as humans read them.

The text is divided into smaller units called tokens.

A token can be a word, part of a word, punctuation or another small textual unit.

Conceptually:

Your question → Tokens → Model

Tokens matter because they are the basic units the LLM processes. They also influence context limits, computational requirements, latency and, for many cloud services, cost.

3. The model processes the context

The LLM receives those tokens together with the other information available in its context.

It then uses the patterns encoded in its parameters to determine what should come next.

This is the stage where the distinction between Dense and Mixture of Experts (MoE) becomes relevant.

In a Dense model, broadly speaking, the model’s parameters participate throughout the computation.

In an MoE model, a routing mechanism selects only some specialised groups of parameters — the “experts” — for each token.

The objective is similar.

The architecture used to perform the computation is different.

4. The answer is generated token by token

An LLM does not normally formulate an entire paragraph internally and then display it.

It generates the response progressively.

Given the context and the tokens already processed, it predicts an appropriate next token. That token becomes part of the context, and the process repeats.

Conceptually:

Prompt → Tokens → Model processing → Next token → Next token → Next token → Answer

This happens extremely quickly, which creates the conversational experience we associate with ChatGPT and other AI assistants.

5. The answer returns to the employee

The generated tokens are converted back into text and presented through the chatbot interface.

To the employee, the entire process looks very simple:

Question → Answer

But from an organisational perspective, the real picture is closer to:

User → Application → Prompt & Context → Tokens → LLM → Generated Tokens → Application → User

And increasingly there may be other components in between: identity systems, RAG platforms, vector databases, corporate repositories, APIs, agents and external tools.

Why IT and Security should care

This simple diagram reveals something important.

The chatbot is only the visible interface.

The real enterprise AI system exists behind it.

IT needs to understand where the model runs, how much infrastructure it requires, its latency, scalability and cost.

Security needs to understand what information enters the context, where it travels, which model processes it, what may be logged or retained, what systems the AI can access, and where the generated information goes afterwards.

And the organisation needs to understand something even broader:

Who is allowed to ask what, using which data, to which model, for which purpose?

Once we understand this basic journey from human question → tokens → computation → answer, the differences between Dense and MoE — and later between full-precision and quantised models — become much easier to understand.

We are no longer comparing mysterious AI technologies.

We are deciding how and where that computation should happen inside the architecture of the organisation.

Dense, MoE and Quantised Models: What IT and Security Teams Need to Know

For many organisations, the conversation about Large Language Models is moving into a new phase.

The first question was: Which LLM should we use?

Increasingly, that is not enough.

As organisations move from experimentation to production, they need to understand not only which model performs well, but how that model is built, how it will run, where it will run, what resources it requires, and what security and governance implications follow from those choices.

You do not need to understand the mathematics behind transformers to make these decisions. But you do need to understand a few architectural distinctions.

Three terms appear increasingly often: Dense, Mixture of Experts (MoE), and Quantised models.

The first important point is that these are not three mutually exclusive alternatives.

They describe two different dimensions of an LLM.

Architecture: Dense ↔ MoE
Representation and optimisation: Full precision ↔ Quantised

Understanding this distinction already makes the landscape considerably easier to navigate.

Dense: the conventional architecture

A dense model is conceptually the simplest case.

When the model processes a request, its parameters are broadly involved in the computation. The entire network represents the intelligence available to the task.

For organisations, this has an important advantage: simplicity.

Dense models are generally easier to reason about operationally. Their infrastructure requirements, performance characteristics and deployment behaviour can be relatively predictable.

The disadvantage is efficiency.

As models become larger, running all those parameters becomes increasingly expensive in terms of GPU capacity, memory, energy and ultimately cost.

Dense models therefore make sense when an organisation values predictable behaviour, operational simplicity and consistent model quality more than maximum scale efficiency.

MoE: not every expert needs to work on every problem

Mixture of Experts takes a different approach.

Instead of using the entire model in the same way for every token, an MoE architecture contains multiple groups of parameters – the “experts” – and activates only a subset of them for a particular computation.

Think of it as an organisation itself.

A company may employ hundreds of specialists, but it does not invite every specialist to every meeting. It routes a problem towards the capabilities that are most relevant.

This allows an MoE model to have a very large overall capacity while using only part of that capacity for each inference.

That can create an attractive relationship between capability and computational cost.

But there is a trade-off. MoE architectures introduce additional complexity. Routing, infrastructure, memory requirements and operational behaviour can become more complicated.

For an IT department, therefore, the question should not simply be whether an MoE model has more parameters. The more useful question is:

What proportion of the model is actually active during inference, and what infrastructure is required to operate it effectively?

A model with an enormous headline parameter count is not necessarily proportionally more expensive to run.

Quantisation: changing the economics of deployment

Quantisation addresses a different problem.

It does not determine whether the model is Dense or MoE. Instead, it changes how precisely the model’s numerical parameters are represented.

A model that normally uses 16-bit numerical representations, for example, may be converted to 8-bit or 4-bit representations.

The practical consequence can be significant.

Less precision means less memory. Less memory can mean cheaper hardware, faster inference and the possibility of running models on infrastructure where the full-precision version would simply not fit.

This becomes particularly interesting for organisations considering private AI, edge AI, local inference or on-premises deployment.

A smaller quantised model may run inside the organisation’s own infrastructure – or potentially on a workstation or edge device – rather than requiring every prompt and document to travel to an external cloud service.

There can be a quality trade-off, although modern quantisation techniques can preserve surprisingly high levels of model capability.

The organisational question therefore becomes:

How much precision do we actually need for this use case?

A customer-facing reasoning system and an internal document-classification service may have very different answers.

The choices can be combined

This is perhaps the most important concept.

An organisation could deploy:

• a full-precision Dense model;
• a quantised Dense model;
• a full-precision MoE model;
• or a quantised MoE model.

Architecture and numerical representation are separate decisions.

And neither decision should be made independently of the deployment environment.

A third dimension therefore needs to enter the discussion:

Where does the model run?

Public API, private cloud, sovereign cloud, on-premises infrastructure and local or edge devices represent very different operating models.

At this point, LLM selection stops being purely an AI question.

It becomes an enterprise architecture question.

And then comes security

This is where Security teams need to enter the conversation early rather than being asked to approve an architecture after it has already been selected.

The security implications of an LLM implementation extend well beyond the model itself.

• Where will prompts be processed?
• Where will data be stored?
• Can information be retained by an external provider?
• Who can access the model?
• Can the model access internal systems?
• Which documents can it retrieve?
• How are identities and permissions propagated into a RAG or agentic architecture?
• How are prompts, outputs and tool calls logged?
• How are models and dependencies patched?
• And where did the model itself come from?

That last question is becoming increasingly important.

As organisations download models, fine-tunes, adapters and other components from external repositories, model provenance and the AI supply chain become part of cybersecurity.

A locally deployed model is not automatically more secure than a cloud model.

It gives the organisation more control, but it also transfers more responsibility to the organisation.

The same applies to quantisation. A small model that can run locally may improve data sovereignty, but the organisation must still secure the model files, inference environment, interfaces, logs and connected data sources.

From model selection to risk-based architecture

This suggests a different way for organisations to evaluate LLMs.

Do not begin with the leaderboard.

Begin with the use case.

Ask what level of intelligence is required, what information the system will process, how sensitive that information is, what latency is acceptable, what infrastructure is available and what level of operational control is necessary.

Only then ask which model architecture fits those constraints.

The decision becomes a balance between five variables:

Capability. Cost. Performance. Control. Risk.

Dense versus MoE helps answer the architecture question.

Full precision versus quantised helps answer the resource-efficiency question.

Cloud versus private versus local deployment helps answer part of the operational-control question.

Security and governance determine whether the resulting architecture is acceptable for the institution.

There is therefore no universally “best” LLM.

There is only an LLM architecture that is more or less appropriate for a particular organisational context.

And that may ultimately be the more important shift.

As AI becomes infrastructure, choosing a model will increasingly resemble choosing databases, networks or cloud architectures. The technology matters, but the real decision concerns the system around it: where it runs, what it touches, who controls it, what it costs, and what happens when something goes wrong.

The organisations that understand these trade-offs will move beyond simply adopting AI.

They will start architecting it.