AI LLM Tool Ollama: Architecture and Conversation Processing Flow Analysis
- Overview of Ollama
- Ollama Architecture
- Ollama Storage Structure
- Ollama Conversation Processing Flow
- Conclusion
- References:
Overview of Ollama
Ollama is a convenient tool for running LLM (Large Language Models) quickly. With Ollama, users can easily interact with large language models without complex environment configurations.
This article will analyze the overall architecture of Ollama and detail the specific processing flow when users engage in conversations with Ollama.
Ollama Architecture

Ollama employs a classic CS (Client-Server) architecture, where:
- Client: Interacts with the user via the command line.
- Server: Can be started via the command line, desktop application (based on the Electron framework), or Docker. Regardless of the startup method, the same executable file is ultimately invoked.
- Communication: Client and Server communicate using HTTP.
The Ollama Server consists of two core components:
ollama-http-server: Handles interactions with the client.llama.cpp: Serves as the LLM inference engine, responsible for loading and running large language models, processing inference requests, and returning results.- Interaction:
ollama-http-serverandllama.cppalso communicate via HTTP.
Note: llama.cpp is an independent open-source project that is cross-platform and hardware-friendly, capable of running on devices without GPUs, including Raspberry Pi.
Ollama Storage Structure

Ollama uses the default local storage folder path $HOME/.ollama. The file structure is as follows:
$HOME/.ollama/
├── blobs/
├── manifests/
├── logs/
│ └── server.log
├── history
├── id_ed25519
└── id_ed25519.pub
Files can be categorized into three types:
- Log Files: Includes the
historyfile, which records user conversation inputs, and thelogs/server.logserver log file. - Key Files: Includes the
id_ed25519private key andid_ed25519.pubpublic key. - Model Files: Includes
blobsraw data files andmanifestsmetadata files.
The metadata files, such as models/manifests/registry.ollama.ai/library/llama3.2/latest, contain the following content:

{
"digest": "sha256:abcdef1234567890",
"size": 123456789,
"annotations": {
"org.opencontainers.image.title": "llama3.2",
"org.opencontainers.image.version": "latest"
}
}
As shown above, manifests files are in JSON format and draw inspiration from the OCI spec (Open Container Initiative Specification) in cloud-native and container domains. The digest field in manifests corresponds to blobs.
Ollama Conversation Processing Flow

The general flow of a user conversation with Ollama is as follows:
-
User Initiates Conversation: The user executes the command
ollama run llama3.2via the CLI to start a conversation (llama3.2is an open-source large language model; you can also use other LLMs). -
Preparation Phase:
- The CLI client sends an HTTP request to
ollama-http-serverto fetch model information. The server attempts to read the localmanifestsmetadata files. If not found, it responds with a 404 not found. - When the model does not exist, the CLI client requests
ollama-http-serverto pull the model from the remote repository to the local storage. - The CLI client then requests the model information again.
- The CLI client sends an HTTP request to
-
Interactive Conversation Phase:
- The CLI sends an empty message
/api/generaterequest toollama-http-server, which internally handles some channel processing (using Go's channels). - If the model information contains
messages, they are printed. Users can save the current model and session conversation history as a new model, and the conversation history is saved asmessages. - The conversation officially begins: The CLI calls the
/api/chatinterface to requestollama-http-server, which relies on thellama.cppengine to load the model and perform inference (llama.cppalso serves as an HTTP server).ollama-http-serverfirst sends a/healthrequest tollama.cppto confirm its health status, then sends a/completionrequest to obtain the conversation response, and finally returns it to the CLI for display.
- The CLI sends an empty message
Through the above steps, Ollama completes the interaction between the user and the large language model.
Conclusion
Ollama integrates the llama.cpp inference engine and further encapsulates it, making complex LLM technology accessible. It provides developers and technical personnel with an efficient and flexible tool, effectively supporting large language model inference and interaction in various application scenarios.