← All Models
|
Mistral - French AI startup known for efficient open models (founded 2023):
Mistral-7B-v0.1
Mistral-Nemo-Base-2407
Codestral-22B-v0.1
Mistral-Small-24B-Base-2501
Ministral-8B-Instruct-2410
Mistral-Large-Instruct-2411
Mixtral-8x7B-v0.1
Mixtral-8x22B-v0.1
Mamba-Codestral-7B-v0.1
Pixtral-12B-Base-2409
Voxtral-Mini-3B-2507
Voxtral-Small-24B-2507
Mistral-Small-3.2-24B-Instruct-2506
Ministral-3-3B-Base-2512
Ministral-3-8B-Base-2512
Ministral-3-14B-Base-2512
Mistral-Large-3-675B-Base-2512
Devstral-Small-2-24B-Instruct-2512
Devstral-2-123B-Instruct-2512
mistralai/Ministral-8B-Instruct-2410
📊 Model Parameters
Total Parameters
8,019,808,256
Context Length
32,768
Hidden Size
4096
Layers
36
Attention Heads
32
KV Heads
8
💾 Memory Requirements
FP32 (Full)
29.88 GB
FP16 (Half)
14.94 GB
INT8 (Quantized)
7.47 GB
INT4 (Quantized)
3.73 GB
🔑 KV Cache (Inference)
Per Token (FP16)
147.46 KB
Max Context FP32
9.00 GB
Max Context FP16
4.50 GB
Max Context INT8
2.25 GB
⚙️ Model Configuration
Core Architecture
Vocabulary Size
131,072
Hidden Size
4,096
FFN Intermediate Size
12,288
Number of Layers
36
Attention Heads
32
Head Dimension
128
KV Heads
8
Context & Position
Max Context Length
32,768
Sliding Window Size
32,768
RoPE Base Frequency
100000000.0
Layer Attention Types
[36 items]
Attention Configuration
Tied Embeddings
No
Attention Dropout
0%
Activation & Normalization
Activation Function
silu
RMSNorm Epsilon
1e-05
Special Tokens
BOS Token ID
1
Pad Token ID
Not set
EOS Token ID
2
Data Type
Model Dtype
bfloat16
Layer Types:
Attention
MLP/FFN
Normalization
Embedding
Attention
MLP
Norm
Embedding
Clear
Expand All
Collapse All