arcee-ai/Trinity-Mini

📊 Model Parameters

Total Parameters 26,123,974,400
Context Length 131,072
Hidden Size 2048
Layers 32
Attention Heads 32
KV Heads 4

💾 Memory Requirements

FP32 (Full) 97.32 GB
FP16 (Half) 48.66 GB
INT8 (Quantized) 24.33 GB
INT4 (Quantized) 12.16 GB

🔑 KV Cache (Inference)

Per Token (FP16) 65.54 KB
Max Context FP32 16.00 GB
Max Context FP16 8.00 GB
Max Context INT8 4.00 GB

⚙️ Model Configuration

Core Architecture

Vocabulary Size200,192
Hidden Size2,048
FFN Intermediate Size6,144
Number of Layers32
Attention Heads32
Head Dimension128
KV Heads4

Context & Position

Max Context Length131,072
RoPE Base Frequency10,000
Sliding Window Size2,048
Layer Attention Types[32 items]

Attention Configuration

Attention BiasNo
Attention Dropout0%
Tied EmbeddingsNo

Mixture of Experts

Expert FFN Size1,024
Experts per Token8
Number of Experts128
Expert Groups1
Groups per Token1

Activation & Normalization

Activation Functionsilu
RMSNorm Epsilon1e-05

Special Tokens

EOS Token IDNot set
Pad Token IDNot set
BOS Token IDNot set

Data Type

Model Dtypebfloat16
Layer Types:
Attention
MLP/FFN
Normalization
Embedding