arcee-ai/Trinity-Nano-Base

📊 Model Parameters

Total Parameters 6,120,003,328
Context Length 131,072
Hidden Size 1024
Layers 56
Attention Heads 8
KV Heads 2

💾 Memory Requirements

FP32 (Full) 22.80 GB
FP16 (Half) 11.40 GB
INT8 (Quantized) 5.70 GB
INT4 (Quantized) 2.85 GB

🔑 KV Cache (Inference)

Per Token (FP16) 57.34 KB
Max Context FP32 14.00 GB
Max Context FP16 7.00 GB
Max Context INT8 3.50 GB

⚙️ Model Configuration

Core Architecture

Vocabulary Size200,192
Hidden Size1,024
FFN Intermediate Size3,072
Number of Layers56
Attention Heads8
Head Dimension128
KV Heads2

Context & Position

Max Context Length131,072
RoPE Base Frequency10,000
Sliding Window Size2,048
Layer Attention Types[56 items]

Attention Configuration

Attention BiasNo
Attention Dropout0%
Tied EmbeddingsNo

Mixture of Experts

Expert FFN Size256
Experts per Token8
Number of Experts128
Expert Groups1
Groups per Token1

Activation & Normalization

Activation Functionsilu
RMSNorm Epsilon1e-05

Special Tokens

EOS Token IDNot set
Pad Token IDNot set
BOS Token IDNot set

Data Type

Model Dtypebfloat16
Layer Types:
Attention
MLP/FFN
Normalization
Embedding