← All Models
|
Gemma - Google's lightweight open models built from Gemini research:
gemma-2b
gemma-7b
gemma-2-2b
gemma-2-9b
gemma-2-27b
gemma-3-270m
gemma-3-1b-pt
gemma-3-4b-pt
gemma-3-12b-pt
gemma-3-27b-pt
google/gemma-2b
📊 Model Parameters
Total Parameters
3,030,460,416
Context Length
8,192
Hidden Size
2048
Layers
18
Attention Heads
8
KV Heads
1
💾 Memory Requirements
FP32 (Full)
11.29 GB
FP16 (Half)
5.64 GB
INT8 (Quantized)
2.82 GB
INT4 (Quantized)
1.41 GB
🔑 KV Cache (Inference)
Per Token (FP16)
18.43 KB
Max Context FP32
288.0 MB
Max Context FP16
144.0 MB
Max Context INT8
72.0 MB
⚙️ Model Configuration
Core Architecture
Vocabulary Size
256,000
Hidden Size
2,048
FFN Intermediate Size
16,384
Number of Layers
18
Attention Heads
8
Head Dimension
256
KV Heads
1
Context & Position
Max Context Length
8,192
RoPE Base Frequency
10000.0
RoPE Scaling
Not set
Attention Configuration
Attention Bias
No
Attention Dropout
0%
Tied Embeddings
Yes
Activation & Normalization
Activation Function
gelu
RMSNorm Epsilon
1e-06
Special Tokens
BOS Token ID
2
Pad Token ID
0
EOS Token ID
1
Data Type
Model Dtype
bfloat16
Layer Types:
Attention
MLP/FFN
Normalization
Embedding
Attention
MLP
Norm
Embedding
Clear
Expand All
Collapse All