← All Models
|
T5 - Google's Text-to-Text Transfer Transformer:
t5-small
t5-base
t5-large
google-t5/t5-base
📊 Model Parameters
Total Parameters
272,252,160
Context Length
2,048
Hidden Size
768
Layers
12
Attention Heads
12
KV Heads
12
💾 Memory Requirements
FP32 (Full)
1.01 GB
FP16 (Half)
519.3 MB
INT8 (Quantized)
259.6 MB
INT4 (Quantized)
129.8 MB
🔑 KV Cache (Inference)
Per Token (FP16)
36.86 KB
Max Context FP32
144.0 MB
Max Context FP16
72.0 MB
Max Context INT8
36.0 MB
⚙️ Model Configuration
Core Architecture
Vocabulary Size
32,128
FFN Intermediate Size
3,072
Number of Layers
12
Attention Heads
12
Context & Position
Max Context Length
512
Attention Configuration
Tied Embeddings
Yes
Activation & Normalization
RMSNorm Epsilon
1e-06
Activation Function
relu
Dropout (Training)
Hidden Dropout
10.0%
Special Tokens
BOS Token ID
Not set
Pad Token ID
0
EOS Token ID
1
Data Type
Model Dtype
Not set
Layer Types:
Attention
MLP/FFN
Normalization
Embedding
Attention
MLP
Norm
Embedding
Clear
Expand All
Collapse All