← All Models
|
T5 - Google's Text-to-Text Transfer Transformer:
t5-small
t5-base
t5-large
google-t5/t5-small
📊 Model Parameters
Total Parameters
93,405,696
Context Length
2,048
Hidden Size
512
Layers
6
Attention Heads
8
KV Heads
8
💾 Memory Requirements
FP32 (Full)
356.3 MB
FP16 (Half)
178.2 MB
INT8 (Quantized)
89.1 MB
INT4 (Quantized)
44.5 MB
🔑 KV Cache (Inference)
Per Token (FP16)
12.29 KB
Max Context FP32
48.0 MB
Max Context FP16
24.0 MB
Max Context INT8
12.0 MB
⚙️ Model Configuration
Core Architecture
Vocabulary Size
32,128
FFN Intermediate Size
2,048
Number of Layers
6
Attention Heads
8
Context & Position
Max Context Length
512
Attention Configuration
Tied Embeddings
Yes
Activation & Normalization
RMSNorm Epsilon
1e-06
Activation Function
relu
Dropout (Training)
Hidden Dropout
10.0%
Special Tokens
BOS Token ID
Not set
Pad Token ID
0
EOS Token ID
1
Data Type
Model Dtype
Not set
Layer Types:
Attention
MLP/FFN
Normalization
Embedding
Attention
MLP
Norm
Embedding
Clear
Expand All
Collapse All