← All Models | Other Chinese/Asian AI Labs:

meituan-longcat/LongCat-Flash-Thinking-2601

📊 Model Parameters

Total Parameters 0
Context Length 983,040
Hidden Size 6144
Layers 0
Attention Heads 64
KV Heads 64

💾 Memory Requirements

FP32 (Full) 0.0 MB
FP16 (Half) 0.0 MB
INT8 (Quantized) 0.0 MB
INT4 (Quantized) 0.0 MB

🔑 KV Cache (Inference)

Per Token (FP16) 0 B
Max Context FP32 0.0 MB
Max Context FP16 0.0 MB
Max Context INT8 0.0 MB

⚙️ Model Configuration

Core Architecture

Vocabulary Size131,072
Hidden Size6,144
FFN Intermediate Size12,288
Number of Layers28
Attention Heads64
Head Dimension64
KV Heads64

Context & Position

Max Context Length983,040
RoPE Base Frequency1000000.0

Attention Configuration

Attention BiasNo
Attention Dropout0%
Tied EmbeddingsNo

Multi-Head Latent Attention

KV LoRA Rank512
Query LoRA Rank1,536
QK RoPE Head Dimension64
Value Head Dimension128
QK Non-RoPE Head Dimension128

Mixture of Experts

Expert FFN Size2,048
Number of Experts512
Routing Scale Factor6.0
Experts per Token12
Normalize TopK ProbabilitiesNo

Activation & Normalization

Activation Functionsilu
RMSNorm Epsilon1e-05

Special Tokens

Pad Token IDNot set
BOS Token ID1
EOS Token ID2

Data Type

Model DtypeNot set
Layer Types:
Attention
MLP/FFN
Normalization
Embedding