Install
Please confirm you are human
This browser or connection looks automated. Press and continuously hold the control for 3 seconds to enable Google-hosted web results and, when separately allowed, AI-assisted answers.
A successful check enables 100 search requests. Interactive access does not authorize scraping, systematic collection, or reuse of search output.
News
model
3+ hour, 21+ min ago (68+ words) Inference-only HY V4 model compatible with HuggingFace weights (NVIDIA). One HY V4 decoder layer: MLA attention plus a dense or MoE MLP. Whenconfig.enable_ihc is set the layer runs on hc_mult residual channels and each sub-block is wrapped by an HYV4HCLayer boundary; otherwise it uses…...
attention
3+ hour, 21+ min ago (434+ words) MLA attention and lightning indexer for HY V4 (NVIDIA). The per-head learnable sink is supported through.flashmla_sparse, which subclasses the platform's sparse MLA backend to forward attn_sink. The sink is applied by binding the sink-capable backend from.flashmla_sparse; if no backend on this platform…...
hy_v4_reasoning_parser
3+ hour, 21+ min ago (286+ words) Reasoning extraction for HYV4, on plain data. Count reasoning tokens (0 in no_think). The start token is injected at the END of the prompt, so it is normally absent from token_ids. Only skip a leading start token when it actually appears first (legacy / no…...
flashmla_sparse
3+ hour, 21+ min ago (254+ words) Sink-capable FlashMLA sparse backend for HY V4 (NVIDIA). HY V4 adds a per-head learnable attention sink on top of sparse MLA. The vendored FlashMLA kernels already accept an attn_sink argument, but vLLM's shared FLASHMLA_SPARSE backend neither advertises sink support nor forwards the tensor, so…...
LMCache Examples
3+ mon, 2+ week ago (81+ words) This folder demonstrates how to use LMCache with vLLM v1 for KV cache offloading, disaggregated prefilling, and KV cache sharing. LMCache integrates with vLLM v1 in two ways: This example demonstrates how to run LMCache with disaggregated prefill using NIXL on a…...
sharded_rdt_engine
6+ day, 22+ hour ago (1746+ words) Sharded Ray Direct Transport (RDT) weight transfer engine (consumer side). Pulls only the slice each vLLM worker consumes under tensor/expert parallelism, not the full HF-format tensor. One thing at four resolutions, over three lifetimes. FetchKey -- (name, op_chain), "which slice of…...
Sharded RDT Engine
6+ day, 22+ hour ago (259+ words) Discovery is expensive, so it happens once, at init_transfer_engine, as a dry run over model.load_weights with every parameter on meta. Nothing is transferred; the engine just records, per leaf module, which slice feeds which destination region. Every later sync is pure…...
glm52_low_latency_gemm
3+ week, 4+ day ago (26+ words) vLLM GLM-5.2 decode GEMM selection for unquantized BF16 on SM103. Plan for a weight the walk below cannot reach (a plain nn.Linear )....
glm_ocr
7+ mon, 3+ day ago (10+ words) vLLM Inference-only GLM-OCR model compatible with HuggingFace weights....
audio
2+ week, 1+ day ago (22+ words) vLLM _forward_speech_encoder(mel_features, input_seq_lens, audio_sample_lens) ¶ Run the eager speech encoder without server-side slicing/batching....