Meituan LongCat has officially announced the open-sourcing of its native multimodal model, LongCat-2.5, under the MIT license. This move makes the powerful model available for commercial inference deployment and further training without requiring additional authorization.
Addressing Long-Context and Multimodal Challenges
LongCat-2.5 is specifically designed for long-context intelligent agent tasks. It boasts a massive total parameter count of 1.6 trillion, with approximately 48 billion parameters activated per token. A key highlight is its support for an impressive one million token context window, significantly enhancing its ability to process and understand extensive information.
The model demonstrates notable improvements in end-to-end delivery capabilities across various complex scenarios. These include developing intricate codebases, building complete projects, Visual Coding tasks, and professional office applications. This advancement is particularly significant for AI agents that need to maintain context and coherence over long interactions or large documents.
Innovative Architecture for Enhanced Visual Understanding
A significant innovation in LongCat-2.5 lies in its approach to visual representation. Instead of relying on separate visual modules, the model integrates visual signals directly into its core processing. It represents visual information as multi-level discrete tokens, processed uniformly by the main model alongside text tokens. This unified treatment allows for a deeper synergy between visual understanding and code generation.
Specifically, the model achieves this by applying residual vector quantization to native resolution visual features, progressively enriching details through successive layers. These visual tokens can be pre-calculated and cached offline, which reduces the computational load during online processing and balances the workload for distributed training. Furthermore, the visual codebook and the main backbone are trained jointly. This shared parameter approach within the same semantic space enables collaborative processing of visual comprehension and code generation, leading to more cohesive and contextually relevant outputs.
Optimizations for Diverse Hardware and Efficient Inference
Recognizing the importance of accessibility and performance on various hardware, the Meituan LongCat team has implemented deep synergistic optimizations. These efforts specifically target model architecture, chip adaptation, and deployment strategies, with a focus on supporting domestic computing chips that may have limitations in memory and bandwidth.
For inference, LongCat-2.5 utilizes an efficient W4A4 quantization scheme. This method quantizes both expert weights and activations to 4-bit precision, allowing computations to be performed using native INT4 matrix multiplication units. This approach significantly reduces memory footprint and computational overhead, making high-performance AI accessible on a wider range of hardware.
Empowering Developers and Researchers
The open-sourcing of LongCat-2.5 is a significant contribution to the AI community. It provides developers and researchers with a state-of-the-art model capable of handling complex, long-context, and multimodal tasks. The availability under the MIT license ensures broad adoption and fosters further innovation in the field of intelligent agents and AI-powered applications.
Interested parties can explore the model further through the following resources:
- Model on Hugging Face: https://huggingface.co/meituan-longcat/LongCat-2.5
- Technical Blog: https://longcat.ai/blog/longcat-2.5/
- API Platform: https://longcat.chat/platform/product
