Tencent Hunyuan Hy4 LLM Now Open Source

0
31

Tencent’s Hunyuan team has officially announced the open-sourcing of its Hy4 large language model. The model weights are now available for developers to download and deploy across four major platforms: GitHub, Hugging Face, ModelScope, and GitCode. This move signifies the complete open-sourcing of the Hunyuan LLM series, which now includes versions like Hy3 preview, Hy3, Hy4 preview, Hy4, and Hy4 flash.

Hy4: A Powerful MoE Model

The officially released Hy4 model, first previewed on November 4th, utilizes a Mixture-of-Experts (MoE) architecture. It boasts a total of 770 billion parameters, with 49 billion activated parameters. A standout feature of Hy4 is its massive context window, which extends to over 1 million tokens, enabling it to process and understand exceptionally long pieces of text.

Hy4 Flash: Cost-Effective Performance

Alongside the flagship Hy4, Tencent is also making the Hy4 flash model available. Designed for cost-effectiveness, Hy4 flash features approximately 220 billion total parameters, including around 63.5 billion Engram parameters. Crucially, it activates only about 12 billion parameters per token, making it a more efficient option for large-scale deployments. Like its flagship counterpart, Hy4 flash also supports a context window of up to 1 million tokens and includes multimodal capabilities, accepting both text and image inputs. This dual release strategy provides a tiered approach, catering to both top-tier performance needs and scalable, cost-efficient implementations.

Licensing and Security

For developers and businesses, the Hy4 and Hy4 preview models are released under the Apache 2.0 license. This permissive license allows for commercial use and modifications, encouraging broad adoption and innovation. Tencent has also proactively released a Hy4 model System Card, which details security assessments across various dimensions, including network and biological security.

According to official statements, the training data for Hy4 was carefully curated to include only defensive cybersecurity content, such as Proof-of-Concept generation and vulnerability patching. Offensive data, including exploit development, reverse engineering, and penetration testing, was deliberately excluded. Audits conducted by Tencent’s Xuanwu Lab and Keen Security Lab reportedly found no significant risks related to offensive capabilities.

Accessibility and Resources

Developers interested in exploring or utilizing the Hy4 models can access them via the following links:

The open-source license can be found on Hugging Face: https://huggingface.co/tencent/Hy4/blob/main/LICENSE

The detailed model security assessment card is available at: https://hy.tencent.ai/Hy4_system_card.pdf

The complete open-sourcing of the Hy4 model marks a significant step for Tencent’s AI efforts, providing the developer community with powerful tools to build upon and advance the field of large language models.

Source: https://www.ithome.com/1/029/841.htm

LEAVE A REPLY

Please enter your comment!
Please enter your name here