Meta Unveils Open-Source Code Llama 70B to Challenge GitHub Copilot

By siliconindia  |  Tuesday, 30 January 2024, 10:37 IST

Meta Unveils Open-Source Code Llama 70B to Challenge GitHub Copilot
The highly advanced Code Llama, Meta's latest large language model, is now accessible for researchers and developers as a free-to-use download. With an impressive 70 billion parameters, this generative AI model was initially introduced in August 2023. “We’re open-sourcing a new and improved Code Llama, including a larger 70B parameter model. Writing and editing code has emerged as one of the most important uses of AI models today. The ability to code has also proven to be essential for AI models to process information in other domains more rigorously and logically. I’m proud of the progress here and look forward to including these advances in Llama 3 and future models as well", said Meta CEO Mark Zuckerberg.
According to Meta, Code Llama stands out as the most advanced and high-performing model within the Llama family. It is offered for free in three versions: CodeLlama – 70B, the foundational code model; CodeLlama – 70B – Python, a version specifically tailored for Python; and Code Llama – 70B – Instruct, a fine-tuned version incorporating human instruction and self-instruction code synthesis.
Code Llama is built on top of Llama and is capable of generating code. According to the company, the model has scored 67.8 on HumanEval, a generative AI benchmark, while the GPT-4 Turbo, a much bigger model, has scored 81.7. Meta also claims that Code Llama is tuned for code generation, and the best part is that it is an open-source model.
Meta's newest generative AI model possesses the capability to generate code in C++, Java, PHP, and Python. Trained on a vast dataset comprising 500 billion tokens of code and related information, the model showcases an expanded context window. It can now accommodate inputs with up to 100,000 tokens or approximately 75,000 words.
It's important to highlight that Code Llama is built upon Llama, a general-purpose AI model with 175 billion parameters. Users have the option to utilize Code Llama through platforms such as Hugging Face or PyTorch. Additionally, Meta has confirmed that certain enhancements introduced in Code Llama will be incorporated into the upcoming Llama 3 and other future models.

ON THE DECK