It’s hard to imagine Meta without open source AI, given its significant contribution to the ecosystem. Fueling several innovations, it has also in many ways been helping the likes of OpenAI, which have been syphoning off of open source contributions without batting an eyelid.
“OpenAI does not have a monopoly on good ideas. They’re not going to get to AGI by themselves, in-fact they’re using PyTorch and Transformers, which were published by many of us. They’re profiting from the open research landscape,” said Yann LeCun, not mincing words during a roundtable discussion at Davos.
Transformers was first proposed by Google in 2017 in its paper ‘Attention is all you Need’.
Recently, OpenAI released new embedding models and API updates created by Indian developers Aditya Kusupati, a researcher at Google, and Prateek Jain, a senior staff research scientist at Google, two years ago.
Meta is shamelessly open source
Till date, Meta has over 600 open source projects and counting. The company’s influence in the AI landscape is essentially in diverting the larger community from having to pay its competitors. Further, the collaboration with the ecosystem is necessary to set the industry standards in finding use cases with the AI models.
Clem Delangue, founder of Hugging Face, posted on X that Meta has the highest number of open source models on the platform with 689 models, a number which has grown since then.
In this week’s earnings call Zuckerberg addressed how open sourcing benefits Meta saying, “The short version is that open sourcing improves our models, and because there’s still significant work to turn our models into products, we find that there are mostly advantages to being the open source leader.”
Meta is “playing to win,” added Zuckerberg, pointing out that training and operating future models will be even more compute intensive. Its aspiration of building a full general intelligence would require many years of dedicated research and development.
Zuckerberg in a conversation with Lex Fridman said that Meta’s bet is on the open source community as it believes instead of hopping onto the wagon of claiming super intelligent AI, it would be beneficial to let the community use it for research purposes, and building more efficient and aligned AI.
Meta is clearly the winner of the open source AI race, and will continue to be as long as it continues its ways. Llama 2 currently has around 4 million downloads on Hugging Face, and the number is only going to increase when Llama 3 comes out soon.
Open source AGI?
After feeding off of the open source contributions, OpenAI had raised alarm about open source and its risks. In August, Jan Leike, ML researcher and alignment team lead at OpenAI, painted a doomsday picture, saying, “An important test for humanity will be whether we can collectively decide not to open source LLMs that can reliably survive and spread on their own.”
Although the initial reason Zuckerberg gave for their decision to open source its models was, “It drives innovation because it enables many more developers to build with new technology,” it is clear that Meta is benefiting in more than just goodwill from the developer community.
“Zuck and Yann LeCun will go down as heroes in human history! Fighting for ‘Open AI’ when the incumbents sought to shut it down! Unbelievable how much the vibes from Meta have changed over the last year. That, and maybe it’s time for a name change – Meta to OpenAI,” wrote Bindu Reddy, Chief at Abacus.ai, on X.
Likewise, Perplexity chief Arvind Srinivas said,” Open Source AGI is an amazing vision. You (Meta) are building a very powerful technology, and actually aligning to what makes sense for the world: more people have a say in what makes sense and doesn’t”.
Not for long
Meta now faces stiff competition from Chinese companies like Alibaba and Tencent who are continuously releasing open source models. Alibaba released Qwen-7B and Qwen-7B-Chat, each with 7 billion parameters, becoming the first major Chinese tech company to open-source LLMs. These models, part of the Tongyi Qianwen series, aim to help businesses adopt AI.
DeepSeek, a 67 billion parameter model outperformed Llama 2, Claude-2, and Grok-1 on various metrics. The best part is that the model from China is open sourced, and uses the same architecture as LLaMA.
Another hint of China’s open source AI dominance is the Yi-34B model released by 01.AI startup, reaching a unicorn status after the release. The AI startup by Kai-Fu Lee is developing AI systems for the Chinese market. The interesting part is that the second and third models on the Open LLM Leaderboard are also models based on Yi-34B.
As long as Chinese giants open source their models, they are not a threat to anyone, except Meta, and its AGI goals. On the other hand, Meta can also benefit from the success of Chinese open source models, just like it did with TikTok and made Reels on Instagram.
The post Where Would Meta Be without Open Source AI? appeared first on Analytics India Magazine.
