Kaggle competitions, the hardcore community that organises challenges for machine learning experts, brings out both the best and the worst in participants. It clearly prioritises application-based expertise over theoretical knowledge of the subject.
Register for this Free GenAI Workshop >
In a hot take, Jeremy Howard, the co-founder of Fast.ai, said that though PhD students can succeed at Kaggle competitions, it’s the ones who assume an easy win “generally fail miserably”.
This conversation was brought in as Thomas Wolf, the co-founder & CSO of Hugging Face, shared about a super-impressive AI competition, Artificial Intelligence Math Olympiad (AIMO), where AI models, not individuals, competed to solve challenging maths problems of the International Math Olympiad-level for a $1M prize money.
There was a super impressive AI competition that happened last week that many people missed in the noise of AI world. I happen to know several participants so let me tell you a bit of this story as a Sunday morning coffee time.
You probably know the Millennium Prize Problems… pic.twitter.com/LeWnVbjH8m— Thomas Wolf (@Thom_Wolf) July 7, 2024
A team from Numina, collaborating with Hugging Face, trained models using high-quality Chain-of-Thought (CoT) data and innovative methods. Despite initial performance challenges, they significantly improved their models, eventually winning the competition by solving 29 challenges.
This showcases AI’s potential in advancing complex mathematical problem-solving, hinting at a broader impact on scientific progress in the near future. As Terence Tao, regarded as of one the greatest living mathematicians himself set it up, this is “higher than expected”. Sometimes, better than PhD researchers.
This is something that has been agreed upon by several researchers in the field. Max Mynter, a self-taught ML engineer, replied to Howard saying that he had tried his hand at a few competitions and was utterly humbled. “Was a slap in the face that made me learn (again) that there is a difference between theoretical knowledge and practical skill,” said Mynter.
This led Mynter to gain more expertise and experience in the field by putting in the work required and “getting off the high horse”. “I was pretty good at walking through institutions and talking the talk of what prestigious institutions wanted to hear. But after a couple of years, I feel that I am building real skill.”
Similar thoughts were shared by others. “19 yo me pursuing bachelor’s degree thought I could win ‘few’ competitions easily because I self learnt ML and I was so good at it. By the time I finally won one, 2 years later, I realised how underskilled I am.”
Not everyone agrees though. Venu Vasudevan, the director of AI and founding team member of Zigbee, said, “Kaggle is the Rube Goldberg of CS. Not clear it represents street smart problem solving.” He added that he had participated in one of the competitions and came in the 5th percentile. He decided it wasn’t his cup of tea.
Kaggle’s Reality Check
For some people, even getting the data for the Kaggle competition is an unexpectedly difficult task. It is crucial for academic people to get into Kaggle competitions to learn what the real field is working on.
Jean-Francois Puget from NVIDIA, who is also a 3X Kaggle Grandmaster, believes that it should be mandatory for an academic working in machine learning to enter a Kaggle competition.
There is another interesting tale which narrates that Kaggle can land you jobs, not by winning them, but also learning from being a part of them. Yann LeCun shared that Jure Zbontar, a computer scientist from the University of Ljubljana, joined Meta’s FAIR team and worked on various projects with him.
LeCun said that Zbontar’s advisor emailed him about how he kept winning Kaggle competitions and asked if he would co-advice Zbontar. He then joined NYU with LeCun and completed his PhD and topped the leaderboard of the stereo vision benchmark.
Though it is not advisable for people to completely depend on Kaggle prizes for survival. It is also quite interesting to see that most of the top performers on Kaggle are either PhD or MS graduates in 2022. Kaggle definitely helps in getting people the skill they need for jobs.
When speaking with AIM in 2022, Ruchi Bhatia, the youngest 3X Kaggle Grandmaster, also advised people that getting into Kaggle is something necessary to “hone our competitive side, but at the same time not forget about the underlying goal: learning”.
As for the PhD students, it is definitely necessary for them to try their hands on practical skills instead of just focusing on theory.