Рубрика: AI

In the early days of the Internet, there were four ‘horsemen’ of the Internet

1. Cisco
2. EMC
3. Oracle and the now eclipsed
4. Sun Microsystems

With IBM’s 4.5 billion investment in Hugging face today, the generative AI landscape is becoming a bit clearer. There are four Generative AI leaders emerging – others lagging – and one unknown

Lets look at the four leaders of Generative AI as it stands today

1. Azure OpenAI
2. Meta/facebook with Llama
3. IBM Hugging face and
4. Databricks with its acquisition of Mosaic

Not in this list of leaders are

1. Google – still a bit unclear. We are fans of Anthropic though in which Google has invested
2. AWS – still a niche strategy
3. Tesla – experiencing a bit of a Transformers moment 🙂 having contributed to OpenAI – now not a leader
4. Oracle – unclear as of now

and the big unknown – Apple 🙂 no real idea / plan etc

There are really two implications of this

a) The battleground is in the Enterprise – although consumer gets a lot of attention in the media and

b) The talent race for AI will be hot!

We live in interesting times for sure 🙂

Image source: https://pixabay.com/photos/horse-horse-race-jockey-7863936/

An Overnight Sensation, 30 Years in the Making: The Parallel Paths of AI and Quantum Computing August 29, 2023 by Yuval Boger

In the world of technology, overnight sensations are rarely born overnight. They are the result of decades of research, development, and perseverance. The recent success of chatGPT, a state-of-the-art language model, is a prime example of this phenomenon. It has turned heads with its capabilities, but the road to this breakthrough was long and winding. This story of gradual progress leading to sudden recognition has a striking parallel in the field of quantum computing, a technology that could soon have its own "GPT moment."

The Rise of AI: A Slow Burn to Success

The development of AI, particularly deep learning models like ChatGPT, has been a journey decades in the making. The development of ChatGPT has roots in the early artificial neural networks of the 1950s and 1960s. The rise of deep learning in the 1980s and 1990s, including the creation of backpropagation algorithms, set the stage for more advanced models.

Before OpenAI's formation in 2015, significant progress in natural language processing laid the groundwork for sophisticated language models. Innovations like Word2Vec and attention mechanisms were pivotal.

OpenAI's GPT series, starting with the first GPT model in 2018, built upon these foundations. Subsequent iterations like GPT-2 and GPT-3 expanded the architecture, leading to the current state-of-the-art ChatGPT.

Quantum Computing: Getting Closer to a Breakthrough

Quantum computing is getting closer to a significant breakthrough, as the progress made in recent years is remarkable.

Some of the things that quantum computers can do today seemed unimaginable just a few short years ago. Quantum computers are still in their early stages of development, but they have already achieved some remarkable things that were unimaginable a few years ago. For example:

• In 2019, Google announced that its Sycamore quantum computer had solved a problem that would have taken a classical computer 10,000 years to solve. This was a major milestone for quantum computing, and it showed that quantum computers can solve problems that are far beyond the reach of classical computers.
• In 2021, researchers at the University of Science and Technology of China used a quantum computer to simulate the behavior of a molecule of hydrogen fluoride. This was the first time that a quantum computer had been used to simulate a molecule with more than two atoms.
• In 2022, researchers at the University of Toronto used a quantum computer to develop a new algorithm for machine learning. This algorithm is more efficient than traditional machine learning algorithms, and it could revolutionize the way that we use machine learning to solve problems.

The trajectory of quantum computing shares a similar path with AI, and the field is ripe for innovation and discovery. Just as ChatGPT revolutionized natural language processing, a breakthrough in quantum computing could redefine the landscape of computation. The journey has been long, but the quantum "GPT moment" may be closer than we think, and the potential impact could be profound.

The Panic of Unpreparedness

The sudden success of ChatGPT caught many organizations off guard. They found themselves unprepared, lacking the relevant people, processes, and expertise to leverage this new technology.
A similar scenario could unfold with quantum computing. When the quantum GPT moment arrives, organizations may panic, realizing they lack the infrastructure and expertise to harness the power of quantum technology.

Preparing for the Quantum Future

So, what should companies do to avoid being caught off guard? Here's a roadmap:

• Educate and Train: Invest in education and training to build a workforce skilled in quantum computing. This includes not only technical training but also fostering a culture of continuous learning.
• Collaborate and Partner: Collaborate with academia, research institutions, and industry leaders. Partnerships can accelerate innovation and provide access to cutting-edge technology.
• Invest in Research and Development: Dedicate resources to research and development. Experimentation and innovation are key to staying ahead of the curve.
• Develop a Strategic Roadmap: Create a clear and strategic roadmap for integrating quantum computing into the business. This includes understanding the potential applications and aligning them with business goals.
• Embrace Ethical Considerations: Quantum computing, like AI, raises ethical considerations. Organizations must be mindful of the social and ethical implications of the technology.

Conclusion

The story of ChatGPT's success is a reminder that "overnight sensations" are often the result of years of hard work and dedication. It's a lesson that applies equally to quantum computing.

As we stand on the cusp of a quantum revolution, organizations must take proactive steps to prepare. The quantum GPT moment may be closer than we think, and those who are prepared will be best positioned to seize the opportunities it presents.

In the words of the great scientist Isaac Newton, "If I have seen further, it is by standing on the shoulders of giants." The giants of AI have paved the way, and the giants of quantum computing are poised to take the next leap. The future is bright, and the possibilities are limitless. But preparation, foresight, and a commitment to innovation will be key to unlocking the potential of this exciting frontier.

About the Author

Yuval is focused on quantum computing and its broad societal impact. In his career, he has served as CEO and CMO of frontier-tech companies in markets including quantum computing software, wireless power, and virtual reality. His "Superposition Guy's Podcast" hosts CEOs and other thought leaders in quantum computing, quantum sensing, and quantum communications to discuss business and technical aspects that impact the quantum ecosystem.

Related

Exactly three months after OpenAI’s ChatGPT made a grand entry, Google introduced Bard. Since then, Google has been riding the generative AI wave with new features across various domains including Vertex AI, Model Garden and more. Over 70% of AI unicorns operate on Google Cloud infrastructure, exemplified by companies like Anthropic, AI21 Labs, and Character. AI, leveraging partnerships for global service delivery.

After Google I/O Connect, the big tech is back with Google Cloud Next, scheduled to be held in San Francisco from August 29th to 31st. Let’s take a look at major announcements and upgrades made at the event.

Vertex AI Gets Bigger & Better

Unveiled at Google I/O 2021, Vertex AI, the cornerstone suite of generative AI products for Google, gets new upgrades at Next. Now their proprietary LLM PaLM has four times more input token length and is available in 38 languages. Their AI-powered coding bot Codey’s new version has up to 25% quality improvement in code generation. Text-to-image diffusion model Imagen also has new updates with higher image quality, with a new feature called Style Tuning. It allows customers to align generated images with their brand guidelines using reference images.

Additionally, Meta’s Llama 2, Anthropic’s Claude 2, and TII’s Falcon join Model Garden, expanding model variety. Vertex AI Extensions aid model-data connectivity, Data Connectors grant read-only access to Google and third-party data, Vertex AI Search brings Google Search to business data, and Vertex AI Conversation enables easy chatbot deployment. Additionally, Colab Enterprise combines user-friendly Colab notebooks with enterprise-level security. Vertex AI Search is now generally available.

Duet AI for All

Good news for all as Duet AI in Google Workspace is now publicly available.

Now in Google Meet, Duet AI can make your appearance better and has improved sound quality. It will also take notes, and can even attend meetings on your behalf. Google Chat’s Duet AI answers queries, summarises content, and aids in recapping conversations, now featuring an improved UI and connectivity options.

Besides Workspace, Duet AI in Google Cloud is expanding its capabilities for various user groups, including developers, data practitioners, and cybersecurity professionals. It is being integrated into products like BigQuery, Looker, and the Database Migration Service to offer contextual assistance, auto-suggestions, and collaboration features. The launch of BigQuery Studio provides an all-in-one interface for data engineering, analytics, and machine learning. Moreover, Google Cloud is facilitating the integration of Vertex AI models into BigQuery and unveiling AlloyDB Omni, a downloadable edition that runs on multiple platforms.

Google is also introducing AlloyDB AI, enabling easy construction of enterprise AI applications, and announcing Cloud Spanner Data Boost for efficient operational data processing. Developer tools like Jump Start Solutions and Application Integration are becoming generally available, along with an extended partnership with GitLab for secure DevOps solutions. This expanded suite of Duet AI tools includes Cloud Workstations and Application Integration support.

Advancements in Google’s Supercomputing & Cloud Services

Bringing back the focus on supercomputing, Google revealed that their Cloud A3 Supercomputer, based on NVIDIA H100 designed for demanding generative AI workloads and LLMs, will be available generally from next month. The speciality of A3 is it leverages Google infrastructure processing units, claiming to provide three times faster training and better networking than the previous generation. Alongside, the Cross-Cloud Network is also introduced which is a new platform ensuring secure inter-cloud application connectivity.

Plus, now with Google Distributed Cloud (GDC), customers can have the flexibility to deploy Al and data workloads using select Vertex Al services and AlloyDB Omni. Additionally, the premium GKE Enterprise edition offers multi-cluster horizontal scaling, meeting the needs of today’s rigorous AI and ML workloads.

Prioritising Sustainability

Amid updates focused on generative AI, Google Maps is introducing three Environment APIs – Solar, Air Quality, and Pollen – aimed at aiding businesses in reducing emissions and adjusting to environmental changes. Designed to cater to diverse health needs, it offers localised pollen count data, heatmap visuals, comprehensive plant allergen insights, and practical strategies for minimising pollen exposure.

The API computes daily pollen grain levels across a 1×1 km2 grid in over 65 countries, including a five-day forecast, encompassing three plant categories and 15 species. By integrating factors like land cover, climate data, and pollen production, the model generates dependable predictions of local pollen levels and associated risks.

Expanding Partnerships

Google Cloud is partnering with various customers and organisations, ranging from startups to enterprises, to enhance their generative AI. A121 Labs is integrating industry-specific generative AI into BigQuery, Bayer is using Vertex AI Search and Med-PaLM 2 for medical purposes and Culture Amp is using Vertex AI for employee feedback.

While MSCI is aiding investors with risk management, Runway is deploying AI for creative processes while Six Flags is implementing AI for park guidance. Google Cloud also expanded partnerships with SAP and Docusign for industry solutions and contract assistance. Furthermore, domain-specific datasets from partners like Acxiom, Bloomberg, CoreLogic, Equifax, NIQ, and Transunion are now available on Google Cloud for AI training.

Security Cloud

Google Cloud’s always-on AI collaborator, Duet AI, offers generative AI-powered support for cloud users, extending to cybersecurity professionals through Duet AI’s integration in three core solution realms: Mandiant Threat Intelligence for streamlined threat assessment, Chronicle Security Operations for swift threat discovery, and Security Command Center for risk comprehension and remediation suggestions. These features are in Preview and set to be widely available later this year, with Mandiant Hunt for Chronicle enabling proactive search for hidden attacks using frontline intel. Further details are available in the blog post draft.

The post Key Highlights of Google Cloud Next ‘23 appeared first on Analytics India Magazine.

Image by Author

In an era where technology is advancing at an unprecedented pace, Artificial Intelligence?—?or AI for friends 🤓 stands out as one of the most transformative forces.

From automating mundane tasks to predicting complex patterns, AI is reshaping industries and redefining possibilities.

And as we stand on this AI revolution, it’s imperative for us to understand its potential and integrate it into our daily workflow.

However… I know it can be overwhelming to get started with these new technologies.

So, if you are wondering how to get started with AI, especially with models like ChatGPT…

Today I am bringing a set of 7 projects to learn from scratch how to deal with it.

Let’s discover them all together! 👇🏻

1. Generate a language translator using the OpenAI API

LLMs present a wide variety of applications. And one of the most useful?—?and easiest?—?to apply is precisely its ability to translate from any language to any other one.

In the tutorial Building a Multilingual Translation Tool with OpenAI ChatGPT API by Kaushal Trivedi, readers are guided through creating an AI-driven translation application using OpenAI’s gpt-3.5-turbo model via its API.

Screenshot of the tutorial.

The process involves the following steps:

1. Setting up OpenAI API credentials.
2. Defining a translation function using Python and the OpenAI API.
3. Testing the function.
4. Creating a user interface with Python’s Tkinter library.
5. Testing the user interface.

The key lesson is the potential of the GPT-3.5 Chat API in building powerful AI-powered tools. In this case, used for creating a translation tool.

2. Use ChatGPT to build a sentiment analysis AI system for your business

Another common application for LLM is dealing with huge amounts of text. Imagine you run an e-commerce that receives thousands of comments every single day?—?you could take advantage of AI-powered tools to deal with them.

This is precisely what Courtlin Holt-Nguyen shows us throughout his tutorial Sentiment Analysis with ChatGPT, OpenAI, and Python?—?Use ChatGPT to build a sentiment analysis AI system for your business. He performs the whole tutorial on Google Colab and tries to emphasize the versatility of ChatGPT in handling various NLP tasks, the importance of structured data for effective analysis, and the capability of ChatGPT to reason and explain its responses.

Screenshot of the tutorial.

Here are the key steps:

1. Describes the dataset to be used. You can use his dataset or choose any other one you prefer.
2. Introduces the OpenAI API.
3. Installation of required libraries in Google Colab and starts using ChatGPT OpenAI API for Sentiment Analysis.
4. Specific applications of the GPT model dealing with reviews.

ChatGPT’s powerful AI capabilities can be harnessed for comprehensive sentiment analysis, summarization, and actionable insights from customer reviews.

3. Basic usage of LangChain and OpenAI

Last month I wrote an easy-to-follow basic introduction to LangChain called Transforming AI with LangChain: A Text Data Game Changer, a Python library designed to maximize the potential of Large Language Models for text data processing.

Screenshot of the tutorial

The versatility of LangChain when handling large text data and its capability to provide structured output has allowed it to become one of the most used Python libraries to deal with LLM and create real-live tools.

The tutorial explains two simple use cases of this library that can be applied in multiple applications.

1. Summarization:

• Short Text Summarization: Using LangChain and ChatGPT to summarize short texts.
• Long Text Summarization: Handling longer texts by splitting them into smaller chunks and summarizing each chunk.

1. Extraction:

• Extracting Specific Words: Identifying specific words within a text.
• Using LangChain’s Response Schema: Structuring the output from the LLM into a Python object.

LangChain offers a robust framework for text summarization and extraction, simplifying the process of natural language processing applications.

4. Automating PDF Interaction with LangChain and ChatGPT

Following the previous tutorial, there is a more advanced article that teaches how to ingest a PDF and interact with it using the GPT model of OpenAI.

Screenshot of the tutorial.

Lucas Soares shows us throughout his tutorial Automating PDF Interaction with LangChain and ChatGPT how to leverage ChatGPT and the LangChain framework to interact with PDFs. The process is divided into three main steps:

1. Loading the document.
2. Generating embeddings and vectorizing the content.
3. Querying the PDF for specific information.

This approach allows users to ask questions directly to a PDF, streamlining information retrieval. You can either follow his written article or watch his YouTube channel. Whatever you prefer!

The key lesson is the potential of AI in simplifying interactions with traditionally static documents, making data access more dynamic and intuitive.

5. Building up a Resume parser with ChatGPT

Reo Ogusu brings an easy-to-follow project to end up with a Resume Parser using the OpenAI API and LangChain. Throughout the tutorial Transforming Unstructured Documents to Standardized Formats with GPT: Building a Resume Parser he demonstrates how to transform unstructured documents, specifically resumes, into a standardized YAML format using GPT.

Screenshot of the tutorial

Here are the key steps:

1. Extract text from PDFs using the PyPDF2 library.
2. Utilize LangChain, a community-driven framework, to streamline the development of Language Model-powered applications.
3. Define a YAML template for structuring the resume data.
4. Call the OpenAI API using LangChain to instruct GPT to format the data according to the YAML template.

GPT proves to be a powerful tool for converting unstructured data into structured formats, offering the potential for various data conversion applications.

6. Generating a simple chatbot using OpenAI API

To generate a simple ChatBot we can follows Avra tutorial called How to build a Chatbot with ChatGPT API and a Conversational Memory in Python, where he explains how to build a chatbot implementation using the ChatGPT API and the GPT-3.5-Turbo model.

It integrates LangChain AI’s ConversationChain memory module and features a Streamlit front-end.

Screenshot of the tutorial.

The article emphasizes the importance of conversational memory in chatbots, highlighting that traditional chatbots, being stateless, lack the ability to remember past interactions.

By incorporating memory, chatbots can offer a more seamless and natural conversational experience, resembling human-like interactions.

The key takeaway is the significance of context retention in enhancing chatbot-human communication.

7. An End-to-End Data Science Project with ChatGPT

As a final project, I am bringing a really interesting data science tutorial that uses the ChatGPT interface directly.

Abid Ali Awan teaches us through his tutorial A Guide to Using ChatGPT For Data Science Projects on integrating ChatGPT into various stages of a data science project. It showcases the power of ChatGPT in the realm of data science.

From project planning and exploratory data analysis to feature engineering, model selection, and deployment, ChatGPT can assist in every step.

The end product?

A fully functional web app for loan approval classification!

Screenshot of the tutorial.

the tutorial covers:

1. Project Planning: Engaging with ChatGPT to outline the project.
2. Exploratory Data Analysis (EDA): Leveraging Python for data visualization and understanding.
3. Feature Engineering: Enhancing data by creating new features.
4. Preprocessing: Cleaning data, handling class imbalances, and scaling features.
5. Model Selection: Training various models and evaluating their performance.
6. Hyperparameter Tuning: Optimizing the chosen model.
7. Web App Creation: Designing a Gradio-based web app for the loan data classifier.
8. Deployment: Launching the app on Hugging Face Spaces.

The tutorial emphasizes the power of ChatGPT in automating and enhancing various data science tasks, especially in project planning and code generation.

The key takeaway is the synergy between AI tools like ChatGPT and human expertise, where both complement each other to achieve optimal results.

Concluding Thoughts

The set of projects described above is just the tip of the iceberg when it comes to the potential of ChatGPT.

The open-source community is actively working to develop new tools and improve existing ones that can help you craft anything you can think of. LangChain is only one of the many examples out there.

This is why whether you’re still a learner of ChatGPT or a senior pro, always remember that in the world of AI, the only limit is your imagination!

So, why wait?

Dive in, experiment, and let the world of generative AI models open doors to endless possibilities!
Josep Ferrer is an analytics engineer from Barcelona. He graduated in physics engineering and is currently working in the Data Science field applied to human mobility. He is a part-time content creator focused on data science and technology. You can contact him on LinkedIn, Twitter or Medium.

More On This Topic

• Beginner Friendly Python Projects That Are Fun!
• 20 Machine Learning Projects That Will Get You Hired
• How to get started managing data quality with SQL and scale
• What Are NVIDIA NGC Containers & How to Get Started Using Them
• Federated Learning: Collaborative Machine Learning with a Tutorial on How…
• Why Upskilling in Data Vis Matters (& How to Get Started)

30 years.

That’s how long KDnuggets has been running and providing data-related knowledge.

Over the past 30 years, KDnuggets has worked hard to take data professionals on a learning journey by keeping up with the new tools and technologies, and providing tutorials, resources, cheat sheets, and guides.

Our readers have allowed us to keep on for so long, and we hope to see you in another 30 years!

We wouldn’t have reached this milestone without you all! To show our appreciation, we are conducting a 30 for 30 book giveaway with O’Reilly, a company well-known for publishing high quality books, as well as for their online learning platform.

This September, KDnuggets and O’Reilly would like to help our readers get back to their self-development plans and learning journeys with 30 back-to-study books.

About the 30 for 30 Giveaway

For 30 days, KDnuggets and O’Reilly will select a winner each day, who will be able to choose any book they would like from the O’Reilly Website.

The books will be emailed to you in PDF format, to ensure everybody receives their book regardless of where they are in the world.

Every day from August 31 to September 29, KDnuggets will publish a poll on their LinkedIn group and company page, as well as Twitter:

• LinkedIn Page
• LinkedIn Data Science & Machine Learning Group
• Twitter

To ensure that we can reach everyone globally, the poll will be posted at different times.

To enter, simply respond to the poll. At the end of each day, our Community Manager will randomly select a winner from the list of daily participants and contact them via direct message.

Please check your direct messages, as a failure to reply within 48 hours will mean we will have to move on to the next selected winner.

At the end of the 30 days, the winners will receive their selected book via email.

How to Take Part

In order to take part in the KDnuggets and O’Reilly 30 for 30 giveaway, you will need to:

1. Be subscribed to the KDnuggets newsletter
2. Submit a response to our daily poll
3. If selected as a random daily winner, once our Community Manager reaches out to you, please provide them with the email you used to sign up for the KDnuggets newsletter and your selected book by O’Reilly

That’s it!

This is an exciting time for us at KDnuggets, and we want to share this moment with you.

Good luck and happy learning!

Happy 30th Birthday KDnuggets 🎉

More On This Topic

• Top KDnuggets tweets, Dec 2-8: How to do visualization using #Python from…
• KDnuggets™ News 20:n45, Dec 2: TabPy: Combining Python and Tableau;…
• KDnuggets Applied Data Science Survey
• KDnuggets News, September 14: Free Python for Data Science Course •…
• KDnuggets News, October 5: Top Free Git GUI Clients for Beginners • A Day…
• KDnuggets News, November 2: The Current State of Data Science Careers • 15…

The conversation around building AI in India is finally taking its shape. Sam Altman during his visit to India said that no one would be able to replicate what OpenAI has done with ChatGPT. To which, Tech Mahindra CEO, CP Gurnani said, “Challenge accepted.” It took close to two months to present their goals to the world and launched Project Indus, an indigenous LLM for speaking in many Indic languages.

Most recently, at Reliance’s 46th Annual General Meeting, Mukesh Ambani also announced his plans to build India-specific AI models. The Jio boss is probably one of the persons from India who has the capability of actually challenging OpenAI.

“India has scale. India has data. India has talent. But we also need AI-ready digital infrastructure that can handle AI’s immense computational demands,” said Ambani. Through Jio, Ambani was able to give broadband connectivity to everyone and definitely delivered the promise he made. Now with AI, is it really possible?

A little too late?

Last year, Reliance acquired a 25% stake in Silicon Valley-based Two Platforms, a company focused on advanced technological projects aimed at creating interactive and immersive experiences through AI interactions with a sum of $15 million. The company is also investing in developing up to 2,000 MW of AI-ready computing capacity.

When it comes to capital and investment, there is nothing that is going to stop Ambani from making a comparatively great model. For the talent, Reliance can hire people from across the globe, to some extent even existing leading companies if they wish to. If not that, Ambani can simply outsource it to some other company and build an AI model for India. (Reliance’s AI can also be called AI-JIO)

Arguably, it might be a little late to come up in the AI space. Even with Tech Mahindra’s Project Indus is still in the infancy stage. Moreover, for the project to be successful, Gurnani has said that it requires contributions from every Indian to build the dataset. An Indic language dataset would just be the first step towards building an AI model to compete with ChatGPT. Still the bid is on and the way from here is only upwards.

Emerging stronger

On similar lines, India’s ambition of becoming a semiconductor superpower is still in the making. The government has been investing thousands of crores for the projects, but the delays and inefficiencies in the process have resulted in the loss of one-and-a-half years. To make up for the loss, Semicon India 2023 has given a little hope with all the promising investments and partnerships that Vedanta has promised with a ‘world-class technology partner. Even AMD and Micron are planning to invest in India.

The recent success of Chandrayaan-3 mission has also brought India into the spotlight. Becoming the fourth country to land on the moon, and first to land on the south-pole has been a huge feat for the company. This has also caused countries like Japan to already announce their partnership for the future missions with ISRO.

Elon Musk’s Starlink is also expected to offer satellite services in India. The company has already been in talks with the Indian government for making the process faster. Musk also finds it “impressive” that many of the CEOs of one of the biggest companies in the world are also of Indian-origin.

Impressive

— Elon Musk (@elonmusk) August 26, 2023

Meanwhile, no one can deny the success of UPI. The whole world has been trying to replicate its success in as many ways as possible. Countries like France, Australia, Singapore, Saudi Arabia, Bhutan, Nepal, Oman, Sri Lanka, and Pakistan have already received the UPI technology from India.

So even though it might seem like the step towards building AI systems might be a little late for India, the country has already proven itself as one of the leading countries when it comes to technological advancements, and is striving to do even more. Possibly, AI could be the next step for the country. Further, the country can lead the way for quantum computing soon if IBM comes back to India.

The post India Catapults the AI Mission appeared first on Analytics India Magazine.

It won’t be wrong to say that at this moment Hugging Face is minting moolah. The leading open-source proponent is now valued at $4.5 billion after raising $235 million in round D funding from Google, Amazon, Nvidia, Intel, AMD, Salesforce and others.

With this recent round, Hugging Face has now raised a total of $395 million, CEO Clement Delangue revealed in an interview.

Hugging Face hosts more than 500,000 AI models and an extensive collection of 250,000 datasets. The platform allows users to freely share their models and datasets, fostering a spirit of collaboration and innovation within the AI community.

What Future Holds for Hugging Face

While Hugging Face’s open approach has garnered much appreciation, questions about its future direction have emerged.

As its influence continues to grow, concerns have risen regarding the platform’s possible transition from its original open and collaborative ethos to a more profit-driven model. This potential shift has raised discussions reminiscent of OpenAI’s journey.

The cautionary “enshittification cycle,” discussed on platforms like Hacker News, is worth considering—a pattern wherein a company’s user-centric services, initially supported by venture capital and investors, give way to business-oriented approaches and eventual monetisation that could diminish user satisfaction and engagement.

When asked during a recent interview about their plans to put the fresh infusion of capital to work, Hugging Face CEO, Clement Delangue said that they would use it to double down on hiring more people and on investments in open-source AI.

However, he also spoke about profit and validating the revenue generation from AI.

“I think we’ve validated last year that there was massive usage for generative AI. This year we are validating that there is massive revenue for generative AI…..we are on track to 5x our revenue this year with over 10,000 customers today,” Clem said in a recent interview. Seemingly indicating that we are in the second phase of the ‘enshittification cycle’.

Way Too Much Power

Over the years, Hugging Face has created a thriving ecosystem that caters to all participants involved. However, discussions have now moved to the potential consequences of Hugging Face’s growing dominance in model hosting and frameworks. Some stakeholders worry about the platform’s future control over these resources, which could stifle innovation and centralise power dynamics.

For instance, the recent licensing of their TGI library raised concerns.

The spectre of Hugging Face potentially becoming less community-friendly may lead to possible forks or alternatives emerging if necessary. And this is a pattern visible in other community-based platforms like Reddit which blocked access to its API to save its data.

The open-source community’s discussion has been filled with mixed views on the potential limitations and future restrictions on the platform. Some users speculate that venture capitalists and investors may eventually impose restrictions on the platform in a move to monetise, similar to Docker‘s monetisation model—the change in licensing for Hugging Face’s text generation inference library is an indicator.

Docker at a certain point sent an email titled ‘Sunsetting free team organisations’ to any Docker Hub user who had created an Open source ‘organisation’, telling them their account would be deleted unless they did not upgrade to a paid plan. The price went from 0 to 420 USD per year, and naturally, users were not pleased. Docker went from being the darling of the open-source community to being hated.

Unprecedented Growth

Despite these concerns, Hugging Face’s journey showcases its transformative growth since 2016. The balance between commercial success and a collaborative spirit remains an ongoing quest for the company.

The evolving landscape raises the question: Is Hugging Face treading a path similar to OpenAI’s, where the quest for profit could lead to a shift from open to closed source?

Indeed, Hugging Face’s trajectory evokes comparisons with OpenAI’s evolution. OpenAI began as an open-source initiative and later transitioned to a more controlled model. Hugging Face’s trajectory is at a crossroads, where it must decide how to harness its exponential growth while maintaining its community-driven ethos.

Striking Balance is the way to go

Striking the right balance will determine whether Hugging Face becomes the next OpenAI—or charts a unique path that marries profit and collaboration in a harmonious synthesis.

As Hugging Face ponders its future direction, the tech community watches with a mix of excitement and caution, hoping that the platform’s journey continues to be guided by principles that have fueled its rise.

Hugging Face is also going to do a whole lot of good with the investment and Clem recently announced a flurry of hiring initiatives on X, emphasising diversity and inviting individuals from non-traditional backgrounds to join the collaborative AI efforts of the company.

The move was applauded for tapping into underrepresented talent that may not follow conventional paths, challenging the notion of capability.

The company’s job openings cover a wide spectrum, ranging from Machine Learning Engineers to Technical Support Engineers, Account Executives, Cloud Machine Learning Engineers, and more. The positions are spread across various locations, including the US and Europe, and all offer remote work opportunities.

This marks Hugging Face’s commitment to fostering an inclusive environment and expanding its team with professionals from diverse backgrounds. The range of roles also highlights the company’s multidisciplinary approach to AI, spanning technical, customer support, and marketing domains.

Conclusively, the potential pivot towards closed-source profitability poses both challenges and opportunities, with implications that extend beyond the company itself, shaping the landscape of AI collaboration and innovation.

The post Is Hugging Face the Next OpenAI? appeared first on Analytics India Magazine.

Meta has been open sourcing almost all of their creations. Undoubtedly, it has been good for the company. Ever since the release of LLaMA, Meta has been touted as the open source champion. Now, with Llama 2 and Code Llama, the praise for the company from the developer ecosystem has been at an all time high. But it is quite interesting how Meta does not care about anything apart from gathering all the praises.

Recently, Jason Wei from OpenAI, posted on X with his alternate account that he overheard at a Meta GenAI social event that the company plans to build Llama 3 and 4, which is expected to be much more powerful than GPT-4. “Wow, if Llama-3 is as good as GPT-4, will you guys still open source it?,” someone asked. To which, the person from Meta AI said that it would be, adding, “Sorry, alignment people.”

Overheard at a Meta GenAI social:
"We have compute to train Llama 3 and 4. The plan is for Llama-3 to be as good as GPT-4."
"Wow, if Llama-3 is as good as GPT-4, will you guys still open source it?"
"Yeah we will. Sorry alignment people."

— jason (@agikoala) August 25, 2023

This remark definitely points towards the closed-door systems that OpenAI, Google, and many others have been building in a bid to make them more aligned. Whatever the case might be, Meta’s bid towards open sourcing one of the most “dooming” technologies of all time, as many put it, is a little concerning.

Is open source really that good?

Sam Altman has been asked several times about the “kill switch” that he keeps carrying in his blue backpack to put an end to OpenAI’s systems if they get rogue. In a recent interview, he laughingly said it was just a joke. But this still does not kill the fear that people have about AI systems getting out of control.

With AI systems getting better and better everyday, the conversation about OpenAI’s system going rogue has steered away. It has come to open source models. Meta wants to open source a GPT-5 level model. This means, as pointed out in an X discussion, that there would be no kill-switch for this. Which means that if some bad actor wants to use an open source model and weaponise it, there is no way to turn it off.

Moreover, all the research for AI safety can possibly become meaningless. Companies that have been trying to make AI systems aligned, honest, and ethical would have no say in what happens. Everyone would be able to fine-tune the open source models however they want to. Arguably, this might be a little more dangerous than just giving away your data to OpenAI through ChatGPT.

Meta’s love for open source isn’t quite clearly well founded on its own beliefs. There is no proof that the company had plans to open source LLaMA in the first place, it only happened when the model got leaked, and was hailed as a game changer by many developers.

Open source is still under control

Yann LeCun, the Meta AI chief, posted on X that, “Once AI systems become more intelligent than humans, humans will still be the apex species.” AI doomers still disagree with this statement. But even if humans remain the “apex species”, one thing is certain that OpenAI’s GPT is going to remain the “apex model” for a very long time. And OpenAI knows that.

The hype around open source models outperforming closed door ones would be nothing if every single model was not compared against GPT-3 and GPT-4.

Every model in the open source ecosystem compares itself against GPT’s capabilities, and on the HumanEval benchmark which has been created by OpenAI itself. Arguably, no one would even bat an eye on the open source models if they were not compared against GPT-3 and GPT-4 in terms of performance.

Furthermore, even if Meta decides to release an open source Llama 3, which would be on par with GPT-4 in terms of capabilities, it would still be measured on HumanEvals. Additionally, by the time this happens, OpenAI might have already released GPT-5 and created another evaluation benchmark for open source models. There is no way open source would be able to escape this.

Adding to all of this is the fact that OpenAI, partnered with Anthropic, Google, and Microsoft, launched Frontier Model Forum, for ensuring safe and responsible development of AI models. So if Llama and further models go rogue in the future, they can be pulled down from Hugging Face and GitHub in a moment.

In May, Meta was not invited to the White House for AI discussion and it is not part of this forum as well. The company is being left behind, maybe voluntarily, and is trying to build an open source league of its own, which interestingly is still being controlled by OpenAI and others. So Meta’s bid to be the good guy of AI through the open source community might not last for too long.

The post Meta Launches Open Source Models, OpenAI Controls Them appeared first on Analytics India Magazine.

A few days ago, fine-tuned Code Llama-based models WizardCoder 34B by Wizard LM and Phind were released. As of now it seems both of them are engaged in a heated argument over whether Phind used Wizard LM’s WizardCoder-style dataset to train their V1 model. However, Phind dismissed the claims, and the debate is still on!

Everybody is rigorously evaluating OpenAI’s HumanEvals, trying to beat GPT-4 on various tasks. Just two days after the launch of Code Llama, Wizard LM introduced WizardCoder 34B, a fine-tuned version based on Code Llama. The company proudly claimed that WizardCoder 34B performed even better than GPT-4, ChatGPT-3.5, and Claude-2 on HumanEval, with a pass rate of 73.2% on the first try.

It seemed like Wizard LM attempted to deceive developers by cleverly omitting the fact that it had compared the 73.2% score with the HumanEval rating of GPT-4’s March version, rather than the August version, where GPT-4 achieved an 82% which Wizard LM itself calculated. Notably, HumanEval results of GPT-4 and ChatGPT-3.5 are 67.0 and 48.1 respectively as per GPT4- Technical Report (2023/03/15) — this seems odd that HumanEval Of GPT-4 by OpenAI is lower than that of Wizard LM.

Amazing!
But shouldn't the 73.2 be compared against 82.0 then since both are from your runs?

— TDM (e/flλ) (@cto_junior) August 26, 2023

However, Wizard LM isn’t the only player in this race. Another startup, Phind, also claimed that their fine-tuned versions, CodeLlama-34B and CodeLlama-34B-Python, achieved pass rates of 67.6% and 69.5% on HumanEval, using their own Phind dataset. These numbers are almost equivalent to GPT-4’s.

Obsession with GPT-4

It clearly shows that the open source community considers GPT-4 to be the ultimate benchmark. Pick up any research paper based on LLMs by Meta, they compare their results with GPT-based models, particularly, OpenAI’s HumanEvals.

Ironically, Meta needs OpenAI and vice versa. In the paper ‘Code Llama: Open Foundation Models for Code’, the word ‘GPT’ was used 37 times, on the other hand OpenAI didn’t use the word ‘Meta’ or ‘LLaMA’ in their ‘GPT-4 Technical Report’. What would happen if the open-source community stopped comparing itself with closed source models? Apparently the evaluation metrics created by OpenAI give purpose to the existence of open source models, otherwise, it would be difficult to assess their performance and to assess where they stand.

In the Code Llama research paper, Meta did not utilize any evaluation metric of its own making. Besides HumanEval, the only other metric employed was MBPP(Mostly Basic Python Programming), which Google created.

Another important thing to note is that GPT-4 does more than just coding tasks. On the other hand, Meta is creating models meant for specific tasks, and they’re trying to surpass GPT-4 in those particular tasks.

If a model is designed specifically for coding, there’s a good chance it might outperform GPT-4. Phind’s performance is also pretty much the same as GPT-4’s on HumanEval.

Moreover, there’s a strong likelihood that Code Llama was trained using datasets generated by GPT-4. Otherwise, it would be quite challenging for an open-source model to come close to competing with GPT-4.

Is HumanEval enough?

A discussion has been going on Reddit whether HumanEval is a suitable parameter to measure efficiency of coding abilities of large language models. The thread says HumanEval solving 160 programming questions in Python is not everything one would expect from a code model and real-world usage of code models is not captured by a single number based on 160 programs.

The thread further said that factors like code explanation, docstring generation, code infilling, SO questions, writing tests, etc, are not captured by HumanEval.

One of the users of X expressed the same sentiment and said sadly real life performance is still way beyond GPT-4 for Python Code. “Tried different, real life examples for creating minimal flask microservices (which I test on a bunch of LLMs) and GPT-4 still outperforms all open-source LLMs,” he added praising GPT-4 capabilities on real world usage.

Interestingly, Can Xu, a senior researcher at Wizard LM replied and said that he will look into it and try to improve the model. “Thank you for pointing out the points of potential improvement, we will work on the real life examples soon,” Xu said.

In another conversation, an X user expressed that he finds these benchmarks for models tend to be poor metrics for measuring how well they perform in actual real-world workflows.

Phind cofounder Michael Royzen replied to it and said that it was an early experiment to reproduce (and exceed) the “Unnatural CodeLlama” results from the paper. He said more work will be done in the future to make these models production ready.

“In the future, we’ll have a Mixture of Experts of different Code Llama models and I think that those will be competitive in real-world workflows”, Royzen added optimistically.

While open-source models might not yet match GPT-4’s standards and are striving to catch up, it’s heartening to see that they’re openly discussing with the community and acknowledging their shortcomings. The discussion between Wizard LM and Phind on X is a good sign and it shows that the open source community is pretty much dedicated.

This transparency in the open source community is a positive step towards practicing ‘Responsible AI’. In contrast, OpenAI keeps its trade secrets hidden, leaving everyone guessing about their upcoming plans.

The post Code Llamas Fight Over GPT-4 appeared first on Analytics India Magazine.

Today, OpenAI released ChatGPT Enterprise, an enterprise-grade version of its popular generative AI chatbot. ChatGPT Enterprise has enhanced security and privacy meant for business use and unlimited access to a high-speed version of ChatGPT‘s underlying large language model GPT-4. It also includes the ability to process longer inputs, advanced data analysis capabilities and per-organization customization options.

Jump to:

• What is ChatGPT Enterprise?
• How can you get ChatGPT Enterprise?
• ChatGPT Enterprise’s place in the market

What is ChatGPT Enterprise?

ChatGPT Enterprise is a generative AI chatbot software platform. OpenAI sees it as an evolution of the type of chatbot assistant that is currently in vogue.

People from more than 80% of Fortune 500 companies have set up ChatGPT accounts, OpenAI said, according to its records of email address accounts with corporate domains. However, there has been some concern across industries that chatbots aren’t private because data from conversations held within the bots may be fed back into the massive pool of data the bots use for training. OpenAI intends to nip that concern in the bud by making sure ChatGPT Enterprise doesn’t use any conversations for training.

Enhanced security and admin options

Admins can access domain verification, single sign-on, usage insights and manage users through a dedicated console.

SEE: Salesforce released a public policy prohibiting inappropriate uses of its AI products. (TechRepublic)

Plus, ChatGPT Enterprise data is encrypted in transit and at rest and is SOC 2 compliant, which the consumer version is not. Usage and advanced data analysis are both unlimited in ChatGPT Enterprise, unlike the metered consumer version.

“With the integration of ChatGPT Enterprise, we’re aimed at achieving a new level of employee empowerment, enhancing both our team’s performance and the customer experience,” Sebastian Siemiatkowski, chief executive officer at fintech company Klarna, told OpenAI in a blog post.

Upcoming additions to ChatGPT Enterprise

OpenAI plans to add even more to ChatGPT Enterprise in the future. Those enhancements include:

• Custom connections to other enterprise applications.
• A ChatGPT Business variant for smaller organizations.
• Improved Advanced Data Analysis and browsing.
• Tools made specifically for data analysts, marketers, customer support and other roles.

How can you get ChatGPT Enterprise?

ChatGPT Enterprise is available now. To get it for your business, you’ll need to contact OpenAI’s sales team. There is a page with more information, but we were getting a 404 error trying to visit it at the time of this writing. Pricing is also unknown for now, although we reached out to OpenAI for more information about availability.

ChatGPT Enterprise’s place in the market

OpenAI has several rivals in the area of generative AI for business. Anthropic makes the AI assistant Claude, which doesn’t have an enterprise version with privacy guardrails comparable to ChatGPT Enterprise. Google plays in the same space with its Bard AI for Workspace products, which has an admin panel for enterprise use. Because Meta’s Llama 2 AI is open source, it can be integrated into enterprise use cases in a variety of ways, including inside the Microsoft Azure AI catalog.

There is a chance OpenAI might end up competing with itself because customers can get ChatGPT through Azure.

“Customers can choose which platform is right for their business,” an OpenAI spokesperson told Reuters. We have reached out to OpenAI for more information about possibly competing with themselves.

ChatGPT has been an expensive project, with some estimates putting the cost of running it at $700,000 per day. Adding ChatGPT Enterprise as a new way to bring in some cash may help offset that drain.

Subscribe to the Innovation Insider Newsletter

Catch up on the latest tech innovations that are changing the world, including IoT, 5G, the latest about phones, security, smart cities, AI, robotics, and more.

Delivered Tuesdays and Fridays Sign up today