Рубрика: AI

Meta built a code-generating AI model similar to Copilot Kyle Wiggers 9 hours

Meta says it’s created a generative AI tool for coding similar to GitHub’s Copilot.

The company made the announcement at an event focused on its AI infrastructure efforts, including custom chips Meta’s building to accelerate the training of generative AI models. The coding tool, called CodeCompose, isn’t available publicly — at least not yet. But Meta says its teams use it internally to get code suggestions for Python and other languages as they type in IDEs like VS Code.

“The underlying model is built on top of public research from [Meta] that we have tuned for our internal use cases and codebases,” Michael Bolin, a software engineer at Meta, said in a prerecorded video. “On the product side, we’re able to integrate CodeCompose into any surface where our developers or data scientists work with code.”

The largest of several CodeCompose models Meta trained has 6.7 billion parameters, a little over half the number of parameters in the model on which Copilot is based. Parameters are the parts of the model learned from historical training data and essentially define the skill of the model on a problem, such as generating text.

CodeCompose was fine-tuned on Meta’s first-party code, including internal libraries and frameworks written in Hack, a Meta-developed programming language, so it can incorporate those into its programming suggestions. And its base training data set was filtered of poor coding practices and errors, like deprecated APIs, to reduce the chance that the model recommends a problematic slice of code.

Meta’s CodeCompose tool, powered by AI.

In practice, CodeCompose makes suggestions like annotations and import statements as a user types. The system can complete single lines of code or multiple lines, optionally filling in entire large chunks of code.

“CodeCompose can take advantage of the surrounding code to provide better suggestions,” Bolin continued. “It can also uses code comments as a signal in generating code.”

Meta claims that thousands of employees are accepting suggestions from CodeCompose every week and that the acceptance rate is over 20%.

The company didn’t address, however, the controversies around code-generating AI.

Microsoft, GitHub and OpenAI are being sued in a class action lawsuit that accuses them of violating copyright law by allowing Copilot to regurgitate sections of licensed code without providing credit. Liability aside, some legal experts have suggested that AI like Copilot could put companies at risk if they were to unwittingly incorporate copyrighted suggestions from the tool into their production software.

It’s unclear whether CodeCompose, too, was trained on licensed or copyrighted code — even accidentally. When reached for comment, a Meta spokesperson had this to say:

“CodeCompose was trained on InCoder, which was released by Meta’s AI research division. In a paper detailing InCoder, we note that, to train InCoder, ‘We collect a corpus of (1) public code with permissive, non-copyleft, open source licenses from GitHub and GitLab and (2) StackOverflow questions, answers and comments.’ The only additional training we do for CodeCompose is on Meta’s internal code.”

Generative coding tools can also introduce insecure code. According to a recent study out of Stanford, software engineers who use code-generating AI systems are more likely to cause security vulnerabilities in the apps they develop. While the study didn’t look at CodeCompose specifically, it stands to reason that developers who use it would fall victim to the same.

Bolin stressed that developers needn’t follow CodeCompose’s suggestions and that security was a “major consideration” in creating the model. “We are extremely excited with our progress on CodeCompose to date, and we believe that our developers are best served by bringing this work in house,” he added

KPI Partners, a leading global provider of Analytics and Digital Transformation solutions, announced today its achievement of Microsoft Azure Solution Partner status in Data & AI. With expertise in leveraging Azure Synapse Analytics, Azure Data Lake, Azure Data Factory, and Azure Databricks, KPI Partners excels in designing and implementing customized Microsoft Analytics solutions.

The company’s proficiency enables the construction of robust analytics and AI solutions tailored to meet the unique requirements of each client, offering streamlined data management processes. Recognizing KPI Partners’ competence in workload analysis, schema modeling, and ETL operations for data migration to cloud-based data warehouses, Microsoft acknowledges their designation as a Solution Partner.

Through their Azure Center of Excellence, KPI Partners has strengthened their data management and governance capabilities, facilitating the development of cutting-edge analytics using advanced technologies such as Generative AI, ChatGPT, and AI solutions on the Azure Cloud Platform. This new capability empowers businesses to transform data into a competitive advantage, driving accelerated ROI and value realization while minimizing implementation costs.

Sid Goel, CTO & Partner at KPI Partners, emphasized, “Becoming a Solutions Partner for Data and AI (Azure) is a testament to KPI Partners’ unwavering commitment to technical expertise and performance. Our solutions are designed to drive customer success, positioning us as the trusted partner of choice for those seeking top-notch Data and AI solutions. Our expertise and innovation have enabled us to provide unparalleled support to global customers across various sectors, including high technology, manufacturing, healthcare, financial services, energy, and others. By partnering with us, clients can align their long-term strategic goals for greater profitability and growth.”

The post KPI Partners Attains Microsoft Azure Solution Partner Status in Data & AI appeared first on Analytics India Magazine.

Sponsored Post

Machine learning system design is complex. The successful ML engineer needs to navigate a multistep process that demands skills from many different fields and roles.

Machine Learning System Design: With end-to-end examples is a practical guide for planning and designing successful ML applications. It lays out a clear, repeatable framework for building, maintaining, and improving systems at any scale.

Authors Arseny Kravchenko and Valeri Babushkin have filled this unique handbook with campfire stories and personal tips from their own extensive careers. You’ll learn directly from their experience as you consider every facet of a machine learning system, from requirements gathering and data sourcing to deployment and management of the finished system.

"This is not about MLOps! This book is about a more important question: how to make sure that an ML project does not end up on the shelf?"

—Boris Tseytlin, Senior Machine Learning Engineer, Planet Farms

In the book you’ll follow two example companies each building a new ML system, exploring how their needs are expressed in design documents and learning best practices by writing your own. Along the way, you’ll learn how to ace ML system design interviews, even at highly competitive FAANG-like companies, and improve existing ML systems by identifying bottlenecks and optimizing system performance.

Machine Learning System Design is available from its publisher Manning Publications.

Our 35% discount code (good for all our products in all formats): nlkdnuggets21

One free eBook code for Machine Learning System Design: mlstds-AEBF

More On This Topic

• Machine Learning Systems Design: A Free Stanford Course
• Toloka 101 Live Demo: Learn how to get reliable training data for machine…
• Effective Testing for Machine Learning
• Design Patterns in Machine Learning for MLOps
• Top KDnuggets tweets, Sep 02-08: Training alone is never enough to generate…
• Design patterns in machine learning

Image by Author

It cannot be emphasized enough how crucial data is in making informed decisions.In today’s world, businesses rely on data to drive their strategies, optimize their operations, and gain a competitive edge.

However, as the volume of data grows exponentially, organizations or even developers in personal projects might face the challenge of efficiently scaling their data science projects to handle this deluge of information.

To address this issue, we will discuss five key components that contribute to the successful scaling of data science projects:

1. Data Collection using APIs
2. Data Storage in the Cloud
3. Data Cleaning and Preprocessing
4. Automation with Airflow
5. Power of Data Visualization

These components are crucial in ensuring that businesses collect more data, and store it securely in the cloud for easy access, clean and process data using pre-written scripts, automate processes, and harness the power of data visualization through interactive dashboards connected to cloud-based storage.

Simply, these are the methods that we will cover in this article to scale your data science projects.

But to understand its importance, let’s take a look at, how you might scale your projects before cloud computing.

Before Cloud Computing

Image by Author

Before cloud computing, businesses had to rely on local servers to store and manage their data.

Data scientists had to move data from central servers to their systems for analysis, which was a time-consuming and complex process. Setting up and maintaining on-premise servers, can be highly costly and requires ongoing maintenance and backups.

Cloud computing has revolutionized the way businesses handle data by eliminating the need for physical servers and providing scalable resources on demand.

Now, let’s get started with Data Collection, to scale your data science projects.

Image by Author Data Collection using API
Image by Author

In every data project the first stage will be data collection.

Feeding your project and model with constant, up-to-date data is crucial for increasing your model's performance and ensuring its relevance.

One of the most efficient ways to collect data is through API, which allows you to programmatically access and retrieve data from various sources.

APIs have become a popular method for data collection due to their ability to provide data from numerous sources including social media platforms or financial institutions and other web services.

Let’s cover different use cases to see how this can be done.

Youtube API

In this video, coding was done on Google Colab and testing was conducted using the Requests Library.

The YouTube API was used to retrieve data, and the response from making an API call was obtained.

The data was found to be stored in the 'items' key.

The data was parsed through, and a loop was created to go through the items.

A second API call was made, and the data was saved to a Pandas DataFrame.

This is a great example of using API in your data science project.

Quandl's API

Another example is the Quandl API, which can be used to access financial data.

In Data Vigo's video, here, he explains how to install Quandl using Python, find the desired data on Quandl's official website, and access the financial data using the API.

This approach allows you to easily feed your financial data project with the necessary information.

Rapid API

As you can see, there are many different options available to scale up your data by using different APIs. To discover the right API for your needs, you can explore platforms like RapidAPI, which offers a wide range of APIs covering various domains and industries. By leveraging these APIs, you can ensure that your data science project is always supplied with the latest data, enabling you to make well-informed, data-driven decisions.

Data Storage in the Cloud
Image by Author

Now, you collect your data, but where to store it?

The need for secure and accessible data storage is paramount in a data science project.

Ensuring that your data is both safe from unauthorized access and easily available to authorized users allows for smooth operations and efficient collaboration among team members.

Cloud-based databases have emerged as a popular solution for addressing these requirements.

Some popular cloud-based databases include Amazon RDS, Google Cloud SQL, and Azure SQL Database.

These solutions can handle large volumes of data.

Notable applications that utilize these cloud-based databases include ChatGPT, which runs on Microsoft Azure, demonstrating the power and effectiveness of cloud storage.

Let’s look at this use case.

Google Cloud SQL

To set up a Google Cloud SQL instance, follow these steps.

1. Go to the Cloud SQL Instances page.
2. Click "Create instance."
3. Click "Choose SQL Server."
4. Enter an ID for your instance.
5. Enter a password.
6. Select the database version you want to use.
7. Select the region where your instance will be hosted.
8. Update the settings according to your preferences.

For more detailed instructions, refer to the official Google Cloud SQL documentation. Additionally, you can read this article that explains Google Cloud SQL for practitioners, providing a comprehensive guide to help you get started.

By utilizing cloud-based databases, you can ensure that your data is securely stored and easily accessible, enabling your data science project to run smoothly and efficiently.

Data Cleaning and Preprocessing
Image by Author

You collect your data and store it in the cloud. Now, it is time to transform your data for further stages.

Because raw data often contains errors, inconsistencies, and missing values that can negatively impact the performance and accuracy of your models.

Proper data cleaning and preprocessing are essential steps to ensure that your data is ready for analysis and modeling.

Pandas and NumPy

Creating a script for cleaning and preprocessing involves the use of programming languages like Python and leveraging popular libraries such as Pandas and NumPy.

Pandas is a widely used library that offers data manipulation and analysis tools, while NumPy is a fundamental l?brary for numerical computing in Python. Both libraries provide essential functions for cleaning and preprocessing data, including handling missing values, filtering data, reshaping datasets, and more.

Pandas and NumPy are crucial in data cleaning and preprocessing because they offer a robust and efficient way to manipulate and transform data into a structured format, that can be easily consumed by machine learning algorithms and data visualization tools.

Once you have created a data cleaning and preprocessing script, you can deploy it on the cloud for automation. This ensures that your data is consistently and automatically cleaned and preprocessed, streamlining your data science project.

Data Cleaning on AWS Lambda

To deploy a data cleaning script on AWS Lambda, you can follow the steps outlined in this beginner example on processing a CSV file using AWS Lambda. This example demonstrates how to set up a Lambda function, configure the necessary resources, and execute the script in the cloud.

By leveraging the power of cloud-based automation and the capabilities of libraries like Pandas and NumPy, you can ensure that your data is clean, well-structured, and ready for analysis, ultimately leading to more accurate and reliable insights from your data science project.

Automation
Image by Author

Now, how can we automate this process?

Apache Airflow

Apache Airflow is well-suited for this particular task as it enables the programmable creation, scheduling, and monitoring of workflows.

It allows you to define complex, multi-stage pipelines using Python code, making it an ideal tool for automating data collection, cleaning, and preprocessing tasks in data analytics projects.

Automating a COVID Data Analysis using Apache Airflow

Let’s see its usage in the example project.

Example project: Automating a COVID data analysis using Apache Airflow.

In this example project, here, the author demonstrated how to automate a COVID data analysis pipeline using Apache Airflow.

1. Create a DAG (Directed Acyclic Graph) file
2. Load data from the data source.
3. Clean and preprocess the data.
4. Load the processed data into BigQueryç
5. Send an email notification:
6. Upload the DAG to Apache Airflow

By following these steps, you can create an automated pipeline for COVID data analysis using Apache Airflow.

This pipeline will handle data collection, cleaning, preprocessing, and storage, while also sending notifications upon successful completion.

Automation with Airflow streamlines your data science project, ensuring that your data is consistently processed and updated, enabling you to make well-informed decisions based on the latest information.

Power of Data Visualization
Image by Author

Data visualization plays a crucial role in data science projects by transforming complex data into easily understandable visuals, enabling stakeholders to quickly grasp insights, identify trends and make more informed decisions based on the presented information.

Simply put, it will offer you information in interactive ways.

There are several tools available for creating interactive dashboards including Tableau, Power BI, and Google Data Studio.

Each of these tools offers unique features and capabilities to help users create visually appealing and informative dashboards.

Connecting Dashboard to your cloud-based database

To integrate cloud data into a dashboard, start by choosing a cloud-based data integration tool that aligns with your needs. Connect the tool to your preferred cloud data source and map the data fields you want to display on your dashboard.

Next, select the appropriate visualization tools to represent your data in a clean and concise manner. Enhance data exploration by incorporating filters, grouping options, and drill-down capabilities.

Ensure that your dashboard automatically refreshes the data or configure manual updates as needed.

Lastly, test the dashboard thoroughly for accuracy and usability, making any necessary adjustments to improve the user experience.

Connecting Tableau to your cloud-based database — use case

Tableau offers seamless integration with cloud-based databases, making it simple to connect your cloud data to your dashboard.

First, identify the type of database you are using, as Tableau supports various database technologies such as Amazon Web Services(AWS), Google Cloud, and Microsoft Azure.

Then, establish a connection between your cloud database and Tableau, typically using API keys for secure access.

Tableau also provides a variety of cloud-based data connectors that can be easily configured to access data from multiple cloud sources.

For a step-by-step guide on deploying a single Tableau server on AWS, refer to this detailed documentation.

Alternatively, you can explore a use case that demonstrates the connection between Amazon Athena and Tableau, complete with screenshots and explanations.

Conclusion

The benefits of scaling data science projects with cloud computing include improved resource management, cost savings, flexibility, and the ability to focus on data analysis rather than infrastructure management.

By embracing cloud computing technologies and integrating them into your data science projects, you can enhance the scalability, efficiency, and overall success of your data-driven initiatives.

Improved decision-making and insights from data are achievable too by adopting cloud computing technologies in your data science projects. As you continue to explore and adopt cloud-based solutions, you'll be better equipped to handle the ever-growing volume and complexity of data.

This will ultimately empower your organization to make smarter, data-driven decisions based on the valuable insights derived from well-structured and efficiently managed data pipelines.

In this article, we discussed the importance of data collection using APIs and explored various tools and techniques to streamline data storage, cleaning, and preprocessing in the cloud. We also covered the powerful impact of data visualization in decision-making and highlighted the benefits of automating data pipelines using Apache Airflow.

Embracing the benefits of cloud computing for scaling your data science projects will enable you to fully harness the potential of your data and drive your organization towards success in the increasingly competitive landscape of data-driven industries.
Nate Rosidi is a data scientist and in product strategy. He's also an adjunct professor teaching analytics, and is the founder of StrataScratch, a platform helping data scientists prepare for their interviews with real interview questions from top companies. Connect with him on Twitter: StrataScratch or LinkedIn.

More On This Topic

• Cloud Computing, Data Science and ML Trends in 2020–2022: The battle of…
• Cloud-Native Super Computing
• Beginner's Guide to Cloud Computing
• Discover What It Takes to Scale Innovation & Data Science
• 11 Best Practices of Cloud and Data Migration to AWS Cloud
• Data Science in the Cloud with Dask

Almost overnight, ChatGPT, the runaway, conversational, large language model (LLM) AI hit created by OpenAI has found itself yet another role to play within American society — that of stock market guru.

In a survey of 2,000 Americans conducted by investment advice website The Motley Fool, 47% of US adults reported using ChatGPT to glean advice on stock market picks. In a sign of things to come, 45% said that they would be comfortable with only using the AI model for stock picking.

Also: How to use ChatGPT: Everything you need to know

Hopefully, the survey respondents — as well as all other aspirant Midases in waiting — are aware that ChatGPT-3.5 was trained on contents of the internet up to 2021. To be closer to the pulse of today's market, they would have to pay for ChatGPT Plus, powered by GPT-4.

Even then, GPT-4 is already dated. There is no current system of feeding generative language AIs with new and dynamic information on the internet, such as stock price or interest rate fluctuations. So, if you're thinking about taking a plunge into day-trading, this would not work well for you.

Who is turning to AI for investing advice?

However, for those looking at slightly broader movements, ChatGPT seems to serve admirably well, and younger Americans, many of them digital natives, have wholeheartedly embraced AI for investing advice.

According to The Motley Fool survey, 50% of Millennials and 53% of Gen Zers used the AI LLM to unearth stock picks. Meanwhile, only 25% of Baby Boomers — a cohort that still remembers how fax machines and floppy discs work — felt comfortable doing so.

Also: How to use ChatGPT to write Excel formulas

Unsurprisingly, the survey also found that income levels can be good predictors of who tends to use the service for stock research. As many as 77% of high-income Americans say they have used ChatGPT for investment recommendations, compared to 43% of middle-income Americans and just 23% of low-income Americans.

Gender differences also turned out to be significant: women — who have outperformed men as investors recently — tend to be more conservative in money matters, according to The Motley Fool analyst Asit Sarma, being less impulsive and calmer during market volatility. So, it's no surprise that only 41% of women used ChatGPT versus 55% of men.

In total, just over two-thirds (69%) of American adults said that they would consider using ChatGPT for investment advice in the future. This move to AI could be nothing short of a seismic upheaval in the economics of money management, thanks to the democratization of a tool that even someone like a hedge fund trader is simultaneously salivating over.

Also: 6 major risks of using ChatGPT, according to a new study

One recent survey of the top 50 hedge fund managers by London-based Market Makers found nine out of 10 hedge fund traders are planning to use AI to manage their portfolios for the rest of 2023.

What ChatGPT can do

Already, the news doesn't look too rosy for the money men and women — such as institutional fund managers — who control and attempt to grow the vast assets of American savers.

For instance, a hypothetical fund of 38 stocks, chosen by ChatGPT and based on criteria (such as low debt, high growth) culled from the portfolios it was competing against, rose by 4.93% in the first eight weeks since its creation on March 6, 2023, versus an average of -0.78% posted by the 10 most popular funds in the UK. In fact, the hypothetical fund outperformed the top 10 on 34 of the 39 days the market was open.

Also: ChatGPT is the most sought out tech skill in the workforce, says learning platform

Another potential nail in the coffin is a recent ChatGPT-based study conducted at the University of Florida, which suggests even more dire implications for fund managers worldwide.

In a paper published this week in the Social Science Research Network, professors Alejandro Lopez-Lira and Yuehua Tang described how they decided to test ChatGPT in how well it could conduct 'Sentimental Analysis' — essentially looking at headlines in articles to determine stock-picking strategy.

Again, ChatGPT is not trained beyond September 2021, so the researchers fed the AI model 67,586 headlines pertaining to 4,138 unique companies between then and now.

This kind of analysis has already been taking place in the trading rooms of hedge funds for some time now, but it was the first time that ChatGPT was tested to perform tasks almost identical to expensive proprietary trading platforms and with customized sentiment analysis built-in.

Also: This new technology could blow away GPT-4 and everything like it

The ChatGPT trading model, using sentiment analysis, posted returns in excess of 500% during this period against the -12% from buying and holding an S&P 500 ETF during the same period.

If that isn't bad news for the finance industry — and lower-rung research analysts, in particular, almost every day marks the emergence of new APIs and plug-ins that can integrate with ChatGPT.

For example, PortfolioPilot is a freshly released and verified ChatGPT plugin that allows portfolios to be dumped into it for analysis and recommendations — all for free.

Considering AI's existing ability to best the most blue-chip of money managers out there, the days of paying mutual funds management fees for middling returns, at least in the current structure, may be winding to a close.

What ChatGPT can't do

All of this sounds like investing nirvana, but before you plunge into the fray, buying and selling with your favorite investing bot, there are a few things you should keep in mind.

ChatGPT's training ended in September 2021, so anything you ask it will not reflect the time period since. And that gap brings us to the tendency for generative AI to make up things when it doesn't have answers to your questions — referred to as AI hallucinations.

Also: How I used ChatGPT and AI art tools to launch my Etsy business fast

Generative AI detects patterns really well, which it does by scraping data from pre-existing texts. However, it does not do well in causal reasoning and you could be lured into believing what it says through its glib conversational abilities. It's also bad at math. Therefore, information or insights have to be double-checked for accuracy.

It also doesn't read facial expressions well, which many investigative journalists and stock pickers rely on when watching CEO or CFO interviews to gauge the actual health of a company.

Finally, according to investment professionals, while it has amazed and delighted with its polished responses, many of these suggestions are still way too generalized to be helpful. It doesn't ask the kind of sophisticated questions that any portfolio manager would.

Also: How does ChatGPT work?

Of course, AI can get better at all of these tasks in time, and each version that has come out has proven to be astonishingly better than its predecessor. But we're not quite at investing nirvana yet.

And, when it gets there, we may have to negotiate a slightly larger headache — how do you make money in a market where information and analysis for any conceivable asset anywhere in the world is not at a premium, but just an AI prompt away?

Artificial Intelligence

Microsoft's Bing AI chatbot has impressed many with its advanced features including GPT-4 and access to the internet, while still remaining free to use.

The only barrier to entry has been having a Microsoft account — until now.

Also: The best AI chatbots to try

On Tuesday, Michael Schechter, Vice President of Growth and Distribution at Microsoft, shared via Twitter that Microsoft is rolling out unauthenticated chat access on Bing.

Through this update, anyone can experience the Bing Chat hype, even if you are a loyal Google user who refuses to create a Microsoft account.

Also: These 4 popular Microsoft apps are getting a big AI boost

However, there is a catch: non-authenticated user conversations are limited to five chat turns per session.

If you like your limited experience with Bing Chat, you can easily make a Microsoft account and get full access to the chatbot with 20 chat turns per session instead of five.

Also: How to use Bing Chat (and how it's different from ChatGPT)

This update follows a steady stream of AI upgrades Microsoft has been releasing for the last couple of weeks across its platforms.

Most recently, Microsoft introduced a wave of updates to its Bing, Edge, Swiftkey, and Skype apps.

Artificial Intelligence

Meta bets big on AI with custom chips — and a supercomputer Kyle Wiggers 7 hours

At a virtual event this morning, Meta lifted the curtains on its efforts to develop in-house infrastructure for AI workloads, including generative AI like the type that underpins its recently launched ad design and creation tools.

It was an attempt at a projection of strength from Meta, which historically has been slow to adopt AI-friendly hardware systems — hobbling its ability to keep pace with rivals such as Google and Microsoft.

“Building our own [hardware] capabilities gives us control at every layer of the stack, from datacenter design to training frameworks,” Alexis Bjorlin, VP of Infrastructure at Meta, told TechCrunch. “This level of vertical integration is needed to push the boundaries of AI research at scale.”

Over the past decade or so, Meta has spent billions of dollars recruiting top data scientists and building new kinds of AI, including AI that now powers the discovery engines, moderation filters and ad recommenders found throughout its apps and services. But the company has struggled to turn many of its more ambitious AI research innovations into products, particularly on the generative AI front.

Until 2022, Meta largely ran its AI workloads using a combination of CPUs — which tend to be less efficient for those sorts of tasks than GPUs — and a custom chip designed for accelerating AI algorithms. Meta pulled the plug on a large-scale rollout of the custom chip, which was planned for 2022, and instead placed orders for billions of dollars’ worth of Nvidia GPUs that required major redesigns of several of its datacenters.

In an effort to turn things around, Meta made plans to start developing a more ambitious in-house chip, due out in 2025, capable of both training AI models and running them. And that was the main topic of today’s presentation.

Meta calls the new chip the Meta Training and Inference Accelerator, or MTIA for short, and describes it as a part of a “family” of chips for accelerating AI training and inferencing workloads. (“Inferencing” refers to running a trained model.) The MTIA is an ASIC, a kind of chip that combines different circuits on one board, allowing it to be programmed to carry out one or many tasks in parallel.

An AI chip Meta custom-designed for AI workloads.

“To gain better levels of efficiency and performance across our important workloads, we needed a tailored solution that’s co-designed with the model, software stack and the system hardware,” Bjorlin continued. “This provides a better experience for our users across a variety of services.”

Custom AI chips are increasingly the name of the game among the Big Tech players. Google created a processor, the TPU (short for “tensor processing unit”), to train large generative AI systems like PaLM-2 and Imagen. Amazon offers proprietary chips to AWS customers both for training (Trainium) and inferencing (Inferentia). And Microsoft, reportedly, is working with AMD to develop an in-house AI chip called Athena.

Meta says that it created the first generation of the MTIA — MTIA v1 — in 2020, built on a 7-nanometer process. It can scale beyond its internal 128MB of memory to up to 128GB, and in a Meta-designed benchmark test — which, of course, has to be taken with a grain of salt — Meta claims that the MTIA handled “low-complexity” and “medium-complexity” AI models more efficiently than a GPU.

Work remains to be done in the memory and networking areas of the chip, Meta says, which present bottlenecks as the size of AI models grow, requiring workloads to be split up across several chips. (Not coincidentally, Meta recently acquired an Oslo-based team building AI networking tech at British chip unicorn Graphcore.) And for now, the MTIA’s focus is strictly on inference — not training — for “recommendation workloads” across Meta’s app family.

But Meta stressed that the MTIA, which it continues to refine, “greatly” increases the company’s efficiency in terms of performance per Watt when running recommendation workloads — in turn allowing Meta to run “more enhanced” and “cutting-edge” (ostensibly) AI workloads.

A supercomputer for AI

Perhaps one day, Meta will relegate the bulk of its AI workloads to banks of MTIAs. But for now, the social network’s relying on the GPUs in its research-focused supercomputer, the Research SuperCluster (RSC).

First unveiled in January 2022, the RSC — assembled in partnership with Penguin Computing, Nvidia and Pure Storage — has completed its second-phase buildout. Meta says that it now contains a total of 2,000 Nvidia DGX A100 systems sporting 16,000 Nvidia A100 GPUs.

So why build an in-house supercomputer? Well, for one, there’s peer pressure. Several years ago, Microsoft made a big to-do about its AI supercomputer built in partnership with OpenAI, and more recently said that it would team up with Nvidia to build a new AI supercomputer in the Azure cloud. Elsewhere, Google’s been touting its own AI-focused supercomputer, which has 26,000 Nvidia H100 GPUs — putting it ahead of Meta’s.

Meta’s supercomputer for AI research.

But beyond keeping up with the Joneses, Meta says that the RSC confers the benefit of allowing its researchers to train models using real-world examples from Meta’s production systems. That’s unlike the company’s previous AI infrastructure, which leveraged only open source and publicly available data sets.

“The RSC AI supercomputer is used for pushing the boundaries of AI research in several domains, including generative AI,” a Meta spokesperson said. “It’s really about AI research productivity. We wanted to provide AI researchers with a state-of-the-art infrastructure for them to be able to develop models and empower them with a training platform to advance AI.”

At its peak, the RSC can reach nearly 5 exaflops of computing power, which the company claims makes it among the world’s fastest. (Lest that impress, it’s worth noting some experts view the exaflops performance metric with a pinch of salt and that the RSC is far outgunned by many of the world’s fastest supercomputers.)

Meta says that it used the RSC to train LLaMA, a tortured acronym for “Large Language Model Meta AI” — a large language model that the company shared as a “gated release” to researchers earlier in the year (and which subsequently leaked in various internet communities). The largest LLaMA model was trained on 2,048 A100 GPUs, Meta says, which took 21 days.

“Building our own supercomputing capabilities gives us control at every layer of the stack; from datacenter design to training frameworks,” the spokesperson added. “RSC will help Meta’s AI researchers build new and better AI models that can learn from trillions of examples; work across hundreds of different languages; seamlessly analyze text, images, and video together; develop new augmented reality tools; and much more.”

Video transcoder

In addition to MTIA, Meta is developing another chip to handle particular types of computing workloads, the company revealed at today’s event. Called the Meta Scalable Video Processor, or MSVP, Meta says that it’s its first in-house-developed ASIC solution designed for the processing needs of video on demand and live streaming.

Meta began ideating custom server-side video chips years ago, readers might recall, announcing an ASIC for video transcoding and inferencing work in 2019. This is the fruit of some of those efforts, as well as a renewed push for a competitive advantage in the area of live video specifically.

“On Facebook alone, people spend 50% of their time on the app watching video,” Meta technical lead managers Harikrishna Reddy and Yunqing Chen wrote in a co-authored blog post published this morning. “To serve the wide variety of devices all over the world (mobile devices, laptops, TVs, etc.), videos uploaded to Facebook or Instagram, for example, are transcoded into multiple bitstreams, with different encoding formats, resolutions and quality … MSVP is programmable and scalable, and can be configured to efficiently support both the high-quality transcoding needed for VOD as well as the low latency and faster processing times that live streaming requires.”

Meta’s custom chip designed to accelerate video workloads, like streaming and transcoding.

Meta says that its plan is to eventually offload the majority of its “stable and mature” video processing workloads to the MSVP and use software video encoding only for workloads that require specific customization and “significantly” higher quality. Work continues on improving video quality with MSVP using preprocessing methods like smart denoising and image enhancement, Meta says, as well as post-processing methods such as artifact removal and super-resolution.

“In the future, MSVP will allow us to support even more of Meta’s most important use cases and needs, including short-form videos — enabling efficient delivery of generative AI, AR/VR and other metaverse content,” Reddy and Chen said.

AI focus

If there’s a common thread in today’s hardware announcements, it’s that Meta’s attempting desperately to pick up the pace where it concerns AI, specifically generative AI.

As much had been telegraphed prior. In February, CEO Mark Zuckerberg — which has reportedly made upping Meta’s compute capacity for AI a top priority — announced a new top-level generative AI team to, in his words, “turbocharge” the company’s R&D. CTO Andrew Bosworth likewise said recently that generative AI was the area where he and Zuckerberg were spending the most time. And chief scientist Yann LeCun has said that Meta plans to deploy generative AI tools to create items in virtual reality,

“We’re exploring chat experiences in WhatsApp and Messenger, visual creation tools for posts in Facebook and Instagram and ads, and over time video and multi-modal experiences as well,” Zuckerberg said during Meta’s Q1 earnings call in April. “I expect that these tools will be valuable for everyone from regular people to creators to businesses. For example, I expect that a lot of interest in AI agents for business messaging and customer support will come once we nail that experience. Over time, this will extend to our work on the metaverse, too, where people will much more easily be able to create avatars, objects, worlds, and code to tie all of them together.”

In part, Meta’s feeling increasing pressure from investors concerned that the company’s not moving fast enough to capture the (potentially large) market for generative AI. It has no answer — yet — to chatbots like Bard, Bing Chat or ChatGPT. Nor has it made much progress on image generation, another key segment that’s seen explosive growth.

If the predictions are right, the total addressable market for generative AI software could be $150 billion. Goldman Sachs predicts that it’ll raise GDP by 7%.

Even a small slice of that could erase the billions Meta’s lost in investments in “metaverse” technologies like augmented reality headsets, meetings software and VR playgrounds like Horizon Worlds. Reality Labs, Meta’s division responsible for augmented reality tech, reported a net loss of $4 billion last quarter, and the company said during its Q1 call that it expects “operating losses to increase year over year in 2023.”

It’s the year 1995. AMD just won a groundbreaking settlement against Intel, the pioneer behind the x86 standard, that allows it to continue manufacturing chips using this method. Almost three decades later, AMD has not only leapfrogged Intel’s technology, but dominates Team Blue (Intel) in every segment of the semiconductor market.

As Intel is plagued with more delays to its roadmap, AMD is belting out chips like hotcakes. From the Z1 series chips made for handheld gaming, to the 7000 series chips for desktops and laptops, to the EPYC series chips for the enterprise, Team Red (AMD) is handily beating Intel at every turn.

On the other hand, both Intel and Apple — currently AMD’s biggest competitors — have moved to collaborating with ARM. This IP giant relies on a big.LITTLE design for their chips. This design uses smaller cores for less power-intensive tasks while reserving bigger, more power-hungry cores for powerful tasks, improving overall efficiency.

AMD, on the other hand, is relying on a chiplet design to scale processor size and power. Instead of packing all of the chips’ features into a single big piece or ‘die’, AMD opted to split up the chip into separate, smaller parts known as ‘chiplets’ and connect them together using ‘Infinity Fabric’. With the launch of AMD’s Zen 5 on the horizon, a question arises — Will AMD go the ARM way, or double down on its winning strategy?

Can chiplets keep winning?

When AMD wanted to compete against Intel in the server chip market in 2015, they turned to chiplets as their last resort. This technology debuted with the launch of the first EPYC server chip, to great success.

Moving to this process not only helped AMD to cut manufacturing costs by 40%, it also enabled the creation of chiplet-based desktop CPU. The Zen 2 lineup of chips brought chiplets to consumers, further optimising the manufacturing process. What’s more, the company could simply scale up the number of chiplets in the package to increase the number of cores or power of the chips, subverting the limitations put in place by Moore’s Law.

Speaking of Moore’s Law, the 1965 article that spawned this rule also spoke about chiplets as the chip technology of the future. While it didn’t mention the technology by name, the article stated, “It may prove to be more economical to build large systems out of smaller functions, which are separately packaged and interconnected. The availability of large functions, combined with functional design and construction, should allow the manufacturer…to design and construct a considerable variety of equipment both rapidly and economically.”

The economical and manufacturing advancements that have come with the last 5 decades of chip manufacturing have finally made the economy of manufacturing chiplets possible. The approach also allows for a mixed bag approach to manufacturing chips, provided a chiplet standard is in place.

The industry also seems to think along the same lines, as seen by the introduction of the universal chip interconnect express standard. Supported by AMD, ARM, Google Cloud, Microsoft, Meta, and even Intel, this new chiplet ecosystem promises to go beyond Moore’s Law to make the computing systems of the future.

ARM and Intel’s support of this standard shows that even AMD’s direct competitors see the benefits of chiplets design. In a way, it seems that they are admitting defeat to the superior chip design methodology.

AMD also seems to be doubling down on chiplets for its next generation of CPUs, which are rumoured to provide up to an 18% performance increase while reducing power consumption by 34%. This improvement will be made thanks to a move from TSMC’s N5 manufacturing process to the N3 manufacturing nodes. However, for the time being, it seems that Intel is content to push its existing technology to the brink of obsolescence.

big.LITTLE loses out

Manufacturing a non-homogenous chip is no joke, which is something that Intel found out the hard way. Intel’s 12th Gen Alder Lake CPUs, which were the trial run for the efficiency cores and performance cores design (Intel’s version of big.LITTLE), were delivered without a hitch. However, it encountered problems when moving towards more complex architectures in the same design.

Another of Intel’s line of chips called Sapphire Rapids, better known by the market name ‘4th Gen Xeon Scalable’ CPUs, were plagued with a host of delays. In May 2022, it was reported that Intel ran into a major technical flaw in the design of Sapphire Rapids, resulting in the delay of the chips to the beginning of 2023.

However, this was already too late for Intel, as AMD handily beat the company with the 96-core EPYC Genoa CPU. The competing offering beat Intel’s chips by over 70%, using chiplets to gain the edge. To add salt to the wound, Intel’s next lineup of chips, codenamed Meteor Lake, has also been hit with delays due to production issues.

To gain back some of the ground Intel lost with the botched launch of Sapphire Rapids, it entered into a partnership with ARM. This partnership mainly focused on the manufacturing of mobile chips in the form of SoC or System on Chip approach. Where Intel’s Foundry arm got a deeper insight into ARM’s intellectual property, ARM got access to Intel’s 18A manufacturing process.

Interestingly, another emerging player in the ARM-based processor game, Apple, has also been facing issues. While the chips are undeniably well-suited for their deployed applications, especially in terms of power efficiency, it seems that this is not enough to carry the MacBook’s sales. Reportedly, the next generation of M-series chips will include even more cores in an attempt to entice customers to buy them.

However, it seems that the big.LITTLE architecture is beginning to show its age. As Moore’s Law begins to reach the point of obscurity, older manufacturing designs are beginning to see diminishing returns when it comes to performance. Chiplets are the way of the future, and it even seems like Intel and ARM believe this. With its commitment to the chiplet design, it seems that AMD is uniquely positioned to make the most out of the next decade of chips.

The post AMD Takes On Both Intel and Apple: Will It Win? appeared first on Analytics India Magazine.

Image by Author Introducing PaLM 2

The Pathways Language Model (PaLM) has been updated with improved multilingual, reasoning, and coding capabilities. This new model is more capable of understanding and generating text in multiple languages, as well as reasoning and coding.

PaLM 2 was trained on a massive dataset of text and code in over 100 languages. To improve its reasoning capabilities, the developers included scientific papers and web pages with mathematical expressions. PaLM 2 was also pre-trained on publicly available source code in various programming languages. As a result, it is a top-of-the-line, next-generation language model that is powering various Google services.

Bard on PaLM 2

According to Google Keynote (Google I/O ‘23), Bard is now running on the PaLM 2 model. It is far better at coding, reasoning, and creative writing problems than LaMDA.

Image from Google Keynote (Google I/O ‘23)

I have been using the old Bard (LaMDA) for 30 days and the new Bard (PaLM 2) for 7 days. I have seen drastic changes in the way Bard handles coding problems. Bard is not perfect, but I think Google is on the right track.

For example, when I asked Bard to create a snake game using Pygame, the old Bard was able to create the game, but it had several bugs and reduced functionality. The new Bard was able to create a working snake game with all of the expected features.

I am still seeing some bugs with the new Bard, but overall I am impressed with the progress that Google has made.

Image from Bard

I asked both ChatGPT and HuggingChat to generate code to solve a similar problem. ChatGPT generated bug-free code with additional functionality, while HuggingChat generated code with several errors, missing libraries, and security vulnerabilities.

Image by Author | Using ChatGPT

How Bard is different from ChatGPT?

Whenever you write a prompt, it will provide you three drafts to choses from. It is fast in producing the results, and comes with Google services integrations.

To access the drafts, you need to click on “view other drafts”.

Image from Bard

To access Google integrations, click on the up arrow at the bottom left. It is a code response. You will get the option to run your code on Google Colab.

Image from Bard Bard for Data Science

I have been using Bard for all kinds of data science taks, from understanding the project to producing high quality data reports. I believe that Bard is the best large language model available for the following reasons:

• Grammar and Writing: Bard is good at improving grammar and coming up with realistic text that can be used to improve your writing overall. It is better at this than ChatGPT, which can be overly dramatic.
• Machine Learning Research: Bard is good at researching machine learning topics. It can provide you with accurate information on a wide range of topics, even the latest research.
• Translation: Bard is good at translation. It can translate between many different languages, including Python code to JavaScript or English to Japanese.
• Brainstorming, Project Planning, and Understanding Context: Bard is good at brainstorming, project planning, and understanding context. It will evaluate chat history to provide appropriate answers, instead of giving random responses.
• Generating DALL-E 2, Midjourney, and Stable Diffusion Prompts: Bard is good at generating DALL-E 2, Midjourney, and Stable Diffusion prompts. It can help you create realistic images and art from text descriptions.
• Providing Links to External Sources: Bard is good at providing links to external sources. This can be helpful if you want to learn more about a topic or see an example of something that Bard has generated.

Image from Bard

"Apart from code generation, I am using Bard for everything."

Super Bard: Bard + Tools

Now, let’s talk about the Super Bard that can do all. In the upcoming month, Google has announced Google service and third-party integrations. It means you can prompt in Bard and move the final response to Google Docs, Colab, Email, or any third-party software that you use for work.

Till now, we know you can use Bard to perform research, convert it into a table, modify the table, and export the response to Google Sheets. Moreover, you can use Google Lens service to interact with the image. For example, "Can you describe the image in detail?" Similar to GPT-4.

But it is better than GPT-4.

In the future, you will be able to use Adobe Firefly to generate Images directly from the Bard. You will be able to automate most of your tasks just by typing prompts.

Image by Author from Google I/O ‘23 Conclusion

In conclusion, I believe that Bard has the potential to be a one-stop solution for all of your work-related tasks. The team is constantly working to improve the model and add new features, and they are on the right track to overtake GPT-4. However, there are still a few areas where Bard could improve, such as its ability to handle code-related problems and its integration with Google Search. If Bard can address these issues, I believe it will be a truly revolutionary tool that can change the way we work.
Abid Ali Awan (@1abidaliawan) is a certified data scientist professional who loves building machine learning models. Currently, he is focusing on content creation and writing technical blogs on machine learning and data science technologies. Abid holds a Master's degree in Technology Management and a bachelor's degree in Telecommunication Engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.

More On This Topic

• What is Google AI Bard?
• 8 Open-Source Alternative to ChatGPT and Bard
• ChatGPT vs Google Bard: A Comparison of the Technical Differences
• Why Data Scientists Expect Flawed Advice From Google Bard
• Cloud-Native Super Computing
• 7 Super Cheat Sheets You Need To Ace Machine Learning Interview

Imagine building products and then asking the government to regulate other similar ones. OpenAI is asking the US Senate to do exactly that. During the Senate appearance, Sam Altman asked the government to form a new agency to regulate the industry, which includes both giving and revoking permits from companies. On the other hand, he believes that OpenAI’s technology, though can be risky, is going to remain under control, and thus should be audited differently.

“Regulation of AI is essential,” said Altman. OpenAI is probably just trying to shoo away the competition, just to earn some profits, pay off its investments, and not because it is actually scared of the technology. To achieve this, the company has taken sides with the government to restrict the ease of entry into the market for other players.

"They don't know I'm using GPT-5 to answer all of their questions." pic.twitter.com/2sZULP4xN0

— Smoke-away (@SmokeAwayyy) May 16, 2023

While this is happening from OpenAI’s side, StabilityAI, one of the bigger proponents of open-source, have filed a letter to the US Senate, advocating f0r open models. “Open models and open datasets will help to improve safety through transparency, foster competition, and ensure the United States retains strategic leadership in critical AI capabilities,” the letter read.

Similarly, in February, amid the discussion around the EU AI Act about open source, GitHub CEO Thomas Dohmke, said that open source developers should be exempted from the AI act. He explained that, “The compliance burden should fall on companies that are shipping products,” like OpenAI, Google, or Microsoft.

Fear Mongering AI

By now, it is quite clear that OpenAI has stopped caring about “open” AI altogether. With the Senate discussion, it is clear that Altman played on the side of the government by tapping into their fears that these regulators have. A lot of them do not really understand how it works. Senator Josh Hawley said that he was there “to try to get my head around what these models can do”.

OpenAI has been paving the way all this while to come to this conclusion. Last month, the company published a blog talking about deploying AI systems safely. In the blog, the company spoke about how it is important to understand the benefits and risks of deploying such models before releasing them into the public. The company claimed that they did the same with GPT-4 for six months.

Eventually, Altman decided to put a pause on GPT-5 citing safety concerns.

At the Senate, Altman repeated the same thing. “Appropriate safety requirements, including internal and external testing prior to release”, is essential for any AI model, and the government should now watch over this as more and more open-source models are coming up. But OpenAI’s version of AI safety has been very fluffed – all words, no action. The technology was even banned in several countries.

After becoming a “for-profit” company, OpenAI’s Ilya Sutskever said, “we were wrong”, and if AI becomes potent someday, “it doesn’t make sense to be open source”. The company believes that “open-sourcing AI is not wise”.

Looks like instead of building a product which would be the moat for the company, it has decided to form a regulatory moat to ensure corporate dominance. If observed closely, this can be an anti-competitive practice. This definitely could ensure more legal and regulatory backlash for OpenAI itself.

On the other hand, it can be that OpenAI being the leader of the pack has been able to understand the potential danger better than anyone else. Take the example of OpenAI hiring a killswitch engineer, to pull the plug on the system if it ever gets out of control. It can all just be a facade as well, who knows?

Altman Got Off Easy

The big names in AI have already been against a lot of what OpenAI was doing. The closed-door approach was criticised by Elon Musk which eventually led to a petition to pause giant AI experiments beyond GPT-4, which is now signed by more than 27,000 people. Even though it wasn’t mentioned, this was clearly a move against the monopoly that OpenAI and Microsoft have over AI right now.

One of the leading petitioners along with Musk, Steve Wozniak, was Gary Marcus, who has been critical of deep learning technologies for a long time now. Interestingly enough, he was one of the people to be called at the Senate for discussion about the developing AI technology. To great surprise, a lot of ideas of Marcus and Altman regarding restricting the technology coincided with each other. The reasons can be completely different though.

To counter the pause petition, LAION (Large-scale AI Open Network), a Germany-based research organisation, filed a petition and sent a letter to the European Parliament to speed up the process of opening AI models for a “secure future”. The letter was signed by several European AI experts like Jurgen Schmidhuber.

This is similar to what StabilityAI has been asking for now. Interestingly, LAION received a lot of backlash for this as it was one of the dataset providers for Stable Diffusion, StabilityAI’s image generator for copyright infringement.

Meanwhile, amid this struggle between OpenAI and open source, LAION collaborated with Together and Ontocord.ai, to release an open source alternative to ChatGPT, called OpenChatKit.

Looks like everyone is juggling between what to do with AI and open source. But for now, OpenAI has taken up another enemy – the open source developer community. The company was anyway being criticised for filing for a trademark on ‘GPT’.

Sitting on the fence, but not because of altruism

Marcus and Altman’s collision of approach towards regulating the developing AI technology might be just a coincidence. But this should not be confused with the altruistic approach that Altman sells to the public by acknowledging the risks of the models his company is building.

Moreover, Altman has been touted for being a very conscientious guy for not acquiring any equity in OpenAI. The truth might be that he wants to stay away from the company in case anything goes wrong, while he earns money by investing in other companies through Y Combinator.

In Google’s leaked document, Google and OpenAI not having a moat, it was clearly mentioned that open source is heading for the win when it comes to the developing AI technology. While it might look like the GPT technology that OpenAI has developed might be the moat for them, and now Google’s Bard might hint towards that, it is not sure if it is actually the case.

Sam Altman wants ai regulations so free and truly open source projects get killed before they can take over his paid solution.
It’s business.

— Michal Malewicz (@michalmalewicz) May 18, 2023

Ever since Meta’s LLaMA got leaked, the open source community has been able to replicate what OpenAI’s technology has been able to achieve without requiring as much computational power. This clearly would appear like a threat to OpenAI. Moreover, Google’s PaLM and Hugging Face have been creating some tension for OpenAI with their open products. Moreover, the GPT-4 paper did not reveal much about its model, making researchers and developers again.

Amidst this battle whether LLM-like models should be open or closed, Meta has sided with the open community. Yann LeCun explained that companies hiding their technologies are just for commercial purposes, and keeping it closed is more dangerous.

Thus, instead of calling it out as a completely competitive move, OpenAI has taken the approach of siding with the government regulators, a lot of them who are AI doomers, and playing with their fear to regulate AI and remove competition.

Meanwhile, to get in the good books of the open source community, the company has been taking steps to open source some of its models. Earlier, the company used to extract data and models from the open community, but did not contribute anything back.

Recently, according to reports, OpenAI is releasing another open source model, which might possibly have nothing to do with GPT at all. Might be just a gimmick to call themselves “open” again because the company’s business model is based on providing proprietary models through APIs.

Meanwhile, the US can learn from the debate going on in Europe to avoid harms to the open source community that might be entailed with the drafted AI Act. Instead of imposing an all-out restriction on AI models that would include the developer community as well, the U.S. can be wise and not fall into the words of Altman. Probably they need some advocates from the open source community like Emad Mostaque, Harrison Chase, and Clem Delangue to speak at the senate, on behalf of open source.

The post OpenAI has Stopped Caring about Open AI Altogether appeared first on Analytics India Magazine.