Рубрика: AI

During the ongoing Neural Information Processing Systems (NeurIPS) annual conference, reviewers and chairpersons are currently evaluating tens of thousands of submissions.

Out of the 13,321 papers submitted by authors and researchers worldwide, the top of the lot have won the outstanding awards this year. Here are the 6 outstanding papers announced by NeurIPS in 2023:

Outstanding Main Track Papers

Privacy Auditing with One (1) Training Run

Steinke, Nasr, and Jagielski propose an efficient auditing scheme for assessing the privacy of differentially private machine learning (ML) systems in a single training run. They leverage the parallelism of adding or removing multiple training examples independently. They avoid the computational cost of group privacy by analysing the connection between differential privacy and statistical generalisation.

Their approach works in both black-box and white-box settings, requiring minimal assumptions about the algorithm. They demonstrate the effectiveness of their framework on DP-SGD, achieving meaningful privacy bounds with just one model, while standard methods would need hundreds of models.

Are Emergent Abilities of Large Language Models a Mirage?

Schaeffer, Miranda, and Koyejo challenge the idea that large language models (LLMs) exhibit true emergent abilities. They propose that perceived emergent abilities are often a result of the researcher’s metric choices rather than fundamental changes in model behaviour with scale. They support this with a mathematical model and three analyses:

1. Confirming predictions on metric effects using InstructGPT/GPT-3
2. Validating predictions in a meta-analysis on BIG-Bench
3. Demonstrating how metric choices can create apparent emergent abilities in vision tasks across different networks.

Their findings suggest that alleged emergent abilities may vanish with different metrics, questioning the notion that they are intrinsic to scaled AI models.

Runner-Ups

Scaling Data-Constrained Language Models

In the paper, researchers explored scaling language models in data-limited scenarios, given the potential constraint on internet text data. They conducted extensive experiments, varying data repetition and computed budgets of up to 900 billion tokens and 9 billion parameters. Results showed that with limited data and a fixed computing budget, up to 4 epochs of repeated data had minimal impact on loss. However, further repetition diminished the value of additional compute.

They proposed a scaling law for compute optimality, considering the declining value of repeated tokens and excess parameters. Additionally, they tested methods to alleviate data scarcity, such as augmenting with code data or removing common filters.

Models and datasets from 400 training runs are freely available on GitHub.

Direct Preference Optimization: Your Language Model is Secretly a Reward Model

Here, researchers introduced Direct Preference Optimization (DPO) as a streamlined alternative to Reinforcement Learning from Human Feedback (RLHF) for controlling large unsupervised language models. Unlike RLHF, DPO avoids the complexity and instability of fitting reward models and fine-tuning. Leveraging a mapping between reward functions and optimal policies, DPO directly optimises a single-stage policy training process, solving a classification problem on human preference data.

The experiments demonstrate that DPO can effectively align language models with human preferences, outperforming RLHF in sentiment control and improving response quality in summarization and dialogue. Notably, DPO is more straightforward to implement and train.

Datasets and Benchmarks Papers:

ClimSim: A large multi-scale dataset for hybrid physics-ML climate emulation

Machine learning experts have introduced ClimSim, the largest hybrid ML-physics dataset, co-created by climate scientists and ML researchers. With 5.7 billion pairs of input-output vectors, it isolates the impact of high-resolution physics on macro-scale climate states. Global and spanning multiple years, the dataset facilitates emulators compatible with operational climate simulators.

The data and code are released openly to support the development of hybrid ML-physics and high-fidelity climate simulations.

DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models

With the rise of GPT models, practitioners are considering using them for sensitive applications like healthcare and finance, but research reveals undisclosed vulnerabilities. GPT models, including GPT-4, can produce biassed, toxic outputs and unintentionally leak private information.

Despite GPT-4’s generally improved trustworthiness, it exhibits vulnerability to jailbreaking systems or misleading user prompts. This study highlights previously unrecognised trustworthiness gaps in GPT models.

The benchmark is publicly available on GitHub.

The post 6 Outstanding Papers Presented at NeurIPS 2023 appeared first on Analytics India Magazine.

In the ever-evolving landscape of artificial intelligence, we have witnessed the launch of groundbreaking vision models that pushed the boundaries of computer vision. These cutting-edge models harnessed advanced neural network architectures, sophisticated training techniques, and unprecedented data sets to redefine the capabilities of visual perception — from enhanced object recognition to nuanced scene understanding.

Here is a list of the top 7 vision models launched this year.

DINOv2

Meta AI has developed DINOv2, an innovative method for training high-performance computer vision models that delivers exceptional performance and does not require fine-tuning. As a result, it is well-suited to serve as a backbone for various computer vision tasks.

Thanks to its self-supervised learning approach, DINOv2 can learn from any image collection and acquire features that the current standard approach cannot, such as depth estimation.

In 2023, DINOv2 was open-sourced, becoming the first method for training computer vision models to use self-supervised learning to achieve results that match or surpass the standard approach used in the field. Self-supervised learning is a potent and adaptable way to train AI models since it does not require vast amounts of labelled data.

The models can be trained on any image collection, regardless of whether they have associated metadata, and learn from all images they are given. This approach expands the potential applications of computer vision models, making them more versatile and powerful than ever before.

YOLOv8

The YOLO (You Only Look Once) series of models are very well-known in the computer vision world. YOLO is famous because it achieves high accuracy while having a small model size. It can be trained on a single GPU. Machine learning practitioners can deploy it on edge hardware or in the cloud at a low cost.

YOLOv8 is the latest YOLO model that uses advanced technology to detect objects, classify images, and segment instances. It was created by Ultralytics, the same team that developed the highly influential YOLOv5 model. YOLOv8 brings various architectural and developmental improvements over its predecessor YOLOv5.

As of now, Ultralytics is actively developing YOLOv8 and working on new features while considering feedback from the community. The organisation ensures that their models receive long-term support and collaborates with the community to improve the model’s performance.

EfficientViT

Vision transformers are a widely used framework in computer vision, as they provide high computational capabilities and superior performance. However, while these models continue to improve in accuracy and performance, they also come with higher operational costs and computational overhead. The EfficientNet model was developed to address this issue.

It seeks to enhance the performance of vision transformers and determine the principles for designing efficient and effective transformer-based framework architectures. The EfficientViT model is built on existing vision transformer frameworks such as Swim and DeiT. It analyses three critical factors that affect the speed of model interference, including computation redundancy, memory access, and parameter usage.

SWIN Transformer

The field of medical image segmentation can be challenging due to the need for large amounts of pre-training data, which is difficult to acquire. However, recent advancements in large-scale Vision Transformers have led to significant progress in improving pre-trained models for this purpose.

One such advancement is the Masked Multi-view with Swin Transformers (SwinMM), a novel multi-view pipeline that enables accurate and data-efficient self-supervised medical image analysis. SwinMM utilises a masked multi-view encoder during the pre-training phase to train masked multi-view observations through image reconstruction, rotation, contrastive learning, and a novel task that capitalises on the consistency between predictions from various perspectives.

During the fine-tuning stage, a cross-view decoder aggregates the multi-view information through a cross-attention block. SwinMM outperforms the previous state-of-the-art self-supervised learning method. SwinMM shows great potential for future applications in medical imaging.

SimCLR

The SimCLR vision model is designed to learn image representations from unlabeled data by generating positive and negative image samples using image augmentation. It then minimises the contrastive loss function to explore more underlying structural information. SimCLR-Inception model, the new version launched in 2023, achieves better results and the accuracy is at least 4% higher than the other compared models such as LeNet, VGG16, Inception V3, and EfficientNet V2, indicating that this model could work better for robot vision.

SimCLR maps the previous conceptual components onto a deep neural network architecture, inspired by residual neural networks (ResNet). Initially, SimCLR randomly selects examples from the original dataset, transforming each example twice using a combination of simple augmentations (random cropping, random colour distortion, and Gaussian blur), creating two sets of corresponding views.

StyleGAN3 by NVIDIA

The progress in generating artificial images has been impressive, thanks to the StyleGAN architecture, which is skilled in creating realistic facial images. In 2023, a group of researchers from NVIDIA and Aalto University introduced StyleGAN 3, which addressed a significant weakness in current generative models. This breakthrough has created numerous possibilities for using these models in video and animation.

Developing this new model was made easier by the well-documented and organised code base and the high level of compatibility with previous versions. It didn’t take long before people began to guide the outputs of StyleGAN3 with CLIP, resulting in beautiful outcomes. In StyleGAN3, each feature’s precise sub-pixel location is solely passed down from the underlying coarse features, resulting in a more natural transformation hierarchy.

MUnit

MUnit is a testing framework for Mule applications that enables you to create automated tests for your APIs and integrations with ease. It offers a comprehensive set of integration and unit testing capabilities and comes fully integrated with Maven and Surefire, making it ideal for use in continuous deployment environments.

In MUnit3, it is assumed that images can be broken down into a content code that is invariant across domains, and a style code that captures domain-specific properties. To translate an image to another domain, the content code is combined with a random style code sampled from the target domain’s style space.

Finally, the latest version makes it possible for users to control the style of translation outputs by providing an example style image.

The post Top 7 Vision Models Transforming the Future of AI in 2023 appeared first on Analytics India Magazine.

Oracle Corporation witnessed a significant drop of more than 9% in its shares during extended trading on Monday. The decline follows the release of the software giant’s fiscal second-quarter results, which not only fell short of Wall Street expectations but also prompted a downward revision in quarterly revenue guidance.

The financial figures, when compared to consensus estimates from LSEG (formerly Refinitiv), revealed several key areas of concern. According to the reported results:

• Earnings per Share: Oracle posted adjusted earnings of $1.34 per share, slightly exceeding the expected $1.32 per share.
• Revenue: The company’s revenue amounted to $12.94 billion, falling short of the anticipated $13.05 billion.

Despite a 5% year-over-year growth in revenue for the quarter ending November 30, Oracle reported a substantial 44% increase in net income to $2.5 billion, equivalent to 89 cents per share, compared to $1.74 billion or 63 cents per share in the same period a year ago.

On a positive note, Oracle reported a remarkable 52% increase in cloud infrastructure revenue, reaching $1.6 billion. Notable clients during this period included Elon Musk’s artificial intelligence startup xAI, Halliburton, and Samsung.

However, Oracle faced challenges meeting the demand from Musk’s company, which sought more AI chips than Oracle could supply.

Read: How Oracle is Fuelling Musk’s Ambitions

During the earnings call with analysts, Oracle co-founder Larry Ellison acknowledged the chip shortage, stating that the company had to choose between a smaller buildout to recognise revenue in the quarter or a larger buildout with a wait for capacity availability, leading to the current constraints.

However, the company’s guidance for the fiscal third quarter raised eyebrows, with adjusted net income projected to range between $1.35 and $1.39 per share and a modest 6% to 8% revenue growth. Analysts polled by LSEG had expected $1.37 in adjusted earnings per share and $13.34 billion in revenue, reflecting a 7.6% growth.

Read: Oracle’s Symbiotic Connection with AMD and NVIDIA

Oracle’s performance in key segments further contributed to the market’s reaction. Notably, revenue from cloud services and license support reached $9.64 billion, showing a 12% increase but still falling short of the StreetAccount consensus of $9.71 billion.

Revenue from cloud and on-premises licenses saw an 18% decline to $1.18 billion, slightly below the $1.21 billion consensus. Additionally, services revenue amounted to $1.37 billion, missing the consensus of $1.40 billion.

Oracle’s strategic moves during the quarter included securing cloud business from larger rival Microsoft and announcing the availability of its database software on Microsoft’s Azure public cloud. The company plans to activate 20 data centers connected with Azure in the coming months.

Read: Oracle’s Grand Multicloud Gamble

Despite these efforts, Oracle’s shares are now up only about 41% for the year, a significant drop from previous performance, still outperforming the S&P 500 index, which has gained 20% during the same period. Investors are left to assess the impact of these results on Oracle’s future prospects and market standing.

The post Oracle Shares Plummet Over 9% Over Q2 Results appeared first on Analytics India Magazine.

Elon Musk’s xAI, and its chatbot Grok, recently has come under fire for alleged bugs. Public attention was drawn to a snapshot shared by ChatGPT’s X account that a security tester posted of Grok rejecting a query, citing OpenAI’s use case guidelines.

To this, Musk replied, “Well, son, since you scraped all the data from this platform for your training, you ought to know.”

Igor Babuschkin, an xAI representative, recognised the problem and blamed it on ChatGPT outputs that were inadvertently included when Grok was being trained on web data.

The explanation presented doubts to specialists, implying that Grok may have been purposefully adjusted using OpenAI model output data.

This is not the first time that an AI model has been trained on OpenAI’s output. The practice of fine-tuning AI models with synthetic data, generated by other language models, has become more common. Most of this is via ShareGPT, where people share their responses while talking to ChatGPT. It allows models like Grok to specialise in specific tasks, such as coding.

Despite claims that the issue is rare, some experts question the likelihood of it being an unintentional accident, suggesting that Grok’s behaviour was trained deliberately. Might be that Grok just wanted to mess around with ChatGPT, after all it’s a funny and “based” chatbot.

On the other hand, in the recent podcast with Lex Fridman, Elon Musk suggested that he likes the idea of open source AI, and would probably open source Grok. “I am generally in favour of open sourcing, like biassed towards open sourcing,” he said.

He is also planning to double the compute power at xAI every month. Currently, Grok is trained on 8,000 NVIDIA A100 GPUs.

The spat between Musk and OpenAI has been long going. Some weeks back, Sam Altman posted a photo of building Grok with a single prompt, to which Musk replied with a funny poem on how GPT-4 is boring.

The post Elon Musk’s Grok Copies OpenAI’s ChatGPT appeared first on Analytics India Magazine.

Since generative AI has become omnipresent among the big tech, the companies have been funnelling money not just to develop AI models but also help their customers ride the tide. In one such attempt, in June, Amazon’s cloud unit invested a hundred million dollars to assist their clientele to develop and implement solutions through the new Generative AI Innovation Center.

At the recent AWS re: Invent AIM caught up with Sri Elaprolu, the Global Head to learn about the recently launched innovation centre’s nitty gritty.

“From ideating, identifying and building a solution that proves the value and meets requirements, that’s what the team does.” pinpointed Elaprolu. More than 1,000 customers have come in across all industries since the project started five months ago.

Elaborating on AWS clientele, he mentioned that the customer base has both global enterprises and public sectors. For instance, the Singapore Ministry of Education is a customer.

Elaprolu also mentioned that AWS is helping its clients understand what to build and how to build it at an enterprise level. “It’s now getting more into the core of the enterprise, not just on the chatbot style application,” he elaborated.

Diving deeper geographically, Elaprolu mentioned that usually, the US tends to be relatively high on the list. This is a trend usually seen in the past.

But generative AI has seen a different beginning. “The thing that’s different this time”, said the industry veteran, “is that it has started to be hot across and not just one active geography. We’re supporting customers across all areas, including Latin America, Africa, and the Middle East — locations that we normally don’t see a jump right into emerging tech.”

Amazon has been at the forefront of AI, he highlighted by stating the examples of AI in Amazon through retail, AWS, and Alexa. “GenAI is new. But in our view, it’s an extension and not a completely radical thing. It’s just that until now, you’ve been able to predict, but now you can create. But very few companies know what that means for them,” he said.

A Broad Spectrum

Speaking about companies moving towards generative AI under the industry’s pressure, Ela said, “Maybe early on, it was a lot of experimentation. Now, there are two ways you can think of how companies are leveraging hardware.”

One is how a business process can be improved and optimised by bringing generative AI. For example, Bridgewater Associates is the world’s largest hedge fund manager. They have investment analysis, which needs intensive maths crunching. “The company is leveraging Amazon’s Bedrock and Anthropic’s Claude models to simplify many things that took weeks and now take hours. They’ve also built investment analysts that allow junior analysts to move quickly. That’s more of an internal optimisation of what you already do,” Elaprolu said.

In terms of cutting down costs time, you can look at customers who are building new capabilities that did not exist before. “That’s another area where we see a lot of customers,” the AWS employee stated. He gave the example of Lonely Planet building a solution for personalised itinerary planning so that their customers can get a much better experience than what they perhaps were doing previously with manual clicking and dragging. “So it’s a wide spectrum of how companies are evolving,” Elaprolu said in conclusion.

Stepwise Breakdown

Building and maintaining a client base of over 1,000 customers in less than six months is not a piece of cake. AWS is following a strategy that is helping them work with many enterprises.

“The first step that we do there is working with the business leadership and technical leadership to understand their vision, pain points, and opportunity areas for their business. Then we work backwards from there to identify what we can do now with AI and methods that are still traditionally valid. Then we do a business impact analysis to check for positive returns,” Elaprolu revealed.

He further explained that this process is done with the customer at the table, not AWS going off and doing their work. “We then move into understanding their environment, data, technical abilities, and various requirements. If you’re operating in a regulated world, you have compliance regimes to meet,” he added.

Once Elaprolu’s team knows the details, they build that solution. That usually can go roughly 4-8 weeks, on average. AWS then demonstrates the value of that solution with the customer data with their users in mind. “That gives them a lot more confidence now because now it’s not just theory,” Elaprolu stated.

The post What Happens at AWS’ $100M Generative AI Innovation Center appeared first on Analytics India Magazine.

The introduction of custom AI-generated wallpapers was a big development on the Google Pixel 8 and Pixel 8 Pro, and now Google is giving a little AI love to its Chrome browser.

It was just a few days ago that Google announced a "Help Me Write" AI feature was headed to Chrome. But now it appears Chrome users will soon have the ability to create a custom AI-generated theme for their browser. The feature was first spotted by X, formerly Twitter, user Leopeva64, who dove deep into the code of the latest unreleased Canary version of the browser.

Also: Why Google's cheaper Pixel 8 is the real star of its Android phone lineup

Like on the Pixel 8, the feature starts off by asking the user to choose a theme. But the themes are quite a bit more robust than what's offered for Google's flagship phones. Under the subjects tab, the X post shows, there are categories like Buildings, Food, Everyday Objects, Nature, Space, US Cities & Parks, and more.

Those categories expand into further options to choose from. Buildings, for example, breaks down into Airport, Cafe, Castle, Lighthouse, Office, and so on. Everyday Objects shows dozens of household objects that a theme can be built around. Under Space, you can build a theme around Constellations, Satellites, Moon, Sun, Stars, Solar system, Spaceships, and more.

US cities is the category I'm most excited to see. A glance shows options for Arches National Park, Chicago, the Grand Canyon, Houston, Los Angeles, New York City, Philadelphia, Phoenix, San Diego, San Francisco, Seattle, and the Everglades among others.

Also: Google's Gemini continues the dangerous obfuscation of AI technology

Once that theme is selected, the user can even further fine-tune their theme with color and mood options — say, a steampunk sad Chicago in blue hues or an expressionist romantic airport with red tones.

Since the feature isn't actually live yet, we don't have an idea of what the wallpapers might look like. But I found myself creating dozens of wallpapers with the Pixel 8 Pro, and it appears Chrome's version will only be better.

Given how long things usually take from first appearing in Chrome's code to actually being deployed for use, it seems likely we'll see a full rollout of this feature within the coming months.

Featured

Last night, I felt restless, so what did I do? I opened up my favorite meditation and sleep application, Calm, and picked a sleep story to play on the Bluetooth speaker. I typically go for one of my favorites, such as "Daddy Pig Reads the Wonderful World of Concrete." But to my surprise, I found a new one, narrated by the great American actor Jimmy Stewart.

Jimmy Stewart… Didn't he die in 1997?

Also: I fact-checked ChatGPT with Bard, Claude, and Copilot — and this AI was the most confidently incorrect

No, Calm did not find an old recording to adapt to its app, as it has done with Bob Ross. Instead, the company enlisted the help of Respeecher, a voice cloning software company that involved a voice actor and AI technology to bring the narration to life.

The company is also known for recreating the voices of Luke Skywalker for "The Mandalorian" TV series and Darth Vader for the series "Obi-Wan Kenobi." Stewart's family and estate, whose likeness is managed by CMG Worldwide, fully approved of the AI voice adaptation.

The 45-minute Calm story, "It's a Wonderful Sleep Story," has Jimmy Stewart narrating a tale about a downtrodden technology entrepreneur, Stanley J. Montgomery — not unlike George Bailey, the small-town banker, Stewart portrays in the 1946 Christmas classic, "It's a Wonderful Life." The synthetic Stewart recites the tale in rhyme and verse to the backdrop of calming, sleepy, floating music. Calm posted this preview to Instagram below.

It's hypnotizing and, dare I say, spookily effective. I don't remember listening to the entire thing, and I know it knocked my wife out pretty quickly.

Also: AI in 2023: A year of breakthroughs that left no human thing unchanged

There's certainly a very uncanny valley feeling hearing the voice of someone who has departed. However, Stewart has been gone long enough that many people listening to it will likely feel nostalgia rather than shock.

I'm unsure how realistic the Stewart voice is, and the cadence is odd. But it works, given the rhyme and intonation of the story and desired application. Still, these AI voice actors have a lot of potential for further exploration, especially by Calm. After Stewart, what is next, Burl Ives, with a new Rudolph story? Boris Karloff with a modern Grinch tale? I'm looking forward to finding out.