This Indian Company is Harnessing AI for Traffic Regulation

Earlier this year, Noida-based smart security and surveillance solutions provider Vehant Technology secured Delhi Police as its client. The law enforcement agency has installed or is in the process of installing 535 Automatic Number Plate Recognition (ANPR) software at strategic locations across the city.

The AI-powered ANPR system’s integration with police station control rooms enhances traffic surveillance by efficiently analysing video streams to recognise and capture vehicle number plates.

“We have covered over 2,000 lanes with our number plate reading cameras, which are either deployed or are being deployed which essentially track vehicle movement in the city with regards to stolen or missing vehicles,” Kapil Bardeja, CEO and co-founder of Vehant Technologies, told AIM.

Worldwide, computer vision algorithms are becoming an indispensable tool for law enforcement agencies. Computer vision algorithms analyse video feeds in real-time, identifying and tracking people, vehicles, and helping law enforcement agencies to enhance public safety, investigate incidents, and respond promptly to potential security threats.

( Vehant co-founders Kapil Bardeja and Anoop G Prabhu)

AI for traffic enforcement

Vehant’s technology is also being utilised by Gurugram police, who have deployed approximately 300 state-of-the-art cameras in the city for traffic enforcement. The models detect violations such as speeding, no helmet, tripling, etc.

The company leverages image classification, object detection, object tracking, and image retrieval algorithms to improve the accuracy in the search and retrieval of objects of reference and expedite their clients’ decision-making capability.

“The models generally available are trained on generic data sets. The ones we use are application-oriented, and we customise them by training them for specific problem-solving. We run the models on edge devices. We take ideas from the literature and develop in-house solutions with performance at par with the state-of-the-art,” Anoop G Prabhu, co-founder & CTO at Vehant Technologies, told AIM.

“Our cameras are spread across 250 different lanes in Gurugram. Our system detects violations; let’s take, for example, someone riding without a helmet. It captures a snapshot, reads the number plate, and provides an image of the number plate. This information is bundled into a challan form and presented to the user for analysis in real-time with the help of AI,” Bardeja told AIM.

However, a manual intervention is involved as law enforcement agencies are mandated by law to manually verify if the challans are accurate and whether they should be subject to a fine.

“Once verified, it gets integrated with the NIC database of Sarthi and Vahaan where the information is fed and an auto-challan is generated. So this is a completely integrated seamless system,” Bardeja said.

Currently, around 2,000 challans are validated on a daily basis. However, Bardeja said that the number of challans generated by AI is way higher. Vehant’s number plate scanning technology is also deployed in 50 Smart Cities across India and the system manages close to 1 billion transactions a year.

AI-powered X-Ray baggage scanning

Vehant is the only Indian company approved by the Bureau of Civil Aviation Security to deploy dual-view X-ray scanners in Indian airports. So far, Vedant’s scanners have been deployed at 70 airports in cities, including Chennai, Kolkata, Trichy, Pune, Udaipur, Varanasi, and Jodhpur, among others.

This year, Vehant also bagged a substantial order worth Rs. 37 crores from the Airports Authority of India (AAI) and Mumbai Airport to deploy its AI-powered dual-view X-ray scanners in Mumbai Airport, which is one of the busiest airports in the world.

The object detection algorithm, integrated into the devices, auto-detects contraband items such as guns, gun parts, knives, and scissors among other things.

“We use AI for detecting threats and restricted objects. The computer collects the information from the machine and identifies the objects using identification algorithms. We use AI-based multi-modal solutions for object and contraband detection,” Anoop G Prabhu, co-founder & CTO of Vehant Technologies, told AIM.

( Dual view X-Ray scanners)

The state-of-the-art machines are powered by AI algorithms, which automatically stops the conveyor immediately when it detects any contraband objects and alerts the operator of critical threats. This results in accurate and reliable detection for security operators with low false alarm rates.

“These are high-end technologies, and even though there are other Indian companies developing single view scanners, in the aviation market, only dual view scanners are sold, and we are the only Indian company which has managed to develop the technology,” Bardeja said.

Vehant’s ‘Make in India’ journey

Previously, most of the security and surveillance technology was imported from other nations. However, after the parliament attack in 2001, the government of India approached IIT Delhi for indigenously developed and under-vehicle scanning systems. Previously, such systems were imported from the UK and cost around INR 1 crore. This kickstarted Vedant Technology’s journey.

Recently, the company deployed its under-vehicle scanning systems at the new parliament building in New Delhi and the Telangana secretariat building. However, back in 2005, when the company started out, according to Bardeja, the whole technology stack was not ready.

“Back then, the computer resolutions were not that good, and the hardware infrastructure, including compute power and GPU accelerators, was not as advanced as it is today. Running algorithms for specific use cases required substantial computational resources, making it economically impractical. Customers were hesitant to invest significant funds for use cases that demanded extensive computational resources,” Bardeja said.

However, in the last few years, significant progress has been made. Today, we have high resolution cameras and the compute cost has also come down significantly.

Even software algorithms, according to Bardeja, have progressed a lot over the years. The cost of bandwidth has also drastically reduced over the years in the country.

“Now, we can feed the data from the camera to the cloud at a relatively low cost. I believe the amalgamation of these factors has allowed us to provide a reasonably accurate solution using our proprietary algorithm at a cost-effective price,” Bardeja concluded.

The post This Indian Company is Harnessing AI for Traffic Regulation appeared first on Analytics India Magazine.

Why Microsoft’s Orca-2 AI Model Marks a Significant Stride in Sustainable AI?

Despite the notable advancements made by artificial intelligence in the last decade, which include defeating human champions in strategic games like Chess and GO and predicting the 3D structure of proteins, the widespread adoption of large language models (LLMs) signifies a paradigm shift. These models, poised to transform human-computer interactions, have become indispensable across various sectors, including education, customer services, information retrieval, software development, media, and healthcare. While these technological strides unlock scientific breakthroughs and fuel industrial growth, a notable downside for the planet exists.

The process of training and utilizing LLMs consumes an immense amount of energy, resulting in a substantial environmental impact marked by an increased carbon footprint and greenhouse gas emissions. A recent study from the College of Information and Computer Sciences at the University of Massachusetts Amherst revealed that training LLMs can emit over 626,000 pounds of carbon dioxide, roughly equivalent to the lifetime emissions of five cars. Hugging Face, an AI startup, found that the training of BLOOM, a large language model launched earlier in the year, led to 25 metric tons of carbon dioxide emissions. Similarly, Facebook's AI model, Meena, accumulates a carbon footprint on par with the environmental impact of driving a car for more than 240,000 miles throughout its training process.

Despite training LLMs, the demand for cloud computing, crucial for LLMs, now contributes more emissions than the entire airline industry. A single data centre can consume as much power as 50,000 homes. Another study highlights that training a single large language model can release as much CO2 as five cars using energy throughout their entire lifetimes. Predictions suggest that AI emissions will surge by 300% by 2025, emphasizing the urgency of balancing AI progress with environmental responsibility and prompting initiatives to make AI more eco-friendly. To address the adverse environmental impact of AI advancements, sustainable AI is emerging as a crucial field of study.

Sustainable AI

Sustainable AI represents a paradigm shift in the development and deployment of artificial intelligence systems, focusing on minimizing environmental impact, ethical considerations, and long-term societal benefits. The approach aims to create intelligent systems that are energy-efficient, environmentally responsible, and aligned with human values. Sustainable AI focuses on using clean energy for computers, smart algorithms that use less power, and following ethical guidelines to ensure fair and transparent decisions. It is important to note that there is a difference between AI for sustainability and sustainable AI; the former may involve using AI to optimize existing processes without necessarily considering its environmental or societal consequences, while the latter actively integrates principles of sustainability into every phase of AI development, from design to deployment, to create a positive and lasting impact on the planet and society.

From LLMs towards Small Language Models (SLMs)

In the pursuit of sustainable AI, Microsoft is working on developing Small Language Models (SLMs) to align with the capabilities of Large Language Models (LLMs). In this effort, they recently introduce Orca-2, designed to reason like GPT-4. Unlike its predecessor, Orca-1, boasting 13 billion parameters, Orca-2 contains 7 billion parameters using two key techniques.

  1. Instruction Tuning: Orca-2 improves by learning from examples, enhancing its content quality, zero-shot capabilities, and reasoning skills across various tasks.
  2. Explanation Tuning: Recognizing limitations in instruction tuning, Orca-2 introduces Explanation Tuning. This involves creating detailed explanations for teacher models, enriching reasoning signals, and improving overall understanding.

Orca-2 uses these techniques to achieve highly efficient reasoning, comparable to what LLMs achieve with many more parameters. The main idea is to enable the model to figure out the best way to solve a problem, whether it's giving a quick answer or thinking through it step by step. Microsoft calls this “Cautious Reasoning.”

To train Orca-2, Microsoft builds a new set of training data using FLAN annotations, Orca-1, and the Orca-2 dataset. They start with easy questions, add in some tricky ones, and then use data from talking models to make it even smarter.

Orca-2 undergoes a thorough evaluation, covering reasoning, text completion, grounding, truthfulness, and safety. The results show the potential of enhancing SLM reasoning through specialized training on synthetic data. Despite some limitations, Orca-2 models show promise for future improvements in reasoning, control, and safety, proving the effectiveness of applying synthetic data strategically in refining the model after training.

Significance of Orca-2 Towards Sustainable AI

Orca-2 represents a significant leap towards sustainable AI, challenging the prevailing belief that only larger models, with their substantial energy consumption, can truly advance AI capabilities. This small language model presents an alternative perspective, suggesting that achieving excellence in language models doesn't necessarily require enormous datasets and extensive computing power. Instead, it underscores the importance of intelligent design and effective integration.

This breakthrough opens new possibilities by advocating a shift in focus—from simply enlarging AI to concentrating on how we design it. This marks a crucial step in making advanced AI more accessible to a broader audience, ensuring that innovation is inclusive and reaches a wider range of people and organizations.

Orca-2 has the potential to significantly impact the development of future language models. Whether it's improving tasks related to natural language processing or enabling more sophisticated AI applications across various industries, these smaller models are poised to bring about substantial positive changes. Moreover, they act as pioneers in promoting more sustainable AI practices, aligning technological progress with a commitment to environmental responsibility.

The Bottom Line:

Microsoft's Orca-2 represents a groundbreaking move towards sustainable AI, challenging the belief that only large models can advance AI. By prioritizing intelligent design over size, Orca-2 opens new possibilities, offering a more inclusive and environmentally responsible approach to advanced AI development. This shift marks a significant step towards a new paradigm in intelligent system design.

With AI Studio, Google launches an easy-to-use tool for developing apps and chatbots based on its Gemini model

With AI Studio, Google launches an easy-to-use tool for developing apps and chatbots based on its Gemini model Frederic Lardinois @fredericl / 8 hours

After announcing its family of Gemini models last week and bringing it to its Bard chatbot experience, Google is now bringing Gemini to developers by launching a slew of new and updated services today. One of these services is AI Studio — which was previously known as MakerSuite.

AI Studio is a web-based tool for developers that functions a bit like a gateway into the wider Gemini ecosystem, starting with Gemini Pro and then, at some point next year, also Gemini Ultra. Using the service, developers can quickly develop prompts and Gemini-based chatbots — and then get API keys to use them in their apps or get access to the code to work on it in a more fully-featured IDE.

It’s important to note that there is a relatively generous free quota, with up to 60 requests per second, which should be enough to quickly iterate on ideas without facing onerous restrictions and maybe even enough to power some lesser-used applications in productions.

There is a price to pay here though: for developers using the free tier (and that’s pretty much everyone for now, as Google only plans to launch a paid version early next year — likely to coincide with the GA launch of the Gemini Ultra model), Google’s reviewers can see the input and output of the API and web app to “improve product quality.” Google notes that this data is de-identified from the user’s Google Account and API key, though.

Compared to the earlier version of Makersuite/AI Studio, this updated edition feels quite a bit more substantial. Among other things, it will offer support for both Gemini Pro and the Gemini Pro Vision model, allowing developers to work with both text and imagery.

“We’ve designed it really to be the fastest way to build with Gemini,” Josh Woodward, Google’s VP for Google Labs, told me. “We really want to invite developers to come play with it. It is the first version and we’ve got a lot of fine-tuning we’re already doing now for future updates, too, but we’re trying to design it in a way where people can just get in and really start building with it.”

In the web interface, developers can choose their models, adjust the temperature to control the output’s creative range, provide examples to provide tone and style instructions. You can also tune the model’s safety settings. It’s worth noting that in MakerSuite, you could turn off all the guardrails by telling the system not to block any responses, while in AI Studio, the lowest setting now is “block few.”

There are also different workflows for creating freeform, structured and chat prompts, too.

Woodward noted that the team tried to design AI Studio so even the free tier wouldn’t feel like a trial or gated product. Indeed, assuming the free tier’s rate limits are sufficient for their use cases, developers can start publishing their AI Studio applications or use them through the API or Google’s SDKs right away.

Jeanine Banks, the VP and GM for Google’s Developer X teams and head of developer relations, also stressed that AI Studio is a gateway into Google’s wider AI ecosystem and in particular to Vertex AI, Google’s enterprise-ready generative AI developer platform.

“[We have] this idea of ‘growing with Google,’ where you can get in, build something, actually run it, deploy it, let people use it, and have that generous free tier. But then we’re also shipping a whole set of SDKs that enable developers to run and build apps with Gemini Pro that can run pretty much everywhere, from the back-end with support for Node.js and Python, to mobile, with support for Java, Kotlin, and Swift, and to web, of course, with JavaScript,” she explained.”

She noted that the team wants this transition between going from AI Studio and to Vertex to be as seamless as possible. Woodward added that the strong SDK support came directly from user feedback. “The first version we showed people, they said: ‘I love how easy it is to prompt. Now I want to go to code.’ And there was sort of this cliff that we had to fill in,” he told me.

Talking about the overall ecosystem, Banks also explained that Google plans to bring Gemini to the Chrome Dev Tools and Google’s Firebase mobile development platform early next year.

Given the speed at which generative AI is developing, it’s hard to even predict what developers will want to use these tools for next, but Banks and Woodward stressed that Google plans to build out AI Studio as an easy onramp for developers of all skill levels.

“I hope that AI Studio, in some ways, won’t just be seen as a prompting tool or something that only developers go to, but it’s actually, in some ways, a developer and creativity tool, where people can come up with ideas for working with these models and all the capabilities that will come out over the next year or so,” Woodward said.

OpenAI Partners with Axel Springer, Enriching ChatGPT with POLITICO and BUSINESS INSIDER Content

OpenAI announced that it has partnered with Axel Springer, a media and technology company, to strengthen independent journalism in the age of AI. This initiative seeks to enhance users’ interactions with ChatGPT by integrating recent and authoritative content across a broad spectrum of topics. The collaboration explicitly recognises the pivotal role of publishers in contributing to OpenAI’s products.

We have formed a new global partnership with @AxelSpringer and its news products.
Real-time information from @politico, @BusinessInsider, European properties @BILD and @welt, and other publications will soon be available to ChatGPT users.
ChatGPT’s answers to user queries will…

— OpenAI (@OpenAI) December 13, 2023

As a result of this collaboration, ChatGPT users globally can now access summaries of selected global news content from Axel Springer’s media brands, including POLITICO, BUSINESS INSIDER, and European properties such as BILD and WELT. Notably, this encompasses otherwise paid content, enriching the user experience with valuable insights.

“This partnership with Axel Springer will help provide people with new ways to access quality, real-time news content through our AI tools. We are deeply committed to working with publishers and creators around the world and ensuring they benefit from advanced AI technology and new revenue models,” said Brad Lightcap, COO of OpenAI.

Transparency is a key element of this partnership. ChatGPT’s responses to user queries will feature attribution and links to the full articles, ensuring openness and providing users with avenues for further information.

Beyond content delivery, the partnership extends its support to Axel Springer’s ongoing AI-driven ventures that leverage OpenAI’s technology. This collaboration also involves the utilisation of quality content from Axel Springer’s media brands to advance the training of OpenAI’s sophisticated large language models.

“We are excited to have shaped this global partnership between Axel Springer and OpenAI – the first of its kind. We want to explore the opportunities of AI empowered journalism – to bring quality, societal relevance and the business model of journalism to the next level,”said Mathias Döpfner, CEO of Axel Springer.

The post OpenAI Partners with Axel Springer, Enriching ChatGPT with POLITICO and BUSINESS INSIDER Content appeared first on Analytics India Magazine.

5 Rare Data Science Skills That Can Help You Get Employed

5 Rare Data Science Skills That Can Help You Get Employed
Image by Author

If you know how to create a machine learning decision tree, congratulations, you have the same level of code expertise as ChatGPT and the thousands of other data scientists competing for the job you want.

One fascinating trend among hiring managers lately is that raw coding ability just doesn’t cut it anymore. To get hired, you need to go a step above knowing languages, frameworks, and how to search on StackOverflow. You need far more conceptual understanding, and a grasp of today’s data science landscape – including things you think only the CEO of a company should be worried about, like data governance and ethics.

There are many technical and non-technical data science skills that you should know but If you’re having a hard time getting hired, these less common data science skills might be the ticket to getting your foot in the employment door.

1. Model visualization

Previously, data scientists worked in isolation, in dark underground basements producing models. The models would create predictions or insights; those would be passed onto C-suite execs who would act on them with no understanding of the model that had produced these predictions. (I’m exaggerating a little, but not by that much.)

Today, leadership takes a much more active role in understanding the products of data scientists. That means that you, as a data scientist, need to be able to explain why models do what they do, how they work, and why they came up with that particular prediction.

While you could show your boss the actual code running your model, it’s much more useful (read: employable) to be able to show them how your model works through visualization. For example, imagine you've developed an ML model that predicts customer churn for a telecom company. Instead of a screenshot of your lines of code, you could use a flowchart or decision tree diagram to visually explain how the model segments customers and identifies those at risk of churning. This makes the model's logic transparent and easier to grasp.

Knowing how to illustrate code is a rare skill, but certainly one worth developing. There aren’t any courses yet, but I recommend you try a free tool like Miro to create a flowchart documenting your decision tree. Better yet, try to explain your code to a non-data scientist friend or family member. The more lay, the better.

2. Feature engineering 5 Rare Data Science Skills That Can Help You Get Employed
Image by Author

Many data scientists tend to focus more on model algorithms than on the nuances of the input data. Feature engineering is the process of selecting, modifying, and creating features (input variables) to improve the performance of machine learning models.

For example, if you're working on a predictive model for real estate prices, you might start with basic features like square footage, number of bedrooms, and location. However, through feature engineering, you could create more nuanced features. You might calculate the distance to the nearest public transport station or create a feature that represents the age of the property. You could even combine existing features to create new ones, such as a "location desirability score" based on crime rates, school ratings, and proximity to amenities.

It’s a rare skill because it requires not just technical know-how, but also deep domain knowledge and creativity. You need to really get your data and the problem at hand, and then creatively transform the data to make it more useful for modeling.

Feature engineering is often covered as part of broader machine learning courses on platforms like Coursera, edX, or Udacity. But I find the best way to learn is through hands-on experience. Work on real-world data and experiment with different feature engineering strategies.

3. Understanding data governance

Here is a hypothetical question: imagine you're a data scientist at a healthcare company. You’ve been tasked with developing a predictive model to identify patients at risk of a certain disease. What is likely to be your biggest challenge?

If you answered, “grappling with ETL pipelines,” you’re wrong. Your biggest challenge is likely to be making sure your model is not only effective but also compliant, ethical, and sustainable. That includes ensuring that any data you collect for the model complies with regulations like HIPAA and GDPR, depending on your location. You need to know when it’s even legal to use that data, how you need to anonymize it, what consent you require from patients, and how to get that consent.

And you need to be able to document data sources, transformations, and model decisions so that a non-expert would be able to audit the model. This traceability is vital not just for regulatory compliance but also for future model audits and improvements.

Where to learn data governance: It’s dense, but one great resource is the Global Data Management Community.

5 Rare Data Science Skills That Can Help You Get Employed
Image from dataedo 4. Ethics

“I know data science basically can know statistics, create models, find trends, but if you asked me, I couldn't think of any real ethical dilemmas, I think data science just spills out the real facts,” said Reddit user Carlos_tec17, wrongly.

Beyond legal compliance, there's an ethical aspect to consider. You need to ensure that any model you create doesn't inadvertently introduce biases that could lead to unequal treatment of certain groups.

I love the example of Amazon’s old recruitment model to illustrate why ethics matter. If you’re not familiar with it, Amazon data scientists tried to speed up their hiring workflow by creating a model that could pick out potential hires based on resumes. The problem was that they trained the model on their existing base of resumes, which was very male-dominated. Their new model was biased towards male hires. That is extremely unethical.

We are so far past the “move fast and break things” stage of data science. Now, as a data scientist, you need to know that your decisions will have a real impact on people. Ignorance is no longer an excuse; you need to be fully aware of all the possible ramifications your model could have, and why it makes the decisions it makes.

UMichigan has a helpful course on “data science ethics.” I also liked this book to illustrate why and how ethics crop up in even “number-based” science like data science.

5. Marketing

One secret life hack is that the better you know how to market, the easier you’ll find it to get a job. And by “market,” I mean “know how to make things sexy.” With the ability to market, you’ll be better at making a resume that sells your skills. You’ll be better at charming an interviewer. And in data science specifically, you’ll be better at explaining why your model – and the results of your model – matter.

Remember, it doesn’t matter how good your model is if you can’t convince anyone else it’s necessary. For example, imagine you've developed a model that can predict equipment failures in a manufacturing plant. In theory, your model could save the company millions in unplanned downtime. But if you can't communicate that fact to the C-suite, your model will languish unused on your computer.

With marketing skills, you can prove your use and the need for your model with a compelling presentation that highlights the financial benefits, the potential for increased productivity, and the long-term advantages of adopting your model.

This is a very rare skill in the data science world because most data scientists are numbers people at heart. Most would-be data scientists really believe that simply doing your best and keeping your head down is a winning employment strategy. Unfortunately, computers are not the ones hiring you – people are. Being able to market yourself, your skills, and your products is a real advantage in today’s job market.

To learn how to market, I recommend a few beginner, free courses like “Marketing in a Digital World,” offered by Coursera. I especially liked the section on “Offering product ideas that stick in a digital world.” There aren’t any data science-specific marketing courses out there, but I liked this blog post that walks through how to market yourself as a data scientist.

Final Thoughts

It’s tough out there. Despite there being a projected growth of data scientist employment, according to the Bureau of Labor Statistics, many more entry-level data science aspirants are finding it hard to land a job, as these Reddit posts illustrate. There’s competition from ChatGPT and the layoff vultures are circling.

To compete and stand out in the job market, you have to go above just technical chops. Data governance, ethics, model viz, feature engineering, and marketing skills make you a more thoughtful, robust, and intriguing candidate for hiring managers.

Nate Rosidi is a data scientist and in product strategy. He's also an adjunct professor teaching analytics, and is the founder of StrataScratch, a platform helping data scientists prepare for their interviews with real interview questions from top companies. Connect with him on Twitter: StrataScratch or LinkedIn.

More On This Topic

  • Data Science Projects That Can Help You Solve Real World Problems
  • If You Can Write Functions, You Can Use Dask
  • AI in Dating: Can Algorithms Help You Find Love?
  • Data Science Portfolio Project Ideas That Can Get You Hired (Or Not)
  • How Data Scientists Can Get the Ear of CFOs (And Why You Want It)
  • Data access is severely lacking in most companies, and 71% believe…

7 Wittiest AI Memes of 2023

Memes have become an integral part of online culture within the broad and constantly changing internet. These oddball, frequently humorous images combined with astute commentary have evolved into the digital equivalent of a secret handshake, providing a quick and efficient means of communication for people to connect, exchange ideas, and convey a whole range of human emotions.

However, this virtual story now has a twist! AI has transformed meme production into a whole new territory. Memes are firmly establishing themselves as the undisputed kings of internet humor, but they’re more than just timely jokes —they’re a science and they’re art. To master the meme game, producers must strike a careful balance between things that make people laugh and things that could set off the internet’s laughter detector.

Let’s catch a few AI memes that went viral this year:

Sam’s not f**king leaving

This meme perfectly captures Sam Altman’s return after being ousted by the OpenAI board.


Altman’s office records at OpenAI now featured an infamous three-day holiday. So, hold on to your AI systems, meme engineers – for this was a special case of pink slip and rehire.

One course is all you need


When you’re halfway through your PhD in machine learning, you realize that you and Andrew Ng are practically best friends because of his online courses. Read more about the latest course by Ng, your genius friend here.

It’s all the same thing

When using ChatGPT to play mind games, you may display the same image with different faces, and it will react by saying things like, “Wait a byte! Did I just see a face-making pixel? The AI world of made up perplexity got interesting in 2023 with the ultimate face-show.

We go hand in hand

When your illustration abilities are a work of abstract wonder, do not be alarmed. With AI art generators like Emu Edit, Imagen, DALL-E 3, Stable Diffusion, and Midjourney, AI has you covered. With the help of these AI tools that can instantly transform doodles into masterpieces through neural network processing, even your stick figures can now shine on the digital canvas.

You got it right


Across the globe, it’s a “large language model”, but stroll down the boulevards of Paris, just “le big model”. Because in the City of Light, even our AI speaks the language of chic sophistication.

Late realisation


When you realize ChatGPT can do your job, and you’re like, “Wait, is my role just a glorified conversation starter?” You’re left wondering if you’re the one being helped or the one providing assistance when ChatGPT begins creating its own job description.

What did he just say?

Murthy’s move: Unintentional joke or 4D chess? Workers contemplate, as the timer continues to run.

How are meme generators powered by AI?

AI and machine learning techniques are utilized by AI meme generators to produce memes in an automated manner. These generators examine pre-existing memes and produce new ones based on discovered patterns by utilizing a variety of methods, including computer vision and natural language processing.

AI systems learn comedy, trends, and the cultural context needed to create memes that connect with viewers by being trained on enormous volumes of meme data.

The post 7 Wittiest AI Memes of 2023 appeared first on Analytics India Magazine.

This AI Meeting Assistant is Now Just $34.97 for a 1-Year Subscription

Laxis AI Meeting Assistant subscription.
Image: StackCommerce

We communicate through many channels nowadays, but business happens face-to-face. Getting client calls and meetings right is essential if you want to close deals consistently. Laxis AI is intuitive software that makes meeting management and recall easier and better than ever.

This powerful AI tool allows you to query a database of past conversations in real-time and draft follow-up notes in seconds. In a limited-time price drop, you can get a one-year subscription for only $34.97 at TechRepublic Academy — an attractive last-minute gift option for business professionals.

Think about the last time you booked a meeting with a client. You probably sent out an invitation, including a few notes about the topics to be discussed. During the meeting, you either had someone take notes or you recorded the call. And afterward, you sent a personalized follow-up.

All these tasks take time. With Laxis AI, you can speed things up drastically.

Using cutting-edge artificial intelligence, Laxis AI crafts customized invitations on your behalf. It works with Zoom, Microsoft Teams, Google Meet and other platforms.

When the meeting begins, Laxis AI automatically records and transcribes everything. It works on both regular phone calls and video meetings. At the same time, you can interact with the AI to access information from previous conversations with the same contact.

You also have the option to record your own audio and upload it after the meeting. Laxis AI will perform the same analysis and pull out key insights.

Existing users seem to love it. Laxis AI has a perfect 5-star rating on Product Hunt, along with 4.9/5 stars on G2 and 4.7/5 stars on Capterra.

Order by midnight on Christmas Day to get a Laxis AI Meeting Assistant: 1-Yr Premium Subscription covering up to 2,000 transcription mins/month for only $34.97 — a huge saving worth $125.

Prices and availability are subject to change.

Is AlphaCode 2 a Q* Moment for Google?

Google DeepMind last week released AlphaCode 2, an update to AlphaCode, along with Gemini. This version has improved problem-solving capabilities for competitive programming. Last year when AlphaCode was released, it was compared to Tabnine, Codex and Copilot. But with this update AlphaCode definitely stands way ahead.

AlphaCode 2 approaches problem-solving by using a set of “policy models” that produce various code samples for each problem. It then eliminates code samples that don’t match the problem description. AlphaCode 2 employs a multimodal approach that integrates data from diverse sources, including web documents, books, coding resources, and multimedia content.

This approach has been compared to the curious Q* from OpenAI. Instead of being a tool that regurgitates information, Q* is rumoured to be able to solve maths problems it has previously not seen before. The technology is speculated to be an advancement in solving basic maths problems, a challenging task for existing AI models.

Now while Q* is only a speculation, AlphaCode performed better than 85% of competitors on average. It solved 43% of problems within 10 attempts across 12 coding contests with more than 8,000 participants, doubling the success rate as the original AlphaCode’s success rate was 25%.

However, like any AI model, AlphaCode 2 has its limitations. The whitepaper notes that AlphaCode 2 involves substantial trial and error, operates with high costs at scale, and depends significantly on its ability to discard clearly inappropriate code samples. The whitepaper suggests that upgrading to a more advanced version of Gemini, such as Gemini Ultra, could potentially address some of these issues.

What sets AlphaCode 2 apart

The AlphaCode 2 Technical Report presents significant improvements. Enhanced by the Gemini model, AlphaCode 2 solves 1.7 times more problems and surpasses 85% of participants in competitive programming. Its architecture includes powerful language models, policy models for code generation, mechanisms for diverse sampling, and systems for filtering and clustering code samples.

To reduce redundancy, a clustering algorithm groups together code samples that are “semantically similar.” The final step involves a scoring model within AlphaCode 2, which identifies the most suitable solution from the largest 10 clusters of code samples, forming AlphaCode 2’s response to the problem.

The fine-tuning process involves two stages using the GOLD training objective. The system generates a vast number of code samples per problem, prioritising C++ for quality. Clustering and a scoring model help in selecting optimal solutions.

Tested on Codeforces, AlphaCode 2 shows remarkable performance gains. However, the system still faces challenges in trial and error and operational costs, marking a significant advancement in AI’s role in solving complex programming problems.

When compared to other code generators, AlphaCode 2, unlike its counterparts, shows a unique strength in competitive programming. On the other hand, GitHub Copilot, powered by OpenAI Codex, serves as a broader coding assistant. Codex, an AI system developed by OpenAI, is particularly adept at code generation due to its training on a vast array of public source code.

In the emerging field, other notable tools like EleutherAI’s Llemma and Meta’s Code Llama bring their distinct advantages. Llemma, with its 34-billion parameter model, specialises in mathematics, even outperforming Google’s Minerva. Code Llama, based on Llama 2, focuses on enabling open-source development of AI coding assistants, offering a unique advantage in creating company-specific AI tools.

AlphaCode 2 has a different approach compared to other AI coding tools. It uses machine learning, code sampling, and problem-solving strategies for competitive programming. These features are tailored for complex coding problems. Other tools like GitHub Copilot and EleutherAI’s Llemma focus on general coding help and maths problems.

A Close Contest

For OpenAI, Q* represents a significant advancement in AI capable of solving maths problems it hadn’t seen before. This breakthrough, involving Sutskever’s work, led to the creation of models with enhanced problem-solving abilities.

However, the rapid advancement in this technology has raised concerns within OpenAI about the pace of progress and the need for adequate safeguards for such powerful AI models.

While both AlphaCode 2 by Google DeepMind and the speculated Q* represent significant advancements in AI, they are not yet widely available to the public.

The post Is AlphaCode 2 a Q* Moment for Google? appeared first on Analytics India Magazine.

Undersampling Techniques Using Python

With the evolving digital landscape, a wealth of data is being generated and captured from diverse sources. While immensely valuable, this vast universe of information often reflects the imbalanced distribution of real-world phenomena. The problem of imbalanced data is not merely a statistical challenge; it has far-reaching implications for the accuracy and reliability of the data-driven models.

Take, for example, the ever-growing and prevalent concern of fraud detection in the financial industry. As much as we want to avoid fraud due to its highly damaging nature, machines (and even humans) inevitably need to learn from the examples of fraudulent transactions (albeit rare) to distinguish them from the number of daily legitimate transactions.

This imbalance in data distribution between fraudulent and non-fraudulent transactions poses significant challenges for the machine-learning models aimed at detecting such anomalous activities. Without appropriate handling of the data imbalance, these models risk becoming biased toward predicting transactions as legitimate, potentially overlooking the rare instances of fraud.

Healthcare is another field where machine learning models are leveraged to predict imbalanced outcomes, such as diseases like cancer or rare genetic disorders. Such outcomes occur far less frequently than their benign counterparts. Hence, the models trained on such imbalanced data are more susceptible to incorrect predictions and diagnoses. Such missed health alert defeats the purpose of the model in the first place, i.e., to detect early disease.

These are just a few instances highlighting the profound impact of data imbalance, i.e., where one class significantly outnumbers the other. Oversampling and Undersampling are two standard data preprocessing techniques to balance the dataset, of which we will focus on undersampling in this article.

Let us discuss some popular methods for undersampling a given distribution.

Understanding the Downside of Imbalance

Let's start with an illustrative example to grasp the significance of under-sampling techniques better. The following visualization demonstrates the impact of the relative quantity of points per class, as executed by a Support Vector Machine with a linear kernel. The below code and plots are referred from the Kaggle notebook.

import matplotlib.pyplot as plt  from sklearn.svm import LinearSVC  import numpy as np  from collections import Counter  from sklearn.datasets import make_classification      def create_dataset(      n_samples=1000, weights=(0.01, 0.01, 0.98), n_classes=3, class_sep=0.8, n_clusters=1  ):      return make_classification(          n_samples=n_samples,          n_features=2,          n_informative=2,          n_redundant=0,          n_repeated=0,          n_classes=n_classes,          n_clusters_per_class=n_clusters,          weights=list(weights),          class_sep=class_sep,          random_state=0,      )      def plot_decision_function(X, y, clf, ax):      plot_step = 0.02      x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1      y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1      xx, yy = np.meshgrid(          np.arange(x_min, x_max, plot_step), np.arange(y_min, y_max, plot_step)      )        Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])      Z = Z.reshape(xx.shape)      ax.contourf(xx, yy, Z, alpha=0.4)      ax.scatter(X[:, 0], X[:, 1], alpha=0.8, c=y, edgecolor="k")      fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2, 2, figsize=(15, 12))    ax_arr = (ax1, ax2, ax3, ax4)  weights_arr = (      (0.01, 0.01, 0.98),      (0.01, 0.05, 0.94),      (0.2, 0.1, 0.7),      (0.33, 0.33, 0.33),  )  for ax, weights in zip(ax_arr, weights_arr):      X, y = create_dataset(n_samples=1000, weights=weights)      clf = LinearSVC().fit(X, y)      plot_decision_function(X, y, clf, ax)      ax.set_title("Linear SVC with y={}".format(Counter(y)))  

The code above generates plots for four different distributions starting from a highly imbalanced dataset with one class dominating 97% of the instances. The second and third plots have 93% and 69% of the instances from a single class, respectively, while the last plot has a perfectly balanced distribution, i.e., all three classes contribute a third of the instances. Plots of the datasets from the most imbalanced to the least are displayed below. Upon fitting SVM over this data, the hyperplane in the first plot (highly imbalanced) is pushed to a side of the chart, mainly because the algorithm treats each instance equally, irrespective of the class, and tries to separate the classes with maximum margin. Hence, a majority yellow population near the center pushes the hyperplane to the corner, making the algorithm misclassify the minority classes.

Undersampling Techniques Using Python

The algorithm successfully classifies all interest classes as we move towards a more balanced distribution.

In summary, when a dataset is dominated by one or a few classes, the resulting solution often results in a model with higher misclassifications. However, the classifier exhibits diminishing bias as the distribution of observations per class approaches an even split.

In this case, undersampling the yellow points presents the simplest solution to address model errors originating from the problem of rare classes. It's worth noting that not all datasets encounter this issue, but for those that do, rectifying this imbalance forms a crucial preliminary step in modeling the data.

Imbalanced-Learn Library

We'll use the Imbalanced-Learn Python library (imbalanced-learn or imblearn). We can install it using pip:

pip install -U imbalanced-learn

Hands-On!

Let us discuss and experiment with some of the most popular undersampling techniques. Suppose you have a binary classification dataset where class '0' significantly outnumbers class '1'.

NearMiss Undersampling

NearMiss is an undersampling technique that reduces the number of majority samples closer to the minority class. This would facilitate clean classification by any algorithm using space separation or splitting the dimensional space between the two classes. There are three versions of NearMiss:

NearMiss-1: Majority class samples with a minimum average distance to the three closest minority class samples.

NearMiss-2: Majority class samples with a minimum average distance to three furthest minority class samples.

NearMiss-3: Majority class samples with minimum distance to each minority class sample.

Let’s demonstrate the NearMiss-1 undersampling algorithm through a code example:

# Import necessary libraries and modules  import numpy as np  import matplotlib.pyplot as plt  from collections import Counter  from sklearn.datasets import make_classification  from imblearn.under_sampling import NearMiss    # Generate the dataset with different class weights  features, labels = make_classification(      n_samples=1000,      n_features=2,      n_redundant=0,      n_clusters_per_class=1,      weights=[0.95, 0.05],      flip_y=0,      random_state=0,  )    # Print the distribution of classes  dist_classes = Counter(labels)  print("Before Undersampling:")  print(dist_classes)    # Generate a scatter plot of instances, labeled by class  for class_label, _ in dist_classes.items():      instances = np.where(labels == class_label)[0]      plt.scatter(features[instances, 0], features[instances, 1], label=str(class_label))  plt.legend()  plt.show()    # Set up the undersampling method  undersampler = NearMiss(version=1, n_neighbors=3)    # Apply the transformation to the dataset  features, labels = undersampler.fit_resample(features, labels)    # Print the new distribution of classes  dist_classes = Counter(labels)  print("After Undersampling:")  print(dist_classes)    # Generate a scatter plot of instances, labeled by class  for class_label, _ in dist_classes.items():      instances = np.where(labels == class_label)[0]      plt.scatter(features[instances, 0], features[instances, 1], label=str(class_label))  plt.legend()  plt.show()

Undersampling Techniques Using Python

Change version=1 to version=2 or version=3 in the NearMiss() class to use the NearMiss-2 or NearMiss-3 undersampling algorithm.

Undersampling Techniques Using Python Undersampling Techniques Using Python

NearMiss-2 selects instances at the core of the overlap region between the two classes. With the NeverMiss-3 algorithm, we observe that every instance in the minority class, which overlaps with the majority class region, has up to three neighbors from the majority class. The attribute n_neighbors in the code sample above defines this.

Condensed Nearest Neighbor (CNN) Rule

This method starts by considering a subset of the majority class as noise. Then, it uses a 1-Nearest Neighbor algorithm to classify instances. If an instance from the majority class is misclassified, it's included in the subset. The process continues until no more instances are included in the subset.

from imblearn.under_sampling import CondensedNearestNeighbour    cnn = CondensedNearestNeighbour(random_state=42)  X_res, y_res = cnn.fit_resample(X, y)

Tomek Links Undersampling

Tomek Links are closely located pairs of opposite-class instances. Removing the instances of the majority class of each pair increases the space between the two classes, facilitating the classification process.

from imblearn.under_sampling import TomekLinks    tl = TomekLinks()  X_res, y_res = tl.fit_resample(X, y)    print('Original dataset shape:', Counter(y))  print('Resample dataset shape:', Counter(y_res))

With this, we have delved into the essential aspects of undersampling techniques in Python, covering three prominent methods: Near Miss Undersampling, Condensed Nearest Neighbour, and Tomek Links Undersampling.

Undersampling is a crucial data processing step to address class imbalance problems in machine learning and also helps improve the model performance and fairness. Each of these techniques offers unique advantages and can be tailored to specific datasets and the goals of machine learning projects.

This article provides a comprehensive understanding of the undersampling methods and their application in Python. I hope it enables you to make informed decisions on tackling class imbalance challenges in your machine-learning projects.

Vidhi Chugh is an AI strategist and a digital transformation leader working at the intersection of product, sciences, and engineering to build scalable machine learning systems. She is an award-winning innovation leader, an author, and an international speaker. She is on a mission to democratize machine learning and break the jargon for everyone to be a part of this transformation.

More On This Topic

  • Exploring Data Cleaning Techniques With Python
  • Ensemble Learning Techniques: A Walkthrough with Random Forests in Python
  • 7 Techniques to Handle Imbalanced Data
  • Knowledge Graph Conference, covering tools, techniques, case…
  • Guide To Finding The Right Predictive Maintenance Machine Learning…
  • Visual Scoring Techniques for Classification Models

Snezhana is Bringing Tech into Fashion and Films the Perfect Way

Snezhana is Bringing Tech into Fashion and Films The Perfect Way

Born in Russia, Snezhana Paderina is an art director in fashion and technology. Based out of New York, she works with CG cinematics and 3D designing as a fashion film director. With a drive to learn design in a US university at a time when English was not her first language, Paderina combined her technical know-how to help her get through.

“Imagine coming from a tech background and going to art school into the fashion industry in the US,” said Padreina, sitting at her office in Turkey and a Meta Quest headset sitting on the shelf behind her, in an exclusive interview with AIM.

“I had to write these long essays on art. So, I created a neural network to do it,” she spoke about her experience at design school. Being a big fan of Cyberpunk, she used William Gibson‘s books on Blade Runner and Neuromancer to build a bot that could write essays for her. “That’s how I first used AI.

Paderina said that she always wanted to be a programmer. “When I was eight years old, I watched The Matrix for the first time, and that is how I got into computers. I never wanted to go into fashion.”

Currently, Paderina works at DeV Virtual Production as an Art and Cinematics Director where she uses Unreal Engine for AR, VR, and 3D filmmaking, along with fashion and game development. Her website Snezhana.nyc also highlights her love for using tech in fashion and film.

Paderina completed her education in cyber security but was always inclined towards using it in some creative ways. One fine day she got a sewing machine and suddenly realised how her experience with programming and coding could actually augment the fashion industry. Then she got into fashion.

“It doesn’t matter if it’s fashion or film, it is like a canvas for me,” said Paderina. In 2013, Paderina did her first fashion show, where she presented her CGI and 3D printing collection. “I realised that I needed some education in fashion,” she added laughingly. She then joined Parson School of Design in New York.

“Using the Unreal Engine in filmmaking allowed me to combine everything that I have learnt as there are no constraints like in real-life film making.” Being an avid gamer with games such as Cyberpunk, she also aims to go into developing games in the future.

“I think like a tech person, more than anything else”

In 2018, when NVIDIA was announcing the launch of its RTX series GPUs, the Russian branch of the company contacted Paderina for a collaboration for the fashion industry at the Mercedes Benz Fashion Week, where she made her ‘RAY’ collection. “From a designer’s point of view, it was very challenging because I had to create a collection that would be appealing to both audiences, which are very different gamers, and fashion insiders,” she explained.

RAY collection from Mercedes Benz Fashion Week

Today, Paderina uses Midjourney to create storyboards for her movies. “It’s the fastest way to convey my vision to the team,” she added. She also uses ChatGPT to analyse scripts. “I put my scripts in ChatGPT, and ask it to analyse it from a viewer standpoint to find weak spots, and actually it gives very good results by suggesting weak spots in the character arc and many more things.”

“I feel that people should learn how to utilise AI,” Paderina talks about the fear of AI. “I see a lot of artists who feel threatened by AI because Midjourney can draw better than us,” she laughed. “I spent 15 years drawing and now I do it in seconds. But these people feel that AI is a threat, they are kind of closing themselves to it, instead of adopting it in their work, just as they did with sewing machines to remove the tedious work.”

In a wearable tech competition in Spain, Paderina with her collaborator, Nikita Replyanski, got first place for building GS[3], which is a corset with a graduated spine support system provides dynamic back support for patients suffering from medical conditions which cause joint hypermobility and chronic musculoskeletal issues that require daily spinal support.

GS[3] corset

“We made this corset very fashionable on purpose because we wanted to normalise medical devices as part of your outfit, instead of hiding under clothes,” Paderina explained.

Embracing technology is all we need

“With the team in 3D, we also use AI for motion capture and taking visual reference,” Paderina continued. “You put the models through AI and you get the motion capture data for 3D characters animation.” The team also uses AI on Unreal Engine for generating textures and meshes, which Paderina highlights is pretty groundbreaking.

Furthermore, she highlighted that with the development of deep fake, it is very interesting to see the future of entire films being made just using AI with actors, scripts, sound, and even speech. “I think that is what people in Hollywood are afraid of and are giving a pushback to AI in the field.”

When it comes to fashion, “AR has a very big potential because the fashion industry is very dependent on consumers,” she added. “It would be so cool for fashion if you could see other people’s clothes changing. For example, you wear a t-shirt with the marker that you want, and other people see changes on your clothes when they see them through glasses.”

“I expect a lot from Apple Vision Pro and other such upcoming wearables,” Paderina spoke about the future of AR/VR in films and fashion. “There is something really good about interaction and interactivity that these gadgets can provide, as we saw with games.”

The post Snezhana is Bringing Tech into Fashion and Films the Perfect Way appeared first on Analytics India Magazine.