Giga ML wants to help companies deploy LLMs offline

Giga ML wants to help companies deploy LLMs offline Kyle Wiggers 9 hours

AI is all the rage — particularly text-generating AI, also known as large language models (think models along the lines of ChatGPT). In one recent survey of ~1,000 enterprise organizations, 67.2% say that they see adopting large language models (LLMs) as a top priority by early 2024.

But barriers stand in the way. According to the same survey, a lack of customization and flexibility, paired with the inability to preserve company knowledge and IP, were — and are — preventing many businesses from deploying LLMs into production.

That got Varun Vummadi and Esha Manideep Dinne thinking: What might a solution to the enterprise LLM adoption challenge look like? In search of one, they founded Giga ML, a startup building a platform that lets companies deploy LLMs on-premise — ostensibly cutting costs and preserving privacy in the process.

“Data privacy and customizing LLMs are some of the biggest challenges faced by enterprises when adopting LLMs to solve problems,” Vummadi told TechCrunch in an email interview. “Giga ML addresses both of these challenges.”

Giga ML offers its own set of LLMs, the “X1 series,” for tasks like generating code and answering common customer questions (e.g. “When can I expect my order to arrive?”). The startup claims the models, built atop Meta’s Llama 2, outperform popular LLMs on certain benchmarks, particularly the MT-Bench test set for dialogs. But it’s tough to say how X1 compares qualitatively; this reporter tried Giga ML’s online demo but ran into technical issues. (The app timed out no matter what prompt I typed.)

Even if Giga ML’s models are superior in some aspects, though, can they really make a splash in the ocean of open source, offline LLMs?

In talking to Vummadi, I got the sense that Giga ML isn’t so much trying to create the best-performing LLMs out there but instead building tools to allow businesses to fine-tune LLMs locally without having to rely on third-party resources and platforms.

“Giga ML’s mission is to help enterprises safely and efficiently deploy LLMs on their own on-premises infrastructure or virtual private cloud,” Vummadi said. “Giga ML simplifies the process of training, fine-tuning and running LLMs by taking care of it through an easy-to-use API, eliminating any associated hassle.”

Vummadi emphasized the privacy advantages of running models offline — advantages likely to be persuasive for some businesses.

Predibase, the low-code AI dev platform, found that less than a quarter of enterprises are comfortable using commercial LLMs because of concerns over sharing sensitive or proprietary data with vendors. Nearly 77% of respondents to the survey said that they either don’t use or don’t plan to use commercial LLMs beyond prototypes in production — citing issues relating to privacy, cost and lack of customization.

“IT managers at the C-suite level find Giga ML’s offerings valuable because of the secure on-premise deployment of LLMs, customizable models tailored to their specific use case and fast inference, which ensures data compliance and maximum efficiency,” Vummadi said.

Giga ML, which has raised ~$3.74 million in VC funding to date from Nexus Venture Partners, Y Combinator, Liquid 2 Ventures, 8vdx and several others, plans in the near term to grow its two-person team and ramp up product R&D. A portion of the capital is going toward supporting Giga ML’s customer base, as well, Vummadi said, which currently includes unnamed “enterprise” companies in finance and healthcare.

Apple’s Leap into the AI Frontier: Navigating the MLX Framework and Its Impact on Next-Gen MacBook AI Experiences

The realm of artificial intelligence is currently experiencing a significant transformation, driven by the widespread integration and accessibility of generative AI within open-source ecosystems. This transformative wave not only enhances productivity and efficiency but also fosters innovation, providing a vital tool for staying competitive in the modern era. Breaking away from its traditional closed ecosystem, Apple has recently embraced this paradigm shift by introducing MLX, an open-source framework designed to empower AI developers to efficiently harness the capabilities of Apple Silicon chips. In this article, we will take a deep dive into the MLX framework, unravelling its implications for Apple and the potential impact it holds for the broader AI ecosystem.

Unveiling MLX

Developed by Apple's Artificial Intelligence (AI) research team, MLX stands as a cutting-edge framework tailored for AI research and development on Apple silicon chips. The framework encompasses a set of tools that empowers AI developers to create advanced models, spanning chatbots, text generation, speech recognition, and image generation. MLX goes beyond by including pretrained foundational models like Meta’s LlaMA for text generation, Stability AI’s Stable Diffusion for image generation, and OpenAI’s Whisper for speech recognition.

Inspired by well-established frameworks such as NumPy, PyTorch, Jax, and ArrayFire, MLX places a strong emphasis on user-friendly design and efficient model training and deployment. Noteworthy features include user-friendly APIs, including a Python API reminiscent of NumPy, and a detailed C++ API. Specialized packages like mlx.nn and mlx.optimizers streamline the construction of complex models, adopting the familiar style of PyTorch.

MLX utilizes a deferred computation approach, generating arrays only when necessary. Its dynamic graph construction capability enables the spontaneous generation of computation graphs, guaranteeing that alterations to function argument do not hinder performance, all while keeping the debugging process straightforward and intuitive. MLX offers a broad compatibility across devices by seamlessly performing operations on both CPUs and GPUs. A key aspect of MLX is its unified memory model, preserving arrays in shared memory. This unique feature facilitates seamless operations on MLX arrays across various supported devices, eliminating the need for data transfers.

Distinguishing CoreML and MLX

Apple has developed both CoreML and MLX frameworks to assist AI developers on Apple systems, but each framework has its own unique features. CoreML is designed for easy integration of pre-trained machine learning models from open-source toolkits like TensorFlow into applications on Apple devices, including iOS, macOS, watchOS, and tvOS. It optimizes model execution using specialized hardware components like the GPU and Neural Engine, ensuring accelerated and efficient processing. CoreML supports popular model formats such as TensorFlow and ONNX, making it versatile for applications like image recognition and natural language processing. An essential feature of CoreML is on-device execution, ensuring models run directly on the user's device without relying on external servers. While CoreML simplifies the integration of pre-trained machine learning models with Apple’s systems, MLX serves as a development framework specifically designed to facilitate the development of AI models on Apple silicon.

Analyzing Apple’s Motives Behind MLX

The introduction of MLX indicates that Apple is stepping into the expanding field of generative AI, an area currently dominated by tech giants such as Microsoft and Google. Although Apple has integrated AI technology, like Siri, into its products, the company has traditionally refrained from entering the generative AI landscape. However, the significant increase in Apple's AI development efforts in September 2023, with a particular emphasis on assessing foundational models for broader applications and the introduction of MLX, suggests a potential shift towards exploring generative AI. Analysts suggest that Apple could use MLX frameworks to bring creative generative AI features to its services and devices. However, in line with Apple's strong commitment to privacy, a careful evaluation of ethical considerations is expected before making any significant advancements. Currently, Apple has not shared additional details or comments on its specific intentions regarding MLX, MLX Data, and generative AI.

Importance of MLX Beyond Apple

Beyond Apple's world, MLX's unified memory model offers a practical edge, setting it apart from frameworks like PyTorch and Jax. This feature lets arrays share memory, making operations on different devices simpler without unnecessary data duplications. This becomes especially crucial as AI increasingly depends on efficient GPUs. Instead of the usual setup involving powerful PCs and dedicated GPUs with a lot of VRAM, MLX allows GPUs to share VRAM with the computer's RAM. This subtle change has the potential to quietly redefine AI hardware needs, making them more accessible and efficient. It also affects AI on edge devices, proposing a more adaptable and resource-conscious approach than what we're used to.

The Bottom Line

Apple's venture into the realm of generative AI with the MLX framework marks a significant shift in the landscape of artificial intelligence. By embracing open-source practices, Apple is not only democratizing advanced AI but also positioning itself as a contender in a field dominated by tech giants like Microsoft and Google. MLX's user-friendly design, dynamic graph construction, and unified memory model offer a practical advantage beyond Apple's ecosystem, especially as AI increasingly relies on efficient GPUs. The framework's potential impact on hardware requirements and its adaptability for AI on edge devices suggest a transformative future. As Apple navigates this new frontier, the emphasis on privacy and ethical considerations remains paramount, shaping the trajectory of MLX's role in the broader AI ecosystem.

Cybersecurity teams need new skills even as they struggle to manage legacy systems

Already facing a dearth of talent, cybersecurity teams now need additional skillsets to deal with the growing adoption of generative artificial intelligence (AI) and machine learning. This is further complicated by a threat landscape that continues to evolve and a widening attack surface that needs safeguarding, including legacy systems that organizations are finding tough to let go of.

As it is, they are struggling to hire enough cybersecurity talent.

Also: Security first in software? AI may help make this an everyday practice

While the number of cybersecurity professionals in Asia-Pacific grew 11.8% year-on-year to just under 1 million in 2023, the region still needs another 2.67 million to adequately secure digital assets. This cybersecurity workforce gap is a record high for the region, widening by 23.4%, according to the 2023 ISC2 Cybersecurity Workforce Study, which polled 14,865 respondents, including 3,685 from Asia-Pacific.

Worldwide, the gap grew 12.6% from 2022 to almost 4 million cybersecurity professionals, according to estimates by ISC2 (International Information Systems Security Certification Consortium), a non-profit association comprising certified cybersecurity professionals.

The global cybersecurity workforce currently is at 5.45 million, up 8.7% from 2022, and will need to almost double to hit full capacity, ISC2 said.

The association's CISO Jon France told ZDNET that the biggest gap is in Asia-Pacific, but there are promising signs that this is narrowing. Singapore, for instance, decreased its cybersecurity workforce gap by 34% this year. Another 4,000 professionals in the sector are needed to sufficiently protect digital assets, ISC2 projects.

Globally, 92% of cybersecurity professionals believe their organization has skills gaps in at least one area, including technical skills such as penetration testing and zero trust implementation, according to the study. Cloud security and AI and machine learning top the list of skills that companies lack, at 35% and 32%, respectively.

Also: Generative AI can easily be made malicious despite guardrails

This demand will continue to grow as organizations incorporate AI into more processes, further driving the need for cloud computing, and the need for both skillsets, France noted. It means cybersecurity professionals will need to understand how AI is integrated and secure the applications and workflows it powers, he said.

Left unplugged, gaps in cybersecurity skills and staff will result in teams being overloaded and this can lead to oversights in addressing vulnerabilities, he cautioned. Misconfiguration and falling behind security patches are among the most common mistakes that can lead to breaches, he added.

AI adoption driving the need for new skills

Things are likely to get more complex with the emergence of generative AI.

Tools such as ChatGPT and Stable Diffusion have enabled attackers to improve the credibility of messages and imagery, making it easier to fool their targets. This significantly improves the quality of phishing email and websites, said Jess Burn, principal analyst at Forrester, who contributes to the analyst firm's research on the role of CISOs and security talent management.

And while these tools help bad actors create and launch attacks on a greater scale, Burn noted that this does not change how defenders respond to such threats. "We expect cyberattacks to increase in number as they've done for years now, [but] the threats themselves are not novel," she said in an email interview. "Security practitioners already know how to identify, resolve, and mitigate them."

To stay ahead, though, security leaders should incorporate prompt engineering training for their team, so they can better understand how generative AI prompts function, the analyst said.

Also: Six skills you need to become an AI prompt engineer

She also underscored the need for penetration testers and red teams to include prompt-driven engagements in their assessment of solutions powered by generative AI and large language models.

They need to develop offensive AI security skills to ensure models are not tainted or stolen by cybercriminals seeking intellectual property. They also have to ensure sensitive data used to train these models are not exposed or leaked, she said.

In addition to the ability to write more convincing phishing email, generative AI tools can be manipulated to write malware despite limitations put in place to prevent this, noted Jeremy Pizzala, EY's Asia-Pacific cybersecurity consulting leader. He noted that researchers, including himself, have been able to circumvent ethical restrictions that guide platforms such as ChatGPT and prompt them to write malware.

Also: What is phishing? Everything you need to know to protect yourself from scammers

There also is potential for threat actors to build their own large language models, trained on datasets with known exploits and malware, and create a "super strain" of malware that is more difficult to defend against, Pizzala said in an interview with ZDNET.

This pivots to a broader debate about AI and the associated business risks, where many large language and AI models have inherent and in-built biases. Hackers, too, can target AI algorithms, strip out the ethics guidelines and manipulate them to do things they are not programmed to do, he said, referring to the risk of algorithm poisoning.

All of these risks stress the need for organizations to have a governance plan, with safeguards and risk management policies to guide their AI use, Pizzala said. These also should address issues such as hallucinations.

With the right guardrails in place, he noted that generative AI can benefit cyber defenders themselves. Deployed in a security operations center (SOC), for instance, chatbots can more quickly provide insights on security incidents, giving responses to prompts asked in simple language. Without generative AI, this would have required a series of complex queries and responses that security teams then needed time to decipher.

Also: AI safety and bias: Untangling the complex chain of AI training

AI lowers the entry level for cybersecurity skills. Without the aid of generative AI, organizations would need specialized experience to interpret data generated by traditional monitoring and detection tools at SOCs, he said. He noted that some organizations have started training and hiring based on this model of governance.

Echoing Burn's comments on the need for generative AI knowledge, Pizzala also urged companies to build up the relevant technical skillsets and knowledge of the underlying algorithms. While coding for machine learning and AI models is not new, such foundational skills still are short in supply, he said.

The growing adoption of generative AI also calls for a different lens from a cybersecurity point of view, he added, noting that there are data scientists who specialize in security. Such skillsets will need to evolve and continue to upskill, he said.

In Asia-Pacific, 44% also point to inadequate cybersecurity budget as the biggest challenge, compared to the global average of 36%, Pizzala said, citing EY's 2023 Global Cybersecurity Leadership survey.

Also: AI at the edge: 5G and the Internet of Things see fast times ahead

A widening attack surface is the most cited internal challenge, fuelled by the adoption of cloud computing at scale and the Internet of Things (IoT). With AI now paving new ways to infiltrate systems and third-party supply chain attacks still a concern, the EY consultant said it all adds up to an ever-growing attack surface.

Burn further noted: "Most organizations were not prepared for the rapid migration to cloud environments a few years ago and they've been scrambling to acquire cloud security skills ever since, often opting to work with MDR (managed detection and response) services providers to fill those gaps.

"There's also a need for more proficiency with API security given how ubiquitous APIs are, how many systems they connect, and how much data flows through them," the Forrester analyst said.

Also: Will AI hurt or help workers? It's complicated

To address these requirements, she said organizations are tapping the knowledge that security operations and software development or product security teams have on infrastructure and adjusting this for the new environments. "So it's about finding the right training and upskilling resources and giving teams the time to train," she added.

"Having an underskilled team can be as risky as having an understaffed one," she said. Citing Forrester's 2022 Business Technographics survey on data security, she said companies that had six or more data breaches in the past year were more likely to report the unavailability of security employees with the right skills as one of their biggest IT security challenges in the past 12 months.

Tech stacks need simplifying to ease security management

Should organizations engage managed security services providers to plug the gaps, Pizzala recommends they do so while remaining involved. Similar to a cloud management strategy, there should be shared responsibility, with the companies doing their own checks and scanning, he said.

He also supported the need for businesses to reassess their legacy systems and work to simplify their tech stack. Having too many cybersecurity tools in itself presents a risk, he added.

Operational technology (OT) sectors, in particular, have significant legacy systems, France said.

With a growing attack surface and complex digital and threat landscape, he expressed concerns for companies that are unwilling to let go of their legacy assets even as they adopt new technology. This increases the burden on their cybersecurity teams that have to continue monitoring and protecting old toolsets alongside newly acquired systems.

Also: What the 'new automation' means for technology careers

To plug the resource gap, Curtis Simpson, CISO for security vendor Armis, advocated the need to look at technology, such as automation and orchestration. Much of this will be powered by AI, he said.

"People won't help us close this gap. Technology will," Simpson said in a video interview.

Attacks are going to be AI-powered and continue to evolve, further stressing the need for orchestration and automation so companies can move quickly enough to respond to potential threats, he noted.

Defense in depth remains critical, which means organizations need to have complete visibility and understanding of their entire environment and risk exposure. This then enables them to have the necessary mediation plan and minimize the impact of a cyber attack when one occurs, Simpson said.

It also means that legacy defense capabilities will prove disastrous in the face of modern AI-driven attacks, he said.

Also: How AI can improve cybersecurity by harnessing diversity

Stressing that security teams need fundamental visibility, he noted: "If you can only see half of your environment, you don't know if you're doing the right or wrong things."

Half of Singapore businesses, for instance, say they lack complete visibility of owned and managed assets in their environment, he said, citing recent research from Armis. These companies cannot account for 39% of their asset attributes, such as where the asset is located or how or whether it is supported.

In fact, Singapore respondents cite IoT security and concerns over outdated legacy infrastructure as their top challenges.

Such issues often are compounded by a lack of funding over time to facilitate a company's digital transformation efforts, Simpson noted.

Funds typically are scheduled to slow progressively along with expectations that legacy infrastructures will reduce over time, as microservices and workflows are pushed to the cloud.

Also: State of IT report: Generative AI will soon go mainstream, say 9 out of 10 IT leaders

However, shutting down legacy systems would end up taking longer than expected because companies lack understanding of how these assets continue to be used across the organization, he explained.

"The general stance is to retire legacy, but the reality is that these systems are running across different regions and different customers. Orders are still being processed on [legacy] backend systems," he said, adding that the lack of visibility makes it difficult to identify which customers are using legacy systems and the applications that are running on these assets.

Most struggle to shut down legacy infrastructures or rid of their technical debt, which leaves them unable to recoup software and maintenance costs, he noted.

Their risk landscape then is comprised of cloud services as well as legacy systems, the latter of which are pushing data into a modern cloud architecture and workloads. They also are likely to introduce vulnerabilities along the chain by opening new ports and integration, Simpson added.

Also: The 3 biggest risks from generative AI — and how to deal with them

Their IT and security teams also have more solutions to manage and threat intel collected from different sources to decipher, often manually.

Few organizations, unless they have the necessary capabilities, have a collective view of this mixed environment of modern and legacy systems, he said.

"New technologies are intended to benefit businesses, but when left unmonitored and unmanaged, can become dangerous additions to an organization's attack surface," he noted. "Attackers will look to exploit any weakness possible to gain access to an organization's network. The responsibility lies on organizations to ensure they have the needed oversight to see, protect, and manage all physical and digital assets based on what matters most to their business."

Security

5 Books to Unlock AI in Your Kids 

Children love to play with and are fascinated by robots. Some of them also enjoy reading books very much. Here is AIM’s list of the Top 5 AI Kids Books list which includes fiction, activity books, and enjoyable reading material, all of which can inspire children to prepare for and have optimism for AI technology in the future.

20 Fables for kids on AI

By R F

’20 Fables for Kids On AI’ by R F is a thrilling and hilarious book that explores the world of artificial intelligence. It features enchanting stories of dancing robots, puzzle-solving wonders, and captivating characters. The book also introduces readers to the core concepts of AI, including machine learning, algorithms, and the power of data. With vibrant illustrations, it is a treasure for parents, educators, and anyone curious about the AI revolution. The book sparks discussions, ignites imaginations, and leaves readers craving more of the AI magic.

Artificial Intelligence: 101 Things You Must Know Today About Our Future

By Lasse Rouhiainen

The best gift you can offer your child is Artificial Intelligence: 101 Things you must know about our future if they are constantly thrilled about it. Digital specialist and thought leader Lasse Rouhiainen provides a thorough overview of the AI revolution, answering questions on the effects of AI on industry, society, and humanity. The post-COVID-19 edition of Artificial Intelligence: 101 Things You Must Know Today About Our Future offers helpful tools and ideas for anticipating the future on subjects including chatbots, robotic healthcare, and self-driving automobiles.

Coding for Kids in Scratch 3

By Raj Sidhu

Coding for Kids in Scratch 3 is an engaging guide by Raj Sindhu for kids aged 8+ teaches coding fundamentals and innovative projects using Scratch 3. It inspires them to explore technology frontiers like AI, video game bots, machine learning, augmented reality, and multiplayer computer games.

A.I. Encyclopedia for Kids

By A. Onkar

A.I. Encyclopedia for Kids by A. Onkar is an engaging and informative guide for young minds, providing a clear understanding of Artificial Intelligence (AI). It covers its history, types, applications, and ethical considerations, offering real-life examples and a dictionary of common terms. It sparks curiosity and equips readers with skills for AI success.

STEM Starters for Kids Artificial Intelligence Activity Book

By Sam Hutchinson (Author), Steven Johnson (Illustrator), Dan Pidcock (Contributor)

STEM Starters for Kids Artificial Intelligence full-color activity book teaches kids about AI science, machine learning, decision trees, inputs, and outputs. It combines non-fiction and activities, preparing them for the evolving world and fostering a lifelong passion for artificial intelligence.

The post 5 Books to Unlock AI in Your Kids appeared first on Analytics India Magazine.

25 Free Books to Master SQL, Python, Data Science, Machine Learning, and Natural Language Processing

25 Free Books to Master SQL, Python, Data Science, Machine Learning, and Natural Language Processing
Image by Author

Learning the essential skills like SQL, Python, data science, machine learning, and natural language processing can open up exciting career opportunities. However, courses and books on these topics can be expensive. The good news is that free resources are available online to help you master these skills. Online books provide foundational knowledge, practical examples, and code snippets you can apply right away.

I would like to share with you a list of 25 high-quality books covering various topics such as SQL, Python, data science, machine learning, and NLP that you can access for free online. I have gathered the best posts on free books from KDnuggets to create this collection. These books are of high quality, and I suggest bookmarking this page for future use.

SQL

  • SQL Learning by Stack Overflows: Learn all the important syntax and functions of SQL language to get better at handling data.
  • Introduction to SQL by Bobby Iliev: Learn to use relational databases for your SysOps, DevOps, and Dev projects.
  • Essential SQL by Stack Overflow: Learn clear and concise explanation of SQL topics for both beginner and advanced programmers.

Learn more about individual books by reading 5 Free Books to Master SQL

Python

Learn more about individual books by reading 5 Free Books to Help You Master Python

Data Science

  • Think Stats by Allen B. Downey: For students who want to earn Probability and Statistics with Python code examples.
  • Data Science for Business by Tom Fawcett: For business people, developers, and aspiring data scientists who want to work with, manage, implement, or invest in data science solutions.

Learn more about individual books by reading 5 Free Books to Master Data Science

Machine Learning

Learn more about individual books by reading 5 Free Books to Master Machine Learning

Natural Language Processing

Learn more about individual books by reading 5 Free Books on Natural Language Processing to Read in 2023

Conclusion

The collection of 25 free books provides a wealth of knowledge on essential data skills such as SQL, Python, data science, machine learning, and natural language processing. With the help of practical examples and code snippets, you can gain hands-on experience and apply what you learn to build your own applications. Whether you're a beginner or looking to advance your skills, these books offer valuable resources. All you need is dedication and hard work to become the best data professional you can be.

Abid Ali Awan (@1abidaliawan) is a certified data scientist professional who loves building machine learning models. Currently, he is focusing on content creation and writing technical blogs on machine learning and data science technologies. Abid holds a Master's degree in Technology Management and a bachelor's degree in Telecommunication Engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.

More On This Topic

  • 5 Free Books on Natural Language Processing to Read in 2023
  • N-gram Language Modeling in Natural Language Processing
  • Transfer Learning for Image Recognition and Natural Language Processing
  • Great New Resource for Natural Language Processing Research and…
  • 5 Free Books to Master SQL
  • Data Representation for Natural Language Processing Tasks

Gemini Pro vs GPT-4V: Has Google Killed it This Time?

Despite Google’s release of its competitor, Gemini Pro, there are claims that it has fallen short of expectations compared to OpenAI’s GPT-4. The ongoing debate revolves around whether Gemini or GPT-4V are comprehensively superior. While many opinions lean towards GPT-4V, it’s important to acknowledge that Google’s Gemini Pro is not far behind.

Recently, a research paper by researchers from Hong Kong and Shanghai, titled Gemini Pro vs GPT-4V: A Preliminary Comparison and Combination of Vision-Language Models Through Qualitative Cases, compared both the model’s vision capabilities, and the results are quite interesting.

Gemini demonstrated superior performance in specific reasoning tasks, particularly logical reasoning and factual accuracy. This positions Gemini as a suitable choice for tasks requiring robust comprehension and analytical capabilities. Hence, it’s essential to recognise the strengths of both models. Favouring GPT-4V in discussions may not be entirely justified.

GPT-4V vs Gemini

The study shows that GPT-4 V exhibited precision and succinctness in its responses, showcasing a notable strength in contextual understanding. On the other hand, Gemini Pro excelled in providing detailed and expansive answers, coupled with relevant imagery and links, highlighting its capacity for rich content generation. In industrial application scenarios, both models demonstrated competency, albeit with nuanced differences.

Gemini’s limitation to inputting a single image at a time, dependent on accompanying textual instructions, contrasts with GPT-4 V(ision)’s ability to continuously ingest multiple images, enhancing its memory capabilities. While both models exhibit comparable proficiency in basic image recognition tasks, GPT-4 Vision shines in real-world object localization, particularly in abstract image (tangram) localization.

Text extraction from images is a strength for both models, but Gemini surpasses GPT-4 Vision in reading table information. Both models demonstrate common-sense understanding in advanced reasoning tasks, with Gemini slightly trailing in certain intelligence tests. Notably, both models excel in emotional understanding and expression.

The choice between GPT-4 and Gemini hinges on the specific task requirements. GPT -4 is favoured for multimodal and prompted tasks and Gemini for code-related endeavours or scenarios prioritising computational efficiency.

Did Gemini pass the test?

When Google showcased the multi-modal capabilities of Gemini Ultra through a demo video on launch, everyone was awestruck. However, later it was found that the video was staged.

The six-minute video uploaded by Google guides us through various examples where Gemini engages in fluent conversations, responding to queries and participating in activities such as playing games like rock-paper-scissors with a person.

In the demo, everything seems to be happening in real time and Gemini can quickly respond. On the contrary, the Youtube description of the video reads, “For the purposes of this demo, latency has been reduced and Gemini outputs have been shortened for brevity.” But that is not the case with Gemini Pro regarding performance.

In highlighting the impressive capabilities of GPT-4 V within benchmark scenarios, it’s crucial to recognise the parallel strengths that Gemini Pro shares with it. Gemini distinguishes itself through its ability to provide concise and direct responses, offering a significant advantage in tasks that demand factual accuracy and prompt information retrieval.

This commonality underscores the nuanced effectiveness of both models in addressing specific challenges and reinforces the notion that advancements in one can often resonate with the capabilities of the other. Its strong reasoning ability, particularly in expert tasks, and improved identification accuracy, especially in recognising celebrities, showcase its prowess in specialised domains.

Gemini stands out in code-related tasks, demonstrating proficiency in code generation, comprehension, translation, and bug detection, making it a preferred choice for developers. It also boasts general reasoning capabilities and is touted for its scalability and efficiency.

However, both models share weaknesses, including limitations in spatial awareness, unreliable OCR, inconsistencies in reasoning, and sensitivity to prompts. The absence of the specific report’s details hinders a more in-depth analysis, emphasising the need for quantitative benchmarks and staying abreast of ongoing developments in both models, which are actively evolving. Although Gemini Ultra will be released next year, if you prioritise practicality, efficiency, and wider accessibility, Pro is likely the better choice.

The post Gemini Pro vs GPT-4V: Has Google Killed it This Time? appeared first on Analytics India Magazine.

LangChain, OpenAI’s Alter Ego

LangChain, the framework for Large Language Model (LLM) powered applications, has had a rough but exciting year in 2023 with a series of rapid adaptations and innovative expansions.

They’ve been quick to keep up with the evolving AI landscape. The speed at which LangChain keeps pace with OpenAI’s rapid advancements with ChatGPT mirrors a parallel growth trajectory and matches OpenAI’s own pace of innovation.

We saw this in the consecutive updates LangChain launched after every development at OpenAI. For instance, in March LangChain released the API integration hours after OpenAI released the same feature. In July they quickly updated their function calling feature within hours of the OpenAI announcements. Last week, just as OpenAI incorporated Pinecone vector storage integrations, LangChain adopted it on the same day.

🦜🔗LangChain Release Notes Week of 12/11
🎯LangChain Core, LangChain Community, and LangChain v0.1
💬New LLM integrations like Mixtral and Gemini
📃5 New LangChain Templates
🧑‍🤝‍🧑Lot's of community apps to highlight
Read here & sign up to get it in your inbox every other week:… pic.twitter.com/UKT4RHd7nL

— LangChain (@LangChainAI) December 16, 2023

Apart from this they also quickly integrated ChatGPT 3.5, Davinci Codex, Fine tuning capabilities and Multimodal support. These updates often released within a few hours of OpenAI’s own releases correspond to the increasing pace LangChain sets itself. This allows developers to use OpenAI models easily, while also adding a range of AI tools.

LangChain is not only limited to OpenAI’s updates on ChatGPT but also extended its support to other LLMs like Cohere, Anthropic, and a diverse array of models on HuggingFace. Recently, they also released updates for Gemini and Mistral.

The LangChain way

Although LangChain can be used for ChatGPT, agent agnostic frameworks will play a core role in most serious implementations. You just can’t rely on one LLM provider alone.

The platform’s extensive support for different LLMs, including the latest versions, positions it as a complementary framework that enhances and extends the capabilities of these models.

This support is a critical aspect of LangChain, allowing developers to leverage OpenAI’s powerful models within a versatile and expansive framework.

Similar to OpenAI, LangChain has fostered a robust and active community. This community engagement mirrors OpenAI’s own approach to building a strong user base and responding to feedback, further aligning LangChain with the ethos and practices of OpenAI.

Most recently, LangChain introduced LangChain Expression Language (LCEL), which allows users to easily create and manage sequences of actions or instructions for AI models. It is a simple way to tell an AI model what steps to follow in order to complete a task. Instead of writing complex code, you can use LCEL to clearly define these steps, making it easier to build and modify AI applications.

Had an old RAG using @LangChainAI's older agents and Chain parent class. Migrated to LCEL and holy cow what a difference. Deleted tons of code. Steps are more clear. Parallel execution means the chain is blazing fast. Great work y'all such a fun one!

— Austin Vance (@austinbv) December 27, 2023

Additionally, LangChain underwent a structural evolution, splitting into ‘langchain-core’, ‘langchain-community’, and ‘langchain’. This modular approach has streamlined the platform, facilitating stable and efficient deployment in production environments.

Competitive edge

LangChain’s diverse support for LLMs and its advanced features have given it a competitive edge in the AI market. The platform’s broad range of integrations, including various vector stores and model providers, positions it as a versatile and powerful tool for LLM applications. Its approach to advanced retrieval strategies further underlines the importance LangChain places on enhancing the capabilities and effectiveness of LLMs in practical applications.

In comparison, competitors like PromptChainer and AutoChain cater to specific market segments. PromptChainer, with its visual programming interface, appeals to users seeking simpler, more accessible LLM workflows, while AutoChain’s minimalist design attracts those who prefer direct control and rapid prototyping.

AgentGPT, offering browser-based AI agent development, is noted for its user-friendly interface and is popular for browser-centric applications. BabyAGI, still in early development, promises innovative task-driven AI but currently occupies a niche space.

Other competitors like LlamaIndex address particular aspects of LLM application development and do not directly compete with LangChain’s extensive feature set and adaptability.

Overall, LangChain’s comprehensive and adaptable framework appears to have broader appeal, especially for sophisticated LLM applications.

The platform’s journey through 2023 has been characterised by rapid adaptation, innovative development, and a steadfast commitment to serving its community.

Its ability to keep pace with the latest advancements in AI, combined with its expanding range of features and integrations, cements LangChain’s position as a leading framework in the realm of LLM applications.

The Problem with Speed

LangChain, despite its rapid development pace, faces challenges with accuracy and user concerns, particularly regarding documentation and platform complexity.

Users on social media platforms and HackerNews have raised complaints about the platform’s handling of multiple tasks simultaneously. In response, Harrison Chase, actively engaged with users, acknowledged the difficulties of managing a small team amidst the fast-paced nature of the industry. LangChain is working on consistent updates, overhauling documentation, tooling, and customisability.

Despite criticisms, some users recognise LangChain as superior to alternatives, likening it to democracy with widespread complaints but acknowledged effectiveness.

The post LangChain, OpenAI’s Alter Ego appeared first on Analytics India Magazine.

TSMC Unveils Ambitious Roadmap for Trillion-Transistor Chips

Recently, TSMC, the global giant in chip foundries, announced its audacious goal to create chips equipped with a trillion transistors by the year 2030 at the International Electron Devices Meeting (IEDM) conference held in San Francisco, signalling a remarkable advancement in chip complexity.

Concurrently, the company is pushing fabrication boundaries, aiming for 200 billion transistors on a single silicon piece. Committing to 2nm-class N2 and N2P nodes, plus 1.4nm-class A14 and 1nm-class A10 processes by 2030.

TSMC’s strategy involves a dual focus on advanced 3D packaging of multiple chiplets and monolithic chips, leveraging a 3D chipmaking approach to overcome existing limitations. Key milestones include the commencement of 2nm production in 2025, propelling advancements in miniaturisation, and the long-term vision of creating 1nm monolithic chips, each with a staggering 200 billion transistors.

Recent developments, such as Samsung’s introduction of 3nm Gate-All-Around (GAA) transistor technology, add layers of competition and innovation to the landscape. The company looks forward to tech advancements like CoWoS, InFO, and SoIC, aiming to create multi-chiplet solutions with over a trillion transistors by 2030.

TSMC’s strategic shift

The semiconductor industry has witnessed a trend towards multi-chiplet designs in recent years, as seen in AMD’s Instinct MI300X and Intel’s Ponte Vecchio, each comprising dozens of chiplets. TSMC predicts that this trend will intensify, with multi-chiplet solutions featuring over a trillion transistors becoming a reality in the coming years.

TSMC has played a pivotal role in advancing next-gen cloud solutions and AI accelerators, as demonstrated by the utilisation of its technology in AWS Graviton4, Microsoft Azure’s custom chips, and Google’s Tensor Processing Units. Beyond cloud computing and smartphones, rumours of Apple incorporating TSMC’s 3nm chips for advanced AI features in iOS 18 hint at the broader impact of these technological strides.

This interconnected ecosystem, where TSMC’s innovations fuel the demands of industry giants, forms a symbiotic loop of progress with global implications. However, manufacturing hurdles, cost considerations, and geopolitical tensions pose significant uncertainties.

Despite these challenges, TSMC’s initiative can potentially revolutionise fields ranging from artificial intelligence and high-performance computing to autonomous vehicles and medical technology. As the chipmaking sector navigates uncharted territory, it is evident that TSMC’s vision is steering the industry toward greater complexity and miniaturisation.

The post TSMC Unveils Ambitious Roadmap for Trillion-Transistor Chips appeared first on Analytics India Magazine.

What the Paytm Layoff Saga Tell Us About AI

What the Paytm Layoff Saga Tell Us About AI

The layoffs because of AI continue. “That’s 1000 families not having a happy new year,” wrote a person on X talking about the layoffs that happened at Paytm. According to reports, the fintech major and mobile payments pioneer has laid off 1,000 employees, which is around 10% of its total, from its sales, operations, and engineering team, and replaced them with AI.

“We are transforming our operations with AI-powered automation to drive efficiency, eliminating repetitive tasks and roles to drive efficiency across growth and costs, resulting in a slight reduction in our workforce in operations and marketing,” a Paytm spokesperson told. The person further added that this way the company would be able to save 10-15% in employee cost as AI is delivering better than they initially anticipated.

CEO Vijay Shekar Sharma has been urging his employees and engineers to use Microsoft and Google’s AI tools for reducing development time and costs, and that is possibly leading to a lot more layoffs.

Paytm’s employee payment problem

According to sources, most of the layoffs are part of the lending team at Paytm, a segment which has around 30% of the entire workforce. Paytm has also recently shut down small ticket loans and BNPL services to cut costs. Regardless, people say their lending business, Paytm Money, is going very strong. Since most of the work for it is now automated with AI-powered chatbots, the company is now willing to just expand businesses.

“Young customers are headed towards this. So we are saying, okay, let’s just build our trading platform with investor protection and with recommendations, the power of AI and a high transaction success rate,” said Sharma in an interview.

The Billionaire CEO has allegedly over hired a lot of employees to boost up the company’s services and thus increase revenue during the pandemic. But since the $2.5 billion IPO in 2021, the company has fallen roughly 70% because of losses and rivals such as PhonePE in the market.

This is not the first time that the One97 Communications owned Paytm has done a large scale layoff. In 2021, the company let go of around 600 employees based on their performance. This has been constant every year as the company believes it is essential to evaluate performance and cut costs over time.

Just last week, Sharma also announced the company is planning to hire around 15,000 salespeople for boosting its online wealth management services, essentially getting more merchants on its network, in a bid to hit profitability sooner. Laying off people from its customer services and engineering team might just be a move to cut off on the expenses.

Sharma had also said that with more merchants push and AI automated cost savings could help Paytm operate in profit within a year. Though Paytm had claimed it achieved operating profitability in February, that was excluding the cost of employee stock ownership plans.

Concerns around AI

Sharma is taking Paytm all in on AI by announcing several partnerships. He has also launched a $3.6 million fund for investing in AI and electric vehicles.

At the recent GPAI Summit, Sharma said that India should develop its own AI solutions instead of relying on western products. He said that the potential of AI is immense in healthcare, education, and financial services.

It might be possible that the company is building its own tech stack to compete with others coming up in the market such as Ola building Krutrim.

We reached out to Paytm for comments, but they did not respond.

In November, Paytm also announced that it is partnering with Amadeus for bringing AI solutions to its travel platform and optimising travel experience. This is also probably one of the hints that the company was looking to automate most of its customer-facing work. It is interesting to note that Amadeus uses IBM’s Watson platform for its generative AI tools and hybrid cloud.

In 2022, Paytm had started its partnership with AWS for upskilling 2,800 of its employees that included IT architects and developers.

The concerns around AI stealing jobs have been increasing everyday. In a podcast with Lex Fridman, Sam Altman has said, “AI would be able to do anything that you’d be happy with a remote coworker doing just behind a computer, which includes learning how to be a doctor or learning how to be a very competent coder.”

It seems Altman suggests that a ‘median human’, that could be any of us for him, could be soon out of job. Trying to copy Altman and come on top as legendary, Suumit Shah, the founder of Dukaan, recently drew a lot of flak for firing 90% of his workers to replace them with an AI chatbot. He also later called it a “no-brainer” move.

Mark Cuban predicted six years back that a lot of jobs would become obsolete with AI and customer services are definitely taking a hit.

The post What the Paytm Layoff Saga Tell Us About AI appeared first on Analytics India Magazine.

UP to Build India’s First AI City in Lucknow

India AI city

The Uttar Pradesh government has announced plans to establish India’s inaugural AI city in Lucknow, aimed at nurturing and advancing the AI ecosystem. The initiative, led by UP Electronics Corporation Ltd, the project’s nodal agency, has issued an expression of interest (EoI) soliciting proposals for the design, development, and operation of this groundbreaking AI city.

The envisioned AI city aspires to be a dynamic hub fostering innovative ideas, incorporating state-of-the-art technology, research facilities, and educational institutions. The EoI invites real estate developers to actively participate in constructing and managing the AI city, with the goal of cultivating the workforce of the future.

Lucknow already hosts Centers of Excellence specialising in AI and medtech, showcasing significant integration of AI technologies. The AI Centre of Excellence at IIIT Lucknow plays a pivotal role in supporting over 15 AI/ML startups, contributing to the development of a vibrant culture of creativity and entrepreneurship.

Furthermore, a report underscores the planned AI city’s components, including Grade-A certified commercial spaces for IT firms, data centre, Grade-A flexible workspaces, and technology laboratories.

The IT and Electronics Department has identified potential land parcels in strategic locations across Lucknow for the AI city’s development, with a 40-acre parcel in the Nadarganj industrial area being of particular significance due to its proximity to key infrastructure and connectivity to the Lucknow-Kanpur road and the Lucknow International Airport.

India boasts over 70 generative AI startups that have raised over $440 million in capital between 2019 and Q3 2023. The domestic generative AI market is anticipated to expand from $1.1 billion in 2023 to $17 billion by 2030.

The government of UP has embarked on a mission to achieve a $1 trillion economy within the next five years, with a specific focus on the IT and IT-enabled Services (ITeS) sectors as pivotal drivers for economic growth. UP, ranking as the sixth-largest state in terms of the IT ecosystem, is also recognised for hosting a substantial number of startups within India.

Noida is emerging as a global hub for IT and ITeS, and UP aims to enhance this sector by fostering Tier 2 cities like Lucknow, identified by prominent IT industry associations as emerging technology hubs. EoI document emphasises the strategic role of AI as a key driver for global growth in the IT and ITeS sectors, which the state intends to capitalise on.

Lucknow already boasts the presence of major IT players, including HCL and TCS, contributing to a flourishing tech ecosystem. The city’s tech landscape is further enriched by a skilled workforce, comprising over 75,000 tech professionals, 23,000 STEM graduates, and 300+ colleges, featuring esteemed institutions like IIM-Lucknow, IIIT-Lucknow, BBDU, and Amity, as highlighted in the EoI.

Simultaneously, the government is actively addressing AI data and regulation issues. The central government has initiated the formulation of regulations to foster the growth, safeguard interests, and promote innovation in the AI sector, as per IT Secretary S Krishnan.

Minister of State (MoS) for Information Technology Rajeev Chandrasekhar’s recent announcement of plans to fund and support AI startups in the country is also going to foster the AI ecosystem.

The post UP to Build India’s First AI City in Lucknow appeared first on Analytics India Magazine.