LangChain, Redis Collaborate to Create a Tool to Improve Accuracy in Financial Document Analysis

The emergence of advanced AI models has revolutionised the field of natural language processing, enabling machines to analyse, interpret, and respond to human language with increasing accuracy and sophistication. However, despite the significant advancements achieved in these models, some AI-powered assistants, such as ChatGPT, still face challenges in accurately answering complex questions derived from Securities and Exchange Commission filings. Researchers from Patronus AI discovered that even the best-performing AI model configuration, OpenAI’s GPT-4-Turbo, can only answer 79% of questions correctly on Patronus AI’s new test.

Partnering with LangChain, Redis has produced the Redis RAG template, optimized for creating factually consistent, LLM-powered chat applications. By leveraging Redis as the vector database, the template ensures rapid context retrieval and grounded prompt construction, making it a crucial tool for developers to create chat applications that provide responsive and precise AI responses.

🦜LangChain & Redis over Financial PDFs
Excited to work with the @Redisinc team to create a RAG template for developers to chat with public financial PDF documents such as Nike’s 10k filings. The application uses:
🦄FastAPI and Uvicorn to serve client requests via HTTP
🔃… pic.twitter.com/YOd3CaxDUc

— LangChain (@LangChainAI) December 19, 2023

The Redis RAG template is a REST API that allows developers to interact with public financial documents, such as Nike’s 10k filings. This application uses FastAPI and Uvicorn to serve client requests via HTTP. It also uses UnstructuredFileLoader to parse PDF documents into raw text, RecursiveCharacterTextSplitter to split the text into smaller chunks, and the ‘all-MiniLM-L6-v2’ sentence transformer from HuggingFace to embed text chunks into vectors. Moreover, it utilizes Redis as the vector database for real-time context retrieval and the OpenAI ‘gpt-3.5-turbo-16k’ LLM to generate answers to user queries.

In a recent interaction withAIM, Redis CTO Yiftach Shoolmansaid “Your data is everywhere, on your laptop on the organization repository on AWS s3 on Google Cloud Storage, whatever. You need a platform to bring the data to a vector database like Redis. cut to pieces based on the relevant knowledge”.

Criticising ChatGPT he said, “ ChatGPT doesn’t know anything because it was not trained on your data, adding users need to look for the data that is relevant to their request in their knowledge base that they just created.

The RAG template offers deployable reference architectures that blend efficiency with adaptability, providing developers with a comprehensive set of options to create factually consistent, LLM-powered chat applications with responsive and precise AI responses.

LangChain’s hub of deployable architectures also includes tool-specific chains, LL M chains, and technique-specific chains, which reduce the friction in deploying APIs. LangServe is central to deploying these templates, using FastAPI to transform LLM-based Chains or Agents into operational REST APIs, enhancing accessibility, and production-ready.

The post LangChain, Redis Collaborate to Create a Tool to Improve Accuracy in Financial Document Analysis appeared first on Analytics India Magazine.

A Simple Guide to Running LlaMA 2 Locally

A Simple Guide to Running LlaMA 2 Locally
Image by Author

New open source models like LLaMA 2 have become quite advanced and are free to use. You can use them commercially or fine-tune them on your own data to develop specialized versions. With their ease of use, you can now run them locally on your own device.

In this post, we will learn how to download the necessary files and the LLaMA 2 model to run the CLI program and interact with an AI assistant. The setup is simple enough that even non-technical users or students can get it running by following a few basic steps.

Downloading Llama.cpp for GPU machine

To install llama.cpp locally, the simplest method is to download the pre-built executable from the llama.cpp releases.

To install it on Windows 11 with the NVIDIA GPU, we need to first download the llama-master-eb542d3-bin-win-cublas-[version]-x64.zip file. After downloading, extract it in the directory of your choice. It is recommended to create a new folder and extract all the files in it.

Next, we will download the cuBLAS drivers cudart-llama-bin-win-[version]-x64.zip and extract them in the main directory. For using the GPU acceleration, you have two options: cuBLAS for NVIDIA GPUs and clBLAS for AMD GPUs.

Note: The [version] is the version of the CUDA installed on your local system. You can check it by running nvcc --version in the terminal.

A Simple Guide to Running LlaMA 2 Locally Downloading the Model

To begin, create a folder named “Models” in the main directory. Within the Models folder, create a new folder named “llama2_7b”. Next, download the LLaMA 2 model file from the Hugging Face hub. You can choose any version you prefer, but for this guide, we will be downloading the llama-2-7b-chat.Q5_K_M.gguf file. Once the download is complete, move the file into the “llama2_7b” folder you just created.

A Simple Guide to Running LlaMA 2 Locally

Note: To avoid any errors, please make sure to download only the .gguf model files before running the mode.

Initiating the AI Assistant CLI Program

You can now open the terminal in the main directory. By right clicking and selecting “Open in Terminal” option. You can also open PowerShell and the us “cd” to change directory.

A Simple Guide to Running LlaMA 2 Locally

Copy and paste the command below and press “Enter”. We are executing the main.exe file with model directory location, gpu, color, and system prompt arguments.

./main.exe -m .Modelsllama2_7bllama-2-7b-chat.Q5_K_M.gguf -i --n-gpu-layers 32 -ins --color -p "<<SYS>> As an AI assistant, your core values include being supportive, considerate, and truthful, ensuring that every interaction is guided by clarity and helpfulness. <</SYS>>"

A Simple Guide to Running LlaMA 2 Locally

Our llama.ccp CLI program has been successfully initialized with the system prompt. It tells us it's a helpful AI assistant and shows various commands to use.

Using LLaMA 2 Locally in PowerShell

Let’s test out the LLaMA 2 in the PowerShell by providing the prompt. We have asked a simple question about the age of the earth.

The answer is accurate. Let’s ask a follow up question about earth.

A Simple Guide to Running LlaMA 2 Locally

As you can see, the model has provided us with multiple interesting facts about our planet.

A Simple Guide to Running LlaMA 2 Locally

You can ask the AI assistant to generate code and an explanation in the terminal, which you can easily copy and use in your IDE.

A Simple Guide to Running LlaMA 2 Locally

Perfect.

Conclusion

Running Llama 2 locally provides a powerful yet easy-to-use chatbot experience that is customized to your needs. By following this simple guide, you can learn to build your own private chatbot set up in no time without needing to rely on paid services.

The main benefits of running LlaMA 2 locally are full control over your data and conversations as well as no usage limits. You can chat with your bot as much as you want and even tweak it to improve responses.

While less convenient than an instantly available cloud AI API, local setup brings peace of mind regarding data privacy.

Abid Ali Awan (@1abidaliawan) is a certified data scientist professional who loves building machine learning models. Currently, he is focusing on content creation and writing technical blogs on machine learning and data science technologies. Abid holds a Master's degree in Technology Management and a bachelor's degree in Telecommunication Engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.

More On This Topic

  • Development & Testing of ETL Pipelines for AWS Locally
  • A Simple Guide to Machine Learning Visualisations
  • Simple Salary Guide for Tech Experts 2022
  • Personalized AI Made Simple: Your No-Code Guide to Adapting GPTs
  • How to Get Up and Running with SQL — A List of Free Learning Resources
  • 7 Steps to Running a Small Language Model on a Local CPU

OpenAI Teases GPT-V

OpenAI loves drama. Recently, the company teased everyone with GPT-V. The link ‘openai.com/v-test’ was recently discovered by a user on X, which now shows 404 error.

This is just a few days after the same user earlier discovered a leaked document saying that GPT-4.5 is about to drop. The document also mentioned that Google rushed the release of Gemini because of a possible GPT-4.5 drop.

However, Altman put all the speculations to rest by replying with a simple ‘nah’ to the query ‘gpt4.5 leak legit or no?’ by a user on X. Interestingly, The GPT-4.5 leak also included unrealistic pricing. Therefore, it cannot be conclusively determined whether Altman dismissed just the pricing or the existence of GPT-4.5 altogether, and instead are likely to release GPT-V or GPT-5.

Dimitris Papailiopoulos posted on X a funny screenshot of ChatGPT Plus responding that it is running on ‘gpt5-feeltheAGI-this-is-what-ilya-saw-turbo-512k’.

It was believed that behind the firing of Sam Altman was the leaked Q* model, which was able to solve maths problems it hadn’t seen before, touted as a step closer to AGI. This was after the company released GPT-4 Turbo at its DevDay conference in November. There were also rumours that OpenAI is silently working on GPT-5, which seems to be actually true.

Some people predict that it can be a video generator model given all the recent trends such as Pika AI, Stable Video Diffusion, and RunwayML, and also the fact that it is called ‘v-test’.

Forget GPT-4.5, GPT-V is here

It only makes sense for OpenAI to release another version of GPT as soon as possible. The company needs to stay ahead of every other AI company, as everyone has been comparing their models with GPT-3.5 or GPT-4 as soon as they release them. OpenAI has set a benchmark for all the AI models in the industry.

OpenAI has all the resources to go ahead with this. Altman recently posted on X that OpenAI has found new GPUs, which could possibly be AMD MI300X announced at the AMD Advancing AI event, where it was revealed that OpenAI will use MI300X in its programming system, Triton 3.0. Meanwhile, Microsoft has also announced that it will use the new AMD chip in its cloud computing segment, Azure.

Currently, some of the models claim to outperform GPT-4 on several tasks and benchmarks. Gemini Ultra is set to be released by early next year. If it lives up to its on-paper prowess over GPT-4, it could indeed pose a significant challenge. It is high time OpenAI sets another benchmark with GPT-5, while starting working on better things.

While GPT-5 is likely to be more sophisticated than its predecessors, Altman has said that it was technically hard to predict exactly what new capabilities and skills the model might have.

“Until we train that model, it’s like a fun guessing game for us,” he said. “We’re trying to get better at it, because I think it’s important from a safety perspective to predict the capabilities. But I can’t tell you exactly what it’s going to do that GPT-4 didn’t.” said Altman.

Brace yourselves, AGI is coming

Interestingly, OpenAI is suddenly going all out on Super Alignment, publishing blogs and papers. A 26-page preparedness framework released on Monday detailed OpenAI’s process for assessing “catastrophic risks” in AI models before making them available to the public. This development follows Anthropic’s publication of a 22-page document outlining its proactive approach to addressing the potential dangers associated with its AI.

The preparedness team at OpenAI, led by Aleksander Madry was born around the same time as Ilya Sutskever’s ‘superalignment’ team, which served the purpose of reducing the company’s push towards building superintelligent AI systems, as they possess risk to humanity, which OpenAI’s non-profit arm is clearly against.

At least for the time being, the preparedness and superalignment teams are endeavouring to operate independently. Jan Leike, who leads the superalignment team and is an ally of Sutskever, stated that their primary focus is on safeguarding against the emergence of superintelligent models in the “distant future.”

This is after the company had devised a new method in which it plans to supervise superior AI models with smaller models, clearly indicating its work on a model that surpasses GPT-4 and requires supervision. Some people also found the blog ironic coming from OpenAI.

All these GPT-5 leaks, along with the focus on AI safety hints towards a powerful AI model coming out soon. A portion of the announcement also indicates that OpenAI aims to demonstrate a commitment to balancing the requirements of the non-profit governed by the board, which has a fiduciary duty to safeguard humanity, with the demands of a business led by Altman.

The post OpenAI Teases GPT-V appeared first on Analytics India Magazine.

Apple Optimises LLMs for Edge Use Cases

Apple has published a paper ‘LLM in a flash: Efficient Large Language Model Inference with Limited Memory’ outlining a method for running LLMs on devices that surpass the available DRAM capacity. This involves storing the model parameters on flash memory and bringing them on demand to DRAM.

https://twitter.com/1littlecoder/status/1737353316634374312

Their method involves constructing an inference cost model that aligns with the behavior of flash memory, guiding optimization efforts in two crucial areas: reducing the volume of data transferred from flash and reading data in larger, more contiguous chunks

Within this flash memory-informed framework, Apple employs two principal techniques. First, “windowing” strategically reduces data transfer by reusing previously activated neurons, and second, “row-column bundling,” tailored to the sequential data access strengths of flash memory, increases the size of data chunks read from flash memory.

These methods collectively enable running models up to twice the size of the available DRAM, with a 4-5x and 20-25x increase in inference speed compared to naive loading approaches in CPU and GPU, respectively.

This research is significant as Apple plans to integrate generative AI capabilities into iOS 18. The new OS will leverage generative AI technology to enhance Siri and the Messages app, enabling them to answer questions and auto-complete sentences more effectively. Apple is also exploring the potential use of generative AI in apps such as Apple Music, Pages, Keynote, and Xcode.

Apart from Apple, Samsung recently introduced Gauss, its own on-device LLM. According to reports, Gauss is set to be incorporated into the upcoming Galaxy S24 smartphone, slated for release in early 2024. The company intends to integrate this language model into its devices such as phones, laptops, and tablets to enhance the capabilities of its smart devices.

Moreover, Google has announced its on-device LLM, called Gemini Nano, which is set to be introduced in the upcoming Google Pixel 8 phones, offering capabilities such as Summarize in the Recorder app and Smart Reply in Gboard.

The post Apple Optimises LLMs for Edge Use Cases appeared first on Analytics India Magazine.

Apple Smoothly Crafts ‘Mouse Traps’ for Humans

Apple surely knows how to develop amazing products, and thrives on its ever evolving ecosystem. But, what many don’t know is, it’s a trap, or rather ‘Apple Trap’ – once you’re in, you are a happy captive.

The tech giant recently released a new software update, iOS 17.2. With this update, iPhone15 Pro and iPhone 15 Pro Max users can now record spatial videos, which can be brought to life with an Apple Vision Pro, which is expected to be released early next year.

Experiencing spatial videos on Apple Vision Pro. Source: Apple Blog

In a Walled Garden

It’s not new that Apple has always pushed for a closed ecosystem. It serves as a way to not only give its users products exclusive to the club, but to retain those users through new products that work best when all Apple products are owned. With the Continuity feature, one can use their Mac with other Apple devices, allowing for smarter work and seamless transitions between the devices.

The first Apple wearable, Apple watch, which was released in 2015, continues to allow sync only via iPhones. At the time of launch, though the iOS market was only 29%, when Android had over 70%, Apple continued its philosophy of launching it only for their niche club. Similarly, the easy pairing and content sync within iPhones and Macbooks.

Similarly, Apple TV that operates on tvOS, the operating system developed by Apple, includes related products such as Siri Remote and an Apple TV+ subscription. Clubbing this with other home devices such as Apple Pods, the whole home setup is addressed.

Push Towards Hardware

As per a recent Bloomberg report, Apple is said to heavily concentrate on its wearable segment in 2024. The iPhone, which has always been at the center of annual Apple product launches, will also see the launch of a new iPhone version, however, it will have no significant upgrades. Instead, Apple will be focusing on their hardware products this year.

In addition to Vision Pro, the most anticipated Apple product for the coming year, the company will be releasing advanced versions of AirPods and watches. The healthcare industry, which has been one of the sought-after categories that Apple is now chasing, will be the main focus for the product launches.

The company has announced plans to add more health detection features in their next series of watches, particularly hypertension and apnea among others. The Apple AirPods that also assist the hearing-disabled, will likely have more advanced features. Apple’s new patent shows AirPods with brain wave-detecting sensors that could measure brain activity, muscle movement, blood volume, and more features. It is a given that these features will require an iPhone for use, and the way the company will integrate it across other devices is to be seen.

AI, rather ML, For its Users

Being adamant on referring to machine learning and not ‘AI’ while announcing Apple hardware updates, Apple is going big on building an ecosystem in the PC category too. Moving away from wearables, Apple announced a string of next-generation chips for Macbook on the eve of Halloween. The M3, M3 Pro and M3 Max built with 3-nanometer process technology with an improved neural engine, will boost high-performace ML models, at improved speed and efficiency.

By announcing chips, Apple is going head-to-head against Intel, AMD and Qualcomm. With the chip offerings, Apple is trying its best to capture the chip market which is currently dominated by Intel with 68.4% followed by AMD with 31.6%. In 2020, Apple parted ways with Intel after 15 years when they launched their M1 chip, to power their Macbooks, indicating Apple’s long-term vision of non-reliability from any potential competitor.

Apple is also building its own internal chatbot, Apple GPT, on its framework Ajax, and is also working towards bringing generative AI features into its voice-assisted platform Siri and other Mac products. The move is further fortifying Apple’s closed ecosystem, and pushing for more user adoption. When other companies such as OpenAI and Microsoft are way ahead on AI developments, Apple’s presence in hardware and software gives it an advantage. Thereby, locking its customers forever.

The post Apple Smoothly Crafts ‘Mouse Traps’ for Humans appeared first on Analytics India Magazine.

The Ultimate Satellite Internet Showdown: Kuiper vs Starlink

When Amazon chief Andey Jesse shared a post about Project Kuiper broadband satellites on Linkedin, it looked like SpaceX’s Starlink but on steroids.

During a podcast with Lex Fridman, Jeff Bezos hesitantly said that while “SpaceX is going to be successful for sure,” there’s “room for a bunch of winners” in space, including Blue Origin and others—drawing a parallel to the internet, with numerous successful entities.

While this is in context to his space venture, Blue Origin, it also stands true for Amazon’s Kuiper—an up-and-coming rival to SpaceX’s Starlink in the rapidly evolving satellite internet market. Interestingly, the 3,236 satellite constellation, which Musk had once labelled a copycat mission, is being launched on SpaceX’s back.

With the promise of high-speed, low-latency internet access, both aim to capture a significant share of a market projected to reach $19 billion by the end of the decade.

Moreover, Amazon recently announced that it has achieved a breakthrough with its ‘Project Kuiper’ internet satellite system, completing successful tests of an optical mesh network in low-Earth orbit and demonstrating 100 Gbps optical links between its prototype satellites.

This will enhance throughput and reduce latency for its satellite internet service, which employs infrared lasers for data transmission between spacecraft in orbit. Unlike earlier generations, OISL or ‘optical inter-satellite links’ used in Kuiper’s satellites can simultaneously establish connections with multiple other spacecraft, forming a space-based mesh network.

The mesh network allows Kuiper to move data approximately 30% faster than terrestrial fibre optic cables over equivalent distances. However, while competitor Starlink has already achieved the laser capability, Amazon overcame challenges in implementing OISL, like the need for precise, narrow laser beams to maintain contact up to 1,616 miles away, accounting for spacecraft speeds of up to 15,534 miles per hour and complex orbital dynamics.

Interestingly, Kuiper, led by Rajeev Badyal, a former president of Starlink, still has a long way to go when it comes to matching the capabilities of Starlink, which has more than 2 million active customers and is available on all seven continents. Not just that, Starlink has also secured government contracts in the billions.

Kuiper vs Starlink

Amazon initiated Project Kuiper in 2019 to launch a constellation of 3,276 satellites orbiting at 590 to 630 kilometres, slated to deploy at least 50% of its constellation by 2026. In contrast, Starlink, developed by SpaceX, has already deployed over 5,000 satellites and aims for a massive constellation of 42,000.

However, analysts believe that Amazon’s Project Kuiper is pioneering advanced technology, such as space lasers, while also employing straightforward, reliable components that help manage costs and weight. “Kuiper is designing spacecraft that are fewer in number, bigger in size, higher in power” than SpaceX’s first generation of vehicles, said Caleb Henry, a researcher at Quilty Space.

Performance and Hardware

Pricing details for Project Kuiper are still undisclosed. Still, Amazon is expected to enter the market with competitive rates, possibly subsidised by its core businesses, such as e-commerce and cloud services. Starlink, however, offers tiered pricing based on location and chosen plan, with customers in the United States paying between $90 to $110 per month for the Residential plan and hardware fees.

In the hardware realm, Project Kuiper plans to offer phased-array antennas with various customer terminal options, including a compact and affordable 7-inch square design suitable for ground mobility and IoT applications. Starlink, meanwhile, provides a range of hardware sets to accommodate different use cases, including high-performance antennas for business customers and mobile solutions for on-the-go users.

The performance of both services depends on several factors, including the customer terminal selected and network congestion. Project Kuiper’s terminals are projected to offer speeds ranging from 100 Mbps for the mini version to 1 Gbps for the high-performance version. Starlink initially promised download speeds of up to 350 Mbps with low latency, but recent revisions have lowered these figures to 220 Mbps for certain plans. Starlink’s performance may vary as more users join the network, impacting bandwidth availability.

In conclusion, while both Project Kuiper and Starlink aim to offer satellite-based internet services, they differ in several key aspects, including satellite constellation size, launch partnerships, availability, pricing, hardware offerings, performance projections, and investment scale. The direction these projects take, and their ultimate impact on the global internet landscape will become clearer as they progress towards full deployment and commercial operation.

Generative AI to the Rescue

Amazon has also indicated the usage of Generative AI in constellation management to manage these massive constellations of satellites better. In a conversation with AIM, Director of Aerospace and Satellite Solutions at AWS, Clint Crosier emphasised the immense computational power and complex processes required for managing such satellite constellations. These processes involve daily maintenance updates, solar array charging, battery management, and other technical tasks necessary to keep satellites operational. Currently, these processes are predominantly executed by humans.

However, generative AI promises to change this paradigm. Crosier described how generative AI could handle the intricacies of constellation management efficiently. He explained, “Generative AI is going to allow you to feed 600 satellites to the generative AI system and say, go build the maintenance plan that maximizes the maintenance on the satellite while reducing the impact to users on the ground. That only does maintenance on a satellite when it’s out of view.” This approach could significantly enhance the operational efficiency of satellite constellations, reducing human intervention and maximising performance.

Partnerships and Availability

Amazon has announced critical partnerships with Verizon Communications Inc. (US), Vodafone Group Plc (Europe and Africa), and Japan’s NTT targetting customer base including enterprises, governments, schools, hospitals, and mobile operators, emphasizing diversification in over 15 countries, including Brazil, Canada, France, Mexico, and the US where it has obtained licenses to operate.

Leveraging AWS infrastructure and services can also create a flexible and secure end-to-end connectivity network for customers, allowing them to connect people, facilities, and equipment efficiently. On the other hand, in terms of availability, Starlink has been operational since November 2020 and has expanded its coverage to over 60 countries, offering a variety of internet plans.

While Starlink benefits from in-house launch capabilities using SpaceX’s reliable and cost-effective Falcon 9 and Heavy rockets, Project Kuiper has contracted United Launch Alliance (ULA), ArianeGroup, and Blue Origin and plans to collaborate with ABL.

The post The Ultimate Satellite Internet Showdown: Kuiper vs Starlink appeared first on Analytics India Magazine.

Google Unveils VideoPoet, a New LLM for Video Generation 

Google researchers recently introduced VideoPoet, a new large language model (LLM) for video generation. This model is designed to perform a range of tasks including text-to-video, image-to-video, video stylisation, video inpainting and outpainting, and video-to-audio conversion.

The introduction of VideoPoet addresses the challenge of creating coherent large motions in videos, a limitation in current video generation technologies.

This new model differentiates itself by integrating multiple video generation capabilities within a single LLM framework, contrasting with the segmented approach of existing models. It uses various modalities and is trained with multiple tokenizers, such as MAGVIT V2 for video and image, and SoundStream for audio. This allows VideoPoet to perform diverse tasks, from animating images to editing and stylising videos based on text inputs.

How it compares with others

In the evolving landscape of AI-generated video technology, VideoPoet emerges as a significant advancement, distinguishing itself from existing models like Imagen Video, RunwayML, Stable Video Diffusion, Pika, and the most recent ‘Animate Anyone’ from Alibaba Group. through its enhanced capabilities in text fidelity and motion interestingness. This new model outshines its counterparts by more accurately following text prompts and generating videos with more engaging motions.

Key points of comparison include: Zero-Shot Capabilities, VideoPoet, like other contemporary models, excels in generating content from minimal input, such as a single text prompt or image, without needing specific training on that content.

However, unlike other models which may struggle with large motion coherence, VideoPoet showcases a higher degree of accuracy in translating text prompts into video, enhancing the user experience. Where other models often face challenges in creating large, artefact-free motions, VideoPoet demonstrates a marked improvement, creating more dynamic and fluid videos.

As with Google announcements

Amidst the announcement of VideoPoet by Google Research on December 19, 2023, some scepticism exists within the community regarding its practical application and effectiveness. While VideoPoet showcases advancements in text fidelity and motion interestingness in video generation, critics question the reliance on specific prompting techniques.

There are observations that the use of terms like “8k” in prompts, a trick from previous AI models like VQGAN + CLIP and Stable Diffusion, may be employed to artificially enhance photorealism, raising concerns about the model’s genuine capability.

Overall, while VideoPoet represents a significant step in video generation technology, its real-world application, effectiveness, and impact remain subjects of debate and speculation within the community.

The post Google Unveils VideoPoet, a New LLM for Video Generation appeared first on Analytics India Magazine.

AI adds new fuel to autonomous enterprises, but don’t write off humans

Arc of light

The reality of a completely automated enterprise may still be some way off, but there are many processes and roles that can now be carried out by machines. Many IT executives and managers even agree that a substantially automated enterprise is now a realistic goal, thanks to advances in artificial intelligence (AI) and machine learning (ML), a new survey shows.

We use a lot of autonomous systems in our everyday lives — when we purchase items from online platforms, download and test software, or make research inquiries.

Also: If AI is the future of your business, should the CIO be the one in control?

Here's an apt description of what autonomous enterprises should look like: "Much like an autonomous car warns the driver about obstacles in the road, an autonomous enterprise helps business leaders avoid hidden potholes on their customer journeys, such as inefficient onboarding processes or a product offer that doesn't fit the customer's needs," Don Schuerman, CTO of Pegasystems, writes in Forbes. "Responsible and well-governed AI models can identify these issues and dynamically adjust business strategies in real time. And it happens continually, optimizing hundreds, thousands or millions of customer interactions a day across all channels."

Lately, there has been a "marked shift" toward the autonomous enterprise, according to authors of a recent survey from Digitate. Around a quarter (26%) of the survey's 601 respondents plan to implement machine-operated tasks that require limited human input, or to fully transition to autonomous systems, within the next five years.

Also: Businesses need a new operating model to compete in an AI-powered economy

Technology departments — particularly IT operations and ticket management — are the early use cases for autonomy. The survey suggests 90% of IT functions plan to implement additional automation within the next year and 58% within the next six months.

As automation proliferates, customer support, supply chain sourcing and procurement (45%), business operations (37%), and finance and accounting (36%) are the next areas in line for autonomous operations, the survey suggests.

But don't count out the role of humans just yet. One-third (33%) of organizations use machines to assist with tasks, but still rely on human input, the Digitate survey's authors point out. Close to another third (32%) have progressed to an "equal balance" of automation and human input.

While AI and ML are tilting the balance toward greater machine autonomy, the survey suggests only 12% of respondents are implementing automation systems that learn and adapt to changing environments, workloads, technologies, and policies, requiring little to no human input, or fully transitioning to self-healing automation with proactive resolution.

Also: 4 ways generative AI can stimulate the economy

However, 26% of respondents plan to implement machine-operated tasks that require limited human input, or to fully transition to autonomous systems, within the next five years.

It's also important to consider how the rise of autonomous processes also means there's a demand for a skilled workforce that can design, build, maintain, and train these systems. Businesses will also need creative leaders who can make sure these autonomous processes are focused on things that are worthwhile to the organization and its markets.

The survey makes it clear that the march to intelligent automation is not without headwinds. More than one in four IT executives (26%) feel that the main obstacle to automation in their organization is a fear that employees will leave, or the perception there will be job redundancy. This opinion is particularly strong among heads of department and VPs (41%), "underlining the tough balancing act these executives face in improving productivity while ensuring staff feel valued and perceive their jobs to be safe," the survey's authors surmise. In addition, IT executives face a lack of availability of the right tools and solutions (24%), as well as lack of tangible or measurable return on investment (23%).

So, despite the vision of machines and software running businesses smoothly and efficiently, automation requires people who can give its systems purpose — and remain within the guardrails of security and fairness.

Artificial Intelligence

Hugging Face’s Philipp Schmid on Rethinking AI Evaluation

Two weeks ago, Google pulled the curtain back on Gemini. In the release, they were chest-thumping, saying their new AI model outshines OpenAI’s GPT-4 on specific benchmarks. But can these benchmarks be solely trusted? Philipp Schmid, a Technical Lead at Hugging Face (HF) does not believe so.

“Benchmarks are critical to compare models with each other to get a first level of understanding where a company or someone should start building similar models.” But they cannot be completely relied upon, “Especially academic benchmarks,” said Schmid. “Those are very static and do not represent real-world use cases. Also, there’s a lot of potential data contamination,” he clarified in an interview with AIM.

For example, a popular benchmark is massive multitask language understanding (MMLU), a wealth of knowledge test with multiple choice questions. If this data is on the internet and was not removed from the training set of the model, the model might have seen all of those questions before and could perform better than a different model, which has yet to see the questions, he explained.

“You should always be careful about what marketing content is shared,” he proposed. While Google parades Gemini’s superior performance, they conveniently tucked away GPT-4’s victories. “It’s not a very fair comparison,” Schmid said.

Further, he described that Gemini uses the chain of thought 32. As per the technical report, they also evaluated Gemini on the five-shot concept, where GPT-4 performs better than Gemini, but was not included in the marketing blog post to show that Gemini is equally performing or better than GPT.

Schmid pointed out something else he came across recently on X: Microsoft researchers again evaluated GPT-4 on MMLU, and now it performs 90.10% Gemini with the correct prompts. I’m not sure how much you should trust those academic benchmarks. “You always need to evaluate the models based on your needs and use cases,” he added.

Leave it to Hugging Face to keep it real; the platform rolled out the Open LLM leaderboard earlier this year. It is a central place for people to start looking at what the current model is in their compute budget range.

“We evaluate models not only for the biggest one but all different classes of models with three, seven, thirteen, thirty billion parameters and even bigger. It’s more like starting at one and not zero,” Schmid declared.

New Tech, Same Old Problems

Generative AI’s been stirring the pot, no doubt. People are up in arms about it, churning out fake news and conjuring faces out of thin air. But Schmid is not buying the hype—or the panic.

“The tech is not creating something new which we haven’t seen before. There was already Photoshop which could be used to create images which might not be safe for work or could harm other people. It’s not that we are creating new problems to which we need to find a new solution,” Schmid specified.

Generative AI might be new, but the headaches it brings are old news. Suggesting a solution, he name-dropped Meta’s recent Purple Llama initiative, which can be used as an additional step in conversational application to make sure that the prompt the user provides is safe and the user can define their own safety rules.

Schmid’s Open Source Devotion

Exactly three years back, Schmid started working on the HF platform. He joined in the early days to start working on HF’s cloud integration when it began its partnership with Amazon Web Services (AWS). Then came the chat with Clem Delangue, the CEO of Hugging Face. Next thing he knew, he was the guy making the AWS integration happen; he expanded it to Azure and dabbled with NVIDIA and all the big cloud and hardware partners, he recounted.

Why the open-source devotion? Schmid doesn’t mince words: “Open source is the only right way on how we should solve AI. We have learned from regular software development that open source is way more secure and robust than closed source since more people are looking at it.”

The young gun, just 27 but with the gravitas of a seasoned vet, argues that when dealing with the unpredictable outputs of generative AI, transparency is king. When users hit a snag or something weird in generative AI models, open source lets them trace it back to the source. Furthermore, questions like ‘Was there some biased information in my training data set? Was there some bias inside the model? Can I see how the different tokens are predicted?’ can be answered via open source, Schmid pointed out.

“That’s just not possible if you have a black box API where you don’t know anything about it since, I mean, for example, we don’t know if any of the closed source providers add a layer of a different model which does some checks or which removes some other words and like behaviour changes, which is I think, supercritical for companies if you want to adopt AI in productive use cases,” he said.

The post Hugging Face’s Philipp Schmid on Rethinking AI Evaluation appeared first on Analytics India Magazine.

Rakuten India is Certified as Best Firm for Women in Tech

Rakuten India has been certified as ‘Best Firm for Women in Tech’ by Analytics India Magazine (AIM) through its workplace recognition programme. This is a corporate certification provided to organisations that have demonstrated a commitment to promoting workplace equality and empowering women in the technology industry.

“At Rakuten, we’re not just building technology; we’re fostering a community that is empowering society through technology. Our commitment to empowerment goes beyond words—it’s woven into our everyday fabric. Here, our women employees are valued contributors to our collective success story.

“They drive critical pillars ranging from our CSR engagements to e-commerce, product marketing, new business incubation and other strategic initiatives. By bringing people from diverse backgrounds together, we don’t just solve critical business challenges — we create success stories that inspire us all,” said Sunil Gopinath, CEO, Rakuten India.

“Receiving AIM’s ‘Best Firm for Women in Tech’ certification makes extremely proud. It’s a testimony to Rakuten’s consistent and concerted efforts in creating an inclusive workplace and I’m glad that our efforts are being recognised. We celebrate the diverse voices, a myriad of experiences, perspectives, and talents that make Rakuten great.

“Our mentorship programs, scrupulously tailored development paths, supportive amenities and work arrangements ensure that everyone at Rakuten feels valued, seen, heard, and ready to contribute to the future of technology and society,” said Nalini George, chief people officer, Rakuten India.

Meanwhile, Anirban Nandi, vice president – AI products & business analytics, believes that diversity has actually been Rakuten’s strength.

“We are dedicated to creating an environment where everyone, regardless of their gender, feels empowered to thrive. This recognition is a testament to our commitment to diversity, equality, and fostering an inclusive workplace. This accolade is not just a reflection of our achievements but a reminder of the work that still lies ahead in promoting gender equality in the tech industry,” said Nandi.

The ‘Best Firm’ certification by Analytics India Magazine is considered a gold standard in identifying firms that have achieved a high standard of excellence in creating a diverse and inclusive work environment for women in tech.

Get recognised as a company championing diversity and empowering women in the tech space. To nominate your organisation for the certification, please follow the link to fill out the form.

The post Rakuten India is Certified as Best Firm for Women in Tech appeared first on Analytics India Magazine.