Рубрика: AI

Apple is actively hiring people for generative AI roles. In India, the company is looking to hire an ML engineer for a generative AI role in Hyderabad. Certainly, a position advertised on the App Store platform mentions the company’s efforts in developing an internal generative AI-based developer experience platform to support and aid our app development team.

Click here to apply.

A different position within Apple Retail highlights involvement in developing a “conversational AI platform (voice and chat)” for customer engagement. Apple’s job description also outlines responsibilities related to creating text-generation technologies, including “long-form text generation, summarisation, and question-answering.”

Apple’s job portal features essential AI positions like Generative AI Quality Engineer and AIML Software Engineer, among others. Several of these roles demand expertise in training and deploying AI and/or ML models. Qualifications often include a PhD in Computer Science or related fields, encompassing Mathematics and Computer Engineering, among others.

Apple says it is working on a Generative AI based developer experience platform for internal use and assisting its app development team. The selected candidate will work with data scientists, software engineers and operations to deliver an end to end ML enabled solution for this platform.

They will collaborate closely with data scientists to implement and assess various models, fine-tuning them to meet specific requirements. Additionally, they will explore numerous integrations employing tools such as LangChain, IDE plugins, and git-based hooks, aiming to innovate and enhance the developer experience.

This development comes in the background of Apple, which had been relatively passive amid the AI boom, is now gearing up to develop generative AI features across its entire range of devices, including iOS, Siri, and other apps. Apple’s CEO Tim Cook, now asserts that the company has been working on generative AI technology for several years.

The post Apple to Hire ML Engineer for Generative AI in Hyderabad appeared first on Analytics India Magazine.

Qualcomm, the global leader in smartphone chip manufacturing, is collaborating closely with the Indian government and Tata Group to investigate the possibility of locally packaging its latest AI PC chip in the near future, according to Qualcomm’s chief financial officer, Akash Palkhiwala, Economic Times reported.

Palkhiwala said that Qualcomm has established a strong collaboration with the Prime Minister’s Office (PMO) and Tata Group to explore the utilization of Indian manufacturing facilities for our chips.

“Many of our chips are at the forefront of technology, making it challenging to fully leverage India’s manufacturing capabilities until we have leading-edge technology manufacturing here. However, there is a significant opportunity in terms of chip packaging. We are closely working with Tata Group, our longstanding partners, and this partnership presents a clear synergy for us.” added Palkhiwala.

Palkhiwala emphasised Qualcomm’s substantial presence in India, underscoring its pivotal role within the company. The dedicated India team is integral to the development of our entire roadmap, with their significant contributions, particularly in shaping this crucial chip.

Qualcomm recently announced that its redesigned Snapdragon Elite X chip, optimised for handling AI tasks such as summarising emails, text generation, and image creation, will be available in laptops from next year. Qualcomm also claims that its chip will deliver “50% faster peak multi-thread performance” than Apple’s M2 chip.

Qualcomm also introduced Snapdragon Seamless, a cross-platform technology enabling Android, Windows, and Snapdragon devices with different operating systems to discover and share information, operating as one integrated system.

The post Qualcomm Likely to Partner with Tata Group and the Indian Government appeared first on Analytics India Magazine.

Image generated with DALLE 3

One of the challenges that data practitioners face is having to code everything from scratch for every new use case. This can be a time-consuming and inefficient process. no-code or low-code solutions help data scientists create reusable solutions that can be applied to a wide range of use cases. This can save time and effort and improve the quality of data science projects.

You can do almost everything in data science without writing a single line of code. "No-code or low-code solutions are the future of data science," commented Ingo Mierswa, SVP of Product Development at Altair and founder of RapidMiner, a data science platform. As an established inventor in and of the No code data science field, his expertise and contributions have influenced the adoption and implementation of these functionalities in the industry. "These functionalities," Mierswa during our interview call remarked, "make it possible for people without a lot of programming experience to build and deploy data science models. This can help to democratize data science and make it more accessible to everyone."

"There was no no-code or low-code platform out there when I found myself being a computer scientist that I kind of recreated, very similar solutions for every new use case. It was an inefficient process, which felt like a huge waste of time," Miesrwa shares. Humoring with the basics, he articulated, "If you solve a problem for the second time and you are still coding, it means that you did not solve it correctly the first time. You should have created a solution that can be reused to solve the same or similar problems over and over again. "People, he asserts, "often don't realize how similar their problems are, and as a result, they end up coding the same thing repeatedly. "The question they should be asking is, 'Why am I still coding?' Perhaps they shouldn't in order to save time and effort."

Diverse Acceleration

No-code or low-code data science solutions can be very rewarding. "The first and most important benefit is that they can lead to better forms of collaboration," Miesrwa underscores. "Everyone can understand visual workflows or models if they are explained, however, not everyone is a computer scientist or programmer, and not everyone can understand code." So, in order to collaborate effectively, you need to understand what assets the team is collectively producing. "Data science is, at the end of the day, a team sport. You need people who understand the business problems, whether or not they can code, as coding may not be their daily business."

Then you have other people who have access to data, who are saturated in computational thinking, who think like, "Okay, well, look, if I want to build, for example, some machine learning model, I need to transform my data in a specific way." That's a great skill, and they need to collaborate too, but again, for skills like that, we know ETL products have been out for ages. "Yes, in rare cases, in special, very custom situations, you still need to code. Even in those situations, that's the one percent exception," Miesrwa pointed out. "It shouldn't be the norm, but the real magic happens when you bring together all the different skills, data, people, and expertise."

"You will never see that with a pure code-based approach. You will never get the buy-in from stakeholders. That often leads to what I call dead projects. We should be treating data science as a solution for problems. We should not treat it as a scientific approach, where it doesn't matter if we actually create a solution or not." Miesrwa reasoned. "It matters. We are solving multi-million dollar business problems here. We should actually work towards the working solution, getting the buy-in, get it deployed, and really improve our situation here. Not saying, 'Yeah, I know what if it fails, I don't care.' So collaboration is a huge benefit," he affirmed.

Acceleration is another one, Miesrwa explains. When you do repetitive tasks by coding, you're not working in the fastest possible way. If I create, for example, a RapidMiner workflow consisting of five or ten operators, that often is the equivalent of thousands of lines of code. Copying and pasting code can slow you down, but low-code platforms can help you create custom solutions faster.

Accountability, often easily overlooked, is the most important benefit. When you create a code-based solution, it can be difficult to track who made changes and why. "This can lead to problems when someone else needs to take over the project or when there is a bug in the code. On the other hand, low-code platforms are self-documenting. This means that the visual workflows that you create are also accompanied by documentation that explains what the workflow does. "This makes it easier to understand and maintain the code, and it also helps to ensure accountability," Miesrwa said. "People understand it. They buy into this, but they also can take ownership of those results. Collectively, as a team."

Open Ecosystem

The torrent of AI advancements is transforming the data science landscape, and companies that want to stay ahead of the curve are open, using open source and open standards, and not hiding anything that is very important in the data science market.

Companies that have remained open have had a winning position because the market moves quickly and requires constant iteration. "This is true for the overall data science market over the past 10 to 20 years," Miesrwa reflected, "the fast-paced nature of the market requires constant iteration, making it exceedingly unwise to close down the ecosystem. This is part of why some companies that have traditionally been closed have opened up and even adopted a vendor-neutral approach to support more programming languages and integrations."

While the code-optional approach allows researchers to perform complex data analysis tasks without writing a single line of code, there are situations where coding may be necessary. In such cases, most low-code platforms integrate with programming languages, machine learning libraries, and deep learning environments. They also offer users the ability to explore the marketplace for third-party solutions, Miesrwa specified. "RapidMiner even provides an operator framework that allows users to create their own visual workflows. This operating framework makes it easy to extend and reuse workflows, providing a flexible and customizable approach to data analysis."

The Path Ahead

Altair, a leader in computational science and AI, conducted a survey that revealed the widespread adoption of data and AI strategies in organizations worldwide.

The research, which involved over 2,000 professionals from various industries and 10 different countries, revealed a significant failure rate (ranging from 36% to 56%) for AI and data analytics projects when there is friction between different departments within an organization.

The study identified three main sources of friction that hinder the success of data and AI projects: organizational, technological, and financial.

• Organizational friction arises from challenges in finding qualified individuals to fill data science roles and a lack of AI knowledge among the workforce.
• Technological friction stems from limitations in data processing speed and issues with data quality.
• Financial friction is caused by constraints in funding, a focus on upfront costs by leadership, and the perception of high implementation costs.

James R. Scapa, founder and CEO of Altair, in the news release emphasized the importance of organizations leveraging their data as a strategic asset to gain a competitive edge.

Friction paralyzes mission-critical projects. To overcome these challenges and achieve what Altair terms as 'Frictionless AI,' businesses must adopt self-service data analytics tools. These tools," Scapa highlights, "empower non-technical users to navigate complex technology systems easily and cost-effectively, eliminating the friction that hampers progress."

He also acknowledged that obstacles exist in the form of people, technology, and investment, hindering organizations from harnessing data-driven insights effectively. And by closing the skill gaps, organizations can help build sound knowledge between cross-functional teams to overcome friction.

Saqib Jan is a writer and technology analyst with a passion for data science, automation, and cloud computing.

More On This Topic

• The Drag-and-Drop UI for Building LLM Flows: Flowise AI
• Analyze Python Code in Jupyter Notebooks
• Data Profession Job Satisfaction: Beware Of The Drop
• The Data Matters: Choosing the right data to analyze can make or break your…
• The Rise of Vector Data
• GitHub Copilot and the Rise of AI Language Models in Programming Automation

There is no doubt that generative AI has taken the world by storm. While there are issues of reliability and accuracy in tools like ChatGPT, AI chatbots can also provide a substantial productivity benefit to many workers.

Beyond the benefits that come from using generative AI as a tool, there are also opportunities for you to improve your employment situation by developing skills in the generative AI arena.

Also: 6 skills you need to become an AI prompt engineer

Although there are many courses, degree programs, micro degrees (which are not degrees, for the record), and boot camps that will charge you big bucks with the promise of earning big bucks, that's not the only direction you can take. There are also many free resources that can help you expand your skill set.

For example, there is the collection of thirteen free short courses offered by OpenAI (the folks who make ChatGPT) and DeepLearning (an education provider). While DeepLearning does sell longer courses, the thirteen I'm going to spotlight here are a great place to start — and they are completely free.

I signed up and took the first course, "ChatGPT Prompt Engineering for Developers." It consists of nine short videos of roughly 10 minutes each, along with exercises and a test bench that allows you to try out the code taught in the course and see what it does.

Also: 8 ways to reduce ChatGPT hallucinations

I enjoyed the course and got a lot out of it. It helped me understand how to connect ChatGPT to code and how traditional programmers can include generative AI prompts in their coding kit bags. I did it over the course of a weekend, learned a lot, and spent nothing.

It doesn't get better than that.

Beginner courses

Cours offerings are listed as "beginner" or "intermediate." Keep in mind that beginner refers to your AI development experience, not your technical experience. You need to have a fairly good programming background to really understand the content of these "beginner" level courses.

• ChatGPT Prompt Engineering for Developers: Go beyond the chat box. Use API access to leverage LLMs into your own applications, and learn to build a custom chatbot.
• Building Systems with the ChatGPT API: Level up your use of LLMs. Learn to break down complex tasks, automate workflows, chain LLM calls, and get better outputs.
• LangChain for LLM Application Development: Learn the framework to take LLMs out of the box. Learn to use LangChain to call LLMs into new environments.
• LangChain: Chat with Your Data: Create a chatbot to interface with your private data and documents using LangChain.
• Large Language Models with Semantic Search: Learn to use LLMs to enhance search and summarize results.
• Building Generative AI Applications with Gradio: Create and demo machine learning applications quickly. Share your app with the world on Hugging Face Spaces.
• Pair Programming with a Large Language Model: Learn how to effectively prompt an LLM to help you improve, debug, understand, and document your code.
• Understanding and Applying Text Embeddings: Learn how to accelerate the application development process with text embeddings.
• How Business Thinkers Can Start Building AI Plugins With Semantic Kernel: Learn Microsoft's open-source orchestrator, Semantic Kernel, and develop business applications using LLMs.

Intermediate courses

The following set of courses is considered "intermediate." I'd recommend completing the above nine courses first, then taking on this next set. Remember that they're all free, so your only cost is the time it takes to learn some tasty goodness.

• Functions, Tools and Agents with LangChain: Learn and apply the new capabilities of LLMs as a developer tool.
• Finetuning Large Language Models: Learn to fine-tune an LLM in minutes and specialize it to use your own data.
• Evaluating and Debugging Generative AI Models Using Weights and Biases: Learn MLOps tools for managing, versioning, debugging, and experimenting in your ML workflow.
• How Diffusion Models Work: Learn and build diffusion models from the ground up.

Grow your career without growing your credit card balance

AI is hot, hot, hot. As such, companies and even long-established educational institutions will do everything they can to convince you a career in AI will change your life — and they're the folks who will help you make that change.

Some of those programs are very good. I taught degree-credit programming courses at the UC Berkeley extension for six years and have been an advisory board member at the university for more than 16 years. Many students have told me these types of formal courses helped them change careers. So I'm not saying to avoid formal courses and for-pay education.

Also: Mid-career professionals, watch out. You're the most exposed to AI

But with the wealth of free material out there, I strongly recommend you take advantage of those resources first, especially if you're on a budget or if you want to get started in this field without making a substantial financial commitment.

What do you think? Are you going to take any of these courses? Have you taken any? What did you learn? Let us know in the comments below.

You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter on Substack, and follow me on Twitter at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.

Credal aims to connect company data to LLMs ‘securely’ Kyle Wiggers 8 hours

Credal.ai, a Y Combinator-backed startup that gives enterprises a way to connect their internal data to text-generating, cloud-hosted AI models, has raised $4.8 million in a seed round led by Spark Capital.

Credal was founded by Jack Fischer and Ranavin Thambapillaid, who previously worked at Palantir and bonded over a mutual interest in security and compliance. Fischer is an ex-Googler, while Ravin taught himself to code after studying philosophy, politics and economics at Oxford.

“We realized that, with our backgrounds in enterprise data security and AI from Palantir, we were in a unique position from which to build an AI data platform that enterprises could actually trust,” Fischer told TechCrunch in an email interview.

Fischer and Thambapillaid initially set out to build what they describe as a “decision-making assistant” for enterprises that’d use large language models (LLMs) — models along the lines of ChatGPT — to read documents and give advice on strategic, C-Suite-level decisions. But the project eventually morphed into something broader: a tool to connect data from internal data sources to outside LLMs.

As the platform exists today, Credal can be used to build general knowledge or domain-specific, AI-powered chatbots for a range of use cases. For example, a company could tap Credal to create a bot that answers security questions about software that the company licenses, drawing on the latest documentation.

Credal doesn’t serve LLMs itself. Rather, it sits between users submitting prompts (e.g. “What’s the latest version of this software?”) and an API from a third-party LLM provider like OpenAI or Anthropic, acting as a “co-pilot” that can be deployed in existing apps like Slack.

Credal attempts to automatically direct prompts to the “most appropriate” LLM if a company’s using more than one, based on factors like the sensitivity of the data being submitted, cost, company policy and a model’s technical capabilities. In some cases, it employs more than one LLM to accomplish a task — for instance using Anthropic’s Claude and GPT-4 to structure company documents.

Image Credits: Credal

Plenty of platforms offer ways to connect company data to LLMs — see Unstructured, Deasie and LlamaIndex. And OpenAI’s expanding its built-in plugin framework. But Credal’s unique spin on this is a strong emphasis on compliance and security — at least the way Fischer tells it.

Credal attempts to automatically redact, anonymize and otherwise warn when sensitive data is about to be sent off-network, say to an LLM hosted on a public cloud. And it provides logs that show what data’s been shared with which LLMs.

Data sent to Credal is retained by default and kept for 30 days after accounts expire — which might give some companies pause. But admins can change this and choose to wipe data at any time, Fischer emphasizes.

Fischer also claims that Credal is one of the few vendors of its kind to be registered under the Data Privacy Framework, the recent U.S.-EU agreement that governs the transfer of personal data between the two countries. That’s enabled it to win contracts with publicly-traded, regulated European enterprises like Wise, Fischer says.

“IT departments at enterprises want visibility, and control, over how AI is being used within their organization,” he added. “Credal gives them [this transparency] in a standard format across multiple LLM providers, providing fine-grained data controls over who can access which models, what data can be used by each user and for what purposes … Unlike other ‘AI on your data’ systems, Credal automatically mirrors the permissions of the source systems it connects to, so when a user asks a question, the AI responds from only company documents that are both relevant and accessible to that user.”

Since launching in April, Fischer claims that Credal has handled over a quarter of a million LLM calls and ingested around 100,000 corporate documents. The company has 11 customers currently, several of which have signed “six-figure” contracts, according to Fischer.

With the capital from the seed round, Credal plans to expand its headcount (which stands at five employees at present) and “expand the product to cover more data sources and perform more sophisticated data retrieval,” Fischer says.

“The AI industry at the moment is suffering a relative imbalance between the huge enthusiasm and the as-yet still relatively small number of companies using LLMs to create real-world value,” Fischer said. “Credal is solving that by embedding deeply with a small number of amazing enterprises and actually solving their problems end-to-end.”

Mahindra Lifespace Developers Limited (MLDL), a subsidiary of the Mahindra Group specialising in real estate and infrastructure development, has introduced India’s inaugural Metaverse home-buying experience. This novel offering, known as Bastion at Mahindra Citadel, marks the launch of Phase 2 of the project.

Last night, 600 drones illuminated Pune skies to form a QR code, launching India's first metaverse home-buying experience.
This achievement is the result of collaboration between @life_spaces & @tech_mahindra, exemplifying our commitment to synergy & innovation.… pic.twitter.com/6GqxXdLZ8F

— Mahindra Group (@MahindraRise) October 26, 2023

What Does Metaverse Bring?

The announcement was made through a drone display held at the project site in Pimpri-Chinchwad. Over 500 drones illuminated the night sky, showcasing key features of the project, including the ecotone design and home automation capabilities. Prospective buyers can explore their potential future homes prior to making a commitment. Users also have the ability to engage with various elements within these virtual homes and customise the interiors according to their preferences.

Amit Kumar Sinha, MD & CEO of Mahindra Lifespace Developers Ltd, expressed enthusiasm for the launch and its relevance to Pune, which is a significant market for Mahindra Lifespaces. He emphasised Pune’s dynamic real estate environment and its propensity for innovation, underscoring the immersive and interactive nature of the Metaverse experience

Future-Forward

The initiative is a first-of-its-kind where a home-buying experience is brought to a Metaverse, in India. However, it’s not the first time for the Mahindra group to enable business via Metaverse. In 2022, the company announced that it is working on 60 metaverse projects, spanning over projects in ed-tech, retail, automotive dealer management, and others.

Last year, Mahindra also launched a metaverse platform called XUV400Verse for buyers to experience a virtual showroom for their upcoming car XUV400.

In yesterday’s Q2 earnings call, C.P Gurnani, CEO of Tech Mahindra had also emphasised on how the company will continue to innovate and diversify into different vertices.

The post Mahindra Group launches India’s First Home-Buying Experience on Metaverse appeared first on Analytics India Magazine.

In a recent earnings call, Tech Mahindra announced to achieve big milestones in generative AI. During the call, CP Gurnani, Managing Director, Tech Mahindra said, “Tech Mahindra is now working in about 60 customer locations on actually using generative AI to enhance operations, innovation, and productivity.”

In contrast, TCS is engaged in hundreds of Gen-AI projects for its clients across segments. Meanwhile, Infosys is working on 90 generative AI programs, Wipro said it has doubled down on the number of customers as compared to the last quarter. HCLTech is working with a handful of customers with generative AI projects. LTIMindtree is engaged in over 20 clients for generative AI.

Tech Mahindra currently sees generative AI as a means to automate routine tasks and freeing up talent for more innovative work among other initiatives. In August, the company had announced that it plans to train about 8000 employees as it readies itself to cater to demand around generative AI and quantum computing.

Source: Belamy (AI Weekly Newsletter)

Slew of GenAI Solutions

In April, Tech Mahindra became the first IT giant to launch something like a Generative AI Studio. ​​The IT solution provider introduced TechM amplifAI0->∞, a comprehensive suite of Artificial Intelligence (AI) offerings and solutions aimed at democratizing and responsibly scaling AI deployment. Over time, the company has continuously integrated new tools into this suite.

In August, Tech Mahindra partnered with Google AI to develop Generative AI Powered Email AmplifAIer, a solution that uses generative AI to automate email responses and to personalize email communications.

Last month, the company introduced another solution called ‘Ops amplifAIer’ solution. This solution aims to enhance the productivity of support engineers by offering a unified integrated view with all the contextual information and tools needed to resolve issues. It also facilitates team collaboration and generative AI assistance capabilities, ensuring that processes are future-proofed in a responsible manner.

Tech Mahindra’s Ops amplifAIer solution integrates with existing ITOps tools to collect the contextual information related to an IT ticket/alert and uses generative AI to analyze the collected data. It further identifies the probable root cause, diagnose, recommend remediation actions, and generate the corresponding automation scripts. The solution comes with an enterprise automation catalogue that enables the reuse of automation artefacts like scripts or workflows across the enterprise.

Early this month, Tech Mahindra released ‘Vision amplifAIer’ solution. This solution is designed to enhance computer vision-related use cases for enterprises, offering comprehensive end-to-end lifecycle management for computer vision (CV) projects. The emphasis is placed on streamlining the entire process, making it more accessible and efficient.

Apart from the enterprise solutions , Tech Mahindra is working on an indigenous LLM called Project Indus that would have the ability to speak in many Indic languages, most notably Hindi. The model will have the ability to speak in 40 different Indic languages, to begin with. More languages that have originated in the country will also be added subsequently.

Tech Mahindra is not alone

During a recent earnings call, TCS’ executive vice president and global head of human resources, Milind Lakkad, highlighted their workforce’s readiness for generative AI, boasting over 100,000 GenAI-ready employees. TCS CEO K Krithivasan emphasised their success with TCS Cognix, utilising generative AI for operating model transformation deals, enabling businesses through dashboards and predictive analytics.

Infosys, under CEO Salil Parekh, discussed their generative AI capability, Topaz, which significantly increased their market share. Parekh revealed Infosys’ engagement in over 90 generative AI programs, emphasising their commitment to helping clients navigate the future with deep technical expertise. To support this, Infosys trained 57,000 employees in generative AI principles.

“We have trained as many as 180,000 employees in basic GenAI principles,” said Wipro chief Thierry Delaporte. He said that the company has rolled out personal-based learning pathways to create a pool of specialised talent with deeper technical expertise. It has ambitions to train over 250,000 employees in the coming months. Recently, it also launched a new GenAI Center of Excellence in collaboration with IIT Delhi.

Delaporte said it is rapidly integrating generative AI into its processes, solutions and offerings. “Thousands of our employees have or are starting to use generative AI. Today, we are seeing a doubling of GenAI active projects than we did just one quarter ago,” added Delaporte, pointing out how healthcare, consumer and financial services, high tech and utilities are seeing rapid adoption.

HCLTech also announced a slew of new deals, where it will be involved in digital and cloud transformation, alongside generative AI initiatives.

HCLTech chief C Vijayakumar said that the company is working on a two-pronged approach – for its clients, it looks to generate early-stage opportunities, alongside training its delivery organisation to leverage generative AI for core development, deployment, and testing and managed services. So far, it looks to train close to 18,000 employees.

LTIMindtree LTIMindtree has expressed its commitment to integrating generative AI into its products and solutions, revealing that they have participated in more than 100 discussions and currently have over 20 active engagements with clients, according to LTIMindtree Chief, Debashis Chatterjee.

Additionally, the company is set to launch two new offerings: Canvas Lite, designed for developers focusing on productivity use cases, and Canvas Control Claim, which facilitates secure, moderated, and responsible AI use. This offering provides clients with the option to choose from commercial open-source and custom element models. Chatterjee also shared the company’s plan to train over 10,000 employees by the end of the third quarter.

In conclusion, Tech Mahindra is a unique player in Indian IT, who is making strides in generative AI, while others – TCS, Infosys, Wipro, HCLTech, and LTIMindtree – are focusing on enterprise solutions, upskilling and reskilling, alongside experimenting with real use cases.

The post While TCS Leads, Tech Mahindra Makes Strides in Generative AI appeared first on Analytics India Magazine.

Image by Editor Introduction

In recent years, the Large Language Model (LLM) has changed how people work and has been used in many fields, such as education, marketing, research, etc. Given the potential, LLM can be enhanced to solve our business problems better. This is why we could perform LLM fine-tuning.

We want to fine-tune our LLM for several reasons, including adopting specific domain use cases, improving the accuracy, data privacy and security, controlling the model bias, and many others. With all these benefits, it’s essential to learn how to fine-tune our LLM to have one in production.

One way to perform LLM fine-tuning automatically is by using Hugging Face’s AutoTrain. The HF AutoTrain is a no-code platform with Python API to train state-of-the-art models for various tasks such as Computer Vision, Tabular, and NLP tasks. We can use the AutoTrain capability even if we don’t understand much about the LLM fine-tuning process.

So, how does it work? Let’s explore further.

Getting Started with AutoTrain

Even if HF AutoTrain is a no-code solution, we can develop it on top of the AutoTrain using Python API. We would explore the code routes as the no-code platform isn’t stable for training. However, if you want to use the no-code platform, We can create the AutoTrain space using the following page. The overall platform will be shown in the image below.

Image by Author

To fine-tune the LLM with Python API, we need to install the Python package, which you can run using the following code.

pip install -U autotrain-advanced

Also, we would use the Alpaca sample dataset from HuggingFace, which required datasets package to acquire.

pip install datasets

Then, use the following code to acquire the data we need.

from datasets import load_dataset     # Load the dataset  dataset = load_dataset("tatsu-lab/alpaca")   train = dataset['train']

Additionally, we would save the data in the CSV format as we would need them for our fine-tuning.

train.to_csv('train.csv', index = False)

With the environment and the dataset ready, let’s try to use HuggingFace AutoTrain to fine-tune our LLM.

Fine-tuning Procedure and Evaluation

I would adapt the fine-tuning process from the AutoTrain example, which we can find here. To start the process, we put the data we would use to fine-tune in the folder called data.

Image by Author

For this tutorial, I try to sample only 100 row data so our training process can be much more swifter. After we have our data ready, we could use our Jupyter Notebook to fine-tune our model. Make sure the data contain ‘text’ column as the AutoTrain would read from that column only.

First, let’s run the AutoTrain setup using the following command.

!autotrain setup

Next, we would provide an information required for AutoTrain to run. For the following one is the information about the project name and the pre-trained model you want. You can only choose the model that was available in the HuggingFace.

project_name = 'my_autotrain_llm'  model_name = 'tiiuae/falcon-7b'

Then we would add HF information, if you want push your model to teh repository or using a private model.

push_to_hub = False  hf_token = "YOUR HF TOKEN"  repo_id = "username/repo_name"

Lastly, we would initiate the model parameter information in the variables below. You can change them as you like to see if the result is good or not.

learning_rate = 2e-4  num_epochs = 4  batch_size = 1  block_size = 1024  trainer = "sft"  warmup_ratio = 0.1  weight_decay = 0.01  gradient_accumulation = 4  use_fp16 = True  use_peft = True  use_int4 = True  lora_r = 16  lora_alpha = 32  lora_dropout = 0.045

With all the information is ready, we would set up the environment to accept all the information we have set up previously.

import os  os.environ["PROJECT_NAME"] = project_name  os.environ["MODEL_NAME"] = model_name  os.environ["PUSH_TO_HUB"] = str(push_to_hub)  os.environ["HF_TOKEN"] = hf_token  os.environ["REPO_ID"] = repo_id  os.environ["LEARNING_RATE"] = str(learning_rate)  os.environ["NUM_EPOCHS"] = str(num_epochs)  os.environ["BATCH_SIZE"] = str(batch_size)  os.environ["BLOCK_SIZE"] = str(block_size)  os.environ["WARMUP_RATIO"] = str(warmup_ratio)  os.environ["WEIGHT_DECAY"] = str(weight_decay)  os.environ["GRADIENT_ACCUMULATION"] = str(gradient_accumulation)  os.environ["USE_FP16"] = str(use_fp16)  os.environ["USE_PEFT"] = str(use_peft)  os.environ["USE_INT4"] = str(use_int4)  os.environ["LORA_R"] = str(lora_r)  os.environ["LORA_ALPHA"] = str(lora_alpha)  os.environ["LORA_DROPOUT"] = str(lora_dropout)

To run the AutoTrain in our notebook, we would use the following command.

!autotrain llm   --train   --model ${MODEL_NAME}   --project-name ${PROJECT_NAME}   --data-path data/   --text-column text   --lr ${LEARNING_RATE}   --batch-size ${BATCH_SIZE}   --epochs ${NUM_EPOCHS}   --block-size ${BLOCK_SIZE}   --warmup-ratio ${WARMUP_RATIO}   --lora-r ${LORA_R}   --lora-alpha ${LORA_ALPHA}   --lora-dropout ${LORA_DROPOUT}   --weight-decay ${WEIGHT_DECAY}   --gradient-accumulation ${GRADIENT_ACCUMULATION}   $( [[ "$USE_FP16" == "True" ]] && echo "--fp16" )   $( [[ "$USE_PEFT" == "True" ]] && echo "--use-peft" )   $( [[ "$USE_INT4" == "True" ]] && echo "--use-int4" )   $( [[ "$PUSH_TO_HUB" == "True" ]] && echo "--push-to-hub --token ${HF_TOKEN} --repo-id ${REPO_ID}" )

If you run the AutoTrain successfully, you should find the following folder in your directory with all the model and tokenizer producer by AutoTrain.

Image by Author

To test the model, we would use the HuggingFace transformers package with the following code.

from transformers import AutoModelForCausalLM, AutoTokenizer    model_path = "my_autotrain_llm"  tokenizer = AutoTokenizer.from_pretrained(model_path)  model = AutoModelForCausalLM.from_pretrained(model_path)

Then, we can try to evaluate our model based on the training input we have given. For example, we use the "Health benefits of regular exercise" as the input.

input_text = "Health benefits of regular exercise"  input_ids = tokenizer.encode(input_text, return_tensors="pt")  output = model.generate(input_ids)  predicted_text = tokenizer.decode(output[0], skip_special_tokens=False)  print(predicted_text)

The result is certainly still could be better, but at least it’s closer to the sample data we have provided. We can try to playing around with the pre-trained model and the parameter to improve the fine-tuning.

Tips for Successful Fine-tuning

There are few best practices that you might want to know to improve the fine-tuning process, including:

1. Prepare our dataset with the quality matching the representative task,
2. Study the pre-trained model that we used,
3. Use an appropriate regularization techniques to avoid overfitting,
4. Trying out the learning rate from smaller and gradually become bigger,
5. Use fewer epoch as the training as LLM usually learn the new data quite fast,
6. Don’t ignore the computational cost, as it would become higher with bigger data, parameter, and model,
7. Make sure you follow the ethical consideration regarding the data you use.

Conclusion

Fine-tuning our Large Language Model is beneficial to our business process, especially if there are certain requirements that we required. With the HuggingFace AutoTrain, we can boost up our training process and easily using the available pre-trained model to fine-tune the model.

Cornellius Yudha Wijaya is a data science assistant manager and data writer. While working full-time at Allianz Indonesia, he loves to share Python and Data tips via social media and writing media.

More On This Topic

• Hugging Face Transformers Package — What Is It and How To Use It
• Overview of AutoNLP from Hugging Face with Example Project
• Understanding BERT with Hugging Face
• Training BPE, WordPiece, and Unigram Tokenizers from Scratch using Hugging…
• Top 10 Machine Learning Demos: Hugging Face Spaces Edition
• A community developing a Hugging Face for customer data modeling