Sam Altman to Attend  Intel’s Event, Indicates Venturing into AI Chips

Intel Foundry Services (IFS) is set to host IFS Direct Connect on February 21, 2024, at the San Jose McEnery Convention Center. Sam Altman, Chief Executive of OpenAI, will mark his presence at the event as a luminary speaker.

“Thrilled Sam Altman is joining me at Direct Connect on Feb 21. Sam is a renowned leader on AI & its impact on the world. Looking forward to discussing the role semis play in enabling modern society,” wrote Pat Gelsinger, Intel chief.

“Infinite possibilities ahead in the age of AI & no better convo for Intel’s 1st Foundry event. See you soon, Sam!” he added.”

The dialogue is expected to delve into the critical role played by semiconductors in shaping modern society, exploring the limitless possibilities in the age of AI.The IFS Direct Connect event promises exclusive insights from Intel executives and ecosystem partners.

Attendees can anticipate learning about Intel’s strategies, process technology, advanced packaging, and ecosystem collaborations. Of particular interest will be the discussion on how Intel Foundry Services can empower silicon designs, leveraging Intel’s resilient, secure, and sustainable supply chain.

Recently, reports surfaced that Altman is planning to raise billions for an AI chip venture aimed at developing a ‘network of factories’ for fabrication that would stretch around the globe, involving collaboration with unnamed ‘top chip manufacturers.’

Last week, Altman visited South Korea to meet with executives from Samsung Electronics and SK Hynix, exploring the potential for an alliance in the field of AI chips. Other reports suggest that Altman is in discussions with Middle Eastern investors and chip fabricators, including TSMC, about launching a new chip venture.

The post Sam Altman to Attend Intel’s Event, Indicates Venturing into AI Chips appeared first on Analytics India Magazine.

From MLOps to LLMOps— and hardware headaches ahead

A model on its own is typically not enough. It requires the data, which comes in a very specific format and has to be the same format that will be used at the time of inference or prediction. But the reality, on the other hand, is that data changes all the time, and sometimes even data formats change. So, typically, you need to put something in front of the model, which makes sure that the data coming in still fits the template, header, or schema the model has seen when it was trained. And that’s a very special factor—an artifact, really—for MLOps.

MLOps and LLMOps
Image by Product School on Unsplash

The only way to effectively productionize any machine learning project at scale is with MLOps tools and principles. But there are unique problems that, by their intrinsic nature, are universally challenging to the MLOps engineering teams. And those challenges are leading to the rise of foundation model operations (FMOps) and large language model operations (LLMOps)—a subset of FMOps.

The models get larger and larger, such as generative AI models and large language models. “They are so large that you need specialized infrastructure, and you sometimes need to come up with new creative ways so you can ensure that responses are delivered in a timely fashion,” says Dr. Ingo Mierswa, an industry-veteran computer scientist and founder of Altair RapidMiner. “And you need to start understanding if you can maybe sacrifice some precision of your models to actually reduce the memory footprint.”

All of that is very new. “This problem didn’t exist about 20 years ago when I started in this field,” Mierswa reflected during our call. We had already been running into all kinds of memory issues, and this was not a problem. Now, thanks to generative AI and LLMs, that problem is back—we are, because of how much data is being generated, developing resource-intensive models that the consumer-based hardware before us is not sufficient to work with. ‘And if you need to use specialized hardware’ — he implied it also means that the engineer needs specialty skills to work with that hardware and make it scalable or reduce the memory footprint of models.

Consider the GPT-based series of applications, since most of them are chat-based (text-to-text) applications. You will always see—you’ll type something, and then it streams a set of tokens or text. “One of the reasons behind that is the inference time of GPT is very slow, on the order of several seconds. And for deployment, the challenges that engineering teams face when deploying large language models for search applications, recommendation applications, or ad applications is that the latency requirements are on the order of several milliseconds—not seconds,” says Raghavan Muthuregunathan, Senior Engineering Manager at LinkedIn, leading the Typeahead and whole-page optimization of LinkedIn Search.

And how are big tech engineering teams trying to solve for that? “The knowledge distillation, and the fine-tuning where engineers are trying to deploy a very small, fine-tuned model for that specific task within a GPU. ‘This helps decrease the inference time from several seconds to just a few hundred milliseconds,’ Muthuregunathan articulated. ‘In fact, there is a technique called Look ahead decoding.’ It still is a very active area of research where engineers are trying to see if LLMs’ inference time can be reduced.”

Google has a very limited preview of their AI-powered research search where you can ask a question, such as ‘tell me about Pier 57 in NYC,’ and within seconds, you’ll see everything loading with the search results. It does take a few seconds to load, and that delay is likely due to the extremely slow inference time of a large language model. That’s why, for instance, if you were to type a query like ‘Donald Trump‘ on Google, they won’t provide AI-generated results immediately because they know it would take more time for this specific query, and users probably don’t need it instantly. The user’s intent is likely to navigate to some specific web page rather than consume the LLM summarized content.So, they’ve introduced a ‘generate’ button feature. If you choose to, you can wait for those several seconds, and then the results will be generated. “The way people are circumventing this latency issue is through a product experience rather than relying on advanced AI or infrastructure techniques,” Muthuregunathan explained. They’ve created a button so that when people click it, they’re okay with waiting a few seconds, as opposed to waiting several seconds of delay after typing a query, which otherwise might not create a good user experience.

“And everyone is trying to do streaming of LLMs to circumvent the limitation puzzle instead of making a single call to LLMs and getting all responses at once,” he mentioned. “Why? Because every application is more of a streaming application—that’s part of why most of these applications are chat applications instead of search engine applications.”

This is imperative because reducing the memory footprint is exceedingly difficult. We don’t have good hardware for vision-related tasks, and the availability of GPUs for production use cases today is a challenge. But there are sparks of progress. Nvidia’s founder and CEO, Jensen Huang, announced their new AI chip, the H200, at the last AWS re:Invent, which will be available for AWS customers. Google is introducing Cloud TPU v5p and the AI Hypercomputer for GPU-accelerated computing to run deep learning workloads. OpenAI is engaging in the arms race by developing its own chipset. And Tesla? They’re definitely forging ahead.

The extent to which these efforts will deliver on their seemingly great promise of overcoming computational limitations is still up in the air. But because of these limitations, it continues to be common among engineering teams to add more boxes, given that AI workloads rely heavily on extremely high-performance computing nodes.

A Step by Step Guide to Reading and Understanding SQL Queries

A Step by Step Guide to Reading and Understanding SQL Queries
Image by Freepik

SQL, or Standard Query Language, is a programming language for managing and manipulating data within a relational database management system (RDBMS). It’s a standard language used in many companies to help businesses access data smoothly. Because it’s widely used, the employment usually cites SQL as one of the necessary skills. That’s why it’s essential to learn about SQL.

One of the common problems for people when learning SQL is understanding the queries, mainly when another person writes them. We would work as a team in the companies and often need to read and understand their SQL queries. So, we need to practice deconstructing the SQL queries and understand them.

This article will walk through a step-by-step process to read and understand SQL queries. How do we do that? Let’s get into it.

1. Understand the General SQL Query Structure

The first thing we need to do when encountering an SQL query is to understand the general intention of the SQL queries. The general intention doesn’t mean we know entirely about the query's structure; it’s more about the overall flows.

We should understand the standard SQL queries to understand the general SQL query. Most SQL queries start with the SELECT clause and follow with the FROM clause. Continuing from there, the queries are most often followed by JOIN, WHERE, GROUP BY, ORDER BY, and HAVING clauses.

The clauses above are standard in the SQL queries we need to understand. For each clause, their functions are:

  1. SELECT: What columns are to be taken from the table
  2. FROM: Which table the data came from
  3. JOIN: Combine tables with the specified identifier
  4. WHERE: Data filter based on the condition
  5. GROUP BY: Organize the data based on the column's value and allow aggregation function to be performed.
  6. ORDER BY: Arrange the data result order based on the specific column
  7. HAVING: The filter condition for aggregation function that cannot be specified with WHERE

These are the standard clauses and what you should find in the beginning when understanding the general SQL query structure. Let’s use the example code to learn further.

SELECT     customers.name,     purchases.product,     SUM(price) as total_price   FROM     purchases     JOIN customers ON purchases.cust_id = customers.id   WHERE     purchases.category = 'kitchen'   GROUP BY     customers.name,     purchases.product   HAVING     total_price > 10000   ORDER BY     total_price DESC;  

When you look at the query above, try to identify the standard clauses. The clause would provide you with an understanding of what data are selected (SELECT), where it is from (FROM and JOIN), and the condition (WHERE, GROUP BY, ORDER BY, and HAVING).

For example, reading the query above would provide you understanding of the following:

  1. we try to get three different kinds of data: the Name from a table called customers, the Product from a table called purchases, and the aggregation of price columns that do not identify where the table is from and with the alias total_price (Information from clause SELECT).
  1. The overall data would come from purchases and customers tables that join together using the cust_id column from purchases and the id column from the customer's table (Information from clause FROM) and JOIN).
  1. We would only select the data with the category column value in the purchases table as ‘kitchen’ (Information from clause WHERE),
  1. group for the aggregation function with the name and product column that came from the respective table (Information from clause GROUP BY),
  1. filtered as well from the aggregation function result sum where the total_price is more than 10000 (information from clause HAVING), and
  1. order the data descending according to the total_price (information from clause ORDER BY).

That’s the general SQL query structure you need to know and identify. From there, we can explore further from the advanced query. Let’s get on with the next step.

2. Understand the Final Select

There would be a time when you would encounter a complex query where so many SELECT clauses exist in the same query. In this case, we should understand the final result of the query or the final (first) SELECT you see in the query. The key is to know what the query output wants to be.

Let’s use a more complex code like the one below.

WITH customerspending AS (    SELECT       customers.id,       SUM(purchases.price) as total_spending     FROM       purchases       JOIN customers ON purchases.cust_id = customers.id     GROUP BY       customers.id  )   SELECT     c.name,     pd.product,     pd.total_product_price,     cs.total_spending   FROM     (      SELECT         purchases.cust_id,         purchases.product,         SUM(purchases.price) as total_product_price       FROM         purchases       WHERE         purchases.category = 'kitchen'       GROUP BY         purchases.cust_id,         purchases.product       HAVING         SUM(purchases.price) > 10000    ) AS pd     JOIN customers c ON pd.cust_id = c.id     JOIN customerspending cs ON c.id = cs.id   ORDER BY     pd.total_product_price DESC;  

The query seems more complex and longer now, but the initial focus should be on the final SELECT, which seems to try to produce the customer’s total spending and purchase history. Try to assess what the final result wants to be and break it down from there.

3. Understand the Final Conditions Clause

We have the insight of the result should be from the queries. The next part is to see what the conditions for the final SELECT are. The conditions clause, including WHERE, GROUP BY, ORDER BY, and HAVING were the ones that controlled the overall data result.

Try to read and understand our query's conditions, and we will better understand our query's final result. For example, in our previous SQL query, the final condition is only the ORDER BY. This means the final result would be ordered by the total product price in descending order.

Knowing the final conditions would help you understand a significant part of the query and the overall query intention.

4. Understand the Final Join

Lastly, we need to understand where the data comes from. After we know about the data to select and the conditions to get them, we need to understand the source. The final JOIN clause would give us an understand of how the tables interact and the data flow.

For example, the previous complex query shows that we performed Join twice. It means we used at least three data sources for the final result. This information would be necessary in the later steps to understand further how each data source comes by, especially when the data source is coming from the subquery.

5. Reverse Order Reading and Repeat

After understanding how the final result should be and where it comes from, we need to look closer at the details. From here, we would backtrack into each subqueries and understand why they are structured like that.

However, we don’t try to look at them in the top-down structures. Instead, we should try to look at the subqueries that are closer to the final result, and we move up to the one furthest from the final result. From the code example above, we should try to understand this code first:

SELECT     purchases.cust_id,     purchases.product,     SUM(purchases.price) as total_product_price   FROM     purchases   WHERE     purchases.category = 'kitchen'   GROUP BY     purchases.cust_id,     purchases.product   HAVING     SUM(purchases.price) > 10000

Then, we are moving to the furthest code which is this one:

WITH customerspending AS (    SELECT       customers.id,       SUM(purchases.price) as total_spending     FROM       purchases       JOIN customers ON purchases.cust_id = customers.id     GROUP BY       customers.id  )

We can track down the author's thought process clearly when we break down each subquery from the one closer to the result to the furthest.

Try to repeat the process above if you need help understanding each subquery. With some practice, you will have a better experience in reading and understanding the query.

Conclusion

Reading and understanding SQL queries is a skill that everyone should have in the modern era, as every company deals with them. By using the following step-by-step guide, you would have a better time to understand the complex SQL queries. The steps include:

  1. Understand the General SQL Query Structure
  2. Understand the Final Select
  3. Understand the Final Conditions Clause
  4. Understand the Final Join
  5. Reverse Order Reading and Repeat

Cornellius Yudha Wijaya is a data science assistant manager and data writer. While working full-time at Allianz Indonesia, he loves to share Python and Data tips via social media and writing media.

More On This Topic

  • A Step-by-Step Guide to Web Scraping with Python and Beautiful Soup
  • A Guide On How To Become A Data Scientist (Step By Step Approach)
  • How To Structure a Data Science Project: A Step-by-Step Guide
  • Text-2-Video Generation: Step-by-Step Guide
  • SQL Simplified: Crafting Modular and Understandable Queries with CTEs
  • Multilabel Document Categorization, step by step example

Choosing the right technique: Prompt engineering vs fine-tuning

Artificial intelligence and machine learning applications have been revolutionizing many industries for the last decade, but due to generative AI models like ChatGPT, Bard, Midjourney, etc., they have become more popular and are being used by individuals and businesses that might never have previously considered using them.

Prompt-Engineering-vs-Fine-tuning-copy

Despite demonstrating tremendous potential, AI models, in reality, don’t have true intelligence. Instead, AI and ML applications are systems that can identify patterns within large and complex datasets and then present these relationships to users in a meaningful way. These systems heavily depend on model optimization techniques to improve performance and drive desirable, context/ domain-specific results.

Prompt engineering and fine-tuning are two key model optimization techniques used in training GenAI models, especially large language models.

What is prompt engineering?

Choosing the right technique: Prompt engineering vs fine-tuning

User inputs to a generative AI model play a pivotal role in influencing the result it produces. These inputs are referred to as prompts, and the process of writing these prompts is referred to as prompt engineering. A prompt engineer crafts optimal prompts to interact with other inputs in a GenAI tool. These prompts help coax generative AI applications into giving better answers, leading the model to improve its performance, such as writing articles, generating codes, interacting with customers, and even producing music and videos.

Prompt engineering to elicit better answers from the AI model

Your results are only as good as your prompts. Prompt optimization is the art of crafting more precise and detailed prompts or experimenting with words and phrases to elicit better outputs from a generative AI model. In other words, prompt engineers write a prompt in different ways to get the most desirable and contextually relevant results.

Here is an example. Say you need to write an email to inform about the new feature on your loan app, InstaCash. You write a simple prompt in ChatGPT: “Write a crisp email regarding the additional feature to our loan app.” The output looks like this:

Image-02

The content is quite ordinary and is not likely to catch the attention and interest of your target audience.

Let’s try again with a more carefully phrased and specific prompt: “Craft an email announcing the addition of the ‘Personal Loan Calculator’ feature to our InstaCash app. This feature allows borrowers to calculate monthly payments and the overall cost of their loans.” The output is:

Image-01

As you can see a prompt can be engineered by providing a generative AI model with more detailed and specific input.

What is fine-tuning

Choosing the right technique: Prompt engineering vs fine-tuning

Fine-tuning is another technique to unlock the full potential of generative AI models. Businesses can leverage fine-tuning for harnessing LLM capabilities to tackle specific tasks and achieve optimal results. The process involves building on the pre-existing GenAI model that has been trained and customizing it to suit the specific needs of individuals seeking to make the most of AI and align it with their personalized goals.

Essentially, fine-tuning is used to refine pre-trained models to deliver better performance on specific tasks by training them on a more carefully labeled dataset that is closely related to the task at hand. It enables models to adapt to niche domains, such as customer support, medical research, legal analysis, etc. The power of fine-tuning depends on the additional data and training. Additional datasets refer to new raw information, and training is often coupled with a feedback or rating system that evaluates the outputs produced by the model, and guides it to better results.

Fine-tuning an AI model incurs additional costs, but it is often much more cost-effective in the long run compared to training a language model from scratch. Moreover, using prompt engineering techniques for every single exchange with a chatbot is impractical.

Prompt engineering vs fine-tuning

Both prompt engineering and fine-tuning strategies play an important role in enhancing the performance of AI models. However, they are different from each other in several important aspects:

Prompt engineering empowers users to elicit highly accurate responses, whereas fine-tuning helps in optimizing the performance of a pre-training AI model on specific tasks.

Prompt engineering requires users to craft the prompt in different ways and provide more context by giving effective inputs, adding new information, clarifying the query, and even making requests in sequence. Fine-tuning, on the other hand, focuses on training an AI model on additional datasets to improve knowledge and tailor it to meet specific requirements of businesses.

While prompt engineering is a precision-focused approach that offers more control over a model’s actions and outputs, fine-tuning focuses on adding more in-depth information to topics that are relevant to the language model.

As prompts are created by humans, prompt engineering needs no or minimal computer resources. Fine-tuning is resource-intensive and involves training a language model on additional datasets, which may require substantial computing resources.

Prompt engineering and fine-tuning are both effective techniques for enhancing model performance in delivering accurate and desirable outputs. However, they play different roles in developing a model: while fine-tuning involves updating the model’s internal settings to customize it for specific needs, prompt engineering is the process of crafting optimal prompts during the interface to obtain better outputs from a generative AI model. The effectiveness of both techniques depends on the human experts in the loop.

Inference.ai matches AI workloads with cloud GPU compute

Inference.ai matches AI workloads with cloud GPU compute Kyle Wiggers 9 hours

GPUs’ ability to perform many computations in parallel make them well-suited to running today’s most capable AI. But GPUs are becoming tougher to procure, as companies of all sizes increase their investments in AI-powered products.

Nvidia’s best-performing AI cards sold out last year, and the CEO of chipmaker TSMC suggested that general supply could be constrained into 2025. The problem’s so acute, in fact, that it has the U.S. Federal Trade Commission’s attention — the agency recently announced it’s investigating several partnerships between AI startups and cloud giants like Google and AWS over whether the startups might have anti-competitive, privileged access to GPU compute.

What’s the solution? It depends on your resources, really. Tech giants like Meta, Google, Amazon and Microsoft are buying up what GPUs they can and developing their own custom chips. Ventures with fewer resources are at the mercy of the market — but it doesn’t have to be that way forever, say John Yue and Michael Yu.

Yue and Yu are the co-founders of Inference.ai, a platform that provides infrastructure-as-a-service cloud GPU compute through partnerships with third-party data centers. Inference uses algorithms to match companies’ workloads with GPU resources, Yue says — aiming to take the guesswork out of choosing and acquiring infrastructure.

“Inference brings clarity to the confusing hardware landscape for founders and developers with new chips coming from Nvidia, Intel, AMD, Groq [and so on] — allowing higher throughput, lower latency and lower cost,” Yue said. “Our tools and team allow for decision-makers to filter out a lot of the noise and quickly find the right fit for their project.”

Inference essentially provides customers a GPU instance in the cloud, along with 5TB of object storage. The company claims that — thanks to its algorithmic matching tech and deals with data center operators — it can offer dramatically cheaper GPU compute with better availability than major public cloud providers.

“The hosted GPU market is confusing and changes daily,” Yue said. “Plus, we’ve seen pricing vary up to 1000% for the same configuration. Our tools and team allow for decision makers to filter out a lot of the noise and quickly find the right fit for their project.”

Now, TechCrunch wasn’t able to put those claims to the test. But regardless of whether they’re true, Inference has competition — and lots of it.

See: CoreWeave, a crypto mining operation-turned-GPU provider, which is reportedly expected to rake in around $1.5 billion in revenue by 2024. Its close competitor, Lambda Labs, secured $300 million in venture capital last October. There’s also Together — a GPU cloud — not to mention startups like Run.ai and Exafunction, which aim to reduce AI dev costs by abstracting away the underlying hardware.

Inference’s investors seem to think there’s room for another player, though. The startup recently closed a $4 million round from Cherubic Ventures, Maple VC and Fusion Fund, which Yue says is being put toward build out Inference’s deployment infrastructure.

In an emailed statement, Cherubic’s Matt Cheng added:

“The requirements for processing capacity will keep on increasing as AI is the foundation of so many of today’s products and systems. We’re confident that the Inference team, with their past knowledge in hardware and cloud infrastructure, has what it takes to succeed. We decided to invest because accelerated computing and storage services are driving the AI revolution, and Inference product will fuel the next wave of AI growth.”

Jio Launches ‘Jio Brain’, 5G Integrated Generative AI Service

Jio Platforms Launches ‘Jio Brain’, Providing ML-as-a-service

Aayush Bhatnagar, Senior Vice President at Jio, revealed the launch of Jio Brain by Jio Platforms, positioned as the “Industry’s first 5G-integrated ML platform. Jio Brain aims to empower telecom networks, enterprise networks, and industry-specific IT environments to seamlessly incorporate ML tools into their day-to-day operations.

The cloud-native platform boasts over 500 REST APIs and data APIs, providing companies with the flexibility to craft tailored ML services according to their specific requirements.

A culmination of two years of intensive research involving hundreds of engineers, Jio Brain introduces an enterprise and mobile-ready LLM-as-a-service feature, allowing clients to harness generative AI capabilities.

Bhatnagar emphasised the transformative potential of Jio Brain, stating, “Jio Brain will help create new 5G services, transform enterprises, optimise networks, as well as set the stage for 6G development – where ML is a key capability.”

The services will include AI capabilities for images, videos, text, documents, and speech, a cloud-native solution with plug-and-play architecture, data integration functionalities, and multiple AI/ML embedded mobile and web applications.

Enterprises can leverage Jio Brain for natural language processing (NLP) and AI generation tasks, spanning image-to-video generation, text-to-music generation, text-to-image and video generation, speech-to-speech translation, speech-to-text translation, and more.

For coding purposes, the platform facilitates code generation, explanation, optimisation, and debugging. Additionally, Jio Brain encompasses core ML capabilities such as ML chaining, hyperparameter tuning, feature engineering, and more, which can be utilised collectively or as standalone services.

Despite the comprehensive unveiling, Jio has not disclosed a specific launch date for Jio Brain. The company expressed openness to collaboration with AI and ML researchers, aiming to scale up the platform and enhance its value proposition.

Recently, Akash Ambani, the chair of Jio also announced that it is partnering with IIT Bombay for the BharatGPT initiative. This is also after the announcement that Reliance is partnering with NVIDIA for utilising GH200 GPUs for building AI models in India.

The post Jio Launches ‘Jio Brain’, 5G Integrated Generative AI Service appeared first on Analytics India Magazine.

High-performance computing’s role in real-time graph analytics

A podcast with CEO Ricky Sun of Ultipa

High-performance computing’s role in real-time graph analytics

Image by Gerd Altmann from Pixabay

Relationship-rich graph structures can be quite complex and resource consuming to process at scale when using conventional technology. This is particularly the case when it comes to searches that demand the computation to reach 30 hops or more into the graphs.

Moreover, a key benefit of graph technology is ease of large-scale integration. When it comes to analytics, bringing all the relevant information together and processing it quickly is critical to effective discovery.

For that reason, high performance computing (HPC) methods that enable the processing of over a trillion floating point operations per second have been desirable for efficient, large=scale enterprise graph analytics. In 2012, for example, back in the early days of data lakes and rising demand for big data analytics, supercomputer provider Cray launched a subsidiary called YarcData that targeted the enterprise market for graph DBMSes.

YarcData’s Urika in-memory appliance available in 2013 featured a maximum of 512 terabytes of contiguous random access memory (RAM)–plenty of space to load entire large graphs using a range of different algorithms and visualization techniques.

Urika’s first users came from the knowledge-intensive verticals that needed to work with giant graphs for discovery purposes, including financial services (fraud detection, for example), pharmaceuticals (drug discovery) and cybersecurity. Customers paid upwards of $200K for each appliance. The high cost of the appliances compelled YarcData to offer a subscription pricing model as an alternative to buying the appliances outright.

HPC has moved up the performance curve and down the cost curve since then. Today, HPC Graph systems provider Ultipa claims to deliver real-time graph analytics at scale using high-performance computing methods cost effectively. Ultipa’s proprietary technologies include these:

  • Hybrid Transaction/Analytical Processing (HTAP): HTAP, a term Gartner coined in 2014, refers to scalable transactional processing ability in addition to analytical. Ultipa’s HTAP offers both horizontal (distributed across clusters) and vertical (per server) scaling.
  • High Density Parallel Computing (HDPC): HDPC is Ultipa’s term for its patent-pending concurrency capability, which offers near linear scaling. As the number of instances grows, so does the scale.

During this podcast recorded in mid-January, Ultipa CEO Ricky Sun shared a number of insights about how large banks are using the company’s HPC graph systems for real-time liquidity risk evaluation. Silicon Valley Bank’s collapse in March 2023 underscored why more banks are using HPC graph technology for their risk assessments now.

Hope you enjoy the podcast as much as I did recording it.

Podcast with Ricky Sun, CEO of Ultipa

Banks are Working on Tech to Counter Deepfakes

The realism of deepfakes has raised significant concerns within the financial sector. Many times, during the registration process for financial services, we are asked to submit a photo while holding our Aadhar card. With the advancement of generative AI, creating a virtually identical picture within seconds has become a feasible and worrisome prospect.

With generative AI, it is now even possible to generate hyper-realistic videos of individuals and this could enable fraudsters to exploit this technology to circumvent the verification processes established by banks.

“As deepfakes improve, I think it will only become harder over time to validate if the person on the other side is real or AI-generated. This problem will be bigger for banks that have more stringent regulatory requirements during onboarding,” Nitin Kamath, CEO of Zerodha, posted on X.

Cognizant of the advancements, the government has also already urged banks to prepare for the potential implications of deepfakes.

Banks have a deepfake problem

Banks and other financial services companies use eKYC/ Video-KYC features to onboard customers to their platforms. The Reserve Bank of India (RBI) does mandate KYC to be done through a live feed only, however, fraudsters can easily deceive live video verification sessions through a method known as a camera injection attack, introducing deepfake videos into the system.

For example, a man from Kerala fell victim to a scam where another man, pretending to be his friend, used deep fake technology over a call, resulting in a loss of INR 40,000.

For banks, deepfakes are also exacerbating the pre-existing issue of financial fraud. For decades, banks worldwide have contended with counterfeit documents and forged identity cards. In fact, fraudulent documentation emerged as one of the primary forms of bank fraud in 2022, as indicated by a survey conducted by Deloitte.

Now, with the advent of generative AI, the financial sector is presented with a fundamental problem. As Video KYC features can now be easily manipulated, should banks contemplate mandating customers to visit physical branches for identity verification?

Banks are working on countermeasures

Experts AIM has spoken to opine that forsaking the digital process is not currently under consideration by banks. Instead, the challenge provides banks with an opportunity to innovate and create technologies to combat the issue of deepfakes.

According to Aditya Gupta, chief executive officer & founder at Credilio, the traditional methods of eKYC and video-KYC authentication were developed before the onset of generative AI. Hence, banks and financial institutions will now need to introduce additional layers of security checks to nullify this threat.

Credilio, which uses video KYC to onboard customers to its platform, is working closely with its partner banks to analyse the risks emerging from advancements in deepfake tech. The solutions for introducing additional layers of security are currently in development and expected to be rolled out during the year,” he told AIM.

Moreover, digital KYC processes cannot be substituted with physical processes, given the mammoth scale of the financial services sector. “Going back to physical KYC processes may increase customer onboarding costs, making financial services inaccessible, hindering India’s financial inclusion and also the Digital India vision,” Supratim Chakraborty, Partner at Khaitan & Co, told AIM.

He believes that adopting future-proof technologies, such as ‘liveness detection systems’ and advanced image processing frameworks developed by certain startups, could significantly enhance the ability to identify and flag deepfakes.

These innovative solutions may prove instrumental in filtering out suspicious profiles, offering an effective means for banks to counter the challenges posed by the proliferation of deepfake technology in identity verification.

“Additional due diligence measures such as asking randomised questions to customers during the onboarding process, as well as recognising choppy sentences, varying tone, inflection in speech, phrasing, etc. may also help identify and flag suspicious users,” Chakraborty said.

Biometrics verification in KYC

However, according to a report from Netherland-based Sensity AI, liveness tests used for ID verification are susceptible to deepfake attacks. This is where technology which tracks one’s biometrics like fingerprints and iris can come in handy to tackle deepfake.

Banks’ onboarding partners are working on technology to facilitate fingerprint and iris scanning without the need for separate hardware.

Accura Scan, India’s only certified biometric solution provider, has unveiled the integration of contactless finger biometrics and document liveness checks into its digital ID products.

The company’s biometric fingerprint verification eliminates the necessity for extra hardware, utilising a device’s built-in camera to capture and analyse finger vein and fingerprint ridge patterns. This entirely touchless process is designed to be conducted remotely.

“As a step towards innovation, ‘identity fingerprinting’, or uniquely authenticating users through their facial, gestural, and vocal mannerisms in a privacy-preserving manner, may also be considered towards building trust in digital KYC processes,” Chakraborty added.

Moreover, as an alternative, Gupta suggests the creation of a neutral KYC repository, which is conducted in person for consumers and has validity for a longer duration for use by all regulated entities. “This should have biometric / iris checks to neutralise the threat of eKYC and Video KYC,” he pointed out.

AI against deepfake

Many financial services experts believe AI could also possibly be the strongest defence banks have against the deepfake menace. The right approach for banks is to stay ahead of evolving deepfake technologies and continually enhance their security measures.

Identity Verification and Digital KYC solutions providers across the globe are trying to use AI to track biometrics and make KYC processes safer and more secure. Moreover, AI can also be leveraged to detect adversarial attacks and other AI-based fraud tactics like camera injection attacks.

The post Banks are Working on Tech to Counter Deepfakes appeared first on Analytics India Magazine.

5 Free University Courses to Ace Coding Interviews

5 Free University Courses to Ace Coding Interviews
Image generated with Segmind SSD-1B model

Given how competitive the tech job market is right now, you should constantly upskill and improve your technical chops. For any role in data and software engineering, the interview process typically starts with a round or two of coding interviews.

While projects and technical expertise will help you in the later rounds of the interview, coding interviews are often hard to crack—especially if you haven’t been practicing for a while. And having a rock solid foundation in data structures and algorithms is necessary.

Even if you don't have a CS degree, taking university-level courses in programming, data structures, and algorithms will help you prepare for coding interviews. Because learning the fundamentals followed by several weeks of deliberate practice are both required for cracking coding interviews.

We’ve compiled a list of free university courses to help you learn data structures and algorithms. So let’s go over them.

1. Programming, Data Structures, and Algorithms Using Python – CMI

Programming, Data Structures, and Algorithms Using Python taught by Prof. Madhavan Mukund at Chennai Mathematical Institute is a great first course in data structures and algorithms using Python.

When preparing for coding interviews, you often have to understand advanced concepts. And you may find some university courses difficult to follow along. So this is a good first course if you haven't previously taken a course in data structures in algorithms.

I took this course during my undergrad days and found it super helpful. I highly recommend taking this course first before proceeding to the other courses.

This course has about 8 weeks of content. Here’s an overview of what the course covers:

  • Introduction to programming
  • Basics of Python
  • Search algorithms
  • Sorting algorithms
  • Built-in data structures in Python
  • Exception handling, file I/O, and string processing
  • Backtracking
  • Data structures such as stacks, queues, and heaps
  • Classes, objects, and user-defined data types
  • Dynamic programming

Course link: Programming, Data Structures and Algorithms Using Python

2. Algorithmic Toolbox – UCSD

Algorithmic Toolbox from UC San Diego is a great course to learn the fundamentals of problem solving techniques that’ll help you tackle coding interviews.

You’ll learn to first code a brute-force solution that works, gradually moving to more optimal solutions while learning techniques like dynamic programming. You can audit the course for free on Coursera and use a language that you’re comfortable programming in.

This course should take you a few weeks to work through. If you’re interested, you can also audit the entire Data Structures and Algorithms specialization for a more complete learning path.

The course contents include:

  • Programming challenges
  • Searching and sorting algorithms
  • Greedy algorithms
  • Divide and conquer
  • Dynamic programming

Course link: Algorithmic Toolbox

3. Introduction to Algorithms – MIT

Introduction to Algorithms from MIT is one of the most popular highly recommended algorithms courses.

If you have some programming experience and are already familiar with the basics of data structures and algorithms, then this course will help you level up. And learn the basics of common data structures algorithms and algorithmic paradigms.

You can access the course materials: lecture notes, problem sets, and solutions for free on the course website. Here’s an overview of what the course covers:

  • Computational complexity of algorithms
  • Searching and sorting
  • Graph algorithms
  • Dynamic programming

Course link: Introduction to Algorithms

4. Algorithms: Design and Analysis, Part 1 – Stanford

Thought by Prof. Tim Roughgarden during his time at Stanford university, the Design and Analysis of Algorithms courses (this part and the next) will help you push yourself hard to improve your algorithmic thinking and problem-solving skills.

If you have the time during interview prep, I recommend taking this course and the next. It’ll be helpful to have a strong foundation from one or more of the previous courses before you dive into this algorithms course.

In part 1 of this course on design and analysis of algorithms you will learn:

  • Big-O notation
  • Searching and sorting
  • Divide and conquer
  • Randomized algorithms
  • Data structures such as hash tables and Bloom filters
  • Algorithms on graphs

Course link: Algorithms: Design and Analysis, Part 1

5. Algorithms: Design and Analysis, Part 2 – Stanford

In this part 2 of the Design and Analysis of Algorithms course, you’ll get to learn more advanced concepts including:

  • Greedy algorithms
  • Dynamic programming
  • NP completeness
  • Heuristics analysis
  • Local search

You can watch the lectures on YouTube or audit the course for free on edX. These courses are also available as a five-course specialization on Coursera. So if you prefer this version, you can audit this Algorithms Specialization for free on Coursera.

Course link: Algorithms: Design and Analysis, Part 2

Wrapping Up

I hope you found useful resources to help in your coding interview prep.

Before you start preparing for coding interviews, however, you should refresh programming concepts and focus on becoming familiar with the features of the specific language. This will help you choose the right built-in data structures to design algorithms with the optimal space and runtime complexity.

Good luck cracking coding interviews and landing your dream role! If you're looking for some actionable tips on landing data science jobs, check out 7 Reasons Why You're Struggling to Land a Data Science Job.

Bala Priya C is a developer and technical writer from India. She likes working at the intersection of math, programming, data science, and content creation. Her areas of interest and expertise include DevOps, data science, and natural language processing. She enjoys reading, writing, coding, and coffee! Currently, she's working on learning and sharing her knowledge with the developer community by authoring tutorials, how-to guides, opinion pieces, and more.

More On This Topic

  • 7 Must-Know Python Tips for Coding Interviews
  • KDnuggets News, May 4: 9 Free Harvard Courses to Learn Data…
  • 5 Free University Courses on Data Analytics
  • 5 Free University Courses to Learn Python
  • 5 Free University Courses to Learn Data Science
  • Free Python Project Coding Course

Kore.ai, a startup building conversational AI for enterprises, raises $150M

Kore.ai, a startup building conversational AI for enterprises, raises $150M Kyle Wiggers 8 hours

In the midst of a wave of tech industry layoffs, it’s heartening to see some startups succeeding despite the dour market outlook.

Kore.ai, a company developing enterprise-focused conversational AI and GenAI products, today announced that it raised $150 million in a funding round led by FTV Capital, Nvidia, Vistara Growth, Sweetwater PE, NextEquity, Nicola and Beedie. Bringing the company’s total raised to ~$223 million, the new cash will be put toward product development and scaling up Kore.ai’s workforce, co-founder and CEO Raj Koneru told me in an interview.

Koneru started Kore.ai in 2014 after launching Kony, a mobile app development startup, and several other small companies including iTouchPoint (an outsourcing firm) and Intelligroup (a tech consultancy). He says he was inspired to found Kore.ai after seeing the potential of AI, particularly large language models (LLMs) along the lines of OpenAI’s ChatGPT, to transform user experiences.

“With the introduction of GenAI and LLMs, the tech landscape turned out to be very chaotic and uncertain due to rapid advancements,” Koneru said via email. “There were more questions than answers … but I saw conversational AI and LLMs as an opportunity to innovate.”

GenAI being a newer discipline, Kore.ai wasn’t developing GenAI products in 2014 per se. But Koneru says that the company was laying the foundations for GenAI products to come — investing heavily in text-generating and -analyzing models.

So how’s Kore.ai innovating? Well, as Koneru describes it, the startup provides a no-code platform to help companies power various “business interactions” via AI — essentially any customer-to-employee or employee-to-employee interaction over the phone or text (think support chats with an IT/HR service desk). Kore.ai offers workflows and tools designed to give companies in industries such as banking, healthcare and retail the ability to create custom conversational AI apps or deploy pre-built, “domain-trained” chatbots.

“Kore.ai’s platform encompasses intelligent virtual agent, contact center AI, agent AI and search and answer capabilities for all kinds of customer experience and employee experience use cases,” Koneru said. “In addition, Kore.ai’s array of industry and horizontal solutions address the needs of specific industries and enterprise functions.”

But aren’t there lots of vendors building GenAI- and LLM-powered solutions for search, question-answering and the other sorts of applications Kore.ai advertises supporting? Indeed, there are.

See Acree, which hosts a platform for building corporate GenAI apps, and Giga ML, which offers tools to help companies deploy LLMs offline. Reka and Contextual AI both recently emerged from stealth to help create custom AI models for organizations, while Fixie is crafting tools to make it easier for companies to code on top of LLMs.

What Kore.ai does differently, Koneru asserts, is offer great flexibility where it concerns where companies can deploy their AI apps — in the cloud, locally or in virtual machines — and the degree to which they can fine-tune these apps. For certain applications (e.g. text summarization, finding and generating answers, topic discovery and sentiment analysis), Koneru makes the case that fine-tuned models — Kore.ai’s speciality — are superior to the larger, more powerful models available from vendors like Anthropic and OpenAI, as well as more cost-effective.

There’s a privacy argument to be made, too, for smaller, offline models.

A 2023 Predibase survey found that more than 75% of enterprises don’t plan on using commercial, cloud-hosted LLMs in production over fears that the models will compromise sensitive info. In a separate poll from GenAI platform Portal26 and data research firm CensusWide, 85% of businesses said that they’re concerned about GenAI’s privacy and security risks.

Kore.ai

Creating a GenAI or conversational AI workflow using Kore.ai’s web tooling.

“Over the past 18 months, we’ve observed that fine-tuned models are very effective compared to pre-trained models for specific enterprise use cases,” Koneru said. “Compared to a large pre-trained model, it takes less than 2% of the enterprise data to train and create a fine-tuned model that companies can deploy safely for enterprise use cases. We’ve successfully built smaller enterprise LLMs that provide higher efficiency, better accuracy, the ability to control responses and — most importantly — reduce latency and cost.”

Also unlike some rivals, Kore.ai offers ways for organizations to scale up their AI as needed, Koneru says, and expand their use of AI into new and diverse domains.

“Kore.ai sits above the infrastructure and fragmentation of all the LLM layers with a platform-driven approach, offering freedom of choice with built-in guardrails for effective AI implementation,” Koneru added.

Now, the extent to which these capabilities are truly differentiating is subject to debate. Vendors like Google Cloud, Azure and AWS offer robust scaling solutions for conversational AI and GenAI apps, and Kore.ai isn’t the only platform to let customers deploy models in a range of local and cloud compute environments.

But — whether on the strength of its platform, nearly-1,000-person-workforce, marketing campaign or all three — Orlando, Florida-based Kore.ai has established an impressive foothold in the competitive AI field. The company’s customer base eclipsed 400 brands (including PNC, AT&T, Cigna, Coca-Cola, Airbus and Roche) last year, and its annual recurring revenue now stands north of $100 million — thanks to income from licensing and usage fees in addition to consulting services.

It probably helps that funding for GenAI startups of all stripes remains strong. According to a recent survey from GlobalData, the London-based data analytics and consulting firm, GenAI startups raised a record $10 billion in 2023 — a 110% increase compared to 2021.

The question is whether the growth is sustainable, given that GenAI isn’t a home run in the enterprise — at least not yet. Koneru argues that it is, pointing to surveys like Gartner’s from last October, which found that 55% of organizations are already piloting or deploying GenAI tech into production for functions such as customer service, marketing and sales.

“We haven’t observed any slowdown in the market,” Koneru said. “The most pressing challenge [we’re facing] is to operate and innovate in a market that’s not just seen rapid growth but also disruption driven by advancements in technology, changing user expectations and a broader integration of newer AI capabilities that are evolving each day. Enterprise players need to take advantage of the benefits of technology while avoiding security, privacy and compliance pitfalls.”

Added FTV Capital’s Kapil Venkatachalam in a statement: “While the advanced AI market has experienced rapid growth in recent years, many enterprises are grappling with how to responsibly and effectively deploy AI across their organizations. We were impressed with Kore.ai’s open platform approach for leveraging AI models, scalability, vertical specific out-of-the-box applications and low-code no-code capabilities, making them well-positioned to take advantage of the growing demand from global brands looking for innovative AI solutions to enhance business interactions and drive value.”