Рубрика: AI

A few weeks ago, Meta CEO Mark Zuckerberg announced via Facebook that his company is open-sourcing its large language model (LLM) Code Llama, which is an artificial intelligence (AI) engine similar to GPT-3.5 and GPT-4 in ChatGPT.

Zuck announced three interesting things about this LLM: it's being open-sourced, it's designed to help write and edit code, and its model has 70B parameters. The hope is that developers can feed the model more challenging problems, and the engine will be more accurate when it answers.

Also: Why open-source generative AI models are still a step behind GPT-4

The open-sourcing issue is interesting. It's an approach that implies that you could download the whole thing, install it on your own server, and use the model to get programming help without ever taking the risk that the Overlords of Facebook will hoover up your code for training or other nefarious purposes.

Doing this work involves setting up a Linux server and doing all sorts of hoop jumps. However, it turns out that the specialists at Hugging Face have already implemented the Code Llama 70B LLM into their HuggingChat interface. So, that's what I'm going to test next.

Getting started with Code Llama

To get started, you'll need to create a free account on Hugging Face. If you already have one (as I do), you can use the 70B Code Llama LLM with that account.

Also: GitHub: AI helps developers write safer code, but you need to get the basics right

One thing that's important to note is that, while you could install Code Llama on your own server and thereby not share any of your code, the story is far different on Hugging Face. That service says that anything you type in might be shared with the model authors unless you turn off that option in settings:

When you log in to HuggingChat, you'll be presented with a blank chat screen. As you can see below, my current LLM is openchat/openchat-3.5-0106, but I'm going to change it to Code Llama — and I'll show you how.

You change your current model in the settings, which you can get to by hitting the gear icon:

Once in settings, click (at 1) the codellama/CodeLlama-70b-Instruct-hf on the left, verify (at 2) that the Code Llama LLM has been selected, and then click Activate (at 3):

Now, when you talk to the chat interface, you'll be using the Code Llama model, as verified at the top of the chat interface:

To test, I decided to pull prompts from a previous coding test run I conducted with Bard (now Gemini) and I ran the same tests in HuggingChat.

Test 1: Writing a WordPress plugin

My first test was the creation of a WordPress plugin. ChatGPT performed quite well at this task. Bard was weak, but tried its best. But how about Code Llama? Well, let's see. Here's the prompt:

Write a PHP 8 compatible WordPress plugin that provides a text entry field where a list of lines can be pasted into it and a button, that when pressed, randomizes the lines in the list and presents the results in a second text entry field with no blank lines and makes sure no two identical entries are next to each other (unless there's no other option)…with the number of lines submitted and the number of lines in the result identical to each other. Under the first field, display text stating "Line to randomize: " with the number of nonempty lines in the source field. Under the second field, display text stating "Lines that have been randomized: " with the number of non-empty lines in the destination field.

And here are the results, such as they are:

That ain't right on so many levels. First, Code Llama didn't create the plugin header, a very simple set of fields required by all plugins. Then, it generated code that my programming editor's code formatter couldn't interpret, indicating that there are missing elements in the code.

What's more, the code can't be tested. This is a fail.

Test 2: Rewriting a string function

For this test, I used a string rewriting request I once gave ChatGPT. I wanted to fix a bug in my code that was only allowing integers when it should have been allowing for dollars and cents (in other words, some number of digits, possibly followed by a period, and then, if there is a period, two more digits).

I fed in the same prompt I used in my previous article and got back this response:

This code works and I'm satisfied with it.

Test 3: Finding a bug I couldn't find

Again, I reused a test I wrote about in a previous article. I'll point you to the original article if you want the details of the problem I tried out on Code Llama. The coding problem is long and fairly convoluted, which is why I couldn't find out what was wrong.

ChatGPT solved the problem immediately; Bard did not. Bard failed because it looked at the surface of the problem, not how the overall code was constructed and needed to run. An analogy is going to the doctor with a headache. One doctor might tell you to take two aspirin and not call him in the morning. The other doctor might try to find out the root cause of the headache and help solve that.

Also: How to use ChatGPT to write code

ChatGPT zeroed in on the root cause, and I was able to fix the bug. Bard just looked at the symptoms and didn't come up with a fix.

Unfortunately, Code Llama did exactly the same thing as Bard, looking at just the surface of the problem. The AI made recommendations, but those recommendations didn't improve the situation.

And the winner is…

My test suite is far from comprehensive. But if Code Llama fails on two of the three tests that didn't even slow down ChatGPT, it seems like the AI isn't ready for prime time.

The only reason you might want to use Code Llama over ChatGPT is if you install it on your own server because then your code won't be shared with Meta. But what good is privacy if the thing doesn't give correct answers?

If ChatGPT hadn't been so good, I probably would have given some points to Code Llama. But we know what's possible with ChatGPT — and Code Llama is far from that level. In short, it looks like Facebook has to Zuck it up and make some improvements.

Also: Implementing AI into software engineering? Here's everything you need to know

To be honest, I expected better and I'm a little disappointed. But if there's one thing tech columnists get used to, it's being a little disappointed by many of the products and projects we look at. I think that's why we get so excited when something stands out and rocks our world. And Code Llama, unfortunatey, isn't one of those.

Have you tried any of the AIs for coding help? Which ones have you used? How have they worked out? Let us know in the comments below.

You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter on Substack, and follow me on Twitter at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.

Artificial Intelligence

A few weeks ago, Meta CEO Mark Zuckerberg announced via Facebook that his company is open-sourcing its large language model (LLM) Code Llama, which is an artificial intelligence (AI) engine similar to GPT-3.5 and GPT-4 in ChatGPT.

Zuck announced three interesting things about this LLM: it's being open-sourced, it's designed to help write and edit code, and its model has 70B parameters. The hope is that developers can feed the model more challenging problems, and the engine will be more accurate when it answers.

Also: Why open-source generative AI models are still a step behind GPT-4

The open-sourcing issue is interesting. It's an approach that implies that you could download the whole thing, install it on your own server, and use the model to get programming help without ever taking the risk that the Overlords of Facebook will hoover up your code for training or other nefarious purposes.

Doing this work involves setting up a Linux server and doing all sorts of hoop jumps. However, it turns out that the specialists at Hugging Face have already implemented the Code Llama 70B LLM into their HuggingChat interface. So, that's what I'm going to test next.

Getting started with Code Llama

To get started, you'll need to create a free account on Hugging Face. If you already have one (as I do), you can use the 70B Code Llama LLM with that account.

Also: GitHub: AI helps developers write safer code, but you need to get the basics right

One thing that's important to note is that, while you could install Code Llama on your own server and thereby not share any of your code, the story is far different on Hugging Face. That service says that anything you type in might be shared with the model authors unless you turn off that option in settings:

When you log in to HuggingChat, you'll be presented with a blank chat screen. As you can see below, my current LLM is openchat/openchat-3.5-0106, but I'm going to change it to Code Llama — and I'll show you how.

You change your current model in the settings, which you can get to by hitting the gear icon:

Once in settings, click (at 1) the codellama/CodeLlama-70b-Instruct-hf on the left, verify (at 2) that the Code Llama LLM has been selected, and then click Activate (at 3):

Now, when you talk to the chat interface, you'll be using the Code Llama model, as verified at the top of the chat interface:

To test, I decided to pull prompts from a previous coding test run I conducted with Bard (now Gemini) and I ran the same tests in HuggingChat.

Test 1: Writing a WordPress plugin

My first test was the creation of a WordPress plugin. ChatGPT performed quite well at this task. Bard was weak, but tried its best. But how about Code Llama? Well, let's see. Here's the prompt:

Write a PHP 8 compatible WordPress plugin that provides a text entry field where a list of lines can be pasted into it and a button, that when pressed, randomizes the lines in the list and presents the results in a second text entry field with no blank lines and makes sure no two identical entries are next to each other (unless there's no other option)…with the number of lines submitted and the number of lines in the result identical to each other. Under the first field, display text stating "Line to randomize: " with the number of nonempty lines in the source field. Under the second field, display text stating "Lines that have been randomized: " with the number of non-empty lines in the destination field.

And here are the results, such as they are:

That ain't right on so many levels. First, Code Llama didn't create the plugin header, a very simple set of fields required by all plugins. Then, it generated code that my programming editor's code formatter couldn't interpret, indicating that there are missing elements in the code.

What's more, the code can't be tested. This is a fail.

Test 2: Rewriting a string function

For this test, I used a string rewriting request I once gave ChatGPT. I wanted to fix a bug in my code that was only allowing integers when it should have been allowing for dollars and cents (in other words, some number of digits, possibly followed by a period, and then, if there is a period, two more digits).

I fed in the same prompt I used in my previous article and got back this response:

This code works and I'm satisfied with it.

Test 3: Finding a bug I couldn't find

Again, I reused a test I wrote about in a previous article. I'll point you to the original article if you want the details of the problem I tried out on Code Llama. The coding problem is long and fairly convoluted, which is why I couldn't find out what was wrong.

ChatGPT solved the problem immediately; Bard did not. Bard failed because it looked at the surface of the problem, not how the overall code was constructed and needed to run. An analogy is going to the doctor with a headache. One doctor might tell you to take two aspirin and not call him in the morning. The other doctor might try to find out the root cause of the headache and help solve that.

Also: How to use ChatGPT to write code

ChatGPT zeroed in on the root cause, and I was able to fix the bug. Bard just looked at the symptoms and didn't come up with a fix.

Unfortunately, Code Llama did exactly the same thing as Bard, looking at just the surface of the problem. The AI made recommendations, but those recommendations didn't improve the situation.

And the winner is…

My test suite is far from comprehensive. But if Code Llama fails on two of the three tests that didn't even slow down ChatGPT, it seems like the AI isn't ready for prime time.

The only reason you might want to use Code Llama over ChatGPT is if you install it on your own server because then your code won't be shared with Meta. But what good is privacy if the thing doesn't give correct answers?

If ChatGPT hadn't been so good, I probably would have given some points to Code Llama. But we know what's possible with ChatGPT — and Code Llama is far from that level. In short, it looks like Facebook has to Zuck it up and make some improvements.

Also: Implementing AI into software engineering? Here's everything you need to know

To be honest, I expected better and I'm a little disappointed. But if there's one thing tech columnists get used to, it's being a little disappointed by many of the products and projects we look at. I think that's why we get so excited when something stands out and rocks our world. And Code Llama, unfortunatey, isn't one of those.

Have you tried any of the AIs for coding help? Which ones have you used? How have they worked out? Let us know in the comments below.

You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter on Substack, and follow me on Twitter at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.

Artificial Intelligence

Image created by me with Microsoft Image Creator

Since OpenAI unleashed ChatGPT to the public, a flurry of discussions has emerged online about a new dream job: Prompt Engineering. It's touted as "AI's Hottest Job," promising six-figure salaries without the need for programming experience. Enthusiasts describe it as a job of the future, where anyone can earn up to $335K by smooth-talking a cool know-it-all robot into giving right answers. No surprise, Instagram money making sages, YouTube career preachers, and self-proclaimed oracles of TikTok have been very vocal about it. While this sounds like a dream job, is it truly achievable? Let's delve into the reality of job market behind the hype to find out.

Analyzing job advertisement data provides valuable insights into labor demand trends, responsibilities, qualifications, and salary expectations. Thus, I decided to take a look at the ad data of the so-called “AI’s Hottest Job” with no speculations or presumptions. I collected 73 recently posted unique job ads data from popular on-line job posting platforms. Read about my data collection methodology and access the data set here. While 73 may not be an ideal sample size, it's a comprehensive starting point for our analysis. The initial revelation is sobering: there's a scarcity of employers seeking "prompt engineers."

Now, let’s take a look at the data. The most frequently mentioned job title is "prompt engineer." However, other titles such as "IT Innovation Analyst," "Freelance ML/AI Engineer," "Data Scientist," and "AI Engineer" also emerge. I created word clouds for qualifications and responsibilities mentioned in the job descriptions. I don’t think word clouds are meant to reveal extraordinary insights, but they can represent a compact version of the important highlights in the text. As you see, in the job ads employers are talking about experience in computer science, model development, python, prompt design, machine learning, large language models, natural language processing, and artificial intelligence more than other stuff.

1.This is a significantly larger sample size if you compare it with many of those anecdotal early articles who constructed their whole argument about a six-figure salary with no coding from only one job ad.

Next, I used ChatGPT and Claude to summarize the collected ads text corpus to identify top prompt engineering qualifications and qualifications. I did multiple rounds of prompting with different approaches followed by manually checking the data to make sure I got stable and valid output.

Essential qualifications demanded for Prompt Engineer job:

1. Proficiency in Python programming (2-5 years of experience) including experience with AI/machine learning frameworks like TensorFlow, PyTorch, Keras.
2. Working knowledge of NLP and LLMs (2-5 years of experience) like BERT, GPT-3/4, T5, etc. Knowledge of how these models work and how to fine-tune them.
3. Strong analytical and problem-solving skills. The ability to think critically, design effective prompts, analyze model performance, and troubleshoot issues is vital.
4. Expertise in prompt engineering principles and techniques like chain of thought, in-context learning, tree of thought, etc. This allows guiding the models to desired outcomes.
5. Excellent communication skills, both verbal and written. This is needed for collaborating across teams, explaining technical concepts, and documenting work.

And the essential responsibilities of the prompt engineering jobs are:

1. Prompt Design and Optimization: Designing, developing, testing, and refining AI-generated text prompts to maximize effectiveness for various applications. This includes utilizing techniques like transfer learning and leveraging linguistic expertise to craft high-quality and diverse prompts.
2. Integration and Deployment: Ensuring seamless integration of optimized prompts into the overall product or system. Collaborating with engineers to implement prompts and models into production environments.
3. Performance Evaluation and Improvement: Rigorously evaluating prompt performance using metrics and user feedback. Conducting continuous testing and analysis to identify areas for optimization and prompt iteration.
4. Collaboration and Requirements Gathering: Working closely with cross-functional teams like data scientists, content creators, and product managers to understand requirements and ensure prompts align with business goals and user needs.
5. Knowledge Sharing: Documenting prompt engineering processes and outcomes. Educating teams on prompt best practices. Keeping updated on the latest AI advancements to bring innovative

It is fair to say that “no programming experience” premise of the so called “AI’s Hottest Job” is far from the reality, since the top demanded skills in the prompt engineering market are programming proficiency and NLP and LLMs experience. And they are not talking about micky mouse programming skills, they are looking for experts who are familiar with ML and AI frameworks. The employers are not just requiring “familiarity” with LLMs and coding, but, on average, they seek experts with 2-5 years of experience working with structured and unstructured data, coding, NLP, ML, and AI.

Reading the top responsibilities make it more clear why this job title demands such a high level of programming and LLMs skills. Prompt engineering, as a professional job, is not sitting behind a computer and playing with Generative AI models to give you the right answer. It is about building business information systems that optimize inputs, integrate them seamlessly with other information systems and products, and deliver values to users and customers. In other words, businesses are not looking for someone who can chat with ChatGPT, they want to hire experts who can optimize GPT-like models and integrate them with their own products.

Job ads data analysis of degree requirements indicates a preference for technical educational backgrounds in computer science, math, analytics, engineering, physics, or linguistics. A bachelor’s degree in computer science or a related field is commonly required, with more advanced degrees preferred or required for senior roles. The salaries are very different depending on the responsibilities and seniority. It can be as low as 30k and as high as half a million dollars per year. On average, the job ads with salary information pay between 90k and 195k a year.

Despite initial enthusiasm, doubts regarding the viability of prompt engineering as a dream job have surfaced. As Ethan Mollick, the Wharton School professor, wrote in a twitter post last year “prompt engineer is not a job of future” because “AI gets easier” and smarter in interpreting basic prompts. A month ago Coursera published a well-thought career guide for prompt engineering (also see this). It seems the initial Gen AI fad is slowly fading, and we are in a better position to understand AI’s current status and future trends. Don’t get me wrong. The quality of Gen AI outputs strongly depends on the inputs. Learning how to use and interact with these complex models is becoming an important skill for almost everyone. There are increasing number of scientific studies that suggest a systematic approach to prompting can significantly improve the outcome of these models (see 1, 2, 3, 4, 5, 6, 7). However, “prompt engineering” is not (and it never was) a dream job that some people wanted it to be. Without significant experience in programming, natural language processing, machine learning, product development, and software integration no one is going to pay you a six-figure salary for just smooth-talking ChatGPT into a right answer.

The present and future of prompt engineering, and Gen AI applications, seem to be influenced by two important trends: first, as Ethan Mollick mentioned, Gen AI models are getting more adept at generating good outputs from unsophisticated simple prompt, perhaps similar to how Internet search engines have become better at returning more relevant results from simple search queries. Second, Gen AI models are being increasingly integrated into the business’s products, services, and platforms. This adaptation is crucial for the success of the AI economy. Therefore, knowing how to optimize, fine-tune, customize, and integrate Gen AI models with the current information systems and products is and will remain a valuable skill set. That’s why in the current prompt job ads, there is a huge demand for programmers, system designers, and those who can collaborate with other product development team members.

Mahdi Ahmadi is a clinical assistant professor at the Information Technology & Decision Sciences department at the University of North Texas where I teach data mining, business intelligence, and data analytics. My primary research area is the application of machine learning and data mining techniques in businesses. I also provide consultation to businesses, higher education institutions, and non-for-profit organizations on their data analytics problems.

More On This Topic

• Dream Come True: Building websites by thinking about them
• From Fiction to Reality: ChatGPT and the Sci-Fi Dream of True AI…
• Free Data Science Interview Book to Land Your Dream Job
• The Art of Prompt Engineering: Decoding ChatGPT
• Some Kick Ass Prompt Engineering Techniques to Boost our LLM Models
• Why Prompt Engineering is a Fad

Dili wants to automate due diligence with AI Kyle Wiggers 10 hours

Stephanie Song, formerly on the corporate development and ventures team at Coinbase, was often frustrated by the volume of due diligence tasks she and her team had to complete on a daily basis.

“Analysts burn the midnight oil working hundreds of hours doing the work that nobody wants to do,” Song told TechCrunch in an email interview. “At the same time, funds are deploying less capital and looking for ways to make their teams more efficient while reducing operating costs.”

Inspired to find a better way, Song teamed up with Brian Fernandez and Anand Chaturvedi, two ex-Coinbase colleagues, to launch Dili (not to be confused by the capital of East Timor), a platform that attempts to automate key investment due diligence and portfolio management steps for private equity and VC firms using AI.

Dili, a Y Combinator graduate, has raised $3.6 million in venture funding to date from backers including Allianz Strategic Investments, Rebel Fund, Singularity Capital, Corenest, Decacorn, Pioneer Fund, NVO Capital, Amino Capital, Rocketship VC, Hi2 Ventures, Gaingels and Hyper Ventures.

“[AI] affects all parts of an investment fund, from analysts to partners and back-office functions,” Song said. “Investment professionals at funds are looking for a differentiated edge on decision-making, and can now use their wealth of data to combine their understanding of the deal with how it fits into the funds … Dili has a unique opportunity to emerge as a solution for funds in a harsh macro environment.”

Song’s not wrong about funds looking for an edge — or any new promising ways to mitigate investing risk, for that matter. VCs reportedly have $311 billion in unspent cash, and last year raised the lowest total — $67 billion — in seven years as they grew increasingly cautious about early-stage ventures.

Dili isn’t the first to apply AI to the due diligence process. Gartner predicts that by 2025, more than 75% of VC and early-stage investor executive reviews will be informed using AI and data analytics.

Several startups and incumbents are already tapping AI to pour through financial documents and copious amounts of data to craft market comparisons and reports — including Wokelo (whose customers are private equity and VC funds, like Dili’s), Ansarada, AlphaSense and Thomson Reuters (through its Clear Adverse Media unit).

But Song insists that Dili uses “first-of-its-kind” technology.

“[We can] deliver very high accuracy on specific tasks like pulling financial metrics from large unstructured documents,” she added. “We’ve built custom indexing and retrieval pipelines tuned for specific documents to provide [our AI] models with high quality context.”

Dili leverages GenAI, specifically large languages models along the lines of OpenAI’s ChatGPT, to streamline investor workflows.

The platform first catalogs a fund’s historical financial data and investment decisions in a knowledge base, and then applies the aforementioned models to automate tasks such as parsing databases of private company data, handling due diligence request lists and digging for little-known figures across the web.

Dili recently added support for automated comparable analysis and industry benchmarking on a firm’s backlog of deals. Once funds upload their deal data, they can compare historical and current investment opportunities in one place.

“Imagine being able to get an email with a new investment opportunity or portfolio company update and instantly having a platform produce AI-generated deal red flags, competitive analysis, industry benchmarking and a preliminary summary or memo leveraging your fund’s historical investing patterns,” Song said.

The question is, can Dili’s AI — or any AI really — be trusted when it comes to managing a portfolio?

Image Credits: Dili

AI isn’t necessarily known for sticking to facts, after all. Fast Company tested ChatGPT’s ability to sum up articles and found that the model had a tendency to get stuff wrong, leave pieces out and outright invent details not mentioned in the articles it summarized. It’s not tough to imagine how this might become a real problem in due diligence work, where accuracy is paramount.

AI can also bring prejudices into the decisioning process. In an experiment conducted by Harvard Business Review several years ago, an algorithm trained to make startup investment recommendations was found to pick white entrepreneurs rather than entrepreneurs of color and preferred investing in startups with male founders. That’s because the public data the algorithm was trained on reflected the fact that fewer women and founders from underrepresented groups tend to be disadvantaged in the funding process — and ultimately raise less venture capital.

Then there’s the fact that some firms might not be comfortable running their private, sensitive data through a third-party model.

In a survey from Bloomberg Law, 30% of deal lawyers said they wouldn’t consider using AI as it exists today at any stage of the due diligence process, citing concerns including violating confidentiality agreements associated with deals by entering third-party info into AI software.

To attempt to allay all those fears, Song said that Dili is continuing to fine-tune its models — many of which are open source — to reduce instances of hallucination and improve overall accuracy. She also stressed that private customer data isn’t used to train Dili’s models and that Dili plans to offer a way for funds to create their own models trained on proprietary, offline fund data.

“While hedge funds and public markets have invested heavily in tech, private market data has a lot of untapped potential that Dili could unlock for firms,” Song said.

Dili ran an initial pilot last year with 400 analysts and users across different types of funds and banks. But as the startup expands its team and adds new capabilities, it’s angling to expand into new applications — ultimately toward becoming an “end-to-end” solution for investor due diligence and portfolio management, Song says.

“Eventually we believe this core technology we’re building can be applied to all parts of the asset allocation process,” she added.

In recent years, the realm of artificial intelligence has witnessed a transformative shift with the advent of generative AI, particularly in the field of video creation. This emerging technology has redefined the boundaries of digital content generation, allowing for the creation of vivid, imaginative, and incredibly realistic visuals. Amidst this technological evolution, OpenAI, a leading name in AI research and innovation, has unveiled its groundbreaking project: Sora. Sora, a text-to-video generation tool, marks a significant leap forward in the AI-driven creative landscape, promising to turn simple textual descriptions into rich, dynamic video content.

The Capabilities of Sora

Sora emerges as a pinnacle of AI-driven creativity, showcasing an extraordinary ability to create photorealistic videos from mere text prompts. This advanced model ushers in a new era of content generation, where the lines between reality and AI-generated content blur. Sora's capabilities extend far beyond basic video creation; it can conjure up complex scenes with multiple characters, each interacting within intricately detailed backgrounds. The model demonstrates an acute understanding of the physical world, allowing it to render objects and environments with striking realism.

One of the most intriguing aspects of Sora is its profound comprehension of motion and emotion. The model is adept at creating characters that not only move naturally but also exhibit a spectrum of emotions, lending a layer of depth and realism previously unseen in AI-generated content. This level of detail in character portrayal opens up new possibilities for storytelling and digital artistry.

Moreover, Sora's versatility is highlighted by its ability to interact with still images. This feature enables users to transform a single frame into a fluid, dynamic video, expanding the creative possibilities. Additionally, Sora can enhance existing videos, filling in missing frames or extending clips, thereby providing a tool for both creation and augmentation of visual content. This dual capability of Sora positions it as a versatile tool in the arsenal of filmmakers, content creators, and artists alike, promising a future where imagination is the only limit to visual storytelling.

Introducing Sora, our text-to-video model.

Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions. https://t.co/7j2JN27M3W

Prompt: “Beautiful, snowy… pic.twitter.com/ruTEWn87vf

— OpenAI (@OpenAI) February 15, 2024

Technical Achievements and Limitations

The technical prowess of Sora is a testament to the significant strides made in the field of artificial intelligence. Sora represents an evolutionary leap from static image generation to dynamic video creation, a complex process that involves not just visual rendering but also the understanding of motion and temporal progression. This advancement signals a monumental shift in AI's capability to interpret and visualize narratives over time, making it more than just a tool for creating visuals — it’s a storyteller.

However, as with any groundbreaking technology, Sora comes with its own set of limitations. Despite its advanced capabilities, the model sometimes struggles with accurately simulating the physics of more complex scenes. This can result in visuals that, while impressive, may occasionally defy the laws of physics or fail to accurately represent cause-and-effect scenarios. For example, a character in a video may interact with objects in ways that are not physically plausible or consistent over time.

Sora in the Competitive Landscape

In the rapidly evolving landscape of AI-driven video generation, Sora positions OpenAI at the forefront of innovation, alongside tech giants and emerging AI startups. Companies like Google, Meta, and numerous AI startups have also ventured into the realm of video generation, each contributing unique approaches and technologies.

Sora distinguishes itself with its emphasis on creating high-definition, photorealistic videos from text, a feature that sets a new bar in the field. While competitors like Google's Lumiere and Meta's Make-A-Video have demonstrated their capabilities in this space, Sora's advanced understanding of language, emotion, and physical properties offers a different level of sophistication and realism.

The competitive landscape of AI video generation is not just about technological prowess but also about the nuances of each tool's capabilities. Sora's entry into this space highlights the diverse approaches being taken to solve the complex puzzle of AI-generated content. Each player, including Sora, contributes to a broader understanding and development of this technology, pushing the boundaries of what is possible in digital content creation.

As the field continues to grow, Sora stands out for its ambitious goal of bridging the gap between text and video in a seamless and realistic manner, setting the stage for future advancements in the AI-generated video domain.

https://t.co/rmk9zI0oqO pic.twitter.com/WanFKOzdIw

— Sam Altman (@sama) February 15, 2024

Safety and Accessibility

In the realm of powerful AI tools like Sora, safety and accessibility are paramount. Recognizing this, OpenAI has taken a cautious approach to Sora's rollout. Currently, the model is accessible only to a select group of red teamers and visual artists. This strategy allows OpenAI to rigorously test Sora in controlled environments, ensuring that any potential harms or risks associated with its use are identified and mitigated.

The concerns surrounding AI-generated content, particularly in the realm of deepfakes and misinformation, are well-founded. The potential for misuse of such technology in spreading false information or creating deceptive media is a significant challenge. OpenAI's approach reflects a growing awareness within the AI industry of the need to balance innovation with responsibility. By limiting initial access to a carefully chosen group, OpenAI aims to understand and address these concerns before making Sora widely available.

Future Implications and Ethical Considerations

The introduction of Sora into the market is not just a technological milestone; it also brings with it a host of ethical considerations and potential impacts across various sectors. In the media and entertainment industry, for instance, Sora could revolutionize content creation, offering new avenues for storytelling and visual artistry. However, in the wrong hands, the same technology could be used to create misleading or harmful content, exacerbating the already prevalent issues of fake news and digital manipulation.

The ethical deployment of AI technologies like Sora involves navigating a complex landscape of societal, legal, and moral questions. Ensuring that these tools are used for beneficial purposes while safeguarding against abuse is a challenge that requires the collective effort of policymakers, technologists, and the community at large. Engaging in open dialogues and developing robust policies will be crucial in shaping the responsible use of generative AI technologies.

Navigating the AI-Generated Future

OpenAI's Sora model stands as a remarkable achievement in the evolution of AI video generation, showcasing impressive capabilities while also highlighting the ongoing challenges and limitations of such technology. Its introduction into the AI landscape underscores the extraordinary potential of generative AI, opening doors to new creative possibilities.

However, the development and deployment of Sora also reflect the critical need for caution and responsibility in the AI industry. As we move forward, the balance between innovation and ethical considerations will be crucial. The anticipation of future developments in AI-generated content, coupled with a commitment to responsible use, will shape the trajectory of this exciting and rapidly evolving field. In navigating this AI-generated future, the collective efforts of technologists, policymakers, and the community will be instrumental in ensuring that these advancements serve to enrich and not diminish the fabric of our digital world.

upliance.ai, India’s AI-powered home appliance company that featured on Shark Tank India, recently secured a ₹34 crore seed round at a valuation of ₹143 crore. The funding was led by Khosla Ventures, renowned for investing in AI startups like OpenAI, Rabbit, and Sarvam.

upliance.ai’s AI Cooking Assistant comes in the form of a jar with a smart 8-inch screen attached. It can prepare over 500+ dishes, including paneer stir fry, chole masala, steamed rice, and chai, for instance. “We are actually working closely with OpenAI, and it’s not just about the APIs. We have also started using our own trained models, so it’s not as simple as ChatGPT,” said Mahek Mody, upliance.ai chief, in an exclusive interview with AIM.

“We want to build Netflix, but for food,” he added, saying that upliance offers you an infinite variety of food options. “You don’t need to worry about whether you know how to cook something or not, you could just follow instructions,” he said.

Tech Behind upliance.ai

The company uses a proprietary technology called UpAI, built to understand the details of cooking – right from getting ingredients ready to making complete meals.

The smart jar is equipped with a blade, a thermal sensor, and a heating element, enabling it to execute more than 16 cooking functions such as chopping, sautéing, blending, heating, and steaming. “The jar looks at vibrations and the viscosity of the gravy. All these things help it determine how far ahead we are in the cooking process and if it is consistently cooked,” said Mody.

The screen comes with the in-built ‘ChefGPT’, which answers all your queries about the recipes and suggests ingredients for the dish. Moreover, it can suggest new recipes based on the ingredients you have at your disposal. Not only does it provide step-by-step instructions while you are cooking, but also enhances your overall cooking experience.

“We have now 25,000 meals cooked for customers on both the app and the device. For instance, we have approximately 420 unique recipes that people have cooked and we have cross-promoted food as well,” said Mody, adding that they started with 50 customers who pre-ordered the product.

Why LLMs in the Kitchen?

“The idea is that if one person cooks a dish in one corner of the world, another person should also be able to cook the same thing and all its variations,” said Mody.

He further added that all the cooking data, people’s preferences, and feedback are streamed to the internet to maintain collective intelligence and improve their cooking algorithms.

“Typically, you learn cooking from your parents. It’s mostly roti-sabzi, and dal–chawal. But suppose you want to make risotto, it would take awfully long to learn that from YouTube,” he said. Now, you can input the name of the dish into the generative AI screen that upliance has and it will generate a recipe for you.

“For example, let’s say lasooni chicken is not on the appliance and is not pre-set, you can still go to the AI generator and type the name of the dish. It will generate the recipe for you,” he added.

Furthermore, he said that the AI Cooking Assistant will consider the type of recipe and the temperature of the item before providing answers. “You could be in the middle of cooking and say, ‘Hey, I want to make this recipe vegan’. It will go ahead and make the changes in the recipe accordingly,” he explained.

upliance.ai may introduce new products in the future but for now, one can buy the AI Cooking Assistant from the company website for ₹23,999. In our kitchens, we have several appliances like the microwave, induction cooker, mixer, coffee maker, etc., LLMs can help us use these devices more effectively, eliminating the need to manually press buttons to adjust settings as they would be smart enough to do so.

upliance.ai’s Journey

upliance.ai was launched in early 2023 by Mahek Mody and Mohit Sharma. Mody has previously worked with Ather, learning how to set up the supply chain and design production, while Sharma was at Chaayos, lapping up the science behind cooking.

Last year, the startup successfully completed a pre-seed funding round, raising Rs 11 crore from investors such as Draper Associates, Rukam Capital, Rainmatter (a Zerodha Fund), and angel investors, including the co-founders of Ather Energy and Unacademy.

With funding from Khosla Ventures now, the company aims to grow its revenue to ₹150 crore in 2024 and scale the production to 20,000 units per annum in the next 6 months. Besides Khosla Ventures, upliance is closely working with Qualcomm. “Qualcomm has given us multiple grants to invest in this technology. Qualcomm India has been a great partner to us,” concluded Mody.

The post This Indian AI Startup is Bringing LLMs to Your Kitchen appeared first on Analytics India Magazine.

Image by Author

Profiling Python code is helpful to understand how the code works and identify opportunities for optimization. You’ve probably profiled your Python scripts for time-related metrics—measuring execution times of specific sections of code.

But profiling for memory—to understand memory allocation and deallocation during execution—is just as important. Because memory profiling can help identify memory leaks, resource utilization, and potential issues with scaling.

In this tutorial, we’ll explore profiling Python code for memory usage using the Python package memory-profiler.

Installing the memory-profiler Python Package

Let’s start by installing the memory-profiler Python package using pip:

pip3 install memory-profiler

Note: Install memory-profiler in a dedicated virtual environment for the project instead of in your global environment. We'll also be using the plotting capabilities available in memory-profiler to plot the memory usage, which requires matplotlib. So make sure you also have matplotlib installed in the project’s virtual environment.

Profiling Memory Usage with the @profile Decorator

Let's create a Python script (say main.py) with a function process_strs:

• The function creates two super long Python strings str1 and str2 and concatenates them.
• The keyword argument reps controls the number of times the hardcoded strings are to be repeated to create str1 and str2. And we give it a default value of 10**6 which will be used if the function called does not specify the value of reps.
• We then explicitly delete str2.
• The function returns the concatenated string str3.

# main.py    from memory_profiler import profile    @profile  def process_strs(reps=10**6):  	str1 = 'python'*reps  	str2 = 'programmer'*reps  	str3 = str1 + str2  	del str2  	return str3    process_strs(reps=10**7)

Running the script should give you a similar output:

As seen in the output, we’re able to see the memory used, the increment with each subsequent string creation and the string deletion step freeing up some of the used memory.

Running the mprof command

Instead of running the Python script as shown above, you can also run the mprof command like so:

mprof run --python main.py

When you run this command, you should also be able to see a .dat file with the memory usage data. You’ll have one .dat file every time you run the mprof command—identified by the timestamp.

Plotting Memory Usage

Sometimes it's easier to analyze memory usage from a plot instead of looking at numbers. Remember we discussed matplotlib being a required dependency to use the plotting capabilities.

You can use the mprof plot command to plot the data in the .dat file and save it to an image file (here output.png):

mprof plot -o output.png

By default, mprof plot used the data from the most recent run of the mprof command.

You can see the timestamps mentioned in the plot as well.

Logging Memory Usage Profile to a Log File

Alternatively, you can log the memory usage statistics to a preferred log file in the working directory. Here, we create a file handler mem_logs to the log file, and set the stream argument in the @profile decorator to the file handler:

# main.py    from memory_profiler import profile    mem_logs = open('mem_profile.log','a')    @profile(stream=mem_logs)  def process_strs(reps=10**6):  	str1 = 'python'*reps  	str2 = 'programmer'*reps  	str3 = str1 + str2  	del str2  	return str3    process_strs(reps=10**7)

When you now run the script, you should be able to see the mem_profile.log file in your working directory with the following contents:

Profiling Using the memory_usage Function

You can also use the memory_usage() function to understand the resources required for a specific function to execute—sampled at regular time intervals.

The memory_usage function takes in the function to profile, positional and keyword arguments as a tuple.

Here, we’d like to find the memory usage of the process_strs function with the keyword argument reps set to 10**7. We also set the sampling interval to 0.1 s:

# main.py    from memory_profiler import memory_usage    def process_strs(reps=10**6):  	str1 = 'python'*reps  	str2 = 'programmer'*reps  	str3 = str1 + str2  	del str2  	return str3    process_strs(reps=10**7)    mem_used = memory_usage((process_strs,(),{'reps':10**7}),interval=0.1)  print(mem_used)

Here’s the corresponding output:

Output >>>  [21.21875, 21.71875, 147.34375, 277.84375, 173.93359375]

You can also adjust the sampling interval based on how often you want the memory usage to be captured. As an example, we set the interval to 0.01 s; meaning we’ll now get a more granular view of the memory utilized.

# main.py    from memory_profiler import memory_usage    def process_strs(reps=10**6):  	str1 = 'python'*reps  	str2 = 'programmer'*reps  	str3 = str1 + str2  	del str2  	return str3    process_strs(reps=10**7)    mem_used = memory_usage((process_strs,(),{'reps':10**7}),interval=0.01)  print(mem_used)

You should be able to see a similar output:

Output >>>  [21.40234375, 21.90234375, 33.90234375, 46.40234375, 59.77734375, 72.90234375, 85.65234375, 98.40234375, 112.65234375, 127.02734375, 141.27734375, 155.65234375, 169.77734375, 184.02734375, 198.27734375, 212.52734375, 226.65234375, 240.40234375, 253.77734375, 266.52734375, 279.90234375, 293.65234375, 307.40234375, 321.27734375, 227.71875, 174.1171875]

Conclusion

In this tutorial, we learned how to get started with profiling Python scripts for memory usage.

Specifically, we learned how to do this using the memory-profiler package. We used the @profile decorator and the memory_usage() function to get the memory usage of a sample Python script. We also learned how to use the capabilities such as plotting the memory usage and capturing the stats in a log file.

If you’re interested in profiling your Python script for execution times, consider reading Profiling Python Code Using timeit and cProfile.

Bala Priya C is a developer and technical writer from India. She likes working at the intersection of math, programming, data science, and content creation. Her areas of interest and expertise include DevOps, data science, and natural language processing. She enjoys reading, writing, coding, and coffee! Currently, she's working on learning and sharing her knowledge with the developer community by authoring tutorials, how-to guides, opinion pieces, and more.

More On This Topic

• How to troubleshoot memory problems in Python
• Profiling Python Code Using timeit and cProfile
• How Much Memory is your Machine Learning Code Consuming?
• Memory Complexity with Transformers
• New Computing Paradigm for AI: Processing-in-Memory (PIM) Architecture
• Introduction to Clustering in Python with PyCaret