Рубрика: AI

It is difficult to follow all the new developments in AI. How can you discriminate between fundamental technology here to stay, and the hype? How to make sure that you are not missing important developments? The goal of this article is to provide a short summary, presented as a glossary. I focus on recent, well-established methods and architecture.

I do not cover the different types of deep neural networks, loss functions, or gradient descent methods: in the end, these are the core components of many modern techniques, but they have a long history and are well documented. Instead, I focus on new trends and emerging concepts such as RAG, LangChain, embeddings, diffusion, and so on. Some may be quite old (embeddings), but have gained considerable popularity in recent times, due to widespread use in new ground-breaking applications such as GPT.

New Trends

The landscape evolves in two opposite directions. On one side, well established GenAI companies implement neural networks with trillions of parameters, growing more and more in size, using considerable amounts of GPU, and very expensive. People working on these products believe that the easiest fix to current problems is to use the same tools, but with bigger training sets. Afterall, it also generates more revenue. And indeed, it can solve some sampling issues and deliver better results. There is some emphasis on faster implementations, but speed and especially size, are not top priorities. In short, more brute force is key to optimization.

On the other side, new startups including myself focus on specialization. The goal is to extract as much useful data as you can from much smaller, carefully selected training sets, to deliver highly relevant results to specific audiences. Afterall, there is no best evaluation metric: depending on whether you are a layman or an expert, your criteria to assess quality are very different, even opposite. In many cases, the end users are looking for solutions to deal with their small internal repositories and relatively small number of users. More and more companies are concerned with costs and ROI on GenAI initiatives. Thus, in my opinion, this approach has more long-term potential.

Still, even with specialization, you can process the entire human knowledge — the whole Internet — with a fraction of what OpenAI needs (much less than one terabyte), much faster, with better results, even without neural networks: in many instances, much faster algorithms can do the job, and it can do it better, for instance by reconstructing and leveraging taxonomies. One potential architecture consists of multiple specialized LLMs or sub-LLMs, one per top category. Each one has its own set of tables and embeddings. The cost is dramatically lower, and the results more relevant to the user who can specify categories along with his prompt. If in addition you allow the user to choose the parameters of his liking, you end up with self-tuned LLMs and/or customized output. I discuss some of these new trends in more details, in the next section. It is not limited to LLMs only.

Key Concepts

The list below is in alphabetical order. In many cases, the description highlights how I use the concepts in question in my own open-source technology.

• ANN (Approximate nearest neighbor). Similar to the K-NN algorithm used in supervised classification, but faster and applied to retrieving information in vector databases, such as LLM embeddings stored as vectors. I designed a probabilistic version called pANN, especially useful for model evaluation and improvement, with applications to GenAI, synthetic data, and LLMs. See here.

• Diffusion. Diffusion models use a Markov chain with diffusion steps to slowly add random noise to data and then learn to reverse the diffusion process to construct desired data samples from the noise. The output is usually a dataset or image similar but different from the original ones. Unlike variational autoencoders, diffusion models have high dimensionality in the latent space (latent variables): the same dimension as the original data. Very popular in computer vision and image generation.

• Embedding. In LLMs, embeddings are typically attached to a keyword, paragraph, or element of text; they consist of tokens. The concept has been extended to computer vision, where images are summarized in small dimensions by a number of numerical features (far smaller than the number of pixels). Likewise, in LLMs, tokens are treated as the features in your dataset, especially when embeddings are represented by fixed-size vectors. The dimension is the number of tokens per embedding. See tokens.

• Encoder. An autoencoder is (typically) a neural network to compress and reconstruct unlabeled data. It has two parts: an encoder that compacts the input, and a decoder that reverses the transformation. The original transformer model was an autoencoder with both encoder and decoder. However, OpenAI (GPT) uses only a decoder. Variational autoencoders (VAE) are very popular.

• GAN (generative adversarial network). One of the many types of DNN (deep neural network) architecture. It consists of two DNNs: the generator and the discriminator, competing against each other until reaching an equilibrium. Good at generating synthetic images similar to those in your training set (computer vision). Key components include a loss function, a stochastic gradient descent algorithm such as ADAM to find a local minimum to the loss function, and hyperparameters to fine-tune the results. Not good at synthesizing tabular data, thus the reason I created NoGAN: see here.

• GPT. In case you did not know, GPT stands for Generative Pre-trained Transformer. The main application is LLMs. See Transformer.

• Graph database. My LLMs rely on taxonomies attached to the crawled content. Taxonomies consist of categories, subcategories and so on. When all subcategories have only one parent category, you use a tree to represent the structure. Otherwise, you use a graph structure.

• Key-value database. Also known as hash table or dictionary in Python. In my LLMs, embeddings have variable size. I store them as short key-value tables rather than long vectors. Keys are tokens, and a value is the association between a token, and the word attached to the parent embedding.

• LangChain. Available as a Python library or API, it helps you build applications that read data from internal documents and summarize them. It allows you to build customized GPTs, and blend results to user queries or prompts with local information retrieved from your environment, such as internal documentation or PDFs.

• LLaMA. An LLM model that predicts the next word in a word sequence, given previous words. See here how I use them to predict the next DNA subsequence in DNA sequencing. Typically associated to auto-regressive models or Markov chains.

• LLM (large language model). Modern version of NLP (natural language processing) and NLG (natural language generation). Applications include chatbots, sentiment analysis, text summarization, search, and translation.

• Multi-agent system. LLM architecture with multiple specialized LLMs. The input data (a vast repository) is broken down into top categories. Each one has its own LLM, that is, its own embeddings, dictionary, and related tables. Each specialized LLM is sometimes called a simple LLM. See my own version named xLLM, here.

• Multimodal. Any architecture that blends multiple data types: text, videos, sound files, and images. The emphasis is on processing user queries in real-time, to return blended text, images, and so on. For instance, turning text into streaming videos.

• Normalization. Many evaluation metrics take values between 0 and 1 after proper scaling. Likewise, weights attached to tokens in LLM embeddings have a value between -1 and +1. In many algorithms and feature engineering, the input data is usually transformed first (so that each feature has same variance and zero mean), then processed, and finally you apply the inverse transform to the output. These transforms or scaling operations are known as normalization.

• Parameter. This word is mostly used to represent the weights attached to neuron connections in DNNs. Different from hyperparameters. The latter are knobs to fine-tuning models. Also different from the concept of parameter in statistical models despite the same spelling.

• RAG (retrieval-augmentation-generation). In LLMs, retrieving data from summary tables (embeddings) to answer a prompt, using additional sources to augment your training set and thus the summary tables, and generating output. Generation focuses on answering a user query (prompt), on summarizing a document, or producing some content such as synthesized videos.

• Regularization. Turning a standard optimization problem or DNN into constrained optimization, by adding constraints and corresponding Lagrange multipliers in the loss function. Potential goals: to obtain more robust results, or to deal with overparametrized statistical models and ill-conditioned problems. Example: Lasso regression. Different from normalization.

• Reinforcement learning. A semi-supervised machine learning technique to refine predictive or classification algorithms by rewarding good decisions and penalizing bad ones. Good decisions improve future predictions; you achieve this goal by adding new data to your training set, with labels that work best. In my LLMs, I let the user choose the parameters that best suit his needs. This technique leads to self-tuning and/or customized models: the default parameters come from usage.

• Synthetic data. Artificial tabular data with statistical properties (correlations, joint empirical distribution) that mimic those of a real dataset. You use it to augment, balance or anonymize data. Few methods can synthesize outside the range observed in the real data (your training set). I describe how to do it, here. A good metric to assess the quality of synthetic data is the full, multivariate Kolmogorov-Smirnov distance, based on the joint empirical distribution computed both on the real and generated observations. It works both with categorical and numerical features. The word “synthetic data” is also used for generated (artificial) time series, graphs, images, videos and soundtracks in multimodal applications.

• Token. In LLMs or NLP, a token is a single word; embeddings are vectors, with each component being a token. A word such as “San Francisco” is a single token, not two. In my LLMs, I use double tokens, such as “Gaussian distribution” for terms that are frequently found together. I treat them as ordinary (single) tokens. Also, the value attached to a token is its “correlation” (pointwise mutual information) to the word representing its parent embedding, see picture. But in traditional LLMs, the value is simply the normalized token frequency computed in some text repository.

• Transformer. A transformer model is an algorithm that looks for relationships in sequential data, for instance, words in LLM applications. Sometimes the words are not close to each other, allowing you to detect long-range correlations. It transforms original text into a more compact form and relationships, to facilitate further processing. Embeddings and transformers go together.

• Vector search. A technique combined with feature encoding to quickly retrieve embeddings in LLM summary tables, most similar to prompt-derived embeddings attached to a user query in GPT-like applications. Similar to multivariate “vlookup” in Excel. A popular metric to measure the proximity between two embeddings is the cosine similarity. To accelerate vector search, especially in real-time, you can cache popular embeddings and/or use approximate search such as ANN.

Author

Vincent Granville is a pioneering GenAI scientist and machine learning expert, co-founder of Data Science Central (acquired by a publicly traded company in 2020), Chief AI Scientist at MLTechniques.com and GenAItechLab.com, former VC-funded executive, author and patent owner — one related to LLM. Vincent’s past corporate experience includes Visa, Wells Fargo, eBay, NBC, Microsoft, and CNET. Follow Vincent on LinkedIn.

In today's rapidly evolving cloud landscape, reducing cloud costs while enhancing application performance has become a critical priority for both established enterprises and fast-growing digital native businesses.

The “State of Cloud Optimization 2024” report from Granulate, an Intel Company, is a crucial resource in today's cloud-centric IT environment. This comprehensive document provides an in-depth analysis of current practices, challenges, and trends in cloud optimization, offering valuable insights for businesses navigating the complex landscape of cloud computing.

Key Findings

• Prioritization of Cost Cutting: For 2024, the top objective for enterprises has been identified as “Cutting cloud costs.” This highlights the ongoing emphasis on cost-efficiency in cloud operations.
• Demand for Autonomous Tools: The report shows a high average importance rating of 7.29/10 for autonomous optimization tools, revealing a strong market demand for automation in cloud optimization efforts.
• Lack of Optimization Efforts: Alarmingly, 1 in 10 respondents claim their cloud optimization efforts are nearly non-existent, and 40% believe there is significant room for improvement.
• SMBs' Optimization Review Frequency: Over 50% of Small and Medium-sized Businesses (SMBs) review and adjust their cloud expenditure quarterly or less often, suggesting a potential lag in responding to the dynamic cloud environment.
• Correlation Between Costliness and Optimization Difficulty: A notable finding is the strong correlation between the perceived costliness of workloads and the difficulty of optimization. Kubernetes environments, in particular, rank highest in both costliness and optimization difficulty.
• Lack of Dedicated Teams: More than half of the respondents reported the absence of a team dedicated to optimization, indicating a potential gap in focused efforts towards cloud efficiency.
• Rise of AI-Driven Optimization: AI-driven cloud optimization emerged as the most exciting trend for 2024, with 33.7% of respondents affirming this. It reflects the growing interest in leveraging AI for more efficient and effective cloud optimization strategies.

Tool Adoption in the Optimization Tech Stack

The report delves into how organizations are addressing the challenge of cloud optimization through their choice of tools and methodologies. The survey asked participants about the types of optimization tools currently in their tech stacks, offering a range of tools, solutions, and practices as selectable options. The listed options were as following:

1. APM/Monitoring/Observability: These tools, aimed at providing real-time insights into application performance, system health, and user experiences through metrics, logs, and traces, were chosen more frequently than other types. This indicates a strong focus on visibility and organization of cloud resources among organizations.
2. CSP Optimization Tools: Custom solutions designed to enhance efficiency, cost-effectiveness, and performance, provided by Cloud Service Providers and tailored to their services, are critical for specific cloud environments optimization.
3. FinOps Methodologies: These practices, combining financial, operational, and business metrics, are crucial in driving cost-effective cloud usage and maximizing value. This approach reflects a strategic alignment of cloud operations with financial goals.
4. Code Profilers: Tools that analyze program code to identify performance bottlenecks, resource usage, and efficiency opportunities. These are essential for fine-tuning the performance and resource allocation of cloud applications.
5. Log Management and Analysis: Systems focusing on collecting, storing, and analyzing log data play a pivotal role in uncovering trends, diagnosing issues, and enhancing system performance.
6. 3rd Party Troubleshooting Tools: External solutions specializing in areas like network diagnostics or application debugging assist significantly in resolving cloud environment issues.
7. Storage Optimization: Techniques and tools aimed at improving data storage efficiency, accessibility, and cost-effectiveness are increasingly vital in cloud optimization strategies.
8. Rightsizing: This practice of precisely matching cloud resources to workload demands to ensure optimal performance without overprovisioning is becoming more prevalent, though only 23% of businesses reported using it.
9. Runtime Optimization: Focused on enhancing the execution environment of applications, this area is crucial for optimizing performance and resource utilization during operation, but only one in four organizations are using runtime optimization.

The survey results reveal that log management, analysis, and APM/monitoring/observability tools were chosen more often, yet no single type of optimization tool was selected by the majority of respondents​​​​. This suggests a diverse approach to cloud optimization across organizations, with many prioritizing visibility and organization of cloud resources. However, the relatively low usage of rightsizing and runtime optimization indicates that there is likely a significant amount of cloud resources going unutilized.

The findings from the report emphasize the importance of a comprehensive and tailored approach to cloud optimization. It highlights the necessity for businesses to carefully select the right mix of tools and practices to effectively manage and optimize their cloud environments.

The Role of Automation in Cloud Optimization Planning

The report provides critical insights into the role of automation in cloud optimization, particularly in the context of the challenges faced by businesses in managing cloud costs and performance.

Limited Dedicated Teams for Optimization Contrary to expectations, less than half of the surveyed organizations reported having a full-time team responsible for code optimization. Specifically, only 46.2% of respondents affirmed having a dedicated team for this purpose. This trend was consistent across company sizes, with Small and Medium-sized Businesses (SMBs) even less likely to have such teams (affirmative responses under 40%) compared to enterprises (46%).

Turning to Automation In situations where manual efforts in optimization are limited or non-existent, automation becomes a crucial factor. This is underscored by the survey findings, where 60% of respondents considered the autonomous nature of optimization tools to be very to extremely important. The emphasis on automation is further highlighted by the average importance rating of 7.29 out of 10 for autonomous tools, indicating a general high regard for these tools among the survey participants.

Implications for Cloud Optimization The findings from the report suggest a significant reliance on automation to compensate for the lack of dedicated optimization teams. This reliance is driven by the need to manage cloud costs effectively and improve performance, which are key objectives tied closely to cloud optimization. The emphasis on automation reflects a broader industry trend towards more efficient, self-managing cloud environments.

The report's insights into the role of automation in cloud optimization planning offer a clear directive for businesses. To effectively manage cloud resources and achieve optimization goals, organizations must increasingly look towards integrating autonomous tools into their tech stacks. This shift towards automation is critical in addressing the complex challenges of cloud cost management and performance optimization in the absence of dedicated optimization teams​​.

Summary

The “State of Cloud Optimization 2024” report serves as a vital guide for businesses seeking to enhance their cloud infrastructure's efficiency and cost-effectiveness. It provides a clear view of the challenges and opportunities in the realm of cloud optimization, backed by data and trends that will shape strategies in 2024 and beyond.

This Week in AI: Addressing racism in AI image generators Kyle Wiggers Devin Coldewey 8 hours

Keeping up with an industry as fast-moving as AI is a tall order. So until an AI can do it for you, here’s a handy roundup of recent stories in the world of machine learning, along with notable research and experiments we didn’t cover on their own.

This week in AI, Google paused its AI chatbot Gemini’s ability to generate images of people after a segment of users complained about historical inaccuracies. Told to depict “a Roman legion,” for instance, Gemini would show an anachronistic, cartoonish group of racially diverse foot soldiers while rendering “Zulu warriors” as Black.

It appears that Google — like some other AI vendors, including OpenAI — had implemented clumsy hardcoding under the hood to attempt to “correct” for biases in its model. In response to prompts like “show me images of only women” or “show me images of only men,” Gemini would refuse, asserting such images could “contribute to the exclusion and marginalization of other genders.” Gemini was also loath to generate images of people identified solely by their race — e.g. “white people” or “black people” — out of ostensible concern for “reducing individuals to their physical characteristics.”

Right wingers have latched on to the bugs as evidence of a “woke” agenda being perpetuated by the tech elite. But it doesn’t take Occam’s razor to see the less nefarious truth: Google, burned by its tools’ biases before (see: classifying Black men as gorillas, mistaking thermal guns in Black people’s hands as weapons, etc.), is so desperate to avoid history repeating itself that it’s manifesting a less biased world in its image-generating models — however erroneous.

In her best-selling book “White Fragility,” anti-racist educator Robin DiAngelo writes about how the erasure of race — “color blindness,” by another phrase — contributes to systemic racial power imbalances rather than mitigating or alleviating them. By purporting to “not see color” or reinforcing the notion that simply acknowledging the struggle of people of other races is sufficient to label oneself “woke,” people perpetuate harm by avoiding any substantive conservation on the topic, DiAngelo says.

Google’s ginger treatment of race-based prompts in Gemini didn’t avoid the issue, per se — but disingenuously attempted to conceal the worst of the model’s biases. One could argue (and many have) that these biases shouldn’t be ignored or glossed over, but addressed in the broader context of the training data from which they arise — i.e. society on the world wide web.

Yes, the data sets used to train image generators generally contain more white people than Black people, and yes, the images of Black people in those data sets reinforce negative stereotypes. That’s why image generators sexualize certain women of color, depict white men in positions of authority and generally favor wealthy Western perspectives.

Some may argue that there’s no winning for AI vendors. Whether they tackle — or choose not to tackle — models’ biases, they’ll be criticized. And that’s true. But I posit that, either way, these models are lacking in explanation — packaged in a fashion that minimizes the ways in which their biases manifest.

Were AI vendors to address their models’ shortcomings head on, in humble and transparent language, it’d go a lot further than haphazard attempts at “fixing” what’s essentially unfixable bias. We all have bias, the truth is — and we don’t treat people the same as a result. Nor do the models we’re building. And we’d do well to acknowledge that.

‘Embarrassing and wrong’: Google admits it lost control of image-generating AI

Here are some other AI stories of note from the past few days:

• Women in AI: TechCrunch launched a series highlighting notable women in the field of AI. Read the list here.
• Stable Diffusion v3: Stability AI has announced Stable Diffusion 3, the latest and most powerful version of the company’s image-generating AI model, based on a new architecture.
• Chrome gets GenAI: Google’s new Gemini-powered tool in Chrome allows users to rewrite existing text on the web — or generate something completely new.
• Blacker than ChatGPT: Creative ad agency McKinney developed a quiz game, Are You Blacker than ChatGPT?, to shine a light on AI bias.
• Calls for laws: Hundreds of AI luminaries signed a public letter earlier this week calling for anti-deepfake legislation in the U.S.
• Match made in AI: OpenAI has a new customer in Match Group, the owner of apps including Hinge, Tinder and Match, whose employees will use OpenAI’s AI tech to accomplish work-related tasks.
• DeepMind safety: DeepMind, Google’s AI research division, has formed a new org, AI Safety and Alignment, made up of existing teams working on AI safety but also broadened to encompass new, specialized cohorts of GenAI researchers and engineers.
• Open models: Barely a week after launching the latest iteration of its Gemini models, Google released Gemma, a new family of lightweight open-weight models.
• House task force: The U.S. House of Representatives has founded a task force on AI that — as Devin writes — feels like a punt after years of indecision that show no sign of ending.

More machine learnings

AI models seem to know a lot, but what do they actually know? Well, the answer is nothing. But if you phrase the question slightly differently… they do seem to have internalized some “meanings” that are similar to what humans know. Although no AI truly understands what a cat or a dog is, could it have some sense of similarity encoded in its embeddings of those two words that is different from, say, cat and bottle? Amazon researchers believe so.

Their research compared the “trajectories” of similar but distinct sentences, like “the dog barked at the burglar” and “the burglar caused the dog to bark,” with those of grammatically similar but different sentences, like “a cat sleeps all day” and “a girl jogs all afternoon.” They found that the ones humans would find similar were indeed internally treated as more similar despite being grammatically different, and vice versa for the grammatically similar ones. OK, I feel like this paragraph was a little confusing, but suffice it to say that the meanings encoded in LLMs appear to be more robust and sophisticated than expected, not totally naive.

Neural encoding is proving useful in prosthetic vision, Swiss researchers at EPFL have found. Artificial retinas and other ways of replacing parts of the human visual system generally have very limited resolution due to the limitations of microelectrode arrays. So no matter how detailed the image is coming in, it has to be transmitted at a very low fidelity. But there are different ways of downsampling, and this team found that machine learning does a great job at it.

Image Credits: EPFL

“We found that if we applied a learning-based approach, we got improved results in terms of optimized sensory encoding. But more surprising was that when we used an unconstrained neural network, it learned to mimic aspects of retinal processing on its own,” said Diego Ghezzi in a news release. It does perceptual compression, basically. They tested it on mouse retinas, so it isn’t just theoretical.

An interesting application of computer vision by Stanford researchers hints at a mystery in how children develop their drawing skills. The team solicited and analyzed 37,000 drawings by kids of various objects and animals, and also (based on kids’ responses) how recognizable each drawing was. Interestingly, it wasn’t just the inclusion of signature features like a rabbit’s ears that made drawings more recognizable by other kids.

“The kinds of features that lead drawings from older children to be recognizable don’t seem to be driven by just a single feature that all the older kids learn to include in their drawings. It’s something much more complex that these machine learning systems are picking up on,” said lead researcher Judith Fan.

Chemists (also at EPFL) found that LLMs are also surprisingly adept at helping out with their work after minimal training. It’s not just doing chemistry directly, but rather being fine-tuned on a body of work that chemists individually can’t possibly know all of. For instance, in thousands of papers there may be a few hundred statements about whether a high-entropy alloy is single or multiple phase (you don’t have to know what this means — they do). The system (based on GPT-3) can be trained on this type of yes/no question and answer, and soon is able to extrapolate from that.

It’s not some huge advance, just more evidence that LLMs are a useful tool in this sense. “The point is that this is as easy as doing a literature search, which works for many chemical problems,” said researcher Berend Smit. “Querying a foundational model might become a routine way to bootstrap a project.”

Last, a word of caution from Berkeley researchers, though now that I’m reading the post again I see EPFL was involved with this one too. Go Lausanne! The group found that imagery found via Google was much more likely to enforce gender stereotypes for certain jobs and words than text mentioning the same thing. And there were also just way more men present in both cases.

Not only that, but in an experiment, they found that people who viewed images rather than reading text when researching a role associated those roles with one gender more reliably, even days later. “This isn’t only about the frequency of gender bias online,” said researcher Douglas Guilbeault. “Part of the story here is that there’s something very sticky, very potent about images’ representation of people that text just doesn’t have.”

With stuff like the Google image generator diversity fracas going on, it’s easy to lose sight of the established and frequently verified fact that the source of data for many AI models shows serious bias, and this bias has a real effect on people.

These are curious times at the supermarket checkout.

For a while, it looked as if supermarkets were investing more and more in self-checkout technology.

Also: How renaissance technologists are connecting the dots between AI and business

This approach was completely understandable. It felt like a good investment, one that would obviate the need to employ so many people and to help cut costs, which is something a low-margin retail business craves.

That shrinking feeling

Yet the practical deployment of self-checkout technology has been less edifying. Though some customers adore the ability to sweep through the self-checkout lanes without having to talk to anyone, others have been a touch outraged.

One Rhode Island politician even tried to introduce a law preventing supermarkets from having too many self-checkout lanes open at any one time. The politician believes she shouldn't have to do the work of a human member of staff.

Also: How tech professionals can survive and thrive at work in the time of AI

Target, too, has decided to limit self-checkout hours at some stores, in an effort to reduce the sorts of losses other supermarket chains say they've experienced at the automated machines. The cause of these losses? Dishonest shoppers, apparently.

One study suggested that so-called shrink — losses caused when goods aren't paid for — might be as much as 16 times worse at self-checkout than at cashier-operated lanes.

Meanwhile, the BBC published a piece declaring: "'It hasn't delivered': The spectacular failure of self-checkout technology."

This checkout is policed by AI

I started to feel mournful that self-checkout technology had been introduced without the fullest aforethought. However, Germany's Retail Optimiser provided a more optimistic view.

Perhaps, unsurprisingly, this positivity came from Diebold Nixdorf's vice president of retail technology, Matt Redwood. After all, his company has something of a vested interest.

Also: Soon, every employee will be both AI builder and AI consumer

Redwood says that, in Germany, "retailers are recording an increasing number of inventory discrepancies." He concedes that "unintentional or deliberately missed scans, the manipulation of barcodes or when customers leave the checkout zone without making a payment" are all problems.

He believes, however, that there is a solution — and that solution is powered by AI.

Redwood explained in the article that Diebold Nixdorf's AI-powered software suite, launched just this year, will make a considerable difference to non-payment. Known as 'Vynamic Smart Vision | Shrink Reduction', this technology suite communicates with a staff member clutching a tablet or phone.

Also: AI will have a big impact on jobs this year. Here's why that could be good news

The technology brings together several pieces of software. There's one application that automatically identifies an alcohol-purchaser's age, one that claims to instantly recognize fruit and vegetables, which is one of the primary items of frustration for many customers, and one that does a little police work.

The company's YouTube video offers the hearty promise of a "Retail AI Revolution", one that ought — the company believes — to please supermarket owners and customers alike.

In essence, this software package means that you're going to be more closely monitored at the self-checkout. If you happen to — accidentally, or less so — slip an item into your pocket, a supermarket employee will immediately get an alert.

You see, with this system, you're being filmed from above, so smile and please don't be naughty.

Redwood is clear that there are several issues this software should address.

As an example: "If the scanned barcode does not match the item — such as when a label for bananas is attached to a whiskey bottle and items are deliberately or unintentionally not brought to the scanner."

And then there's the issue "when two items are held in front of each other, when customers pass an item past the scanner without actually scanning it or walk away from the self-service checkout without paying after a canceled transaction."

Also: Agile Intelligence: AI gives tech and business collaboration a much-needed boost

You might be wondering how the overhead camera detects such irregular actions. Well, the software quickly performs AI-powered video analysis, which could be seen as the equivalent of a passing police officer muttering: "Hmm, something's not right here."

The software does, though, first appeal to your honesty, notifying you of a perceived peculiarity. But the technology still doesn't rely on your integrity: "At the same time, the employees are notified of the incorrect operation with the help of the intelligent assistant."

And Diebold Nixdorf's software does offer the retailer one more option: it can shut down the self-checkout machine, just like that.

Will the AI get it right?

It seems that the whole approach relies on the speed and accuracy of AI-powered video analysis. In which case, the process better be right or there might be some huffing and scuffling.

Yet Redwood believes his company's software — specifically its claimed accuracy — will benefit store employees: "In cases where the customer has unknowingly made a mistake, they can provide support without directly assuming theft and thus scaring the customer off."

But what about if a customer is simply pulling a fast one?

Redwood says: "In the event of an obvious attempt at fraud, security staff can intervene immediately, and employees are protected from potentially critical situations."

Also: Want to work in AI? How to pivot your career in 5 steps

Personally, I'm fond of my local supermarket's checkout staff. And at Trader Joe's, which insists it'll never house self-checkout machines, there's an extra joy to be bathed in when you can chat to staff about the everyday issues of life, even if only for a moment.

Still, you can't say Diebold Nixdorf isn't trying — to make self-checkout more profitable.

And in an age where many people aren't all that interested in, well, other people, the number of customers I see paying with their phones and not even looking at a store cashier is a touch chilling.

In fact, I believe Diebold Nixdorf may have nixed many of the problems retailers are facing with self-checkout.

But will customers really love it? Oh, perhaps.

Artificial Intelligence

Reddit signed a data licensing agreement with Google for reportedly around $60 million per year to access Reddit’s real-time content through its Data API. This has come close to its initial public offering (IPO) in March this year. The company will be the first major social media company to go public since Pinterest in 2019.

The company will likely be aiming for a valuation of at least $5 billion in an IPO. Since the first announcement in 2021, the company has been making sweeping changes to allegedly increase its revenue and further its valuation. Along with this announcement,The biggest change was that Reddit eliminated free access to most of its API. Developers now need to pay for access, with pricing based on the number of requests made, which were exorbitant, shutting down alternative apps to the site.

The primary purpose of this with Google is to provide Google’s AI with a large amount of conversational data to train and improve their LLMs. The user on Reddit signs off on the rights to the platform to use the content however they see fit, however it is unclear how anonymisation of the data will be applied before being used for training. Reddit also benefits by leveraging Google’s Vertex AI, improving its not-so-good search feature.

Useful Data?

Most of reddit is known for conversation ranges from flippant to hateful. The polarised crowd in each subreddit form their own echo chambers. The recent example of conversations on Russia-Ukraine war showed a bias towards Ukraine often spewing vitriolic speech when countered this stance.

Although the social media platform is regulated by moderators trying to curb violent speech it is not always successful as the subreddit rules vary. Bindu Reddy, CEO and Co-founder of Abacus.AI said on X, “Once they (Google) pre-train their model on this largely uncensored corpus where humans routinely reveal their true opinions, they will spend > $60M suppressing the Reddit content, nerfing, and nudging their model to reflect their ideology!”

Popular for its diverse content, the top comments on most of the posts are funny and satirical. The irony, satire, and humour offers a unique data set for training Google’s AI, contributing to a deeper understanding of complex human communication.

The data on Reddit is also organised, sorted and clear, with information on the upvotes making it structured. This gives Google instant access to all types of data to use this information better. The company plans to use “enhanced signals” to improve how it shows information, including showing more content from Reddit as announced by Google and Reddit.

From training on such content, AI to grasp nuances, identify misinformation through satire, and improve generative models with informal language and creativity. However, challenges arise from the heavy reliance on context, potential bias amplification, and the limited generalizability of niche or slang-dominated data. The company blog further added that Reddit searches are popular on Google, “This partnership will facilitate more content-forward displays of Reddit information that will make our products more helpful for our users and make it easier to participate in Reddit communities and conversations.”

Experts in AI suggest that messy, diverse data can enhance model performance, emphasising the importance of sophisticated filtering and a balanced training approach. Privacy advocates, however, raise concerns about using even anonymized Reddit data for advancing profiling and targeted advertising techniques. The massive downside is the inaccurate information, during the peak of COVID-19 the company said it would leave up subreddits that spread misinformation related to Covid-19. Days later, after protest from many of its own users, Reddit banned the forum in question, saying it had violated other rules.

Google paused its Image Generator and apologised for ‘missing the mark’ for the inaccuracies. The irony here is that the model was inclusive but in showing African, Asian Nazi soldiers. Training on Reddit’s (although structured) data would be easier to train the model, the questionable quality of data to train the AI models could lead to similar problematic outputs, where harmful stereotypes or misinformation are reinforced rather than detected.

The post Why is Google Eyeing Reddit’s data? appeared first on Analytics India Magazine.

Sam Kriegman, a professor at Northwestern's McCormick School of Engineering, is something of a local celebrity.

He can often be found on television shows explaining the wonders of his new invention – an insta-robot.

In one TV appearance, Kriegman places his insta-robot on his palm to show it off — a purple-grey, gelatinous, squishy object with a few holes that looks a little like what you imagine a rhinoceros's ancestor to be.

Also: Robots plus generative AI: Everything you need to know when they work as one

He then attaches a flexible pipe to the robot and starts pumping. The jelly-robot kicks its legs out and — in perhaps the first robotic imitation of the moonwalk — begins to move backward. (You can watch the video here.)

It may all appear a bit underwhelming, a DIY toy from a science fair that you brought home to your five-year-old.

The truth is, however, that it may be a revolutionary moment in robot design with far-reaching implications for many aspects of human life – from exploration to rescue efforts to medicine.

"When people look at this robot, they might see a useless gadget," says Kriegman. "I see the birth of a brand-new organism."

A robot for every occasion

Imagine a future where you need to rescue people trapped under rubble, deliver a life-saving medicine in a hard-to-reach part of the body, or engineer a shutdown of a nuclear reactor in a heavily contaminated space.

Pretty much any of these scenarios would require a lightning-quick turnaround from conceptualizing a solution to fabricating and testing it before deployment.

None of this has been cheap or easy. Just the trial-and-error process of coming up with a solution takes months if not years.

There are evolutionary algorithms that look to nature for ideas when designing robots — but they require churning out vast datasets, which in turn requires a supercomputer and many days to arrive at a solution.

We're not even going to get into the astronomical sums and countless hours spent on a robot design project.

Also: 15 big ideas that will revolutionize industries and economies, led by AI

Yet, here is Kriegman wielding a puny laptop by comparison, typing in the words "Design a Robot that can walk" that results in a blueprint for a functioning prototype in a bewildering 26 seconds.

"Evolving robots previously required weeks of trial and error on a supercomputer, and of course, before any animals could run, swim or fly around our world, there were billions upon billions of years of trial and error,' says Kriegman.

"This is because evolution has no foresight. It cannot see into the future to know if a specific mutation will be beneficial or catastrophic. We found a way to remove this blindfold, thereby compressing billions of years of evolution into an instant."

Also: Generative AI filled us with wonder — but all magic comes with a price

As Kriegman and Matthews explain in their recent paper, in those 26 seconds, the computer methodically built upon each version until it was satisfied that it had generated a workable solution.

From a jello-like bar of soap, to something that bounces in one spot, to something with holes in it (to make it lighter), to an object with fins and three legs, the algorithm arrived at a model that can hop and then shuffle.

"It's interesting because we didn't tell the AI that a robot should have legs," Kriegman said. "It rediscovered that legs are a good way to move around on land. Legged locomotion is, in fact, the most efficient form of terrestrial movement." (Kriegman notes that one exercise which forced the AI to use pre-designed musculature didn't produce legs.)

Also: Generative AI can easily be made malicious despite guardrails

Each version of the robot — out of nine attempts — was an improvement over its predecessor, until the ninth version produced the desired outcome – an object that could walk half its body length per second when air was repeatedly pumped into it to simulate expanding and contracting muscles that produce locomotion.

Of course, to get to that result Kriegman's team had to make a 3D-printed mold of the object's design generated by the algorithm and fill it with liquid silicone rubber, after which it was cured like an ancient dinosaur jello treat.

There are a few intriguing takeaways from this experiment.

First, the feat of compressing millions of years of evolution into just 26 seconds is an indication of the speed at which solutions can be found for intractable problems — how to destroy an asteroid hurtling towards earth, or develop drugs for cancer, or link diseases to genetics.

Second, it takes a leap of imagination to shift your preconceived notions of functional design; for example, from thinking of chairs as four-legged to three-legged.

Also: How renaissance technologists are connecting the dots between AI and business

"When humans design robots, we tend to design them to look like familiar objects," Kriegman said. "But AI can create new possibilities and new paths forward that humans have never even considered. It could help us think and dream differently. And this might help us solve some of the most difficult problems we face."

Ultimately, it is about unlocking the vast store of resources lying in plain sight that we can't quite see as yet.

"The only thing standing in our way of these new tools and therapies is that we have no idea how to design them," Kriegman said. "Lucky for us, AI has ideas of its own."

A bumpy road for EV manufacturers Haje Jan Kamps 8 hours

Welcome to Startups Weekly — your weekly recap of everything you can’t miss from the world of startups. Sign up here to get it in your inbox every Friday.

Rivian launched with a promise to revolutionize the way we think about trucks and SUVs but has now hit a bit of a speed bump. In a move that screams, “Oops, we might’ve gotten a bit ahead of ourselves,” the company announced it’s laying off 10% of its workforce. Why, you ask? Well, it seems the EV market is a tad more cutthroat than anticipated, with pricing pressure mounting like the suspense in a bad thriller movie. Rivian is now faced with the harsh reality that making cars is hard and making them profitable is even harder. The current round of layoffs follows the 6% of staff it laid off a year ago, and another 6% that was shown the door around 18 months ago.

The manufacturer is reportedly launching its R2 series vehicles early next month, which supposedly will improve the company’s profit margins somewhat.

As Rivian navigates these turbulent waters, one can’t help but wonder if this is just a bump in the road or a sign of more ominous clouds on the horizon. Who needs soap operas when you have the EV market?

Most interesting startup stories this week

Image Credits: Hivemapper

Staying in the world of automotive, there’s been a lot of movement from the avant-garde of EV manufacturers.

Lucid Motors, in a valiant effort to make lemonade out of the lemons that were its 2023 sales figures, has decided to slash prices across its lineup of electric sedans. The move screams “Please notice us!” in a market where being the new kid on the block (Lucid was founded in 2007) is about as advantageous as a skateboard in a Formula 1 race. The base model Lucid Air Pure, previously flirting with the $80,000 mark, now bats its eyelashes at potential suitors with a more approachable $69,900 price tag. This comes on the heels of a rather humbling admission that the company only managed to deliver 6,001 cars in the entirety of 2023. The company at one point predicted it would be shipping 90,000 cars in 2024 — it looks like the real number will be a tenth of that.

It isn’t just the new kids on the block that are struggling. Founded a solid 104 years before Lucid, Ford, in a move that echoes the playground tactic of “if they can do it, so can we,” has decided to slash prices on its electric Mustang Mach-E in response to the softening demand for premium EVs. It seems the electric vehicle market is experiencing a bit of a reality check, with consumers suddenly remembering that money doesn’t grow on trees, even if those trees are saved by driving electric cars.

These price adjustments from Ford and Lucid come hot on the heels of EV industry poster child Tesla’s price reductions, suggesting that the EV market is maturing and that customers are becoming more price conscious.

A fire sale on electric motorcycles: The owner of a Florida retail shop scooped up the majority of Swedish electric motorbike brand Cake’s U.S. inventory, including all the Makka and Ösa motorbikes, accessories, and spare parts that had made their way stateside. Cake itself is cruising into bankruptcy.

Despite all their rage, they’re still just a rat in a Faraday cage: Struggling EV startup Faraday Future owes the landlord of its Los Angeles headquarters nearly $1 million after missing the last two months’ rent.

Keeping its eyes on the roads: Hivemapper, the company that’s been buzzing around the tech scene with its innovative mapping technology in an attempt to take on Google Maps, has just unveiled its latest creation: the Bee dashcam. This isn’t your ordinary dashcam: It’s designed to collect and share street-level imagery, contributing to Hivemapper’s global map.

Most interesting fundraising trends this week

Image Credits: Carol Yepes / Getty Images

Global investment firm Partech recently closed its second Africa fund at a whopping $300 million, doubling down on its commitment to the continent’s burgeoning tech ecosystem. The new fund aims to bridge the gap from seed to Series C funding rounds, providing a much-needed continuum of financial support for African startups. The fund’s strategy is to not only inject capital but also offer strategic guidance and access to a global network, empowering startups to scale both within Africa and internationally.

Om nom nom: Bluestein Ventures, a Chicago-based early-stage venture capital firm, has recently closed its third fund, with $45 million in capital commitments. Founded in 2014, the firm focuses on investing in consumer-facing technology across the food supply chain, including health and wellness, proprietary food tech, commerce, and digital technology.

Ready player 2: The global video game industry, despite its immense revenue surpassing that of movies and music combined, faced a challenging year in 2023 with significant layoffs and a five-year low in venture funding. However, optimism remains high among VCs for a turnaround in 2024.

That’s a big bag of moolah, you guys: Moonshot AI, a burgeoning artificial intelligence startup from China, has reportedly secured over $1 billion in a Series B funding round, setting a new record for the largest single funding round for Chinese large language model (LLM) developers. This financial boost propels Moonshot AI’s valuation to an impressive $2.5 billion.

This week’s big trend: The AI train keeps rumblin’ along

Image Credits: OpenAI

OpenAI has introduced a new generative AI model named Sora, capable of creating videos from text descriptions or still images. Sora stands out by generating high-res movie-like scenes that can include multiple characters, various motions, and detailed backgrounds. This model can also extend existing video clips by filling in missing details, demonstrating a deep understanding of language and the physical world.

Sora’s capabilities extend to generating videos in a range of styles, such as photorealistic, animated, and black and white, with durations up to a minute — significantly longer than most existing text-to-video models. Despite some limitations, like occasional inaccuracies in simulating complex physics or specific cause-and-effect scenarios, Sora’s output maintains a high level of coherence, avoiding common pitfalls of “AI weirdness.” The model will likely not be released to the public.

In other OpenAI news, the U.S. Patent and Trademark Office has denied the company’s attempt to trademark “GPT,” ruling that the term is “merely descriptive” and therefore unable to be registered.

Yeah, that checks out: Dili, a platform designed to automate key investment due diligence and portfolio management steps for private equity and VC firms using AI, has raised $3.6 million in venture funding. The company aims to alleviate the burden of due diligence tasks by leveraging generative AI, specifically large language models, to streamline investor workflows.

Is there anything else AI can help you with today?: Sierra’s approach to customer service AI is focused on augmenting human agents rather than replacing them. The company believes that AI can handle routine and repetitive inquiries, freeing up human agents to focus on more complex and nuanced customer interactions. The company has raised $110 million to date.

Focusing on the XX: In a New York Times piece late last year, the Gray Lady broke down how the current boom in AI came to be. The piece went viral — not for what was reported, but instead for what it failed to mention: women. We took a look at the women who are making waves in AI.

Other unmissable TechCrunch stories …

Every week, there are always a few stories I want to share with you, but they somehow don’t fit into the categories above. It’d be a shame if you missed ’em, so here’s a random grab bag of goodies for ya:

Onshoring: Don’t miss Aria’s awesome profile of Chris Power, the founder and CEO of Hadrian, an industrial automation startup. He is on a mission to defy historical patterns of empire decline through innovation in American manufacturing. Drawing from his observations of historical cycles where empires fall due to outsourcing core industries, Power embarked on a journey from Australia to the U.S. in 2019 with a thesis that the U.S. industrial base was in massive decline. In that decline, he saw an opportunity.

Gimme all your money: Ransomware has become a lucrative business model for cybercriminals, generating billions of dollars in revenue annually. This malicious software encrypts the victim’s data, making it inaccessible, and demands a ransom for the decryption key. Carly digs in to see how it became such a lucrative criminal enterprise.

Hello, this is Mr. I: Y Combinator’s newest request for startups (RFS) is well worth reading, and not just because it’s been a while since the incubator shared the ideas and categories its partners “would like to see more people working on.” Including a request for more startups working on cancer treatments.

Domo arigato, bricker roboto: Bricklaying robots aren’t exactly an untapped concept, but Amsterdam-based Monumental specializes in the more familiar red clay variety and caught our reporter’s eye.

Seizing the means of production: Amazon, SpaceX, and Trader Joe’s have recently taken legal actions that challenge the constitutionality of the National Labor Relations Board (NLRB), potentially threatening national worker protections that have been in place for nearly a century.

Most cybersecurity professionals (88%) believe AI will significantly impact their jobs, according to a new survey by the International Information System Security Certification Consortium; with only 35% of the respondents having already witnessed AI’s effects on their jobs (Figure A). The impact is not necessarily a positive or negative impact, but rather an indicator that cybersecurity pros expect their jobs to change. In addition, concerns have arisen about deepfakes, misinformation and social engineering attacks. The survey also covered policies, access and regulation.

How AI might affect cybersecurity pros’ tasks

Survey respondents generally believe that AI will make cybersecurity jobs more efficient (82%) and will free up time for higher-value tasks by taking care of other tasks (56%). In particular, AI and machine learning could take over these aspects of cybersecurity jobs (Figure B):

• Analyzing user behavior patterns (81%).
• Automating repetitive tasks (75%).
• Monitoring network traffic and detecting malware (71%).
• Predicting where breaches might occur (62%).
• Detecting and blocking threats (62%).

The survey doesn’t necessarily rank a response of “AI will make some parts of my job obsolete” as negative; instead, it’s framed as an improvement in efficiency.

Top AI cybersecurity concerns and possible effects

In terms of cybersecurity attacks, the professionals surveyed were most concerned about:

• Deepfakes (76%).
• Disinformation campaigns (70%).
• Social engineering (64%).
• The current lack of regulation (59%).
• Ethical concerns (57%).
• Privacy invasion (55%).
• The risk of data poisoning, intentional or accidental (52%).

The community surveyed was conflicted on whether AI would be better for cyber attackers or defenders. When asked about the statement “AI and ML benefit cybersecurity professionals more than they do criminals,” 28% agreed, 37% disagreed and 32% were unsure.

Of the surveyed professionals, 13% said they were confident they could definitively link a rise in cyber threats over the last six months to AI; 41% said they couldn’t make a definitive connection between AI and the rise in threats. (Both of these statistics are subsets of the group of 54% who said they’ve seen a substantial increase in cyber threats over the last six months.)

SEE: The UK’s National Cyber Security Centre warned generative AI could increase the volume and impact of cyberattacks over the next two years – although it’s a little more complicated than that. (TechRepublic)

Threat actors could take advantage of generative AI in order to launch attacks at speeds and volumes not possible with even a large team of humans. However, it’s still unclear how generative AI has affected the threat landscape.

In flux: Implementation of AI policies and access to AI tools in businesses

Only 27% of ISC2 survey respondents said their organizations have formal policies in place for safe and ethical use of AI; another 15% said their organizations have formal policies on how to secure and deploy AI technology (Figure C). Most organizations are still working on drafting an AI use policy of one kind or another:

• 39% of respondents’ companies are working on AI ethics policies.
• 38% of respondents’ companies are working on AI safe and secure deployment policies.

The survey found a very wide variety of approaches to allowing employees access to AI tools, including:

• My organization has blocked access to all generative AI tools (12%).
• My organization has blocked access to some generative AI tools (32%).
• My organization allows access to all generative AI tools (29%).
• My organization has not had internal discussions about allowing or disallowing generative AI tools (17%).
• I don’t know my organization’s approach to generative AI tools (10%).

The adoption of AI is still in flux and will surely change a lot more as the market grows, falls or stabilizes, and cybersecurity professionals may be at the forefront of awareness about generative AI issues in the workplace since it affects both the threats they respond to and the tools they use for work. A slim majority of cybersecurity professionals (60%) surveyed said they feel confident they could lead the rollout of AI in their organization.

“Cybersecurity professionals anticipate both the opportunities and challenges AI presents, and are concerned their organizations lack the expertise and awareness to introduce AI into their operations securely,” said ISC2 CEO Clar Rosso in a press release. “This creates a tremendous opportunity for cybersecurity professionals to lead, applying their expertise in secure technology and ensuring its safe and ethical use.”

How generative AI should be regulated

The ways in which generative AI is regulated will depend a lot on the interplay between government regulation and major tech organizations. Four out of five survey respondents said they “see a clear need for comprehensive and specific regulations” over generative AI. How that regulation may be done is a complicated matter: 72% of respondents agreed with the statement that different types of AI will need different regulations.

• 63% said regulation of AI should come from collaborative government efforts (ensuring standardization across borders).
• 54% said regulation of AI should come from national governments.
• 61% (polled in a separate question) would like to see AI experts come together to support the regulation effort.
• 28% favor private sector self-regulation.
• 3% want to retain the current unregulated environment.

ISC2’s methodology

The survey was distributed to an international group of 1,123 cybersecurity professionals who are ISC2 members between November and December 2023.

The definition of “AI” can sometimes be uncertain today. While the report uses the general terms “AI” and machine learning throughout, the subject matter is described as “public-facing large language models” like ChatGPT, Google Gemini or Meta’s Llama, usually known as generative AI.