What does GPT stand for? Understanding GPT 3.5, GPT 4, and more

GPT-4 on a laptop

OpenAI's claim to fame has been its AI chatbot, ChatGPT, which has become a household name. According to a recent Pew Research Center survey, about six in ten adults in the US are familiar with ChatGPT. However, how much do you know about the large language model underlying the chatbot?

Artificial Intelligence

Most organizations fear AI failure, but those that implement AI do report benefits

Tech stack creative photo

When generative artificial intelligence first burst upon the scene, the technology showed potential for making people's everyday lives easier. Now, AI solutions have been developed to help enterprises optimize their operations, and here's why you might want to consider using them in your business.

Also: How generative AI will deliver significant benefits to the service industry

Pluralsight's AI skills report surveyed 1,200 executives and IT professionals across the US and the UK to better understand how organizations deploy AI and its effects on businesses and their employees.

The report found that implementing AI in organizations had promising results, with 97% of organizations that have already deployed AI technologies benefiting. Moreover, 18% reported experiencing increased productivity and efficiency, 13% reported improved customer service and repetitive task reduction, and 11% said AI reduced business costs.

Despite the benefits, 25% of these organizations said they don't have plans to deploy AI, while 20% already have and 55% plan to. The hesitation stems from inadequate budget or talent required to use the new tools properly.

A majority of the professionals acknowledged that hesitation could be disastrous in the long run, with 94% of executives and 92% of IT professionals sharing that organizations investing in AI in the near future will be better able to compete, according to the report.

However, the lack of talent to properly use the new tools is an obstacle to the successful implementation of AI, and the report finds that the answer may lie in organizations helping upskill employees.

The report cites IDC research that found investments in skills and digital training of employees will be organizations' most enduring technology investments in 2023 and 2024, even over investments in generative AI solutions.

Out of the professionals surveyed, 95% of executives and 94% of IT professionals believe that AI initiatives will fail due to a lack of employees who do not know how to use the tools efficiently, again pointing to the need for investing in talent and training.

Also: Data is the missing piece of the AI puzzle. Here's how to fill the gap

Some ways to invest in talent and training include creating an AI skill development strategy before adopting AI technology, assessing employees' current AI knowledge and skills, and providing hands-on learning opportunities, according to the report.

Artificial Intelligence

Samsung Galaxy S24, Plus, Ultra compared: Which model should you buy?

Samsung Galaxy S24 models

New product launches are always exciting, especially when you're looking to upgrade your smartphone. But while these launches are fun, they can also be overwhelming, with companies delivering a lot of information in a short amount of time. January's Samsung Unpacked event was no different.

Review: Samsung Galaxy S24 and S24 Plus: A week later, I've forgotten about the Ultra model

Samsung's Unpacked event focused largely on the new features of the Galaxy S24 lineup of smartphones. From increased battery life to artificial intelligence (AI) capabilities, all three of the Samsung Galaxy S24 smartphones are some of the best options available on the market right now.

Now that ZDNET's team of reviewers has spent the past two weeks testing the devices, here are the key differences you should know as you decide on which model to buy.

Specifications

Galaxy S24 Galaxy S24 Plus Galaxy S24 Ultra
Starting price $799 $999 $1,299
Display 6.2" Dynamic AMOLED 2X, 2600 nits peak, Always-on 6.7" Dynamic AMOLED 2X, 2600 nits peak, Always-on 6.8" Dynamic AMOLED 2X, 2600 nits peak, Always-on
Resolution 1080 x 2340 pixels 1440 x 3088 pixels 1440 x 3088 pixels
Refresh rate 120Hz 120Hz 120Hz
Processor Snapdragon 8 Gen 3 Snapdragon 8 Gen 3 Snapdragon 8 Gen 3
RAM 8GB 12GB 12GB
Storage 128GB, 256GB, 512GB 256GB, 512GB 256GB, 512GB, 1TB
Main camera 50MP f/1.8 wide, 12MP ultrawide, 10MP telephoto — 8K at 24/30fps, 4K at 30/60fps 50MP f/1.8 wide, 12MP ultrawide, 10MP telephoto — 8K at 24/30fps, 4K at 30/60/120fps 200MP f/1.7 wide, 12MP ultrawide, 50MP telephoto — 8K at 24/30fps, 4K at 30/60/120fps
Front camera 12MP f2.2 — 4K at 60fps 12MP f2.2 — 4K at 60fps 12MP f2.2 — 4K at 60fps
OS Android 14 Android 14 Android 14
Battery 4,000mAh, 50% in 30 minutes 4,900mAh, 65% in 30 minutes 5,000mAh, 65% in 30 minutes
Durability IP68, Gorilla Glass Victus 2 with aluminum frame IP68, Gorilla Glass Victus 2 with aluminum frame IP68, Gorilla Glass Armor with titanium frame
Colors Onyx Black, Marble Grey, Cobalt Violet, Amber Yellow Onyx Black, Marble Grey, Cobalt Violet, Amber Yellow Titanium Black, Titanium Gray, Titanium Violet, Titanium Yellow

You should buy the Samsung Galaxy S24 Ultra if…

1. You want to make the most of Samsung's new AI features

The brand new Samsung Galaxy S24 Ultra may not be a huge hardware upgrade when compared to its previous version, the S23 Ultra, but its new AI features make daily use a far superior experience in comparison. And the included S Pen and large 6.8-inch display help you make the most of it.

Galaxy S24 users can use 'Circle to Search', a new feature that lets you circle an area in an image and run a Google search. The included S Pen that comes with the Ultra series is perfect to use for this purpose, giving users the precision necessary to encircle what they want to search for and leave out anything they don't.

Also: The best Android phones of 2024: Expert tested and reviewed

Other new AI features coming to the Samsung Galaxy S24 Ultra include: Generative Edit, which lets you move and edit subjects in photos, using AI to generate backgrounds or expand images; Live Translate, which features real-time translation during voice calls; Transcript Assist, which can transcribe voice memos and notes; and Chat Assist, which can help you generate text for emails, text messages, and more.

2. You want a top-of-the-line camera

The Samsung Galaxy S24 Ultra still has a 200MP main camera, but the telephoto module got an upgrade from a 10MP to 50MP with 5x optical zoom. New improvements and the Snapdragon 8 Gen 3 also result in clearer photos in low-light conditions. This capability helps reduce noise in photos taken at night or in dark areas while delivering more detail that typically wouldn't be captured with other cameras.

There's also a new AI Zoom feature that brings up the clarity and detail in long-distance shots by filling in areas that are blurry or not in focus.

You should buy the Samsung Galaxy S24 Plus if…

1. You like a large phone but not the large price tag

The great thing about the Goldilocks Samsung Galaxy S24 Plus is that it offers a lot of the great experiences from the larger S24 Ultra for a lower price. Don't get me wrong — you can't expect the S24 Plus to be an Ultra 'lite', but it is a great alternative if you'd rather have a larger screen and don't need the extra features that a higher model offers.

Also: Samsung Galaxy S24 vs Galaxy S23: Should you make the upgrade?

Starting at $999 for the 256GB model, the S24 Plus has a 6.7-inch display and is perfect for anyone looking for a large phone without all of the Ultra's bells and whistles. It also features a larger battery than the base model, and a 50MP main camera.

2. Battery life is your priority

The Samsung Galaxy S24 Plus has a considerably larger battery than the S24 and its predecessor, the S23 Plus. The Galaxy S24 has the smallest battery of Samsung's latest trio of smartphones, at 4000 mAh, and is capable of reaching 50% in 30 minutes.

At 4900 mAh, the Galaxy S24 Plus' battery can last up to 35 hours of talk time and, like the S24 Ultra's 5000 mAh battery, can charge to up to 65% in just 30 minutes. The fact that the S24 Plus has a battery almost as powerful as the S24 Ultra's, but without the latter's demand to maintain more high-end features, is worth considering when choosing an S24 model.

You should buy the Samsung Galaxy S24 if…

1. You want all the new AI features for less

The Samsung Galaxy S24 is an obvious choice for anyone looking to save some cash, starting at just $799 for the 128GB model. But even if you're saving more than if you get a Plus or Ultra model, you'd still be able to enjoy Samsung's new AI features, which are available on all S24 models.

Also: Samsung Galaxy S24 series hands-on: I did not expect this model to be my favorite

With a 1080p-LTPO AMOLED display, the Galaxy S24 model also borrows some features from the Plus and Ultra, like a 50MP main camera and a Qualcomm Snapdragon 8 Gen 3 chipset for improved performance, all wrapped in Gorilla Glass Victus 3 with an aluminum frame.

2. You want a phone that fits comfortably in your palm

If you're a fan of one-handed Candy Crush sessions, then you might enjoy the smaller Samsung Galaxy S24. The Galaxy S24 is the smallest model in the lineup, at a comfortable 6.2 inches. This size is perfect if you don't like your smartphone to push seven inches, which often results in the device falling out of your pocket or having to maintain an awkward, one-handed hold.

Alternatives to consider

BEST ANDROID ALTERNATIVE

Google Pixel 8 Pro

The Google Pixel 8 Pro is a great alternative to the S24 series of smartphones because it features many of the great AI capabilities that are exclusive to the Galaxy S24, thanks to Google's Assistant.

View at Amazon

BEST iPHONE ALTERNATIVE

iPhone 15

The iPhone 15 is just 0.1 inches smaller than the Samsung Galaxy S24 but has a higher resolution and pixel density. It's also a great fit for those who enjoy the Apple ecosystem.

View at Amazon

BEST SAMSUNG ALTERNATIVE

Samsung Galaxy S23 Ultra

The Samsung Galaxy S23 Ultra is last year's model that is similar to this year's option. Though it doesn't have the new AI features, it's discounted now at $1,055 for the 512GB option.

View at Samsung

Featured reviews

5 Free NoSQL Database Certification Courses

NoSQL databases diverge from traditional relational databases by leveraging non-tabular storage methods, such as document databases tailored for storing data in document formats like JSON and XML. Some of the types of NoSQL databases include key-value stores, wide-column databases, graph databases and more. MongoDB, Cassandra, HBase, Hypertable, and Neo4j are a few big names in this space.

AIM has curated a list of the top five free courses that every developer can take up to understand and implement NoSQL databases.

1. Database Architecture, Scale, and NoSQL with Elasticsearch

The University of Michigan offers a course on database architecture, scale, and NoSQL with Elasticsearch as part of PostgreSQL. Taught in English, but available in over 20 languages, the course covers PostgreSQL architecture, SQL and NoSQL analysis, and a comparison between ACID and BASE style architectures.

The intermediate-level free course requires approximately 10 hours to complete, offering flexibility in scheduling. The learning experience involves assessments through two quizzes, and a shareable career certificate. The final module explores database architecture, PostgreSQL, and scalable deployment configurations, emphasising basic CRUD operations, indexes, and the implementation of ACID requirements.

Register Database Architecture, Scale, and NoSQL with Elasticsearch.

2. Introduction to NoSQL Databases

With over 25 years of industry experience, Sarma Pydipally, principal engineer (database) of Verizon Data Services, brings to you this free introductory course on NoSQL databases.

This training course provides an introduction to the evolution of database technology, specifically focusing on the transition from traditional DBMS to NoSQL databases. The course covers the history and development of NoSQL databases, exploring concepts such as vertical scaling vs horizontal scaling and the scaling challenges faced by previous generations. Additionally, it delves into the Five V’s of big data – velocity, volume, variety, veracity, and value.

The need for NoSQL databases and their promises are examined, along with a discussion on the varied types within the NoSQL landscape. The course addresses document databases, key-value stores, graph databases, wide column stores, and time-series data.

Find out more about the course on the Introduction to NoSQL Databases.

3. NoSQL, Big Data, and Spark Foundations Specialisation

Available for free, the IBM NoSQL, Big Data, and Spark Foundations Specialisation is a three-course series designed to equip individuals with practical skills for a career in big data. The specialisation, taught in English by the IBM Skills Network Team, covers essential concepts in NoSQL databases, Big Data technologies, and Apache Spark.

Participants gain hands-on experience working with popular NoSQL databases like MongoDB, Apache Cassandra, and IBM Cloudant, mastering tasks such as data insertion, updating, querying, and indexing. The curriculum also delves into foundational Big Data knowledge, including Apache Hadoop, MapReduce, and Kubernetes, with a focus on hands-on lab experience.

Participants learn extract, transform, and load (ETL) processing and machine learning model deployment using Apache Spark, Spark SQL, and SparkML. The specialisation is suitable for beginners and targets aspiring data engineers, software developers, IT architects, data scientists, and IT managers.

Sign up for the course NoSQL, Big Data, and Spark Foundations Specialisation

4. NoSQL and DBaaS 101

This course by IBM, NoSQL and DBaaS 101 provides an overview of the NoSQL database landscape, emphasising the advantages of utilising Database-as-a-Service (DBaaS) and introducing the role of IBM Cloudant within this context. The course also covers account setup, database creation and replication, data loading and querying using Cloudant.

The objective is to equip participants with the necessary skills and understanding to leverage a flexible schema for fast application development, handle scalable datasets and concurrent users, and explore the benefits of offloading database administration to a service provider.

The prerequisites for the course include a connected laptop or desktop, a modern browser (preferably Google Chrome), and the installation of cURL for accessing HTTP REST API from the command line. Prior knowledge in databases, HTTP, JSON, and basic familiarity with browsers and cURL on the Linux command line is recommended.

Register NoSQL and DBaaS 101 Course by cognitiveclass

5. MongoDB Aggregation Framework

Learn MongoDB Aggregation Framework through this free course by the database giant itself. It holds an intermediate level, requires no prior experience and can be completed in 18 hours over three weeks, with a flexible schedule allowing you to learn at your own pace.

The curriculum covers schema design, relational data migrations, and machine learning with MongoDB. Upon completion, you’ll possess the skills to effectively use MongoDB and its aggregation framework in your data science workflow.

Find more information MongoDB Agreegation Framework by Coursera

Related Reads

  1. Most Popular NoSQL Database
  2. NoSQL vs SQL – Which is better?

The post 5 Free NoSQL Database Certification Courses appeared first on Analytics India Magazine.

Microsoft’s big bet on AI seems to be paying off

Microsoft front of building in NYC

Since its launch of Copilot (formerly Bing Chat) in February last year, Microsoft has gone full-on artificial intelligence (AI). The company has infused AI across all parts of the tech stack, including enterprise, security, cloud, hardware, and more — it seems to be paying off.

On Tuesday, Microsoft unveiled its second-quarter earnings. The revenue and earnings not only exceeded estimates but also significantly surpassed the numbers from the year before, with AI being a star of the growth.

Also: Microsoft Copilot vs. Copilot Pro: Is the subscription fee worth it?

"We've moved from talking about AI to applying AI at scale," said Microsoft CEO Satya Nadella. "By infusing AI across every layer of our tech stack, we're winning new customers and helping drive new benefits and productivity gains across every sector."

Microsoft reported a revenue of $62 billion, an increase of 18% compared to the prior fiscal year, and a net income of $21.9 billion, reflecting a 33% increase.

Microsoft's Cloud revenue grew significantly, up 24% year-over-year at $33.7 billion. Azure and other cloud services grew 30%, and according to Nadella, AI has a lot to do with it.

"Azure again took share this quarter with our AI advantage," said Nadella on an earning call, according to a report.

Additionally, Office Commercial products also saw a revenue increase of 15%, driven by Office 365 Commercial revenue growth of 17%, according to the earnings report.

These results follow the company making Microsoft Copilot for Microsoft 365, an AI assistant for Microsoft 365 applications, generally available for enterprise customers on Nov. 1, 2023.

Also: ChatGPT Plus users can now add relevant GPTs to their chats — here's how

Even though more time is needed to solely attribute the increase in revenue to that, it could be a factor, especially since the enterprise solution comes at a steep price of $30 per user per month.

Microsoft shows no signs of slowing down on the AI front, just this month unveiling its latest development, Copilot Pro, a supercharged subscription version of Copilot for users at $20 per month. There have also been reports of Microsoft and OpenAI looking into investing in a humanoid robot startup.

Artificial Intelligence

Shopify is rolling out an AI-powered image editor for products

Shopify is rolling out an AI-powered image editor for products Ivan Mehta 10 hours

Commerce platform Shopify is releasing new features for its Winter Editon rollout including an AI-powered media editor to enhance product images, improved semantic search, and a better way for merchants to showcase different variants of a product.

Many companies including Meta, Google, and Amazon have allowed advertisers to generate backgrounds using generative AI since last year. Similarly, Shopify wants to help merchants edit their product images’ backgrounds and scenes using this tech. In the Magic Media Editor, merchants can select a type from seven styles — Minimal, Vibrant, Natural, Urban, Regged, Refined, and Surreal — or type a prompt to generate a new background.

Apart from this, merchants can match an image’s background style with an existing image to maintain brand consistency. The tool can also remove unwanted backgrounds from the images.

Shopify has experimented with generative AI-based tools for merchants in the past. Last June, it launched the Sidekick bot to help merchants navigate different parts of the platform.

The company is also rolling out a semantic search, which goes beyond matching keywords from the search query. For example, a customer can ask a query like “warm clothes for winter” to get various results of thermal wear.

Shopify is also rolling out tools for better merchandising, so merchants can showcase different variants of a product. The company said that based on the change of variables, the product creation and classification tool supports 2,000 variants.

The company has also launched a new app for combined listings to manage that. The app, available for Shopify Plus customers, allows sellers to maintain unique descriptions, galleries, and URLs for different variants of products.

Merchants can add parameters like different colors, materials, and specific activities to target different audiences Image Credits: Shopify

Additionally, Shopify is launching headless storefronts — where businesses can use different custom front-end solutions while using Shopify-specified APIs to access backend — for B2B merchants.

Shopify raked in $1.7 billion in revenue with a 25% year-on-year growth for Q3 2023. The company laid off 20% of its staff (more than 2,000 people) in June 2023 and sold its logistics business to Flexport in exchange for 13% stock.

PyTorch 2.2 Releases New Features like the FlashAttention-v2 and AOTInductor

PyTorch has released version 2.2, featuring integration of FlashAttention-v2 which processes certain calculations about twice as fast as before. It also has the AOTInductor that allows PyTorch programs to be compiled in advance and used in environments where Python isn’t available. This is important for running AI models efficiently in different settings, like on web servers.

In the previous version’s FlashAttention, the main challenge was the quadratic increase in runtime and memory usage as the length of the sequence processed by the AI model increased. FlashAttention-v2 addressed this by optimizing memory usage, reducing it from a quadratic to a linear scale, and achieving a runtime speedup of 2-4 times compared to optimized baselines without any approximation.

It reaches 50-73% of the theoretical maximum performance on A100 GPUs, closely approaching the efficiency of optimized matrix multiplication operations.

The update also includes enhancements in torch.compile for Optimizers, new inductor optimizations, and the introduction of a logging mechanism named TORCH_LOGS.

It is important to note that this will be the last version supporting macOS x64, as macOS x86 support is being phased out.

This release comprises 3,628 commits from 521 contributors and has several key improvements and features. These include Device_mesh in torch.distributed, a tool that helps organize and manage how AI models are split and run on different parts of a computer or across multiple computers.

TORCH_LOGS Logging Mechanism which is a new way to record what happens when its programs run. This helps developers understand how their AI models are performing and troubleshoot any issues.

Further enhancements are made in torch.compile for Optimizers which are a part of PyTorch that helps AI models learn better from data. TorchInductor which improves how it handles Optimizers and TorchInductor. It relates to how PyTorch combines different parts of a program for efficient processing.

Additional features and performance improvements include inductor optimizations, aarch64 optimizations, and support for FlashAttention-2 in torch.nn.functional.scaled_dot_product_attention, which delivers significant speedups on A100 GPUs.

The post PyTorch 2.2 Releases New Features like the FlashAttention-v2 and AOTInductor appeared first on Analytics India Magazine.

8 Job Openings for Professionals Impacted by Recent Tech Layoffs

You thought 2023 was brutal? Think again. The number of techies laid off in January 2024 has surpassed last year’s count in the same month. While the firings continue unabated, there’s still hope as many companies are simultaneously looking to hire for the laid-off roles.

A study by 365 DataScience revealed that 23.7% of the laid-off employees from Twitter were software engineers. The number increased to 28.8% for Amazon and 35.5% at Google – the highest among the big tech companies. Other tech roles to lose out were data scientists and analysts.

Here are eight openings available for the role most affected by the ongoing tech layoffs.

Boeing, Software Engineer (Mid-Level and Experienced)

Boeing is seeking software engineers in Everett, WA, with over 5 years of experience in Java, C++, or Python software development. Additionally, 3+ years in embedded software development, verification, or certification is essential. Responsibilities will focus on avionics, cabin and network systems, common core systems, electronic subsystems, environmental controls, flight controls, or propulsion.

Duties include documenting software requirements, architectures, and interfaces, and ensuring adherence to industry standards, Boeing requirements, and FAA regulations for software certification. The role demands 7-9 years of experience.

Click here to apply.

LinkedIn, Software Engineer – iOS

Join LinkedIn’s team as an experienced iOS engineer focused on the profile completion track. Your role would involve enhancing LinkedIn’s profile experience by leading initiatives to elevate user experience and improve profile quality.

The role will let you collaborate with 12 engineers, contributing to the Profile TopCard++ team and shaping LinkedIn’s profile experience. The opportunity has core features like TopCard, About Card, Highlights, Open To, Badges, and foundational elements, impacting LinkedIn’s profile evolution.

The responsibilities span designing and implementing user-facing features for the iOS app, using cutting-edge technologies, and building scalable mobile apps.

Click here to apply.

US Bank, Senior Software Engineer – Full stack: Java, Spring Boot, React)

This role entails analyzing, designing, testing, developing, and maintaining high-quality software experiences. The ideal candidate, with 5-6 years of experience, is a collaborator capable of working across teams and ensuring software artifacts align with the US Bank standards.

Proficiency in the Agile software development lifecycle and DevOps principles is essential. Key responsibilities include:

  • Full-stack ownership.
  • Crafting production-ready and testable code.
  • Designing optimally while considering scalability, reliability, and performance.

Click here to apply.

eBay, MTS2, Software Engineer

eBay is looking for a staff software engineer for its Member-to-Member communications group, focusing on enabling global email and chat communication among eBay users. In this role, you’ll contribute to large-scale applications, processing millions of transactions per minute.

To qualify, you should bring over 10 years of experience in crafting and developing highly scalable distributed platforms, internet-scale web applications, and substantial hands-on expertise in Java/J2EE, SOAP, SOA services, design patterns, OOA/D, data structures, XML, REST, JSON, and Internet Protocols.

A technical degree (BSCS, BSEE, MSEE, MSCS, PhD, or similar) is required, reflecting your background in software engineering.

Click here to apply.

Mastercard, Lead Software Engineer/Data Engineer

As a lead site reliability/DevOps engineer within the Mastercard Data Platform and Engineering Service program, you’ll create data pipelines feeding into the Mastercard data warehouse.

You would contribute to developing performance metrics, maintaining dashboards reflecting production system health, and introducing proactive monitoring approaches.

Click here to apply.

Snowflake, Senior Software Engineer – Telemetry

Snowflake has an open position for senior software engineers based in San Mateo, CA. The role is focused on empowering Snowflake customers to harness time-series data, optimize alerts, and fine-tune notifications in their data pipelines.

As a Senior Software Engineer at Snowflake, your responsibilities include addressing customer requirements and creating highly reliable software for the largest number of customers. The role involves analyzing fault-tolerance, availability, performance, and scale challenges.

The ideal candidate brings 7+ years of industry experience in designing and building distributed data systems and a background in constructing enterprise-grade, reliable software or services. Proficiency in Java or C++ is essential.

Click here to apply.

AT&T, Lead Software Engineering, ServiceNow

The Lead Software Engineer is important to advance ABS Procurement and Logistics by delivering ServiceNow solutions. This position entails end-to-end responsibilities, including design, development, testing, deployment, and maintenance of applications within the enterprise.

Key responsibilities involve:

  • Gathering and reviewing requirements.
  • Estimating points.
  • Collaborating with various application teams to create design and interface documents.

The role includes supporting applications, solving configuration and environment issues, and overseeing the software deployment process.

Click here to apply.

Walmart, Software Engineer III

Walmart has an opening for a Software Engineer specializing in iOS to join the Acquisition and Discovery Engineering team. In this role, as a Software Engineer focused on the iOS platform for the Item Page team.

The ideal candidate brings at least three years of experience designing and developing software for the iOS platform and a broader frontend stack. Proficiency in mobile architecture is required.

Debugging, production support skills, and experience monitoring mobile app performance are crucial. Familiarity with public cloud technologies is a plus.

Click here to apply.

The post 8 Job Openings for Professionals Impacted by Recent Tech Layoffs appeared first on Analytics India Magazine.

Mastering E-commerce data governance: Best practices, challenges, and future trends for quality, compliance, and growth

data

Data governance is more important than ever in e-commerce, where massive amounts of data are generated and processed daily. Big Data presents opportunities and challenges for e-commerce businesses, requiring a strategic approach to data quality, security, and compliance. This article discusses e-commerce data governance best practices, including understanding data governance, data quality, data security, compliance with regulations, tools, challenges, case studies, and future trends.

Data governance in e-commerce ensures quality data-driven decisions, risk management, and compliance. Businesses rely on data governance to manage and use data. Data governance tools that are both automated and intelligent are utilized in the process of inventorying and categorizing data. Data stewards are subject-matter experts appointed after data is organized and policies are created to manage its use.

E-commerce data governance is important. For decisions to be made in a timely and accurate manner, high-quality data is absolutely necessary. Governance is essential for compliance and security because e-commerce platforms collect sensitive consumer data.

What is data governance?

Let’s dispel some data governance myths before learning more. Data governance is different from data lineage, MDM, or stewardship, but they are often used together. Terms are part of data governance practice, but not everything.

What’s data governance? Data governance involves formally managing an organization’s data and extracting value from it through people, processes, and technology to simplify and automate. Consider data security. The most valuable asset that a company possesses is its confidential information. Data must be secured and protected for confidentiality and legitimate access. People should help determine data access restrictions. This task is simplified and automated by identity management systems and permission management. Data stewards and security analysts cannot reconcile, classify, or integrate new data with existing data as the speed and volume of data increase, especially in the age of big data. Many companies store new data in a holding cell and classify it later. Luckily, your company doesn’t have to do this. Technology providers classify data as it arrives or soon after to solve such problems. These technologies meet authorization requirements and provide quick data access, reducing time to insight.

The importance of data governance

Data governance has only become important recently. It didn’t arrive alone. Events led to it. Data breaches have made data governance a priority for IT professionals.

GDPR and CCPA stem from these beaches and data privacy concerns. These are restricting how organizations use, manage, and collect personal data. Organizations use data governance frameworks to manage data due to breaches and privacy laws.

Big data governance

Big data systems like Hadoop make data governance difficult. Organizations combine clusters with various purposes and data stores, such as files, tables, and streams. The approach violates data governance principles. Even with proper stitching and securing, gaps remain.

Solved by converged architecture systems. Converged architecture integrates subsystems into a single data repository for improved security and governance.

How data governance works in e-commerce

Businesses that participate in e-commerce collect data from a wide range of sources, such as web analytics, email and marketing tools, online transactions, surveys, and a variety of other sources. The stakeholders make use of this data after it has been consolidated. Providing these stakeholders with access to high-quality data is the result of data governance initiatives that have been correctly implemented.

Mastering E-commerce data governance: Best practices, challenges, and future trends for quality, compliance, and growth

Through the process of governing and integrating these data sources, e-commerce teams have the ability to gain actionable insights regarding customers, trends, products, regions, and other topics. These insights matter greatly:

  • Communicating consumer and market trends to gaps.
  • Better client retention and engagement.
  • Inventory, labor, and pricing optimization.
  • Informing innovation and opportunities in the market that have not yet been reached.

Why is data governance important for e-commerce companies?

E-commerce businesses that don’t have a data governance strategy could have data breaches, get fined by the government, and lose the trust of their customers. These companies also run the risk of losing customers. It is possible for both the growth of a business and the reputation of a shirt brand to suffer. It is therefore essential for e-commerce to have data governance.

Improving visibility, relevance, and consistency

E-commerce companies receive diverse data from digital outlets and advanced customer journeys. Information regarding inventory and purchases made by customers must be updated across all platforms. However, multiple teams often take responsibility, risking data silos and outdated data.

E-commerce businesses are able to better manage their distributed data with the assistance of an effective and unified data governance system. This system prevents data silos and keeps data relevant across platforms. Because of this, businesses have increased visibility to expand.

Mastering E-commerce data governance: Best practices, challenges, and future trends for quality, compliance, and growth

Limiting data exposure

E-commerce and retail stakeholders should share data seamlessly. Unrestricted data flow has the potential to improve workflow efficiency and reduce the number of data silos, but it also may present potential risks to safety and security. Still, security breaches involving private customer data often make customers less likely to trust a brand.

For the purpose of protecting sensitive information, data governance systems provide e-commerce shirt brands with the capability to implement two-factor authentication, data encryption, and tokenization, in that order.

Dealing with data inconsistencies

E-commerce data warehouses may have inconsistencies. It is necessary to update all of the other repositories whenever a change is made in one repository. This process can become challenging and overwhelming when it is not coordinated properly.

Sales, revenues, productivity, and strategy are affected by inconsistent data. For the purpose of curating, modifying, and validating raw data, a robust data governance system makes use of process pipelines. Data visualization that is better and data analysis that is faster and more accurate can both help e-commerce companies. These benefits can be realized by businesses including jewelry wholesalers, that engage in e-commerce.

Conceptual framework and discussions

Big Data governance is important. Data governance framework aids big data transitions. Organizations struggle with managing large amounts of data. Big Data governance solutions affect all businesses and operations right away. No governance, difficult Big Data management. Eight components make up the governance framework for big data. Organization structure, stakeholders, Big data scope, policies and standards, optimization and computation, quality measurement and monitoring, data storage, communication, and management. Proposed framework uses governance principles. Big Data governance has seven principles: organization, metadata, privacy, data quality, business process integration, master data integration, and information lifecycle management.

Mastering E-commerce data governance: Best practices, challenges, and future trends for quality, compliance, and growth

Org structure impacts Big Data governance choices. Organizational structure needs study. Goals and vision guide Big Data governance. Organization structure is included. Identifying Big Data governance stakeholders is crucial. Determine the scope of Big Data and check if the organization has defined its scope. Big Data processing tech issues. Establish policies, rules, and standards for data capture, management, consumption, privacy, security, risk, retention, regulatory compliance, and classification.

Big Data governance includes data optimization, privacy, and realization. Verify if framed policies align with traditional system policies. Optimization and computation involve data acquisition and transformation. Orgs benefit from data analysis. Big Data quality measurement and monitoring is crucial. Fix inconsistent or invalid data in the analytics pipeline. Document every change from concept to visualization. Big Data governance improves quality. Cleaning data is necessary for analysis.

Data needs preparation and analysis. Data is stored securely and accessible. Outputs are delivered to clients. Comparing BGF1–BGF12 studies to ISO 8000 data governance framework standard to find gaps. Table compares ISO 8000 standards with FR and PR. The BGF7 framework is unavailable for comparison.

Mastering E-commerce data governance: Best practices, challenges, and future trends for quality, compliance, and growth

Data governance best practices

1. Data management is not the same thing as data governance

Data management provides assistance to software frameworks for data governance. The decisions regarding data management are made by data governance.

2. Frameworks that are the result of collaborative efforts are the most efficient

Employees of the organization who are proficient in data management ought to play a pivotal role in the design of the framework in order to maximize the efficiency of the process.

3. There must be an integration of data governance across the entire organization

After the framework has been functionalized, implementing it throughout the organization will ensure that data collection is consistent, which will assist each team in accomplishing their objectives.

4. Risk indicators and benchmarks

Data is valuable, and when it is shared, it increases the risk that an organization faces. When risk milestones are established, potential risks will be brought to light, and costly breaches will be avoided.

5. Make consistent improvements

As your company expands, you should audit your data governance strategy to ensure that it continues to satisfy the requirements of both your customers and your organization.

Data ownership and accountability in data governance

The following reasons make data ownership crucial to data governance:

1. Accountability and decision-making

In order to establish accountability for data management and integrity, data ownership is necessary. A data set’s owner is responsible for its quality, accuracy, and regulatory compliance. It is possible to make decisions regarding data in a timely and efficient manner thanks to this accountability. Regarding the establishment of data ownership, I believe this to be the most significant reason.

2. Data governance framework

A solid data governance framework starts with data ownership. It provides organizations with the authority to define and enforce data-related policies, standards, and processes, as well as the roles, responsibilities, and decision-making power to undertake these tasks. It is possible for data governance initiatives to become fragmented if there is a lack of clear data ownership, which can lead to inconsistent practices and problems with data management.

3. Data quality and integrity

Data quality and integrity depend on data ownership. The data owner is responsible for accuracy, completeness, and consistency when data ownership is clear. They want to protect data integrity by implementing data quality, validation, and governance measures.

4. Required compliance and regulation

Data ownership is closely related to regulatory compliance. By designating individuals as data owners, individuals are held accountable for comprehending and adhering to laws pertaining to data protection and privacy. Data owners can reduce data breaches and non-compliance risks by monitoring data usage, implementing security measures, and complying with laws.

Data governance through metadata management

By 2020, Gartner predicts 50% of information governance initiatives will use metadata-only policies. Metadata helps understand data (datasets and images), concepts (classification methods), and real-world entities.

Descriptive metadata describes sources for discovery and identification, structural metadata describes data models and reference data, and administrative metadata helps manage/monitor sources. For the purpose of data governance, numerous specifications and frameworks define metadata management.

Metadata management of the entire data infrastructure

ACS Solutions has built and integrated Hadoop clusters for several customers’ Business Intelligence infrastructures. We maintain and update those clusters for changing data flow needs.

Implementations begin with a few data sources, ingestion, transformation, and presentation requirements. As the company adds data sources, users, and applications, these requirements grow. These entities may be outside Hadoop infrastructure.

To ensure data consistency, integrity, and availability to authenticated/authorized users as data moves between systems, ACS Solutions uses Hortonworks Data Platform applications out of the box. We created a communication layer that lets Oracle, Cassandra, Kafka, and Hive exchange metadata governed by Apache Atlas.

The image above shows metadata management in data lakes, which source and receive data.

Atlas is a Hadoop-based open-source metadata repository. It can exchange metadata with SQL Server and Oracle, among other tools and processes, due to its flexibility.

In the dynamic realm of IT, the need for seamless customer support is paramount. Imagine being able to provide top-notch assistance to your clients from virtually anywhere, even while basking in the tranquility of a beach holiday.

Ranger adds data governance features to Atlas using Tags and handles authentication, authorization, and auditing using cluster Kerberization and Apache Knox. Tags, like PII, can be placed on any field like SSN. We can use the governance infrastructure to control data movement and monitoring at the tag level.

Falcon supports data lifecycle management, compliance (lineage and audit), replication, and archiving. Dataflow integration/workflow suites are the fourth data governance pillar and may or may not be part of Hadoop infrastructure.

These create a complete system for a self-service data marketplace in the organization regardless of publisher and consumer numbers. It will enable the data-driven organization to adapt to changing vocabularies, taxonomies, and coding schemes.

Compliance with Data Regulations in E-commerce

Understanding data privacy regulations

To comply with GDPR and CCPA, e-commerce businesses must stay informed. Avoiding legal issues and maintaining customer trust requires understanding these regulations.

Data governance policies and documentation

Comprehensive data governance policies that comply with regulations are essential. Auditing and compliance check processes are documented for accountability and transparency.

Training and awareness programs

Training teams on data privacy and compliance is ongoing. Regular training helps employees understand their roles and changing data protection laws.

What are the benefits of data governance for e-commerce?

The importance of data governance as well as the difficulties that it presents are brought to light, but the benefits that it brings to e-commerce businesses are the positive effects and value that it brings. It is beneficial for e-commerce businesses to have data governance frameworks:

  • Overall performance: Data governance helps teams find accurate data and gain insights faster.
  • The monitoring of data quality metrics exhibits the utilization of data by both the team and the stakeholders.
  • Improved understanding of the business world to include: E-commerce businesses are able to identify areas of weakness, gain a competitive advantage, and find new revenue streams due to their ability to acquire these capabilities.
  • When properly controlled, high-quality data enables individuals to make decisions quickly and accurately while still protecting their privacy and complying with regulations.
  • Ownership of data, responsibility for data, and accountability for data: E-commerce teams are informed by data governance about whom they should consult for data issues.

Employing data governance in e-commerce

When it comes to e-commerce, data governance is everything. E-commerce businesses will be in a better position to take advantage of new opportunities, increased customer confidence and security, increased sales, and growth if they implement an efficient data governance solution.

Businesses that engage in e-commerce will be able to establish and maintain data governance with the assistance of best practices for data governance, which will also position them for growth.

Future Trends in E-commerce Data Governance

AI and ML in data governance

Future technology combines AI for predictive data quality and ML for security and compliance automation. These technologies will support proactive and intelligent data governance.

Blockchain for transparent and secure transactions

Blockchain technology could make e-commerce transactions secure and transparent. In data governance, it can improve supply chain traceability, transparency, and trust.

Conclusion

E-commerce relies on data governance for quality decisions, risk management, and compliance. It involves classifying, organizing, and creating data usage policies. E-commerce requires it due to consumer data sensitivity. Data governance enables actionable insights, customer engagement, and operations optimization. Companies risk data breaches, regulatory fines, and trust loss without it. Improvements include performance, data quality, and informed decision-making. Future trends include AI/ML integration and blockchain for transparent transactions, emphasizing the need for strong e-commerce data governance.

ChatGPT Plus users can now add relevant GPTs to their chats — here’s how

Referencing a custom GPT

Since rolling out early this month, OpenAI's ChatGPT store has attracted thousands of custom GPT chatbots, many with distinctive and useful abilities. Now, you can reference any of those GPTs in a normal chat as a way to tap into their individual skills.

In a Monday post on X, aka Twitter, OpenAI announced the launch of the new feature by saying: "You can now bring GPTs into any conversation in ChatGPT — simply type @ and select the GPT. This allows you to add relevant GPTs with the full context of the conversation."

Also: Microsoft Copilot vs. Copilot Pro: Is the subscription fee worth it?

As an example, maybe you're involved in a conversation with ChatGPT about film and want to get help or information from a film-related GPT. Typing the @ symbol followed by the word "film" displays a list of custom GPTs matching that word. You can then select the GPT you wish to invoke, and it will join the conversation.

Like the custom GPT feature itself, the GPT mention capability is available only to paid ChatGPT Plus and Enterprise subscribers. But if you do subscribe, here's how to try it out.

Sign in to your ChatGPT home page and click the link at the top of the left sidebar for Explore GPTs. Browse through the different GPTs in the store and try a few that interest you, particularly ones devoted to your favorite topics. In my case, film is one of my favorite areas.

Also: ChatGPT vs ChatGPT Plus: Is it worth the subscription fee?

If you like a specific GPT, you can pin it to the sidebar so that it's readily available. To do this, click the name of the GPT at the top and select the menu option for "Keep in sidebar." That GPT will then always be visible in the sidebar.

After you've checked out and pinned a few GPTs, head back to the home page and kick off a conversation with ChatGPT. Type the @ symbol, and a list of recent and pinned GPTs appears. Select the GPT you want to use, and it will add its knowledge and information to your conversation.

Call on a custom GPT to add to your conversation.

As the GPT store grows, finding and using specific custom GPTs will likely become more challenging. The goal behind the new mention feature is to provide quick and easy access to relevant and useful GPTs that can enhance any conversation.

Artificial Intelligence