Everywhere you look, the term “Generative AI” is being talked about as the Next Big Thing. “Generative AI” is often associated with “ChatGPT” for near-sentient chatbots, and “Dall-E” or “StableDiffusion” for breathtaking images. But what is Generative AI? How is it different from regular AI? And how are people actually using it in practice?
Most people are familiar with regular AI concepts – whether we think of facial recognition on our phones, shopping recommendations, or autonomous driving, or tracking defects on a manufacturing line. Behind the scenes, we have captured a set of known data, such as faces, fingerprints, buying preferences, road signs, vehicles or components on a production line. We feed the known data into a system, and the system learns how to identify good from bad, right from wrong, safe from unsafe. We have converted all that data into a series of optimized mathematical parameters, which let us take a new image and create a recommendation or an answer in the blink of an eye.
But if we have an AI model which can interpret the complexity of life, and boil it down to a set of essential characteristics (cat/dog, left/right, two legs good, four legs bad), then can we use a similar technique to generate something new, which would fit seamlessly back into our world?
We want a way for AI not to just recognise a flower, but to describe the flower, place it in context, and share more information in a way which fits with a given writing style. After all, there are many ways to describe a beautiful flower, but only Shakespeare might write something like “A rose by any other name would smell as sweet.”
Enter “Generative AI”
Generative AI extends the existing concepts of AI, by gathering much more information about the context and the rich diversity of the input data – so that we develop the capability to express a wider range of output than simple interpretation. Let’s use language processing as an example, although the concepts extend equally well to images, science, engineering and many other fields.
Let’s imagine we are reading text using AI, then a regular AI system would capture the patterns in the text, and analyze and reduce to identify key characteristics. On the other hand, if we want to generate some new text which corresponds to a certain meaning, we need to have a way to take some key idea, and be able to construct something which sounds natural. The problem is that “natural” is conversational, non-repetitive, and needs to conform to a recognisable style (business vs prosaic, rhyme vs rap, soundbite vs Shakespearean sonnet).
So for Generative AI, we need to capture the many possibilities of word context, the shape of the phrasing, and the evolution of the argument through an essay. In technical terms, this means using the concept of “attention” to analyze the relation between neighboring elements – and “transformers” to evaluate the associations between local relationships and the overall content to derive the higher-level structure of an essay or a picture.
Generative AI in Practice
Creative teams quickly discovered the opportunities of Generative AI to inspire copywriting and graphic design – and every day, we hear of new examples where other types of organisation have been able to integrate ChatGPT into their own workflows, allowing them to innovate within their own problem spaces. For example, an IT Trouble Ticket provider can use ChatGPT to express richer customer interactions which provide context about a user’s request for IT support, or a security and networking organisation can leverage similar technology to provide a natural language interface to configure complex security policies.
But Generative AI is not only about creating text and images, it can be applied to other domains: at our recent Life Sciences Field Day, David Ruau of NVIDIA talked about how pharmaceutical researchers are using Generative AI to create novel drugs and therapies, by understanding the rules of how peptides are made, how they interact, and how they might behave – helping to identify novel therapies much faster than before.
Elsewhere, Generative AI is used to assist software developers become more productive, or to resolve security vulnerabilities, helping to tame the complexity of managing software coding by creating contextual documentation and better design patterns.
Generative AI has the potential to transform many types of organisation and workflow, but ultimately needs to be seen as a way to augment existing skills and enhance productivity, rather than to replace creative talent. Think of Generative AI as your co-pilot, rather than your auto-pilot: you provide the guidance, retain the control, and remain ultimately responsible for what is created.
How to Train Your Generative AI Dragon
All this means building a model with many, many more parameters, and training with much, much more data. As a baseline, the original ChatGPT uses the GPT-3.5 model, which comprises 175 billion parameters, and the next generation GPT-4 is estimated to be 1000x larger. This colossal size is essential to capture the richness and diversity of the possible outputs, as inferred from the input data. In turn, this model needs to be trained on very large data sets with enough variation to express the creativity, and the model needs to read and re-read the training data many times in order to extract and analyze the fine-grained relationships which make up the creative space we want to explore.
Even with very large systems, the model training can take weeks or months, and ultimately, the bottleneck in the process is in driving the training data around the system, and writing out checkpoints to capture intermediate results during the lengthy learning process. Translating this into the real world, we need powerful computation systems, matched with scalable, high throughput storage systems which can tackle the immense scale needed for global-scale generative AI applications.
Watch Our Webinar
Want to learn more about how Generative AI systems are built to scale for the most demanding creative applications? Listen in to our on-demand webinar “Intelligent infrastructure for Generative AI” with James Coomer from DDN, Premal Savla from NVIDIA and Tim Phillips from The Register .
