Generative AI 101: A Quick Introduction

As software developers our goal is simple: build products that make our end users lives easier, faster and smarter.

Generative AI is opening a new world of possibilities — from automating content creation to enhancing customer experiences.

In my past projects, like GO Bermondsey (a coworking space platform) and Writely (a content creation tool), I created AI features like content generation, meta tag automation, social media hashtags, and chatbots. At the time, it felt like pure magic — I used Open AI API without fully understanding how it worked.

Recently, I’ve started learning more again through the ChaiCode Gen AI Cohort (I’ve just attended the few class!) — and it’s been eye-opening.

In this blog, I’ll break down the basics of what I’ve learned so far about how Generative AI works. Let's dive in!

What is Generative AI?

It is a type of artificial intelligence that can generate new content such as text, images, music etc. To do this, you have to describe what want to create to AI. This is called a prompt. The AI model then uses its knowledge to generate something new based on that prompt.

Let’s understand the jargon used in AI development:

Token:

It represents the smallest data unit that the AI model can understand from its dictionary. It can be a character, word, sub-word or number etc.

const OM_AI_MODEL_TOKEN_DICTIONARY = {
    I: 49,
    a: 97,
    am: 20,
    developer: 49,
    by: 59,
    profession: 76,
    and: 70,
    love: 89,
    coding: 99,
    you: 100,
    he: 101,
    she: 102,
    'pani puri': 103
};

Tokenization:

It is a process of breaking down input data like sentences, texts, audio, images or videos into tokens. So that AI model can process and understand each token.

If you want to view what tokenisation looks like, you can use this link: Tokenisation Visualizer.

Encoding:

Converting tokens into numbers that the AI model can understand. Each token is mapped to a unique number using a predefined dictionary.

const OM_AI_MODEL_TOKEN_DICTIONARY = {
    I: 49,
    a: 97,
    am: 20,
    developer: 49,
    by: 59,
    profession: 76,
    and: 70,
    love: 89,
    coding: 99,
    you: 100,
    he: 101,
    she: 102,
};

const mySentence = "I am a developer by profession and I love coding.";
const tokens = mySentence.split(" ");
const encodedTokens = tokens.map(
    (token) => OM_AI_MODEL_TOKEN_DICTIONARY[token]
);
console.log(encodedTokens);
// Output: [ 49, 20, 97, 49, 59, 76, 70, 49, 89, 99 ]

Decoding:

Converting numbers back into tokens that humans can understand.

const OM_AI_MODEL_TOKEN_DICTIONARY = {
    49: "I",
    97: "a",
    20: "am",
    49: "developer",
    59: "by",
    76: "profession",
    70: "and",
    89: "love",
    99: "coding",
    100: "you",
    101: "he",
    102: "she",
};

const myEncodedSentence = [49, 20, 97, 49, 59, 76, 70, 49, 89, 99];
const decodedTokens = myEncodedSentence.map(
    (encodedToken) => OM_AI_MODEL_TOKEN_DICTIONARY[encodedToken]
);
console.log(decodedTokens);
// Output: [ 'I', 'am', 'a', 'developer', 'by', 'profession', 'and', 'I', 'love', 'coding.' ]

Vector Embedding:

Vector embedding means turning words into lists of numbers so that the system can understand their meaning and find relationships between them.

Embedding:

An embedding is like a secret code that turns each word into a list of numbers so the AI model can understand if “chai” and “coffee” are similar.

Positional Encoding:

Keeps track of words or tokens which come first, second, third and so on in a sentence. It helps the AI model to understand the order of words.

Semantic Meaning:

Semantic meaning is the actual meaning of words. For example: understanding “chai” means tea, not just letters C-H-A-I.

Self-Attention:

Self-attention is when each word in a sentence pays attention to all the other words to understand them better.

Softmax:

Softmax function is like a magical decision-maker that helps an AI model choose the best option by turning numbers into chances or probabilities.

Multi-Head Attention:

Multi-Head Attention allows a model to focus on different parts of a statement parallel to better understand various relationships.

Temperature (Temp):

A parameter that controls randomness in the model during output generation. The low values make the model play safe and select the most likely word, while high values let it take creative risks.

Knowledge Cut-off:

The last date AI model was trained on data. For example, if the knowledge cut-off is Dec 2020 then the AI model will not have information about events or developments that occurred after that date.

Vocab Size:

Number of unique tokens in the AI model's dictionary or vocabulary. A larger dictionary size allows the AI model to understand or generate.

Thank you for reading! 🚀

Happy learning! 😊