K8sGPT is a tool for scanning your Kubernetes clusters, diagnosing and triaging issues in simple english. It has SRE experience codified into its analyzers and helps to pull out the most relevant information to enrich it with AI.

K8SGPT is an advance AI algorithm analysis your cluster state and provide intelligent inside for troubleshooting.  

WorkFlow:

Frontier agent are autonomous system that work independent to achieve goals, scale massive to tackle concurrent tasks and run persistenly for hours or days without human intervention.

 Frontier Agent option:

Context Management like telling story when it getting convoluted. It will direct path when AI get confused.

MCP - Model Context Protocol:

MCP like an interface which communicated to MCP client and discover their requirement and offer available services for their requirement. 

RAG - Retrieval Augmented Generation

It converts a data into numerical representation where each piece of data has information about how it relates to others.

Retrieval - when user ask a question or search, RAG turns question or search into own numerical representation (Embedding) and find a data which is similar meanings.

Augmentation - The top search result are then added into prompt and send to back to LLM

Generation - The search results give the LLM some local context and consider as response.

Embedding:

LLM stands for Large Language Model. It is specifically a deep learning model, trained on massive amounts of text data to understand and generate human language, enabling tasks like text generation, translation. It often sing "Transformer" models which are neural networks that can process relationships within language.

AI - System or machines that mimic human intelligence to perform tasks and can iteratively improve themselves based on the information they collect.  Artificial Intelligence capable of generating text, images, videos or other data using generative models often in response to prompts.   Generative AI models learn the patterns and structure of their input training data and then generate new data that has similar characteristics.

AGI - Artificial General Intelligence is a type of AI that can understand, learn and apply knowledge across broad range of tasks similar to human cognitive abilities.   

Example of Cloud Machine Learning:

Different AI Types:

Machine Learning:

Simple Input -> Simple output -> Single topic

Deep Learning:

Complex input -> Simple output -> Single topic

Foundation Model:

Complex inputs -> Complex ouputs -> multiple topics

Foundation Model:

A foundation model is a type of large scale artificial intelligence model that is trained on a broad range of data at massive scale, allowing to develop a wide understanding of many topics and tasks.  These models can be adapted or fine-tuned for various specific applications, demonstrating flexibility and efficiency across different domains. 

 Parameters of foundation models:

• Embedding Vectors : The foundation model dealing with categorical variable (like words or user ID in recommendation systems), embedding vectors are a form of parameter represent a categories in a continuous vector space and capturing semantic similarities among categories. 
• Weights:  This is a numerous parameters in neural network.  Weights are used in various layers of a neural network to scale the input data in a meaningful ways.  For example in a convolutional layer commonly used in image processing, weights determine the importance of neighboring pixel values for feature detection. 
• Biases: Bias parameters are added to the output of weighted inputs to shift the activation function curve up or down.  This is crucial for models to accurately represent patterns in data that do not pass through the origin of the coordinate system. 
• Attention Scores: Attention mechanisms use parameters to weigh the significance of different parts of the input data differently.  For instance, in language models attention scores determine how much emphasis the model places on different words when generating a response or translating text.

LLM:

• A language model (LM) is a probabilistic model of text.

Encoders:

Models that convert a sequence of words to an embedding.

Decoders:

Models take a sequence of words and output next word.

Examples: GPT-4, Llama and bloom

Encoder - decoder module:

We passed english letters and encoder covert into token. Decode passed one tocken at time.

Hallucination:

It is generated text that is non factual and Or ungrounded.

LLM application:

Retrieval Augmented Generation (RAG)

• Primarily used in QA where the model has access to support documents for a query.

Code models:

• Instead of training on written language train on code and comments.

In-context learning and few shot prompting:

• In context learning - conditions an LLM with instructions and Or demonstrate of the task.
• K-shot prompting: Explicitly providing k examples of the intended task in the prompt.
• F-string Or formatted string are a feature in python can be used to create prompt templates for LLM.

Language Agents:

* A Budding area of research where LLM based agents

Some notable work in the space:

* ReAct 

Iterative framework where LLM emits thoughts, then act and observes result

* Toolformer

Pre-training technique where strings are replaced with calls to tools that yield result.

OCI Generative AI service:

* Fully managed service that provides a set of customizable Large Language Models (LLM) avilable via a single API to build generative AI applications.

Generation:

Command -> Command light -> llama 2.7

Dedicated AI cluster:

* Dedciated AI cluster has a GPU based resource that host the customers fine-tuning and inference workloads.

OCI setup:

configuration file : ./oci/config

Model parameters:

Temperature : Determines how creative module should be, default temperature is 1 and maximum temperature is 5.

Length: Approximate length of the summary, choose from short, medium and length.

Embeddings:

Embedding is a numerical represent of piece of text converted into number sequences.

A piece of text could be a word, phrase, sentence or paragraph or more paragraphs.

Models creates a 1024 vector for each embedding.

Max 512 tokens per embedding.

Model create a 384 dimensional vector for each embedding.

It will be very difficult to tune a 2 billion tokens. We are using the In-context Learning/Few shot Prompting:

GPU memory is limited, so switching between models can incur significant overhead due to reloading the full GPU memory.

Dedicated AI cluster units:

* Large cohere - Dedicated AI cluster units for hosting or fine tuning the cohere command

* Small Cohere - Dedicated AI cluster units for  hosting or fine tuning for small cohere command

* Embed Cohere - Dediated AI cluster for hosting the models

* Llama2-70 model - Dedicated AI cluster for hosting the Llamba2 models

Fine tunnning is required 2 units and each cluster is active for five hours.

RAG framework:

Retriever : It is act like search engine. 

Ranker : Evalate and priorites rank based a quality of the data.

Generator : It provide  human like texts.

RAG techniques:

RAG sequence 

RAG token

RAG pipelines:

Documents -> Chunks -> Embedding -> Index [database]

Vector database:

A vector is sequence of numbers called dimensions, used to capture the important "features" of the data.

Semantic search:

It means search by meaning rather than giving a number.