Skip to content

Commit 8e37099

Browse files
koreyspacekoreyspace
committed
added meta lesson
1 parent 078a44d commit 8e37099

File tree

3 files changed

+396
-0
lines changed

3 files changed

+396
-0
lines changed

‎21-meta/README.md‎

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
# Building With the Meta Family Models
2+
3+
## Introduction
4+
5+
This lesson will cover:
6+
7+
- Exploring the two main Meta family models - Llama 3.1 and Llama 3.2
8+
- Understanding the use-cases and scenarios for each model
9+
- Code sample to show the unique features of each model
10+
11+
12+
## The Meta Family of Models
13+
14+
In this lesson, we will explore 2 models from the Meta family or "Llama Herd" - Llama 3.1 and Llama 3.2
15+
16+
These models come in different variants and are available on the Github Model marketplace. Here are more details on using Github Models to [prototype with AI models](https://docs.github.com/en/github-models/prototyping-with-ai-models?WT.mc_id=academic-105485-koreyst).
17+
18+
Model Variants:
19+
Llama 3.1 - 70B Instruct
20+
Llama 3.1 - 405B Instruct
21+
Llama 3.2 - 11B Vision Instruct
22+
Llama 3.2 - 90B Vision Instruct
23+
24+
*Note: Llama 3 is also available on Github Models but won't be covered in this lesson*
25+
26+
## Llama 3.1
27+
28+
At 405 Billion Parameters, Llama 3.1 fits into the open source LLM category.
29+
30+
The mode is an upgrade to the earlier release Llama 3 by offering:
31+
32+
- Larger context window - 128k tokens vs 8k tokens
33+
- Larger Max Output Tokens - 4096 vs 2048
34+
- Better Multilingual Support - due to increase in training tokens
35+
36+
These enables Llama 3.1 to handle more complex use cases when building GenAI applications including:
37+
- Native Function Calling - the ability to call external tools and functions outside of the LLM workflow
38+
- Better RAG Performance - due to the higher context window
39+
- Synthetic Data Generation - the ability to create effective data for tasks such as fine-tuning
40+
41+
### Native Function Calling
42+
43+
Llama 3.1 has been fine-tuned to be more effective at making function or tool calls. It also has two built-in tools that the model can identify as needing to be used based on the prompt from the user. These tools are:
44+
45+
- **Brave Search** - Can be used to get up-to-date information like the weather by performing a web search
46+
- **Wolfram Alpha** - Can be used for more complex mathematical calculations so writing your own functions is not required.
47+
48+
You can also create your own custom tools that LLM can call.
49+
50+
In the code example below:
51+
52+
- We define the available tools (brave_search, wolfram_alpha) in the system prompt.
53+
- Send a user prompt that asks about the weather in a certain city.
54+
- The LLM will respond with a tool call to the Brave Search tool which will look like this `<|python_tag|>brave_search.call(query="Stockholm weather")`
55+
56+
*Note: This example only make the tool call, if you would like to get the results, you will need to create a free account on the Brave API page and define the function itself`
57+
58+
```python
59+
import os
60+
from azure.ai.inference import ChatCompletionsClient
61+
from azure.ai.inference.models import AssistantMessage, SystemMessage, UserMessage
62+
from azure.core.credentials import AzureKeyCredential
63+
64+
token = os.environ["GITHUB_TOKEN"]
65+
endpoint = "https://models.inference.ai.azure.com"
66+
model_name = "meta-llama-3.1-405b-instruct"
67+
68+
client = ChatCompletionsClient(
69+
endpoint=endpoint,
70+
credential=AzureKeyCredential(token),
71+
)
72+
73+
74+
tool_prompt=f"""
75+
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
76+
77+
Environment: ipython
78+
Tools: brave_search, wolfram_alpha
79+
Cutting Knowledge Date: December 2023
80+
Today Date: 23 July 2024
81+
82+
You are a helpful assistant<|eot_id|>
83+
"""
84+
85+
messages = [
86+
SystemMessage(content=tool_prompt),
87+
UserMessage(content="What is the weather in Stockholm?"),
88+
89+
]
90+
91+
response = client.complete(messages=messages, model=model_name)
92+
93+
print(response.choices[0].message.content)
94+
```
95+
96+
## Llama 3.2
97+
98+
Despite being a LLM, one limitation that Llama 3.1 has is multimodality. That is, being able to use different types of input such as images as prompts and providing responses. This ability is one of the main features of Llama 3.2. These features also include:
99+
100+
- Multimodality - has the ability to evaluate both text and image prompts
101+
- Small to Medium size variations (11B and 90B) - this provides flexible deployment options,
102+
- Text-only variations (1B and 3B) - this allows the model to be deployed on edge / mobile devices and provides low latency
103+
104+
The multimodal support represents a big step in the world of open source models. The code example below takes both an image and text prompt to get an analysis of the image from Llama 3.2 90B.
105+
106+
107+
### Multimodal Support with Llama 3.2
108+
109+
```python
110+
import os
111+
from azure.ai.inference import ChatCompletionsClient
112+
from azure.ai.inference.models import (
113+
SystemMessage,
114+
UserMessage,
115+
TextContentItem,
116+
ImageContentItem,
117+
ImageUrl,
118+
ImageDetailLevel,
119+
)
120+
from azure.core.credentials import AzureKeyCredential
121+
122+
token = os.environ["GITHUB_TOKEN"]
123+
endpoint = "https://models.inference.ai.azure.com"
124+
model_name = "Llama-3.2-90B-Vision-Instruct"
125+
126+
client = ChatCompletionsClient(
127+
endpoint=endpoint,
128+
credential=AzureKeyCredential(token),
129+
)
130+
131+
response = client.complete(
132+
messages=[
133+
SystemMessage(
134+
content="You are a helpful assistant that describes images in details."
135+
),
136+
UserMessage(
137+
content=[
138+
TextContentItem(text="What's in this image?"),
139+
ImageContentItem(
140+
image_url=ImageUrl.load(
141+
image_file="sample.jpg",
142+
image_format="jpg",
143+
detail=ImageDetailLevel.LOW)
144+
),
145+
],
146+
),
147+
],
148+
model=model_name,
149+
)
150+
151+
print(response.choices[0].message.content)
152+
```
153+
154+
155+

‎21-meta/python/githubmodels-assignment.ipynb‎

Lines changed: 241 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,241 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Building With the Meta Family Models \n",
8+
"\n",
9+
"## Introduction \n",
10+
"\n",
11+
"This lesson will cover: \n",
12+
"\n",
13+
"- Exploring the two main Meta family models - Llama 3.1 and Llama 3.2 \n",
14+
"- Understanding the use-cases and scenarios for each model \n",
15+
"- Code sample to show the unique features of each model \n",
16+
"\n",
17+
"\n",
18+
"## The Meta Family of Models \n",
19+
"\n",
20+
"In this lesson, we will explore 2 models from the Meta family or \"Llama Herd\" - Llama 3.1 and Llama 3.2 \n",
21+
"\n",
22+
"These models come in different variants and are available on the Github Model marketplace. Here are more details on using Github Models to [prototype with AI models](https://docs.github.com/en/github-models/prototyping-with-ai-models?WT.mc_id=academic-105485-koreyst).\n",
23+
"\n",
24+
"Model Variants: \n",
25+
"Llama 3.1 - 70B Instruct \n",
26+
"Llama 3.1 - 405B Instruct \n",
27+
"Llama 3.2 - 11B Vision Instruct \n",
28+
"Llama 3.2 - 90B Vision Instruct \n",
29+
"\n",
30+
"*Note: Llama 3 is also available on Github Models but won't be covered in this lesson*\n",
31+
"\n"
32+
]
33+
},
34+
{
35+
"cell_type": "markdown",
36+
"metadata": {},
37+
"source": [
38+
"## Llama 3.1 \n",
39+
"\n",
40+
"At 405 Billion Parameters, Llama 3.1 fits into the open source LLM category. \n",
41+
"\n",
42+
"The mode is an upgrade to the earlier release Llama 3 by offering: \n",
43+
"\n",
44+
"- Larger context window - 128k tokens vs 8k tokens \n",
45+
"- Larger Max Output Tokens - 4096 vs 2048 \n",
46+
"- Better Multilingual Support - due to increase in training tokens \n",
47+
"\n",
48+
"These enables Llama 3.1 to handle more complex use cases when building GenAI applications including: \n",
49+
"- Native Function Calling - the ability to call external tools and functions outside of the LLM workflow\n",
50+
"- Better RAG Performance - due to the higher context window \n",
51+
"- Synthetic Data Generation - the ability to create effective data for tasks such as fine-tuning \n",
52+
"\n"
53+
]
54+
},
55+
{
56+
"cell_type": "markdown",
57+
"metadata": {},
58+
"source": [
59+
"### Native Function Calling \n",
60+
"\n",
61+
"Llama 3.1 has been fine-tuned to be more effective at making function or tool calls. It also has two built-in tools that the model can identify as needing to be used based on the prompt from the user. These tools are: \n",
62+
"\n",
63+
"- **Brave Search** - Can be used to get up-to-date information like the weather by performing a web search \n",
64+
"- **Wolfram Alpha** - Can be used for more complex mathematical calculations so writing your own functions is not required. \n",
65+
"\n",
66+
"You can also create your own custom tools that LLM can call. \n",
67+
"\n",
68+
"In the code example below: \n",
69+
"\n",
70+
"- We define the available tools (brave_search, wolfram_alpha) in the system prompt. \n",
71+
"- Send a user prompt that asks about the weather in a certain city. \n",
72+
"- The LLM will respond with a tool call to the Brave Search tool which will look like this `<|python_tag|>brave_search.call(query=\"Stockholm weather\")` \n",
73+
"\n",
74+
"*Note: This example only make the tool call, if you would like to get the results, you will need to create a free account on the Brave API page and define the function itself` "
75+
]
76+
},
77+
{
78+
"cell_type": "code",
79+
"execution_count": 20,
80+
"metadata": {},
81+
"outputs": [
82+
{
83+
"name": "stdout",
84+
"output_type": "stream",
85+
"text": [
86+
"<|python_tag|>brave_search.call(query=\"Stockholm weather\")\n"
87+
]
88+
}
89+
],
90+
"source": [
91+
"import os\n",
92+
"from azure.ai.inference import ChatCompletionsClient\n",
93+
"from azure.ai.inference.models import AssistantMessage, SystemMessage, UserMessage\n",
94+
"from azure.core.credentials import AzureKeyCredential\n",
95+
"\n",
96+
"token = os.environ[\"GITHUB_TOKEN\"]\n",
97+
"endpoint = \"https://models.inference.ai.azure.com\"\n",
98+
"model_name = \"meta-llama-3.1-405b-instruct\"\n",
99+
"\n",
100+
"client = ChatCompletionsClient(\n",
101+
" endpoint=endpoint,\n",
102+
" credential=AzureKeyCredential(token),\n",
103+
")\n",
104+
"\n",
105+
"\n",
106+
"tool_prompt=f\"\"\"\n",
107+
"<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n",
108+
"\n",
109+
"Environment: ipython\n",
110+
"Tools: brave_search, wolfram_alpha\n",
111+
"Cutting Knowledge Date: December 2023\n",
112+
"Today Date: 23 July 2024\n",
113+
"\n",
114+
"You are a helpful assistant<|eot_id|>\n",
115+
"\"\"\"\n",
116+
"\n",
117+
"messages = [\n",
118+
" SystemMessage(content=tool_prompt),\n",
119+
" UserMessage(content=\"What is the weather in Stockholm?\"),\n",
120+
"\n",
121+
"]\n",
122+
"\n",
123+
"response = client.complete(messages=messages, model=model_name)\n",
124+
"\n",
125+
"print(response.choices[0].message.content)\n",
126+
"\n",
127+
"\n",
128+
"\n"
129+
]
130+
},
131+
{
132+
"cell_type": "markdown",
133+
"metadata": {},
134+
"source": [
135+
"### Llama 3.2 \n",
136+
"\n",
137+
"Despite being a LLM, one limitation that Llama 3.1 has is multimodality. That is, being able to use different types of input such as images as prompts and providing responses. This ability is one of the main features of Llama 3.2. These features also include: \n",
138+
"\n",
139+
"- Multimodality - has the ability to evaluate both text and image prompts \n",
140+
"- Small to Medium size variations (11B and 90B) - this provides flexible deployment options, \n",
141+
"- Text-only variations (1B and 3B) - this allows the model to be deployed on edge / mobile devices and provides low latency \n",
142+
"\n",
143+
"The multimodal support represents a big step in the world of open source models. The code example below takes both an image and text prompt to get an analysis of the image from Llama 3.2 90B. \n",
144+
"\n",
145+
"### Multimodal Support with Llama 3.2"
146+
]
147+
},
148+
{
149+
"cell_type": "code",
150+
"execution_count": 4,
151+
"metadata": {},
152+
"outputs": [
153+
{
154+
"name": "stdout",
155+
"output_type": "stream",
156+
"text": [
157+
"The image shows a screenshot from a video or presentation about AI, with a woman on the left side and a catalog of models on the right side. The woman is wearing a gray hoodie and has blonde hair. She is visible from the chest up and appears to be speaking or presenting.\n",
158+
"\n",
159+
"The model catalog is titled \"MODEL CATALOG\" in large text at the top, with the subtitle \"Find the right model to build your custom AI solution\" below it. The catalog lists various models, including \"gpt-4,\" \"gpt-35-turbo-instruct,\" and \"mistralai-Mistral-Rx228-v0-1.\" Each model has a brief description and a button to view more information.\n",
160+
"\n",
161+
"At the bottom of the image, there is a blue bar with white text that reads \"Try it yourself\" and a URL that says \"https://ai.azure.com.\" The background of the image is white, with some colorful graphics and logos in the corners. Overall, the image appears to be a screenshot from a presentation or video about AI and machine learning, showcasing a catalog of models that can be used to build custom AI solutions.\n"
162+
]
163+
}
164+
],
165+
"source": [
166+
"import os\n",
167+
"from azure.ai.inference import ChatCompletionsClient\n",
168+
"from azure.ai.inference.models import (\n",
169+
" SystemMessage,\n",
170+
" UserMessage,\n",
171+
" TextContentItem,\n",
172+
" ImageContentItem,\n",
173+
" ImageUrl,\n",
174+
" ImageDetailLevel,\n",
175+
")\n",
176+
"from azure.core.credentials import AzureKeyCredential\n",
177+
"\n",
178+
"token = os.environ[\"GITHUB_TOKEN\"]\n",
179+
"endpoint = \"https://models.inference.ai.azure.com\"\n",
180+
"model_name = \"Llama-3.2-90B-Vision-Instruct\"\n",
181+
"\n",
182+
"client = ChatCompletionsClient(\n",
183+
" endpoint=endpoint,\n",
184+
" credential=AzureKeyCredential(token),\n",
185+
")\n",
186+
"\n",
187+
"response = client.complete(\n",
188+
" messages=[\n",
189+
" SystemMessage(\n",
190+
" content=\"You are a helpful assistant that describes images in details.\"\n",
191+
" ),\n",
192+
" UserMessage(\n",
193+
" content=[\n",
194+
" TextContentItem(text=\"What's in this image?\"),\n",
195+
" ImageContentItem(\n",
196+
" image_url=ImageUrl.load(\n",
197+
" image_file=\"sample.jpg\",\n",
198+
" image_format=\"jpg\",\n",
199+
" detail=ImageDetailLevel.LOW)\n",
200+
" ),\n",
201+
" ],\n",
202+
" ),\n",
203+
" ],\n",
204+
" model=model_name,\n",
205+
")\n",
206+
"\n",
207+
"print(response.choices[0].message.content)"
208+
]
209+
},
210+
{
211+
"cell_type": "markdown",
212+
"metadata": {},
213+
"source": [
214+
"## Learning does not stop here, continue the Journey\n",
215+
"\n",
216+
"After completing this lesson, check out our [Generative AI Learning collection](https://aka.ms/genai-collection?WT.mc_id=academic-105485-koreyst) to continue leveling up your Generative AI knowledge!"
217+
]
218+
}
219+
],
220+
"metadata": {
221+
"kernelspec": {
222+
"display_name": "Python 3",
223+
"language": "python",
224+
"name": "python3"
225+
},
226+
"language_info": {
227+
"codemirror_mode": {
228+
"name": "ipython",
229+
"version": 3
230+
},
231+
"file_extension": ".py",
232+
"mimetype": "text/x-python",
233+
"name": "python",
234+
"nbconvert_exporter": "python",
235+
"pygments_lexer": "ipython3",
236+
"version": "3.12.1"
237+
}
238+
},
239+
"nbformat": 4,
240+
"nbformat_minor": 2
241+
}

‎21-meta/python/sample.jpg‎

2.27 MB
Loading

0 commit comments

Comments
 (0)