Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,13 +157,13 @@ The Foundry Local SDK makes it easy to integrate local AI models into your appli
new() { Role = "user", Content = "What is the golden ratio?" }
};

await foreach (var chunk in chatClient.CompleteChatStreamingAsync(messages))
await foreach (var chunk in chatClient.CompleteChatStreamingAsync(messages, CancellationToken.None))
{
Console.Write(chunk.Choices[0].Message.Content);
}

// Unload the model when done
await model.Unload();
await model.UnloadAsync();
```

</details>
Expand Down
2 changes: 2 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ Documentation for Foundry Local can be found in the following resources:
- SDK Reference:
- [C# SDK Reference](../sdk/cs/README.md): This documentation provides detailed information about the C# SDK for Foundry Local, including API references, usage examples, and best practices for integrating Foundry Local into your applications.
- [JavaScript SDK Reference](../sdk/js/README.md): This documentation offers detailed information about the JavaScript SDK for Foundry Local, including API references, usage examples, and best practices for integrating Foundry Local into your web applications.
- [Python SDK Reference](../sdk/python/README.md): This documentation provides detailed information about the Python SDK for Foundry Local, including API references, usage examples, and best practices for integrating Foundry Local into your Python applications.
- [Rust SDK Reference](../sdk/rust/README.md): This documentation provides detailed information about the Rust SDK for Foundry Local, including API references, usage examples, and best practices for integrating Foundry Local into your Rust applications.
- [Foundry Local Lab](https://github.com/Microsoft-foundry/foundry-local-lab): This GitHub repository contains a lab designed to help you learn how to use Foundry Local effectively. It includes hands-on exercises, sample code, and step-by-step instructions to guide you through the process of setting up and using Foundry Local in various scenarios.

## Supported Capabilities
Expand Down
12 changes: 6 additions & 6 deletions sdk/cs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ await model.LoadAsync();
var chatClient = await model.GetChatClientAsync();
var response = await chatClient.CompleteChatAsync(new[]
{
ChatMessage.FromUser("Why is the sky blue?")
new ChatMessage { Role = "user", Content = "Why is the sky blue?" }
});

Console.WriteLine(response.Choices![0].Message.Content);
Expand Down Expand Up @@ -159,7 +159,7 @@ var catalog = await FoundryLocalManager.Instance.GetCatalogAsync();
// List all available models
var models = await catalog.ListModelsAsync();
foreach (var m in models)
Console.WriteLine($"{m.Alias} — {m.SelectedVariant.Info.DisplayName}");
Console.WriteLine($"{m.Alias} — {m.Info.DisplayName}");

// Get a specific model by alias
var model = await catalog.GetModelAsync("phi-3.5-mini")
Expand Down Expand Up @@ -214,8 +214,8 @@ var chatClient = await model.GetChatClientAsync();

var response = await chatClient.CompleteChatAsync(new[]
{
ChatMessage.FromSystem("You are a helpful assistant."),
ChatMessage.FromUser("Explain async/await in C#.")
new ChatMessage { Role = "system", Content = "You are a helpful assistant." },
new ChatMessage { Role = "user", Content = "Explain async/await in C#." }
});

Console.WriteLine(response.Choices![0].Message.Content);
Expand All @@ -229,9 +229,9 @@ Use `IAsyncEnumerable` for token-by-token output:
using var cts = new CancellationTokenSource();

await foreach (var chunk in chatClient.CompleteChatStreamingAsync(
new[] { ChatMessage.FromUser("Write a haiku about .NET") }, cts.Token))
new[] { new ChatMessage { Role = "user", Content = "Write a haiku about .NET" } }, cts.Token))
{
Console.Write(chunk.Choices?[0]?.Delta?.Content);
Console.Write(chunk.Choices?[0]?.Message?.Content);
}
```

Expand Down
4 changes: 2 additions & 2 deletions sdk/js/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ console.log('\nTesting streaming completion...');
for await (const chunk of chatClient.completeStreamingChat(
[{ role: 'user', content: 'Write a short poem about programming.' }]
)) {
const content = chunk.choices?.[0]?.message?.content;
const content = chunk.choices?.[0]?.delta?.content;
if (content) {
process.stdout.write(content);
}
Expand Down Expand Up @@ -198,7 +198,7 @@ For real-time output, use streaming:
for await (const chunk of chatClient.completeStreamingChat(
[{ role: 'user', content: 'Write a short poem about programming.' }]
)) {
const content = chunk.choices?.[0]?.message?.content;
const content = chunk.choices?.[0]?.delta?.content;
if (content) {
process.stdout.write(content);
}
Expand Down
9 changes: 3 additions & 6 deletions sdk/python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,12 +232,9 @@ print(result.choices[0].message.content) # "42"
# Streaming chat
messages = [{"role": "user", "content": "Tell me a joke"}]

def on_chunk(chunk):
delta = chunk.choices[0].delta
if delta and delta.content:
print(delta.content, end="", flush=True)

client.complete_streaming_chat(messages, on_chunk)
for chunk in client.complete_streaming_chat(messages):
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)

# Unload when done
model.unload()
Expand Down
5 changes: 3 additions & 2 deletions sdk/rust/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ let loaded = catalog.get_loaded_models().await?;

### Model Lifecycle

Each model may have multiple variants (different quantizations, hardware targets). The SDK auto-selects the best available variant, preferring cached versions. All models implement the `IModel` trait.
Each model may have multiple variants (different quantizations, hardware targets). The SDK auto-selects the best available variant, preferring cached versions. All models are represented by the `Model` type.

```rust
let model = catalog.get_model("phi-3.5-mini").await?;
Expand Down Expand Up @@ -445,6 +445,7 @@ match manager.catalog().get_model("nonexistent").await {
| `Serialization(serde_json::Error)` | JSON serialization/deserialization failed |
| `Validation { reason }` | A validation check on user-supplied input failed |
| `Io(std::io::Error)` | An I/O error occurred |
| `Internal { reason }` | An internal SDK error (e.g. poisoned lock) |

## Configuration

Expand Down Expand Up @@ -520,4 +521,4 @@ cargo run -p native-chat-completions

## License

MIT — see [LICENSE](../../LICENSE) for details.
Microsoft Software License Terms — see [LICENSE](../../LICENSE) for details.
Loading