Skip to content

Add support for parsing AUTO, HETERO and MULTI from json config #605

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 2, 2025
Merged

Add support for parsing AUTO, HETERO and MULTI from json config #605

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
41ecf31
Add support for parsing AUTO, HETERO and MULTI from json config
preetha-intel Mar 6, 2025
1c950b1
Fix lint issues
preetha-intel Mar 10, 2025
2d4d500
Address review comments
preetha-intel Mar 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions onnxruntime/core/providers/openvino/backends/basic_backend.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
// Licensed under the MIT License

#include <map>
#include <unordered_set>

#include <string>
#include <memory>
#include <sstream>
Expand Down Expand Up @@ -222,6 +224,15 @@ void BasicBackend::PopulateConfigValue(ov::AnyMap& device_config) {
}
}
}
auto find_device_type_mode = [&](const std::string& device_type) -> std::string {
std::string device_mode = "";
auto delimiter_pos = device_type.find(':');
if (delimiter_pos != std::string::npos) {
std::stringstream str_stream(device_type.substr(0, delimiter_pos));
std::getline(str_stream, device_mode, ',');
}
return device_mode;
};

// Parse device types like "AUTO:CPU,GPU" and extract individual devices
auto parse_individual_devices = [&](const std::string& device_type) -> std::vector<std::string> {
Expand Down Expand Up @@ -270,8 +281,14 @@ void BasicBackend::PopulateConfigValue(ov::AnyMap& device_config) {
if (session_context_.device_type.find("AUTO") == 0 ||
session_context_.device_type.find("HETERO") == 0 ||
session_context_.device_type.find("MULTI") == 0) {
//// Parse to get the device mode (e.g., "AUTO:CPU,GPU" -> "AUTO")
std::unordered_set<std::string> supported_mode = {"AUTO", "HETERO", "MULTI"};
auto device_mode = find_device_type_mode(session_context_.device_type);
ORT_ENFORCE(supported_mode.find(device_mode)!=supported_mode.end(), " Invalid device mode is passed : " , session_context_.device_type);
// Parse individual devices (e.g., "AUTO:CPU,GPU" -> ["CPU", "GPU"])
auto individual_devices = parse_individual_devices(session_context_.device_type);
if (!device_mode.empty()) individual_devices.emplace_back(device_mode);

// Set properties only for individual devices (e.g., "CPU", "GPU")
for (const std::string& device : individual_devices) {
if (target_config.count(device)) {
Expand Down
6 changes: 4 additions & 2 deletions onnxruntime/core/providers/openvino/openvino_provider_factory.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
// Licensed under the MIT License

#include <map>
#include <set>

#include <utility>
#include "core/providers/shared_library/provider_api.h"
#include "core/providers/openvino/openvino_provider_factory.h"
Expand Down Expand Up @@ -216,9 +218,9 @@ struct OpenVINO_Provider : Provider {

for (auto& [key, value] : json_config.items()) {
ov::AnyMap inner_map;

std::set<std::string> valid_ov_devices = {"CPU", "GPU", "NPU", "AUTO", "HETERO", "MULTI"};
// Ensure the key is one of "CPU", "GPU", or "NPU"
if (key != "CPU" && key != "GPU" && key != "NPU") {
if (valid_ov_devices.find(key) == valid_ov_devices.end()) {
LOGS_DEFAULT(WARNING) << "Unsupported device key: " << key << ". Skipping entry.\n";
continue;
}
Expand Down
Loading