Conversation
Adds a "Speak text" PostGrabAction that reads OCR output aloud using Windows.Media.SpeechSynthesis. Speaks the final transformed text after all other actions run in FullscreenGrab; in GrabFrame, speaks only when the captured text changes to avoid repeating on every OCR tick. - ITtsEngine interface for future engine swappability - WindowsSpeechEngine wraps WinRT SpeechSynthesizer + MediaPlayer - TtsService queues utterances via Channel<string> so new text waits rather than interrupting in-progress speech - TtsSpeakWordLimit setting (default 100) truncates long captures; configurable in General Settings - PostGrabActionManager: new SpeakText_Click action at order 6.6 - GrabFrame: speaks on text change when action is checked - Tests: count updated to 6, "Speak text" assertion, fire-and-forget test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds SpeakInsteadOfToast setting: when enabled, Text Grab speaks the captured text aloud rather than chiming a notification toast. Defaults to enabled. Includes: - Stop-speaking button in GrabFrame to cancel playback mid-sentence - TTS drain-on-shutdown fix so the queue empties cleanly on exit - Stop() method on TtsService to flush the queue and cancel speech
Adds a dedicated Voice Output page in Settings containing: - Voice picker populated from SpeechSynthesizer.AllVoices - Word limit setting - Speak-instead-of-notification toggle (moved from General Settings) - Preview button to hear the selected voice WindowsSpeechEngine now applies the saved TtsVoiceName before synthesising. TtsVoiceName setting added (empty = system default).
Introduce a new user-scoped boolean setting, WindowsAiDescriptionEnabled, to control Windows AI description features. The setting defaults to False and is added to App.config, Settings.settings, and Settings.Designer.cs.
Added WindowsAiDescriptionLang implementing ILanguage for the new "Windows AI Description" language kind. Introduced GeneratedOcrLinesWords, GeneratedOcrLine, and GeneratedOcrWord classes to generate OCR line/word models from text and bounding boxes. Updated HistoryInfo to support the new language kind.
Introduce WindowsAiDescriptionLang for image captioning via Windows AI. Update LanguageKind, LanguageService, and CaptureLanguageUtilities to support selection and caching. Route OCR/image description requests through new async methods in OcrUtilities and WindowsAiUtilities. Add device capability checks, content moderation, and async polling for image description. Exclude WindowsAiDescriptionLang from table output. Improve resource cleanup and error handling.
Improve test coverage for WindowsAiDescriptionLang integration: - Save/restore WindowsAiDescriptionEnabled for isolation - Test inclusion/exclusion in CaptureLanguageUtilities - Assert correct tag/kind handling in LanguageService - Verify fallback and HistoryInfo behavior with new language
Introduce WindowsAiDescriptionLang support in LanguagePicker and filter it from the language list. Add a toggle in LanguageSettings to enable/disable Windows AI Description as a language option, with explanatory text. Refactor settings initialization to use a unified loadingLanguageSettings flag and update event handlers accordingly. Implement methods to load and persist the new setting, and ensure language caches are invalidated when toggled.
Add the five Microsoft.Recognizers.Text.* packages (with BUILT-WITH
attribution) and a fixed BuiltInRecognizer catalog -- numbers, ordinals,
percentages, ages, currencies, dimensions, temperatures, dates/times, phone
numbers, emails, URLs, IPs, GUIDs, and booleans -- plus RecognizerExecutor for
matching and applying them (resolved value or matched text, first/last/all/nth).
GrabTemplate gains {r:Name:mode[:value|text][:separator]} placeholders
(TemplateRecognizerMatch), resolved by GrabTemplateExecutor alongside the
existing {p:} regex patterns.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Introduce PatternItem -- a single user-facing "pattern" that wraps either a saved
StoredRegex or a built-in recognizer -- and PatternExecutor, which dispatches
HasMatch/GetMatches/Apply to the regex or recognizer engine. This lets every UI
surface treat the two kinds uniformly. PatternItem also builds the grouped
("Saved Patterns" / "Smart Patterns") choice lists used by the combo pickers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Expose saved regexes and built-in recognizers together everywhere, split into
"Saved Patterns" and "Smart Patterns" subsections:
- Template inline {...} picker: one Patterns group (InlinePickerItem.Kind drives
whether selection emits {p:} or {r:}); {p:}/{r:} syntax unchanged.
- Edit Text Window: "Apply Pattern" / "Apply Pattern Per Line" list both kinds and
transform in place via PatternExecutor.
- Grab Frame: "Search by Pattern" submenu (saved regexes searchable for the first
time); Quick Simple Lookup and Find & Replace use a unified Patterns combo
(saved regex loads into the find box so replace still works; recognizers are
find-only).
Combo pickers use PatternChoice data items with DisplayMemberPath and
non-selectable subsection headers, fixing the selection-box width/height clipping
that came from using ComboBoxItem objects as data. User-visible "Recognizer"
wording is now "Pattern".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A reusable search input used across the search surfaces: a text box with a placeholder and clear button, a removable smart-pattern chip, a WPF-UI-style accent focus underline on the whole control, a regex toggle rendered as a split-button whose chevron opens a scrollable pattern flyout (saved regexes + smart recognizers), and an optional exact-match toggle. Adds the ToggleSplitSegment and SplitSegmentButton styles the split-button uses. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… Frame Replaces each window's hand-built search box, regex control, and pattern picker with the shared SearchBar and routes searching through its SearchChanged event: - Quick Simple Lookup: swaps its text box, regex toggle, and pattern combo for the control; regex validity now flows through SetRegexValidity. - Find & Replace: the find box becomes a SearchBar; the "Match Exactly" switch moves into the box as a toggle (wired via ExactMatchChanged), and a picked recognizer can be narrowed by typed text. Adds SetFindText for external callers. - Grab Frame: the regex, exact-match, and "search by pattern" options that were hidden in the text box context menu become visible controls, and the status bar grows to fit the taller control. Retargets RegexManager and EditTextWindow to SetFindText and drops the now-unused PatternItem combo helpers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Registers a Ctrl+Shift+F key gesture in the Edit Text Window that opens Find & Replace, and shows the shortcut on the menu item. Ctrl+F stays mapped to Fullscreen Grab. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Expanded GetDefaultPatterns summary to clarify exclusion of patterns already handled by built-in recognizers (emails, phones, URLs, IPs, GUIDs, dates, times, currency, numbers). Removed these patterns from the defaults, retaining only credit card, hex color, SSN, and US zip code regexes.
Recognizers: unify saved regexes and smart recognizers as one "Patterns" concept
Introduce a new user-scoped boolean setting, WindowsAiDescriptionEnabled, to control Windows AI description features. The setting defaults to False and is added to App.config, Settings.settings, and Settings.Designer.cs.
Added WindowsAiDescriptionLang implementing ILanguage for the new "Windows AI Description" language kind. Introduced GeneratedOcrLinesWords, GeneratedOcrLine, and GeneratedOcrWord classes to generate OCR line/word models from text and bounding boxes. Updated HistoryInfo to support the new language kind.
Introduce WindowsAiDescriptionLang for image captioning via Windows AI. Update LanguageKind, LanguageService, and CaptureLanguageUtilities to support selection and caching. Route OCR/image description requests through new async methods in OcrUtilities and WindowsAiUtilities. Add device capability checks, content moderation, and async polling for image description. Exclude WindowsAiDescriptionLang from table output. Improve resource cleanup and error handling.
Improve test coverage for WindowsAiDescriptionLang integration: - Save/restore WindowsAiDescriptionEnabled for isolation - Test inclusion/exclusion in CaptureLanguageUtilities - Assert correct tag/kind handling in LanguageService - Verify fallback and HistoryInfo behavior with new language
Introduce WindowsAiDescriptionLang support in LanguagePicker and filter it from the language list. Add a toggle in LanguageSettings to enable/disable Windows AI Description as a language option, with explanatory text. Refactor settings initialization to use a unified loadingLanguageSettings flag and update event handlers accordingly. Implement methods to load and persist the new setting, and ensure language caches are invalidated when toggled.
…-Grab into WinAI-description
Add overloads to GetTextDescriptionWithWinAI for System.Drawing.Bitmap input and CancellationToken support. Use AsTask to enable cancellation of DescribeAsync, remove manual polling, and improve exception handling to propagate OperationCanceledException and log other errors. Temp files are now cleaned up after use.
Introduce a new user flow for Windows AI image description in fullscreen grab mode. The overlay now freezes the selected region as a background, shows a spinner, and after 2 seconds offers Cancel/Re-grab options. If the AI description fails or is empty, a "Send to Grab Frame" option is provided. Refactor PreviousGrabWindow and FullscreenGrab to support this asynchronous, user-driven workflow and improve code organization.
Replace the flat black strip with a centered rounded panel using the app background, teal border, and drop shadow like the fullscreen grab toolbar. Buttons now use the TealColor style with icon + label content, and the bar slides up from the bottom edge with a fade on first reveal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Use C# collection expressions for English word array in WindowsAiUtilities - Return empty string (not error) if WinAI unavailable for image description - Ignore single clicks for AI Description in fullscreen grab, reset selection instead - Cancel grab if overlay closes without user choice to avoid orphaned tasks - Default TextContent to empty string if OCR result is null
Adds ShowSuccessAsync to PreviousGrabWindow for awaiting overlay close, ensuring text insertion occurs after focus is released. Refactors insertion logic in FullscreenGrab to prevent focus issues, threads cancellation tokens in WindowsAiUtilities, and cleans up redundant code.
Adds a ToggleButton to PreviousGrabWindow for sending OCR results to an Edit Text Window, with visibility and checked state managed during OCR. Introduces a SendToEditTextToggled event and syncs the toggle state across all fullscreen grab windows. Updates running/failed state logic to show or hide the toggle as appropriate.
WinAI Descriptions
Added permissions in settings.local.json to allow web access to app.produckmap.com, read from local pdm skill directories, and execute pdm-related PowerShell and Bash commands for story maps, types, UI elements, and API operations. This enables broader interaction with ProDuckMap data and tools.
Add SaveHiddenSmartPatternIds_FileBackedModeWritesBothStores unit test to verify that saving hidden smart pattern IDs updates both in-memory settings and the backing file when file-backed mode is enabled, and that loading retrieves the correct values from both sources.
Renamed the "Regex Manager" button to "Patterns Manager" in the UI. Enhanced PatternItem to support hiding built-in recognizer patterns, including an IsHidden property and PatternDisplay. Updated the recognizer constructor and GetAll method to handle hidden patterns, allowing the Patterns Manager to show and unhide them as needed.
Added persistence for HiddenSmartPatternIds in SettingsService, including file registration, caching, load/save methods, cache invalidation, and a cloning helper for safe list management.
Renamed all UI and code references from "Regex Manager" to "Patterns Manager". Updated the manager window to display both user-saved regexes and built-in recognizers in a unified, read-only DataGrid. Added the ability to hide/unhide recognizers, with hidden items shown faded and italicized. Introduced a PatternItem abstraction for unified selection and management. Disabled direct add/delete in the grid; patterns are now managed via dialogs. Improved test and explain features to support both pattern types and handle invalid patterns gracefully. Updated comments and tooltips to match new terminology.
Add a custom style for Wpf.Ui DropDownButton to unify colors and hover/pressed states, importing the wpfui namespace. Also reformat bindings in SelectionStyleComboBoxStyle.xaml and the ResourceDictionary tag in SettingsStyles.xaml for readability. No functional changes outside the new style.
- Animate MoreOptionsToggleButton chevron in FindAndReplaceWindow and add tooltip - Remove hardcoded Foreground and min row height in FindAndReplaceWindow - Change RegExIcon to use Foreground instead of IconColor; remove IconColor property and binding - Add PatternDropDownButtonStyle to SearchBar for better contrast in regex mode - Update SearchBar to use new style and Foreground binding for RegExIcon
Improve visual feedback for NextStepDropDownButton by updating background and foreground colors based on selection and hover state, including icon color. Always show Freeze button in GrabFrame unless image is static.
Extended HistoryInfo with source kind, path, and page index to track PDF documents. Added PdfDocument to LookupItemKind and updated LookupItem to assign kind and icon for PDFs. Improved distinction between PDF, edit window, and grab frame entries.
- Added PDF document history with separate limits and menu support - Refactored history logic to distinguish images and PDFs - Enforced strict size/count/dimension limits for .tggf files - Hardened Grab Frame file reading against resource exhaustion - Limited PDF page cache and pixel count in PdfDocumentRenderer - Improved diagnostics and history model for PDF support
Added tests for GrabFrame metadata, search, and alignment logic. Verified HistoryService PDF/grab separation and retention. Tested PdfDocumentRenderer clamping and invalid size handling. Minor import changes for test support.
Added "Recent PDFs" menu to EditTextWindow and GrabFrame for quick PDF access using HistoryService. Refactored history checks for clarity. Enhanced GrabFrame to open specific PDF pages from history and improved page restoration. Introduced GrabFrameSearchMatch/Unit records for more accurate search and selection, updating search logic and "Edit Matches in New Window" behavior. QuickSimpleLookup now displays and opens recent PDFs with removal support. Minor UI text tweaks included.
Added ReplaceFileAtomically to GrabFrameFileUtilities using File.Replace for atomic saves, with fallback to File.Move. Ensured temp/backup cleanup in finally block. Added two unit tests: one for atomic replacement success, one for failure with locked file preserving original.
Added unit test to verify correct handling of rowspan and colspan in HTML tables. Introduced MaxHtmlTableSpan constant to cap span values. Refactored cell placement logic in ConvertHtmlToTabSeparated to prevent colspans from overwriting rowspans, using new FindNextFreeColumnRange helper. Updated attribute parsing to clamp span values.
Added test for ordered list start numbers. Set and serialize WpfList.StartIndex based on Markdown input. Introduced GetOrderedListStart helper to extract start number, defaulting to 1 if missing or invalid.
FindResult now stores RawText for original match content. All match creation sets RawText. Copy Matches uses RawText via GetMatchTextForEditing to preserve formatting. Copy command logic updated to allow copying when a pattern is selected. Added unit test to verify whitespace and line break preservation.
Adds EditorModeChanged event to EditTextWindow and updates FindAndReplaceWindow to refresh search results when editor mode changes, especially for spreadsheet mode. Refactors search source logic into ResolveSearchSourceText with new unit tests. Ensures search methods always use current editor content. Improves event handler cleanup to prevent memory leaks.
Refactored GrabFrame unfreeze logic to use a _freezeTransitionVersion integer, incremented on each freeze/unfreeze/cleanup, to track transition state. Updated FinishUnfreezeWithDiffAsync to check a transitionVersion parameter, ensuring only the latest unfreeze result is applied and preventing race conditions. Added static ShouldApplyUnfreezeResult to encapsulate result application logic. Set Opacity to 0 during unfreeze and restore to 1 only if the result is valid. Added GrabFrameUnfreezeTests to verify ShouldApplyUnfreezeResult behavior.
Refactored DisplayHdrInfo to enumerate all monitors and expose GetAll/GetForRegion. Updated HdrScreenCapture to handle regions spanning multiple HDR monitors, with BuildCaptureSegments and HdrCaptureSegment for mapping and compositing. Implemented CaptureCompositeRegion to blend HDR/SDR content. Added HdrScreenCaptureTests for cross-monitor region validation.
Refactored TtsService to use a dedicated _isBusy flag for accurate busy/idle state tracking and event publishing. Centralized idle state logic in PublishIdleIfDrained for correct sequencing and thread safety. Updated Speak and Stop methods to prevent double-counting or missed events. In GrabFrame.xaml.cs, improved SpeakToggleButton_Click to only speak current frame text when toggling on, using the new ShouldSpeakCurrentFrameWhenEnabled helper. Added unit tests for GrabFrame toggle logic and TtsService to ensure no spurious idle events when speech is queued during drain callbacks.
Expanded OcrTests with cases for IsWrappedParagraph to ensure lines on the same visual row are not merged and embedded line breaks are removed. Updated OcrUtilities to use MakeStringSingleLine for DisplayText and SingleLineText. Refined IsWrappedParagraph logic to require a vertical gap before merging lines. Ensured GrabFrame clipboard output is always single-line per OCR line.
Updated NuGet package versions in Text-Grab.csproj and Tests.csproj: - Magick.NET-Q16-AnyCPU: 14.14.0 → 14.15.0 - Magick.NET.SystemDrawing: 8.0.23 → 8.0.24 - Magick.NET.SystemWindowsMedia: 8.0.23 → 8.0.24 - Microsoft.WindowsAppSDK.AI: 2.2.3 → 2.3.4 - Microsoft.WindowsAppSDK.Foundation: 2.1.0 → 2.3.5 - Microsoft.WindowsAppSDK.Runtime: 2.2.0 → 2.3.1 - Microsoft.WindowsAppSDK.WinUI: 2.2.1 → 2.3.0 - Microsoft.NET.Test.Sdk (Tests.csproj): 18.7.0 → 18.8.1 No code or logic changes; only dependency updates.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Classification
New feature and major UI/UX enhancement: unified pattern/recognizer search, voice output, HDR capture correction, and improved settings.
PR Summary
This PR introduces unified "Smart Pattern" (recognizer) and regex search across the app, adds voice output (TTS) features, supports HDR-aware screen capture, and modernizes the settings UI. It also improves Grab Frame and Quick Simple Lookup workflows and adds .tggf file support.