Deal with non-ascii messages from serial port

[jitterjames]

Sorry. I would have done this the normal way but I can't get GIT to work in Visual Studio and I'm not sure how to do it by the web page.

I propose a change to deal with incoming messages that send non-ascii codes in the string. Currently they come through as unreadable characters and/or truncate the string prematurely.

I can email the updated .cs file but I wanted to highlight the proposed change.

Thanks:

INFO:

in the file: SerialPortActions.cs

Add this method:

private static String decodeHexMessage(string message)
        {
            StringBuilder sb = new StringBuilder();
            foreach (byte b in message)
            {  
                try
                {
                    if (b > 33 && b < 127)
                    {
                        sb.Append((char)b);                       
                    }

                    else
                    {
                        sb.Append(@"\x" + b.ToString("X2"));
                    }
                }
                catch (Exception err)
                {
                    sb.Append("[error:" + err.Message + "]");
                }
            }
            
            return sb.ToString();
        }

and modify the port.DataReceived method :

port.DataReceived += delegate {
                // added try catch to fix crashing when closing VC
                try {
                    Plugin.PortMessage = port.ReadExisting();
                    if (!string.IsNullOrWhiteSpace(Plugin.PortMessage)) {
                        if (PluginOptions.GenEventOnReceive) {
                            Plugin.HostInstance.triggerEvent(
                                "Serial.Received", new List<string> {
                                    port.PortName,
                                   decodeHexMessage( Plugin.PortMessage)
                                }
                            );
                        }
                    }
                }
                catch {
                    //
                }
            };

[F1uctus]

Thanks for that solution, James, but I found one problem.
That function transforms non-ascii letters to "\x" sequence too.
Can you check my solution in latest commit at first, will it work correctly?
I recently tried to change encoding through SerialPort.Encoding = Encoding.UTF8,
and it fixed viewing of non-ascii letters.

[jitterjames]

The problem is that there are other non-readable characters as well that are not UTF either. Things like null, escape, linefeed etc.

We want to transform these non-readable characters to the \x## format.

So anything with a byte value of less than 33.

I am also catching characters with a byte value of greater than 127 so maybe this is the problem for other UTF values?

If some characters are being converted to the \x## format when they should not be, or if they are converted to the wrong character, can you please give me a specific example of what the characters should be and what you are getting instead? Thanks.

[jitterjames]

Maybe this will help us...
https://stackoverflow.com/questions/40564692/c-sharp-regex-to-remove-non-printable-characters-and-control-characters-in-a

[jitterjames]

except we want to replace with the \x## pattern instead of empty string...

[F1uctus]

For example, any cyrillic symbols are encoded to "\x##",
(and probably any non-latin alphabet too)

[F1uctus]

(At first line there are cyrillic symbols in braces)

[F1uctus]

(?@825B)

[F1uctus]

I just tested regex, it works for newlines, not affecting the letters.
but we need to replace each Regex.Match.Value by \x ...

[F1uctus]

Or make function simpler:

private static String decodeHexMessage(string message) {
    StringBuilder sb = new StringBuilder();
    foreach (char c in message) {
        try {
            if (char.IsControl(c)) {
                sb.Append(@"\x" + ((byte)c).ToString("X2"));
            }
            else {
                sb.Append(c);
            }
        }
        catch (Exception err) {
            sb.Append("[error:" + err.Message + "]");
        }
    }

    return sb.ToString();
}

[F1uctus]

It works, I think)

[jitterjames]

Looks great. I did not know about the char.IsControl method.

I'm sure we could use RegEx as well but this is fine and speed is not an issue.

I will try to do some testing on it.

I know I have been very slow but I will be doing a new release of VC soon with a bunch of new stuff including this plugin. Thanks for your help.
👯‍♂️

[F1uctus]

Maybe I'll optimize that method a bit before testing?
It's better to replace foreach by for.
what is try-catch used for? If it's reason is explicit cast, is it better to replace it with unchecked expression, like that:

private static string DecodeHexMessage(string message) {
    var sb = new StringBuilder();
    for (var i = 0; i < message.Length; i++) {
        char c = message[i];
        if (char.IsControl(c)) {
            sb.Append(@"\x")
              .Append(unchecked((byte)c).ToString("X2"));
        }
        else {
            sb.Append(c);
        }
    }

    return sb.ToString();
}

And that method 'll be more powerful

[F1uctus]

And I forgot to mention, in port.DataReceived I moved DecodeHexMessage to Plugin.PortMessage initialization, or there will be bugs.

https://github.com/F1uctus/Serial/blob/239796e03345981ae0753bec52fcce4b53d8c1ff/Source/SerialPortActions.cs#L91

[jitterjames]

Yes the try catch was a leftover from something else with testing. It would be good to have a try catch around the whole method but not on each iteration.

I don't think for next is better in this case but I don't mind if you want to change it. No one will see any difference in speed and for each is easier to read and less likely to have a coding mistake which is why I prefer it.

[F1uctus]

Ok, you convinced me, I know that's unpredictably dangerous to just change some letters in code :)
I added try-catch for whole method.