So.... I found a way to make this work (for my case anyway) with a few changes to Lexer.cs
I change ScanNextToken line 158 to use a more restrictive custom IsLetter(ch) function instead of Char.IsLetter(ch). Using the debugger I observed some really strange values (in the high 200's for example) being returned as true by Char.IsLetter() and I theorized that maybe these should be treated as not letters.
Code:
if (IsLetter(ch))
return this.symbol = ScanKeyword();
which is implemented as follows
Code:
private bool IsLetter(char c)
{
if (c == ' ') return true;
if (c >= 'a' && c <= 'z') return true;
if (c >= 'A' && c <= 'Z') return true;
return false;
}
Then I changed ScanKeyword() in two spots....
Line 307 I replaced Char.IsLetter(ch) with the same custom IsLetter(ch) method.
Code:
while (true)
{
if (IsLetter(ch))
{
this.token.Append(ch);
and line 317 I changed the switch statement to handle errant whitespace characters
Code:
// Check known tokens
switch (this.token.ToString().Trim())
{
Like I say. This worked for me, but there is a very real chance (almost a certainty) that I screwed something up by doing this. I would love feedback from someone who is a little more conversant in the PDF format than I am to tell me what I may have messed up with these changes.
Thanks...
RKM
Login
Register
FAQ
Search
