Jonas Rickert aa21ce4a79 [mlir] Do not set lastToken in AsmParser's resetToken function and add a unit test for AsmParsers's locations (#105529) ...

This changes the function `resetToken` to not update `lastToken`.

The member `lastToken` is the last token that was consumed by the
parser.
Resetting the lexer position to a different position does not cause any
token to be consumed, so `lastToken` should not be updated.
Setting it to `curToken` can cause the scopeLoc.end location of
`OperationDefinition `to be off-by-one, pointing to the
first token after the operation.

An example for an operation for which the scopeLoc.end location was
wrong before is:
```
%0 = torch.vtensor.literal(dense_resource<__elided__> : tensor<768xbf16>) : !torch.vtensor<[768],bf16>
```
Here the scope end loc always pointed to the next token

This also adds a test for the Locations of `OperationDefinitions`.
Without the change to `resetToken` the test failes, with the scope end
location for `llvm.mlir.undef` pointing to the `func.return` in the next
line

2024-09-09 11:48:13 +02:00

6.5 KiB

Raw Blame History

View Raw