diff options
| author | Tor Andersson <tor@ccxvii.net> | 2023-11-06 18:34:45 +0100 |
|---|---|---|
| committer | Tor Andersson <tor@ccxvii.net> | 2023-11-06 18:49:53 +0100 |
| commit | ce732ca77bc0cbef6c053ec6095e81f2c624a327 (patch) | |
| tree | 921f0865af449d58b48bddc70e05bc02d7dd5d53 /cards100/states_11.jpg | |
| parent | c31e8c8052386de10ef85626f508d608c6a55df1 (diff) | |
| download | votes-for-women-ce732ca77bc0cbef6c053ec6095e81f2c624a327.tar.gz | |
Add tokenizer for gencode to allow spaces in lists.
Tokens must be separated by spaces.
If a token contains a ", `, or ' it may contain spaces within the quotes.
If a token contains a (, {, or [ it may contain spaces within the parens.
Thus, the string "foo(a, b, c)" is treated as one token, as is also
the string "{ hello(a, b, c); bar(); return ['x', 'y z'] }"
Diffstat (limited to 'cards100/states_11.jpg')
0 files changed, 0 insertions, 0 deletions