I have run into an issue with spaces when using regex replacement. Not entirely sure if this is expansion related or there is some method to properly preserve the spaces.
The current workflow is that I search for [longer] (including brackets) and want to append something after longer (but inside the brackets still - e.g. [longer~and~longer]. Because I never know what that stuff could be (due to multiple iterations of appending), I need to extract the match first, remove the final ], and then append the new stuff plus a replacement ].
This workflow is fine (and appears in the MWE) and retains any space(s) in the
original token list. However the replacement token list, when updated into the original token list (via another regex replace) gets all of its spaces gobbled up. What I would expect as [longer~and~longer] ends up as [longerandlonger].
What do I need to add/change in this MWE to retain the spaces from the original regex extraction?
I found a workaround for this problem... but it is a hack that I think is the wrong approach. Before the final \regex_replace_once:nVN I can do a replace all on the updated token list (\l_update_stuff_tl) converting \s to something else which is unlikely to appear every (such as ~!@#$%^&()). Then, following the final \regex_replace_once:nVN I do another replace all on that unlikely token list by replacing them with \.
While this hack works, I am looking for more correct way to handle this. This workaround also has the drawback that a user, however unlikely, may actually enter the string and it would cause a probable error or lots of confusion.
Edit: Changed \l_some_stuff_tl to \g_some_stuff_tl because setting its value should persist outside the command.