Here’s a simple and neat approach to using search engines I came up with today:
- Find a tokenizer to use for your search query. In this case our query is
online tokenizer - Add spaces between the tokens from the tokenizer output for your new query:
online tokenizer→online token izer - Potentially pull up new and previously unseen search results
online tokenizer
DuckDuckGo
Original query results:
Tokenized query results:
Some of the result differences:
I intend to study this more in depth eventually, if you want you can find my progress here: github.com/jerlendds/token-queries
to be continued…