Tokenizer

I wanted to write a quick post about a module that I recently put up on the marketplace. The module is called Tokenizer.

The scenario that this module addresses is the need to have a centralized place for content. This could be a specific word, a phrase, a product name, a disclaimer, or a date. Something that needs to be the same across many pages on your site.

The module works by looking for a pattern in rendered fields. The pattern is %%([a-zA-Z0-9]+)%%. When a match is found, the token is looked up in a dictionary. If a match is found in the dictionary, the token is replaced with the value.

The module also comes with a tool for the HTML editor. This allows content authors to insert tokens without having to know the exact syntax. They can select from a list of available tokens and insert them into their content.

The code is fully open source and available on the marketplace. The concept could be expanded exponentially. You could have tokens pull data from third party sources or implement complex logic via query strings or JavaScript injection.