3 Commits

Author	SHA1	Message	Date
Veronica K. B. Olsen	83169d9075	Updated tokenizer, and added html converter as a subclass	2019-05-07 18:47:55 +02:00
Veronica K. B. Olsen	e0936b5f33	Text tokenizer done.	2019-05-07 18:01:19 +02:00
Veronica K. B. Olsen	7166dfbb2e	Started adding framework for document tokenizer.	2019-05-05 23:16:53 +02:00