CSV streaming
While most use cases for Template involve working with Word or Excel document,
Templater also works with plain text/csv formats.
When working in such mode each row is considered a context,
which maps closely to working with CSV files.
The main benefit for using Templater as CSV export comes from customization options which
are possible in those cases.
Same patterns used in Word and Excel can be applied to work with CSV files.
This opens up possibility for large exports since CSV is much more memory friendly than other two formats.
But even with much smaller memory footprint, to create really large documents
streaming needs to be utilized, otherwise all data must be in memory upfront which
is often not feasible.
If Templater is used in a certain way it can perform streaming and thus
flush output and reuse memory.
To perform streaming certain conditions must be met:
- Tags which are getting streamed must be last in the document (there can't be other tags before the ones which require streaming). If there are, they need to be processed first, so that the remaining tags can be streamed
- Resize must be called multiple times. First to create the extra context and then to resize the actual chunk which is getting processed
How it works
All tags before the ones which are streamed must be processed before streaming takes place.
- First by resizing the context (row) once (as shown by green rectangle)
- Then by processing (resizing + replacing) the original row again with the appropriate chunk (as shown by red rectangle)