Data Caching FAQs

Explore data caching FAQs to understand caching vs. ingestion, 'Copy to System,' Fast Query Engine, and performance optimization tips.

This page contains a set of FAQs focused on the nuances of data caching within the Tellius platform.

1. How is data caching different from data ingestion in Tellius memory?

Data caching involves storing data in memory for quick access, ensuring faster response times for repeated queries. On the other hand, data ingestion refers to the process of bringing in data on demand, which may not be immediately stored in memory. While both processes are integral to Tellius, caching provides a performance advantage by reducing the need to repeatedly ingest data.

2. What happens if I disable data caching from these settings?

Disabling data caching can lead to performance issues. Each time you publish a dataset, create a Business View, or run Insights, Tellius will fetch the entire data from the source, apply necessary transformations, and then continue with subsequent operations. This can significantly slow down performance and exert undue pressure on the data source.

3. How is the "Copy to System" option different from data caching when creating a new scripted dataset or during data ingestion?

"Copy to System" refers to persisting data to the disk rather than just in memory. When this option is enabled, a copy of the data is saved on the disk. This ensures that even if the service restarts (which might clear the memory cache), the system doesn't have to retrieve data from the source again.

4. Are "Copy to Disk" and "Copy to System" the same?

Yes, "Copy to Disk" and "Copy to System" are synonymous. They might be represented differently in Settings and the Load page, but they perform the same function of saving data to disk.

5. How does enabling caching for insights and ML models help?

Caching saves datasets for faster access. However, Business Views are treated differently. Since both Insights and ML models use Business Views and go through the data multiple times, enabling caching makes these processes run much faster. Without caching, things could slow down noticeably.

6. What does the "Save to Fast Query Engine" setting mean for a dataset with BV enabled?

The "Save to Fast Query Engine" (FQE) setting determines whether a Business View should be saved to FQE by default. Unless there are specific issues with it, it is always recommended to save to FQE to ensure faster query performance.

It's important to understand that some of these settings might evolve over time based on system upgrades and improvements. For instance, the option for "Copy to System" has been superseded in the 4.3 release with the introduction of the "Permanent Data Storage" feature, which always saves data to disk irrespective of user selection.