Tellius Engine: Comparison of In-Memory vs. Live Mode

Tellius offers two distinct modes for data querying and execution—In-memory mode and Live mode. Each mode is designed to optimize different types of queries and provide flexibility in how data is processed. Understanding the capabilities of these two modes is critical when working with large datasets or real-time analytics.

This document provides a detailed comparison between the In-memory mode and Live mode, with emphasis on how they interact with various functionalities of the Tellius engine. The table below outlines which features each mode supports and offers additional context on where each mode excels.

In-memory mode

In-memory mode loads data into the Tellius engine’s memory for processing, which allows for faster performance in most cases. This mode is ideal when working with cached or pre-aggregated data. It's especially useful for complex data analysis, machine learning (ML) tasks, and data transformations since all relevant data is held within Tellius for quick access.

Live mode

Live mode, on the other hand, executes queries directly against your data sources without requiring data to be ingested into Tellius beforehand. This mode provides real-time data access but comes with limitations, especially when dealing with transformations, ML-based analytics, or actions that require reading large amounts of raw data into the system. Live mode ensures you always work with the most current data, but complex operations that require substantial data processing are best done in-memory.

Key considerations

As a general rule, any functionality that relies on machine learning models or data transformations is only available in in-memory mode. This is because these operations require reading raw data into Tellius for analysis, which live mode does not support.

• Live mode excels when real-time data access is essential. It allows users to run queries directly on the source, ensuring the data is always up-to-date. However, this comes at the cost of speed and complexity, especially when dealing with machine learning tasks or extensive data transformations.

• In-memory mode provides better performance for complex queries since data is already ingested and processed. Live mode offers real-time insights but is limited in handling computationally intensive tasks.

Functionality breakdown

Module	Functionality	In-Memory	Live	Description
Search	Natural Language Search	✔	✔	Kaiya’s query optimization engine efficiently handles both in-memory and live queries.
	Assistant	✔	✔	Assistant works efficiently in both modes, leveraging cached data in-memory or real-time queries in live mode.
	Kaiya Search	✔	✔	Kaiya Search supports in-memory and live queries, adjusting to the most optimal execution path.
Explore (Vizpads)	All Vizpads functionalities	✔	✔	Both modes fully support interactive data exploration using Vizpads, with faster responses in in-memory mode.
Discover (Insights)	Segment Driver	✔	✘	Segment Driver uses ML models that require raw data ingestion, making it available only in in-memory mode.
	Trend Driver	✔	✔	Allows for real-time analysis and trends without needing ML-based transformations.
	Trend Driver (with Segments)	✔	✘	Trend analysis with segments requires raw data transformations, available only in in-memory mode.
	Comparison Driver	✔	✔	Enables comparative insights across datasets without requiring in-depth ML-based processing.
	Comparison Driver (with Segments)	✔	✘	Requires ML processing, available only in in-memory mode.
Data Preparation	Data Load	✔	✔	Data can be loaded into Tellius in both modes, though live mode operates on real-time data without caching.
	Dataset Metadata Management	✔	✔	Both modes allow metadata management, irrespective of data being cached or accessed live.
	Business View Creation	✔	✔	Users can create Business Views without needing extensive transformations.
	Data Transformation	✔	✘	Transformations need the raw data to be read into Tellius, so this is only supported in in-memory mode.
Feed	All Feed functionalities	✔	✔	Feeds are available in both modes, though underlying transformations will only work in in-memory mode.
Predict (ML)	Point n Click	✔	✘	ML models require raw data processing, only available in in-memory mode.
	Auto ML	✔	✘	Auto ML requires training on raw data, making it supported only in in-memory mode.