For the complete documentation index, see llms.txt. This page is also available as Markdown.

Storage Architecture & File Specification

The following section outlines the directory structure and file types within the user project. The root folder of these files and folders are at /automl.

⚠️ Note: The /automl directory stores conversation logs and artifacts that define your session history. While these files are accessible, users should treat this directory as read-only and avoid modifying its contents to ensure data integrity and proper functionality of AutoML Assistant.

Data Cache (Vector & Graph Stores)

  • Cached Data Index (cache/weaviate/)

    • Type: Tarball / Dataset

    • Description: Compressed Weaviate vector database indices used for high-performance semantic search across the metadata of datasets.

  • Cached Dataset Lexical Strand (cache/surrealdb/datasets/)

    • Type: Tarball / Dataset

    • Description: Lexical and relational metadata for specific datasets stored in SurrealDB format to support structured queries.

  • Cached Cohort Lexical Strand (cache/surrealdb/cohorts/)

    • Type: Tarball / Cohort

    • Description: Specific relational data and indices for defined patient or sample cohorts.

User & Session Management

  • User Profile Directory (users/user-[id]/)

    • Type: Directory / User

    • Description: The root container for all user-specific settings, preferences, and session history.

  • Settings (settings.record)

    • Type: Record / User

    • Description: A binary or structured record file containing user-specific application preferences and interface configurations.

  • Conversation-User Links (conversations-v2.record)

    • Type: Record / User

    • Description: A mapping file that associates specific conversation IDs with the user's profile, used for populating the user's chat history list.

Conversation Session Details

The below folders and data are under the users/user-[id]/ folder

Conversation Session (conversations/[conversation_uuid]/)

  • Type: Directory / Conversation

  • Description: A dedicated workspace for a single AI interaction session, containing all logs, plans, and generated artifacts.

Conversation History (history/chunk[nnn].record)

  • Type: Record / Conversation

  • Description: Segmented log files (chunks) containing the actual dialogue exchange between the user and the AI.

Artifact-Conversation Map (artifacts.record)

  • Type: Record / Conversation

  • Description: Metadata linking generated files or datasets to the specific point in the conversation where they were created.

Artifact Data (artifacts/[name].[file-format])

  • Type: File / Artifact

  • Description: The actual output files generated by the system during a session. The file format depends on the type of artifact:

    • Dataframe artifact: .parquet (Parquet tables)

    • Notebook artifact: .ipynb (Jupyter Notebooks)

    • Model artifact: .pkl (Pickled objects of a scikit-learn Estimator)

    • Chart artifact: .png (PNG images)

Plan-Conversation (plans.record)

  • Type: Record / Conversation

  • Description: A record of the execution steps or "thought process" the agent intended to follow to fulfill user requests.

Session Log (log.txt)

  • Type: File / Conversation

  • Description: A plaintext execution log used for debugging and tracking backend system calls during the conversation.

Last updated