Access to a Multimodal Language/Sensor Model for Data Analysis
Overview
This service allows the user to upload their own dataset of sensor or general time-series data, and converse about it in natural language. One model for wearable motion data and one for time series in general are provided. The first one can answer general questions about the data (e.g., where is the motion most intense) as well as questions about human activities (e.g., what activity is being performed in a specific interval). The second one can answer various questions about time series data (e.g., are there anomalies in the data, what is the expected value at a specific point in the future). Their capabilities are beyond the capabilities of general large language models, as they are specifically pre-trained to have a strong semantic understanding of the signals, paired with NLP capabilities. The service is based on models trained on aligned sensor or time series and text data. For wearable motion data, the LLaSA model is used, a 13-billion-parameter model trained on accelerometer and gyroscope data from various wearable devices. For general time series, the ChatTS model is used, based on the 16-billion parameter QWen open large language model.
Outcome
The user can converse about their data interactively in natural language, gaining insights into sensor readings, activity recognition, anomaly detection, and time-series forecasting.
Service Card
| Category | AI development and deployment |
| Domain | Sensing, Data Science |
| Vertical | Health and Biotechnology |
| Target users | Engineers and Researchers Working with Sensor Data |
| Service type | Models, Chat User Interface |
| Outputs | Interactive chat interface for sensor and time-series data analysis. |
| Requirements | Requires NVIDIA GPU node. |
| Limitations | |
| Availability | Q3/2027 |
| Licensing | Apache, MIT |
| Visual content | assets/service-visuals/multimodal-sensors-service-h.png |
| Visual caption |