This dataset group was generated by an Autonomous Driving (AD) simulation application based on a multi-core time-triggered scheduling system. They are primarily used for event and related attribute prediction in time-triggered systems. Each time we run the simulation, a new dataset is generated. Therefore, the name of the dataset corresponds to the time of its generation. The model in our paper is trained on the dataset from "2024.07.15 15h29m36s".
For a dataset, the most important issue is how to efficiently and quickly read the data. Therefore, in the beginning, I provide a simple Python code example to help you quickly retrieve the five sub-datasets and the event attribute name list. The event attribute names correspond one-to-one with the features in the event attribute dataset 'attr'. Next, I will provide a detailed explanation of our simulation and the dataset.
- Get Dataset Quickly
- Autonomous Driving Simulation Based On Multi-processor Time-triggered Scheduling System
First, let me introduce the directory of our dataset files. After extracting the dataset zip file, you will obtain a dataset folder with the following directory structure:
2024.07.15 15h29m36s.zip/
├── Data Set Check/ # Data check folder.
│ ├── Data # Data used for the report file.
│ ├── Figures # Figures used for the report file.
│ ├── Models # Typically empty.
│ ├── PathDic.json # Vehicle trajectory data used for the report file.
│ └── Report.pdf # The simulation report file for showing results and
| # checking the dataset.
├── Config.json # Simulation configuration file, including the parameters
| # of task, event, message, and time-triggered system.
├── FGC.csv # Filtered dataset
├── GlobalContext.csv # Raw dataset generated from the simulation directly.
├── subdataset.npy # Sub-dataset dictionary that can be used directly.
├── system_dataset.csv # CPU utility dataset
└── README.md
The contents of the 'Data Set Check' folder include the simulation results and the dataset evaluation results. These are automatically summarized in the Report.pdf file after the simulation run is completed. Since our time-triggered scheduling system is simulated on a server with RT-Linux kernel, rather than being a real-time triggered system, we must evaluate the generated data to avoid using datasets with severe violations of the schedule. At the end of this file, there are two test results: Start Time Error Test and End Time Error Test. A severe error occurs if a task is started before its dispatch time specified in the schedule. If the delay is too long, it will also trigger a severe error. A severe error will occur if a task’s execution time exceeds the WCET (Worst-Case Execution Time). The error data calculated in these two tests will be displayed in two graphs, and any error messages will be listed below the graphs, if available. Due to other software running on the Linux system, some scheduling errors may occur, which cannot be completely avoided. Therefore, we consider a dataset usable as long as the number of severe errors is not too high, and the error results are not too severe. Note: The usability of a dataset is subjectively determined by us. Therefore, we include the evaluation results in our dataset to help readers independently judge whether they want to use it.
All the simulation parameters are included in the 'Config.json' file, which lists all the tasks, messages, events, and other simulation parameters. This file generally does not change, but as we continue to update our simulation software, this file will undergo corresponding changes.
The 'GlobalContext.csv' file is directly generated by our simulation. The 'FGC.csv' file is a reordered version of the dataset, and its data section is identical to that of 'GlobalContext.csv'. Note: Our simulation usually has missing or anomalous data in the first and last scheduling cycles. Therefore, if you plan to use the 'FGC' file directly, please remove the data from the first and last scheduling cycles. The 'subdataset.npy' file contains five sub-datasets. It is the result of splitting the data from 'FGC' based on functionality, with the data from the first and last scheduling cycles removed. For example, in the '2024.07.15 15h29m36s.zip' dataset, the 'FGC' file contains data for 6905 scheduling cycles, while the 'subdataset.npy' file contains data for 6903 scheduling cycles.
Here, a Python code example is provided to help you retrieve the 5 sub-datasets.
import numpy as np
# Please use the dataset path after unzipping
datasetPath = "<the dataset path>"
# obtain 5 sub-datasets from subdataset.npz file
subdatasetPath = os.path.join(datasetPath, "subdataset.npz")
# load dataset dictionary
_subdataset = np.load(subdatasetPath)
# example for showing you how to obtain these 5 sub-datasets and
# create your own sub-dataset dictionary.
subdataset = {'header':_subdataset['header'],
'task':_subdataset['task'],
'event':_subdataset['event'],
'attr':_subdataset['attr'],
'cpu':_subdataset['cpu']}
# show their shape
print(subdataset['header'].shape,
subdataset['task'].shape,
subdataset['event'].shape,
subdataset['attr'].shape,
subdataset['cpu'].shape)
# show the name of event attributes
attrNameList = list(_subdataset['attr_name'])
print(attrNameList)
# create prefix (data) sequence and suffix (label) sequence based on them
# ...Carla Simulator is an open-source simulation platform designed for developing AD systems. It provides a highly realistic environment for testing autonomous vehicles. Furthermore, it provides Python APIs and several AD examples.
Homepage: https://carla.org/
GitHub: https://github.com/carla-simulator/carla
ROS is an open-source framework for developing robotic applications. In our application, ROS plays a role as a communication system.
Homepage: https://wiki.ros.org/noetic
We've expanded a simulation example of Carla Simulator into a multi-processor time-triggering system. In this example, only the autonomous driving functionality based on the data from the simulation environment is implemented. We want to add a variety of sensors to make the whole system more realistic. Therefore, we refer to the teardown report of the Tesla Model 3 (https://www.eetimes.com/a-tesla-model-3-tear-down-after-a-hardware-retrofit/). Since we don't have access to the functional modules of the software part, we can only design them ourselves based on the web descriptions and our understanding. However, AD simulation isn't our main job. Our main goal is to build an AD simulation based on a multi-processor time-triggered system to obtain event logs. The message dependency diagram of the AD simulation application we designed is as follows:
This AD simulation consists of 21 tasks and 32 events. We refer to the design of Tesla Model 3 and the hardware situation of our server to configure the number of CPU cores, a total of 14 cores. There are three independent computing units, and the number of CPU cores in each unit is different. Moreover, the tasks assigned to each unit cannot be executed in other units. In order to get as close to the real situation as possible, we added a "computational load" (actually a "sleep" function) to each task to consume time.
There are 4 sensor data collection tasks: "GNSS & IMU Data Collection", "Ultrasonic Sensor Data Collect", "Radar Data Collection", and "Camera Video Collection" tasks. These sensor data can be obtained directly from Carla Simulator's APIs. However, in practice, this sensor data takes time to acquire. Therefore, we configured the basic computational load for each sensor task. Since some "sensors" have a long sampling period, their sensor data can be accessed every several scheduling cycles. We configured periodic computational loads for the "GNSS & IMU Data Collection" and "Camera Video Collection" tasks so that the execution time of these two tasks varies periodically. Moreover, their sensor data are only available periodically. The periods of these two tasks are 5 seconds and 10 seconds, respectively. We also take into account the loss of sensor data. Every sensor task has a certain probability, albeit low, of losing data.
There are 3 actuator tasks: "Throttle & Steering Driver Control", "Brake Actuator Control", and "Dash Panel Display". They only have a fixed computational load.
The remaining 14 tasks are logical computing tasks. They are responsible for the implementation of the different functional modules. However, implementing each of these modules is unfeasible for us, and it is not our original purpose. For example, the "Vehicle Detection" task is responsible for identifying and classifying surrounding vehicles and estimating their location by fusing data from radar, ultrasonic, and camera sensors. This requires the cooperation of multiple machine learning algorithms and deep learning models. Furthermore, it requires a lot of time and resources to design and train models. Fortunately, we can access all vehicle information directly through the APIs provided by the Carla Simulator. Therefore, the idea is that we don't really need to implement the functionality of these modules, but rather restore the input and output streams of these modules as much as possible and make the computation time seem reasonable. For example, the "Vehicle Detection" task is implemented as follows: we take the location of all vehicles through APIs and then calculate their distance from the vehicles we control. If the distance of a vehicle is within a detection range, then the information will be appended to the output list of this task. Although the output has nothing to do with the input data, it affects the execution time of the task. Because sensor data are not always accessible due to periodicity and data loss, old data may be used for multiple scheduling cycles. In such a situation, we assume that additional time is needed to estimate the current data based on the old one. The execution time of other logical computing tasks is also adjusted based on the current task inputs.
Carla Simulator is developed using the Unreal Game Engine. Although Carla Simulator presents realistic 3D renderings, the simulation process is not real-time. Since the 3D rendering time is affected by a variety of factors, such as the number of objects in the picture and the complexity, the corresponding real-world rendering time is variable. Therefore, we separated the scheduling system from the simulation process and designed a separate time system for them. Carla Simulator provides an API that allows the simulator to perform a simulation for a specified simulating time period. Thanks to this API, we can separate them. The two systems are executed alternately, as shown in the following diagram:
After initializing the simulation environment and scheduling system parameters, the scheduling system begins to execute the first scheduling cycle. Some parameters of the simulation environment are dynamically configured during this process, rather than just task execution (so the data for the first scheduling cycle of the event log needs to be discarded). When all tasks are completed, the task clock will be paused. Then start the rendering of the first simulation tick. We set each tick to 20ms simulation time. The simulation clock automatically starts timekeeping. When the simulation ends, the simulation clock is automatically paused. Finally, each task sends its own event list to the master node to generate an event log. This will be detailed in the next section. This is the end of the first scheduling cycle. The next scheduling cycle will then be triggered by the scheduler, and the task clock will resume. When all tasks are executed, the task clock will pause again. This process can be simply described as the alternation of the scheduling system and the simulation system. These two clocks also alternate until the vehicle we control reaches the destination or reaches the maximum execution time. When the simulation application starts, the PC clock starts to count until the whole simulation is completed. It is not affected by simulation systems and scheduling systems. Each event will contain information from these 3 clocks. I've taken a screenshot of the event log and presented the time data in the graph below.
The task clock is synchronized with the PC clock, but only when the scheduling system is running. As you can see from the diagram above, the relationship between the task clock and the PC clock is a straight line. The simulation clock is not directly related to the PC clock but is controlled by the simulation environment. We set the simulation time of a tick to 20ms, so it takes very little time to render. From this graph, you can see that the task clock and the simulation clock are counted alternately. At the end of each scheduling cycle, there is a period of time when both clocks stop counting, during which the local event lists are transmitted to the masker node.
Each task has an event monitoring module designed to collect event data. To facilitate the development of future event prediction models, relevant data is recorded every time an event is triggered, forming an event vector
Where
to be updated...
In the simulation, a total of 32 event types are defined. Among them, 25 events are related to environmental inputs and system state changes, called unique events. These events may trigger the adaptive time-triggered system to switch schedules, to which your models should give more attention. The remaining 9 events appear duplicately in every scheduling cycle and are independent of environmental inputs, referred to as unique events. In the following, we introduce these events and the corresponding information in detail. It should be noted that most events and their attributes occur only once within a single scheduling cycle, except for "event_task_start" and "event_task_end", which are triggered for every task. Consequently, their associated attributes appear multiple times. To reduce memory consumption during model training, the event attributes within a single scheduling cycle are merged into one attribute list in the Attribute sub-dataset. Duplicate attributes, namely "start_time", "execution_time", and "slack_time_percentage", are disambiguated by using the task name as a prefix, resulting in 21 unique instances of each of these attributes. Some event attributes always have a value of zero (used as placeholders) because they are not useful for event prediction. These data will be completely removed from the logs in the next version.
| ID | Event Name | Duplicate | Description |
|---|---|---|---|
| 0 | event_task_start | yes | It is triggered at the beginning of each task. This event includes the attribute "start_time", which denotes the task’s start time in seconds [s]. |
| 1 | event_task_end | yes | It is triggered at the completion of each task. This event contains two attributes: "execution_time" indicates the duration of the task execution within the current scheduling cycle in seconds [s]; and "slack_time_percentage" represents the percentage of remaining time (slack time) available for the task in percentage [%], which is calculated as |
| 2 | event_gnss | no | It is triggered upon successfully collecting GNSS sensor data. This event includes two attributes: "longitude" and "latitude". It should be noted that the GNSS sensor is configured with a controllable data loss probability of 0.3%. |
| 3 | event_imu | no | It is triggered upon successfully collecting IMU sensor data. This event contains 7 attributes: three-axis accelerations ("acc_x", "acc_y", and "acc_z"), three-axis angular velocities ("gyro_x", "gyro_y", and "gyro_z"), and the orientation angle estimated from the magnetometer, "compass". It should be noted that the IMU sensor is configured with a controllable data loss probability of 0.1%. |
| 4 | event_ultrasonic | no | It is triggered upon successfully collecting ultrasonic sensor data. This event contains 12 attributes, "s1" to "s12", representing sensor data from the simulation. Note: they are all 0. The CARLA simulator does not provide an ultrasonic sensor model, but a radar sensor is typically used as a substitute. The radar sensor produces point cloud data, the storage of which in the event log would consume a substantial amount of memory. Moreover, such unprocessed raw data offers limited value for the prediction model. |
| 5 | event_radar | no | It is triggered upon successfully collecting radar data. This event contains 1 attribute, "radar_distance". Note: it is 0, for the same reason as with the ultrasonic sensor. |
| 6 | event_camera | no | It is triggered upon successfully collecting camera data. This event contains 8 attributes, "c1" to "c8", representing image data from 8 cameras. Note: they are all 0, because image data is not useful for event prediction. |
| 7 | event_self_localization | no | This event is triggered after the ego vehicle’s current position, velocity, and heading angle are computed through the fusion of GNSS-provided longitude and latitude data with IMU data, followed by coordinate transformation. It comprises seven attributes: three-dimensional positional coordinates ("loc_x", "loc_y", "loc_z"); three-dimensional velocity components ("vel_x", "vel_y", "vel_z"); and the heading angle "rot_yaw". A value of 0 for "rot_yaw" corresponds to geographic north, measured clockwise, with the unit expressed in radians. |
| 8 | event_gnss_is_none | no | It is triggered when GNSS sensor data is not successfully acquired, representing either data loss or a malfunction of the GNSS sensor. |
| 9 | event_red_traffic_light | no | This event is triggered when a red traffic signal is detected. If the event is not triggered, it indicates that either a traffic signal of another color has been detected or no traffic light has been recognized. |
| 10 | event_traffic_sign | no | This event is triggered when a lane marking is detected. It includes three Boolean attributes: "lane_change", indicating whether a lane change is possible in either direction (e.g., in the case of a single lane); "left_lane_change", indicating that a left lane change is possible; and "right_lane_change", indicating that a right lane change is possible. |
| 11 | event_global_path | no | This event is triggered when the global path planning task computes a new route. It contains a single attribute, "remaining_points_len", which represents the number of remaining waypoints along the planned path and is proportional to the remaining travel distance. |
| 12 | event_vehicle_detection | no | This event is triggered when the vehicle detection task identifies the presence of surrounding vehicles. It contains five attributes: "v_distance", representing the distance to the nearest vehicle; three velocity components ("v_x", "v_y", "v_z") describing the detected vehicle’s speed; and "v_yaw", indicating the vehicle’s heading angle. |
| 13 | event_vehicle_detection_fail | no | This event is triggered when the vehicle detection module fails, which is often caused by the absence of camera data. |
| 14 | event_pedestrian_detection | no | This event is triggered when the pedestrian detection task identifies the presence of pedestrians, typically occurring near intersections and crosswalks. It contains five attributes: "p_distance", representing the distance to the nearest pedestrian; three velocity components ("p_x", "p_y", "p_z") describing the detected pedestrian’s speed; and "p_yaw", indicating the pedestrian’s heading angle. |
| 15 | event_pedestrian_detection_fail | no | This event is triggered when the pedestrian detection module fails, which is often caused by the absence of camera data. |
| 16 | event_lane_aware | no | This event is triggered when a lane is detected. It contains three attributes: "l_x", "l_y", and "l_z", representing the waypoint coordinates of the detected lane’s center point. |
| 17 | event_lane_aware_fail | no | This event is triggered when the lane detection module fails. Such failure may result either from the absence of image sensor data or from the inability to detect lane markings, which can occur, for example, when the computed region of interest (ROI) is incorrect or when lane markings are missing, such as in the middle of an intersection. |
| 18 | event_vehicle_following | no | The decision module classifies the current vehicle state based on the vehicle detection task, pedestrian detection task, traffic light detection task, and ego-vehicle information (including position, speed, and orientation). This event is triggered when the ego-vehicle is following another vehicle. |
| 19 | event_lane_keeping | no | This event is triggered when the ego is traveling along an open road. |
| 20 | event_turning_left | no | This event is triggered when the ego is making a left turn or overtaking from the left side. |
| 21 | event_turning_right | no | This event is triggered when the ego is making a right turn or preparing to park on the right side. |
| 22 | event_intersection | no | This event is triggered when the ego is passing through an intersection. |
| 23 | event_local_path | no | This event is triggered when the local path planning task computes a local path. |
| 24 | event_controller | no | This event is triggered when the control task generates control signals. It includes three attributes: "throttle", "steer", and "brake". |
| 25 | event_start_calc_local_context | yes | At the end of each task, all event vectors of the task are integrated to form the local context. This event is triggered when the computation of the local context begins. |
| 26 | event_transmit_local_context | yes | After the local context has been computed, it is transmitted to the master node, at which point this event is triggered. |
| 27 | event_start_calc_global_context | yes | When the master node has received all local contexts, it merges them into a global context, at which point this event is triggered. |
| 28 | event_transmit_global_context | yes | This event is triggered when the master node computes the global context and transmits it back to the other tasks. |
| 29 | event_task_pause_and_resume_carla | yes | This event is triggered when all tasks have been completed and the task clock is paused. It marks the end of a scheduling cycle. |
| 30 | event_task_pause_carla | yes | This event is triggered after the CARLA rendering is completed, when the task clock resumes and a new cycle begins. It represents the start of a new scheduling cycle. |
| 31 | event_new_timebase | yes | In our co-simulation, since the rendering time of CARLA is unpredictable, the scheduling timebase needs to be recalculated after the completion of each CARLA tick. This event is triggered at that point. It contains 1 attribute, "new_timebase", which denotes the new scheduling time base and also marks the start time of the first task (scheduled at time 0) in the subsequent scheduling cycle. |
If you find AD-TTS useful, please cite the paper:
@INPROCEEDINGS{11049265,
author={Li, Dongchen and Onwuchekwa, Daniel and Obermaisser, Roman},
booktitle={2025 14th Mediterranean Conference on Embedded Computing (MECO)},
title={Event and Slack Time Prediction With Deep Learning for Proactive Runtime Scheduling in Time-Triggered Systems},
year={2025},
volume={},
number={},
pages={1-8},
keywords={Deep learning;Resistance;Schedules;Runtime;Processor scheduling;Computational modeling;Stability criteria;Noise;Predictive models;Multitasking;event sequence prediction;slack time prediction;multi-task learning;deep learning;time-triggered scheduling system},
doi={10.1109/MECO66322.2025.11049265}}