Evaluation, conception and prototypical implementation of telemetry dashboard components in motorsport PDF Free Download

Name: Evaluation, conception and prototypical implementation of telemetry dashboard components in motorsport PDF
Author: ChangeFox

1 / 109

0 views•109 pages

Evaluation, conception and prototypical implementation of telemetry dashboard components in motorsport PDF Free Download

Evaluation, conception and prototypical implementation of telemetry dashboard components in motorsport PDF free Download. Think more deeply and widely.

Master’s thesis in the master’s course

User Experience Design

Specialist area: Computer Science

Evaluation, conception and prototypical

implementation of telemetry dashboard

components in motorsport

Issued on 9th June, 2022.

Submitted by

Andreas Riedel

Matriculation-Nr.: 00120225

anr6080@thi.de

at Technische Hochschule Ingolstadt

on 9th December, 2022

to obtain the academic degree of

Master of Science (M. Sc.)

First Examiner: Prof. Dr. Munir Georges

Second Examiner: Prof. Dr. techn. Priv.-Doz. Andreas Riener

Declaration

In accordance with § 18 Para. 4 Nr. 7 APO THI

I hereby declare that this thesis is my own work, that I have not presented it elsewhere

for examination purposes and that I have not used any sources or aids other than those

stated. I have marked verbatim and indirect quotations as such.

Ehrenwörtliche Erklärung

Erklärung nach § 18 Abs. 4 Nr. 7 APO THI

Ich erkläre hiermit, dass ich die Arbeit selbständig verfasst, noch nicht anderweitig für

Prüfungszwecke vorgelegt, keine anderen als die angegebenen Quellen oder Hilfsmittel

benützt sowie wörtliche und sinngemäße Zitate als solche gekennzeichnet habe.

Ingolstadt 08.12.2022

Location Date Signature

Abstract

With the rapid development of technology in recent years, telemetry has become one

of the most important components in motorsport. The remote collection of telemetry

data is crucial to the success of each team. In addition, full compliance with technical

and safety regulations by all vehicles must be monitored at any time by the race

scrutineers. This master's thesis concerns interactive visualisation of telemetry data on

the web for scrutineers.

The first part of the thesis deals with basic UX concepts that describe how huge amount

of data sets can be visualised as comprehensively as possible in a performant and clear

manner in the web. Furthermore, it is investigated whether it is possible to visualise

large amounts of data interactively with web technologies.

Building on this, the second part of the thesis describes the conception and realisation

of prototypical virtualisation components for the web. For this purpose, software

systems and technologies are evaluated based on requirements previously defined

using the qualitative methods of expert interviews and the contextual inquiry method at

the 24 Hours of Le Mans 2022.

Contents

III

Contents

Declaration ....................................................................................................................... I

Abstract ........................................................................................................................... II

Contents ......................................................................................................................... III

List of abbreviations ........................................................................................................ V

List of figures .................................................................................................................. VI

List of tables ................................................................................................................. VIII

1. Introduction ............................................................................................................ 1

1.1. Overview and Relevance ......................................................................... 1

1.2. Research Questions and Goals ............................................................... 3

1.2.1. Evaluation of concepts to design user-friendly real-time

dashboards ................................................................................ 3

1.2.2. Evaluation of performance possibilities for visualising large

amounts of data in the browser ................................................. 4

1.2.3. Prototypical design and development of web-based high

performance graphical components .......................................... 4

1.3. Scope and Limitations ............................................................................. 5

2. Fundamentals ........................................................................................................ 6

2.1. Information Visualisation .......................................................................... 6

2.2. Real-time computing ............................................................................... 8

2.3. Telemetry ................................................................................................. 9

2.4. Challenges ............................................................................................... 9

3. Usability and Technology .................................................................................... 11

3.1. Usability in Dashboards ......................................................................... 11

3.1.1. Visualisation Basics ................................................................. 12

3.1.2. Design principles ...................................................................... 13

3.1.3. Visual Perception ..................................................................... 17

3.1.4. Visual Chart Library .................................................................. 20

3.1.5. Interactions .............................................................................. 23

3.1.6. Placing the usability results in the overall context ................... 24

3.2. Web-Technologies and Performance .................................................... 26

3.2.1. Performance in Data Visualisation ........................................... 26

3.2.2. Render functionality of web-browsers ..................................... 27

3.2.3. Canvas and WebGL ................................................................. 29

3.2.4. JavaScript vs. Web Assembly .................................................. 34

3.2.5. Data Pre-Processing ................................................................ 35

3.2.6. Placing the technical results in the overall context .................. 39

4. Practical Work ...................................................................................................... 41

Contents

4.1. Use case ................................................................................................ 41

4.2. Methods ................................................................................................. 42

4.2.1. Expert Interviews ...................................................................... 42

4.2.2. Contextual Inquiry Method ....................................................... 43

4.2.3. Design Thinking ........................................................................ 44

4.3. Personas and User Stories .................................................................... 45

4.4. Usability Conception .............................................................................. 46

4.4.1. Usability Requirements ............................................................ 46

4.4.2. Conception and Low Fidelity ................................................... 48

4.4.3. Design ...................................................................................... 52

4.5. Performance and Technology ................................................................ 55

4.5.1. Technical Requirements .......................................................... 55

4.5.2. Technology Evaluation Matrix .................................................. 57

4.5.3. Performance and Robustness Test ......................................... 60

4.5.4. Technical Performance Results ............................................... 64

4.6. Prototype Implementation ..................................................................... 65

4.7. Usability Test ......................................................................................... 67

4.7.1. Tasks ........................................................................................ 67

4.7.2. Participants .............................................................................. 68

4.7.3. Result ....................................................................................... 68

4.8. Algorithm Survey .................................................................................... 71

4.8.1. Questions ................................................................................. 71

4.8.2. Participants .............................................................................. 73

4.8.3. Results ..................................................................................... 73

4.9. Evaluation of results and potential for improvement ............................. 75

5. Conclusion and Outlook ...................................................................................... 79

Bibliography .................................................................................................................. 81

Appendix ....................................................................................................................... 85

A.1 WebGL Code Snippet .......................................................................................... 85

A.2 Expert Interviews ................................................................................................. 87

A.3 Personas .............................................................................................................. 92

A.4 Requirements and Evaluation Matrix ................................................................... 95

A.5 Performance Test Suite ....................................................................................... 96

A.6 Clickable Prototype ............................................................................................. 96

A.7 Usability Test ....................................................................................................... 97

A.8 Down sampling example ..................................................................................... 98

List of abbreviations

API

Application Programming Interface

CSS

Cascading Style Sheets

CSSOM

CSS Object Model

DOM

Document Object Model

GUI

Graphical User Interface

FPS

Frames per Second

WASM

WebAssembly

JavaScript

User Experience

SUS

System Usability Scale

UEQ

User Experience Questionnaire

List of figures

Figure 1: Application Architecture ................................................................................................ 5

Figure 2: Scientific and information visualisation ......................................................................... 7

Figure 3: Visualisation pipeline ..................................................................................................... 7

Figure 4: Dashboard gauges with missing context .................................................................... 13

Figure 5: Measurement illustration that fail to express the message ......................................... 14

Figure 6: Eye-tracking study for dashboards ............................................................................. 15

Figure 7: Two graphs with use of non-data ink visualised in red ............................................... 16

Figure 8: Attentive vs. Preattentive Processing .......................................................................... 19

Figure 9: Preattentive Attributes: Size ........................................................................................ 19

Figure 10: Point, Line and Bar diagram encoding same quantitative data ................................ 20

Figure 11: Interaction Techniques. ............................................................................................. 24

Figure 12: Retained mode rendering .......................................................................................... 28

Figure 13: SVG Syntax and outcome ......................................................................................... 28

Figure 14: Render time with varying number of objects for SVG and Canvas. .......................... 29

Figure 15: Immediate Mode rendering ....................................................................................... 30

Figure 16: Canvas Syntax and outcome .................................................................................... 30

Figure 17: Performance Test Canvas vs. WebGL ...................................................................... 32

Figure 18: Performance Test SVG, Canvas and WebGL regarding FPS .................................... 33

Figure 19: Data overload in line diagrams .................................................................................. 35

Figure 20: Full dataset (light grey) vs. Average Aggregation Method (black) ............................. 36

Figure 21: Bucketing of data. ..................................................................................................... 36

Figure 22: Full dataset (light grey) vs. LTTB algorithm (black) .................................................... 38

Figure 23: Largest triangle across three buckets and point C being a temporary point ............ 39

Figure 24: Persona Tom ............................................................................................................. 45

Figure 25: Health-strips Low Fidelity Prototype, first design cycle ............................................ 48

Figure 26: Health-strips Low Fidelity Prototype, second design cycle ...................................... 49

Figure 27: Health-strips Low Fidelity Prototype, third design cycle ........................................... 50

Figure 28: Multi-select box ......................................................................................................... 51

Figure 29: Dashboard Low Fidelity Prototype, 1st design cycle ................................................. 51

Figure 30: Dashboard Low Fidelity Prototype; Header and Control Centre, 2nd design cycle ... 52

Figure 31: Colour palette ............................................................................................................ 53

Figure 32: Dashboard Design ..................................................................................................... 53

Figure 33: Static test with different numbers of points .............................................................. 62

Figure 34: Appending test with different number of elements per graph ................................... 63

Figure 35: Implemented Prototype in Grafana ........................................................................... 66

Figure 36: SUS-Score Telemetry Dashboard ............................................................................. 70

Figure 37: UEQ Result Telemetry Dashboard ............................................................................ 70

Figure 38: Different visual characteristics of telemetry data for the algorithm survey ............... 72

List of figures

VII

Figure 39: Survey result for the first two examples voting the best algorithm ........................... 74

Figure 40: Survey result for voting the best thresholds for LTTB algorithm ............................... 74

Figure 41: Persona Liz Morgan ................................................................................................... 93

Figure 42: Persona Tom Pright ................................................................................................... 94

Figure 43: Algorithm Example with LTTB (1), AVG (2) and Mode Median Bucket (3) ................. 99

List of tables

VIII

List of tables

Table 1: Evaluation Matrix Graphs and Relationships ................................................................ 21

Table 2: Functional Design Requirements .................................................................................. 47

Table 3: Technical Requirements ............................................................................................... 56

Table 4: Additional quality features for evaluating technology ................................................... 57

Table 5: Acceptance Criterion .................................................................................................... 58

Table 6: Performance Test suite Scenarios ................................................................................ 61

Table 7: Age distribution of participants in the algorithm survey ............................................... 73

Introduction

1. Introduction

The introduction provides an overview of this master’s thesis. Chapter 1.1 explains the

general topic and relevance. The objectives and research questions of this thesis are outlined

in more detail in chapter 1.2. Chapter 1.3 defines the scope and its limitations.

1.1. Overview and Relevance

Motorsport

has changed in recent decades and has become increasingly technical (cf.

Menath, 2014). As a result, the sport provides millions of fans with thrilling entertainment.

Racing fans can track the individual performance of each race car via wireless data and video

broadcast subscriptions. This is only possible because of dozens of sensors installed in the

vehicle to collect information and send them continuously to a central data collector (cf.

Reese, 2017). As a result one car generates up to 30 GB of data in one race (cf. Blackstock,

2020). This data is referred to as telemetry data and describes the remote collection and

transmission of information in the form of measured values (cf. Rothe, 2022).

The initial step to the telemetry that is known today was taken in 1978 by the American

physicist Dr. Karl Kempf (cf. Stansell und Marsh 1978 p. 111). He was hired by the British

Formula Two racing driver Ken Tyrell to record as much data from racing cars as possible.

Kempf pioneered telemetry data recording at Goodyear and brought it to Formula One. The

sensors measured speed, suspension movements, directional forces, throttle, braking, and

more (cf. ibid.). Kempf is seen as the founder of today's technical motorsport: Not only the

fans benefit from the technological transition, but also the racing teams.

The teams are nowadays able to monitor the cars in microseconds granularity and decide

their strategy based on the telemetry data. The exponential development of racing tech is

caused by the technological and scientific approach. Telemetry systems can interpret data

without requiring lengthy human translation, giving engineers a scientific perspective on the

vehicle's performance to optimize it in ways that aren't possible for a human (cf. Blackstock,

2020). In addition, racing cars must comply with complex regulations each racing series has

established to ensure fairness. Due to the rapid development of racing cars, there is a

possibility that teams could circumvent the regulations. To ensure that the racing cars meet

the regulations, race control needs a tool for continuous real-time monitoring of the cars.

Therefore, telemetry data is of great interest to race control. The associated process to

For the sake of readability, the generic masculine is used. However, it can mean both male and female persons.

Introduction

combat rule-breaking in motorsport is called scrutineering and is carried out by the race

stewards, the so-called scrutineers (cf. Partridge, 2021). Not only are scrutineers responsible

for preventing and punishing cheating by ensuring that no team gains a competitive technical

advantage, but also for ensuring the safety of the drivers by confirming that all vehicles, parts

and equipment comply with technical regulations. The technical inspection procedure looks

similar in the major racing series. At any time during the race weekend, the vehicles can be

inspected – before, after, or even both. This involves weighing the cars as well as measuring

the size, angle and relative position of all external components (cf. ibid). During the race,

however, the scrutineers receive only minimal data in most race series. Currently, the data

they receive is transmitted to the control centre from a sensor system installed at the

racetracks. The sensors communicate with the cars as they pass the system. Since race cars

reach speeds up to 300 km/h, only very little data is exchanged during the short time they

pass the sensor system.

Therefore, it would be beneficial to develop an application that has access to each race cars’

real-time live telemetry data. Thereby, the data can be visually prepared and provided to the

scrutineers. This ensures that the safety of the vehicles and drivers is guaranteed at all times

during the race and it can be observed whether teams are in breach of the regulations.

Moreover, it would allow the scrutineers to react to critical situations immediately and

intervene during the race.

Since such an application requires real-time access to the telemetry data of all racing cars

simultaneously, it seems reasonable to rely on a cloud solution. This way, the data can be

transmitted and retrieved simultaneously from the cloud. The data is stored centrally and can

be accessed from anywhere via flexible interfaces. Cloud architecture also offers better

scaling, both horizontally and vertically, with corresponding conceptual approaches. In

addition, the data can be accessed during the race, but also afterwards, so that it can be

easily shared within the organisation.

To avoid being tied to possibly expensive hardware or operating systems, it would be

advantageous if the application runs in a conventional browser. Access is possible from

anywhere with any device – only an internet connection is needed. Moreover, a web-

application facilitates automatic and continuous rolling out of updates and new features to all

users at the same time. This has the advantage, that users do not have to maintain the

application themselves. Therefore, the application is the same for all users and on all end

devices without the need to install an app. However, web-applications only have limited

access to special hardware features of a device, so performance problems might occur when

processing a large amount of data.

Introduction

The following master’s thesis is based on the above considerations. It has the main objective

to research how to design and develop user-friendly dashboards and if it is possible to

visualise huge amounts of data in a web-based environment.

1.2. Research Questions and Goals

This thesis aims to research approaches and concepts for the prototypical development of

web-based components for a telemetry dashboard for scrutineers in motorsport. In order to

achieve this goal, two research questions have to be answered:

1. What methods and concepts can be used to design graphical components in a user-

friendly way by highlighting the most important aspects while at the same time

providing a good overview without overwhelming the user?

2. Is it possible to use web-based frontend solutions to develop graphical dashboard

components that can display an immense historical data stream as well as real-time

data in a performant way?

In the following, each research question is assigned to a local objective. Building on this, the

previously gained knowledge is applied practically, which is the third local objective of this

thesis:

1. Evaluation of design concepts to develop user-friendly real-time dashboards.

2. Evaluation of performance possibillities for visualising large amounts of data in the

browser.

3. Prototypical design and development of web-based high-performance graphical

components.

1.2.1. Evaluation of concepts to design user-friendly real-time

dashboards

The first obejctive is explored in detail in chapter 3.1. The local goal evaluates concepts and

methods for gaining insights into how to visualise large amounts of data in a user-friendly

way. Basic visualisation types and best practices for visualising dashboards and components

are described in detail. In addition, psychological factors of human visual perception are

explained to understand how the human brain processes and stores data. Furthermore, it is

discussed how graphic components can be arranged optimally for human perception.

Thus, an overview describing the basic concepts of dashboard visualisations will be

presented to justify the approach for practical implementation.

Introduction

1.2.2. Evaluation of performance possibilities for visualising large

amounts of data in the browser

The second local objective, discussed in chapter 3.2, evaluates the state of the art of web

browsers that enable performant data visualisation on the web. Specifically, web technologies

for performant web visualisations are described and compared. In addition, methods are

discussed to prepare and pre-process data in a way to achieve improved performance.

The aim is to provide an overview of the basic knowledge required to develop powerful

graphical components for the web.

1.2.3. Prototypical design and development of web-based high

performance graphical components

Based on the knowledge gained from the previous two local objectives, online-based

prototype graph components and dashboards will be developed in chapter 4. Therefore,

existing dashboard building block systems, Business Intelligence Solutions, and frontend

graphic libraries are evaluated against previously defined requirements. Subsequently, a

suitable technology is selected to develop performant graph components that are in line with

the researched findings of the theoretical part. On the one hand, a focus is laid on usability

for which intuitive concepts have been developed to meet user design requirements. On the

other hand, technological methods and best practices are used to develop the components

in a particularly performant way. These are then evaluated in chapter 4.7 utilizing a usability-

test from which further recommendations and results are obtained. Before the start of the

thesis, basic requirements were defined, which are explained in the following:

Basic Requirements:

• The user must see all racing cars for a certain sensor data category on one dashboard.

• The user should be able to clearly identify deviations from the norm in the data.

• The user should have the possibility to monitor historical data as well as real-time data.

• The application must be executable on the web in a performant way.

Further requirements for the prototype were derived from expert interviews and the method

of contextual inquiry conducted at the 24 Hours of Le Mans in June 2022 and are presented

in chapter 4.

Introduction

1.3. Scope and Limitations

This thesis is written in cooperation with Bosch Engineering GmbH and Bosch Motorsport.

Internally, a fully comprehensive architecture was designed before the start of the thesis. The

Bosch Engineering GmbH implemented it partially before the start as well as in parallel during

the duration of the thesis. The architecture is shown in Figure 1. It describes the data flow

from the sensors (Logger) in the race car to the cloud-based database and finally the transfer

of the requested data to the end users’ web-application.

Figure 1: Application Architecture

In detail, sensor data is sent from a logger at a sampling rate of 40 hz to 10 kHz over the

mobile LTE network to an adapter in the cloud. Here, the data is passed to a converter, which

encodes and processes it and then feeds it to a cloud based timeseries database. From here,

the data can be retrieved by any application. This subarea is the focus of this thesis. However,

the provision of the data is not yet optimized for a real-time view of the data. Considerations

for an aligned architecture have already been made internally and are marked light grey in the

figure. Since real-time support is a decisive factor for the success of a later product, it was

assumed for the prototypical implementation that this would be given in the future.

Fundamentals

2. Fundamentals

Data visualisation can be based on different requirements and consequently serve different

purposes. To narrow down the concepts fundamental to this thesis, this chapter explains and

defines the essential terms necessary for further understanding. In chapter 2.1, the

information visualisation process is illustrated by introducing modelling techniques and

current tools. Chapter 2.2 explains the terminology of real-time computing in computer

technology for further understanding the development of a real-time dashboard. Finally, the

telemetry data and its characteristics are discussed and described in more detail.

2.1. Information Visualisation

The definition of "to visualise" is "to form a mental picture of something" or "to make

[something] visible" (Spence, 2007, p. 5). Visualisation refers to a human cognitive ability that

makes it possible to gain insights into or knowledge about something. Pictorial

representations support visualisations. Such representations can reduce the complexity of

data and map them into easily understandable concepts, so the human brain can

simultaneously process several items of information. With the help of visualisations, for

example, maximum and minimum values as well as relationships between the data, such as

groupings, trends, or gaps, can be easily recognised (cf. Mazza, 2009, p. 4).

The increasing power of computers has led to the emergence of the field of visualisation in

computer science. Card et al. define visualisation in computer science, which they summarise

as “the use of computer-based, interactive, visual representations of data to enhance

perception” (Card et al., 1999, p. 6). Computers not only allow researchers to create visual

representations of data more quickly and conveniently, but also to interact with these

representations to analyse the data from different angles or highlight certain aspects.

Therefore, effective visualisation supports data exploration by providing a suitable interface.

Visualisation interfaces have been heavily influenced by the information-seeking mantra

presented in the work of Shneiderman (Shneiderman, 1996). The mantra is based on

Shneiderman's experience exploring big data and describes how users want to work with big

data. According to Shneiderman, the primary purpose is to gain an overview of the entire

collection. Users then want to get more details by zooming in on items of interest and filtering

out uninteresting items. In addition, especially with large data sets, users are interested in the

relationships between the elements (cf. Shneiderman, 1996).

Fundamentals

In general, a fundamental distinction is made between scientific visualisation and information

visualisation: the former typically involves physically based, scientific data, while the latter

deals with abstract, non-physical data, such as business information or telemetry data (cf.

Oliveira & Levkowitz, 2003). Figure 2 shows an example visualisation for both classes.

Figure 2: Scientific and information visualisation (a) Scientific visualisation focuses mainly on mapping

physical measurements or simulation data to the inherent spatial reference frame (Image taken from

(UCRL, 2007). (b) Information visualisation transforms abstract data into spatial mappings (Image

taken from (Farcaster, 2013)).

However, both visualisation classes share a common problem: data overload. Presenting

huge amounts of data leads to a unclear display, which makes it hard for the viewer to gain

insights into the data (cf. Spence, 2007). Handling large amounts of data was one of the main

motivations for developing architectural models dealing with this problem. Card et al. present

a model tailored to the information visualisation category that also considers the user’s

interaction with the visualisation (cf. Card, 2008, p. 519). This includes steps like data

transformation, visual mapping and view transformation as seen in Figure 3.

Figure 3: Visualisation pipeline as proposed by Card et al.

The data transformation step involves reducing the data by filtering or aggregating data

points. Visual mapping is the heart of the visualisation pipeline and refers to the mapping of

Fundamentals

data values to visual structures of the data through attributes such as colour and shape.

Finally, the view transformation determines properties such as the position and scale of the

visual structures. Especially filtering and aggregation of data is an important topic when

dealing with telemetry data and thus is explained in more detail in chapter 3.2.5

2.2. Real-time computing

In real-time systems, the accuracy of the system depends on the calculation and transmission

of logical results and on the time in which the system generates the results. A result must be

guaranteed to be delivered within a predefined fixed time interval. In practice an arbitrarily

small time limit cannot always be realised due to a lack of sufficiently fast hardware or

limitations in the software (cf. Stankovic, 1988, p. 10 ff.). Therefore, a system is defined as

real-time when programmes work without any noticeable delay. Shin and Ramanthan defined

three classes of real-time systems, which determine the value of the information after the

execution time, as follows (cf. Shin & Ramanathan, 1994, p. 7):

• Hard: Not meeting the deadline can have catastrophic consequences.

• Firm: The information is useless after the deadline.

• Soft: Information decreases in value after the deadline but is still useful.

Since a system is only real-time if all parts of the system respect the time constraints, it is

difficult, if not impossible, to develop a real-time system that communicates over the internet.

This is due to transmission protocols like TCP or UDP, which do not include a concept of

predictable deadlines and always contain latency (cf. Williams, 2020). For this reason, the

term "near real-time" has been established in telecommunications and computer science.

Near real-time refers to the time delay caused by automated data processing or network

transmission between the occurrence of an event and the usage of the processed data (cf.

Wilson, 2021). A near real-time representation depicts an event or situation as it existed at

the current time minus the processing time, i.e., nearly at the time of the live event. The

distinction between the terms "near real-time" and "real-time" is somewhat blurry and must

be defined for each situation. The delay depends on the type and speed of the transmission.

For the present project, a delay of up to three seconds is acceptable, since large data sets

must be transmitted and processed over the LTE network as explained in chapter 1.3. In the

further course of this thesis, the term real-time refers to a soft real-time system or, more

specifically, a near-real-time system.

Fundamentals

2.3. Telemetry

As described in chapter 1.1 of this thesis, telemetry is an essential component in motorsports,

where race cars can be monitored remotely via highly automated sensor data. Enormous

amounts of data are collected in this process. In this thesis, approximately 19 million data

points per car and per race are awaited, assuming a race duration of 2 hours and 40 minutes.

An average number of 30 cars amounts to approximately 576 million data points that must

be processed per race. The data acquisition is made by 40 to 50 sensors per car, which can

be described with the following characteristics. Data is automatically collected by sensors

that regularly measure various features at a 40 Hz to 10 kHz sampling rate. Among them:

speed of the car, tyre speed for each tyre, tyre pressure, understeer and oversteer, steering

angle, acceleration, braking and gear selection. Furthermore, the data consists of a stream of

events with an ascending timestamp, which is essential for the correct analysis of the data.

Telemetry data is divided into two categories: live data is data that is logged in real-time.

Historical data is recorded data or backfilled telemetry. Engineers can continuously and

immediately access the live data, but also historical data.

Engineers work with the data interactively and analyse them at wide time ranges to identify

trends. It is important to note that anomalies or deviations from the norm must not be

excluded from the visualisation, as the engineers declare this data to be the most important.

In a further step, the telemetry data is thoroughly analysed and must be able to be retrieved

and displayed exactly in microsecond granularity. It is important that the data correspond

congruently to the data stream recorded by the logger. Otherwise, the data is unusable.

2.4. Challenges

In the previously described sections, the most important terms, and the general conditions of

the project have been explained. Based on the given conditions, some challenges can be

identified:

• For the present project, enormous amounts of data need to be processed, exceeding

up to over 500 million data points per race. These must be visualised in a performant

way and interactively analysed by the engineers according to Shneiderman's main tasks

written down in his information seeking mantra explained in chapter 2.1.

• Due to delays in data transmission and processing, web-applications are not able to

display the described amounts of data in real-time as section 2.2 has shown. This is

called a "near real-time" system, or a real-time system classified as soft. Nevertheless,

Fundamentals

it is important to keep the delay as low as possible (less than or equal to three seconds)

to guarantee the engineers a near real-time visualisation.

• Enormous data overflow can lead to high data intensity in information visualisations,

making it impossible to gain insight into the data. Furthermore, the unnecessary retrieval

of data points from the database that cannot be shown on a display due to the available

pixels leads to delayed visualisation. For this problem, concepts need to be developed

that build on the model of Card et al. presented in chapter 2.1.

In chapter 3, concepts and methods are discussed to overcome the challenges while meeting

the goals defined in chapter 1.2.

Usability and Technology

3. Usability and Technology

This chapter is divided into two subchapters. The subchapter 3.1 deals with the principles

and methods of usability design. It explains how to design user-friendly dashboards that give

the user a quick and clear overview without neglecting the details. Moreover, the first

subchapter addresses human visual perception and its importance in dashboard design. The

second subchapter 3.2 deals with optimising performance in the browser. Among other

things, various rendering methods and technologies for generating graphical elements in the

browser are discussed. Finally, the pre-processing of data is highlighted and explained in

detail.

3.1. Usability in Dashboards

Dashboards are a special form of visualisation. Stephen Few defines them as the following:

“A dashboard is a visual display of the most important information needed

to achieve one or more objectives; consolidated and arranged on a single

screen so the information can be monitored at a glance.” (Few, 2006)

This definition outlines that the fundamental challenge of dashboard design is the need to

squeeze a large amount of information into a small amount of space, resulting in a display

that is easily and immediately understandable. Another characteristic of dashboards is that

they can be customised for a specific person, group, or function. Otherwise a dashboard

cannot fulfil its purpose (cf. Few, 2006, p. 23 f.). However, dashboards generally come in two

forms: operational and analytical dashboards.

Operational dashboards aim to quickly communicate important information to users as they

complete time-sensitive tasks. In doing so, they typically present constantly updating and

changing data. Therefore, they need to quickly present data to their users to identify

unacceptable parameters deviations (cf. Laubheimer, 2017).

Analytical dashboards provide users with at-a-glance information used for analysis and

decision-making but are not as time-critical as operational dashboards. They can, for

example, alert the user to fluctuating data that will require careful analysis later (cf. ibid.)

In the following, principles and methods are discussed to design the best possible usability

for dashboards.

Usability and Technology

3.1.1. Visualisation Basics

To design a successful and user-friendly dashboard, it is important to first consider the raw

data and the user. The most valuable and important information comes from the potential

users of the application, who will use the system and decide about its success or failure. For

this reason, some preliminary research must be done before starting to design a

dashboard. This procedure can be outlined in the following steps (cf. Mazza, 2009, p. 25).

Define the problem by spending time with the potential users of the visual representation.

This is necessary to understand which message should be delivered by the dashboard. It is

important to determine the needs of the user and to figure out how they will work with the

dashboard. Moreover, it is necessary to consider the human factors of the target audience

and to know their cognitive and perceptual abilities. This influences the choice of visual

models to be used, so the users understand the information correctly.

Examine the nature of the data to be represented. The data can be quantitative, ordinal,

or categorical. Quantitative data is for example a list of integers or real numbers. Ordinal data

is not numerical in nature, but with an order of their own, such as the days of the week.

Categorical data has no order of its own. An example would be the names of people or cities.

In graphs they mostly represent labels. Depending on the type of data, a different visual

mapping might be appropriate.

The number of dimensions of the data (also called attributes) determines the type of

representation which can be used. The attributes can be independent or dependent. The

dependent attributes are those in which the user is interested. Their behaviour is analysed in

relation to the independent attributes. Depending on the number of dependent attributes, a

data collection can be described as univariate, bivariate, trivariate or multivariate. Univariate

describes the fact that one dimension varies with respect to another. Bivariate has two,

trivariate has three dependent dimensions. In multivariate dimensions, four or more

dependent dimensions vary in relation to independent dimensions.

The type of interaction determines whether the visual representation is static, transformable,

or manipulable. Static can be an image and does not allow the user to perform any interaction.

In transformable interactions, the user can control the process of data modification and

transformation in the pre-processing phase through data filtering. These manipulations

usually influence and modify the visualisations. Manipulable interactions allow the user to

control and change parameters, generating a new view via zooming, rotation, panning, etc.

Usability and Technology

Each of the possible options described may indicate the use of a particular design technique.

The following section discusses design principles that support those techniques.

3.1.2. Design principles

Building on the fundamentals discussed above, Few's preoccupation with the surface and

functional visual characteristics of dashboards have led to a set of principles. The principles

that are most relevant to this project are discussed in this section:

1. Not exceeding the boundaries of a single screen

One of the great benefits of a dashboard is that it allows the user to see information at a

glance to gain better understanding of the data. If dependent values are not on the same

screen, a dashboard might fail its purpose. This is because only a small amount of

information at a time can be stored in the short-term memory as explained in chapter

3.1.3.1. Therefore, dashboards should only be divided into individual screens if there is a

more significant benefit in displaying additional information groups. On the other hand,

scrolling is usually always a bad practice. Most people assume that anything outside their

immediate field of view that only becomes visible by scrolling is less important than what

is immediately visible (cf. Few, 2006, p. 39 ff.).

2. Supplying adequate context for the data

Providing the proper context for key metrics makes the difference between numbers

which are just presented and those that educate and inspire action. The gauges in Figure

4 are an example of minimal useful context. The middle gauge shows that 7,822 units

have been sold which is assumed to be positive (indicated by the green arrow). The

intended goal is that the tick marks represent an approximation of the measure. However,

this could only be achieved by labelling the scale with numbers. This is not the case and

makes the graphs therefore meaningless. Another problem is the colour of the arrows. In

general, it is a good idea to communicate good or bad performance to the user. However,

the explanation of when a performance is good or bad is missing. This could be achieved

by colouring the sections along the scales with appropriate colours (cf. Few, 2006, p. 43).

Figure 4: Dashboard gauges with missing context (Image taken by (Few, 2006, p. 43)

Usability and Technology

3. Choosing the correct measure

For a metric to be meaningful, it must be known to the user what is being measured and

in what units the metric is expressed. A metric is deficient if it does not clearly and

efficiently convey the meaning that the dashboard viewer should identify. An example of

unsatisfied metrics is shown in Figure 5. The graph shows the budget and actual revenue

trend over weeks. The aim is to determine the difference between the actual and the

projected income. In the present example, however, the viewer must calculate the

differences himself. It would be better to indicate the difference between the actual

income and the projected income. Even better would be a percentage indication, as this

shows the difference in a directly understandable way (cf. Few, 2006, p. 46 ff.).

Figure 5: Measurement illustration that fail to express the message (Image taken by (Few, 2006, p. 46))

4. Choosing an appropriate display media

Choosing an inappropriate display medium is one of the most common design mistakes.

For this reason, section 3.1.4 will go into more detail about what graph types exist and

when they are best used. Using the same type of display medium multiple times in a

dashboard does not bore the users. Yet, it can help them use the same perceptual

strategy for interpreting the data, saving them time and energy (cf. Few, 2006, p. 47 ff.).

5. Arranging the data correctly

Dashboards often must present a large amount of information in a limited space. If the

information is not well organised, the result is unclear. Therefore, the correct placement

of information according to the importance and desired viewing order is crucial. This can

be achieved with a visual design that divides the data into meaningful groups. The goal is

Usability and Technology

not just to make the dashboard look good, but to arrange the data in a way that fits its

use case. Data that needs immediate attention should be highlighted. Data that needs to

be compared should be arranged and visualised in a way that encourages comparison.

This can be achieved through the psychological Gestalt principle of similarity. This

principle states, that objects with a similar shape or colour are usually perceived as

belonging together. Furthermore, spatial proximity can indicate semantic grouping (cf.

Few, 2006, p. 56 ff.). However, the individual arrangements should always be in relation

to their importance. In 2016 an eye-tracking study from the visualisation software

company Tableau found that users look on a dashboard in an “F-Pattern” as seen in

Figure 6. The more intensely coloured areas indicate a strong tendency towards initial

fixations and respective retention times. As seen in the figure, a higher focus and retention

time is in the upper left corner and spread to the right and bottom with similar likelihood.

As a result, it can be deduced that users perceived the longest at the top left area -

elements at the bottom right, on the other hand, are perceived the least (cf. Amy Alberts,

2018).

Figure 6: Eye-tracking study for dashboards (Image taken from (Amy Alberts, 2018))

6. Using colours correctly

Colour should not be used indiscriminately. Colour choices should be made with an

understanding of how humans perceive colour. Bright and fully saturated colours demand

humans’ attention. This is because a colour appears as a contrast to the norm, which

alerts the eyes. The human brain then tries to assign a meaning to this contrast. That’s

Usability and Technology

why such colours should be kept to a minimum, using them only to highlight data that

requires attention. More subtle colours, on the other hand, are less visible, and therefore

less noticeable and should be used for content that does not require as much attention.

In general, colours that are common in nature, such as soft greys, browns, and blues,

work well as a standard colour palette for dashboards. They allow the viewer to look at

the dashboard calmly and unbiased. However, it should be considered that colours such

as red, yellow, and green exclude ten percent of men and one percent of women due to

colour blindness. Few suggests to use different intensities of one colour to encode

different meanings instead (cf. Few, 2006, p. 132).

7. Eloquence through simplicity

As already mentioned, the fundamental challenge in designing dashboards is to fit a lot of

useful and often incoherent information into a small space while maintaining clarity. This

can be achieved by simplifying dashboards. Edward Tufte introduced a concept called

the data-to-ink ratio (cf. Tufte, 2013, p. 93). When quantitative data is presented, some

ink represents data, while the rest represents visual content that does not relate to data.

These elements are called non-data ink and are highlighted in Figure 7 as red lines.

Figure 7: Two graphs with use of non-data ink visualised in red

To avoid displaying superfluous and useless information, Tufte defines a very simple

equation for the optimal use of ink.

𝑑𝑎𝑡𝑎𝑖𝑛𝑘𝑟𝑎𝑡𝑖𝑜 =𝑑𝑎𝑡𝑎𝑖𝑛𝑘

𝑡𝑜𝑡𝑎𝑙𝑖𝑛𝑘𝑢𝑠𝑒𝑑𝑡𝑜𝑝𝑟𝑖𝑛𝑡𝑔𝑟𝑎𝑝ℎ𝑖𝑐



The equation calculates how much ink is used to display the actual data compared to the

amount of ink used to enrich the graphic with visual elements. The goal is to design a

Usability and Technology

display with the highest possible data-to-ink ratio. This should be as close as possible to

1.0 without eliminating anything necessary for effective communication. Tufte derives a

design principle from this: “Maximise the relationship between data and ink as much as

possible. Every bit of ink on a graphic needs a reason. And almost always that reason

should be that the ink contains new information.” (Tufte, 2013, p. 96). However, eliminating

unnecessary ink of data is not only achieved by removing less relevant elements, but also

by using data summaries and exceptions. Exceptions are particularly useful for reducing

data in a dashboard to the essentials. Often the state of something does not need to be

displayed unless there is a problem or an opportunity that requires action. However, it is

important to keep in mind that not all non-data and data ink can be eliminated without

losing useful information or context. Especially some non-data ink might support the

structure, organisation, or readability of a dashboard (cf. Few, 2006, p. 85 ff.).

3.1.3. Visual Perception

Seeing and thinking are closely linked. Nevertheless, it is only possible to recognise patterns

in a visual representation of data if they are mapped and encoded in such a way that they

stand out clearly to human perception. This chapter deals with human perception and how

psychological factors can be used to easily perceive and process visual representations.

Chapter 3.1.3.1 discusses the role of human memory in processing visualisations. Section

3.1.3.2 explains how humans can process and encode data faster.

3.1.3.1. Memory

Human eyes do not register everything that is visible around them, but only a part which is in

the respective area of perception. And only a part of what is perceived and focused on is

processed by the brain. This is because without limits and filters, the flood of perception

would overwhelm the human brain. Memory stores information in three basic types from the

moment something is perceived:

• iconic memory (also known as the visual sensory register).

• short-term memory (also known as working memory).

• long-term memory.

In iconic memory, images are stored for a short time until they can be moved to the short-

term memory. Even though the processes in iconic memory are preconscious, a certain type

of processing is known as preattentive processing. This means that certain characteristics of

what is seen is recognised extraordinarily often, leading to certain things being highlighted

Usability and Technology

and certain groups of objects being grouped together. Preattentive processing thus plays an

important role in visual perception (cf. Few, 2006, p. 67).

In the short-term memory, information is stored during conscious processing. The short-term

memory is temporary and has limited storage capacity. Information remains in short-term

memory from a few seconds to as long as a few hours if periodically rehearsed; then it is

flushed. If rehearsed in a particular way, information is moved from short-term memory to

long-term memory, where it is stored more permanently for later recall. Humans can only store

three to nine chunks of visual information simultaneously in short-term memory. If the

capacity of short-term memory is exhausted, something must either be moved to long-term

memory or erased. What constitutes a "chunk" of visual information depends on the type of

objects which are seen. As well as aspects of their design and familiarity with them. For

example, individual numbers on a dashboard are stored as discrete chunks. A well-designed

graphic pattern, such as the pattern formed by one or more lines in a line diagram, can

represent a large amount of information as a single chunk. This is one of the great advantages

of charts over text.

Dashboards should be designed to support the optimal assembly of information so that it can

be perceived and understood as efficiently as possible in large visual gulps. The limited

capacity of the short-term memory is also the reason why information that belongs together

should never be split across multiple dashboards or require the user to scroll. If a user has

everything he needs in view on a single dashboard, information can be swapped in and out

of short-term memory at high speed (cf. Few, 2006, p. 66 f.).

3.1.3.2. Visually encoding data for rapid perception

Based on various studies in psychology, some visual features are defined as preattentive.

Preattentive means, that they can be processed more quickly without fully engaging the user’s

attention, as described in section 3.1.3.1. Attentive processing, on the other hand, is

sequential and therefore much slower.

This phenomenon is illustrated in Figure 8. On the left-hand side, it is much more difficult to

determine the number five than on the right-hand side. This is because of the different colour

intensities. According to Colin Ware, preattentive attributes can be divided into four basic

categories: colour, shape, movement and spatial position (cf. Ware, 2004).

Usability and Technology

Figure 8: Attentive vs. Preattentive Processing (a) Attentive processing, (b) Preattentive processing

through differing colour intensity

However, most charts are two-dimensional, with one axis vertical and the other horizontal.

Charts mainly visualise quantitative data, usually numbers, characterised by categorical data

that describe or define the quantitative data. (cf. Few, 2005, p. 5). As Few and Mazza point

out, two-dimensional diagrams use the two strongest properties to encode quantitative

values: line length (or the length of a shape that resembles a line with an insignificant width,

such as bars in a bar chart) and 2D position. Of all the attributes of visual perception, only

these two do an effective job of graphically representing quantitative values (cf. Mazza, 2009,

p. 40). This fact is discussed and explained further in chapter 3.1.4.

Size, on the other hand, is of very limited use with quantitative data. As Figure 9 shows, it is

easy to see that the right-hand circle is significantly larger than the left-hand circle. However,

it is difficult to specify exactly how much larger the circle is. This is because humans’ ability

to accurately compare 2D surfaces is not well developed. That is why 2D surfaces should be

avoided to encode quantitative values in graphs, including slices in a pie chart (cf. Few, 2006,

p. 72).

Figure 9: Preattentive Attributes: Size

For categorical data, such as labels, Mazza recommends distinguishing different categories

based on preattentive attributes, such as unique colours or shapes (circle, triangles), because

people tend to group objects that are similar in colour, size or shape (cf. Mazza, 2009, p. 40).

However, as stated in chapter 3.1.3.1, visual mapping cannot be done endlessly. The human

brain limits the number of preattentive attributes that can be processed in a single

visualisation. Ware suggests limiting attributes to no more than eight different colours, four

Usability and Technology

different orientations, four different sizes, and all other visual preattentive attributes to fewer

than ten different values. Few instead suggests limiting the number of distinctions for each

attribute to no more than four (cf. Mazza, 2009, p. 38).

3.1.4. Visual Chart Library

As explained in the previous section, the designer of a visual application must carefully

consider which attributes to use for a graph. This process is called visual mapping and is

explained in section 2.1. The correct data representation must also be chosen wisely to reach

its unique potential for clear and immediate communication.

Before the design and correct use of charts are discussed, it is essential to highlight a general

concept that applies in all circumstances: the appropriate use of tables versus charts.

Tables are most appropriate when the visualisation is used to look up individual values or

when the quantitative values need to be precise. Charts, on the other hand, are most

appropriate when the message is in the form of data. For example, if a single value needs to

be looked up, such as the exchange rate of the Euro against the US-Dollar at a particular

time, a table supports this need very well. If the trend or a pattern of the euro against the US

dollar over a period of time needs to be examined, then a chart is much more suitable than a

table (cf. Few, 2005, p. 4).

Few found out in his work with data visualisations that most meaningful relationships in

quantitative data can be divided into seven types: nominal comparison, time series, ranking,

part-to-whole, deviation, frequency distribution and correlation (cf. Few, 2005, p. 6). Since

Few stated, that line length and 2D position are the best ways to encode quantitative data

(see section 3.1.3.2), four types of objects work best for encoding quantitative values in

graphs: points, lines, bars, and boxes as seen in Figure 10.

Figure 10: Point, Line and Bar diagram encoding same quantitative data (Image taken by (Few, 2004))

All three charts have the same quantitative vertical scale and the same categorical horizontal

scale. The diagram on the left uses points to encode the quantitative values, the same as the

Usability and Technology

middle one. But this time a line is added to connect the dots. The right diagram uses vertical

bars to encode quantitative values.

Points visually highlight the individual values they encode. Points are rarely used alone in

charts, but rather in combination with a line connecting them. An exception is made with

scatter plots. They are using two axes with a quantitative scale. Representing this kind of data

is only possible with scatter plots and not by using lines or bars (cf. Few, 2005, p. 13).

Lines can encode quantitative data by connecting individual data points or by plotting a trend

of a series of data points. That’s why lines are particularly useful for showing values over time,

clearly representing the change from point to point. They reveal patterns that might otherwise

not be visible, such as a clear deviation from the norm (cf. ibid p. 13).

By combining location and line length to encode quantitative values, bars and boxes highlight

the individual values of the measured object per categorical subdivision. Because they give

visual weight to individual values, they are not as well suited as lines to show data pattern

over time. Nevertheless, if primarily attention should be drawn to individual values and

support comparison between them, bars might be the right choice (cf. ibid p. 14).

Researchers have found that other chart types, such as pie charts, require users to visually

assess angle, area, or volume to understand the data. These tend to be much more difficult

to process and incorporate (cf. Moran, 2022). In the evaluation matrix in Table 1, the

mentioned graph types are attributed to the seven kinds of relationships mentioned above.

The matrix evaluates the best visualisation methods for each relationship.

Table 1: Evaluation Matrix Graphs and Relationships (cf. Few, 2005)

Relationships

Encoding Methods

Nominal Comparison

A simple comparison of the categorical

subdivisions of one or more measures in no

particular order

• Bars only (horizontal or vertical)

Time Series

Multiple instances of one or more measures

taken at equidistant points in time

• Lines to emphasize overall pattern

• Bars to emphasize individual values

• Points connected by lines to slightly emphasize individual

values while still highlighting the overall pattern

• Always place time on the horizontal axis

Ranking

Categorical subdivisions of a measure

ordered by size (either descending or

ascending)

• Bars only (horizontal or vertical)

• To highlight high values, sort in descending order

• To highlight low values, sort in ascending order

Usability and Technology

Part-to-Whole

Measures of individual categorical

subdivisions as ratios to the whole

• Bars only (horizontal or vertical)

• Stacked bars when measures of the whole as well as the

parts must be displayed

Deviation

Categorical subdivisions of a measure

compared to a reference measure,

expressed as the differences between them

• Lines to emphasize the overall pattern only when displaying

deviation and time-series relationships together

• Points connected by lines to slightly emphasize individual

data points while also highlighting the overall pattern when

displaying deviation and time-series relationships together

• Vertical bars to emphasize individual values when a time-

series relationship is included

• Always include a reference line to compare the measures

Frequency Distribution

Counts of something per categorical

subdivisions (intervals) of a quantitative

range

• Vertical bars to emphasize individual values (called a

histogram)

• Lines to emphasize the overall pattern (called a frequency

polygon)

Correlation

Comparisons of two paired sets of measures

to determine if as one set goes up the other

set goes up or down in a corresponding

manner, and if so, how strongly

• Scatter plot

• Bars may be used, arranged as a paired bar graph or a

correlation bar graph, if scatter plots are unfamiliar

Two basic principles have guided the selection of each kind of representation in this library.

Firstly, it must be the best way to represent a certain type of information usually found on

dashboards. Secondly, it must serve its purpose, even if it is used in a small space (Few,

2006, p. 106).

If the right graph is chosen, some further steps need to be considered (cf. Few, 2005, p. 15

ff.). Firstly, it must be decided which variables need to be encoded along the axes. In a line

graph, the most important variable is placed along the x-axis. If data should be highlighted,

colours should be used with caution as explained in chapter 3.1.2. It might be better to

highlight data by making lines thicker or points larger.

Furthermore, the range for the quantitative scale must be determined. If bars where chosen,

they must start from zero. For other chart types, the scale could be reduced to make the

differences between the values more visible for the analysis. However, the user should be

made aware of changing the scale (cf. Few, 2005, p. 18)

Lastly, it must be decided if grid lines are required. They are only useful when values cannot

be interpreted with the necessary degree of accuracy or subsets of points in multiple related

scatter plots must be compared (cf. ibid p. 20).

Usability and Technology

3.1.5. Interactions

As mentioned in chapter 2.1 one of the biggest problems of information visualisation is data

overload. If a certain amount of data is exceeded, there is insufficient space to display all the

data intended for visualisation. However, if only a small part of the data is shown in detail, the

user misses the overall context. Therefore, users need to interact with the visualisation to

explore the data according to their needs. Schneiderman's information mantra explains how

an information visualisation system can support the user. It can be summarised as follows:

First, overview, then, zoom and filtering, finally, details on demand (cf. Mazza, 2009, p. 107).

To achieve this, there are visual mapping techniques for manipulating the view. Those will be

discussed in the following sections.

3.1.5.1. Overview and Details

The basic idea of the Overview and Details technique is to show a detailed part of the view

on the screen while having a global overview of the entire data at the same time. This

technique allows the user to get a first impression and understanding of the data structure. It

also provides the possibility of getting a more detailed view by enlarging and diminishing the

displayed area through zooming or panning. It is important that the user can quickly switch

between the two views if necessary. An example would be a time series data where the user

sees the entire data history in a small line chart. Here the user can interact, zoom, and pan.

This displays a detailed view of the selected area in a different chart, as shown in Figure 11

(a).

3.1.5.2. Focus and Context

The Focus and Context technique presents the user with detailed and contextual data

simultaneously in the same graph. The aim is to give all the space on the screen to the detailed

view that interests the user, while preserving the context in which the details are located. This

can be done by distorting the view with a fisheye technique for example: When the user

hovers over the data he gets a carefully selected distortion and therefore a focus on the

detailed data. Another solution is eliding data through selective filters or aggregations first.

While hovering over or clicking on the elided data, the user gets a focus and a detailed view

of the underlying data as seen in Figure 11 (b).

Usability and Technology

Figure 11: Interaction Techniques. (a) Overview and Details; (b) Focus + Context

3.1.6. Placing the usability results in the overall context

This chapter summarised the most important methods and principles for designing

information dashboards.

The biggest challenge in designing a dashboard is concisely presenting the most useful

information in a small space. Therefore, careful design decisions must be made when

implementing dashboards. To achieve this, the user's way of working and thinking and the

processes to be executed must be observed and understood.

Consequently, for the practical work, a scientific method must be found to observe the user

group, so that requirements can be established. This preliminary work is important to meet

user requirements in a later prototype and should not be underestimated.

When designing the graphical components within the dashboard, an essential task is to

bundle the data in such a way that the context is maintained while details can still be

accessed. This is not only important to give the user the possibility to keep an overview and

analyse individual data sets at the same time. It also corresponds to the design principle

"Eloquence through Simplicity" explained in chapter 3.1.2, which states that by summarising

and pointing out exceptions, a better overview is created and thus the presentation of

unnecessary information is avoided. Thus, the user's view can be focussed on the essentials.

In the present case, the dashboard contains all vehicles with a certain number of sensors as

outlined in the basic requirements defined in chapter 1.2.3. Thus, it can be expected that a

large amount of data must be displayed, which might lead to data overload. The flood of data

provides the viewer with limited information. In order to give the user the possibility to get an

overview of the data and at the same time to have a detailed view available for analysis if

needed, a concept has to be derived from the techniques Overview and Detail or Focus and

Context, as explained in chapter 3.1.5. This enables visual mapping in the visualisation

pipeline, creating a manipulable view. Thereby, the overall goal is always to facilitate the

understanding and analysis of big data.

Usability and Technology

This could be achieved, through a concept where the user can first see bundled data

packages elided through selective filters to get an overview of the data. By hovering over or

clicking on those data packages, he then gets a detailed view of those specific packages.

However, this decision must be accompanied by the correct selection of a chart type. For

this, the first step is to determine the dimensions of the data. After that, the evaluation matrix

described in chapter 3.1.4 can be helpful to choose the right visualisation type. The matrix

relates the most important diagram types to the most common relationships that quantitative

data possess. These are summarised by Few as follows: nominal comparison, time series,

ranking, part-to-whole, deviation, frequency distribution and correlation.

In the present case of scrutineering, care must always be taken when selecting the interaction

concepts and the right graphs to ensure that deviations from the norm can be detected

quickly and clearly. This essential requirement was defined at the beginning of the project, as

explained in section 1.2.3. To fulfil this requirement, the design of the attributes of a

dashboard must be adapted to the conditions of human visual perception as discussed in

chapter 3.1.3. Preattentive attributes, which humans perceive partly unconsciously, are

beneficial in this context and lead to a faster, more efficient perception. The two most

important characteristics for visualising and perceiving quantitative data are line length or the

length of a shape that resembles a line with a small width, such as bars and boxes, and the

2D position of elements. To enable the user to concentrate on the essentials, care should be

taken to arrange the individual components according to their meaning. The most important

components should be at the top left, and the least important at the bottom right as the study

in chapter 3.1.2 has shown. However, since the scrutineers must observe the same data for

each car, the individual components might have the same importance. As Few has described,

the repetitive repetition of elements can even help the user to apply the same recognition

mechanisms and thus process visualisations faster.

When it comes to the design of the individual components, care should be taken that colours

are used thoughtfully. Intense, saturated colours should only be used to highlight data that

requires attention. Colours that frequently occur in nature are more pleasant for the viewer

and are therefore suitable as a standard colour palette. In general, care should be taken to

use most of the ink to visualise the data and to avoid flourishes and embellishments. In

addition, it is important to note that a dashboard should never be scrollable. Users tend to

declare data that is not immediately visible as unimportant.

Usability and Technology

The results derived in this chapter will form the basis for creating the dashboards and their

components. The aim is to realise a dashboard that offers users a user-friendly way to analyse

telemetry data effectively and appropriately.

3.2. Web-Technologies and Performance

To understand how to improve performance for visualising large amounts of data in a browser,

it is essential to understand what performance means in the context of Data Visualisation.

This is explained in chapter 3.2.1. Chapter 3.2.2 provides information about basic web-page

rendering technologies. Limitations and bottlenecks are described in section 3.2.2.1.

Furthermore, technologies that improve the performance of visualising big data on the web

are discussed in chapter 3.2.3 and 3.2.4. Finally, chapter 3.2.5 explains concepts for avoiding

data overload in the visualisation pipeline.

3.2.1. Performance in Data Visualisation

Website performance measures how quickly the pages of a website are loaded and displayed

in the web-browser. In the field of data visualisation, performance essentially means how fast

the visualisation solution is in visualising huge amounts of data.

Performance is required to render all data smoothly and present it in a comprehensive visual

format. This is closely linked to an improved user experience when interacting with the

visualisation. Two common terms are used to describe performance in web data

visualisations: Frames Per Second (FPS) and rendering time (cf. Juviler, 2021).

FPS is a metric that indicates how many times an image is rendered or drawn in one second.

For data visualisations, this means how smoothly the charts respond when zooming, panning

or other human-computer interactions occur. At 60 FPS, smooth interaction can be expected.

At FPS values below 20, mouse interaction feels sluggish to the user as the image is rendered

with a delay.

Rendering time is used to measure how long it takes to render all elements. For chart libraries,

it means how quickly the data is displayed on the screen after it was added to a chart.

Slow display of data, lags or even crashes of the visualisation lead to user frustration. There

is no definitive benchmark for how fast a visualisation needs to be before the user is frustrated.

However, there is a consensus in the industry that an application should not take longer than

three seconds to load (cf. Juviler, 2021).

Usability and Technology

3.2.2. Render functionality of web-browsers

This chapter explains how browsers render HTML and CSS code via the rendering engine

and display it in the browser window.

The browser's rendering engine creates a document object model tree (DOM tree) from an

HTML file. A DOM tree translates an HTML or XML document into a tree structure of node

objects.

This defines the logical structure of the documents and defines how the nodes of a document

relate to each other. Each node is an object such as a paragraph of text, a heading, or an

SVG-element. The rendering engine creates a CSS Object Model (CSSOM) from the CSS file.

The CSSOM defines how the HTML objects are styled (cf. Ilya Grigorik, 2018).

The rendering engine assembles the independent trees into the "render tree" and renders the

content. Before the content can be displayed, the display area is mathematically calculated

and then painted in the browser. If either the DOM or the CSSOM is changed, the process

just described must be repeated to find out which pixels on the screen need to be re-rendered

(cf. Ilya Grigorik, 2018).

3.2.2.1. DOM-Rendering and Limits

The described rendering process is called “Retained Mode” and is visualised in Figure 12 (cf.

Perna, 2021). In detail, the rendering of the render tree is handled by the browser's rendering

engine which uses the inbuild Graphic API for this process. Application programming

interfaces (APIs) are constructs provided in programming languages to make it easier for

developers to create complex functions. They abstract more complex code and instead

provide a simpler syntax (cf. MDN contributors, 2022).

The Graphic API creates an in-memory model (often called a scene) from the input that

determines how the final output should look like. The time needed to render the render tree

depends on the size of the document, the number of objects, the styles applied and the device

on which it runs. The larger the document, the more work the browser has, and the longer the

browser needs to render (cf. Perna, 2021).

Usability and Technology

Figure 12: Retained mode rendering

When visualising and drawing graphic elements and shapes in the browser, SVG-elements

are mainly used. They are part of the DOM like a div-element and have the advantage of being

vector-based. Therefore, they are visually clear and smaller in file size than raster graphics

such as JPGs. In addition, SVG-elements can be styled with CSS and have a wide range of

pre-configured shapes, such as a circle or a polygon. Figure 13 shows how to create a yellow

circle with a green border in a simple and understandable syntax, styled by CSS properties

(cf. Coyier, 2019).

Figure 13: SVG Syntax and outcome

SVG is convenient to code and therefore suitable when simple graphics need to be rendered.

However, as explained above, the drawing and redrawing of elements are handled

automatically by the browser's Graphic API. This means the developer has no control over

how the engine renders elements. This makes it less flexible for developers to improve

rendering performance but on the other hand reduces a lot of complex rendering

mechanisms. With many complex objects, the rendering time increases enormously because

the browser's memory is limited (cf. Wan, 2019). This behaviour is illustrated in Figure 14. The

figure shows a benchmark test in which an increasing number of SVG-objects were rendered.

The experiment was run on Google Chrome v73 on a 2018 MacBook Pro with an Intel Iris

Plus graphics card by the Frontend Monitoring company LogRocket. The graph shows the

time needed to render a number of objects in milliseconds. To draw 1000 SVG-objects, the

Usability and Technology

browser needs approximately ten milliseconds. The rendering time increases exponentially

with an increasing number of objects.

Figure 14: Render time with varying number of objects for SVG and Canvas. (Image taken by (Wan,

2019))

The benchmark test also introduces a technology called Canvas that significantly outperforms

the rendering time of SVGs as the number of objects increases. This technology will be

explained in detail in the next chapter.

3.2.3. Canvas and WebGL

The Canvas API is part of the HTML5 declaration. The Canvas API enables the drawing of

graphics via JavaScript and the HTML element <canvas> (as seen in Figure 16). It is a high-

level abstraction focused on 2D graphics that provides direct support for rendering

sophisticated graphical elements, e.g., lines, all types of polygons and shapes, colour

gradients, and filters such as blur. Canvas provides the developer a finer grained control over

rendering than with SVG (cf. Perna, 2021).

Another possibility to draw elements in the browser is the WebGL (Web Graphics Library)

technology. It is a JavaScript API for rendering interactive 3D and 2D graphics with high

performance. Compared to the Canvas API, it is a much lower-level API, which means that a

developer must code the rasterization of elements himself. This means, that the API does not

provide any preconfigured sophisticated graphic elements like it is the case in the Canvas

API. The user must supply the code that creates lines, surfaces, and effects such as light and

shadows. WebGL supplies an API that can be used in HTML <canvas> element (cf. Caballero,

2011).

Usability and Technology

3.2.3.1. Rendering of Canvas and WebGL

The initial rendering process is the same in both technologies. They both use the immediate

mode to render shapes in the browser. In an immediate mode system, as seen in Figure 15,

the developer handles the entire scene himself instead of the browsers Graphic API. This

means that the developer needs to specify the drawing commands, so the browser knows

what to update. Therefore, the developer not only specifies what needs to be drawn but also

creates and maintains the model himself. The browser's Graphic API now only takes the

drawing commands and sends them to the browser for execution (cf. Perna, 2021).

Figure 15: Immediate Mode rendering

This means that the rendering pipeline can be customised in much more detail by the

developer, which offers the opportunity to improve rendering performance, but also leads to

more complex code. The same circle as illustrated in Figure 13 in the SVG syntax, is much

more complex to develop with the Canvas syntax, as shown in Figure 16:

Figure 16: Canvas Syntax and outcome

Usability and Technology

In the first line, the canvas element is created and queried in the script in line four. Afterwards

the 2D rendering context is created. Furthermore:

• beginPath() creates a buffer

• arc() generates the points for triangles that make a circle

• fill() takes the points in that internal buffer, generates the correct set of triangles for

the points in that internal buffer into a GL buffer, calls drawArrays() or drawElements()

to draw the circle

It is important to note that Canvas does not cache the buffer. Fill(), the function that does

the actual work does not know what is in the internal buffer of points. Fill() needs to always

look at all the points. This means that if more circles are drawn, all steps must be repeated

each time.

In WebGL, even more code is required to create a simple yellow circle with a green border

because it is a lower-level API than Canvas as mentioned above. A detailed illustration of the

necessary code can be found in the appendix A.1. To reduce the complexity of the code, it

can be simplified as follows:

• User-defined shaders must be written.

• A buffer must be coded and filled with the triangles for a circle (this already saves time

because the intermediate buffer with the points is skipped).

• The functions drawArrays() or drawElements() is called to draw the circle.

WebGL allows the developer to control the rendering pipeline at a much lower level, skipping

steps that Canvas cannot. It also reuses data that Canvas cannot reuse. For example, if

100,000 circles need to be drawn in WebGL, either the points for 100,000 circles can be

generated at once, or instantiation can be used. Both techniques are much faster than in

Canvas, where arc() has to be called 100,000 times and the points for 100,000 circles have

to be generated in every single frame instead of just once at the beginning.

3.2.3.2. Performance: Canvas vs. WebGL

To understand the performance differences between Canvas and WebGL, it is beneficial to

know the basics of a CPU (Central Processing Unit) and a GPU (Graphic Processing Unit).

Graphic visualisation processes typically require many simultaneous and complex

mathematical calculations, which become too power-intensive for a CPU at a certain point.

CPUs are designed to perform various tasks quickly but are limited in processing concurrent

tasks. This means that if the process is fully occupied by one task, no resource is available

Usability and Technology

for any other task. The solution to these complex tasks has led to the development of the

GPU technology. Nowadays, a GPU is usually built into all PC processors. It relieves the load

on the CPU and is designed for complex rendering processes of graphic objects while

executing them simultaneously (cf. Johns, 2020).

Canvas normally uses the CPU but can be configured also to use the GPU of the hardware

where it is executed. However, this is unreliable because this depends on the browser

heuristics, whether a particular canvas is a CPU canvas or a GPU canvas. These can change

from browser to browser and even from browser version to browser version. In one version,

the browser might decide that a particular canvas should be a GPU canvas. In another

version, it might decide the opposite (cf. Johns, 2020). On the other hand, WebGL always

uses the GPU of the used hardware.

To investigate the performance differences between Canvas and WebGL, a benchmark test

was conducted in which a graph with 100,000 data points was created comparing the CPU

Canvas API against the GPU WebGL API. The CPU Canvas was chosen because in a real

scenario - as just explained - it cannot be guaranteed that a GPU Canvas will be used in every

browser.

In the benchmark test, the quantitative data points were simulated randomly in a range from

0 to 1000 and are related to a time axis that increases in time. The experiment was conducted

on a MacBook Pro 2020 with 16GB RAM and a M1 chip and yielded the following results as

seen in Figure 17.

Figure 17: Performance Test Canvas vs. WebGL

Usability and Technology

As the graphic shows, the initial loading of the CPU Canvas (~0,06ms) is much faster than the

initial loading of the WebGL Canvas (~7,9ms). However, once the WebGL Canvas is loaded,

the drawing command is executed extremely fast (~0,1ms) compared to the Canvas drawing

command (~4,8ms). This allows the WebGL diagram to be re-rendered faster during zoom or

panning interactions. Horak et al. observed similar results in their experiment comparing the

rendering performance of Canvas, WebGL and SVG. The outcome can be seen in Figure 18.

They paid attention to the FPS (Frames per second) during zoom and pan interactions, as an

initial longer loading time can still be acceptable. Whereas a slow or even delayed interaction

with an interface cannot. Consequently, FPS can more accurately represent the perceived

effects for the user while analysing huge amount of data (cf. Horak et al., 2018).

Figure 18: Performance Test SVG, Canvas and WebGL regarding FPS (Image taken by (Horak et al.,

2018))

The measurement shows that performance degradation occurs above 400 nodes (about

8,000 elements) for all three technologies. SVG and Canvas perform almost equally.

Nevertheless, the FPS drop only occurs with WebGL when text is displayed in the WebGL

node. Without text elements, the FPS is not affected by the number of nodes; even in an

extreme setup with 400,000 nodes (about 8 million elements). The reason for the FPS drop

with the text is that in WebGL text is rendered as an image (cf. Horak et al., 2018).

However, it must be mentioned that browsers currently have a limit of 16 WebGL contexts.

Mobile browsers have a limit of eight WebGL contexts. This can be circumvented by using

for example a single WebGL canvas that covers the page. Placeholder elements can then be

used to mark where elements should be drawn.

Usability and Technology

3.2.4. JavaScript vs. Web Assembly

When it comes to web-applications performance, not only the actual rendering of the

visualisation is important, but also the overall performance of the web-application. In this

chapter, two technologies that can be used on the web for dynamic content are compared in

terms of their performance.

JavaScript has been used in web browsers since 1995. It is a scripting language that allows

to implement complex functions on web pages. Any time a web page does more than

displaying static information, JavaScript is most likely involved.

JavaScript is an interpreted language that is executed in the browser with a “Just In Time

Compiler”. This means that the human readable code is compiled into a faster binary format

which is understandable to the computer. This happens while the script is being processed,

which slows down the execution process considerably (cf. Gardón, 2022).

WebAssembly (WASM) is a low-level compiled language that can run in all major browsers. It

is compiled down to binary code beforehand and is therefore fast to fetch for browsers and

fast in runtime and execution speed. Moreover, it allows source code written in other

programming languages such as C/C++, C#, and Rust to be compiled and executed on the

web. Performance depends on various factors, such as the browser and the complexity of

the code. (cf. Durmus, 2022).

Most benchmarks indicate that WASM is significantly faster than JavaScript, but only when it

comes to complicated computations. In a study conducted by Aaron Turner, he found that

WASM is about 30 percent faster than JavaScript on desktop Chrome and 90 percent faster

than JavaScript on desktop Firefox for WASM’s intended use case (cf. Turner, 2020).

In a benchmark test Ashle Peacock conducted, WASM was faster when he calculated the

factorial of a given number. By sorting arrays and cubing numbers, JavaScript was faster (cf.

Peacock, 2022). Similar observations were conducted by Durmus: In his case WASM was

only faster when he ran complicated algorithms (cf. Durmus, 2022).

This is because every time a function is called in WASM, it must be loaded into memory in a

WASM instance, which produces a significant overhead (cf. Peacock, 2022). In contrast to

the execution of JavaScript code, where the execution can begin immediately, with WASM

the WASM modules must first be loaded. Only then the modules can be executed. This means

that JavaScript is faster when calculations need to be performed that are not complicated.

However, if the calculation has a certain complexity, it is faster to load the WASM modules

first and use their speed of the previously compiled binary code.

Usability and Technology

In the case of visualisation this means, that WASM only makes sense when there are

complicated calculations and data processing mechanism in place. Otherwise, JavaScript is

faster.

3.2.5. Data Pre-Processing

All performance is useless when the resulting graph looks cluttered because too many data

points are presented. Chapter 3.1.5 introduces two design methods for dealing with data

overload. These approaches are applied during the visual mapping step of the visualisation

pipeline, creating a manipulable representation.

This chapter deals with the problem from a technical point of view and intervenes one step

before. This means that a transformable view is created by data that is pre-processed in the

visualisation pipeline during the transformation step.

From a technical point of view, data overload occurs when more points are rendered than

given pixels are available. This results in high density and provides limited information to the

user, as seen in Figure 19. Therefore, fluctuations can be challenging to determine because

multiple data points are plotted on the same vertical pixel line.

Figure 19: Data overload in line diagrams

When transforming data in the visualisation pipeline, some necessary actions must be taken

to avoid overloading the diagrams. For some data sets, it may be acceptable to filter data by

averaging some data points. This can be achieved with common aggregation methods.

Alternatively, down-sampling algorithms can be used to return a subset of the data points

present in the original data. This chapter introduces and compares different methods and

algorithms.

3.2.5.1. Aggregation functions

For time series data, aggregation methods like summarising, averaging, maximising values,

minimising values or data binning are common to use. In this thesis, the focus lies on the

average and binning methods. This is because the aim is to obtain a transformed visualisation

Usability and Technology

that corresponds to the original data set as close as possible and does not calculate a single

extreme value such as the maximum or the minimum value.

Average Aggregation

An average function returns the arithmetic mean of all input values given to it (cf. Adam Hayes,

2022). First, the points of the series are interpolated to subsequently apply the aggregation

function. The average function then creates so-called buckets. For each bucket a median of

the points is calculated. However, the user does not have the option of specifying the bucket

size. The result is seen in Figure 20, where the average function is visualised (black) and

compared with the full dataset (light grey). The aggregated dataset has 2,100 data points left

from original 7,800 data points.

Figure 20: Full dataset (light grey) vs. Average Aggregation Method (black)

The most obvious problem with this method is that the median calculation is very likely to

exclude local maximums and minimums. Another problem is that the data is distorted

because new values are calculated for the mean calculation in each bucket.

Data Binning Aggregation

Data Binning can reduce the problem of excluding local maximums and minimums by giving

the user the possibility to adjust the bucket size. Like the average aggregation, the binning

technique reduces the original data points that fall within a specific bucket, by replacing the

representative value of each bucket with a median value as sketched in Figure 12.

Figure 21: Bucketing of data. Light grey: Full dataset; Black: Median of buckets

Usability and Technology

In time series data, the sizes of the buckets are usually defined by time intervals that are

dynamically determined by a simple calculation. If only as many data points need to be plotted

as pixels can be displayed on the horizontal axis, the following calculation can be used:

𝐵𝑖𝑛𝑆𝑖𝑧𝑒 = 𝑇𝑖𝑚𝑒𝐼𝑛𝑡𝑒𝑟𝑣𝑎𝑙 =𝑇𝑖𝑚𝑒𝑅𝑎𝑛𝑔𝑒

𝑊𝑖𝑑𝑡ℎ𝑜𝑓𝑐𝑎𝑛𝑣𝑎𝑠𝑖𝑛𝑝𝑥

This ensures that data overload is avoided. Nevertheless, it also means that the larger the

time range, the more points fall into a bucket and the more likely it is that the median

calculation will factor out local maximums and minimums. The smaller the time range is, the

smaller the buckets are, which means more accuracy in the medians of the values in the

buckets and less factoring out of local peaks and troughs.

However, this technique might lead to performance issues when interacting with the graph.

To ensure that the bucket sizes correspond to the user's desired time interval, a new data

query must be triggered for each zoom or pan interaction. In an extensive dataset, querying

data and recalculating the median value of each bucket needs some time and slows down

the visualisation. Moreover, data binning still has the problem that new values are generated,

as the mean value of all data points in a bucket is calculated.

3.2.5.2. Algorithms

A better approach might be to use algorithms that return a subset of data points existing in

the original data. A choice can be made between simpler intuitive algorithms and more

complex algorithms. The common feature is that both categories, like the aggregation

functions, use the general concept of bucketing.

Intuitive Algorithm: Mode-Median-Bucket

Mode-Median-Bucket is a simple algorithm. Each bucket is examined for the frequency of the

y-values. If there is a single most frequent value (mode) in the bucket, that y-value is selected

to represent it. If there are multiple values with the same frequency, a tie must be established

to select a mode. However, if there is no mode, the median is taken. Again, one of the most

obvious problems with this algorithm is that it is very likely to factor out local peaks and

troughs within a bucket. This is because the algorithm does not consider how high the y-value

is but only how often it occurs within each bucket. Also, the performance of this algorithm is

not very good because the object that tracks the modes must be constantly changed (cf.

Steinarsson, 2013, p. 5 ff.).

Usability and Technology

Complex Algorithm: Largest-Triangle-Three-Bucket:

The Largest-Triangle-Three-Bucket (LTTB) algorithm is more complex. The idea is to keep

only the points that have the most visually important impact. Therefore, a cartography

approach is used where triangles are formed between neighbouring data points. The size of

those triangles is used to determine the perceived importance of each point. This means that

the algorithm aims to determine the point in a bucket that forms the largest triangle when two

pre-determined points are given. Since the importance of a point is determined by the size of

its effective area, extreme values naturally have a higher probability of being selected. The

outcome is illustrated in Figure 22. While local peaks were still eliminated in the average

aggregation method, the peaks in the LTTB algorithm are almost congruent with the original

data set. The same initial dataset (~7,800 points) as in Figure 20 was used. The threshold for

the LTTB algorithm was set at 2,100 data points to show the effectiveness and accuracy over

the average aggregation method.

Figure 22: Full dataset (light grey) vs. LTTB algorithm (black)

In detail the algorithm works as follows: The data set is divided into equal-sized buckets which

are determined through a fixed number of output samples that should be shown (threshold).

Then, two predetermined points are selected from the buckets before and after the one from

which the point needs to be calculated. The first point is the one representing the bucket

before the current one. The second point is a "ghost point" representing the next bucket,

which is simply an average of all the points in the bucket. If one were to take a brute force

approach, the size of the triangle with all the points in the next bucket needs to be calculated.

This would make the algorithm inefficient. The average method has therefore proven itself

and seems to work just as well as the brute force approach but is much more efficient. The

point in the current area forming the largest triangle with the two fixed points in the adjacent

areas is selected (cf. Steinarsson, 2013, p. 19 ff.). Figure 23 shows how point B forms the

largest triangle over the areas with fixed point A (previously selected) and temporary point C.

Usability and Technology

Figure 23: Largest triangle across three buckets and point C being a temporary point (Image taken by

(Steinarsson, 2013, p. 22))

An advantage of this approach is that a threshold value for a dataset can be used at the

beginning, allowing zooming, and panning without re-querying of the data, without

compromising the correctness and accuracy of the data. Furthermore, no new values are

created with this algorithm. Only points from the original data set are selected.

3.2.6. Placing the technical results in the overall context

This chapter has given a detailed overview of the performance of web browsers. Different

technologies and rendering options in the browser were discussed, as well as different

programming languages and methods. In addition, algorithms, and aggregation functions to

avoid data overload were examined from different angles.

When it comes to the performance of dashboard visualisation, two attributes are particularly

important: the rendering time and the FPS. It is stated in the industry, that a visualisation that

takes longer than three seconds to render will frustrate the user. An FPS number below 20

makes the visualisation appear sluggish while interacting with it and should therefore be

avoided.

To integrate the gained knowledge into the overall concept, the following considerations

should be made: As research has shown, SVG-elements are unsuitable for developing

performant visualisations on the web. While this technology is user-friendly and easy to

develop, it suffers from significant performance degradation due to the standardized

rendering technique of browsers, as discussed in chapter 3.2.2. It is therefore more common

to use the Canvas API or WebGL API. Through these technologies, a developer has the

possibility to control the rendering process more precisely and thus make it more performant.

In addition, the APIs can access the CPU and GPU of the executing machine, unlike

Usability and Technology

conventional methods, which also results in better performance. Accordingly, it is advisable

to use a library that uses one of these APIs. Therefore, the following considerations should

be taken into account: A first benchmark test in section 3.2.3.2 has shown that the Canvas

API is faster at start-up but has a deficit when rendering many objects. Rendering time

increases steadily the more objects are drawn. In addition, the FPS decreases as the number

of objects increases, leading to a delay in interactions and thus to user frustration.

With WebGL, the initial start-up time takes longer. However, the rendering time and FPS

remain constant even when more objects are rendered. As Horak et al. have pointed out in

section 3.2.3.2, an initial longer rendering time that is still within a tolerable time range is not

as frustrating for the user as a visualisation that is unusable due to a poor FPS performance.

WebGL seems to be therefore the best choice. However, this behaviour must be tested more

deeply in the practical part and in close connection with a possible library that corresponds

to the requirements which will be discussed in chapter 4.4.1 and 4.5.1.

To avoid possible data overload, the use of an algorithm should be considered. This is

accomplished by an early intervention in the visualisation pipeline, namely during the data

transformation step. Thus, this can support the design methods discussed in chapter 3.1.5,

which also aim to reduce data overload in data visualisations. Care should be taken that the

algorithm does not create new average points and does not exclude highs and lows from the

visualisation. The Largest-Triangle-Three-Bucket algorithm presented in section 3.2.5.2 could

be used for this purpose. This algorithm is designed to reduce the original data set in a way

that preserves the extreme values and does not calculate new mean points. However, further

user testing is needed to determine which algorithm is most useful for a potential end-user.

Finally, in combination with the decisions made before, it should be decided whether the use

of WASM technology is beneficial. Based on the findings from chapter 3.2.4, this only seems

to be useful when complex algorithm calculations for data processing need to be done.

The following part of this thesis describes the steps for creating individual graphical

components in detail. In doing so, the usability findings are considered and combined with

the performance possibilities.

Practical Work

4. Practical Work

In this chapter, the conception of the practical work is presented. First, the use case is

explained in chapter 4.1. Chapter 4.2 discusses the applied methods. Then the chapter is

divided into a Usability and a Performance and Technology section. Chapter 4.4 deals with

the conception and design of the prototype. In chapter 4.5, various technologies were

evaluated and considered for a prototypical implementation.

4.1. Use case

As described in chapter 1, telemetry dashboards are used, among other things, for

scrutineering in motorsport. Scrutineering aims to ensure the safety of all cars and to detect

and report violations of the vehicles during the race.

There are five telemetry sensor categories that are of interest to the scrutineers: Tyre

Pressure, Fuel, High-Voltage Systems, and High-Voltage Monitoring. Each dashboard

contains graphical components for one sensor category for all race cars that are eligible for

the race. Therefore, each scrutineer is responsible for one category and thus observes the

corresponding dashboard. For this reason, it is crucial for the scrutineers to have a clear and

comprehensible overview and to be able to identify violations and deviations from the norm

quickly.

This master’s thesis focuses on the tyre pressure sensor category. The reason for this is that

this sensor category is exemplary for the characteristics of scrutineer telemetry data. Binary

sensor data is sent first, stating whether the tyre pressure is within a specified state. A

distinction is made between whether the pressure is in a good state, in a still acceptable but

slightly critical state or in a critical state. This data is sent for all four tyres. The exact values

for each tyre pressure are then sent in a further data record.

However, it is not only the direct practical benefit that is important for this thesis. The

prototype should give an overview of how web-based technologies can be used to visualise

large amounts of data on the web. It is also intended to provide a foundation for further

development, showing which design methods can be used to meet the specific requirements

of technical scrutineering.

Practical Work

4.2. Methods

Chapter 1.2.3 already explained the basic requirements for the prototype. This chapter will

take a closer look at the methods that contributed to the collection of the usability and

technical requirements. The requirements were defined at the start of the thesis and served

as authoritative guidelines.

To gain further insights into the matter, expert interviews as well as a contextual inquiry were

conducted at the beginning of the research for this thesis. Both took place at the 24 Hours of

Le Mans 2022 race. The obtained insights lay the foundation for all further considerations in

this master’s thesis.

4.2.1. Expert Interviews

An expert interview is a UX research method in which a researcher asks user questions about

a specific topic with the aim of learning more about the user’s opinion about or experience

with it. UX interviews are usually used in lean and agile environments. They provide insights

into what users think about a website, application, product, or process. They can therefore

reveal, what users consider important and what suggestions they have for improvement (cf.

Pernice, 2018). Since the topic of telemetry in motorsport is very complex and a deeper

understanding of how to deal with telemetry data and the processes was missing at the

beginning, this method was chosen as a suitable way to gain initial insights into the work and

processes of a telemetry engineer.

In the present case, two expert interviews were conducted. During the interview, the answers

were written down in bullet points and extra notes were taken. The first one was with the

Telemetry Senior Application Engineer from Bosch Motorsport who is responsible for defining

the dashboard architecture and the overall requirements. He has a broad overview of the

entire telemetry data, their preparation, and their characteristics as well as the overall

processes which are connected to telemetry dashboards. He is also in daily contact with

various engineers and knows their frustrations, pain points and wishes regarding the telemetry

data and the respective dashboards. A second expert interview was conducted with a

telemetry engineer from Corvette who is responsible for the telemetry monitoring of the

Chevrolet Corvette C8.R during races. She was chosen because she works with different

telemetry dashboards on a daily basis and therefore knows the advantages and

disadvantages of already existing telemetry dashboards. She knows how to work with

telemetry data and understands the processes and the frustrations and benefits that come

Practical Work

with it. She can give first-hand insights and is ultimately part of the addressed user group

working with the final telemetry dashboards.

The main goal of the interviews was to get a first impression of the topic and to collect the

most important requirements. They helped to understand what kind of data is most important

to the engineers and how they work with the data. In the interviews, frustrations that arose

with past solutions were debated and initial approaches to solving them were discussed. In

addition, the interviews were used to ask about specific tasks, such as which diagram types

they use most often and why or which interactions they perform most. Moreover, it was

discussed which concepts and interactions they think would help them to work more

efficiently. Questions about general processes were asked, such as when graphs need to be

created. Also discussed was what the process is like from vehicle selection to detailed post-

race debriefing. Special care was taken to delve into the processes and tasks that the

engineers considered particularly important. This was supported by open-ended and follow-

up questions. A detailed account of the interviews can be found in the appendix A.2.

4.2.2. Contextual Inquiry Method

The contextual inquiry method is a method for field studies where users are observed and

explain loudly how and why they do what they do. This way, a solid understanding of working

methods and behaviours can be gained. The study takes place in the users' natural

environment, where they perform their activities as usual. The method is carried out in the

early stages of a project, as the research data obtained is crucial for designing requirements

and personas. The contextual inquiry method was chosen in addition to expert interviews for

the following reasons: users can more easily talk about what they do and why they do it while

they are doing it. For this reason, contextual inquiry can provide more affluent, and more

relevant information about how users complete processes. Details can be uncovered that the

user has become accustomed to and no longer notices himself. In this way, results from the

interviews can be confirmed, and new hidden insights can be uncovered (cf. Kim Salazar,

2020).

In this case, the method was carried out during the 24 Hours of Le Mans race training. Two

telemetry engineers from the Corvette team were observed each monitoring multiple sensors

from one Corvette race car. This circumstance is not exactly the same as in scrutineering

where an engineer monitors a sensor category of several vehicles at the same time.

Nevertheless, it gives a good impression of how engineers generally work with telemetry

dashboards.

Practical Work

It was observed that the engineers often switch back and forth between different previously

created dashboards, which were not mentioned in the interviews. When asked, the reason

was that the graphs and data were too detailed and therefore did not all fit on one dashboard.

It was also observed that one engineer repeatedly compared both cars, although he was only

responsible for one car. The reason given for this was that comparing the cars is essential to

understand why the car under observation is slower than the other car of the same type in

certain situations. This way, instructions could be given on what the other driver should be

doing differently. However, quickly adding the other car with the same data to the dashboard

was very complex and frustrated the user. Moreover, it was analysed how an engineer selects

the cars, and the sensors he must monitor. It was observed how he aligned the components

on the dashboards, which components were the most important, which chart types he used

and how he interacted with them individually and with the whole dashboard. Special attention

was paid to which tasks were performed most frequently and which mechanisms and

procedures were used.

From the methods just described, the gathered insights provided an overall understanding.

Personas and user stories were derived defining usability and technical requirements. These

were then weighted and thus assigned a specific importance (1: less important, 3: more

important) as seen in Table 2 and Table 3 in chapter 4.4.1 and 4.5.1.

4.2.3. Design Thinking

Design thinking is a creative, iterative approach that applies design perspectives and

processes to problems. Design thinking looks for innovative solutions to complex problems

that are technologically realizable, desirable for the end user and practical and cost effective

for the company. Design thinking uses an iterative process in five steps that can be described

as follows (cf. Browne, 2021):

• Empathy phase: personifying the end users to understand their challenges and needs.

• Defining phase: identifying end-user problems to be solved.

• Ideating phase: selecting solutions to be prototyped.

• Prototyping phase: prototyping solutions; initially with a low level of detail, but with

increasing levels of detail through iterations.

• Testing phase: testing solution prototypes with end users.

This approach was preferred to other creative approaches such as the User Centred Design

process, as Design Thinking not only considers a comprehensive knowledge of the user, but

also incorporates the feasibility and business objectives.

Practical Work

In this thesis, the iterative process was used to find innovative solutions for the development

of the prototypes. First, the needs and problems of the users were identified. The information

gathered through the expert interviews and the contextual inquiry method was crucial for

defining the requirements. Concepts and ideas for the implementation of the prototypes were

then designed from which prototypes were developed in an iterative process. These were

tested, discussed internally and then revised and supplemented with new ideas.

4.3. Personas and User Stories

Two user personas, Tom, and Liz were developed from the expert interviews and the

contextual inquiry method. Care was taken to ensure that different characteristics were

represented which were observed during the contextual inquiry. Tom, for example, is a very

disciplined, precise, detail-oriented, and a structural person with an affinity for cars, as seen

in the persona in Figure 24. However, he is hesitant to learn new technologies and

applications and wants to keep things as he knows them. Liz, on the other hand, comes from

a computer science background and is very technically minded. She likes to think her way

into new applications and tries out innovative and novel things. Furthermore, she is interested

in artificial intelligence in the field of data visualisation. She is satisfied when things work as

expected and are visually appealing. She is more interested in data structures and how these

get collected and analysed than in the actual cars (see her persona in the appendix A.3).

Figure 24: Persona Tom

Practical Work

A total of five user stories were mapped from the two personas:

“As a user of telemetry dashboards, I want to be notified immediately and clearly when a

vehicle deviates from the, so that I can act quickly and take the next steps to ensure that the

race can continue without any problems.”

“As a user of telemetry dashboards, I want to be able to analyse the data in detail to

understand what happened to the car and why it happened, while keeping a good overview

of all the other cars so that I don't lose sight of the big picture and can act quickly if another

car is in violation with the rules.”

“As a user of telemetry dashboards, I want to be able to get the data as accurately and as

quickly as possible, but also to be able to retrieve it later, so that I have a good overview of

the race and can understand and evaluate which cars were violating the rules.”

“As a user of telemetry dashboards, I want to use innovative and intelligent algorithms that

support me to easily analyse the data and make accurate predictions and forecasts

depending on the history of the data.”

“As a user of telemetry dashboards, I want to get the data pre-processed by artificial

intelligence, so only the data which is necessary for me is displayed.”

4.4. Usability Conception

This chapter describes the design process and the conception of the dashboard. Chapter

4.4.1 defines the usability requirements. The conception of the dashboard and the process of

creating the low-fidelity prototypes are discussed in detail in chapter 4.4.2. Section 4.4.3

explains the considerations and the elaboration of the design for the final dashboard.

4.4.1. Usability Requirements

This chapter discusses the functional usability requirements. They were defined using the

methods described in section 4.2 and served as guidelines for the following conceptions in

section 4.4.2. Table 2 gives an overview of the design requirements for the telemetry

components and dashboards. The usability requirements were divided into graphical

components and dashboard. Graphical components are the actual visualisation components

and parts like selecting boxes. The dashboard refers to the surrounding basic framework and

its overall functionality. A detailed requirements catalogue with acceptance criteria for the

individual requirements can be found in the appendix of this thesis (see A.4).

Practical Work

Table 2: Functional Design Requirements

Nr.

Title

Explanation

Weighting

Graphical Components

Space-saving

visualisations

Graphs visualisations should be designed space

savingly.

Flexible

visualisation

possibilities

Graphs should be as flexible as possible, e.g.,

combining line graphs with status graphs or bar graphs

if necessary. Graphs should also have two y-axis

support.

Overview and

Context

The system must give the user the possibility to change

fast between a contextual overview and a detailed view.

Visualisation of

violations and

deviations

The user must be able to clearly and quickly see when

a car does not comply with the rules or is in a critical

situation.

Compare and

focus on data

The user must be able to compare different data with

each other through a well-thought-out interface and

hide datasets if needed.

Lap distance and

time axis

The user must be able to see the time and lap distance

on the x-axis and select specific time ranges within the

selected range.

Dashboard

Changing between

dashboards

The system should allow the user to change quickly

between different dashboards.

Intuitive Multi-

Select-Box

The system must allow for selecting different cars when

opening the dashboard. All cars must be always

selectable.

Notifications

The system must notify the user when the race is under

caution.

Interaction

concept: Zooming

and Panning

The user must be able to easily navigate through the

datasets by using user-friendly interaction concepts.

Clarity and user-

friendly

arrangement

The individual components must be arranged in a way

that lets the users get a clear overview and does not

overwhelm them.

Selecting Time or

Lap

The user must be able to select a specific lap and time

interval.

Shortcuts

The user must be able to navigate through the

dashboard by using shortcuts for a better user-

experience.

The fundamental requirements formed an important basis for the creation and conception of

the dashboards. The individual implementation steps were carried out and evaluated

regarding the defined requirements.

Practical Work

4.4.2. Conception and Low Fidelity

After the individual requirements and use cases have been explained in more detail, this

chapter will take a closer look at the conception of the low-fidelity prototypes. Special

attention is given to the main implementations and methods used, which were introduced in

the research part. In doing so, they align with the user requirements defined by the interviews

and the contextual inquiry method conducted at the 24 Hours of Le Mans 2022.

4.4.2.1. Graphical Components

As discussed in chapter 4.1, the focus on implementing graphic elements lies on tyre pressure

data. The main goal for the tyre pressure data is to inform the scrutineer that tyre pressure

values are outside the norm.

For this purpose, a concept was developed that uses so-called health-strips for each tyre, as

shown in Figure 25. This makes use of the Focus and Context interaction concept which was

introduced in section 3.1.5.2. In the first design cycle, one strip represents one lap which can

be seen on the lower x-axis. A secondary x-axis with time values is presented on top of the

diagram. The strips slowly fill up per lap and change their colour intensity as soon as the

respective tyre pressure is outside the norm. If the user wants to see a detailed view of each

lap, he can click on the respective strip. The graph then expands and a line diagram with the

exact values appears under the respective strip. The background of the line chart is coloured

according to the value states so the user can quickly see if the line is out of range.

Figure 25: Health-strips Low Fidelity Prototype, first design cycle

However, this design concept has a problem: the user cannot zoom in or out in the health-

strips. The strips change their colour intensity for the whole lap as soon as a violation is

Practical Work

detected. The exact time range of the violation cannot be determined in the health-strip

overview.

Another problem with this concept is that it is difficult to compare the individual values in the

line chart overview. For example, if the user wants to see a detailed view of the left and front

right tyre, he needs to click on both strips to see the detailed values. And even then, a quick

comparison of the data is complicated because they appear in two different line charts above

each other.

As seen in Figure 26, a solution was found for both problems. The health-strips are no longer

mapped per round. They are now designed as a continuous strip that changes its colour

intensity exactly when the violation happens. This allows the user to zoom in and out. To

compare the four different tyre pressure values, the four health-strips can now be toggled into

one line diagram via a button in the upper right corner. In this way, comparisons can be made

easily. Moreover, care was taken to use the effective preattentive attributes of line and boxes

and 2D position described by Few and Mazza explained in chapter 3.1.3.2. This prepares and

visualises the data in a user-friendly and efficient way.

Figure 26: Health-strips Low Fidelity Prototype, second design cycle

In a further design cycle, a solution was sought to make the component more compact. The

aim is to give the user a more precise overview of the component and at the same time make

it more space-saving. This eventually results in more components that can be displayed per

dashboard without overwhelming or distracting the user.

The third concept which was considered combines all four strips into one single strip per car.

As soon as one of the four tyre pressure values is outside the norm, the colour intensity

changes for this period. If the user wants to see more details, he can click on the strip or the

“plus-icon” in the upper right corner. The strip then expands into the view where all four strips

can be monitored at once.

Practical Work

In addition, a button has been added through which the user can display all the values shown

in a table. This should simplify the determination of exact partial values if needed. Here too,

extreme values are marked by colour intensities so that they can be quickly recognised. Figure

27 shows all four states of the diagram.

Figure 27: Health-strips Low Fidelity Prototype, third design cycle

The individual components for several vehicles were arranged in a grid, it became apparent

that the different component heights between the compact view and the other components

contributed to a lack of clarity. For this reason, it was decided to bring all components to the

same height. In addition, the navigation between the individual views was simplified by

replacing the individual icon buttons with a switch button with the corresponding labels: View,

Strips, Line and Table.

Another aspect was the design of multi-select boxes. These can be used to select series,

sensors, or cars. Figure 28 shows an example of a car-select list. After two design cycles, a

hybrid approach with two opposing lists and filter options proved to be the best solution. On

the left-hand list, all available cars are listed. On the right-hand list, the already selected cars

are shown.

This approach was chosen because the list of values is usually very long. The search function

allows the user to search for specific values. This filtering mechanism helps the engineers to

filter the data in the list, which they usually know very well. Therefore, it makes sense to

provide the user with a search field that allows him to filter the desired values quickly and

effectively. At the same time, the user has the possibility to keep an eye on his selection and

can edit it quickly if necessary.

Practical Work

Figure 28: Multi-select box

4.4.2.2. Dashboard

The concept for the individual dashboard forms the framework in which the individual graphic

components are located. Since the dashboards contain the same graphs for each car, a graph

variation with its details can be defined in the settings and is then used for all cars.

In the first design cycle, attention was paid to the fact that the user can select the series and

the cars he wants to observe. In addition, the user can specify the time interval via a date-

time picker as seen in Figure 29.

Figure 29: Dashboard Low Fidelity Prototype, 1st design cycle

However, this design has the disadvantage that the user can only select individual laps in a

cumbersome way. For this reason, a lap-select-box was introduced at the top of the screen

in a second design cycle as seen in Figure 30. This allows the scrutineer to select the

individual laps he wants to monitor. If no lap is selected the whole data is displayed. In a

control centre located at the bottom of the screen, the time range can be adjusted granularly

within the previously selected laps. In order to do that, the user can zoom and pan all the

graphs simultaneously with an intuitive slider interaction. To monitor a smaller range within

the selected laps, the front and the back part of the slider can be pulled together. In addition,

common icons known from music and film streaming can be used to pause or start the

stream. With the rewind and forward icon, the user can jump to the start and end of the

Practical Work

selected time range. The fast backward and forward icon lets the user jump to the beginning

or to the latest recorded data.

On the left site of the control centre, the user can switch the whole dashboard x-axis unit

between distance per lap and time. Next to it, he can switch between the graph types. On the

right site, the user gets notified when the race is under caution.

The user can switch between individual dashboards via a tab menu located above the control

centre. The control centre and the header bar form a coherent framework in which the graphs

can be embedded.

Figure 30: Dashboard Low Fidelity Prototype; Header and Control Centre, 2nd design cycle

4.4.3. Design

As suggested in the research in chapter 3.1.2, the chosen colour palette for the design

contains subtle colours to avoid overwhelming the user during prolonged viewing. The

primary colour is dark blue. All other colours are derived from this primary colour, as seen in

Figure 31. Care has been taken to keep contrasts to a minimum to avoid unnecessary strain

for the user's eyes. This allows the user to concentrate on what is necessary without being

stressed or distracted. An exception is the signal colour red as well as the highlight colour,

which was chosen in light blue. The signal colour indicates to the user that there is a violation

which requires an action and therefore stands out. As soon as this fully saturated colour

appears on the screen, the user's eyes are alerted. This causes the user to assign a meaning

to the contrast as research has shown in chapter 3.1.2. the user can then take a closer look

at the situation and take the necessary steps. The light blue accent colour indicates to the

user that various user actions can be performed as soon as this colour is used. That is why a

contrasting colour was used here as well.

When designing the dashboard, care was taken to use the accent colour only in a few cases

and only where necessary. This is to avoid distraction from the essentials, which is the actual

data.

Practical Work

Figure 31: Colour palette

The individual components were designed using the colour identity. This results in a

dashboard that gives the user a structured and clear overview of the individual graphical

components, as can be seen in Figure 32.

Figure 32: Dashboard Design

The final dashboard design with its components combines the principles that were explained

in detail in chapter 3.1.

As already stated, the general interaction concept is based on the Focus and Context

approach presented in chapter 3.1.5.2. By dividing the tyre pressure data into different states,

the user has a clear overview of the overall context of the data. The data is thus elided to

allow the user to concentrate on the essentials. With this design concept, a method was found

to avoid data overload while still conveying the most important aspects.

Practical Work

The health strips present the values in a visually appealing way. This is by using the

preattentive attribute of line length in the form of boxes that continuously build up and grow

along the x-axis.

The thoughtful use of different colour intensities for the various states of the tyre pressure

values enables the user to efficiently recognise when the values deviate from the norm.

The compression of the four values per car to one value and thus minimising to one health-

strip per car has the goal to simplify the visualisation for the user. This should ensure less

distraction, an even more bundled concentration of the most important aspects and a clearer

overview of the component – especially when there are many components within a

dashboard. These adaptations follow the principle of Eloquence through Simplicity,

described in chapter 3.1.2.

Moreover, the user also can get a very detailed view of the data if needed. First, the user has

the possibility to get all health-strips for each tyre. In the next step, the user is given the

opportunity to analyse the exact values in two different granular detail views. If the user wants

to compare the values over the selected time, he can use the line diagram view. If the user

wants to see individual values, he can use the table overview. The results of the previously

conducted research were used as a guideline.

For the comparison of time series data, line charts with visible data points were used, as

proposed by Few in chapter 3.1.4. However, if the user wants to retrieve individual values, he

can review them in the table view. This might be helpful if he needs a specific value to an

exact millisecond. In addition, the correct context, indicating whether the values are in the

norm or a critical state, is shown by colouring the graph with different colour intensities along

the quantitative y-axis.

Since the elements have the same importance, they have been laid out in an even three-by-

seven grid. Thus, up to 18 cars can be observed at the same time. The sizes of the individual

elements are still sufficient. Moreover, the user can use the same cognitive strategy to decode

the visual components due to the redundant arrangement of the elements.

The elaborate design concept for zooming and panning in the control centre can be

understood as a variation of the Overview and Detail method discussed in chapter 3.1.5.1.

This gives the user the possibility to get a more detailed view. However, in combination with

the lap select box, the user is not losing the overview and always has the context of the overall

data.

Practical Work

In conclusion, it can be said that special attention was paid to simplicity and clarity. Concise,

preattentive attributes were used to guide the user's attention to the most critical aspects of

the data - consciously and subconsciously. Space-saving visualisation components were

designed to give the user an overview of the data and quickly switch to the detailed view if

needed. Care has been taken to limit the entire dashboard to the necessary data elements

required for a general understanding of the data. The objective was to monitor all vehicles at

a glance and recognise when a car violates the rules or deviates from the standard.

The concepts and ideas just explained, which form the basis for the design of the dashboards,

are evaluated, and tested for their suitability in a usability test, which is explained in more

detail in chapter 4.7.

4.5. Performance and Technology

In this chapter, the technical aspects of this thesis are discussed in more detail. Chapter 4.5.1

defines the technical requirements for visual panels for a telemetry dashboard. Chapter 4.5.2

explains the approach and implementation of evaluating different technical solutions to

develop a telemetry dashboard. These are then analysed for robustness and performance in

chapter 4.5.3. The usability and user experience of the previously developed practical

solutions are tested in chapter 4.7 and 4.8 with a usability test and an algorithm survey.

Finally, chapter 4.9 discusses the results obtained in this thesis and presents them in the

overall context.

4.5.1. Technical Requirements

This chapter summarises the technical requirements. On the one hand, these were defined

by the methods described in chapter 4.2. On the other hand, they were determined by Bosch

Engineering GmbH before the start of this thesis. Table 3 gives a collected overview of the

technical requirements for the telemetry components and the surrounding dashboards. They

are divided into Must Have and Nice To Have requirements. Must Have requirements need to

be fulfilled by the chosen visualisation solution. Nice To Have requirements are considered

as optional.

Practical Work

Table 3: Technical Requirements

Nr.

Title

Explanation

Weighting

Must Have

Robustness

The system must be able to handle the huge amount of data

generated by the race cars without freezing or crashing:

• A graph must be able to handle at least one million

datapoints without lagging while user interactions like

zooming or panning

• A dashboard must contain at least 15 graphs

Performance

The system must be able react in a performant way to user

actions:

• FPS should not drop under 20

• The system must support a refresh rate of at least 40Hz

• Loading times should not exceed 3s by plotting 3 million

datapoints

Browser support

The diagrams and visualisation must be executable in the

browser. The application must be compatible for the common

browsers: Google Chrome, Firefox, and Safari in the latest

versions.

Custom

visualisations

User-defined diagrams should be able to be integrated into

the library to be used to fulfil the previously defined design

requirements.

Cloud-based and

flexibility

The application must support a cloud-based architecture and

be able to connect to the existing architecture.

Real-time and

history

The system must support both real-time data, as described in

section 2.2, as well as historical data.

Creating graphs

without code

It must be possible to create the individual diagrams without

the use of code. A query language to request the data is

acceptable and can be used.

Nice To Have

Export reports

The system must support the export of reports from individual

charts.

Authentication

The system must support authentication and provide user

management.

Pre-defined

configurations

The system must support the uploading of diagram

configurations or provide a possibility for the global

configuration of individual diagram types.

In chapter 4.5.3, special attention was paid to the performance and robustness of the system.

All other requirements depend on these two main requirements. If the system cannot fulfil the

basic requirement of handling the amount of data, the solution is unusable which will lead to

user frustration.

Practical Work

4.5.2. Technology Evaluation Matrix

For the evaluation matrix, intensive research was conducted on possible graph libraries

(Canvas and WebGL-based), business intelligence solutions, analytics, and interactive web

visualisation applications. For this purpose, a discussion was held with McLaren, a leading

provider of telemetry dashboards for desktop applications. Various web solutions were

debated, which provided further input for the research and a pre-selection of possible

solutions. To finalise this list, a stress test was carried out to test the performance and

robustness of each solution. An attempt was made to render 100,000 elements as a baseline.

This number was chosen because it had already caused visualisation problems with possible

solutions in previous tests. This stress test minimised the pre-selection to the following

products: Amazon Quicksight, Grafana, SciChart.js, LightningChart JS, μPlot and eCharts.

The requirements defined in chapters 4.4.1 and 4.5.1 were used for the evaluation. In addition,

further quality criteria were defined and used to ensure the quality of the visualisation solution.

They relate to aspects of maintainability, learnability and complexity, community, and

licensing. In detail the aspects are described in Table 4 (cf. Salomaa, 2020).

Table 4: Additional quality features for evaluating technology

Nr.

Title

Explanation

Maintainability

Attention is paid if the visualisation solution is difficult to be

understood or enhanced. How many resources are needed to scale

and/or maintain the solution?

Learnability & Complexity

Attention is paid to whether new programming languages or systems

need to be learned or built from scratch. Consideration is given to

how complex it is to integrate or use the system. This includes the

time and development factor required to develop the solution.

Community

This criterion refers to the size and activity of the community. This is

helpful in the event of arising problems.

Licencing & Supportability

This criterion refers to the security aspect. How easy is it to

understand how the project is run and the development of the

software managed? Is there official support? How does licensing

look like? Evidence of current/future developer support?

Each requirement is assigned to an acceptance criterion. The higher the acceptance criterion

the better. Therefore, four acceptance criteria have been defined and are explained in detail

in Table 5. In the evaluation matrix, the weighted requirements were multiplied by the

weighted acceptance criteria. It can be concluded that the higher the sum of the evaluation

result of the individual solutions, the better it fulfils the user's goals and requirements. As

mentioned earlier, the robustness and performance requirements were considered in a

Practical Work

separate benchmark test, which is described in chapter 4.5.3. The performance requirements

must be fully met. Otherwise, the technology cannot be used.

Table 5: Acceptance Criterion

Acceptance

Criterion

Explanation

Weighting

Acceptable

The requirement has been satisfactorily fulfilled.

Tolerable

The requirement is mostly satisfied.

Undesirable

The requirement is only fulfilled to a small extent and has an

undesirable outcome.

Unacceptable

The requirements are not fulfilled at all.

However, it must be mentioned that AWS Quicksight and Grafana are "platform-ready

solutions" where a certain environment is already given. This means that pre-built

visualisations can be embedded with relatively little effort. The other solutions being evaluated

are graph libraries. While these contain certain chart types, they need to be developed and

configured from scratch so that the end-user has a similar product and user experience as

he has with a platform-ready solution. In addition, the surrounding platform and dashboard

architecture must also be designed and developed. This includes steps such as

authentication, data handling and pre-processing, database connection and user

management. Compared to a platform solution, this has the advantage that requirements can

be implemented much more flexibly and exactly as the user needs it. At the same time,

however, it has the major disadvantage that, in most cases, an enormous amount of

development work must be done for the actual implementation, which ties up developer

resources, time, and budget. These economic requirements could not be fully reflected in the

evaluation matrix. However, these requirements can be crucial for the success or failure of a

project. They represent a high risk that management may be unwilling or unable to invest in.

Nevertheless, each situation must be considered individually. Therefore, this problem will be

discussed in more detail in the result section of the evaluation of the matrix in chapter 4.5.4.

The following sections describe the visualisation solution which were evaluated.

4.5.2.1. Amazon Quicksight

Quicksight, the data visualisation application developed by the Amazon Web Services cloud

provider, is the only business intelligence application which made it to the final selection. The

Amazon QuickSight Business Intelligence utility enables the user to create and analyse

interactive data visualisations to provide insights for decision-making depending on the data.

Practical Work

Visualisations can either be created using a drag-and-drop mechanism or with the in-house

query language. The interactive dashboards can be seamlessly embedded into many

applications, portals, and websites. In addition, users can identify trends as well as make

predictions through built-in machine learning algorithms. The application is comparatively

easy to use and flexible in connecting different data resources. Quicksight thus offers a

complete ecosystem for visualising, analysing and predicting data forecasts (cf. Awati, 2021).

4.5.2.2. Grafana

Grafana is an open-source interactive data visualisation platform developed by Grafana Labs,

that allows users to observe their data in the form of charts and graphs. Grafana is mainly

used to visualise timeseries data. Each dashboard is built from modular panels that can be

connected to several data sources. There are many different prefabricated visualisation

types. However, custom visualisations can also be developed and integrated

programmatically. The data can be queried using a query language and then visualised and

configured in graph panels via a drag-and-drop mechanism. The exact query language

depends on the connected database. Grafana is available as a cloud solution, but it can also

be self-hosted (cf. Grafana, 2022).

4.5.2.3. SciChart.js

SciChart is a Financial & Scientific software company. They create cross-platform charting

libraries for several platforms, including the web. SciChart.js is a JavaScript graphics library

based on WebGL and WebAssembly. The charts are based on a C++ code and compiled with

WASM to JavaScript. They support real-time performance, rich interactive charts and have

up to 40 2D and 3D chart types. However, to build the charts, active coding is required. The

industry leader in telemetry dashboards for desktop applications uses SciChart charts for

desktop platforms. SciChart has a licensing model for enterprise customers (cf. SciChart,

2022).

4.5.2.4. LightningChart JS

LightningChart JS is a cross-platform charting library using WebGL and is designed to deliver

high-performance visualisations. It uses GPU acceleration to ensure that the device's GPU is

used efficiently, resulting in fast execution performance. The library can be integrated into

web, mobile, and desktop applications. To build the charts, active coding is required.

Lightning Chart JS has an enterprise licencing plan (cf. LightningChart, 2019).

Practical Work

4.5.2.5. μPlot

μPlot is an open-source and memory-efficient 2D-based canvas plotter for plotting time series

data. The library was developed by Leon Sorokin and is the smallest time series plotter in the

list at ~40 KB. It does not use any WebGL or WebAssembly technology but is fully build with

the Canvas API. Furthermore, according to the founder, the library does not contain any

transitions or animations – as they only distract from the essentials. To build the visualisations

active coding is required. μPlot is usable under an open-source licence (cf. Sorokin, 2022).

4.5.2.6. eCharts

eCharts is an open-source JavaScript visualisation library. The motivation is driven by three

goals: ease of use, rich built-in interactions, and high-performance. It offers more than 20

chart types out of the box, along with a dozen components, and any of them can be combined

arbitrarily. The visualisation can be configured to use either the Canvas API or SVG rendering.

The active and large open-source community ensures healthy development of the project and

contributes to a wide range of third-party extensions. To build the visualisations and

dashboards active coding is required (cf. Deqing Lia et al., 2018).

4.5.3. Performance and Robustness Test

To test the robustness and performance requirements, an intensive benchmark test was

carried out. The aim of this was to test the possible visualisation solutions listed in chapter

4.5.2 against the performance and robustness requirements defined in chapter 4.5.1. The

focus was on the line diagram type of the respective solutions. This has the following

background: The line diagram is one of the most frequently used diagram types and is

available in every library. Line charts plot every data point that is made available to them for

rendering. This makes data overload and associated performance problems most visible. If a

library can render millions of data points in a line chart, it is likely to be able to do so for most

of the other chart types as well.

To ensure reliable testing, a test suite has been developed that guarantees repetitive testing

under the same conditions. To avoid possible latency of web frameworks, the test suite was

developed in plain JavaScript. The simplified structure is as follows. In a configuration file, the

user can set certain parameters, including:

Practical Work

• the visualisation solution which should be evaluated.

• the number of maximum points to be rendered.

• the number of channels.

• the test scenario approach.

After the parameters have been configured, the user can start the application (settings can

be configured without restarting the application). The configured parameters are then queried

in a general benchmark file and the test suite is built. All data points are queried recurrently

from a JSON file with a dataset of about 16,000 randomly generated data points ordered by

ascending x-values. The data and the user configuration are then passed on to the respective

file for rendering the chosen visualisation solution. First, it is ensured that all external scripts

that could block the rendering of the visualisations are loaded. Then, the application waits for

a few frames before starting the actual benchmark. Now, the diagram solution is rendered

with the necessary configurations and data. The benchmark results can be seen in the

console of the browser. Care was taken to ensure the most performant implementation for

each solution was developed.

Three scenarios were developed to test different performance and robustness focuses. The

performance parameters rendering time and FPS described in chapter 3.2.1 were investigated

in more detail for each solution. The scenarios, parameters, and performance targets which

were tested are shown in Table 6.

Table 6: Performance Test suite Scenarios

Scenario

Explanation

Parameters to set

Focus

Static Test

This scenario tests the basic

performance and robustness of the

visualisation. In this stress test, a

different number of points are

visualised statically.

• Number of points

• Number of channels per diagram

• Rendering

time

Appending

Test

In this scenario, a certain number of

points are added to a chart every

second. This tests how the graph

behaves during interaction e.g.,

zooming and panning.

• Number of points of the initial

diagram

• Number of channels per diagram

• New samples per second

• Cleaning interval of not visible data

• FPS

Refreshing

Test

This scenario tests how fast a data

set can be refreshed, in the unit

frequency of Hz before it starts

lagging. This tests the real-time

support of a chart solution.

• Refresh rate in Hz

• FPS

Practical Work

For each solution, the newest version (as of November 2022) was used. The benchmark test

was performed on a MacBook Pro 2020 with the Apple M1 chip and 16GB RAM in Google

Chrome V.106 in an incognito tab. Each solution was tested five times for each scenario. For

the result of each test, a mean value was calculated from each run.

However, the self-contained platform-ready solutions Grafana and QuickSight could not be

tested with this test suite. Grafana, however, has its own performance tool with which at least

static tests were carried out as similar as possible to the test suite. A documentation of the

Performance Test Suite can be found in the appendix A.5

4.5.3.1. Static Test Scenario

The static test scenario was tested with 500,000, 1,000,000, 3,000,000 and 5,000,000 data

points divided into ten channels per chart while rendering in one chunk. It turned out that

QuickSight is not able to visualise even 500,000 data points. For this reason, it was removed

from the list as it did not meet the basic requirements. Grafana's data points were retrieved

from a database. The time the application needed to query the data was factored out.

Figure 33: Static test with different numbers of points

As seen in Figure 33, eCharts crashed at more than one million data points. Grafana managed

to render up to three million data points. Only SciChart.js, LightningChart JS and μPlot

managed to plot five million data points and more.

The canvas-based library μPlot is the fastest solution for rendering a maximum of one million

data points. For this, the library needs only 282 milliseconds. The WebGL-based libraries

require 511 (SciChart.js) and 712 milliseconds (LightningChart JS). This could be because the

Practical Work

WebGL canvas context takes longer to initially load than the canvas-based context, as

explained in section 3.2.3.2. eCharts and Grafana take significantly more time.

If more than one million data points need to be plotted, the WebGL-based solutions are faster,

which becomes more evident as the number of data points increases. For more than three

million data points, SciChart.js is the fastest solution.

4.5.3.2. Appending Test Scenario

The appending test was tested with up to 3 million elements per graph. Unfortunately, this

test could not be carried out reliably with Grafana, which is why the solution was left out here.

As can be seen in Figure 34, all charts have 60 FPS at 1500 elements. eCharts, however,

collapses very quickly and has just under 15 FPS at around 15 thousand elements. Shortly

after the application is barely usable. μPlot falls below the 20 FPS mark at just over 300

thousand elements. SciChart.js at just under a million. LightningChart JS, on the other hand,

still performs with 58 FPS even at over 3 million data points per chart.

Figure 34: Appending test with different number of elements per graph

4.5.3.3. Refreshing Test Scenario

In the refresh test, refresh rates from 40 Hz to 10 kHz were tested. All graphs had a constant

60 FPS capacity utilisation. This means that all graphs can handle the high frequency of

incoming data. A real-time use is possible with all tested solutions.

Practical Work

4.5.4. Technical Performance Results

The objective of the technical analysis was to find out whether and if so, which possibilities

exist to display large amounts of data (at least three million data points) in a user-friendly time

(maximum three seconds) in a web-based environment as stated in the second research

question. Moreover, it was determined if it is also possible to interact with the huge data sets

after it was plotted.

The evaluation matrix concludes that SciChart.js fulfils the most requirements with 141 points.

A total of 180 points could be achieved. μPlot and the platform-ready solution Grafana came

second and third with each 136 points, followed by LightningChart JS with 127 points. The

last two places were taken by eCharts with 125 and QuickSight with 83. The complete list

can be found in the appendix A.4.

In the performance and robustness tests, the results are not so clear. μPlot could convince

with the rendering time for up to three million data points. After that, the solution which is

based on the Canvas API was doomed by the characteristics of Canvas explained in chapter

3.2.3. In the appending test scenario, which focuses on FPS, there was an obvious

degradation in the μPlot library from around one million data points. At three million datapoints

μPlot was barley usable anymore.

SciChart.js was able to convince in the performance test; it was the fastest solution for

plotting 3 million data points (1,395ms) or more and only fell below the required 20 FPS at

just under one million attached data points. From then on, it remained fairly constant at around

18 FPS. This was slightly noticeable during the test but was not perceived as enormously

disturbing. LightningChart JS was slightly slower than the two previously mentioned solutions

but was convincing in the appending test with a constant 58 FPS with increasing points.

eCharts hardly fulfilled the performance requirements. This could be since they might delete

data points that are no longer visible to the user. QuickSight could not fulfil the performance

requirements.

For the prototypical implementation, a hybrid approach of two solutions was chosen: μPlot

for the implementation of the graphs and Grafana for the environment in which the graphs

can be embedded. This is for the following reasons:

Implementing a whole and fully functional platform additional to the graph panels is too time-

consuming. While testing possible graph solutions and designing the prototype, it became

clear that the successful implementation of a prototype as conceived in chapter 4.4 requires

Practical Work

more than just the visualisation panels. It turned out that the visualisation panels are only the

tip of the iceberg.

More would have been needed for a functioning and satisfactory prototype: A complete

architecture would have had to be developed that would have handled the querying and

processing of the enormous amounts of data from the database into a custom backend. This

has proven to be a critical and very complex undertaking. In addition, the interaction concept

for zooming and panning within the dashboard would have had to be developed in a custom-

made way as well as the selection for the laps. Considering the short time available to develop

a prototype as part of the thesis, this was impossible to realise and would have exceeded the

time frame and probably resulted in an unfinished product.

The platform ready solution Grafana on the other hand supports the integration of custom

graph panels. Furthermore, it offers a platform that connects to a database out of the box and

enables data processing, user management and user-friendly operation of the platform. Even

though it has some downgrades in performance it delivers the overall best experience by

providing a manageable development effort fitting the needs and requirements of the

prototype.

For developing the panels within Grafana, μPlot was chosen instead of the most performant

library SciChart.js. The reason for this decision was that Grafana supports the integration of

diagrams developed with μPlot and even uses them in some of its diagrams itself. This means

better support and less development effort when implementing new graphic panels. This

decision can even be supported by the argument that μPlot performed convincingly in the

performance and robustness tests and in meeting the overall requirements.

Another reason for not using SciChart.js for the prototype was, that they do not provide an

open-source licence. While this is understandable, as it is a commercial product, in the

present case it leads to licensing discussions and various, possibly lengthy processes that

would tie up further (financial) resources. μPlot on the other hand is under an open-source

licence and can be easily integrated. Under these circumstances, μPlot and Grafana were

determined to be the best solution for the prototype.

4.6. Prototype Implementation

As described in chapter 4.5.4, Grafana was used as the surrounding platform. This makes it

possible to integrate self-developed, custom panel plug-ins. These panel plugins are the

actual visualisations and are based on a React-template provided by Grafana.

Practical Work

React is a JavaScript library developed by Facebook. It is used to create interactive user

interfaces and web applications quickly and efficiently with significantly less code than with

vanilla JavaScript. Reusable components are created in React. These components are

individual parts of a final user interface. (cf. Herbert, 2022). The graph visualisations were

developed using the μPlot library described earlier. In addition, Grafana-specific components

and helper functions were used, which Grafana Labs provides for the development of custom

panels.

The platform initially runs locally but can be started simply and easily with the Docker

container system. Docker is an open-source platform that allows developers to create,

deploy, run, update, and manage containers. These are standardised executable components

that combine application source code with the operating system libraries and dependencies

required to run that code in any environment (cf. Carey, 2021). The user thus can integrate

the plugin into his dashboard and configure it using the configuration options provided. Since

not all functions listed in the design concept could be realised due to the platform restrictions

previously discussed and presented in the evaluation matrix, the focus was placed on the

essential aspects: The user can create and configure graphs for all his cars on one dashboard.

The thresholds can be determined and coloured in green, yellow and red. The graph shows

the four health strips for the individual tyre pressure values. A button in the upper right corner

of the graph allows the user to switch between the health strips and the detailed line graph

view as seen in Figure 35.

With this prototype, it can be shown that it is possible to implement the design concepts and

methods and how they perform in a real use-case scenario. Furthermore, real telemetry data

can be displayed, and the user gets a first feeling of what it is like to work with such a

dashboard.

Figure 35: Implemented Prototype in Grafana

Practical Work

4.7. Usability Test

To evaluate whether the concepts and methods of the prototype work out, and to uncover

possible design flaws, a usability test was conducted. A qualitative test was chosen that

focused on gathering insights, findings and anecdotes describing how people use the

dashboard. Therefore, a clickable prototype was created in the prototyping software Figma.

In contrast to the prototype developed in Grafana, where some compromises must be made,

in this prototype all concepts can be presented and implemented exactly as they were

intended.

Due to the distance between the participants and the moderator, a remote test was

conducted. Participants were observed and recorded via the Microsoft Teams VOIP tool while

sharing their screen displaying the prototype with the facilitator. Participants were given tasks

and performed them one by one. The technique of thinking aloud was used. Through this

method, it is possible to find out what the users really think about the design. One learns what

is misunderstood so that recommendations can be derived that might lead to a redesign.

4.7.1. Tasks

The main objective of the usability test is to map the user scenarios presented in chapter 4.3

to individual tasks. For this purpose, four realistic tasks were set. Before the participants had

to work on the tasks, the general scenario of scrutineering during the race was explained to

them.

In the first scenario, the users are supposed to familiarise themselves with the dashboard and

especially with the individual detailed views and decide which overview they like best to

monitor all cars. In doing so, they should name the car they think violates the rules most often,

why they think so and whether this seems logical to them.

In the second scenario, the users must deal with the upper taskbar. Here they have to assess

whether the selection of rounds is coherent and easy to understand.

This task is followed by a detailed examination of the control centre. Thereby it is observed

whether the individual icons are coherent and whether the interaction concepts appeal to the

participants. Special attention is paid to whether the zoom and pan interactions are logical

and easy to understand.

In the final scenario, the detailed views of an individual panel are examined to see if the panels

are clear and easy to understand. Attention is paid to whether the concept of "first overview,

Practical Work

then zoom and detail if required" ensures that the users still retain an overview of all vehicles.

At the same time, they can also enter an initial detailed analysis of a specific vehicle.

4.7.2. Participants

Studies have shown that a usability test delivers a valid result with three or more participants

(cf. Six & Macefield, 2016).

The usability test was conducted with three participants over the course of about two weeks.

Two participants were potential end users of the dashboard. One participant is familiar with

the subject matter but not an end user. This was to test whether new users could also use

the dashboard without lengthy introduction and to get their unbiased insights and

recommendations. The participants were between 25 and 50 years old and all had at least an

academic bachelor's degree. All participants also work in the engineering sector.

4.7.3. Result

The usability test was designed to determine whether the user requirements elaborated in the

user stories could be fulfilled. Specifically, attention was paid to whether the developed

concepts and methods highlight the most important aspects for scrutineers while providing a

good overview without overwhelming the user, as stated in the first research question.

Moreover, it should provide an indication on whether the general usability and user-

friendliness of the dashboard was given.

In the first scenario, the individual detail and overview views were examined more closely.

Here, it turned out that all users considered the multiple health strips view to be the most

useful for an overview scenario. Reducing the individual strips to one summary strip tended

to lead to confusion. None of the participants immediately understood that this was a

summary of all four tyre pressure values.

In the second scenario, the participants focused on the header bar. Here, a design flaw

became visible. Since the dashboards should also be used during the 24 Hours race, up to

2000 laps need to be displayed there. In the current concept, this is difficult to implement and

most likely user-unfriendly because it would require horizontal scrolling in a small space.

Therefore, a new concept and design must be developed. In addition, one participant was

only able to solve the scenario by asking questions, as he could not figure out that the row of

numbers represented the individual rounds.

Practical Work

In the penultimate scenario, the focus was on the dashboard’s control centre. The change

from distance to time was clearly understandable and easy to use for all participants. The

novel interaction concept was welcomed by all participants. The participants indicated that

they would prefer this interaction to the traditional zoom and pan interactions in the individual

graph panel. However, one participant missed a possibility to select a complete lap with just

one click over a button system. Moreover, the buttons with MP3-like functions – which are

used to navigate between laps – caused major confusion for one participant. Here, the

participant stated that a familiarisation phase would be necessary to understand the

functionality behind each button.

In the last scenario, the views of an individual car were examined more closely, through finding

an exact value at a certain point in time. Here, it became most visible how differently the

individual participants operate using the dashboard. For one participant, the tabular view

seemed superfluous. This participant prefers to zoom into the line chart as much as possible

and find out the value via the tooltip while hovering over the single points. The other

participants disagreed with this procedure and considered it useful to find the values in tabular

form. However, concerns were expressed about the height of the table. Since the granularity

of the telemetry data is very fine, the participants were afraid that a user-unfriendly scrolling

of the table would be necessary, even if the time-period was pinned down to a minimum. The

scroll scenario could not be reproduced with the prototype and therefore no clear statement

but only an assumption from the participants could be made.

To get a general picture of the mood and the complete perception of the user experience,

each participant filled out a standardized System Usability Scale (SUS) and a User Experience

Questionnaire (UEQ) after the scenarios.

The SUS gives a good and quick overview of the general usability of a product. A system

receives a score between 0 and 100, while 100 is the best result. Furthermore, there are many

benchmarks in the industry with which to compare a system. With more than 80 points, a

digital product is above average (cf. Schmidt-Rauch, 2017).

The UEQ was included because it asks about other emotional constructs besides usability

and consequently also reveals difficulties beyond the mere ease of interaction. The constructs

are attractiveness, transparency, efficiency, controllability, stimulation and originality (cf.

Schmidt-Rauch, 2017).

In the SUS, the dashboard received an average score of 69 as seen in Figure 36. This score

is at the top of the scale between OK and Good. From this, it can be deduced that the

dashboard has acceptable and user-friendly concepts, but as the scenarios have also shown,

Practical Work

there is still room for improvement in their implementation. These range from missing labels

for better comprehensibility to concepts that need to be reworked from ground on, such as

the icon interaction concept.

Figure 36: SUS-Score Telemetry Dashboard (Own representation based on (Schmidt-Rauch, 2017))

In the UEQ, the dashboard received an overall positive rating. All answers, even the relatively

low ones, are in the positive range as seen in Figure 37. The participants were particularly

unanimous on attractiveness, efficiency, and stimulation. Here, the variance was very low. For

the constructs Novelty, Perspicuity, and Dependability, the evaluation was positive, but the

results show a greater variance. This perhaps indicates the different ways of working and the

different perceptions of the individual participants. As discussed above, these became

particularly visible when operating the individual detailed interviews.

Figure 37: UEQ Result Telemetry Dashboard

Practical Work

4.8. Algorithm Survey

To obtain a more neutral view of the quality of possible algorithms and aggregation functions,

a survey was conducted to compare the visual properties of an original line graph with the

down sampling methods presented in chapter 3.2.5. The survey was designed in such a way

that the participants were asked to rank the down sampled line graphs from the best

representation to the worst. The goal was to gain insights on how good or bad a

representation of a down sampled line graph is.

For generating the visual representations, a web-based test environment was developed in

which the algorithms that should be tested were integrated. The data used for the visualisation

is real production telemetry data that is queried from a cloud-based time series database. To

render the visualisations, a data set stored in a cloud-based database can be selected via a

drop-down menu. The application then first renders the full data set followed by the sampled

down visualisations for the following algorithms: Largest Triangle Three Bucket, Mode Median

Bucket, and the Average Aggregations method. For the LTTB algorithm, the threshold can be

adjusted via a slider.

4.8.1. Questions

To keep the survey short, it consisted of only five questions. For each question, the participant

was shown the original line chart and a series of down sampled line charts (in a random order

for each participant) with different methods and settings.

The participant's task was then to rate the diagrams from best to worst in comparison to the

original line diagram. It was specifically stated that participants should only rank the reduced

line charts according to their own preferences and should not spend more than thirty seconds

on each question.

A good visual representation is not really a well-defined measure. The instruction was

intentionally kept vague so that the participants could decide for themselves what a good

visual representation meant to them. The aim is to find out whether, for example, the removal

of extremes is noticed and perceived as important. An example of the down sampled

variations can be found in the appendix A.8.

Practical Work

Figure 38: Different visual characteristics of telemetry data for the algorithm survey

The line diagrams shown in Figure 38 were chosen because they have different visual

characteristics. The first line diagram is quite irregular, with parts being relatively smooth.

Nevertheless, it has some concise deflections with sharp peaks. The second and third graph

show irregular patterns with sharp and round deflections and changes in direction. The fourth

diagram shows an almost periodic course with sharp deflections in positive and negative

direction as well as a visible band, that diagonally runs through the visualisation. The last

diagram is also a periodic course. However, this is interrupted by various gradations.

The first two questions concerned the Largest-Triangle-Three and Mode-Median-Bucket

algorithms. These were compared with the average aggregation function, applying different

thresholds for down sampling. This was to determine if an algorithm was generally the most

effective or if it varied depending on the down sampling threshold and the data source.

The next three questions compared the original line plot with down sampled versions of the

Largest-Triangle-Three-Bucket algorithm with different thresholds. These were also

compared to the average aggregation method. The goal was to determine if the Largest-

Triangle-Three-Buckets algorithm consistently performed better than the aggregation

function, as would be expected based on the research conducted in chapter 3.2.5. In addition,

it should be determined whether there is a threshold value in relation to the complete data set

up to which the algorithm still works sufficiently and provides a good representation.

Practical Work

4.8.2. Participants

Fifteen participants took the survey. Most participants were between 25 and 40 years old as

you can see in Table 7.

Table 7: Age distribution of participants in the algorithm survey

Age range

Count

18-24

25-30

31-40

41-50

All participants had a college degree and relation to engineering as well as experience viewing

at line diagrams. Care was taken to select participants who were familiar with the topic and

had a certain level of expertise. On the other hand, participants were also selected who were

familiar with the overall topic but explicitly not telemetry engineers. This is to test the extent

to which layers assess the algorithms in contrast to the experts.

4.8.3. Results

The aim of the algorithm survey was to find out if there is a general threshold that can be

applied to any data set to generate a satisfactory down sampled visual representation for the

user. Or – if this is not the case – whether a general principle can be derived about how far

data can be down sampled for certain characteristics of specific telemetry data sets.

As shown in Figure 39, most participants voted the Mode-Median-Bucket algorithm last for

the first two visualisations, considering it the worst representation of the original line graph.

The most effective algorithm was the Largest-Triangle-Three-Bucket, which the majority of

participants ranked first, attributing it the relatively best representation of the original line

diagram. The average method convinced two participants in the first example and even four

participants in the second example.

Practical Work

Figure 39: Survey result for the first two examples voting the best algorithm

Based on this, the next questions investigated whether there is a general threshold that can

be applied to any data set to produce a satisfactory down sampled visual representation. In

order to cover different telemetry data sets and to obtain a meaningful result, examples were

selected that had different characteristics. On the one hand, they contain very periodic trends

and a very dense data distribution, on the other hand, the data also contains random peaks

and lows.

Figure 40: Survey result for voting the best thresholds for LTTB algorithm

Participants had the option to choose between a threshold of 75 %, 50 % and 5 % of the

original data set. The result was quite conclusive as Figure 40 shows: The threshold value of

the Largest-Triangle-Three-Bucket algorithm for sufficient visual representation was

determined to be 50 % in all three examples. Only in the second example five participants

choose the 75 % threshold. This dataset is a highly periodic dataset with a high density of

information. From this, it can be derived that for these participants a threshold of 50 %

excludes too much information that they consider important and that they therefore prefer a

higher threshold and thus a smaller bucket size with a more granular representation of data.

Practical Work

4.9. Evaluation of results and potential for improvement

In this master’s thesis, concepts were developed to present specific telemetry data in a

visually appealing, user-friendly, and compact way in a single dashboard. Furthermore,

explicit graph libraries were extensively and scientifically tested for their performance and

robustness. In addition, down sampling algorithms were explained and tested for their visual

precision at different thresholds. The main results of this thesis and their interpretations are

summarised in the following:

Although a fully functional prototype could not yet be developed within the scope of this

thesis, a solid basis was established that can be used in future scientific work to further

develop a prototype. Thus, the idea of a performant, user-friendly telemetry dashboard in a

web environment can be further pursued.

The technical prototype which was developed in this thesis met the minimal requirements.

This includes the basic design concepts of Focus and Context, which were discussed in

chapter 3.1.5.2. This concept implies that the user can switch between the health strip view

and the line view. The different views and their specific use cases have been discussed in

detail in chapter 4.4.2.1. In addition, the user has the option in the prototype to configure the

panels in terms of colour, size, and certain display options.

These panels have been implemented in the platform-ready solution Grafana. The reason for

not developing a comprehensive and independent dashboard is that it became apparent

during the process that instead of just a dashboard with visual panels, a full application would

have had to be developed, which would have been beyond the scope of this thesis. The

development and conceptualisation of the various aspects would have required too many

sub-areas, which were not the focus of this thesis. For example, querying millions of records

from a time series database has proven to be a major challenge and requires a well-designed

architecture. Considering the individual studies, tests and surveys that were carried out for

this thesis, finding an adequate solution for this topic, which was not in the focus of this

master’s thesis, would have exceeded the time frame.

In the design and usability part of this thesis in chapter 4.4, it was investigated and

experimentally tested whether and how dashboards must be designed to give the user a clear

overview of the respective sensor data of all vehicles. At the same time, the question was to

be answered whether it is possible to analyse individual vehicles and their data in more detail

without losing the overview and context of the dashboard.

Practical Work

The general design concept of presenting the overview of up to 18 cars on a dashboard could

be satisfactorily mapped by applying a variation of the Focus and Context method. Concepts

were developed that meet the mantra developed by Shneiderman for interacting with huge

amount of data: First, overview, then, zoom and filtering, finally, details on demand (cf.

Shneiderman, 1996).

The fact that the developed concept works was confirmed in the usability test and manifested

by the feedback from the participants. The attractiveness of the dashboard was particularly

emphasised by the participants, as described in chapter 4.7. This could indicate a general

enthusiasm of using such a product in a real scenario. This might be because users expect a

better, more convenient, and more efficient way of working with such a dashboard. This result

is also supported by the UEQ construct of effectiveness, which was perceived as very positive

by the participants during the usability test. The various ways and granularities in which the

data can be analysed without losing the overall context of the data have satisfied the

individual needs of different types of users. By using the different detailed views, users have

the possibility to compare different values of a vehicle. However, it would be of interest to

determine in a further study whether these concepts can be applied to other telemetry data

than the tyre pressure data presented in this thesis.

For the interaction concept, new and innovative zoom and pan interactions were designed.

Although these could be evaluated and tested in the usability test, they would have to be

observed more closely in a real scenario to see whether these interactions would also work

during real-time use cases.

The most important data for scrutineers, as explained in the requirements, are those that

deviate from the norm. Through the thoughtful use of the signal colour red, the most important

aspects of the telemetry data are indicated to the user. The underlying traffic light principle

makes it clear to the participants that red means danger and a detailed consideration is

required, while the colour green means that the value is within the norm and no analysis is

needed. This concept helps the scrutineers to work more efficient, as the usability test has

shown. However, the perception of the individual aspects could only be tested in a historical

view. The real-time view was not tested due to technical difficulties. Here it would have been

interesting to see how the participants would react to rapidly changing values. Also, the use

of an eye-tracking device could be advantageous here. This would also give the opportunity

to check whether the eye-tracking study mentioned in the research section (see section 3.1.2)

can be confirmed even in real-time scenarios with fast switching values. This must be

investigated in further tests.

Practical Work

In the technical implementation and research in chapter 4.5, great attention was paid finding

out whether it is possible to display the enormous amounts of data in a web-based

environment. Another focus was the question of whether it is subsequently possible to

interact with the huge amounts of data after it has been rendered. This was an important part

of the research and practical part, as interacting with huge amounts of data is considered to

be a very resource-intensive task, often causing web browsers to freeze or crash. For this

reason, traditional web rendering methods were investigated and challenged by new methods

and technologies to find the best possible answer to these questions.

Eventually, it was proven that it is possible to render up to five million data points in a line

chart within the prescribed three seconds and therefore two million datapoints more than

required. This is not only interesting for the telemetry and motorsport industry but could also

be relevant in other industries, where the visualisation of large amounts of data is also

important. For example, this could be the case in the medical or finance industry. The

advantages of a web application could also be important here. Whereas the biggest

advantage is certainly the easy and unrestricted access without hardware or operating system

limitations.

Smooth interaction with large amounts of data could also be demonstrated with web-based

graph libraries. The WebGL based chart libraries SciChart.js and LightningChart JS were

particularly convincing here. The WebGL based graph libraries were superior to the Canvas

based graphs especially in the interaction tests (Appending and Refreshing Test in chapter

4.5.3.2 and 4.5.3.3). The reason for this might be, that they always use the GPU of the

executing machine and thus have more computing power at their disposal as explained in the

research part in section 3.2.3.2.

During the thesis and the examination of the topic, however, the question arose whether it

really makes sense to plot millions of datapoints in a limited pixel space – even if this is

technically possible as just explained.

For this reason, another focus of the thesis was formed, in which methods and concepts were

sought to filter the data before rendering, not only in terms of design but also technically. A

particular difficulty was to find methods that did not filter out important information. One of

the tested algorithms and aggregation functions eventually proved to be sufficient to do this

and still meet the requirements.

In a first test, the LTTB algorithm has shown that a sufficient visual representation of the entire

dataset is possible even with a dataset reduced by up to 50 %. In retrospect, it would have

been interesting to have a more granular gradation among the 50 % of the original data, to

Practical Work

determine the exact threshold at which the data visualisation no longer provides sufficient

information for the participants. However, it can be concluded from the results that the data

can be rendered even faster, as at least only half of the data needs to be queried and drawn.

This could also have the positive effect of avoiding data overload, so that the user is provided

with a diagram that always gives him a good overview of the data. However, the research has

shown that there were also study participants who preferred a lower sampling rate of only

25 % of the full dataset for datasets with certain characteristics. In detail, this was the case

for a data set that had very dense highs and lows. That suggests that in another study, such

datasets with these characteristics should be examined more closely. Nevertheless, it can be

concluded that pre-filtering by an algorithm seems reasonable considering the other

requirements of rendering time, robustness, and general usability of a visualisation.

Yet, the question remains whether it is possible to query at least one and a half million data

records from a database at once. In an initial test conducted during the development of the

algorithm test suit explained in chapter 4.8, it was possible to query up to two million records

from a cloud-based time series database. However, the basic query-only approach used for

this task proved to be very unstable and error-prone. In another attempt, it should be

investigated which other concepts are possible to get better results.

Conclusion and Outlook

5. Conclusion and Outlook

By carrying out the individual experiments and tests, it becomes clear that it is possible to

visualise huge amounts of data on the web in a performant way. The design concepts and

methods developed in this thesis have also shown that it is possible to structure the flood of

telemetry data in such a way that the user is provided with a well-structured visualisation that

offers a good overview. In addition, the concepts presented in this work also allow the user a

subsequent detailed analysis of the data, while keeping an overview of the big picture.

Nevertheless, it also became clear that the presentation and visualisation of telemetry data is

only the tip of the iceberg when it comes to developing a complete dashboard.

For this reason, this thesis has pointed out the missing components needed to build a fully

functional dashboard. Furthermore, the recommendation was made to use the graph library

SciChart.js for the implementation of the visual panels. This library proved to be the most

suitable solution in the evaluation of the technical possibilities and in the performance tests

carried out. Nevertheless, it must be noted that, as described in chapter 4.5.4, a considerable

development effort is necessary to adapt the library to the requirements. The use of platform-

ready solutions is generally not recommended, as they are too inflexible for the requirements

of the motorsport world and cannot provide the necessary performance. Major cutbacks are

to be expected here.

In summary, this thesis demonstrates that it is possible to visualise large amounts of data in

a performant way in a web-based dashboard that makes the race scrutineers' work easier

and more effective and avoids frustration. This finding is not only relevant for this thesis, but

also for the scientific aspect, as this thesis can be seen as preliminary work for various

industries where the visualisation of large amounts of data is necessary. This is especially

interesting because, as described in chapter 1, a web-application can solve many of the

problems that a desktop application brings with it: the easy accessibility of a web application

compared to a desktop application for example, as well as the fact that with a web-application

it is no longer necessary for the user to install and maintain an application himself. This work

is therefore intended to be a first impulse for further scientific work that builds on the findings

presented here.

The next step is to first address the technical limitations described in chapter 4.9 and propose

solutions for developing a fully functional dashboard. For a successful application, it is

important to investigate all other sub-areas and find the best solutions for each.

Conclusion and Outlook

Building on this, there is the possibility of incorporating algorithms that filter the data in such

a way that they do not extract any important information. As studies in this thesis have shown,

the LTTB algorithm explained in chapter 3.2.5.2 is best suited for this purpose. A further step

could be the implementation of an Artificial Intelligence that pre-analyses data or outputs

forecasts. This would have several advantages: On the one hand, the scrutineers would be

relieved of a lot of work, more precise decisions could possibly be made, and potentially

dangerous situations could be detected better and in advance. This is part of the scrutineers'

requirements but could not be covered further within the scope of this thesis.

The possibility of literature work could not be fully exhausted within the scope of this master's

thesis. When elaborating design principles for dashboards, only an excerpt and the most

important principles could be explained in the first part of the thesis, as researching, and

describing all principles would have been too extensive. In the practical part, further

explanations were omitted in some parts. For example, further implementation of possible

algorithms was dispensed with, as this was not possible due to time constraints.

Even if the results might at first seem to indicate that the successful use of a telemetry

dashboard is still a long way off, the thesis shows through the numerous tests and examples

described that a basis for a fully comprehensive dashboard has been established. The results

show that visualising several million data points in a web-based environment is possible in a

performant and user-friendly way. The developed usability concepts allow a clear visualisation

of many complex panels on one dashboard, but at the same time give the user the possibility

for a detailed analysis of specific individual panels without losing the context.

Bibliography

Adam Hayes. (2022, June 30). Aggregate Function. Investopedia.

https://www.investopedia.com/terms/a/aggregate-function.asp

Amy Alberts. (2018, August 23). How design thinking will affect today’s analysts.

ComputerWeekly.Com. https://www.computerweekly.com/opinion/How-design-

thinking-will-affect-todays-analysts

Awati, R. (2021, June). What is Amazon QuickSight? SearchAWS.

https://www.techtarget.com/searchaws/definition/Amazon-QuickSight

Blackstock, E. (2020, December 6). Formula One Telemetry: Explained. Jalopnik.

https://jalopnik.com/formula-one-telemetry-explained-1845819678

Browne, C. (2021, August 5). Design Thinking vs. User-Centered Design [Which is Best?].

https://careerfoundry.com/en/blog/ux-design/design-thinking-vs-user-centered/

Caballero, L. (2011, October 13). Dev.Opera—An Introduction to WebGL — Part 1.

https://dev.opera.com/articles/introduction-to-webgl-part-1/

Card, S. (2008). Information Visualization (pp. 509–543).

Card, S., Mackinlay, J., & Shneiderman, B. (1999). Readings in Information Visualization:

Using Vision To Think. In Information Visualization—IVS.

Carey, S. (2021, August 2). What is Docker? The spark for the container revolution.

InfoWorld. https://www.infoworld.com/article/3204171/what-is-docker-the-spark-for-

the-container-revolution.html

Coyier, C. (2019, November 12). When to Use SVG vs. When to Use Canvas. CSS-Tricks.

https://css-tricks.com/when-to-use-svg-vs-when-to-use-canvas/

Deqing Lia, Honghui Meib, Yi Shena, Shuang Sua, , Wenli Zhanga, Junting Wanga, Ming

Zua, & Wei Chenb,. (2018). ECharts: A Declarative Framework for Rapid Construction

of Web-based Visualization

✩

Durmus, C. (2022, April 25). WebAssembly vs JavaScript: Can WASM Beat JavaScript In

Benchmark? Medium. https://javascript.plainenglish.io/webassembly-vs-javascript-

can-wasm-beat-javascript-in-benchmark-cd7c30faaf7a

Farcaster. (2013). Relationship between inflation and the unemployment rate. Inflation and

Unemployment rate data by month from Federal Reserve Database.

https://commons.wikimedia.org/wiki/File:U.S._Phillips_Curve_2000_to_2013.png

Few, S. (2004). Eenie, Meenie, Minie, Moe: Selecting the Right Graph for Your Message.

Few, S. (2005). Effectively Communicating Numbers: Selecting the Best Means and Manner

of Display. 34.

Few, S. (2006). Information dashboard design: The effective visual communication of data

(1st ed). O’Reilly.

Bibliography

Gardón, D. S. (2022, March 21). WebAssembly vs. JavaScript: The Complete Guide.

Snipcart. https://snipcart.com/blog/webassembly-vs-javascript

Grafana. (2022, October 23). Grafana: The open observability platform. Grafana Labs.

https://grafana.com/

Herbert, D. (2022, June 27). What is React.js? (Uses, Examples, & More).

https://blog.hubspot.com/website/react-js

Horak, T., Kister, U., & Dachselt, R. (2018). Comparing Rendering Performance of Common

Web Technologies for Large Graphs. 2.

Ilya Grigorik. (2018, August 17). Render-tree Construction, Layout, and Paint. Web.Dev.

https://web.dev/critical-rendering-path-render-tree-construction/

Johns, S. (2020, August 21). CPU versus GPU with the Canvas Web API.

https://www.middle-engine.com/blog/posts/2020/08/21/cpu-versus-gpu-with-the-

canvas-web-api

Juviler, J. (2021, January 12). The Ultimate Guide to Website Performance.

https://blog.hubspot.com/website/website-performance

Kim Salazar. (2020, December 6). Contextual Inquiry: Inspire Design by Observing and

Interviewing Users in Their Context. Nielsen Norman Group.

https://www.nngroup.com/articles/contextual-inquiry/

Laubheimer, P. (2017, June 18). Dashboards: Making Charts and Graphs Easier to

Understand. Nielsen Norman Group. https://www.nngroup.com/articles/dashboards-

preattentive/

LightningChart. (2019, October 28). High-Performance JavaScript Charts | WebGL Charts

Library. https://lightningchart.com/js-charts/

Mazza, R. (2009). Introduction to information visualization. Springer.

MDN contributors. (2022, September 14). Introduction to web APIs—Learn web

development | MDN. https://developer.mozilla.org/en-

US/docs/Learn/JavaScript/Client-side_web_APIs/Introduction

Menath, C. (2014, August 10). Technik—So funktioniert die Telemetrie—Teil 1—Das sehen

die Fahrer—Formel 1. Motorsport-Magazin.com. https://www.motorsport-

magazin.com/formel1/news-197898-technik-so-funktioniert-die-telemetrie-teil-1/

Moran, K. (2022, January 30). Choosing Chart Types: Consider Context. Nielsen Norman

Group. https://www.nngroup.com/articles/choosing-chart-types/

Oliveira, M. C., & Levkowitz, H. (2003). From Visual Data Exploration to Visual Data Mining:

A Survey. Visualization and Computer Graphics, IEEE Transactions On, 9, 378–394.

https://doi.org/10.1109/TVCG.2003.1207445

Partridge, J. (2021, January 21). How Does Scrutineering In F1 Work? | Formula 1

Technology. //f1chronicle.com/how-does-scrutineering-in-f1-work-f1-technology/

Bibliography

Peacock, A. (2022, February 23). How Fast is WebAssembly Versus JavaScript? Medium.

https://betterprogramming.pub/how-fast-is-webassembly-versus-javascript-

bc0eca058a54

Perna, M. A. (2021, February 24). Canvas vs SVG: Choosing the Right Tool for the Job -

SitePoint. https://www.sitepoint.com/canvas-vs-svg/

Pernice, K. (2018, October 7). User Interviews: How, When, and Why to Conduct Them.

Nielsen Norman Group. https://www.nngroup.com/articles/user-interviews/

Reese, L. (2017, July 7). Telemetry in Auto Racing | Sensor Technology.

https://eu.mouser.com/applications/automotive-racing-telemetry/

Rothe, C. (2022, June 7). What F1 racing can teach us about telemetry. Red Canary.

https://redcanary.com/blog/f1-racing-telemetry/

Salomaa, J. (2020). Evaluating JavaScript frameworks. 43.

Schmidt-Rauch, S. (2017, February 16). SUS, UEQ, … – Nutzlose Werkzeuge oder hilfreiche

Indikatoren? evux. https://www.evux.ch/blog/sus-ueq-nutzlose-werkzeuge-oder-

hilfreiche-indikatoren/

SciChart. (2022, October 23). JavaScript Chart Library | JavaScript Charts | SciChart.js.

SciChart. https://www.scichart.com/javascript-chart/

Shin, K. G., & Ramanathan, P. (1994). Real-time computing: A new discipline of computer

science and engineering. Proceedings of the IEEE, 82(1), 6–24.

https://doi.org/10.1109/5.259423

Shneiderman, B. (1996). The eyes have it: A task by data type taxonomy for information

visualizations. Proceedings 1996 IEEE Symposium on Visual Languages, 336–343.

https://doi.org/10.1109/VL.1996.545307

Six, J. M., & Macefield, R. (2016, January 4). How to Determine the Right Number of

Participants for Usability Studies: UXmatters.

https://www.uxmatters.com/mt/archives/2016/01/how-to-determine-the-right-

number-of-participants-for-usability-studies.php

Sorokin, L. (2022). Leeoniya/uPlot [JavaScript]. https://github.com/leeoniya/uPlot

Spence, R. (2007). Information visualization: Design for interaction (2. ed). Pearson/Prentice

Hall.

Stankovic, J. A. (1988). Misconceptions about real-time computing: A serious problem for

next-generation systems. Computer, 21(10), 10–19. https://doi.org/10.1109/2.7053

Stansell, J., & Marsh, P. (1978, July 13). Intelligence comes to motor racing. New

Scientist(79), 161.

Steinarsson, S. (2013). Downsampling time series for visual representation.

Tufte, E. R. (2013). The visual display of quantitative information (2nd ed., 8th print).

Graphics Press.

Bibliography

Turner, A. (2020, January 9). WebAssembly Is Fast: A Real-World Benchmark of

WebAssembly vs. ES6. Medium. https://medium.com/@torch2424/webassembly-is-

fast-a-real-world-benchmark-of-webassembly-vs-es6-d85a23f8e193

UCRL. (2007). Three Dimensional Radiation Hydrodynamics Calculations of Massive Star

Supernovae Explosions. Visualizations that have been created with VisIt. at

wci.llnl.gov.

https://commons.wikimedia.org/wiki/File:Massive_Star_Supernovae_Explosions.jpg

Wan, A. (2019, April 9). When to use HTML5’s canvas. LogRocket Blog.

https://blog.logrocket.com/when-to-use-html5s-canvas-ce992b100ee8/

Ware, C. (2004). Information visualization: Perception for design (2. ed., [Nachdr.]).

Elsevier/Morgan Kaufmann.

Williams, L. (2020, June 29). TCP vs UDP: Key Difference Between Them.

https://www.guru99.com/tcp-vs-udp-understanding-the-difference.html

Wilson, C. (2021, March 24). The Difference Between Real-Time, Near Real-Time & Batch

Processing. Precisely. https://www.precisely.com/blog/big-data/difference-between-

real-time-near-real-time-batch-processing-big-data

Appendix

A.1 WebGL Code Snippet

This appendix describes how to draw a simple yellow circle with a green border in WebGL as

described in simplified form in chapter 3.2.3.1

File: index.html

01: <!DOCTYPE html>

02: <html lang="en">

03: <head>

04: <meta charset="UTF-8" />

05: <meta http-equiv="X-UA-Compatible" content="IE=edge"

06: <meta name="viewport" content="width=device-width, initial-scale=1.0" />

07: <title>Document</title>

08: <script src="https://twgljs.org/dist/4.x/twgl-full.min.js"></script>

09: </head>

10: <body>

11: <canvas id="canvas"></canvas>

12: </body>

13: <script type="text/javascript">

14: const m4 = twgl.m4;

15: const gl = document.querySelector('canvas').getContext('webgl');

16: const vs = `

17: attribute vec4 position;

18: uniform mat4 u_matrix;

19: void main() {gl_Position = u_matrix * position;}`;

20:

21: const fs = `

22: precision mediump float;

23: uniform vec4 u_color;

A.1 WebGL Code Snippet

24: void main() {gl_FragColor = u_color;}`;

25:

26: const program = twgl.createProgram(gl, [vs, fs]);

27: const positionLoc = gl.getAttribLocation(program, 'position');

28: const colorLoc = gl.getUniformLocation(program, 'u_color');

29: const matrixLoc = gl.getUniformLocation(program, 'u_matrix');

30:

31: const positions = [];

32: const radius = 50;

33: const numEdgePoints = 64;

34: for (let i = 0; i < numEdgePoints; ++i) {

35: const angle0 = (i * Math.PI * 2) / numEdgePoints;

36: const angle1 = ((i + 1) * Math.PI * 2) / numEdgePoints;

37: // make a triangle

38: positions.push(

39: 0,

40: 0,

41: Math.cos(angle0) * radius,

42: Math.sin(angle0) * radius,

43: Math.cos(angle1) * radius,

44: Math.sin(angle1) * radius

45: );

46: }

47:

48: const buf = gl.createBuffer();

49: gl.bindBuffer(gl.ARRAY_BUFFER, buf);

50: gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(positions), gl.STATIC_DRAW);

51:

52: gl.enableVertexAttribArray(positionLoc);

A.2 Expert Interviews

53: gl.vertexAttribPointer(positionLoc, 2, gl.FLOAT, false, 0, 0);

54:

55: gl.useProgram(program);

56:

57: const projection = m4.ortho(0, gl.canvas.width, 0, gl.canvas.height, -1, 1);

58:

59: function drawCircle(x, y, color) {

60: const mat = m4.translate(projection, [x, y, 0]);

61: gl.uniform4fv(colorLoc, color);

62: gl.uniformMatrix4fv(matrixLoc, false, mat);

63:

64: gl.drawArrays(gl.TRIANGLES, 0, numEdgePoints * 3);

65: }

66:

67: drawCircle(50, 75, [1, 1, 0, 1]);

68: </script>

69: </html>

A.2 Expert Interviews

User Interview Guide

Introduction:

Today we have an interview about the IMSA Dashboard. IMSA's technical engineers have to

monitor and validate the huge telemetry data sets of all the cars on the race track in real time

to ensure they do not break rules and regulations. This is a very complex process that requires

the creation of flexible dashboards with different chart types and data sets – moreover

engineers need to arrange them, depending on the use case, as flexible as possible. Creating

and managing dashboards needs to happen on the fly without the need of code.

A.2 Expert Interviews

What do the researchers want to find out?

With this interview we want to dive deeper into the project described above and gather

technical, business and user requirements.

The sessions are recorded, and the transcripts are later analysed by the researcher. The

original objectives will be revised and expanded. The transcripts and interviews will help us

to define the best possible interaction and handling of the IMSA dashboards. A subsequent

thematic analysis of the transcripts will help us define instrumental and non-instrumental

needs and determine the UX design goals and opportunities of the system.

Will my participation in this study be kept confidential?

Yes, the data is anonymised so that no person can be assigned to a specific view.

Furthermore, the data is aggregated so that it is impossible to distinguish between

participants.

What will happen to the results of the research study?

The results will be anonymised and should help us to develop the best possible prototype for

IMSA. In addition, some results will be published as part of a master's thesis at Ingolstadt

University of Applied Sciences. In this process, all sensitive data and information, including

what is discussed during the qualitative interview and the recordings about you as an

interviewee, the Bosch Engineering GmbH, and its partners, will be kept confidential (private)

within the research team. No one outside the research team will have access to confidential

information discussed during the interview that could be used to identify you or violate Bosch

Engineering GmbH's confidential and contractual terms. Any publication is subject to prior

review by Bosch Engineering GmbH. It is not possible to withdraw your data.

With taking part in the interview, you agree to the interview being recorded and consent

to the use of this material by the research team (as described above), as part of the

study.

First Interview: Telemetry Senior Application Engineer from Bosch Motorsport

Warm Up Questions:

1. How are you? How was your day so far? How are you feeling today?

2. Where are you working and what are your job tasks?

A.2 Expert Interviews

Main questions:

Let's talk about the process first:

1. How can I imagine the process of a Scrutineer, can you tell me a bit about what

the general task of an engineer is?

• Scrutineer must monitor all the cars from all the manufactures

• Has to focus on same telemetry category of all racing cars

• There are multiple different dashboards that's what makes it complex is a

great way to say

• they have just three or four engineers or something

• If something is suspicious, he needs to analyse and call the respective teams

2. How does he decide which car he's looking at? And how many cars do they

need to monitor at the same time?

• He looks at all the cars at the same time, if something does not look right, he

inspects the car in detail

• A scrutineer must monitor between 12 to 20 cars depending on the race

3. What decision process is behind on which data they need to concentrate on?

• So basically, they are interested on data which looks not correct.

• This is the focus on the data and then they go into detail and look closer at the

datasets

4. Do they switch between cars, or do they always monitor the same cars?

• Always monitor the same (all) cars at the same time

5. How many engineers do they have to monitor all cars?

• There are 5 main categories

• Those have subcategories with different channels they are interested in

• They fit to the main topics

6. How do they monitor the regulations now? Which processes are in hand yet?

Can you please describe the process?

• At the moment it is at least for Scrutineers very limited

• They get only data once per lap from each car

• Very limited channels – not enough for real-time that’s why we need an app

like this

• So, if the telemetry stream goes down, you have no data from the cars. But

what happens is as the cars go over these timing lines, we get data from.

• Build in on the sensor send some data to the stands but not enough

• Currently visualised in Grafana but doubts of performance when more data is

coming in

Let's talk about the application and business requirements:

1. Should this application be a standalone application? Should it be in the Bosch

Style guide?

• For now, standalone

A.2 Expert Interviews

• At some point integrated in RC but only visualisation components

• No Bosch style needed

2. Is login/authentication required and if so, what type of authentication do you

use?

• Yes!

• User management needed, only specific customer is allowed to see their data

• Data is highly confidential

3. Does it need to run in the cloud and what is the purpose of it?

• Cloud is needed to access it from anywhere

• Fast access, use of web -> cheaper, better maintenance, no Hardware

requirement, no OS

• Everyone can access it just with a link and login creds

• Architecture already there (almost)

4. How should it be connected to the RaceConnect Platform?

• Tbd.

• Now no concept, visual panels should be easy integrable within other

platforms

5. What web browsers will be used to display the dashboards on?

• Chrome

6. What hardware is used?

• Normal Performance Notebooks e.g., zBook, ThinkPad

• External monitors, as we have it in the bus

7. Do we have to use a specific database or are we flexible?

• Timestream Database from AWS

8. Performance and real-time is important in this project, do you have any

performance thresholds or key parameters we can test against?

• He’s fine with real-time up to 3-5s that’s enough

• No lags, fluent interaction

9. Is there a budget for the purchase of a web-based solution or should I leave

this firstly out when developing a prototype?

• Need to be debated with management

Let's have a look at the dashboards:

1. What kind of chart types should be visualised? Line-diagram, scatter charts,

gauge diagram, lap diagram...?

• We need map for lap view, line graph, health bar, scatter plots, just numbers,

tables and maybe gauges – line diagram and just numbers most important

2. Do the charts need to be expanded into new separate windows?

• No not necessary

3. What controls do the dashboards need? Time selection, vehicle selection...? Or

e.g., stop the real-time data to have a closer look at a specific action

happened?

• We need time and distance

• Realtime and lap selection not necessary time, we have it now with Grafana

and it is not handy

• Zoom in the graph either simultaneously with all the graphs or only one graph

• Pan but most important is lap selection

A.2 Expert Interviews

4. Do you need to zoom in and zoom out? And if yes, does this need to happen

parallel in all graphs?

• Yes, but would be nice if Engineers can decide this by themselves

5. They need to create new diagrams on the fly, right? How does this process

look like, can you walk me through this process?

• No beforehand the diagrams are created

• They look at what channels are important for them regarding the main

telemetry category they have

• Could also be the case, that we as Bosch create the diagrams for them

• During the race they probably will not change the dashboards

6. Do they know which data they have to monitor beforehand?

• Yes

7. Are the engineers able to do some SQL Queries or should it be a drag-and-drop

dashboard system?

• SQL Queries like it’s now at Grafana is okay

• Drag and Drop or Selection Box would be better

8. How is the relation built between the car setting and the observation?

• What would be slick is a setting where user can define rules for the cars

• Also, what would be nice is if the dashboard handles the data when a race is

under caution so e.g., not all tire pressure data seems to be off

Let's talk about the data:

1. In my research I read that a car is producing up to 3TB for a race. Is this also a

region of data we are talking about in this project?

• Difficult to say by now: Now we have around 50 channels and we log at 40hz

sometimes faster

• We have up to 24h races and up to 20 cars

• But don’t think its 3TB sounds a bit too much

2. How does the data come in? Is there an UDP stream we must handle? Or is it

already in a messaging bus like MQTT or something else?

• So, from the DB to the Dashboard, it’s a query at the moment

• Probably need WebSocket for transferring data in real time?

Thank you for taking part in the interview!

Second Interview: Telemetry Engineer Corvette

Let's talk about the process first:

1. You are Telemetry Engineer at Corvette, right? How can I imagine what your

work looks like? Can you tell me a bit about what the general task of your work

is?

• Here at 24h Race they have 2 of the same cars

• Two engineers monitor each one car

• Need to look for all telemetry data and look if something suspicious happens

• Analyse after each lap in training

• They also get data from the car via USB, because not all data can be sent

over LTE

> Would be too much data

A.3 Personas

2. What decision process is behind on which data you need to concentrate on?

• Has pre-configured dashboards with data which is critical

> Torque sensor, speed, tire speed

• Compares the data to understand where driver could improve

3. Do you switch between cars, or do you always monitor the same cars?

• Always the same car

• Only monitors one car and analyses during the race – after the race driver and

engineers are comparing the cars to see possible flaws, driving error or

technical problems

4. How do you monitor the regulations now? Which processes are in hand yet?

Can you please describe the process?

• We are using Bosch Windarab at the moment

• Also used ATLAS from McLaren which was more robust and faster than

Windarab

> Desktop Application

Let's have a look at the dashboards:

1. What kind of chart types should be visualised? Line-diagram, scatter charts,

gauge diagram, lap diagram...?

• Most times we have scatter plots, line diagrams, tables for some values

• The Lap graph also

• Line Graphs are in parallel for better comparing the values

• Also, just table numbers for easier overview over individual numbers

2. Do the charts need to be expanded into new separate windows?

• Yes, sometimes if she needs bigger for analyse, I drag it to a separate

dashboard to dig deeper

3. What controls do the dashboards need? Time selection, vehicle selection...? Or

e.g., stop the real-time data to have a closer look at a specific action

happened?

• Zoom synchrony in all the graphs at the same time as well as panning

• This is most handy

4. Is there anything else you would like to mention or anything that is important to

you that I have not asked or paid enough attention to?

• To compare its best to have the values in one graph

• Highlighting different lines, to see when they exceed the norm is handy

• KI would be nice, to get easier forecasts

Thank you for taking part in the interview!

A.3 Personas

Liz comes from a computer science background and is very technically minded. She likes to

think her way into new applications and tries out innovative and novel things. Furthermore,

she is very interested in artificial intelligence in connection with data visualisation. She’s

A.3 Personas

satisfied when things work as expected and are visual appealing. She is more interested in

data structures, how these get collected and analysed then in the actual cars.

Figure 41: Persona Liz Morgan

A.3 Personas

Tom is a very disciplined, precise, detail-oriented, and a structural person with an affinity for

cars. However, he is hesitant to learn new technologies and applications and wants to keep

things as he knows them.

Figure 42: Persona Tom Pright

A.4 Requirements and Evaluation Matrix

This appendix summarises in tabular form all the requirements of the project, which were

assessed using the evaluation matrix. The table can also be found here:

https://docs.google.com/spreadsheets/d/1Ew68oshUxG3XGnaTxGFHRVp1AaB4mDIN3o9J

lHWQ238/edit?usp=sharing

          



    





 









  



 









 

















 

 













  







 















  



   



 

 















       

 













     





 





 





 





 

 











       

 













  









  



 



 



 



  





   





 



 



 



 



 















  







   







 

 















  



 





 









 

  









     



 



   



  























  



 





 





 





 





 





  





















  



 





 





 





 





 





  

















 

  











 

 













    





 





 





 





  







  







 



 





 





 





 





 

















    





 





 





 





  





    





 





 





 





  





    





 





 





 





 















  







   





 





 







A.5 Performance Test Suite

The Test Suite can be found on github.com:

https://github.com/AndreasRiedel95/BenchmarkTest.Graph_Libraries

Please visit the repository to see how to start the Performance Test.

Performance comparison of JavaScript chart libraries in line chart visualisation. Line charts

are the most used chart type in all areas of data visualisation. To test their performance in

different types of applications, 3 different application types of line charts were identified:

Static line chart: an XY or Y dataset is loaded and displayed as a line chart.

Refreshing line chart: Dynamic data that changes at regular intervals (refresh rate). Used in

real-time monitoring/analysis.

Appending line chart. Dynamic data, but in this case the previous data is not deleted, only

moved when new data is inserted. Used in several areas with different real-time data sources.

This repository contains performance tests for these 3 application types. The following chart

libraries were tested:

• LightningChart® JS v.3.3

• Highcharts 9.1.0

• SciChart JS v.2.0.2115

• Dygraphs 2.1.0

• μPlot 1.6.17

• Plotly.js 1.58.4

A.6 Clickable Prototype

The clickable Prototype for the Usability Test was created in Figma. An online version can be

found here:

https://www.figma.com/file/av6mwxNFEuxuS44MDmU2kg/IMSA-Dashboard?node-

id=7086%3A19957

A.7 Usability Test

Link to Usability Online Study:

https://docs.google.com/forms/d/e/1FAIpQLScucwX8lKdBNZxTKQ5QDi6pni4wuSBGJ2LJl

wQ7beBW-I2yFg/viewform?usp=sf_link

Introduction:

Hi, I really appreciate you taking time out of your day to participate in this test. I am Andi and

I’m a researcher for Telemetry Dashboards.

So, let me outline how this will go. I’d like to start by asking you some questions about who

you are, your background, and your relevant experience. I will then ask you to perform some

tasks on the telemetry dashboard. Once the tasks have been completed, I’d like to get some

feedback from you about your experience with the dashboard.

I'm doing this usability test to see how users interact with the software, and to hear their

thoughts on it. I’m trying to make this the best it can be, so your honest thoughts are really

important to us.

Let me explain the scenario:

Imagine you are a telemetry engineer. This means that you are responsible for monitoring all

the cars registered for the race. Your job is to ensure the safety of the cars and the racers.

On the other hand, you are responsible for detecting and analysing rule violations.

In today's case, they monitor the tyre pressure of all cars. For this purpose, you have a

telemetry dashboard at your disposal in which you have to complete various tasks (see

below).

Before we start, could you tell me what your current job title is, and a brief overview of what

your job entails?

Tasks

1. Please switch the entire dashboard between different views. How would you go about

doing this? Now take a close look at all four different views. Which car violates the rules

most often and in which view is this most obvious?

Do you think it is important and useful to have different detailed views, or are some of

them too overwhelming for you? In which view do you think you have the best overview

of all cars?

A.8 Down sampling example

2. Please switch to the view with multiple strips. Select the first lap of the race. Now select

the first three laps. Are you clear on how to handle the interactions? Would you prefer to

have a drop-down menu or something similar?

3. Change the unit of the dashboard from time to distance. Now select the complete span

of the 6th lap. Do you understand the icons and how they work? How would you like to

handle it differently?

4. Please find out the value for Car GTD – 12 at the specific timestamp of 2022-09-19

16:45:50.

And that’s the final task finished! I’ve stopped recording your screen.

Wrap up:

Before we finish, I’d like to ask you a few quick questions:

- How would you describe the overall feeling of the dashboard?

- What did I forget to ask about?

- Who else should I talk to?

Please fill now the SUS and UEQ Survey out. Thank you

Well, thank you again so much for taking the time out of your day to take part in this study

with me. Your input today will be extremely useful for us. Take care, I hope to speak to you

soon. Goodbye!

A.8 Down sampling example

In Figure 43 a down sampling example that was presented to the participants in the algorithm

survey is shown.

First, the participants were presented with the full data set. This was followed by down

sampling variants. In this case, the first visualisation is the LTTB algorithm with a threshold of

8,000 data points.

The second visualisation is a representation to which the average aggregation method was

applied. It also still has 8,000 data points. The last visualisation is the Mode Median Bucket

algorithm with 8,000 data points.

A.8 Down sampling example

Figure 43: Algorithm Example with LTTB (1), AVG (2) and Mode Median Bucket (3)

0 views·109 pages

Evaluation, conception and prototypical implementation of telemetry dashboard components in motorsport PDF Free Download

Evaluation, conception and prototypical implementation of telemetry dashboard components in motorsport PDF free Download. Think more deeply and widely.

Uploaded by ChangeFox on 4/18/2026

/109

100%