PRE2024 3 Group4

max van aken

Bram van der pas

Jarno peters

Simon B. Wessel

Javier Basterreche Blasco

Matei Manaila

Start of project

Problem statement and objective

The problem this group wants to tackle is that many swimmers have flaws in their swimming technique, while the quality and quantity of trainers is declining in many amateur clubs. To solve the problem we want to create a swimsuit with sensors that track the position and orientation of the limbs of swimmers. The suit should then be able to give feedback based on the data the sensors aqcuire.

The users

people who swim for sport (as amateurs, professionals generally have a lot of good coaches) and wish to improve their technique (pretty much all of them).

Requirements

The suit should not be too heavy or inhibit motion too much, as the swimmers should be able to swim as normal while the suit is measuring. It should also be a one size fits all solution, as this means swimming clubs have to purchase less suits. The suits should also be as affordable as possible, as the end users are amateur clubs, which are usually not super rich.

Approach

Preferrably we would like to do this using sensors on the suit, as this means that all of the technology would be on the suit, meaning no external infrastructure is required in the swimming pool. The sensors would be placed on top of or near joints in the body, such as the shoulder, elbow and wrist for arms, and the hips, knees and ankles for legs. Distances between joints would be determined using supersonic sensors and orientation would be determined by having gyrosensors at each joint location. With this approach, reference sensors would be required at the base of each arm or leg so the relative position data from the joints can be converted into more absolute data that is more useful.

If the sensor idea turns out to be impossible to implement during this course, the alternative would be the principles of a motion capture suit, where bright white balls are placed on a black suit and their position is determined using 2 cameras. One camera would view from the side in this case, while another would view from above. Based on this data, the same feedback can be constructed as with the sensor principle, but this would require 2 cameras on rails to be installed in the swimming pool, and these cameras would need to follow the suit around. This would possibly make the suit more expensive, as these rails would need to be either 25 or 50 meters long, depending on the swimming pool. This would also be less practical to implement for amateur clubs, as the pools they use would need to agree with installing said rails.

Milestones and deliverables

Due to the time frame and the scope of the course, a full body suit is likely not feasible. To be able to have something to show at the end of the course, a prototype will be built for one arm. There are also multiple ways of swimming, for this project the focus will be on the front crawl.

The milestones for the construction of the arm suit would be as follows:

1. Build the sleeve (for now without any sensors yet). Keep the type of sensor to be used in mind when creating a sleeve.
2. Build a functional prototype, either by attaching sensors, or by making a construction with external cameras. The prototype should be able to send position data for each joint to a computer.
3. Convert the raw position data to usable coordinates, likely with angles and distances between joints.
4. Construct a program that can differentiate between correct and wrong technique. Some technique errors may be distinguished manually, others might require some simple implementations of AI. One method for this would be gathering a bunch of data for wrong and correct arm motion, gathering simple features from the data, like minimal and maximal angle of the elbow joint, and training a simple decision tree.
5. (bonus) if there is some time left, it may be possible to also write a program for a different way of swimming, like backstroke or the butterfly.

Literature study

Wearable motion capture suit with full-body tactile sensors

This article discusses a suit with not only motion sensors, but also tactile sensors. These sensors detect whether a part of the suit is touching something or not. The motion sensors consist of an accelerometer, several gyroscopes, and multiple magnetometers. The data from these sensors is processed in a local cpu and subsequently sent to a central computer, to decrease processing time and ensure real-time calculations. The goal of the suit is to give researchers in the field of sports and rehabilitation more insight in human motion and behavior, as before this, no real motion capture suit with both motion sensors and tactile sensors had been implemented.

Motion tracking: no silver bullet, but a respectable arsenal

This article goes over the different principles of motion sensors and which methods there are. They discuss mechanical, inertial, acoustic, magnetic, optical, radio and microwave sensing.

mechanical sensing: Provides accurate data for a single target, but generally has a small range of motion. These generally work by detecting mechanical stress, which is not a desirable approach for this project.

Inertial sensing: By using gyroscopes and accelerometers, the orientation and acceleration can be determined in the sensor. By compensating for gravity and double integrating over the acceleration, the position can be determined. One downside is that they are quite sensitive to drift and errors, and a small error integrated over time yields massive errors in the final position. For our project this would be very useful, as sensors determining their position wrt each other is difficult to do as the orientation is difficult to determine.

Acoustic sensing: These sensors transmit a small ultrasonic pulse and time how long it takes to get back. This method has multiple challenges, such as the fact that it can only measure relative changes in distance, not absolute distance. It is also very noise sensitive, as the sound wave can reflect off of multiple surfaces. Those reflections can get back at the sensor at different times, causing all sorts of problems. To solve the reflection problems, the sensor can be programmed to only consider the first reflection and ignore the rest, as this first reflection is generally the one that is to be measured.

Magnetic sensing: These sensors rely on magnetometers for static fields and on a change in induced current for changing magnetic fields. One creative way to use this is to have a double coil produce a magnetic field at a given location and estimate the sensors position and orientation based on the field it measures.

optical sensing: These sensors consist of two components; a light source and a sensor. The article discusses these sensors further, but since water and air have different refractive indices, and the sensors will be in and out of the water at random, these sensors will be useless.

radio and microwave sensing: Based on what the article had to say, this is generally used for long range position determination, such as gps. This is likely not useful for this project.

The Use of Motion Capture Technology in 3D Animation

This article reviews literature about motion capture in 3D animation, and it aims to identify the strengths and limitations of different methods and technologies. It starts by describing different motion capture systems, while later on it comes to conclusions about the accessibility, ease of use, and the future of motion capture in general. Although this last part is not super interesting for us, the descriptions of different systems is.

active & passive optical motion capture: The basic idea is that an object or a person has a suit with either active or passive optical elements. Passive elements only reflect external light, and their position is measured using external cameras, generally multiple cameras from several directions. The material is usually selected such that it reflects infrared light. Active markers on the other hand emit their own light, which is again generally in the infrared part of the spectrum. Also for active markers their position is measured using cameras.

Inertial motion capture: This system uses inertia sensors (described in ) to determine the position of key joints and body parts. This system does not depend on lighting and cameras, increasing the freedom of motion. A widely used inertia based system is the Xsens MVN system.

Markerless motion capture: In this case, no markers or sensors are used, but the motion is simply recorded with one or multiple cameras. Software then interprets the data and turns it into something usable for animators. For us this approach is not very usefull.

Surface electromyography: This method is generally used to detect fine motions in the face using sensors that detect the electrical currents produced by contracting muscles. For us again not super useful.

Musculoskeletal model-based inverse dynamic analysis under ambulatory conditions using inertial motion capture

This article discusses the use of inertial motion sensors from xsens, which is currently part of movella. The specific model used here is the xsens MVN link. They constructed a suit using these sensors and let the test subjects perform different movements. The root mean square distance between the determined position and the real position was found to lie between about 3 and 8 degrees, depending on the body part measured. If we can reach these kinds of values for our prototype that would be sufficient. Since this article is from 2019, the current state of the art technology might be even better than this.

Sensor network oriented human motion capture via wearable intelligent system

This article uses 15 wireless inertial motion sensors placed on human limbs and other important locations to capture the motion of the person. The researchers had their focus on a lightweight design with small sensors and a low impact on behavior. The specific sensors used are MPU9250 models, which also only cost about 12 euros.The researchers transform the coordinates gained from the sensors and have an error of about 1.5% in the determined displacement.

Development of a non-invasive motion capture system for swimming biomechanics

I still need to work this one out

Swimming Stroke Phase Segmentation Based on Wearable Motion Capture Technique

In this source, a sensor-based motion capture system is used to distinguish the different phases occurring during a swim stroke. To determine the bodyposition, measurement nodes are placed on different points of the body. Each measurement node contains a 3-axis gyrometer, 3-axis accelerometer,and a 3-axis magnetometer. An orientation estimation algorithm was used to determine the posture from the information gathered by the nodes. Supervised learning is used to create an algorithm capable of separating different phases of the stroke. The accuracy was compared to an optical motion capture system, and it was found that the system is comparatively accurate. But there is quite a big error for a rapidly changing joint angle.

Development of a Methodology for Low-Cost 3D Underwater Motion Capture: Application to the Biomechanics of Horse Swimming

In this article the motion capture is done using an optical marker-based system. This usually involves spherical markers glued to important body parts and camera’s to track them, but in this specific case color pigments were used instead of these markers. It mentions that in general there are three types of Motion Capture: sensor based, optical with markers, and optical without markers. Where optical with markers is considered the gold standard. It also mentions as a downside for sensor based systems that the size of the sensors can cause additional drag. 6 Cameras were used to capture videos of the horses swimming. The video’s were then preprocessed which included things like calibrating the different camera angles and tracking the different markers. Lastly algorithms were used to determine the horse’s movements. The accuracy of the system was in the order of centimeters for segments, and in the order of degrees for angles. Downsides of the method in this article are that the preprocessing requires lots of time-intensive tasks like having to manually track the markers. It therefore recommends training an algorithm for these tasks. It also mentions problems occurring when certain anatomical points lack overlap between different cameras.

Markerless Optical Motion Capture System for Asymmetrical Swimming Stroke

In the article a markerless optical motion capture system is used that only uses one underwater camera to record the swimming motion. The article mentions that an optical motion capture with markers is actually the most used due to its relatively high accuracy, but mentions that there are also downsides like the markers affecting the swimmers movement, it taking quite a long time to properly place the markers, and there being issues discriminating the different markers from eachother. The system involved creating a human body model, this was then matched to the images of the swimmer to track their movement. An additional algorithm was then required to distinguish the left and right side of the body from eachother. The model seemed to match the images quite well. SWUM software was then used to for dynamical analysis, like analysing the force exerted on the water by the swimmer.

SwimXYZ: A large-scale dataset of synthetic swimming motions and videos

According to the article, most limitations of motion capture systems are due to a lack of data. The motion capture systems usually struggle with interpreting aquatic data as they are not trained on that. Research has shown that synthetic data could complement or even replace real images in training the computer models. This paper published a database of 3.4 million synthetic frames of the swimming stroke that can be used for training algorithms. Using their database the researchers also created their own stroke classifier algorithm that seemed to provide accurate results. Limitations of the database are the lack of diversity in the subjects(gender, bodyshape), and lack of diversity in the pool background.

Current swimming techniques: the physics of movement and observations of champions

This paper outlines how former champions swam and how their strokes looked like. This we could use as perfect data. It does this for multiple kinds of strokes for males and females. This paper also dives a bit into what contributes most to drag for swimmers, so how a stroke can be improved. Next to this it has experimental data for drag coefficient for swimmers with respect to the depth of swimming.

Research on distance measurement method based on micro-accelerometer

This paper shows a possibility how to go from accelerometer data to distance measurements. This we probably need to accurately calculate the position and angles of the swimmers body parts

Experiments on human swimming : passive drag experiments and visualization of water flow

This paper investigates the drag force of a swimmer in streamline position. This is done by towing a person (so the person doesnt move any bodyparts). This is also done with a sphere to better understand how to flow of water goes.

Smart Swimming Training: Wearable Body Sensor Networks Empower Technical Evaluation of Competitive Swimming

This paper proposes a body-area sensor network for detailed analysis of swimming posture and movement phases. Wearable inertial sensors placed on 10 body parts capture motion data, which is processed using a motion intensity-based Kalman filter for improved accuracy. Deep learning models combine temporal and graph-based networks for automatic phase segmentation, achieving over 97% accuracy. The system accurately tracks joint angles and body posture, providing valuable feedback for swimming performance enhancement.

Using Wearable Sensors to Capture Posture of the Human Lumbar Spine in Competitive Swimming

This study introduces a motion capture system based on wearable inertial sensors for tracking lumbar spine posture in competitive swimming. Using a multi-sensor fusion algorithm aided by a bio-mechanical model, the system reconstructs swimmers’ posture with high accuracy (errors between 1.65° and 3.66°). Experiments validate its reliability compared to an optical tracking system. Kinematic analysis across four swimming styles (butterfly, breaststroke, freestyle, and backstroke) reveals distinct lumbar movement patterns, helping coaches and athletes optimize

performance. The system offers a practical alternative to video-based methods.

3D Orientation Estimation Using Inertial Sensors

This paper discusses methods for 3D orientation using data orm accelerometers, gyroscopes and magnetometers. The Study focuses on upper limb movements for patients with neural diseases. Additionally it explores techniques for 2D and 3D position tracking by considering joint links and kinematic constrains.

Motion Tracker With Arduino and Gyroscope Sensor

This project focuses on making a 3D motion tracing devises using an ESP and a MPU. The advantage of this approach is that it is wireless and could be mounted as a wearable.

Building a skeleton-based 3D body model with angle sensor data

In this paper a method for constructing a full body motion model using data from wearable sensors.

Using Gyroscopes to Enhance Motion Detection

In this article, the author explores the role of gyro sensors in improving motion detection and their functionality. The article also provides insight into how gyroscopes can be used to enhance the performance of various motion-sensitive technologies.