TheMotionMonitor Blog Team

MARKERLESS MOTION CAPTURE… AN EARLY EXPERIENCE

2022-08-09T08:37:00.002-07:00

Introduction

In our last blog, we described our history with markerless motion tracking dating back to 2009 with early shape fitting methods. We discussed the benefits we saw with AI based tracking methods and, our view of the future of markerless tracking as described in this short video. In this blog we will share our early experience with three of the AI based markerless systems that have powered acceptance of this technology (SwRI, KinaTrax, and Theia 3D). We will place special emphasis on the data generated from each system together with that of a marker-based system. I think you will be just as surprised by what we experienced as we were.

Objectives

But first, it must be said, this is not an attempt to validate any of these systems nor to compare or evaluate one as better than another. The work we did in our early assessment does not have the necessary rigor to arrive at those conclusions. Rather our basic question was simply how accurate is this new technology? What can we do with it? We found there is more to that question than one would think!

There are few validation studies, none that cover multiple systems, and no real way to compare different systems. We’ve all seen the skeletal overlays in the troves of marketing data. How do you measure the accuracy of data derived from these videos? Guidance is often expressed as the subject should cover at least 500 pixels. So, working thru the math, a 1-pixel feature extracted from a 6 foot subject would have an accuracy of 2.7 centimeters? Ok, that is resolution. But overlays which are valuable for throwing out the obvious misalignments offer little in terms of data quality.

Our questions were pretty basic. How do data compare among systems? Are the data reasonable in terms of the movement? Do the data show repeatable and predictable patterns? Are the data useable in higher order computations such as joint forces, moments and power? Do systems do better in one type of activity than another?

Methods

To answer these questions, we used The MotionMonitor’s hybrid capabilities to synchronously collect data from an optical based marker tracking system and 8 digital video cameras, and then created 4 standard biomechanical models using the data generated by each of the 4 technologies. The optical system was a Vicon 8 camera Vero (2.2 MP) collected at 100hz. The digital video system was an 8 camera Basler Ace2Pro (set to 1.2 MP) also collected at 100hz. The system was calibrated to a common global reference frame. The video cameras were calibrated as proscribed by each of the markerless technologies using extrinsic readings common to all. Four, standard 18 segment models with 21 joints were created using data generated by each of the four technologies. The marker-based subject used rigid body clusters to track each segment with joints located using a digitizer and appropriate offsets from digitized points to joint centers.

In most cases, the markerless technologies generated joint data directly. When it did not, joint data was inferred from other reported features. For example, the knee joint center might be located as the centroid of condyles or the toe as offsets from other foot features. Each markerless technology generated 4x4 transformation matrices to locate segments in the global reference frame. Data was collected for an over ground gait walk, treadmill running, a counter movement jump, and a throwing activity. All data was smoothed using a Butterworth 4^th order 12hz digital filter. The output screens identify the subjects as Marker, MLess1, MLess2 and MLess3 which is not the same order as listed in the first paragraph😊

Results

To address our questions, we first looked at the joint separations of each markerless subject relative to the markered subject. In addition, we looked at the absolute reporting of position data, joint angles, acceleration data, joint forces and moment data for each subject. We’ll discuss each in turn.

Joint Separations

In the screen shot from The MotionMonitor below (Figure 1), we have displayed the subjects generated by each of technology. The subject generated by Marker data is in blue, MLess1 is in white, MLess2 is in yellow and MLess3 is in pink. They are remarkable for the degree of overlap.

To determine how closely the markerless data tracked with the markered data, we computed the average separation between the markered subject’s joint centers and each of the markerless subjects. These were averaged over a 1 second interval in the treadmill running trial. We were surprised at how tightly they align. You may need to zoom in on the image to read the graphical data (all data are in meters) but are summarized below in centimeters.

MLess1 MLess2 MLess3 MLess1 MLess2 MLess3

L5/S1 7.8 2.7 8.3

Right Hip 8.0 8.2 7.9 Left Hip 9.0 9.1 8.3

Right Knee 2.8 3.5 5.2 Left Knee 2.7 2.6 4.3

Right Ankle 3.2 2.4 5.7 Left Ankle 3.2 2.6 4.8

C7T1 3.8 9.1 7.1

Right Shldr 1.9 1.6 7.6 Left Shldr 1.1 1.8 6.8

Right Elbow 2.5 2.2 8.5 Left Elbow 3.1 2.6 7.3

Right Wrist 2.5 7.2 5.1 Left Wrist 1.7 8.1 6.7

While the data was generally tightly aligned, we were concerned about the large separations that all the technologies exhibited at L5S1 and the hips. In Figure 2, we plotted the X and Y positions of joint centers. The markered subject’s pelvis in the X direction was most out of line with other technologies. Again, the reader is invited to zoom in to read the data. While the X and Y positions varied by less than a couple centimeters for most joints, the X position of L5S1 was different from the other technologies by as much as 9.5 cm. Upon examination, we discovered that the offset to the L5/S1 center from the digitized point for the markered subject had been entered incorrectly during the model definition!

Passive Optical systems that track reflective markers are considered the “gold standard” in motion capture as they have been demonstrated to track reflective markers with less than 1mm RMS error. But it is instructive that accuracy in tracking a marker does not transfer to accuracy in tracking the human body. Soft tissue movement, surface landmarks vs. internal landmarks, and user errors can all impact this “gold standard” as we just demonstrated. How to turn gold to bronze!

Joint Angles

The impact of the error in L5/S1 is significant as shown in Figure 3. Only the pelvic orientations from the two extreme subjects are shown in the animation. What should be a near neutral pelvic tilt is reported as 52 degrees posterior tilt for the markered subject. This will, in turn, understate hip flexion and overstate thoracic flexion. With normal pelvic tilt, the markered subject would generate hip flexion and thoracic flexion angles that are comparable to the markerless subjects.

Except for the issue discussed above, the hip, knee and ankle sagittal plane angles in Figure 4 show consistency in pattern and generally fall within 10 degrees of one another. The medial/lateral and transverse plane data do not show as strong a consistency in pattern. Good measurements in these planes of motion have always been more difficult to measure accurately.

Hip abduction angles in Figure 5 show similar patterns in each stride for all the subjects, peaking and troughing at consistent points (eg. foot contact) in the activity. But most subjects differ in their expression left to right which may be reflective of the subject's running style.

Hip rotation angles in Figure 6 showed consistent patterns in only two of the subjects.

At the knee, medial/lateral and transvers plane data did not show the consistency of sagittal plan data. Only the markered subject and one of the markerless subjects show discernible patterns of varus/valgus and rotation during foot contact in Figure 7 below. In some areas of research, knee and ankle joints will have constraints placed on the measurements to avoid output that is considered unlikely. For example, some systems will prevent the intersection of tibia and femur or constrain movement as a hinge joint instead of permitting movement with 6 degrees of freedom. This could account for the differences in some of the knee rotation data. Understanding model constraints is important for ensuring that user’s objectives can be met with the chosen technology. For example, tightly constraining the knee does not always make sense if the objective is to identify the degree of actual movement as part of a functional assessment.

Like the knee, ankle data in medial/lateral and transverse planes (Figure 8) did not show the consistency of pattern or amplitude of the sagittal plane. Only two subjects showed consistencies across strides in medial/lateral plane while a different 2 subjects showed consistency in the transverse plane. Some model constraints are suspected again.

Angular Accelerations

One of our questions concerned the ability of AI technologies to generate data suitable for higher order computations such as joint forces, moments and power. Because these measurers rely heavily on acceleration data, an examination of acceleration data is a good place to start. It is generally understood that acceleration data in biomechanics can be noisy. The graph in the upper right corner of Figure 9 displays the raw acceleration data for the flexion/extension angle of the right knee for each of the markerless subjects. Very noisy indeed! The red is the actual signal for reference. The question is “can this data be filtered in such a way as not to destroy the underlying signal?”

To determine if it is possible requires a look at the noise in each system. The MotionMonitor xGen Frequency Domain functions were used to extract the necessary data. The four plots to the left side of Figure 9 are the same angular acceleration data plotted in the frequency domain (frequency on the x axis and power on the y axis). What each show clearly is that the largest frequency is 2.75 hz for each system. But each system does generate varying degrees of high order frequencies (30-50hz). These high frequency elements likely result from the technology rather than the underlying activity. High order frequencies in the marker data could be from marker vibrations that occur on impact. Noise in markerless data could be associated with how accurately features are extracted frame to frame or the way in which global optimization routines are applied to the results. These high order frequencies can be safely removed without affecting the underlying movement data. The Nyquist or Sampling Theorem would suggest that filtering at 2 times the largest frequency would preserve the underlying content of the signal. The RightKnee Angular Acceleration data filtered at 6 hz appears in the right panel of Figure 9 below the unfiltered data. And all 4 technologies appear very similar in both time and amplitude which suggest that acceleration data should not be an issue in higher order computations.

Joint Force, Moments and Power

We would expect good results with force, moment and power data given the consistency we have observed in intermediate data. Moments at the right knee and ankle graphed in Figure 10 for overground walking demonstrate a consistency of both pattern and amplitude for all 4 technologies.

We were also interested in knowing if the type of activity would affect the data. Would these different technologies perform equally well in activities that have faster or more abrupt movement patterns? In Figure 11, a counter movement jump is shown at the point of landing. Here again one sees a consistency of pattern and amplitudes that are within ranges reported in the literature.

Other Considerations

There are other considerations worth mentioning including synchronization and camera calibration. Synchronization of data is always a concern particularly when collecting data from multiple systems. When a markerless system is unable to extract features from a video frame, the missing data can cause a misalignment with the original video file. The data file can start late, end early or have jumps when drop outs occur in the middle of the trial.

While large synching issues are obvious, smaller ones are not. In Figure 13, a 3-frame misalignment is not obvious in the overlay but can be seen in the time series data. The peak height in these two subjects occurs 3 frames apart demonstrating the misalignment. It is important to understand how each technology handles beginning, ending and mid trial drop-outs as alignment is critical when trying to synch AI generated data with other peripheral data such as EMG, EEG, force plates or eye trackers.

Another consideration involves camera calibrations. Each of the markerless technologies has its own calibration process. With different calibration processes it is possible that technologies may accurately extract the features from the video but then mis-report its location in the world due to a poor calibration. The intrinsic and extrinsic calibration data can have a significant impact on the results as seen in this image with an intentionally poor extrinsic calibration.

Conclusions

As stated earlier, our goal was not to evaluate any of the technologies as being “better” or more accurate than the others. We saw early on that “gold standards” are not what they may seem when the difficulties of applying them (soft tissue artifact, operator error, etc.) are considered. But we did come away believing that all these technologies have a real place in the world of motion tracking each being surprisingly consistent with our markered subject.

1. All appear to consistently extract features which are sufficient to generate biomechanical models that behave appropriately and within expected tolerances for all the data categories.

2. Each technology provided similar consistency across activities with different movement patterns and accelerations.

3. The quality of tracking and generation of data certainly seems acceptable for evaluating performance in sporting activities.

4. The markerless data is comparable to markered data and could be used in pre-post training evaluations.

All the technologies supported in The MotionMonitor have strengths and weaknesses. As a result, different purpose, different types of activities and different environments may suggest the use of different technologies. The benefits of markerless tracking are significant as we discussed in our previous blog. And with the accuracy we experienced in our early usage, it is clear that markerless has a place in motion capture!

If you'd like to learn more about The MotionMonitor with markerless, check out this press release.

Want to see these technologies for yourself? Message us to setup a live demo:

Visit our website, www.TheMotionMonitor.com, for more information about our systems and applications.

Until next time,

Lee

MARKERLESS TRACKING IN MOTION CAPTURE

2022-02-07T04:00:00.004-08:00

What is it and Why it is important?

In simplest terms, “markerless tracking” is the ability to track the movement of human beings without instrumentation attached to the body. Most traditional tracking technologies involve attaching reflective markers or 6DOF sensors (Inertial Measurement Units or electro magnetic sensors) to body segments to determine the position and orientation of the segment. Markerless tracking would extract that information with less invasive methods using digital video.

Instrumenting subjects is a problem on many levels. Elite athletes are particularly sensitive to unusual stuff hanging on their bodies and researchers have more generally worried that instrumentation affects the subject’s natural movement of the activity they are studying. The process of instrumenting subjects is time consuming. And instrumentation generally confines a study to laboratory settings. Accurate, realtime, markerless tracking is the holy grail of motion capture!

A Little History

We have witnessed remarkable transformation in the tracking of human motion during our 30 years in the business. Manual digitization of 2D video files first done in the 1800’s was still prevalent in the 1960’s.

It wasn’t until the 1980’s that Oxford Metrics and what would become Vicon could track reflective markers attached to the body and report 3D positions. While a significant advance, low measurement rates, manual identification of markers and post processing still left a lot to be desired. By the early 1990’s, Polhemus and Ascension Technology had introduced 6DOF electromagnetic sensors that could report position and orientation in real time at rates in excess of 100hz. Beginning in 2000, Xsens popularized the use of IMUs in motion tracking. From there we have seen remarkable advances in performance for most all the technologies. Smaller, faster, easier to use; camera companies today offer high speed, real time streaming of not only marker trajectories but also processed human motion inferred from those markers. Despite the advances, we are still faced with instrumenting subjects and limitations on the conditions under which data is collected.

Enter markerless motion tracking. Our first involvement with markerless tracking dates to 2009 and our support for a fledgling company pursuing tracking from digital cameras. Organic Motion, founded in 2002 and now inactive, offered a system that extracted subject positions and orientations from digital video collected from 8 cameras. It had the advantage of no subject instrumentation. Step into the space, take a neutral pose and then record or view subject motion in real time. However, measurement rates were limited to 60hz and collection required special backgrounds that limited its use to laboratory settings.

Many of our clients found the technology very useful. Autistic children that would not submit to instrumentation loved making Donald Duck move on the screen in time to their movements. Stroke patients who could walk only a few paces after lengthy instrumentation sessions would generate significant amounts of data in just a few minutes on a treadmill with Organic Motion’s markerless capture.

But the biggest issue with Organic Motion’s data was a lack of resolution. It could track head, limbs and torso quiet well. When motions were collected with one of our hybrid systems (Vicon marker and Organic Motion’s markerless) we observed good alignment of basic joint angles performed in the subject’s sagittal or frontal planes. Similar results were experienced by Becker & Russ. But we found long bone rotations to be very poor. Also because of the methodology, it was impossible to monitor finer elements of human motion such as finger motion. At the time, all markerless motion tracking systems used a form of shape or silhouette tracking.

How does markerless tracking work?

There are basically two approaches to markerless tracking, i) shape or silhouette fitting and ii) AI/Machine Learning. It is beyond the scope of this post to get into the details of how these two methods are implemented. But a quick overview will help understand the benefits and limitations of each.

Shape or silhouette fitting is basically a process of extracting a subject’s pose from outlines. Outlines obtained by removing backgrounds from multiple digital videos, taken from multiple positions, are “assembled” into a 3d object using various algorithms. Alternatively, a body model is oriented to generate a “best fit” to the multiple silhouettes. The methods and processing speed are such that realistic tracking of the body can be done in real time. But as our experience with Organic Motion showed, the rough shape of limbs does not provide for the kind of detail necessary for many biomechanical applications.

AI and Machine Learning(AI/ML) have gotten much attention in the last 5 years. AI and Convoluted Neural Networks (CNN) seem well suited for extracting finer detail from video images. The process is basically one of taking a set of inputs and mapping them to a set of outputs. But unlike a direct mapping that might occur with linear regression techniques, CNN mapping includes several hidden layers that result in non-linear transformations of input to output. As Gary Marcus of New York University points out in a critical appraisal of deep learning, the methods for achieving that mapping are complex, with results often not understood by the team creating the network. While beyond the scope of this post, for those wanting more insight into CNN and Machine Learning, see Google’s developer section for an introduction to feature identification.

Training a CNN requires that large data sets be run through the fitting routines. In the case of tracking human motion, one needs i) large numbers of images with the relevant features identified, ii) images need to include people in various poses consistent with the activities that are to be tracked, and iii) subjects and backgrounds need to be sufficiently diverse that the models can track in all environments.

Validating the trained CNN is equally daunting. Typically, a model trained on say, 90% of the dataset is used to test its ability to accurately identify the remaining 10%. Randomly picking the 90% in repeated tests allows for a very large validation test.

With images of the features taken from multiple digital videos located in different viewpoints, standard DLT techniques can be used to locate those features in 3D space.

As mentioned, the shape/silhouette approach offers fast processing and reasonable accuracy of gross motion. But its methods are limited in terms of the resolution that can be achieved.

AI/ML on the other hand, enable extracting the position of individual features such as eyes, joint centers and condyles. But the processing of video files is time consuming and not yet real time requiring several minutes to process videos of short duration. The computational requirements are also more expensive involving high end graphics cards and parallel processing on the GPU.

The Future

Our first experience with markerless motion tracking used shape/silhouette technology. However, our current belief is that AI and CNN are especially suited to tracking human motion and are, in fact, the future of motion tracking. Its primary shortfall is the time it takes to process video data, and that will be addressed as computing power and programming techniques evolve.

With AI/ML many of the traditional biomechanical analyses remain in place. For example, rigid body analyses where 3 non-colinear markers are used to track the orientation of a body segment find parallels with markerless tracking. Feature identification that includes a proximal joint center, lateral condyle and distal joint center could be right out of traditional biomechanical marker sets.

The high resolution of MI/ML also enables monitoring of such things as how a ball is held during a pitching activity.

AI/ML approaches are being commercialized by several companies including Theia, KinaTrax, Intel 3Dat, and SwRI. While we are actively evaluating all, Theia and KinaTrax have already demonstrated acceptance in their respective markets.

Thus far we have created tight integrations with Theia and KinaTrax. It is important from an “ease of use” perspective that the steps of recording video, processing video, applying AI modeling, and generating the appropriate analytical output progress without intervention after clicking the record button. The MotionMonitor’s existing structure is being used to collect digital video synchronously with standard laboratory peripherals such as forceplates and EMG. This greatly simplifies and expands the useability of Theia and Kinatrax markerless tracking.

In addition, The MotionMonitor’s unique design supports fast special purpose application development on multiple platforms. The In Game Baseball application developed for KinaTrax shown in the following video is a good example of this capability.

As a result of the progress made thus far, we believe markerless should soon enjoy the same kind of acceptance that standard tracking systems have achieved. We’re excited by the power of markerless technologies to open new markets and applications for motion capture. These technologies partnered with The MotionMonitor’s flexible interface offer innovative solutions for research, clinical applications, sports performance & training, educational applications and more.

We'd welcome the opportunity to talk with you about your applications & experiences with markerless motion capture. Feel free to connect on social media (@motionmonitor) or send us an email at support@TheMotionMonitor.com.

-Lee

www.TheMotionMonitor.com

What technology do I start with for my Player Performance program? Force plates!

2021-03-19T09:32:00.002-07:00

Introduction

Since the early 1990’s there have been significant advances in hardware to track and measure human motion. Cameras, electro-magnetic trackers, and inertial measurement units to track the position and orientation of the body; EMG sensors to track muscle recruitment; EEG sensors to monitor brainwaves that initiate recruitment; and eye-trackers to identify stimuli that generate movement are all examples of hardware used to study human motion. Because of these hardware advances and the research of human motion there has been a huge impact on the academic disciplines of Physical Therapy, Athletic Training and Performance Enhancement.

And these advances have now made their way into athletics at the professional level. Many of the professional baseball and basketball teams have former biomechanists and athletic trainers from academia on their staff as they seek to prevent injury and enhance performance. And the recently formed Baseballbiomechanics.org is one indication of the inroads made thus far. We see this trend accelerating to the university and collegiate level. And private performance enhancement companies such as Driveline and Jenkins Elite are also getting into the act with services targeted at the high school level. Many of these programs rely on advanced hardware.

One question often asked by coaches and staff is where do I start? Equipment that fueled research is often considered too expensive and too difficult to use effectively in a coaching or team setting environment. Our recommendation is usually “force plates.” Force plates are portable, easily connected to computers and work well when evaluating large numbers of subjects. When joined with the right software, force plates will provide information on the power and speed that an athlete can generate. And as budgets increase, force plates provide the base platform of an integrated system for tracking and measuring human performance.

What is a force plate?

On one level it can be described as a glorified bathroom scale. It measures weight or more accurately, the force your body exerts onto the surface of the plate. For use in performance enhancement, most force plates will have 4 sensors that allow the measurement of both the lateral force as well as the vertical force (3D) being applied to the plate. This 3-dimensional force vector is necessary to accurately measure the reaction forces being applied to the bottom of the feet. There are 1- dimensional force plates on the market which measure only vertical force eg. ForceDeck. One dimensional plates typically have a lower price tag because of reduced hardware and engineering costs. However, the tradeoff is that they cannot be expanded to provide kinetic data. As you can see, force plates are much more than a bathroom scale and have a price tag to match!

There are several manufacturers of 3D force plates with the more well-known being Bertec, AMTI, and Kistler. These are manufactured in ISO facilities with a precision acceptable for research and performance. Wherever purchased, it is important to understand the specifications for expandability, frequency, noise and sensitivity before purchase.

Why are people using force plates?

Force plates are being used by many different disciplines and for different reasons. At the academic research level, force plates are an integral part of understanding the biomechanics of movement and the force and moments experienced at different body joints. Force plates are also an important tool for understanding balance and the organization of the vestibular system. At the coaching level force plates are used to quickly evaluate the power and speed of large numbers of candidates. Jump Analysis products provide a consistent testing protocol that quickly generate reliable and relevant measures of subject ability to generate power. These metrics are applicable to a wide-array of sports, including those which may not be obvious, such as running. Dr. Matt Jordan touched on many applications of Jump Mechanics in his recent talk at the 2021 Sports Biometrics Conference. Strength and conditioning coaches can quantitatively monitor progress achieved by each participant over a training cycle or help evaluate the effectiveness of alternative training programs as described on P3’s website. And increasingly, force plates are being used to aid the return to play decision. Balance, symmetrical power output and performance relative to pre-injury levels can be factored into the decision with the aid of force plate data.

Role of Software

Perhaps more important than the Force plate itself is the software that will be used with the plate. Some software is special purpose in the sense it is used for a particular application. SwingCatalyst is an example of a force plate being used with software to analyze a golf or baseball swing. Hawkin Dynamics software is an example used to analyze jump ability. The MotionMonitor xGen is used for a variety of biomechanics-related applications, including jump assessment, kinematics for general sport movements, golf and baseball swing analysis, real-time audio and visual feedback for training, and more. Expandability to include other hardware is an important feature. The ability to synchronously collect motion capture, reference digital video, EMG or eye tracking data allow for more complex analysis of an athlete’s movements. For example, a recent publication from The Sports Surgery Clinic that included the collection of kinematic data revealed injury risks that were not observable from force plate data alone. The MotionMonitor xGen’s Jump Pro offers a scalable solution that can be upgraded to include any of its supported hardware and software applications. The short video of a Baseball application below is an example of an upgraded application that includes full body 3D motion capture in addition to force data.

Opportunities for teams at the collegiate level.

Collegiate athletics offers opportunities that rival the professional teams. Budget sharing with Kinesiology and Sport Science departments might ensure access to research grade hardware while providing researchers with subjects for study. The Auburn Tigers are an excellent example of that kind of collaboration. The 6-minute video below offers a great discussion of such a program.

We'd welcome the opportunity to talk with you about your applications & experiences building a player performance program. Feel free to connect on social media (@motionmonitor) or send us an email at support@TheMotionMonitor.com.

-Meredith

Covid-19 and Equipment Cleaning

2020-07-21T09:08:00.001-07:00

How has COVID-19 Complicated Data Collection?

Data collection has gotten much more complicated because of the COVID-19 pandemic. Keeping subject and researcher safe means following guidelines introduced by research institutions regarding the interaction between subjects and researchers. In addition, the use of instrumentation and proper cleaning add time and uncertainty to the process. Because we support the integration of many hardware systems used to collect data, we did a survey of our suppliers to better understand their cleaning guidelines.

Basics of Cleaning

There have been many suggestions for destroying SARS-CoV-2, or novel coronavirus, on surfaces. They range from exposing surfaces to ultra violet light, application of high heat, to simply leaving surfaces untouched for extended periods of time. More common methods include use of warm water and soap which has been shown to break down the fatty surface of the virus. It is the basis for recommendations on hand washing. The use of disinfectants, most notably 70% solutions isopropyl alcohol (not ethyl alcohol), is also common.

General Guidelines

The manner of subject instrumentation, subject interaction with computer keyboards and monitors, and the number of researchers involved in data collection will all affect cleaning protocols.

Computers, Monitors, Amplifiers – If possible, keep these devices 6 feet or more from subjects to avoid contamination. If subjects must interact with keyboards and monitors or if multiple researchers interact with the hardware, cleaning with 70% solutions of isopropyl alcohol between each use is recommended.

Subject Instrumentation – Most hard surface instrumentation can be cleaned by wiping with 70% solutions of isopropyl alcohol. This includes instrumentation such as Delsys Trigno sensors, NDI trakStar sensors, Polhemus G4 and Liberty sensors, Noraxon EMG and IMU sensors. Straps and other cloth attachments should be washed with soap and water. EEG headcaps and electrodes (not cable connections) can be submerged in alcohol solutions for a few minutes. Reflective markers and clusters of reflective markers can be cleaned by gently shaking them in a solution of warm water and soap and then rinsing with plain water. Substituting a solution of isopropyl alcohol may be used as a disinfectant but there have been no studies of its effectiveness on this type of surface. Note that the oils of your hands and fingers can degrade marker surfaces, so minimize touching the reflective marker surface with bare skin during cleaning or use.

Company Links

The guidelines above were gathered from published descriptions and private correspondence.

Dell - https://youtu.be/dD6Q6vur2sQ

Delsys - https://delsys.com/downloads/USERSGUIDE/trigno/sensor-cleaning-application.pdf

BioSemi - https://www.biosemi.com/faq/covid19.htm

ANT Neuro - https://www.ant-neuro.com/news/cleaning-and-disinfection-ant-neuro-products-light-corona-covid-19-sars-cov-2-ncov-pandemic

Noraxon - https://www.noraxon.com/sanitization-guidelines/.

Vicon - https://www.vicon.com/support/faqs/?q=how-should-you-clean-markers

AMTI, APDM, Bertec, B&L Engineering, Kistler, NDI, PhaseSpace, Polhemus and Xsens all provided recommendations in private correspondence.

Biofeedback & RealTime Motion Capture

2019-02-27T13:05:00.001-08:00

Biofeedback – What are the possibilities?

With the release of The MotionMonitor xGen, our inaugural blog shared some of the things you have been telling us regarding the state of motion capture software. Biofeedback and its use to understand human movement was one topic with lots of interest. In this blog we will explore the topic and user applications in more detail.

What is biofeedback?

A typical definition of Biofeedback is “a treatment in which patients learn to control bodily processes that are normally involuntary such as muscle tension, blood pressure, or heart rate.” In today’s world of research this definition is way too narrow. The common view probably, flows from early use of surface electromyography (sEMG) to treat disorders ranging from tension headache to joint dysfunction. A more appropriate and generalized definition might be: “Biofeedback is a process in which any raw or processed data derived from human movement, or initiation of movement, is provided to the subject in a control feedback loop with the purpose of modifying that movement through a process of learning.”

With this broader definition the concept of “biofeedback” can use more data to analyze and modify behavior in a wide array of areas that range from performance enhancement in sports to injury reduction in ergonomics to rehabilitation of musculoskeletal injuries and to the rehab and study of motor control systems.

Steven Lavender,PhD at The Ohio State University, in a NIOSH grant proposal that was designed to modify the lifting behavior of manual material handlers, drew on Learning Theory and existing research to explain elements that are necessary for biofeedback to be effective. He noted that complex motor skills are best learned when experienced rather than demonstrated; include "transitional cues" associated with modification of the behavior; and provide knowledge of results in the form of objective data to determine if a positive behavioral change has occurred. In his case, the transitional cues were an auditory signal whose pitch was a function of the moment being generated on the L5/S1 joint. Not your typical biofeedback signal!

Biofeedback clearly has possibilities well beyond that narrow definition.

How is Biofeedback Being Used?

So, how is biofeedback being used in the context of this broader definition and what are the types of feedback being provided as transitional cues? One example is from the field of ergonomics. If, as Steve Lavender suggests, “transitional cues” are important for modifying complex motor skills, visual cues alone may actually hinder learning. In this ergonomic application designed to reduce shoulder elevation, an auditory signal indicating “out of bounds” would be more useful as an alert for the subject to immediately “feel” the danger zone. The visual “stop and go” while useful to the coach, is too distracting for the subject.

Rehabilitation is another area that benefits from a broader definition of biofeedback. Ricardo Matias of Setubal Polytechnic Institute, in an article “Effectiveness of Three-Dimensional Kinematic Biofeedback on the Performance of Scapula-focused Exercises” (https://comum.rcaap.pt/handle/10400.26/7051) demonstrated that the use of kinematic feedback was more effective in modifying behavior than sEMG based feedback during certain shoulder motions. Essentially, his research showed that sEMG, while useful in getting subjects to activate particular muscles, did not necessarily result in the desired change of scapular motion. However, by using kinematic data as feedback, subjects not only modified their movement but also activated the correct muscle combination.

James Thomas, PT, PhD and Peter Pidcoe, PT, PhD at Virginia Commonwealth University are analyzing “how real” virtual reality has to be in order to engage the subject. In their virtual Dodge Ball game which encourages people with back pain to move, they have added tactile feedback in the form of vibrators to provide feedback to body segments that have been “hit” by the opposing team. In this example, visual interaction, reward systems and tactile feedback are providing a deeper level of feedback and a more real virtual world.

In golf, the swing lasts a fraction of a second. This is too little time to monitor visual feedback or indeed to even modify the swing while in progress. However, providing immediate feedback in the form of auditory signals during the swing can provide information that aids learning. For example, to achieve the desired amount of shoulder rotation (technically thoracic rotation) a small interval can be set as a targeted success tone. If no tone is heard the golfer did not rotate far enough. Two tones indicate that they have rotated too far as they pass thru the zone on the back swing and again on the downswing. And one tone was just right indicating they reached the zone but did not over rotate.

Biofeedback has been widely used in motor control. It has been used both as a rehabilitation tool in the case of stroke and as a tool to better understand motor control systems. For example, reaching into an immersive display of virtual objects allows the subject to observe their hand movement in a virtual environment that cannot be reproduced in the real world.

Using error augmentation and a representation of the hand with an immersive display, James Patton, PhD at Rehab Institute of Chicago, displayed a perturbated representation of the hand which induces the subject to overcompensate as part of stroke rehabilitation protocol. Using a similar display, Jill Campbell Stewart, PhD writing in Experimental Brain Research (https://link.springer.com/article/10.1007/s00221-014-4025-7) observed repeated reaches to random targets to discern the level of planning and compensatory adjustments undertaken by stroke patients.

Another application where visuals can be useful is during training. Using biofeedback with therapists as they learn manual therapy mobilization techniques is one example. In this excerpt from a demo video, various visual representations can be used to monitor the amount of force that the therapist is applying. Here a slip graph and bar graph are being used to ensure force is reaching the targeted level.

Of course, there are many other examples of biofeedback applications and research such as gait retraining, augmenting running mechanics, movement sonification and more. We encourage you to write to us at support@TheMotionMonitor.com or in the comments below with your suggestions & questions for biofeedback applications.

What is necessary for biofeedback software to be effective?

For software to be effective as a biofeedback tool there are two important considerations. The first is speed of processing and latency in the display of feedback. Measurement rates and software architecture are extremely important for biofeedback. Processing speed must be sufficiently fast to eliminate latency between action and feedback for the training to be beneficial. At the same time, the underlying collection rate has to be fast enough to satisfy Nyquist if the data were to be valid. We discussed these issues in some detail in our 7/24/2018 blog on Real Time Collection of Biomechanical Data (https://www.themotionmonitorblogteam.com/2018/07/real-time-motion-capture-in-biomechanics.html) and should be reviewed carefully before investing in a biofeedback system.

The second consideration is the User Interface and ease with which biofeedback exercises can be constructed. Subject deficiencies and the variety of research studies suggest that the user must be able to set up exercises quickly. When investigating systems for use in biofeedback, this is arguably one of the most important issues. If one is required to program or write exercises using Visual Basic or other programming languages, the setup time and limited interface will interfere with the successful use of biofeedback.

We'd welcome the opportunity to talk with you about your applications & experiences with biofeedback. Feel free to use the comments section below or send us an email at support@TheMotionMonitor.com.

-Ian

Real Time Motion Capture in Biomechanics

2018-07-26T05:24:00.005-07:00

Real Time Data Collection – What is it & Why Does it Matter?

In March, our inaugural blog identified concepts that our clients had highlighted as important features of motion capture systems. One of the most important was real time data collection. This blog expands on that feature and identifies what constitutes real time data collection, the benefits that arise from real time as well as questions that can help ensure your system will generate the benefits you are hoping for.

What is Real Time? Eddie Cramp put it well in a 1/10/2018 post on Biomch-L: “Some manufacturers will say the real-time means data within ten seconds, some have delays of less than one second, and others will provide the data with 0.01 of a second (10ms) - but they all say that their system is ’real-time’." So when is “real time” truly real time or just fast post processing? Our view is that real time means processing data time-period by time-period, as data are being collected. In effect, the process is: read data, compute data, display data, read data, compute data, display data, etc. Of course the requirement is to do this fast enough that the display of data is virtually instantaneous and the underlying collection rate is fast enough to satisfy Nyquist requirements. More on this later.

Could I benefit from Real Time Collection? Obviously the answer depends on your situation. Here are some benefits we have heard mentioned by clients.

Wow Factor – This is probably the least important reason for considering real time. However, clients have commented on how real time lets them demonstrate their work to important constituencies including decision makers that control lab space and other resources. Interaction with subjects is also enhanced in many settings, especially clinical and athletic settings where the live skeletal animation can excite and motivate patients and athletes. And it is a great way to demonstrate the accuracy of collection systems, with photo-realistic objects and the application of basic biomechanical principles as in this demo playing darts.

Time & Money – Real Time data collection can be a source of significant savings in both time and money. These savings generally come from the elimination of rework and increases in processing speed. How many times have you collected data only to find out long after your subject is gone that dropped markers or some other anomaly made your data unusable? If you are compensating subjects it is not just the lost time, it is also the cost of recruiting additional subjects who satisfy the study criteria. With real time data collection, you know immediately if your data is acceptable. And if not, an immediate re-take is possible before instrumentation is removed from the subject or the subject has left the session.

Processing speed is also a big factor in the cost equation. By eliminating post processing, the time consumed ensuring the integrity of marker trajectories and untangling exchanges is eliminated. And this is time that can be spent analyzing data, writing conclusions or searching for funding…all of which have a higher return to you than data processing. There is a corollary to this called data quality which may be more important than the time savings. Eliminating the time spent fixing marker trajectories also eliminates the possibilities of mistakes. The benefit of better data can be immeasurable.

Of course there is no free lunch. There is an up-front, one-time cost associated with real time data collection. First there is the need for adequate camera coverage which sometimes requires more cameras than may be recommended by the vendors. Emphasis is often placed on camera resolution and measurement speed which favors fewer, higher cost cameras. However, the benefit derived from this approach is usually infinitesimal relative to other sources of error such as soft tissue movement which negates the benefit of higher resolution cameras. Real time collection places emphasis on more cameras to achieve better coverage. And in today’s low-cost camera environment, this approach can often be achieved at a lower cost than the alternative. The other cost is the setup of the collection protocol. Typical collection approaches involve the application of markers; recording of the activity; then marker identification and model definition. Real-time reverses the typical process by first defining the model; then ensuring markers or clusters can be tracked during the activity and then collection of the activity data.

The important fact is that the higher cost of real-time are one-time costs while the typical post processing approach involves an on-going cost that quickly exceed the costs of real time.

Creativity is another benefit that is sometimes mentioned. It is not unusual that data is reviewed or analyzed many months after collection. Anomalies in data that can only be guessed at months after collection, often lead to new insights and research ideas when viewed at the time of collection.

Advances in Neuro & Motor Control research have been facilitated with real time collection. Real time collection enables visual, audio and tactile feedback that is based on movement patterns such as joint angles, joint forces and moments, muscle recruitment, brain waves and point of eye focus. This capability is useful for diagnostics, rehabilitation or performance enhancement. This video is an example of measuring response times to visual stimuli that are presented in a randomized fashion.

With real time and virtual reality, it is also possible to control the environment in ways that cannot be done in the real world. For example, it is possible to introduce perturbations in perceived hand movement to observe how the subject responds. Dr. Jim Patton at the Rehabilitation Institute of Chicago has used these concepts and ErrorAugmentation as a rehabilitation aid.

The introduction of interactive control feedback loops where data on the subject’s movement affects the virtual environment while at the same time monitoring the subject’s response to the environment are all possible with real time data collection and can be used for both basic research and rehabilitation. Dr. Jim Thomas (Virginia Commonwealth University) has used these concepts to target the fear-avoidance model of low back pain using bi-directional feedback as shown in this video.

Measurement rates and software architecture are extremely important for the successful use of real time data collection. Earlier I mentioned that the display of visual feedback had to be sufficiently fast as to eliminate the appearance of latency and the underlying collection rate had to be fast enough to satisfy Nyquist. The ability of the subject to observe, process and react to feedback is one level of specification for display measurement rates. Processing speeds and monitor refresh rates can also limit the speed at which feedback can be presented. These measurement rates are “display rates”. To make real time data collection usable for research, the underlying collection rate must be much faster than the display rate. A good ground rule is to insist that the collection rate be the same as used in post processing data collections. For example, force plate data or EMG data, if normally collected at 1000hz or 2000 hz, should still be collected at 1000 or 2000hz in a realtime setting.

Another issue is the latency or the time between data actually happening and when it is available for processing and display. Each separation between the hardware device and the processing of visual data introduces latency. The only way that we have found to ensure minimal latency and to collect data at high measurement rates is to collect data directly from the underlying hardware.

Questions to ask when evaluating systems

So Eddie Cramp was right. There are lots of claims made for real time collection. To help ensure that your system will result in the benefits you are expecting there are a few questions you should ask and understand before committing to a system.

What is the system architecture? Does the software access data directly from the hardware? And at what measurement rate? Is the measurement rate consistent with the frequency content of the activity being collected?

Is the system real time or just really fast post processing? Is data collected, processed, and collected in a repeated sequence? Or just quickly processed at the end of the recording? Is the processed data available to provide real time feedback during the activity?

What is the system latency between occurrence of the data and display of feedback? Is the latency appropriate for the nature of the study? And more importantly, what is the process for determining the latency?

We'd love to hear about your applications & experiences with real-time! Feel free to use the comments section below or send us an email at support@TheMotionMonitor.com.

-Mona

Hybrid Motion Capture Systems

2018-04-27T08:24:00.001-07:00

Hybrid Systems and How They Enhance Biomechanics Research

In our last posting, we outlined several topics that clients have been requesting over the years as ways to extend and deepen the area of motion capture and biomechanics research. One area we discussed briefly was Hybrid systems.

Hybrid Systems: Only a few clients have actually said, “We need a ‘hybrid system.’ ” But lots of you have said, “I wish I could have the unique advantages that each technology offers in one package.”

It is widely recognized that different technologies have different advantages and disadvantages. Putting technologies together to take advantage of the strengths that each technology offers is what makes hybrid systems so important.

What exactly is a hybrid system? In plain English, it is a system that can switch between technologies. A hybrid car that runs on gas or on electric is an example. In the world of motion capture, a hybrid system is one that can use two technologies for the same purpose. It is a system that can seamlessly switch from using one technology to using a second technology.

One supplier of optical systems, Vicon, recently announced the ability to collect their optical data and inertial measurement units (IMU) data thru Nexus. We would not classify that as a hybrid system, though I suspect they might offer a hybrid system in the future. For now, it is simply collecting two technologies in the same software…something akin to collecting forceplates or emg in the same collection session. In The MotionMonitor xGen, a hybrid system might use both optical and IMU data to track body segments and switch from optical (more accurate) to IMU (no line of sight requirement) when the optical markers are temporarily occluded. Essentially it is a system that is better as a result of the advantages offered by each of the technologies.

The variety of tracking technologies exist because each offers some advantage over competing technologies. Optical tracking is generally considered more accurate but if line of sight issues exist because of the complexity of the activity or limitation on the number of cameras, other technologies that do not require line of sight may be preferable. Workplace settings may be particularly susceptible to line of sight issues as shown in this demonstration video.

Differences in the form factor of technologies is another reason for using a hybrid system. Using tiny reflective markers to track finger movement can be complicated by a number of causes.

Photo courtesy of Natural Point Optitrack website

Finger dexterity can create line of sight issues. The physical presence of objects on the external surface of fingers can interfere with activity. The small sized markers may limit the volume in which the activity can be tracked. And given that marker data is only 3DOF means that only position and not orientation can be tracked with each marker. To obtain orientation data, each marker would be replaced with a rigid body of 3 markers further complicating tracking and interfering with finger movement.

With The MotionMonitor xGen we might substitute, Ascension or Polhemus’ mini 6DOF electromagnetic sensors for markers for this application.

Line of sight is not an issue with these sensors and they provide both position and orientation enabling the use of a single sensor to track the entire finger motion. When used inside surgical or sporting gloves these sensors are nearly imperceptible. Dr Carla Pugh has used this technology as part of her simulation engineering for surgical education lab at the University of Wisconsin: http://www2.surgery.wisc.edu/research/researchers-labs/pugh/technology/

In this demonstration video, a hybrid system consisting of a PhaseSpace active optical system is used to track upper body while Ascension 2mm sensors track the fingers.

Another reason for using a hybrid system is related to the complexity of defining the biomechanical model. Some technologies are easier to transform from the rigid body’s position and orientation to the subject’s anatomical joint centers and coordinate system. For example, IMU’s only provide orientation data. As a result, subject setup requires a tedious physical measurement of segment lengths and the creation of a model within the software. A hybrid system can greatly simplify the process. A stylus tracked with an optical system can be used to duplicate methods from our optical or electromagnetic systems. Subject segment endpoints and landmarks are simply digitized resulting in easier, faster and much more accurate biomechanical model. In this video an optical digitizing stylus is used to locate landmarks on a subject instrumented with Xsens IMU’s.

It is worth noting that this might sound like a high cost approach to using a low-cost technology (both optical and IMU systems being required), however, there are low cost, small volume, optical systems that can be used effectively. There are many solutions for this, the Optitrack Trio shown here is an example. This raises another reason for hybrid; a combination of technologies can provide the ability to expand an existing study or capture a new application without requiring a full hardware replacement or extensive additions, like multiple cameras or larger transmitters. A hybrid approach provides unique solutions at a different price point and may have a cost advantage over the traditional approach, depending on the situation and application.

In summary, it may not yet be possible to have all the advantages of the diverse tracking technologies in one hardware package, but it is possible to combine different technologies via software. Hybrid systems offer a novel, modern solution to many biomechanical questions.

Let us know where you see potential for the application of a hybrid system.

-Meredith

The Next Generation: What concepts should drive the design of tomorrow’s biomechanics software?

2018-03-29T12:03:00.001-07:00

In our inaugural blog I thought it would be worthwhile to share some of the things you have been telling us regarding the state of motion capture. During the 24 years that we have been providing software solutions to the research community you have shared your opinions on what works, what doesn’t and what functionality you would like to see in the years ahead. Here are some of the things we have heard:

Real Time One request that you have raised often is the desire to see live data and skeletal animations streaming during data collection. There are so many benefits to this: the ability to quality check while data is collected, the ability to give biofeedback; and the simple “wow” factor that gets subjects, patients, and athletes excited about being part of a data collection. The MotionMonitor Classic has always been able to do this but in a more limited way. And we were not alone, many companies say they are “real-time” but they rarely mean that data is instantaneous. Eddie Cramp put it well in a recent post on Biomch-L: “Some manufacturers will say the real-time means data within ten seconds, some have delays of less than one second, and others will provide the data with 0.01 of a second (10ms) - but they all all say that their system is ’real-time’". It’s not a simple problem; and as Eddie suggests, researchers should take pause and be sure they understand what their vendor means by “real time”. In our next blog, we’ll be going into detail on this topic.

Mobile Another request we often hear is the ability to collect data in the field. Mobile applications provide many benefits, as they provide solutions for common questions, like: “Do the tightly controlled experiments in the lab actually translate to the real world?” or “How do we reach patient populations that cannot make it to the lab?” But there are many challenges to making mobile a reality. The equipment has to work in a variety of settings, and of course, you are always concerned about the time it takes to setup hardware and instrument subjects. A lot of folks in the business have tried to make their systems “mobile”. The optical providers have successfully achieved data collection outdoors, but management of sun-light and setup time still remains a real challenge. And the proliferation of IMU’s and special purpose apps run on iPhones have made it easier to setup and go, but at a sacrifice of informational content and data accuracy. We believe true mobile for research is not a simple problem, but one that must be addressed from the ground up.

Hybrid Only a few clients have actually said, “We need a ‘hybrid system.’” But lots of you have said, “I wish I could have the unique advantages that each technology offers in one package.” While it is not possible to have them in one hardware package, it is possible to combine different technologies via software, so that a subject can be instrumented with multiple hardware systems. Common applications where combined hardware is beneficial include i) the use of IMU’s and optical systems when line of sight is a problem, ii) the use of optical systems to digitize anatomical landmarks on subjects instrumented with IMU’s, and iii) use of 6 degree of freedom electromagnetic sensors for tracking fingers while upper extremities are tracked with optical markers.

Make it More Powerful Research is always about the next question. And software has to be more powerful to help answer those questions. Integrated Muscle Modeling and advanced math and multivariate statistical operators like entropy and fractals; the ability to introduce user code to the software; and, the ability to define data and write code without being a programmer are frequent examples of things you have requested. The benefits of these are easy to see. Implementation that lends itself to continuous enhancement in functionality without major overhaul of the software has been more difficult to achieve.

Make it Simpler Often researchers, athletic trainers or ergonomist have situations where they need to study one motion or exercise repetitively. In these cases, you don’t need full body tracking or diverse data types with many choices, rather you need an application that is simple, with the ability to easily repeat data collection and analysis. We have spent a lot of time trying to reconcile this comment with the previous comment. What we have found is that to make something appear simple (we prefer “elegant”) requires power! We’ll spend a whole posting on this topic where we demonstrate that “Powerful” does not have to mean “Complicated.”

Make it More Flexible This comment is seldom expressed so directly. More frequently it is expressed in the questions our tech support people get…”How do I plot my graph vertically?”, “When locating joint centers, how can I use the functional method for the shoulder and a digitizing method for the knee?”,“Can I use my IMU’s with my camera system?”, “I want text to pop up in the animation window as a signal to the subject.”, and on and on and on. We’ve been keepin’ a list! In future posts we’ll be exploring how we have chosen to implement them in the next generation of The MotionMonitor and how these capabilities can be used. In the meantime, look at the bike fit application as an example of what this flexibility enables a researcher to do.

Support Translation of Client Research. This is a topic that has resulted in much consultation between our engineers and some of our clients. Seeing your research protocols transformed into real world solutions, such as in a clinic, workplace or sports arena, is what motivates you. And when the time comes, The MotioMonitor xGen can be a great tool to help you get there. The Manual Therapy Trainer, championed by Dr. Eric Shamus, is a fine example. We’ll highlight other examples and how they were accomplished in future posts.

Biofeedback and Understanding Human Movement. As technology advances, many of you are seeking the answers to deeper, more complex biofeedback questions. These questions require complex control feedback loops to gain a better understanding of human movement. Many of our clients are pioneers in this respect. When we founded our company, our original raison d’etre was to provide real-time feedback to subjects as a way of improving performance in various sporting activities. So, this comment is one we understand deeply.

At the same time, as clinical and athletic institutions move to embrace technology, the number of applications for biofeedback within society are rapidly expanding. There’s a demand to show data in different ways, beyond just time-series graphs, and to use these data to drive changes in behavior. To convey feedback, you want live animations; ones that you can control and design, but without the need to do your own programming. We’ve worked hard to make this possible; minimizing latency, increasing complexity and providing true flexibility, all while utilizing research-quality data for biofeedback and analysis.

This summary is the collective view of The MotionMonitor team. As such it represents our guide to changes we introduce to the software and the way we organize to provide support and training to our clients. A lot is happening at the home of The MotionMonitor. The biggest of these is the release of our latest version of software that we have dubbed The MotionMonitor xGen. Five years in the making, this is the most significant offering in our history. The ideas and questions you have raised and are summarized in this post were the guiding principles for its development. In future posts, we’ll try and share how you can use these concepts to maximum benefit.

-Lee