casestudy: Associated Motion Capture User Categories

The AMUC e-science demonstrator has provided a unique opportunity for performing artists, biomechanics and computing science specialists to collaborate in practice-driven creative research at Culture Lab, Newcastle University's interdisciplinary digital platform.

AMUC proposes an original query system for motion capture data, where sketched rather than textual inputs serve to retrieve information from a data base. By virtue of this "intuitive" graphic interface which eludes the constraints of multiple specialist languages, the proposed system might conceivably enhance value of a data resource for a diverse range of users. While this proposal in turn raises a set of challenging issues (choices of mappings, feature extraction methods, etc), the demonstrator serves first and foremost as a platform to generate dialogue and pragmatic experimentation amongst researchers from different disciplines engaged with human motion and its traces.

Project partners

Culture Lab

Sally Jane Norman
Martyn Dade-Robertson – RA
Dave Green - Technician

North East Regional e-Science Centre

Paul Watson
Hugo Hiden – NEReSC Technical Manager

Centre for Rehabilitation & Engineering Studies

School of Mechanical & Systems Engineering
Sian Lawson
Anita Chan - RA

Informatics Research Institute

Patrick Olivier
Paul Dunphy – RA
Jonathan Hook – RA
Dan Jackson – Technical developer

Funding body:

AHRC-JISC-EPSRC Arts and Humanities e-Science Initiative (EPSRC)

Type of project:

6 month demonstrator, with a clear view to future development.

Application area

Performance Arts, Visual Arts

Bibliography and links

• Culture Lab
• AMUC Project Report (see attached file)
• Project Website
• OMII UK Project Website
• ISGTW Article

Project contact details

• Principal Investigator: Dr Sally-Jane Norman, University of Newcastle

Description of aims

Associated Motion capture User Categories is an exploratory project focused on interdisciplinary differences encountered in the gathering and processing of motion capture data, and use of these differences to inform design of a motion capture database to be deployed across the Grid. It sought to lay the basis for longer-term development of e-Science resources for Arts and Humanities research involving interdisciplinary collaboration. Moreover, with a view to establishing the reciprocity on which collaboration depends, it aimed to demonstrate the potential value of arts and humanities driven research for the broader e-science technology development community.

In pragmatic terms, the project’s main goal was to propose a prototype data retrieval tool, allowing movement features or sequences to be called up from a motion capture database. Motion capture recordings produce a high dimensional dataset which is both continuous (through its reflection of a dynamic process) and discrete (through its division into sampling intervals or rates), where the tracking and processing of markers on a moving body can potentially be made to yield multiple kinds of information. Exploitation of a dataset of this complexity calls for novel index-based retrieval tools, where annotated data features can be identified in response to multiple, composite query criteria. To answer this challenge, the AMUC Sketch Retrieval Client harnessed graphic input via an electronic tablet so this could be used as a query mechanism, time and position signals obtained from the sketch being mapped/ matched to the properties of data streams stored in the motion capture repository.

Description of methods

The AMUC project was built around a core Arts and Humanities constituency from the live performance sector, engaging with practitioners and theorists from dance and choreography, music (instrumental, compositional, conducting), juggling and acrobatics, and martial arts. Practitioners offered a broad spectrum of movement techniques and approaches, translated as highly diverse metadata environments and requirements. Bioengineering experts accustomed to capturing and analysing large amounts of complex human motion data were closely involved in recording and post-production processing activities, i.e. providing data for integration into the prototype retrieval tool. Computing science skills mobilised for the project ranged from programming to design of interfaces and interactive display systems, development of automatic graphical recognition means, and Grid computing competencies (projection of workflow enactment systems, server distribution logics, etc). Humanities specialist input on information system design and navigation informed overall reflection and discussion. Creative and technical expertise with digital images and motion capture (filming, recording, editing, referencing for optimal access) were key to overall project logistics and organisation.

As it is unique to this project, let us focus on the sketch-based data search and retrieval system. In it the user inputs a sketch using a mouse/pen/on-screen tablet. The user can then choose the features to be investigated. The results are presented in a relevance order, and the user can view a video of each match. Searching in the proposed system consists firstly of identifying a meaningful channel of data and secondly of identifying "features" in the input and mapping them to "features" in the dataset, where useful channels of data may include position, velocity, acceleration, symmetry, potential energy, "excitement", etc. Features are identified by the indexer which detects, for example, a rapid change, a peak, or inactivity. Indexes allow quick feature searches, so that the whole of the data need not be processed and searched for each enquiry.

Description of outcomes, or intended outcomes

Much recent work has involved creation of motion capture libraries for cinematographic and games industries, as well as for edutainment, advertising, training manuals and simulators. This activity tends to focus on commodified products for predefined target groups rather than on building frameworks to optimise shared resources generated by the user communities themselves. Yet the latter resources are increasingly valuable given the current escalation in content growth (e.g. uploads of motion capture sequences via Web 2.0), and the need such content creates for new kinds of search and retrieval methods. The development of motion capture databases thus appears to offer a particularly interesting area for testing novel Grid affordances.

Technologies

• Digital imaging uses the Vicon motion capture equipment configuration and calibration
• Grid specialist storage interface for all elements of the system, including indexing processes to perform feature extractions on stored motion capture data, possibly via multiple "back end" computers with a technology such as GRID-SAM (Grid Submission & Monitoring Service: )
• Retrieval tool is written in C# on the .NET framework

Report

The final report is attached.

This case study was written for AHeSSC, the Arts and Humanities e-Science Support Centre. It is published here with permission from AHeSSC.