The Autism Emulation Project

Home > Projects > Autism

News & Announcements

Aug. 21, 2006

Congratulations to Jennifer Chung for winning the Best Poster Award at the University of Toronto Enginering Science Research Day!

Summary

Contents (hide)

  1. 1. Introduction
  2. 2. Device Implementation
    1. 2.1 Hardware
    2. 2.2 Software
  3. 3. References
  4. 4. Publications
    1. 4.1 Poster Presentations
    2. 4.2 Invited Lectures
  5. 5. Team
    1. 5.1 Research
  6. 6. Links

Understanding alternative perception schemes that reflect some of the experiences of people with autism could help us exploit the advantages of this condition and support the social integration of people with autism by encouraging a more balanced and empathetic adaptation of neuro-typical individuals. These goals may be supported through the design and implementation of mediated-reality devices that approximately emulate the auditory and visual experience of autism. An exhaustive literature review, previously reported case studies and first-hand accounts as well as semi-structured interviews informed our design process. Using this information, a prototype emulator was built by the novel combination of available audio/visual technologies and real-time signal and image processing algorithms. Additionally, a computer-based graphical user interface was implemented to allow for the straightforward customization of the device parameters. Future generations of this proof-of-concept autism sensory emulation device will be validated qualitatively and used to increase our understanding of Autism Spectrum Disorders (ASD).

1.  Introduction

Autism is an umbrella term for a complex spectrum of neurological conditions that may impact, among many other neural processes, sensory information processing by the brain. These atypical sensory information processing patterns usually translate, in varying degrees, into atypical perception schemes in people with autism. The advantages of these altered perception schemes may include, for instance, a hightened sensitivity to visual and auditory stimuli which may result in extraordinary artistic ability. However, social isolation, repetitive and insistent behavior, as well as verbal and non-verbal communication difficulties may also result when there is sensory overload. Understanding the perception schemes of people with autism may help us create tools, places and therapies that take advantage of the benefits of the different perception schemes characteristic of autism, while reducing their less advantageous consequences.

We performed an exhaustive literature review, considered previously reported case studies and first-hand accounts, and completed a number of semi-structured interviews in order to inform the design of a device that may be used to recreate a subset of the autism sensory experience for neuro-typical (i.e. non-autistic) individuals. We focused our design on the emulation of particular auditory and visual perception schemes. As a result, a prototype emulator was built by the novel combination of available audio/visual technologies and real-time signal and image processing algorithms.

2.  Device Implementation

The autism emulation device is a mediated-reality interface between the environment and the user that alters his/her sensory visual and auditory experience according to information extracted from personal accounts by people with autism.

2.1  Hardware


Fig. 1. Autism emulation device hardware.


Fig 2. Eyewear of the autism emulation device.

The hardware can be divided into equipment worn by the user and equipment remote from the user where video and signal processing occurs (figure 1). Some of the equipment worn by the user intakes video and audio of what the user would normally see and hear without the device. In particular, we use a colour board camera and two sub-mini omnidirectional lavalier microphones. The microphones are plugged into phantom power and high-impedance unbalanced to low-impedance balanced transformers. The signals are sent via a 2.4 GHz transmitter worn by the user to its complementary receiver at the remote station. At the remote station, the analog video signal is acquired digitally into a computer. After processing, the video is extracted through a DVI-D port and converted back into an analog video signal before it is sent to another 2.4 GHz transmitter (set at a different channel) on the remote side. In turn, after pre-amplification, the audio signals from the microphone pair are acquired on the computer using an external soundcard. Processed audio signals are sent from the computer, through the same soundcard, to the remote-side transmitter for wireless transmission back to the user.

The video signal is displayed by an eyewear that is buried within lightweight light-occluding foam (figure 2). The audio signal is displayed to the user through noise-blocking canal phones.

2.2  Software

Tables 1 and 2 describe the visual and auditory transformation algorithms implemented, respectively. A computer-based software program and graphical user interface allows for adjustments of the parameters governing these algorithms.

Table 1: Visual Transformation Algorithms

Autism-related condition Description Implemented algorithm
Reduced Gestalt (figure 3.B) Gestalt is the ability to self-organize components of a scene into a cohesive whole. Frames are divided into a grid and the cells are randomly distributed
Hypersensitivity to bright highlights (figure 3.C) Hypersensitivity to brightness is frequently described in individuals with autism. They can also be drawn into brightly coloured patterns. Areas on a frame that pass a threshold of brightness are made brighter while the remaining areas are made duller. This effectively highlights bright areas on each frame.
Visual shutdown (figure 3.D) In response to overwhelming visual stimuli, the amount of visual information being processed can drop significantly as a coping mechanism. A gaussian pattern is used to decrease the visual field in response to the amount of information that must be processed. This amountof information is estimated by quantifying the change between consecutive frames.


Fig. 3. Sample visual transformations implemented with the autism emulation device. Figure 3.A is the original frame. Table 1 includes details on the implementations.

Table 2: Auditory Transformation Algorithms

Autism-related condition Description Implemented algorithm
Hightened frequency sensitivity This condition is characterized by hypersensitivity to certain frequencies. Frequencies in the sensitive range are bandpass filtered and amplified.
Spatial disorientation Reduced ability to judge spatial orientation, which often affects motor coordination. Sense of direction is altered by combining signals from the left and right microphones in different proportions for one earphone, and in opposite proportions for the other earphone.
Voice distortion Reduced ability to distinguish between voice and background noise. The frequency range where voice usually resides is bandpass filtered and equalized to reduce intelligibility.
Audio shutdown In response to overwhelming auditory stimuli, the amount of auditory information being processed can drop significantly as a coping mechanism. The volume of the audio signal is decreased and a lowpass filter is applied to produce a muffled effect. A sudden increase in the amplitude (i.e. volume) of the audio signal triggers this transformation.

3.  References

  1. Dawson, G., Watling, R.. Interventions to Facilitate Auditory, Visual, and Motor Integration in Autism: A Review of the Evidence. Journal of Autism and Developmental Disorders. 30(5), 2000.
  2. Iarocci, G., McDonald, J.. Sensory Integration and Perceptual Experience of Persons with Autism. Journal of Autism and Developmental Disorders. 36(1), 2006.
  3. Rogers, S.J., Ozonoff, S.. Annotation: What do we know about sensory dysfunction in autism? A critical review of the empirical evidence. Journal of Child Psychology and Psychiatry. 46(12), 2005.
  4. Bagdashina, O., Lawson, W.. Sensory Perceptual Issues in Autism: Different Sensory Experiences - Different Perceptual Worlds. Jessica Kingsley, 2003.

4.  Publications

4.1  Poster Presentations

  1. Chung J., Silva J. and Chau T., “A Mediated-Reality Device Based on the Autistic Sensory Experience,” Engineering Science Research Day, University of Toronto, ON., Canada, August 21, 2006. Winner of the Best Poster Award.

4.2  Invited Lectures

  1. Silva J., “The Autism Emulation Project and the New Social Studies of Childhood and Disability,” Child Life Program, McMaster Children’s Hospital, Hamilton ON., Canada, March 10, 2006.

5.  Team

5.1  Research

6.  Links