John Roston's Research

The Simulator allows students to practice their negotiation skills while trying to sell a commuter aircraft to the Vice-President of Olympic Airways in Greece who appears on screen and talks to the student. The student selects a reply from a menu. It has been used in a number of university MBA programs in Canada, the U.S. and Australia.

The initial version was made in 1992 before MPEG video compression (currently mp4) became available. Video was stored on large 12" videodiscs and input to a computer which overlaid graphics on the video. The Simulator was subsequently updated as video technology improved.

Located in the McGill Schulich School of Music's Centre for Interdisciplinary Research in Music Media and Technology (CIRMMT), the Group was mainly concerned with co-presence, the feeling that you are sharing the same space with others who are at a remote location. In addition to being the Coordinator for the projects listed below, my role was the acquisition and display of panoramic high definition video.

Wieslaw Woszczyk, professor of Music, was responsible for the very high quality surround audio. Jeremy Cooperstock, professor of Computer Engineering, was responsible for the control software and the rapid transmission of the video and audio over high bandwidth networks to the far end. There were also major contributions from many colleagues and contributors, to say nothing of the programmers, technicians and graduate students who did the heavy lifting.

An early project addressed the shortage of sign language interpreters for medical appointments and procedures. It simulated a medical examination of a deaf patient by an orthopaedic surgeon with the interpreter being located at an interpretation centre in a remote city.

The interpreter appeared both on a screen at the foot of the bed for the patient and on another screen at the head of the bed for the surgeon. The interpreter could see both patient and surgeon simultaneously. My role was to design the screen and camera arrangement which was awarded a U.S. patent.

In addition to transmitting high quality video and audio, this project added vibration data so that an audience in one city could see, hear and also feel the vibrations created by the music being performed in another city. This emulates a jazz club where the music is not just heard, but also felt through the floor. Vibration platforms were used to pick up vibrations from the performers and reproduce them at the far end underneath the spectators' chairs.

The project used high speed transmission of panoramic video, surround sound, and vibration data to teach music students at McGill and the University of British Columbia with the instructor at one end and the student at the other. In this case, the UBC end was moved to Seattle for the 2005 SuperComputing Conference with the instructor, Gordon Foote, on the conference floor teaching a student jazz ensemble at McGill.

As part of the Bandwidth Challenge competition at the Conference, Gordon and the students played together demonstrating for the first time that it is possible for musicians to play together across the continent, without the use of synchronization devices, if the networks are fast enough. The system won the Most Innovative Use of New Technology award. It has not been possible to play together over even greater distances due to limitations imposed by the speed of light.

The project used panoramic video and surround sound to enable a student alone in a rehearsal room to have the experience of playing in an orchestra. On the left, the camera rig is recording from the first violin position. On the right, a double bass player sees what he would see at that position in the orchestra and the score appears on a touchscreen under the centre monitor.

A separate recording was done at each instrument position with that instrument not playing. Afterward the system would record the student playing the missing instrument in a rehearsal room. The student could play back his or her performance with or without the orchestra. The recording was then automatically submitted to an instructor in another city for comment.

Recordings were done of jazz and classical orchestras at McGill and opera at UBC. For opera, the camera moved as the performer would on stage.

The project was an early effort to transmit broadcast quality high definition video from the seafloor over the Oceans Network Canada undersea internet network connecting instrument observatory clusters designed for marine research as well as earthquake and tsunami early warning. The idea was to transmit video fast enough that scientists at a great distance could control the camera while watching the output. The project was a collaboration between McGill and the University of Victoria.

For one experiment, the camera was deployed to study the underwater decomposition of a dead pig, particularly the skin which resembles that of a human body, for use in the forensic examination of murder victims recovered from water.