About

Integration of Data across Disparate Sensing Systems Over Both Time and Space to Design Smart Environments.

This work addresses a set of general problems during a design of smart spaces. The problems are related to knowing where sensing takes place, at what time, of what measurement type and what the interpretation of acquired measurements is for event detection, recognition and proactive event-driven action purposes.

The work outlines multi-instrument and sensor measurement systems that provide sensing capabilities for smart spaces, as well as theoretical and practical limitations that have to be understood while working with novel sensing technologies. The disparate sensing systems include wireless micro-electro-mechanical systems (MEMS) sensor networks (such as MICA sensors by Crossbow Inc.) and cameras that capture a variety of spectra (such as visible, thermal infrared and hyperspectral information). The sensors represent a mobile system over both time and space by engaging a robot with its robotic arm. The robot performs remotely controlled sensor deployment, and adaptive sensor placement based on the sensing feedback.

Our solution addresses:

  • robotic sensor deployment using various human-computer interfaces,
  • synchronization of sensors and cameras,
  • localization of sensors and objects by fusing the acoustic time-of-flight
  • localization and vision stereo approaches,
  • mutual calibration of measurements from sensors and cameras,
  • proactive camera control based on sensor readings, and
  • hazard detection and understanding.

Our prototype solution demonstrates primarily the integration of data across disparate sensing systems over both time and space.

Hazard aware spaces (HAS) prototype

Figure 1: An overview of the current prototype.

We applied the prototyped solutions to hazard detection and human alert in the context of hazard aware spaces (HAS). The goal of HAS is to alert people in the event of dangers like natural disasters, failures of human hazard attention, and intentionally harmful human behavior. Our prototype HAS system integrates both emerging and standard sensing technologies in order to deliver sensor and camera data streams to a central location. At the central location, our data analysis algorithms detect hazards to alert humans, and possibly analyze hazard sources. Once alerted decision makers know when, where and what events occur, they can use the gesture/voice/keyboard controlled robot to confirm the presence of hazards using sensor and video feedback, as well as to attempt to contain the hazards with the robot.

Localization using Passive RFID Technology.

We investigate the use of passive RFID technology for object localization and tracking. This work is motivated by the desire to design, and build, robust smart spaces (pervasive spaces), and in particular their application in the area of hazard aware spaces. In particular we are interested in building spaces that can detect hazards and autonomously take action to gain more information about the hazard and alert the users of the space. To enable a robot to operate in such a space, accurate localization and tracking must be carried out, allowing the robot to then sense and detect hazards and alert humans about when, where, and what hazards occur. We evaluate the strengths and weaknesses of passive RFID technology as a solution to this problem. In addition, we have researched and developed a methodology for building a sensor model which is required for accurate localization. We applied one specific method of performing probabilistic localization that demonstrated successful global localization of a robot using these approaches.

Tag types used in HAS prototype Tag types used in HAS prototype

Figure 2: Tag types used in our experimental analysis.

Documentation of setup of sensor model Detection range 30deg

Figure 3: Left: documentation of setup used to build the sensor model. Right: Image showing nominal detection range when Tags are distributed on the floor and the Reader is mounted on the robot and tilted forward 30 degrees. The 4th "row" has a radius of roughly 90 cm.

Alien Reader mounted on a robot RFID tags in a regular grid

Figure 4: Experimental setup. The Alien Reader was mounted on the platform of a robot (left). The RFID tags were placed on the floor in a regular grid pattern (right).

Experimental run 1 Experimental run 2 Experimental run 3

Figure 5: These images show three time steps in the middle of an experimental run. Open circles represent undetected tags. Filled blue circles represent detected tags. The red circle represents the average robot location, and a green circle represents a ground truth measurement.


People, Publications, Presentations

Collaborators

  • Peter Bajcsy
    Image Spatial Data Analysis Group, NCSA, UIUC
  • Rob Kooper
    ISDA, NCSA, UIUC
  • David Scherba
    MS student in ECE Department and ISDA, NCSA, UIUC
  • Martin Urban
    MS student in ECE Department and ISDA, NCSA, UIUC
  • Miles Johnson
    MS Student in Aerospace Department and ISDA, NCSA, UIUC
  • Kyaw Soe
    MS student in General Engineering Department and ISDA, NCSA, UIUC

This material is based upon work partially supported by the ONR funding. We acknowledge NCSA/UIUC support of this work.

Conference Papers

  1. "Fusion of Voice, Gesture, and Human-Computer Interface Controls for Remotely Operated Robot"
    M. Urban and P. Bajcsy, 8th International Conference on Information Fusion, July 25-28, 2005, Philadelphia, Pennsylvania
    [pdf]
  2. "Depth Map Calibration by Stereo and Wireless Sensor Network Fusion"
    D. Scherba and P. Bajcsy, 8th International Conference on Information Fusion, July 25-28, 2005, Philadelphia, Pennsylvania
    [pdf]
  3. "Recognition of Arm Gestures Using Multiple Orientation Sensors: Repeatability Assessment"
    M. Urban, P. Bajcsy, R. Kooper and J-C Lementec, 7th International IEEE Conference on Intelligent Transportation Systems, October 3-6, Washington, D.C., 2004, pp 553-558.
    [pdf]
  4. "Recognition of Arm Gestures Using Multiple Orientation Sensors: Gesture Classification"
    J-C Lementec and P. Bajcsy, 7th International IEEE Conference on Intelligent Transportation Systems, October 3-6, Washington, D.C., 2004, pp 965-970.
    [pdf]
  5. "A New Thermal Infrared Camera Calibration Approach Using Wireless MEMS Sensors"
    P. Bajcsy and S. Saha, Communication Networks And Distributed Systems Modeling And Simulation Conference (CNDS 2004), January 19-22 2004, San Diego, California.
    [pdf]
  6. "System Design Issues in a Single-Hop Wireless Sensor Network"
    S. Saha, and P. Bajcsy, 2nd IASTED International Conference on communications, Internet and information technology (CIIT), Novemeber 2003, pp. 743-748
    [pdf]
  7. "Understanding Multi-Instrument Measurement Systems"
    P. Bajcsy, Understanding Complex Systems Symposium, May 17-20, 2004, UIUC
    [Abstract]

Technical Reports

  1. "Toward Hazard Aware Spaces: Localization using Passive RFID Technology"
    P. Bajcsy, R. Kooper, M. Johnson, K. Soe, ISDA06-002, May 25, 2006.
    [pdf]
  2. "Depth Estimation by Fusing Stereo and Wireless Sensor Locations"
    D. Scherba and P. Bajcsy, Technical Report NCSA-ALG-04-0008, October 2004.
    [pdf]
  3. "Recognition of Arm Gestures Using Multiple Orientation Sensors"
    M. Urban, P. Bajcsy and R. Kooper, Technical Report NCSA-ALG-04-0004, July 2004.
    [pdf]
  4. "Communication Models for Monitoring Applications Using Wireless Sensor Networks"
    D. Scherba and P. Bajcsy, Technical Report NCSA-ALG-04-0003, October 2004.
    [pdf]
  5. "System Design Issues for Applications Using Wireless Sensor Networks"
    S. Saha and P. Bajcsy, Technical Report NCSA-ALG-03-0003, August 2003.
    [pdf]