Towards on-site collaborative robotics: Voice control, co-speech gesture and context-specific object recognition via ad-hoc communication

Thibault Schwartz, Sebastian Andraos, Jonathan Nelson, Christopher Knapp, Bertrand Arnold

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

Abstract

This work presents a novel set of accessible, unified hardware and software solutions facilitating the implementation of natural human-machine interactions,as required by collaborative robotics in both indoor and outdoor environments.This extensible framework supports vocal control, co-speech gestures,and object recognition with feature tracking and adaptive resolution. The interactions are based on a new network messaging protocol allowing any device using TCP/IP to share variables with the full abstraction of the original machine software platform and can therefore be used synchronously by a vast array of equipment including CNC machines, industrial robots, construction equipment,mobile devices and PLCs. We conclude with the description of a testing scenario to be deployed during the conference workshop.
Original languageEnglish
Title of host publicationRobotic Fabrication in architecture, art and design 2016
EditorsD Reinhardt, R Saunders, J Burry
Place of PublicationSwitzerland
PublisherSpringer
Pages388-397
Number of pages11
Volumeiv
ISBN (Electronic)978-3-319-26378-6
ISBN (Print)978-3-319-26376-2
DOIs
Publication statusPublished - 2016
EventInternational Conference of the Association for Robotics in Architecture : Robotic Fabrication in Architecture, Art and Design - Pier 2/3 Walsh Bay, Sydney, Australia
Duration: 18 Mar 201619 Mar 2016
http://www.robarch2016.org/conference/

Conference

ConferenceInternational Conference of the Association for Robotics in Architecture
Abbreviated titleROB ARCH
CountryAustralia
CitySydney
Period18/03/1619/03/16
Internet address

Fingerprint

Construction equipment
Gesture recognition
Industrial robots
Object recognition
Programmable logic controllers
Mobile devices
Robotics
Hardware
Communication
Testing

Cite this

Schwartz, T., Andraos, S., Nelson, J., Knapp, C., & Arnold, B. (2016). Towards on-site collaborative robotics: Voice control, co-speech gesture and context-specific object recognition via ad-hoc communication. In D. Reinhardt, R. Saunders, & J. Burry (Eds.), Robotic Fabrication in architecture, art and design 2016 (Vol. iv, pp. 388-397). Switzerland: Springer. https://doi.org/10.1007/978-3-319-26378-6_31
Schwartz, Thibault ; Andraos, Sebastian ; Nelson, Jonathan ; Knapp, Christopher ; Arnold, Bertrand. / Towards on-site collaborative robotics: Voice control, co-speech gesture and context-specific object recognition via ad-hoc communication. Robotic Fabrication in architecture, art and design 2016. editor / D Reinhardt ; R Saunders ; J Burry. Vol. iv Switzerland : Springer, 2016. pp. 388-397
@inproceedings{a8f78977d4e04b829881543304078ae4,
title = "Towards on-site collaborative robotics: Voice control, co-speech gesture and context-specific object recognition via ad-hoc communication",
abstract = "This work presents a novel set of accessible, unified hardware and software solutions facilitating the implementation of natural human-machine interactions,as required by collaborative robotics in both indoor and outdoor environments.This extensible framework supports vocal control, co-speech gestures,and object recognition with feature tracking and adaptive resolution. The interactions are based on a new network messaging protocol allowing any device using TCP/IP to share variables with the full abstraction of the original machine software platform and can therefore be used synchronously by a vast array of equipment including CNC machines, industrial robots, construction equipment,mobile devices and PLCs. We conclude with the description of a testing scenario to be deployed during the conference workshop.",
author = "Thibault Schwartz and Sebastian Andraos and Jonathan Nelson and Christopher Knapp and Bertrand Arnold",
year = "2016",
doi = "10.1007/978-3-319-26378-6_31",
language = "English",
isbn = "978-3-319-26376-2",
volume = "iv",
pages = "388--397",
editor = "D Reinhardt and R Saunders and J Burry",
booktitle = "Robotic Fabrication in architecture, art and design 2016",
publisher = "Springer",
address = "Germany",

}

Schwartz, T, Andraos, S, Nelson, J, Knapp, C & Arnold, B 2016, Towards on-site collaborative robotics: Voice control, co-speech gesture and context-specific object recognition via ad-hoc communication. in D Reinhardt, R Saunders & J Burry (eds), Robotic Fabrication in architecture, art and design 2016. vol. iv, Springer, Switzerland, pp. 388-397, International Conference of the Association for Robotics in Architecture , Sydney, Australia, 18/03/16. https://doi.org/10.1007/978-3-319-26378-6_31

Towards on-site collaborative robotics: Voice control, co-speech gesture and context-specific object recognition via ad-hoc communication. / Schwartz, Thibault; Andraos, Sebastian; Nelson, Jonathan; Knapp, Christopher; Arnold, Bertrand.

Robotic Fabrication in architecture, art and design 2016. ed. / D Reinhardt; R Saunders; J Burry. Vol. iv Switzerland : Springer, 2016. p. 388-397.

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

TY - GEN

T1 - Towards on-site collaborative robotics: Voice control, co-speech gesture and context-specific object recognition via ad-hoc communication

AU - Schwartz, Thibault

AU - Andraos, Sebastian

AU - Nelson, Jonathan

AU - Knapp, Christopher

AU - Arnold, Bertrand

PY - 2016

Y1 - 2016

N2 - This work presents a novel set of accessible, unified hardware and software solutions facilitating the implementation of natural human-machine interactions,as required by collaborative robotics in both indoor and outdoor environments.This extensible framework supports vocal control, co-speech gestures,and object recognition with feature tracking and adaptive resolution. The interactions are based on a new network messaging protocol allowing any device using TCP/IP to share variables with the full abstraction of the original machine software platform and can therefore be used synchronously by a vast array of equipment including CNC machines, industrial robots, construction equipment,mobile devices and PLCs. We conclude with the description of a testing scenario to be deployed during the conference workshop.

AB - This work presents a novel set of accessible, unified hardware and software solutions facilitating the implementation of natural human-machine interactions,as required by collaborative robotics in both indoor and outdoor environments.This extensible framework supports vocal control, co-speech gestures,and object recognition with feature tracking and adaptive resolution. The interactions are based on a new network messaging protocol allowing any device using TCP/IP to share variables with the full abstraction of the original machine software platform and can therefore be used synchronously by a vast array of equipment including CNC machines, industrial robots, construction equipment,mobile devices and PLCs. We conclude with the description of a testing scenario to be deployed during the conference workshop.

U2 - 10.1007/978-3-319-26378-6_31

DO - 10.1007/978-3-319-26378-6_31

M3 - Conference contribution

SN - 978-3-319-26376-2

VL - iv

SP - 388

EP - 397

BT - Robotic Fabrication in architecture, art and design 2016

A2 - Reinhardt, D

A2 - Saunders, R

A2 - Burry, J

PB - Springer

CY - Switzerland

ER -

Schwartz T, Andraos S, Nelson J, Knapp C, Arnold B. Towards on-site collaborative robotics: Voice control, co-speech gesture and context-specific object recognition via ad-hoc communication. In Reinhardt D, Saunders R, Burry J, editors, Robotic Fabrication in architecture, art and design 2016. Vol. iv. Switzerland: Springer. 2016. p. 388-397 https://doi.org/10.1007/978-3-319-26378-6_31