- < NEW >I'm giving an invited talk "From Semantic World Understanding to Collaboration with Deep Representations" at the IROS'18 workshop Language in Robotics.< NEW >Check out the video presenting our new end-to-end probabilistic deep approach to large-scale semantic mapping.
< NEW >I'm serving as an Associate Editor for ICRA 2019.< NEW >Our paper "Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps" was accepted for oral presentation at AAAI 2018. - Sep 2017I'm presenting the paper "Learning Deep Generative Spatial Models for Mobile Robots" in the Deep Learning in Robotics and Automation session at IROS 2017.
Sep 2017I'm chairing the sessions on Deep Learning in Robotics and Automation and RGBD Perception at IROS 2017.Sep 2017I'm serving as an Associate Editor for ICRA 2018. - Jul 2017I'm presenting two papers at the RSS'17 workshop on Spatial-Semantic Representations in Robotics.Jul 2017Our paper "Learning Deep Generative Spatial Models for Mobile Robots" was accepted to IROS 2017.
- Jun 2017I'm presenting the paper "Deep Spatial Affordance Hierarchy: Spatial Knowledge Representation for Planning in Large-scale Environments" during the ICAPS 2017 Workshop on Planning and Robotics.Oct 2016I'm giving an invited talk on representing semantic spatial knowledge at the IROS'16 workshop Integrating Multiple Knowledge Representation and Reasoning Techniques in Robotics (MIRROR-16).Jul 2016Our paper "Autonomous Question Answering with Mobile Robots in Human-Populated Environments" was accepted to IROS 2016.
- May 2016Our paper "Robot Task Planning and Explanation in Open and Uncertain Worlds" is available online.Jul 2015Our paper "Designing Information Gathering Robots for Human-Populated Environments" was accepted to IROS 2015.
I am a research associate in Computer Science & Engineering at the University of Washington in Seattle as well as a senior researcher at the Robotics, Perception and Learning Lab at KTH Royal Institute of Technology in Stockholm, Sweden. At UW, I am working with Rajesh Rao, Dieter Fox and Maya Cakmak, and I am affiliated with the UW Reality Lab.
My goal is to build end-to-end systems that can traverse our world freely, share autonomy with human users, and self-extend through learning and natural interaction.
In Dec 2011, I received my Ph.D in robotics from KTH Royal Institute of Technology. Before joining UW in 2013, I was the Head of Research at OculusAI Technologies, a startup developing mobile applications for vision-based product recognition. Between 2006 and 2012, I was part of the EU integrated projects CoSy, CogX and DIRAC.
News
Projects
Publications
Scholar
Citations: 1900 h-index: 22 i10-index: 33
Selected Publications
- NEW! Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps
, In: AAAI Conference on Artificial Intelligence (AAAI), 2018. Acceptance rate 24.6%, oral presentation. - NEW! Learning Deep Generative Spatial Models for Mobile Robots
, In: International Conference on Intelligent Robots and Systems (IROS), 2017. - NEW! Robot Task Planning and Explanation in Open and Uncertain Worlds
, In: Artificial Intelligence, 247, 2017. - Autonomous Question Answering with Mobile Robots in Human-Populated Environments
, In: International Conference on Intelligent Robots and Systems (IROS), 2016. - Designing Information Gathering Robots for Human-Populated Environments
, In: International Conference on Intelligent Robots and Systems (IROS), 2015. - Active Visual Object Search in Unknown Environments Using Uncertain Semantics
, In: Transactions on Robotics (T-RO), 29(4), 2013. - Large-scale Semantic Mapping and Reasoning with Heterogeneous Modalities
, In: International Conference on Robotics and Automation (ICRA), 2012. Finalist for the Best Cognitive Robotics Paper award. - Semantic Mapping with Mobile Robots
, PhD thesis, KTH Royal Institute of Technology, 2011. Shortlisted for the EURON Georges Giralt Ph.D. award. - Multi-modal Semantic Place Classification
, In: International Journal of Robotics Research (IJRR), 29(2-3), 2010. - The More You Learn, the Less You Store: Memory-controlled Incremental SVM for Visual Place Recognition
, In: Image and Vision Computing (IMAVIS), 28(7), 2010.
Preprints
- NEW! LibSPN: A Library for Learning and Inference with Sum-Product Networks and TensorFlow
, In: ICML 2017 Workshop on Principled Approaches to Deep Learning, 2017.
Peer-reviewed Journal Articles
- NEW! Robot Task Planning and Explanation in Open and Uncertain Worlds
, In: Artificial Intelligence, 247, 2017. - Where Are We After Five Editions?: Robot Vision Challenge, a Competition that Evaluates Solutions for the Visual Place Classification Problem
, In: Robotics & Automation Magazine (RAM), 22(4), 2015. - Active Visual Object Search in Unknown Environments Using Uncertain Semantics
, In: Transactions on Robotics (T-RO), 29(4), 2013. - Multi-modal Semantic Place Classification
, In: International Journal of Robotics Research (IJRR), 29(2-3), 2010. - The More You Learn, the Less You Store: Memory-controlled Incremental SVM for Visual Place Recognition
, In: Image and Vision Computing (IMAVIS), 28(7), 2010. - A Realistic Benchmark for Visual Indoor Place Recognition
, In: Robotics and Autonomous Systems (RAS), 58(1), 2010. - Self-Understanding & Self-Extension: A Systems and Representational Approach
, In: Transactions on Autonomous Mental Development (TAMD), 2(4), 2010. - COLD: COsy Localization Database
, In: International Journal of Robotics Research (IJRR), 28(5), 2009.
Peer Reviewed Conference Papers
- NEW! Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps
, In: AAAI Conference on Artificial Intelligence (AAAI), 2018. Acceptance rate 24.6%, oral presentation. - NEW! Learning Deep Generative Spatial Models for Mobile Robots
, In: International Conference on Intelligent Robots and Systems (IROS), 2017. - Autonomous Question Answering with Mobile Robots in Human-Populated Environments
, In: International Conference on Intelligent Robots and Systems (IROS), 2016. - Designing Information Gathering Robots for Human-Populated Environments
, In: International Conference on Intelligent Robots and Systems (IROS), 2015. - Exploring the Potential of Information Gathering Robots
, In: International Conference on Human-Robot Interaction Extended Abstracts (HRI), 2015. - Adaptive Iterative Closest Keypoint
, In: European Conference on Mobile Robots (ECMR), 2013. - Large-scale Semantic Mapping and Reasoning with Heterogeneous Modalities
, In: International Conference on Robotics and Automation (ICRA), 2012. Finalist for the Best Cognitive Robotics Paper award. - Dora: A Robot that Plans and Acts Under Uncertainty
, In: 35th German Conference on Artificial Intelligence (KI), 2012. - Hierarchical Multi-Modal Place Categorization
, In: European Conference on Mobile Robots (ECMR), 2011. - Plan-based Object Search and Exploration Using Semantic Spatial Knowledge in the Real World
, In: European Conference on Mobile Robots (ECMR), 2011. - Exploiting Probabilistic Knowledge under Uncertain Sensing for Efficient Robot Behaviour
, In: International Joint Conference on Artificial Intelligence (IJCAI), 2011. - Functional Topological Relations for Qualitative Spatial Representation
, In: International Conference on Advanced Robotics (ICAR), 2011. - Search in the Real World: Active Visual Object Search Based on Spatial Relations
, In: International Conference on Robotics and Automation (ICRA), 2011. - Representing Spatial Knowledge in Mobile Cognitive Systems
, In: 11th International Conference on Intelligent Autonomous Systems (IAS), 2010. - A Framework for Robust Cognitive Spatial Mapping
, In: International Conference on Advanced Robotics (ICAR), 2009. - SVM-based Discriminative Accumulation Scheme for Place Recognition
, In: International Conference on Robotics and Automation (ICRA), 2008. - Towards Robust Place Recognition for Robot Localization
, In: International Conference on Robotics and Automation (ICRA), 2008. - Joint Visual Vocabulary For Animal Classification
, In: International Conference on Pattern Recognition (ICPR), 2008. - Confidence-based Cue Integration for Visual Place Recognition
, In: International Conference on Intelligent Robots and Systems (IROS), 2007. - Incremental Learning for Place Recognition in Dynamic Environments
, In: International Conference on Intelligent Robots and Systems (IROS), 2007. - SVM-based Transfer of Visual Knowledge Across Robotic Platforms
, In: International Conference on Computer Vision Systems (ICVS), 2007. - A Discriminative Approach to Robust Visual Place Recognition
, In: International Conference on Intelligent Robots and Systems (IROS), 2006.
Book Chapters
- Semantic Modelling of Space
, In: Cognitive Systems (H. Christensen, G.-J. Kruijff, J. Wyatt, eds.), Volume 8 of Cognitive Systems Monographs, Springer Berlin Heidelberg, 2010. - The Explorer System
, In: Cognitive Systems (H. Christensen, G.-J. Kruijff, J. Wyatt, eds.), Volume 8 of Cognitive Systems Monographs, Springer Berlin Heidelberg, 2010. - Multi-cue Discriminative Place Recognition
, In: Multilingual Information Access Evaluation Vol. II Multimedia Experiments (C. Peters, B. Caputo, J. Gonzalo, G. Jones, J. Kalpathy-Cramer, H. Müller, T. Tsikrika, eds.), Volume 6242 of Lecture Notes in Computer Science, Springer, 2010. - Overview of the ImageCLEF@ICPR 2010 Robot Vision Track
, In: Recognizing Patterns in Signals, Speech, Images and Videos (D. Ünay, Z. Çataltepe, S. Aksoy, eds.), Volume 6388 of Lecture Notes in Computer Science, Springer Berlin Heidelberg, 2010. - The Robot Vision Task
, In: ImageCLEF (H. Müller, P. Clough, T. Deselaers, B. Caputo, eds.), Volume 32 of The Information Retrieval Series, Springer Berlin Heidelberg, 2010. - Overview of the CLEF 2009 Robot Vision Track
, In: Multilingual Information Access Evaluation Vol. II Multimedia Experiments (C. Peters, B. Caputo, J. Gonzalo, G. Jones, J. Kalpathy-Cramer, H. Müller, T. Tsikrika, eds.), Volume 6242 of Lecture Notes in Computer Science, Springer, 2010.
Peer-reviewed Workshop Papers
- NEW! LibSPN: A Library for Learning and Inference with Sum-Product Networks and TensorFlow
, In: ICML 2017 Workshop on Principled Approaches to Deep Learning, 2017. - NEW! Deep Spatial Affordance Hierarchy: Spatial Knowledge Representation for Planning in Large-scale Environments
, In: RSS 2017 Workshop on Spatial-Semantic Representations in Robotics, 2017. - NEW! Learning Deep Generative Spatial Models for Mobile Robots
, In: RSS 2017 Workshop on Spatial-Semantic Representations in Robotics, 2017. - NEW! Learning Semantic Maps with Topological Spatial Relations Using Graph-Structured Sum-Product Networks
, In: IROS 2017 Workshop on Machine Learning Methods for High-Level Cognitive Capabilities in Robotics, 2017. - Deep Spatial Affordance Hierarchy: Spatial Knowledge Representation for Planning in Large-scale Environments
, In: ICAPS 2017 Workshop on Planning and Robotics, 2017. - Enabling Efficient Registration using Adaptive Iterative Closest Keypoint
, In: IROS 2013 Workshop on Planning, Perception and Navigation for Intelligent Vehicles, 2013. - Combining World and Interaction Models for Human-Robot Collaborations
, In: AAAI 2013 Workshop on Intelligent Robotic Systems, 2013. - Productify - Product Recognition in the Wild
, In: CVPR 2012 Vision Industry and Entrepreneur Workshop (VIEW), 2012. - Multi-modal Semantic Mapping
, In: RSS 2011 Workshop on Grounding Human-Robot Dialog for Spatial Tasks, 2011. - Object Search Guided by Semantic Spatial Knowledge
, In: RSS 2011 Workshop on Grounding Human-Robot Dialog for Spatial Tasks, 2011. - A Planning Approach to Active Visual Search in Large Environments
, In: AAAI 2011 Workshop on Automated Action Planning for Autonomous Mobile Robots (PAMR), 2011. - A Framework for Robust Cognitive Spatial Mapping
, In: 2009 Swedish Workshop on Autonomous Robots (SWAR), 2009. - The More you Learn, the Less you Store: Memory-Controlled Incremental SVM
, In: ECCV 2006 2nd International Cognitive Vision Workshop (ICVW), 2006.
Technical Reports
- Understanding the Real World: Combining Objects, Appearance, Geometry and Topology for Semantic Mapping
, Technical report TRITA-CSC-CV 2011:1 CVAP319, KTH Royal Institute of Technology, CVAP/CAS, 2011. - Representing Spatial Knowledge in Mobile Cognitive Systems
, Technical report TRITA-CSC-CV 2010:1 CVAP 316, KTH Royal Institute of Technology, CVAP/CAS, 2010. - The COLD Database
, Technical report TRITA-CSC-CV 2007:1, KTH Royal Institute of Technology, CVAP/CAS, 2007. - The More You Learn, the Less You Store: Memory-Controlled Incremental SVM
, Technical report 51, Idiap Research Institute, 2006. - The KTH-IDOL2 Database
, Technical report CVAP304, KTH Royal Institute of Technology, CVAP/CAS, 2006. - The KTH-INDECS Database
, Technical report CVAP297, KTH Royal Institute of Technology, CVAP/CAS, 2005.
Theses
- Semantic Mapping with Mobile Robots
, PhD thesis, KTH Royal Institute of Technology, 2011. Shortlisted for the EURON Georges Giralt Ph.D. award. - Indoor Place Recognition Using Support Vector Machines
, Master's thesis, KTH Royal Institute of Technology, 2005.
Advising
Co-advised Ph.D. Students
- Junha Roh, Ph.D. Student, UW, Apr 2017–Present
- Francesco Riccio, Visiting Ph.D. Student, UW, Oct 2016–Apr 2017
- Michael Jae-Yoon Chung, Ph.D. Candidate, UW, Jun 2014–Jun 2015
- Johan Ekekrantz, Ph.D. Candidate, KTH, Jan 2012–Feb 2013
- Alper Aydemir, Ph.D. 2012, KTH, Jul 2011–Nov 2012
Advised M.Sc. and B.Sc. Theses
Kaiyu Zheng, UW, B.Sc. 2017
"Learning Large-Scale Semantic Maps with Sum-Product Networks"
Shortlisted for the UW CSE Best Senior Thesis AwardKousuke Ariga, UW, B.Sc. 2017
"Semi-Supervised Spatial Knowledge Transfer with Deep Generative Models"
Shortlisted for the UW CSE Best Senior Thesis AwardBjörgvin Ólafsson, KTH, OculusAI, M.Sc. 2016
"Partially Observable Markov Decision Processes for Faster Object Recognition"Andreas Bruse, KTH, OculusAI, M.Sc. 2015
"Exploiting Cloud Resources For Semantic Scene Understanding On Mobile Robots"Nima Behzad, KTH, M.Sc. 2013
"Modeling 3D Object Context"Nikolaus Demmel, KTH, B.Sc. 2012
"Detecting Novel Concepts in Semantic Mapping with Mobile Robots"André Susano Pinto, KTH, M.Sc. 2011
"Novelty Detection for Semantic Place Categorization"Li Xing, KTH, M.Sc. 2010
"Multi-cue Visual Place Classification for Mobile Robotic Systems"Muhammad Muneeb Ullah, KTH, M.Sc. 2007
"Vision-based Indoor Place Recognition using Local Features"Jie Luo, KTH, M.Sc. 2007
"Incremental Learning for Adaptive Visual Place Recognition in Dynamic Environments"
Best Master’s thesis award of the Swedish AI Society
Teaching
Currently, I'm teaching the following courses at the University of Washington:
I taught the following courses at KTH Royal Institute of Technology:
- EL2310 Scientific Programming (2012, Instructor/Lecturer)
- DD2463 Advanced Individual Course in Computer Science (Project Instructor)
- DD2425 Robotics and Autonomous Systems (2010, 2011, Teaching Assistant)
Other short courses I prepared:
- Modeling Multi-Agent Robot Systems Using Control and Systems Theory
- Dimensionality Reduction
- PhD Course on Probabilistic Graphical Models
Activities & Recognition
Grants & Fellowships
- 2013 - Swedish Research Council Project Grant for Junior Researchers (PI) Prestigious, nation-wide grant in natural and engineering sciences of 3,500,000 SEK (400,000 USD) and acceptance rate of 2.5%.
- 2013 - Toyota Research Grant (Co-PI) Grant for postdoctoral research at the University of Washington (85,000 USD).
- 2017 - Google Research Sponsorship Sponsorship for organizing the ICML Workshop on Principled Approaches to Deep Learning.
- 2008 - DIRAC (EU FP6 Integrated Project) Training Programme Fellowship for research at the Idiap Research Institute.
Awards
- 2017 - Finalist for the Google Best Paper Award at the ICML Workshop on Principled Approaches to Deep Learning Paper: "Library for Learning and Inference with Sum-Product Networks and TensorFlow"
- 2014 - Postdoc Research AwardUniversity of Washington, Computer Science & Engineering
- 2012 - Shortlisted for the 2012 EURON Georges Giralt PhD Award Ph.D. thesis in the top nine Ph.D. theses in robotics in Europe.
- 2012 - Finalist for the ICRA Best Cognitive Robotics Paper Award Paper: "Large-scale Semantic Mapping and Reasoning with Heterogeneous Modalities"
- 2009 - Winning robot in the CogX Cognitive Robot Programming Competition
- 2010 - Winning project in the Excitera 2010 Innovation Challenge
Editorial Appointments
- 2018 - Associate Editor 2019 International Conference on Robotics and Automation (ICRA)
- 2017 - Associate Editor 2018 International Conference on Robotics and Automation (ICRA)
- 2014 - Associate Editor 2015 International Conference on Robotics and Automation (ICRA)
- 2009–2010 - Associate Editor Paladyn, Journal of Behavioral Robotics
Program Committees
- Sep 2014 - ECCV 2014 Workshop on Consumer Depth Cameras for Computer Vision
- Jul 2014 - AAAI 2014 Workshop on Artificial Intelligence and Robotics
- Mar 2014 - AAAI Spring Symposium 2014: Qualitative Representations for Robots
Reviewing
- Conferences: Science and Systems (RSS), International Conference on Intelligent Robots and Systems (IROS), International Conference on Robotics and Automation (ICRA), European Conference on Mobile Robotics (ECMR)
- Journals: International Journal of Robotics Research (IJRR), Transactions on Robotics (T-RO), Robotics and Autonomous Systems (RAS), Autonomous Robots (AURO)
- Grants: University of Washington Royalty Research Fund
Chairing
- 2017 - Chair, IROS session "RGBD Perception"
- 2017 - Co-Chair, IROS session "Deep Learning in Robotics and Automation"
Climbing
Contact
How to Reach Me?
Email :
ude.notgnihsaw.sc@sibonorp
Skype : andrzej_pronobis
Fax : +1 (206) 543-2969
Address
University of Washington
Paul G. Allen School of Computer Science & Engineering
185 E Stevens Way NE
Box 352350
Seattle, WA 98195, USA
Office: 336