|
|
|
# Project: An advanced office assistant
|
|
|
|
|
|
|
|
The goal of the project is to work in a group of four students to develop an advanced office assistant capable of the following functionnalities:
|
|
|
|
|
|
|
|
* *Exploration:* explore its environment to provide a map of interest points, such as location of offices, vending machines and humans. Some of the interest points move over time, and the robot should be able to handle those aspects.
|
|
|
|
* *Human-aware navigation:* navigate around humans in a confortable way
|
|
|
|
* *Natural text commands:* interpret natural text sentences to define goals for the robot
|
|
|
|
* *Decision:* take decision based on a set of goals on what the robot should do
|
|
|
|
|
|
|
|
One and only one student in the group should look at one functionallity, so that the full set of functionnalities is covered. Exceptions might be granted if the project groups is not of size four, which might happen since it is unlikely the number of students is a multiple of four.
|
|
|
|
|
|
|
|
## Expectations
|
|
|
|
|
|
|
|
It is not expected that you deliver a fully functionnal product, but rather a proof of concepts. The development of such a robot would likely take much more effort than what can be covered by the time allocated in this course. Also, the examination is *individual*, you will not be penalized if the other students in your group fail to deliver on their part. However, it is expected that you integrate with the students who are completing the course, and the other part can be handled by the labs:
|
|
|
|
|
|
|
|
* *Exploration:* the exploration of [Lab3](Lab3) and [Lab4](Lab4) should be enough, even though not optimal
|
|
|
|
* *Human-aware navigation:* if the student working on that topic fails to deliver, it is acceptable that the robot collide with humans
|
|
|
|
* *Natural text commands:* the node from [Lab5](Lab5) provides a basic version and can be easilly extended with more simple commands to cover for a missing NLP component
|
|
|
|
* *Decision:* the simple goal to tst node from [Lab5](Lab5) provides a basic version and can be easilly extended to support more goals if needed
|
|
|
|
|
|
|
|
## Deliverables
|
|
|
|
|
|
|
|
* *Individual planning report*: at the end of March, each student should submit an individual planning report. It is a one page document which introduce the topic, list of possible approaches, and indicate which approach the student intend to investigate further. It should include a few relevant bibliographic reference. The main goal of the document is to evaluate the feasability of the project.
|
|
|
|
* *Presentation of the project:* an oral presentation of your work in a seminar during May. The presentation should be hybrid and include a sale pitch of your robot as well as scientific presnetation of the different part ([detailed instructions](group_presentation)).
|
|
|
|
* *Project leaflets:* an A3 document advertising your robot. You should highlight the strong point of your robot to a potential customers (detailed instructions coming soon)
|
|
|
|
* *Individual report:* the individual report should present your individual work as a small scientific paper ([detailed instructions](individual_report)).
|
|
|
|
|
|
|
|
## Information regarding the different sub-parts
|
|
|
|
|
|
|
|
The techniques linked in this section are only suggestions and starting point
|
|
|
|
|
|
|
|
### Exploration
|
|
|
|
|
|
|
|
When the robot is turned on the first time, it knows nothing about the office environment. Your robot should be able to gather information about the environment by driving around and capturing information with its sensors.
|
|
|
|
|
|
|
|
The two techniques implemented in [Lab2](Lab2) and [Lab3](Lab3) works but are inefficient. For the project, you should look at more efficient techniques such as:
|
|
|
|
|
|
|
|
* frontier cell exploration, as presented in the lectures or in _A frontier-based approach for autonomous exploration, B. Yamauchi, Computational Intelligence in Robotics and Automation, 1997_.
|
|
|
|
* _Mobile robots exploration through cnn-based reinforcement learning, L. Tai and M. Liu, Robotics and Biomimetics. 2016_.
|
|
|
|
* Or any other approaches found in the litterature.
|
|
|
|
|
|
|
|
ROS has many modules for exploration that can be used. However, if you choose to use ready made module, you should develop some extra functionnalities on top, such as handling dynamic knowledge. You can also use pre-existing ROS nodes to compare against your own implementation.
|
|
|
|
|
|
|
|
## Human-aware navigation:
|
|
|
|
|
|
|
|
It is not good enough to avoid obstacles, when it comes to humans, it should be done in such a way that the humans feel confortable.
|
|
|
|
|
|
|
|
* Look at Lecture 11 of TDDE05
|
|
|
|
* A good starting point is _Human-aware robot navigation: A survey, T. Kruse, A. Kumar Pandey, R. Alami and A. Kirsch, Robotics and Autonomous Systems, 2013_
|
|
|
|
|
|
|
|
## Natural text commands:
|
|
|
|
|
|
|
|
A natural interface between an human and a robot is for the human to give commands to the robot using natural sentences, such as, "I need to go to George's office and bring us two coffee.". The goal of this part is to decompose this sentence in three goals: _goto George office_ _bring coffee George_ _bring coffee user_.
|
|
|
|
|
|
|
|
You should consider the following approaches:
|
|
|
|
|
|
|
|
* Intent recognition: _Practical guidelines for intent recognition: Bert with minimal training data evaluated in real-world, M. Huggins, S. Alghowinem, S. Jeong, P. Colon-Hernandez, C. Breazeal, and H.W Park, ACM/IEEE International Conference on Human-Robot Interaction. 2021_
|
|
|
|
|
|
|
|
It is not expected that the student investigate speech-to-text, in fact, it is strongly discouraged to look at those expect, you should consider that you receive a sentence as a string. Speech-to-text is a *very* hard problem.
|
|
|
|
|
|
|
|
## Decision
|
|
|
|
|
|
|
|
The robot will receive a set of goals, and should decide on which goals to accomplish and how.
|
|
|
|
|
|
|
|
* This can be done using automated planning, refer to Lecture 07
|
|
|
|
* Using decision theory approach, refer to lecture 08
|
|
|
|
|
|
|
|
## FAQ
|
|
|
|
|
|
|
|
* *Can I use existing software?* Yes, absolutely. It is not expected that your write everything from scratch. You can and should use existing relevant libraries, framework and ROS node and integrate them in your system. However, it is not sufficient to install a ROS node, configure the topics and run that. In case of doubt, ask the assistant if what you intend to do is acceptable. |