README.md 3.53 KB
Newer Older
Axel Wickman's avatar
Axel Wickman committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# LIU Home Wreckers - @home
This is a project at Linköping University's [FIA robotics](https://fiarobotics.se/) association. Our team competes in the [RoboCup](https://www.robocup.org/) competition every year with our Pepper robot (see below) in service robotics. The project has gone on since autumn 2017, and we have competed in many parts of the world. Come by the robotics lab if this sounds like something you would like to parcitipate in.

<img src="https://upload.wikimedia.org/wikipedia/commons/thumb/a/a1/SoftBank_pepper.JPG/800px-SoftBank_pepper.JPG" width="200" />


The project is written in the [Robot Operating System 2-framework](https://index.ros.org/doc/ros2/) using mainly Python3 and C++. This can be hard to get started with, but luckily for you we have already made a Dockerfile to install everything in.
Checkout [INSTALL.md](./INSTALL.md) easily get started.\
More documentation can also be found here:
- General info: https://index.ros.org/doc/ros2/
- ROS Python API: http://docs.ros2.org/latest/api/rclpy/index.html
- ROS C++ API: http://docs.ros2.org/latest/api/rclcpp/index.html
- LibQI: https://github.com/aldebaran/libqi
- Google
- In the heads of @axewi604 and @frelo223


### Important when contributing: 
- Try to follow the ROS Style guide and be consisent with the rest of the project. 
- Name new branches after the features you are implementing. Delete the brances afterwards.
Axel Wickman's avatar
Axel Wickman committed
21
- Create and solve issues at the GitLab page (left menu).
Axel Wickman's avatar
Axel Wickman committed
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
- Test early, test often, and test preferably before pushing to master.
- Don't get stuck researching. It is most often better to rewrite code later.

## 

## Intelligence
Used to make the decisions of what to do next in order to solve the task.
* State machines - gets changes from the core, and supplies back goals.
* AI core - gets a goal and tries to furfill it. Has a lot of data and actions at its' disposal.
* Response discriminator - chooses the best response from the NLP areas.
* Knowledge-base - ontological reasoning about the world. 


## Natural Language Processing (NLP)
To be able to hear, understand, and eventually respond to speech. To generate speech from text, we use the QI-driver.
* Speech to text - taking raw audio and converting it to text.
* DialogFlow - using Google's service for understanding intention.
* GPT2/GPT3 - general language model for talking about anything.
* Wolfram Mathematica - to answer data about the world.

## Vision
This part figures out what objects and people (entities) exists in front of the robot, where they are spatially, and what is what over time.
* DenseSense - for seeing people, what they are wearing and what they are doing.
* YOLO - for seeing all kinds of objects.
* Tracker - takes detections and gives the same objects the same id:s over time.
* 3D estimator - tries to guess a 3D position of the entities.

## Motion
This concerns navigation and body motion. We have to be able to move the robot to an absolute or relative position, and not collide with anything on the way there.
* Local navigation - move small distances without colliding with things.
* Global navigation - use a predrawn map, figure out where the robot is, pathfind there.
* Motion behaviours - can be to scan with the head, to point at something, or to do a dance!

## QI
These are the drivers our code uses to actually interact with the robot. It needs to be fast and is therefore written in C++. It can be to turn the wheels, flash the LED:s, or speak.
* ALVideoDevice - cameras
* ALTextToSpeech - talking.
* ALMotion - pose and navigation.
* ALMemory - sonars, lasers, IR, IMU.
* Many other modules...
62
63