Stereo Vision System

Implementation of a stereo vision system using two calibrated cameras to predict the depth of objects in different scenes

This project explores the problem of collaborative perception where multiple robots co-exist in an environment and one of the robot’s RGB camera undergoes malfunction. As such, that robot may not be able to effectively carry out perception tasks such as navigating in the environment. We consider two of such tasks — segmentation. We are motivated by scenarios where other robots in the environment may be able to assist the malfunctioning robot. We propose to solve this problem using the powerful vision transformer auto encoders. We present TACO, which reconstructs the view for the second robot using only the RGB input from camera 1 and depth input from camera 2. We further make the problem complex by assuming that there is no stereo camera present. Vision transformer, particularly masked autoencoders are comparatively less explored in the context of robotics problem and cannot be directly applied due to their random priors. We evaluate our framework for the downstream task of segmentation in synthetically produced real world dataset. Our results show the potential of computer vision frameworks in real world robotics problems. We extensively evaluate TACO for segmentation in synthetically produced real world dataset for four different environments, our framework leads to a 2.9X improvement compared to without using TACO.

Caption photos easily. On the left, a road goes through a tunnel. Middle, leaves artistically fall in a hipster photoshoot. Right, in another hipster photoshoot, a lumberjack grasps a handful of pine needles.
This image can also have a caption. It's like magic.

You can also put regular text between your rows of images. Say you wanted to write a little bit about your project before you posted the rest of the images. You describe how you toiled, sweated, bled for your project, and then… you reveal its glory in the next row of images.

You can also have artistically styled 2/3 + 1/3 images, like these.

The code is simple. Just wrap your images with <div class="col-sm"> and place them inside <div class="row"> (read more about the Bootstrap Grid system). To make images responsive, add img-fluid class to each; for rounded corners and shadows use rounded and z-depth-1 classes. Here’s the code for the last row of images above:

<div class="row justify-content-sm-center">
    <div class="col-sm-8 mt-3 mt-md-0">
        {% include figure.html path="assets/img/6.jpg" title="example image" class="img-fluid rounded z-depth-1" %}
    </div>
    <div class="col-sm-4 mt-3 mt-md-0">
        {% include figure.html path="assets/img/11.jpg" title="example image" class="img-fluid rounded z-depth-1" %}
    </div>
</div>