Introduction to Carla

CARLA has been developed from the ground up to support development, training, and validation of autonomous driving systems. In addition to open-source code and protocols, CARLA provides open digital assets (urban layouts, buildings, vehicles) that were created for this purpose and can be used freely. The simulation platform supports flexible specification of sensor suites, environmental conditions, full control of all static and dynamic actors, map generation and much more.

• Install CARLA- Carla used in this Project - Drive Link To download Carla 
• Start using CARLA- Official Documentation
• Check the API- There is a handy Python API reference to look up the classes and methods available. Python API

Road-Detection

• Semantic Segmentation is a computer vision task in which we label specific regions of an image according to what's being shown. More specifically, the goal of semantic segmentation is to label each pixel of an image with a corresponding class of what is being represented. Because we're predicting for every pixel in the image, this task is commonly referred to as dense prediction.

• Carla inbuilt Semantic segmentation (here)
• We also trained our own semantic-segmentation model.

• Separating out the road-mask from the output of semantic segmentation Output.

• Random sample consensus (RANSAC) algorithm used to filter and predict a more accurate plane of road. (Video)

Lane-Detection

To implement Lane-Detection we used opencv-canny edge detection over the result of our Road-detection part. Then to draw lanes from image to our real-world we use the feed of depth-camera mounted at the same location of our rgb-camera. 

# Compute x and y coordinates

     for i in range(H):

        for j in range(W):

           X[i, j] = ((j+1 - self.c_u)*depth[i, j]) / self.f

Demo Video (https://www.youtube.com/watch?v=4d8uz27Agvs)

Object-Detection

Model used : yolov4

Full Report : (https://roboticsclub.mnnit.ac.in/projects/detail/8/)

Path-Planning (DWA)

Dynamic Window Approach(DWA) is a local path-planning algorithm. It does not know about the global path. We used to give the positions of the goal, objects and lines. Then it computes the most favoured path.

STEPS:

• Compute Dynamic Window in which our vehicle can move smoothly. 

# Dynamic window from robot specification

Vs = [self._config.min_speed, self._config.max_speed, -self._config.max_yaw_rate, self._config.max_yaw_rate]

# Dynamic window from motion model

Vd = [state[3] - self._config.max_accel * self._config.dt, state[3] + self._config.max_accel * self._config.dt, state[4] - self._config.max_delta_yaw_rate * self._config.dt, state[4] + self._config.max_delta_yaw_rate * self._config.dt]

dw = [max(Vs[0], Vd[0]), min(Vs[1], Vd[1]), max(Vs[2], Vd[2]), min(Vs[3], Vd[3])]

• Fetches all the possible trajectories of the vehicle with varying linear and angular velocity.

def _predict_trajectory(self, x_init, v, y):

        x = np.array(x_init)

        trajectory = np.array(x)

        time = 0

        while time <= self._config.predict_time:

            x = self.motion(x, [v, y], self._config.dt)

            trajectory = np.vstack((trajectory, x))

            time += self._config.dt

        return trajectory

for v in np.arange(dw[0], dw[1], self._config.v_resolution):

     for y in np.arange(dw[2], dw[3], self._config.yaw_rate_resolution):

           trajectory = self._predict_trajectory(x_init, v, y)

• Calculate the cost of all possible trajectories computed in the 2nd step.

There are 4 types of cost.

• Goal Cost  = goal_cost_gain / distance of vehicle to goal
• Velocity Cost = velocity_cost_gain * velocity of vehicle (Smoothness of trajectory)
• Obstacle Cost = objetct_cost_gain / distance from the vehicle  (avoid other vehicles)
• Lane Cost (complex implementation)

      to_goal_cost = goal_cost_gain * calc_to_goal_cost(trajectory)

speed_cost = speed_cost_gain * (self._config.max_speed -  

trajectory[-1, 3])

ob_cost = obstacle_cost_gain * calc_obstacle_cost(trajectory, 

x_init)

lane_cost = lane_cost_gain * calc_lane_cost(trajectory, x_init)

final_cost = to_goal_cost + speed_cost + ob_cost + lane_cost

• Choose the trajectory with the minimum cost.

if min_cost > final_cost :

    min_cost = final_cost

    best_u = [v, y]

    best_trajectory = trajectory

Future Work

• Improvement in controller of Carla
• Implement Different Path-planning algorithms and compare their efficiency.
• Implement the whole algo using ROS. (ROS-bridge for carla)
• Predicting the nature of moving vehicles.
• Avoiding Dynamic vehicles.

Conclusion

• Now the current version of code is able to completely detect the road, lane and obstacle.
• The Avoidance of Static-Vehicle Objects is  done perfectly. (Video).

YouTube (Playlist):

https://youtube.com/playlist?list=PLvF-o_Kkrx182f--pGoFnpKpGXctXnowy