Colon capsule endoscopy (CCE) as a novel 2D biomedical image modality based on visible light provides a higher perspective of the potential gastrointestinal lesions like polyps within the small and large intestines than the conventional colonoscopy. As the quality of images acquired via CCE imagery is low, so the artificial intelligence methods are proposed to help detect and localize polyps within an acceptable level of efficiency and performance. In this paper, a new deep neural network architecture known as AID-U-Net is proposed. AID-U-Net consists of two distinct types of paths: a) Two main contracting/expansive paths, and b) Two sub-contracting/expansive paths. The playing role of the main paths is to localize polyps as the target objectives in high resolution and multi-scale manner, while the two sub paths are responsible for preserving and conveying the information of low resolution and low-scale target objects. Furthermore, the proposed network architecture provides simplicity so that the model can be deployed for real time processing. AID-U-Net with an implementation of a VGG19 backbone shows better performance to detect polyps in CCE images in comparison with the other state-of-the-art U-Net models like conventional U-Net, U-Net++, and U-Net3+ with different pre-trained backbones like ImageNet, VGG19, ResNeXt50, Resnet50, InceptionV3 and InceptionResNetV2.