Colorectal cancer (CRC) is one of the most common and deadly types of cancer worldwide. During standard colonoscopy procedures to detect polyps, which are early-stage precancerous lesions critical for disease prevention, challenges exist, such as overlooking polyps and the inability to accurately segment polyps with weak borders that are integrated with surrounding tissue using current computer-aided methods. This study proposes a new deep learning architecture, called TriaNet (Tri-Fusion Attention Network), to enhance the segmentation accuracy of polyps with weak borders. The fundamental innovation of TriaNet is its unique “triple-fusion” attention mechanism, which combines three complementary information streams. The proposed method dynamically fuses edge feature information obtained from a hybrid block containing Scharr, DoG, and Gabor filters, the semantic feature map from the decoder structure, and an instantaneous boundary map derived from a Scharr operator applied to an upper layer prediction. Furthermore, Deformable Alignment layers are utilized in skip connections to enhance the model's ability to adapt to variable polyp morphologies. The TriaNET architecture was tested on four different benchmark datasets, including Kvasir-SEG, CVC-ColonDB, ETIS-LaribPolypDB, and CVC-300, which demonstrated superior performance compared to state-of-the-art methods.