Accurate classification of breast cancer histopathological images is essential for early diagnosis and effective treatment planning. This study presents a custom-designed Convolutional Neural Network (CNN) model developed to classify breast cancer histopathological images with enhanced accuracy and reliability. The research began by evaluating the performance of eleven pre-trained transfer learning models, including Xception, InceptionV3, MobileNetV2, and EfficientNetV2B1, using a large histopathological dataset. Hyperparameters such as learning rates, loss functions, optimization algorithms, and data augmentation strategies were meticulously optimized during this process. Among the models, Xception and InceptionV3 exhibited the best performance, achieving accuracy rates of 89.89% and 92.17%, respectively, while MobileNetV2 and EfficientNetV2B1 showed significantly lower results. To address the limitations of transfer learning models and further enhance classification performance, a custom CNN model was developed. The proposed model incorporated advanced architectural features, including squeeze-and-excite mechanisms and group normalization, to improve feature extraction and model stability. This custom CNN achieved superior results, with an accuracy of 93.93%, precision of 94.15%, recall of 93.93%, and an F1-score of 93.98%. The findings emphasize the potential of custom deep learning models in advancing breast cancer diagnostics by providing higher accuracy and generalizability compared to traditional transfer learning approaches. The clinical application of the proposed model could significantly improve early detection and treatment planning by offering healthcare professionals a reliable and efficient diagnostic tool, ultimately contributing to better patient outcomes.