Over the last decades, computational scientists have been building high performance scientific codes to make the most of large scale computing resources. These codes form today a rich computing stack exploited to simulate and analyze phenomena in a wide variety of scientific domains. But while research communities were putting a tremendous amount of effort to add numerical methods and solvers on one side, and parallel strategies to exploit heterogeneous architectures on the other side, codes were slowly becoming more complex. In this talk, I will argue that in a growing number of computational sciences, software complexity has become a far more limiting factor than pure computing power. I will discuss the underlying mechanisms that are leading to a wall of complexity that will put heavy constraints on scientific progress. I will review the related challenges from both the computational side and the computer science side, and suggest some interdisciplinary approaches to overcome this upcoming wall. I will discuss in particular how work on software architecture and programming languages can be key to solve this problem. Throughout this talk, numerical cosmology will be taken as a canonical example since the field is facing complexity challenges that no supercomputer can solve. Before concluding on how computer scientists and researchers in application domains can to collaborate to get around the wall of complexity, I will present a few lessons that have been learned over the years in the design of Embedded Domain Specific Languages to try to reduce software complexity in astrophysics codes.