In this paper we describe some of the challenges that solar physicists face in the application of polarized radiative transfer to the modeling of the emergent radiation from the outer layers of the solar atmosphere, where the plane-parallel approximation breaks down, and 3D atmospheric modeling becomes essential. We review the various plasma conditions occurring in the photosphere, chromosphere, and corona, which determine the different regimes of atomic excitation of these regions. Depending on the relative importance of anisotropic irradiation of the gas over collisional thermalization of the atomic populations, the description of the atomic excitation states may necessitate a full quantum-statistical treatment, which exacerbates the numerical complexity of an already computationally intensive problem. Special emphasis is placed on forward modeling and inversion techniques that mitigate this difficulty, making feasible the interpretation of polarization signals in terms of the magnetic field and its connectivity throughout the solar atmosphere.
