Ventricular-arterial coupling plays a vital role in the physiologic function of the cardiopulmonary circulation and progression of cardiovascular diseases. In pulmonary arterial hypertension (PAH), malfunction of one compartment (e.g., microcirculation) of the cardiopulmonary circulation may affect other compartments (e.g., right ventricle) through a positive feedback loop, which in turn, may ultimately result in end-stage heart failure. To investigate this coupling arising from ventricular and vascular remodeling in pediatric PAH, we developed patient-specific image-based computational modeling frameworks based on clinical measurements of ventricular and vascular hemodynamics and geometries. First, we developed a computational framework (based on an efficient adjoint-based optimization scheme) to estimate the microstructural properties of the large pulmonary arteries (PAs) using a constitutive model informed by the pressure and lumen diameter waveforms of the vessels. We show that parameters associated with the PA material stiffness are higher in PAH patients than in a control subject. PA wall stresses are also significantly higher in the PAH patients, suggesting a higher loading and pathological stiffening. We also found that stress and stiffness resultants associated with all constituents are higher in the PAH patients. The effects of these PA microstructural changes on heart function are then investigated in a novel multi-scale modeling framework that bidirectionally couple image-based computational (finite element) models of the large PAs, aorta, and heart (including both ventricles). With this combination of frameworks, we show that hemodynamics of the pulmonary vasculature and RV wall stress are sensitive to both changes in the PA’s microconstituent and vasoconstriction of the distal vessels, albeit more sensitive to the latter.