This paper presents a new housing affordability model, the Housing Affordability Matrix, evaluating the three main components affecting affordability - household incomes, house prices, and mortgage borrowing costs. The model aligns household needs to dwelling size while considering borrowing constraints and restrictions applicable to various households' age and income groups residing in different locations. Using Israeli data, we found that in 2017, 70% of households couldn't afford to purchase a dwelling; among young households, this rate was even higher, at 80%. Over a decade, from 2007 to 2017, housing affordability dropped by 34%, particularly in high-demand Tel Aviv and Jerusalem districts. The suggested approach narrows the gap in existing research, offering a versatile measure applicable across housing and mortgage markets to analyze regional and demographic differences in affordability of housing. Policy implications may include a deeper understanding of affordability factors, which could lead to more efficient housing affordability initiatives.