When you're a ruler, all of the resources, assets and structures on the lands you own are your attributes. Cities take time to grow and take time to be destroyed, just like your Strength or Wisdom does. Castles endure, much like your Constitution does. If you damage the Castle, it's only a little while until you fix it, after all.
But your subjects are a currency, just like your gold is. When you spend your people unwisely, you will be as poor a ruler as one whose treasuries are empty. Dead men, after all, cannot be taxed, and they do not till fields. They do not fight in armies. They do not produce more children. They do not invent. In a sense, your people are your most valuable resource- and by treating them as a currency instead of an attribute, you get neat scenarios like Middle Ages Muscovites, where their neighbors are having social changes and inventing and building cathedrals and all they get to do is fight for a couple hundred years with the Tatar hordes that are on every damned side.
As an added bonus, having population as a currency means that you can have both gold income and population income tracked for each holding and county that you control: A province might provide 1 gold (measured in talents, probably) and 5 manpower per "turn," which means that you can use those five people in tasks that aren't directly related to whatever it is that the province does, like forming an army or building fortifications, or what-have-you. Everybody that's not available
Since we're handwaving away all of the necessary guys in a province, that simultaneously lets us add a layer of abstraction that's also directly calculable by remembering that in the Middle Ages, it took seven people to provide enough food for an 8th*; this means that a 5 manpower (MP here on out) province actually has eight times as many people as that residing in it; our 5 MP province contains a hamlet of some sort. In time, with enough sustained growth, a bit of luck, and maybe a bit of immigration, our hamlet might grow into a useful town.
There's still a whole lot of writing, thinking, and concept work to be done, though, so this is all subject to change. But it's how I've got everything written out for now, and I'm hoping that this will all come together well soon. If anything, it seems a bit overly complex- so it'll be a good bit of work for me to be able to present this in a way that's not absurdly difficult to deal with without a lot of electronic aides. Wish me luck!