Done having kids???

So....I've been struggling with a very hard question for a while now and I need some help..... I wanna hear your stories and opinions.  So, Seasoned Mothers:

How did you know when you were done having children?  How did you make the decision?  What helped you be happy with that decision?

So, as most of you know I have a 4 year old daughter and an 18 month old son.  As many would say, I have the "perfect pair".  Even though there never will be a perfect pair! lol  But society says that One girl and One boy is what everyone strives for.  Well, How do I know if I am done or not? 

This fall our family and my life as a stay at home mom is transitioning and taking a pretty big step!  Cady is heading to All Day Kindergarten and my sister is going to be staying home so I will no longer be babysitting Miss Emma Paige.  So, it will be just me and Nate all day while Cady is at school. 

So, my struggle is do I want to expand our family again and add another member or do I want to enjoy my freedom with only one child all day......  Part of me say yes, the freedom is going to be wonderful!  I can come and go so much easier since I will only have one child to bring with me.  I can get involved in Cady's school because it will be easier to find a sitter for one child.  I can get Nate potty trained and focus solely on him for awhile, and since I am going to be an Auntie THREE more times this year I will have more time to spoil my new nieces or nephews! 

BUT on the other hand, I start to get said that my baby boy is no longer a baby anymore!  Yes he is a momma's boy and will always need me, he loves to cuddle and he can walk by himself so I don't have to carry him much anymore.....But I miss the baby stage....I miss the little baby milestones and tiny little outfits....and I honestly kind of miss being pregnant....But Hubby says he is perfectly happy with 2.

So, tell me, how did you decide that you were done have children?  What helped you to be at peace with your decision?