I came to gardening in what I should probably just go ahead and refer to as midlife, after we bought our first house and moved in in 2010, when I had a two year old and was pregnant with #2. As a child, we didn't grow anything much (although I do have vivid memories of making mud pies in bare dirt, so that's something!). As an adult, I was living in apartments and working fulltime. My husband meanwhile had spent 10 years jazzing up the bare dirt of his rented house, and one of his first romantic gestures was dropping off a bag of his homegrown tomatoes for me. Yet even after I moved in, I was too busy and uninterested.
That changed when we moved in our "own" home and I had a child to teach. I wanted him to understand where food came from, to eat vegetables, to be outside, to learn about science and math from nature as well as from books. My husband was completely occupied with landscaping the front of the house (another bare dirt situation) so the backyard became the fiefdom of me and the kids. I stopped paid work when my younger child was born, so even if I didn't have "free time", I was home more and could step outside without it being a big production.
The backyard was rototilled when we moved in (2010) and basic fill dirt added. Since then, I haven't added any significant new dirt to my 10x20' (approx.) patch. My summer priority is tomatoes. My year-round crops (okay, some months this is still aspirational) are lettuce and cilantro because I am a major cilantrohead. Everything else is added by whim, by accident, or by best-laid plan, and we know what happens to those. I do my best to grow without pesticides; I try to use organic seeds and starts but not exclusively. I care a lot about my plant-babies but some laziness creeps in; as my husband says with a shrug, "It's only plants. You can plant them again!" But you can't grow kids again, so my focus is more on the process than the product.
My older child's excitement and younger child's enthusiasm for gardening are awesome. Some days of course are better than others - this is true of any parenting effort! They usually enjoy being outside with me in the garden, and that's what's most important.