Travelling with kids, especially a little one is very different from traveling alone as a backpacker. There are so many considerations to take into account. Accommodation comfort, suitable food availability and ease of obtaining it, hassle-less transport, keeping the little one interested and most important of all, keeping an eye on the little one.
Our son has just turned 5. In some ways it is easier to travel with him now- food is easier (though he has never been a problem), he can walk on his own most of the time, no need to rush back for afternoon naps and so on. However it has other challenges compared to when younger- he is more easily bored, he has his own mind and argues a lot more, if he is tired and does not want to walk, he is a lot more difficult to carry.
One can probably forget about lingering around to try to take quality photos at the right moments. There is no chance to do so and one can definitely forget about blue hours- it is dinner time!
That said, it is different fun traveling as a family with my beloved wife and my naughty son. There are always fun and memorable moments which we will remember but we doubt he will, at this age. An example was when he gamely and bravely walk across a hanging bridge on his own over a gorge at Taroko Gorge in Taiwan (he was 3+ then). It was so funny to watch him trundle, hopping across on his own without any fear of the height and surroundings. These are moments to cherish.