Most things before Stellae is things Jester either doesnt remember or doesnt care to remember. Having been taken in by Rosemary after she took up position in Stellae, is the start of his life as far as hes concerned. He grew up surrounded by all sorts of individuals who taught him the things he knows now. Basically a "takes a village" chiild.
I grew up here, I didn't need anythign else, and plus I like to think I turned out pretty well.