Brianna is the daughter of Claire a time traveler and can travel. Roger is the descendant of Geillis Duncan who is a time traveler and can travel.
Jemmy is able to hear the gem stones. Gem stones which are typically warm to travelers appear hot to him suggesting is ability is stronger. This makes it likely he is the product of two travelers: Brianna and Roger.
One of Claire’s parents had to be a traveler.
One of Roger’s parents had to be a traveler.
