Jose Ortiz, 62, who was left to babysit two young girls, is accused of harming them instead.
Ortiz was arrested Tuesday on two counts of aggravated sexual assault of a child.
An arrest warrant affidavit stated that the father of a 6-year-old girl contacted San Antonio police on March 8 after his daughter told him that Ortiz had kissed and fondled her.
The affidavit stated that the incident allegedly happened a week prior to that, when Ortiz was babysitting both that child and a 4-year-old girl.
Detectives later questioned the girls and Ortiz, and obtained enough information to make an arrest.
The affidavit stated that the 4-year-old girl said Ortiz also kissed and fondled her.