Man accused of repeatedly slapping 'fussy' baby