Joe Giudice said he and his ex-wife's family have known each other for decades. It might explain Joe Gorga's hard feelings.