The world is designed to encourage the comfort of majority mainstream neurology. Our social rules are based on the needs of the majority; our priorities are based on the comfort of the majority; our understanding of others is based on our expectations of the majority.
A member of the majority can move through the world fairly comfortably confident that they are mostly getting it "right". They can pretty accurately guess at the thought processes and emotional experience of most people, because "most people" are similar to them. They can pretty safely assume that other members of the majority are having a similar experience to their own. It's easy to empathise with their own group.
That said, it's also a pretty safe bet that any group with a common neurology is going to find it easier to relate to members of their own group, to share a common experience, and to more naturally empathise with one another.
If I've learned anything over the past 16 years of parenting an autistic person and mixing with all sorts of people in all sorts of settings, it's that if we look through the right lens, autistic people are not as socially disabled as we have been led to believe. It's really just that the majority neurology dominates the narrative, and all social expectations and rules are based on that reality.
The world is not set up for the comfort or success of autistic people, and the majority group has shown itself to be pretty weak at empathising with the experience of autistic people. The majority group likes to think that autistic people are poor at empathising with majority members, but the reality is that every group tends to be pretty poor at empathising with other groups. So it seems to me, stating that poor empathy skills are exclusive to autism is misleading at best.
In my opinion, all people are socially awkward and clumsy when placed in the wrong context, especially towards people unfamiliar to themselves. It's just that the majority group is lucky enough to be born into a world where the context and rules are designed specifically for them.