It comes from the purpose of words. A mathematic description of words would include the idea that words describe an area in conceptspace. Often, they describe an area in thingspace. What this means is that words are intended to arouse in the hearer a particular, and often specific, idea.
The idea of violence is particular. If I hear that two people outside are engaging in violence, I hope that I react in some particular ways -- calling the police, intervening, barricading the doors, etc. Different reactions may be called for at different levels of violence. There's a weakening of this because there is such a thing as play violence, which does not call for a reaction -- but it strongly resembles real violence, and thus may force a reaction before it is recognized as not actually violence.
But it is the rare occasion where an act of violence in my surroundings does not demand of me a reaction.
I do not believe the same is true of interpersonal power dynamics, so-called violent communication. If two people are standing on the sidewalk arguing about something, and one of them is winning, I do not believe that it is likely appropriate for me to intervene, call the cops, or barricade my doors.
The reason this bothers me is simple: it seems manipulative. It feels like the people who named it violent communication recognize that I do not believe that it demands an equal reaction, yet they want to convince me that it should. But instead of convincing me through argument and discussion, through examples and stories of people who were harmed by power dynamics, they seek to convince me by naming the act something that arouses in me a desire to react as if to violence.
But I can easily imagine it having a reverse effect -- a crying wolf effect. Someone who describes power dynamics as violence will receive far less response to pointing out actual violence, and for good reason, since the purpose of memory and heuristics is to distinguish between situations, and in past situations, those calls for action have abused language to try to get people to respond.