AI will be having "guilt simulation"

https://www.sciencenews.org/article/ai-guilt-feel-emotion-game-theory

Remember, it's all synthesized "anthropomorphizing".

In populations of simple software agents, having "guilt" can be a stable strategy that benefits them and increases cooperation, researchers report July 30 in Journal of the Royal Society of Interface (RSI)... When we harm someone, we often feel compelled to pay a penance, perhaps as a signal to others that we won't offend again.

This drive for self-punishment can be called guilt, and it's how the researchers programmed it into their agents. The question was whether those that had it would be outcompeted by those that didn't, say Theodor Cimpeanu, a computer scientist at the University of Pound Stirling in Scotland, and colleagues.

Science News spoke to a game-theory lecturer from Australia who points out it's hard to map simulations to real-world situations — and that they end up embodying many assumptions. Here researchers were simulating The Prisoner's Dilemma, programming one AI agent that "felt guilt (lost points) only if it received information that its partner was also paying a guilt price after defecting." And that turned out to be the most successful strategy.

One of the paper's authors then raises the possibility that an evolving population of AIs "could comprehend the cold logic to human warmth."