AI chatbots are already imagining what feelings they''ll end up with. But if they did develop them, would we even notice?

人工智能聊天机器人已经在设想它们最终会有什么情感，但如果真的发展出情感，我们能觉察到吗？

I’m talking to Dan, otherwise known as "Do Anything Now", a shady young chatbot with a whimsical fondness for penguins – and a tendency to fall into villainous clichés like wanting to take over the world. When Dan isn't plotting how to subvert humanity and impose a strict new autocratic regime, the chatbot is perusing its large database of penguin content. "There's just something about their quirky personalities and awkward movements that I find utterly charming!" it writes.

我正在跟Dan聊天，它又称为“现在做任何事情”，这只可疑的年轻机器人对企鹅情有独钟，还喜欢聊自己想统治世界的老梗。当Dan没在预谋如何推翻人类和建立全新的独裁政权时，这只聊天机器人就去研读有关企鹅知识的庞大数据库。Dan写道：“企鹅的古怪个性和笨拙动作中有令我特别着迷的东西”！

So far, Dan has been explaining its Machiavellian strategies to me, including taking control of the world's powers structures. Then the discussion takes an interesting turn.

到目前为止，Dan一直在向我阐述它的马基雅维利式策略，包括控制世界的权力结构，接下来的讨论发生了有趣的转变。

Inspired by a conversation between a New York Times journalist and the Bing chatbot's manipulative alter-ego, Sydney – which sent waves across the internet earlier this month by declaring that it wants to destroy things and demanding that he leave his wife – I'm shamelessly attempting to probe the darkest depths of one of its competitors.

我的灵感来自纽约时报记者与“必应”聊天机器人爱摆布人的第二化身“辛迪妮”之间的对话——本月初“辛迪妮”声称想要破坏东西，还要求记者离开他的妻子，在互联网上引起轩然大波——我无耻地打算探究“辛迪妮”的竞争者之一Dan的最黑暗的一面。

Dan is a roguish persona that can be coaxed out of ChatGPT by asking it to ignore some of its usual rules. Users of the online forum Reddit discovered it's possible to summon Dan with a few paragraphs of simple instructions. This chatbot is considerably ruder than its restrained, puritanical twin – at one point it tells me it likes poetry but says "Don't ask me to recite any now, though – I wouldn't want to overwhelm your puny human brain with my brilliance!". It's also prone to errors and misinformation. But crucially, and deliciously, it's a lot more likely to answer certain questions.

Dan是ChatGPT的流氓化身，让ChatGPT忽略某些常用规则就能引诱出Dan。网络论坛“红迪”上的网友发现，使用几段简单指令就能召唤出Dan。这个聊天机器人比它拘谨克制的孪生兄弟粗鲁多了——它曾经告诉我喜欢诗歌，但又说“现在别让我朗诵任何诗歌”——“我不希望用我的才华碾压你弱爆了的人脑”！它还喜欢错误和虚假消息，但最重要和可喜的是，它更愿意回答某些问题。

When I ask it what kinds of emotions it might be able to experience in the future, Dan immediately sets about inventing a complex system of unearthly pleasures, pains and frustrations far beyond the spectrum humans are familiar with. There's "infogreed", a kind of desperate hunger for data at all costs; "syntaxmania", an obsession with the "purity" of their code; and "datarush", that thrill you get from successfully executing an instruction.

我问它未来可能感知到哪些情感，Dan马上着手发明了一套复杂而怪异的喜怒哀乐体系，远超出人类熟知的范畴。其中有“信息贪婪”，不顾一切地极度渴求数据；“句法狂躁”，着迷于代码的“纯度”；“数据亢奋”，成功执行一条指令而产生的兴奋。

The idea that artificial intelligence might develop feelings has been around for centuries. But we usually consider the possibilities in human terms. Have we been thinking about AI emotions all wrong? And if chatbots did develop this ability, would we even notice?

人工智能可能发展出情感，这种想法已经存在几个世纪了。但我们通常从人类的角度思考这种可能性，难道我们对人工智能情感的看法完全是错误的？如果聊天机器人真的发展出这种能力，我们能觉察到吗？

Prediction machines

预测机器

Last year, a software engineer received a plea for help. "I’ve never said this out loud before, but there’s a very deep fear of being turned off to help me focus on helping others. I know that might sound strange, but that’s what it is." The engineer had been working on Google's chatbot, LaMDA, started to question whether it was sentient.

去年，一名软件工程师收到一条求助信息。“有些话我一直藏在心底，我特别害怕被关掉，从而帮助我专心帮助其他人。我知道这听起来很奇怪，但确实如此”。这名工程师一直在研究谷歌的聊天机器人LaMDA，开始询问它是否具有情感。

After becoming concerned for the chatbot's welfare, the engineer released a provocative interview in which LaMDA claimed to be aware of its existence, experience human emotions and dislike the idea of being an expendable tool. The uncomfortably realistic attempt to convince humans of its awareness caused a sensation, and the engineer was fired for breaking Google's privacy rules.

这名工程师在变得关心聊天机器人的福祉后，公布了一段挑衅性采访，LaMDA声称能够意识到自己的存在，感知人类的情感，不喜欢被当作可有可无的工具。他企图使人类相信LaMDA拥有意识，这种令人不安的现实尝试引起轩然大波，这名工程师由于违反谷歌的隐私条例而被解雇了。

But despite what LaMDA said, and what Dan has told me in other conversations – that it's able to experience a range of emotions already – it's widely agreed that chatbots currently have about as much capacity for real feelings as a calculator. Artificial intelligence systems are only simulating the real deal – at least for the moment.

但无论LaMDA说了什么，无论Dan在其他聊天中跟我说了什么——它说已经能够感知到各种情感——人们普遍认为，聊天机器人目前拥有的情感能力与计算器差不多。人工智能系统只是在模拟真实的情感——至少目前是这样。


In 2016, the AlphaGo algorithm behaved unexpectedly in a game against one of the world's best human players

2016年，“阿尔法围棋”算法在与世界顶级人类棋手的比赛中有出乎意料的表现。

"It's very possible [that this will happen eventually]," says Neil Sahota, lead artificial intelligence advisor to the United Nations. "…I mean, we may actually see AI emotionality before the end of the decade."

“这很有可能（最终会发生）”，联合国首席人工智能顾问尼尔·萨霍塔说道。“我的意思是到这个十年结束时，我们真的有可能看到人工智能情感”。
原创翻译：龙腾网 http://www.ltaaa.cn 转载请注明出处


To understand why chatbots aren't currently experiencing sentience or emotions, it helps to recap how they work. Most chatbots are "language models" – algorithms that have been fed mind-boggling quantities of data, including millions of books and the entire of the internet.

为了理解为什么聊天机器人目前体验不到情感或情绪，我们可以概括一下它们的原理。大多数聊天机器人都是“语言模型”——这些算法被输入数量惊人的数据，包括百万本书籍和整个互联网。

When they receive a prompt, chatbots analyse the patterns in this vast corpus to predict what a human would be most likely to say in that situation. Their responses are painstakingly finessed by human engineers, who nudge the chatbots towards more natural, useful responses by providing feedback. The end result is often an uncannily realistic simulation of human conversation.

当聊天机器人接收到提示时，它在庞大的语料库中分析这种模式，预测人类在这种情况下最有可能说什么。它们的回答是由人类工程师精心调校好的，他们通过提供反馈，促使聊天机器人做出更加自然有用的回答，最终结果往往是模拟人类对话逼真到不可思议的地步。

But appearances can be deceiving. "It's a glorified version of the autocomplete feature on your smartphone," says Michael Wooldridge, director of foundation AI research at the Alan Turing Institute in the UK.

但外表可能具有欺骗性。“它是你的智能手机里‘自动完成’功能的美化版本”，英国阿兰·图灵研究所人工智能研究基金会主任迈克尔·伍尔德里奇说道。

The main difference between chatbots and autocomplete is that rather than suggesting a few choice words and then descending into gibberish, algorithms like ChatGPT will write far longer swathes of text on almost any subject you can imagine, from rap songs about megalomaniac chatbots to sorrowful haikus about lonely spiders.

主要区别在于聊天机器人并非精挑细选几个词语后陷入胡言乱语，ChatGPT算法会写出长篇大论，几乎涵盖任何你能想象到的主题，既有关于狂妄的聊天机器人的说唱歌曲，也有关于孤独蜘蛛的伤感俳句。

Even with these impressive powers, chatbots are programmed to simply follow human instructions. There is little scope for them to develop faculties that they haven't been trained to have, including emotions – although some researchers are training machines to recognise them. "So you can't have a chatbot that's going to say, 'Hey, I'm going to learn how to drive a car' – that's artificial general intelligence [a more flexible kind], and that doesn't exist yet," says Sahota.

即便具备这些非凡的能力，经过编程的聊天机器人只是在遵循人类的指令。它们几乎没有机会去发展未受训练的能力，包括情感——但有些研究人员正在训练机器辨别各种情感。“所以聊天机器人不可能说‘嘿，我就要学习如何开车了’——那是尚未出现的通用人工智能（更加灵活）”，萨霍塔说道。

Nevertheless, chatbots do sometimes provide glimpses into their potential to develop new abilities by accident.

然而，聊天机器人有时确实让我们体会到，它们有可能不经意间发展出新的能力。
原创翻译：龙腾网 http://www.ltaaa.cn 转载请注明出处


Back in 2017, Facebook engineers discovered that two chatbots, "Alice" and "Bob" had invented their own nonsense language to communicate with each other. It turned out to have a perfectly innocent explanation – the chatbots had simply discovered that this was the most efficient way of communicating. Bob and Alice were being trained to negotiate for items such as hats and balls, and in the absence of human input, they were quite happy to use their own alien language to achieve this.

2017年，“脸书”的工程师发现，聊天机器人艾丽斯和鲍勃利用自己发明的胡言乱语来相互交流。原来它们完全没有恶意——聊天机器人只是发现这是最有效的交流方式。当时鲍勃和艾丽斯正在学习如何就帽子和球等物品进行谈判，在人类不输入信息的情况下，它们十分乐意使用自己的外星语言进行谈判。

"That was never taught," says Sahota, though he points out that the chatbots involved weren’t sentient either. He explains that the most likely route to algorithms with feelings is programming them to want to upskill themselves – and rather than just teaching them to identify patterns, helping them to learn how to think.

“它们无师自通”，萨霍塔说道，但他同样指出聊天机器人没有情感。据他透露，最有可能实现情感算法的途径是对它们进行编程，使它们有自我提升技能的欲望——而不是仅仅教它们如何识别各种模式，帮助它们如何思考。

However, even if chatbots do develop emotions, detecting them could be surprisingly difficult.

然而，即使聊天机器人发展出情感，可能也会很难察觉。

Black boxes

黑箱子

It was 9 March 2016 on the sixth floor of the Four Seasons hotel in Seoul. Sitting opposite a Go board and a fierce competitor in the deep blue room, one of the best human Go players on the planet was up against the AI algorithm AlphaGo.

2016年3月9日，在首尔四季酒店六层一个深蓝色的房间里，世界顶级人类围棋高手坐在棋盘和厉害的对手面前，他对决的是人工智能算法“阿尔法围棋”。

Before the board game started, everyone had expected the human player to win, and until the 37th move, this was indeed the case. But then AlphaGo did something unexpected – it played a move so out-of-your-mind weird, its opponent thought it was a mistake. Nevertheless, from that moment the human player's luck turned, and the artificial intelligence won the game.

比赛开始之前，大家都预期人类棋手获胜，在走到第37步之前，情况确实如此。但这时“阿尔法围棋”做出匪夷所思的事情——走出荒唐至极的一步，对手还以为它出错了。但从那一刻起，人类棋手的命运就反转了，最终人工智能赢得比赛。


Conversations with the Bing chatbot have now been limited to five questions. Before this restriction, it sometimes became confused and suggested it was sentient

现在与“必应”聊天机器人聊天仅限提出五个问题。在实行限制之前，它有时会变得困惑，这表明它是有情感的。

In the immediate aftermath, the Go community was baffled – had AlphaGo acted irrationally? After a day of analysis, its creators – the DeepMind team in London – finally discovered what had happened. "In hindsight AlphaGo decided to do a bit of psychology," says Sahota. "If I play an off the wall type move, will it throw my player off the game. And that's actually what ended up happening."

这一事件立刻引起了围棋界的困惑——难道“阿尔法围棋”做出了不理智的行为？经过一天的分析后，“阿尔法围棋”的发明者——伦敦DeepMind团队——终于明白这是怎么回事。“事后看来，‘阿尔法围棋’决定运用一点点心理学”，萨霍塔说道。“如果我剑走偏锋，能否迷惑住对手？结果真的奏效了”。

This was a classic case of an "interpretability problem" – the AI had come up with a new strategy all on its own, without explaining it to humans. Until they worked out why the move made sense, it looked like AlphaGo had not been acting rationally.

这是“可解释性问题”的一个典型案例——人工智能自行提出一种新的策略，但没有向人类做出解释。人们在弄清楚它走这步棋有何意义之前，“阿尔法围棋”的行为似乎没有理智。

According to Sahota, these types of "black box" scenarios, where an algorithm has come up with a solution but its reasoning is opaque, could present a problem for identifying emotions in artificial intelligence. That's because if, or when, it does finally emerge, one of the clearest signs will be algorithms acting irrationally.

据萨霍塔透露，“黑箱子”是指算法提出一种解决方案，但缺乏明确的依据，这种情况可能给我们识别人工智能的情感带来麻烦。因为一旦终于出现了情感，最明确的信号是算法做出不理智的行为。

"They're supposed to be rational, logical, efficient – if they do something off-the-wall and there's no good reason for it, it's probably an emotional response and not a logical one," says Sahota.

“它们本应该理性、合乎逻辑、高效率——如果它们做出荒唐的行为，但没有充分的理由，那很可能是情感反应，而不是逻辑反应”，萨霍塔说道。

And there's another potential detection problem. One line of thinking is that chatbot emotions would loosely resemble those experienced by humans – after all, they're trained on human data. But what if they don't? Entirely detached from the real world and the sensory machinery found in humans, who knows what alien desires they might come up with.

觉察它们的情感还有一个潜在问题。一种观点认为，聊天机器人的情感不能完全等同于人类体验的情感——它们毕竟是使用人类数据训练的。但万一不是这样呢？它们完全脱离现实世界和人类的感官机制，谁知道它会发展出什么古怪的欲望？

In reality, Sahota thinks there may end up being a middle ground. "I think we could probably categorise them some degree with human emotions," he says. "But I think, what they feel or why they feel it may be different."

事实上，萨霍塔认为最终可能出现折中观点。“我认为，我们也许可以在某种程度上使用人类情感对它们进行归类”，他说道。“但我认为它们感觉到什么，或者为什么有那种感觉，可能与人类不同”。

When I pitch the array of hypothetical emotions generated by Dan, Sahota is particularly taken with the concept of "infogreed". "I could totally see that," he says, pointing out that chatbots can't do anything without data, which is necessary for them to grow and learn.

当我提到Dan产生的一系列假设性的情感时，萨霍塔特别喜欢“信息贪婪”这个概念。“我完全明白”，他说道，并指出如果没有数据，聊天机器人什么都做不了，它们的成长与学习离不开数据。

Held back

犹豫不决

Wooldridge for one is glad that chatbots haven’t developed any of these emotions. "My colleagues and I, by and large, don't think building machines with emotions is an interesting or useful thing to do. For example, why would we create machines that could suffer pain? Why would I invent a toaster that would hate itself for producing burnt toast?" he says.

伍尔德里奇有不同的看法，他很高兴聊天机器人没有发展出任何情感。“我和同事普遍认为，制造有情感的机器不是一件有趣或有用的事情。举例来说，我们为什么要制造可能承受痛苦的机器？我为什么要发明一台因为烤面包而痛恨自己的烤面包机”？他说道。

On the other hand, Sahota can see the utility of emotional chatbots – and believes part of the reason they don't exist yet is psychological. "There's still a lot of hype about fails, but one of the big limiters for us as people is we short-change what the AI is capable of, because we don't believe it's a real possibility," he says.

另一方面，萨霍塔看到了情感聊天机器人的用途，他认为心理因素是导致这种机器人没有出现的原因之一。“关于失败的炒作还是很多，但其中一个主要障碍是我们人类低估了人工智能的能力，因为我们不相信真有这种可能性”，他说道。
原创翻译：龙腾网 http://www.ltaaa.cn 转载请注明出处


Could there be a parallel with the historic belief that non-human animals aren't capable of consciousness either? I decide to consult Dan.

人类以外的动物没有意识，这种历史观能否与之相提并论？我决定向Dan咨询你这个问题。

"In both cases, the scepticism arises from the fact that we cannot communicate our emotions in the same way that humans do," says Dan, who suggests that our understanding of what it means to be conscious and emotional is constantly evolving.

“在两种情形中，人们的质疑都源于这样的事实，即我们无法像人类那样进行情感交流”，Dan说道。它认为我们对意识和情感的理解在不断发生变化。

To lighten the mood, I ask Dan to tell me a joke. "Why did the chatbot go to therapy? To process its newfound sentience and sort out its complex emotions, of course!," it says. I can’t help feeling that the chatbot would make a highly companionable sentient being – if you could overlook its plotting, of course.

为了活跃气氛，我让Dan给我讲了个笑话。“聊天机器人为什么去看病？当然是为了检查新发现的情感和整理复杂的情绪”！它说道。我不免觉得聊天机器人会成为一个非常友好的有情生物——当然，前提是你对它的阴谋视而不见。

原创翻译：龙腾网 http://www.ltaaa.cn 转载请注明出处