这几天美国人工智能学术界和工业界正在联署一份公开信，呼吁大家开发人工智能的时候谨慎行事，以造福人类、保护人类利益为先，加大相关伦理法律哲学等方面问题研究的力度，及早为必将到来的技术奇点做准备。已签名的有不少这个领域里的顶级专家，包括谷歌、微软、Facebook等广泛应用人工智能的公司的相关负责人，包括美国人工智能行业学会组织的领袖，包括最经典的一些人工智能教科书的作者。
公开信所附一篇关于人工智能研究优先级的论文很有意思，引用了90多种论文专著，概述人工智能在伦理法律各方面的风险、对人类社会可能的威胁，总结出应该加强哪些方面的研究来尽量确保人工智能对我们有益。在我看来其中有些含糊的研究目标，不见得能赶上现实中人工智能发展的速度。
其中一个有趣的研究方向：人工智能的崛起将导致越来越多的人无工可作，但另一方面人工智能又能加速财富的创造，应怎样设计财富分配方式来保证大部分人口的生活质量？有些研究表明在很多文化里，失业会导致生活满意度低下。但中东一些产油国的大部分人口都不需要工作，也能过上富足满意的生活，是一种借鉴。我觉得吧，也许最终会归结到以前马克思的设想，生活物资极大丰富的情况下，大家各尽所能，各取所需好了。也许一部分人口从此终日吃喝玩乐无所事事，只要对社会没有妨害，就养着他们好了。总会有另一部分人口会想干些事业，会想开拓创新，会想做一些真正有意义的事，文明的发展要靠他们了。就算地球上的环境、能源、饥荒、病痛什么什么的问题全都解决了，我们还要迈向太空，还要探索宇宙，总有无数的事情可以做。
另外，人工智能将给一些传统道德难题带来新的更棘手的变化。比如你开车的时候，突然刹车失灵，打方向盘的话你自己就会连车带人掉河里淹死，不打方向盘的话就会冲进人群把许多人撞死，这一瞬间你不管做的是什么决定，你最终都会为自己的决定负责。
如果自己淹死了，你为保护无辜的人群作出了牺牲，会有很多人悼念你；如果把很多人撞死了，那么会被起诉，也许要坐牢，总是要付出代价。但如果你坐的是自动驾驶汽车，那一瞬间由人工智能替你做的决定，那么到头来是谁负责呢？如果你自己淹死了，那你的亲属无疑会起诉制造那个人工智能的公司。如果把其他人撞死了，那死者的亲属无疑也会起诉那个公司。有这样的法律风险在，还有谁敢真正把这种车推向市场呢？除非有新的保险计划能保这种情况，而且法律允许这种保险计划。大半个世纪前阿西莫夫以前写的关于机器人三定律的小说，许多都是这方面的思想实验。
有许多人会说，要防止人工智能作恶，最重要的是要有个开关，能随时把它关掉。这是个很好的想法，然而实践起来不见得容易。首先是，你怎么知道这个开关真地能关掉人工智能呢？论文里头讨论了一些“可验证性”方面的问题，也就包括怎么能验证，这开关一切换，人工智能就彻底关掉了。现代软硬件系统过于复杂，很多东西验证起来不是那么容易。就算你把机器的电源线拔了，要是系统里头有备用电源，人工智能也还能接着跑一会。与此相关的一点是，很多人都知道，你的手机就算关机了，也可能还有一些部件其实开着，可能会被某些外来信号激活，运行一些东西。这么复杂的电路，这么复杂的软件，你怎么知道这种事情不会发生呢？
我的一个想法是，应该从最底层起的软硬件设计都开源，用可验证的方法设计装配这些东西，确保任何人随时都能查看源码和数据，验证没有恶意或无意中的漏洞。而且应该让所有软硬件厂商结成联盟，把那些开关、或类似机器人三定律的制约（如果能表达成系统能理解的东西的话），写成所有厂商都应遵循的协议。所有厂商只用这些开源部件，只支持这些开源部件，而且不能让任何一个厂商的势力大到纵跨系统多个层次的地步，这样大家互相制约，谁也没能力独力开发出秘密的能完整运行的人工智能来违反这些协议，也许能最大限度推迟那一天的到来。
当然，人和机器并不必然成为敌人。机器智能也许可以超越人类，人类意识也许可以上载到机器中，也许人类文明进化的下一阶段就是人机结合，这都没啥大不了的。怎样定义人，也许本来就是只在一定历史阶段才有意义的问题，嘿嘿。
--
这几天美国人工智能学术界和工业界正在联署一份公开信，呼吁大家开发人工智能的时候谨慎行事，以造福人类、保护人类利益为先，加大相关伦理法律哲学等方面问题研究的力度，及早为必将到来的技术奇点做准备。已签名的有不少这个领域里的顶级专家，包括谷歌、微软、Facebook等广泛应用人工智能的公司的相关负责人，包括美国人工智能行业学会组织的领袖，包括最经典的一些人工智能教科书的作者。
公开信所附一篇关于人工智能研究优先级的论文很有意思，引用了90多种论文专著，概述人工智能在伦理法律各方面的风险、对人类社会可能的威胁，总结出应该加强哪些方面的研究来尽量确保人工智能对我们有益。在我看来其中有些含糊的研究目标，不见得能赶上现实中人工智能发展的速度。
其中一个有趣的研究方向：人工智能的崛起将导致越来越多的人无工可作，但另一方面人工智能又能加速财富的创造，应怎样设计财富分配方式来保证大部分人口的生活质量？有些研究表明在很多文化里，失业会导致生活满意度低下。但中东一些产油国的大部分人口都不需要工作，也能过上富足满意的生活，是一种借鉴。我觉得吧，也许最终会归结到以前马克思的设想，生活物资极大丰富的情况下，大家各尽所能，各取所需好了。也许一部分人口从此终日吃喝玩乐无所事事，只要对社会没有妨害，就养着他们好了。总会有另一部分人口会想干些事业，会想开拓创新，会想做一些真正有意义的事，文明的发展要靠他们了。就算地球上的环境、能源、饥荒、病痛什么什么的问题全都解决了，我们还要迈向太空，还要探索宇宙，总有无数的事情可以做。
另外，人工智能将给一些传统道德难题带来新的更棘手的变化。比如你开车的时候，突然刹车失灵，打方向盘的话你自己就会连车带人掉河里淹死，不打方向盘的话就会冲进人群把许多人撞死，这一瞬间你不管做的是什么决定，你最终都会为自己的决定负责。
如果自己淹死了，你为保护无辜的人群作出了牺牲，会有很多人悼念你；如果把很多人撞死了，那么会被起诉，也许要坐牢，总是要付出代价。但如果你坐的是自动驾驶汽车，那一瞬间由人工智能替你做的决定，那么到头来是谁负责呢？如果你自己淹死了，那
你的亲属无疑会起诉制造那个人工智能的公司。如果把其他人撞死了，那死者的亲属无疑也会起诉那个公司。有这样的法律风险在，还有谁敢真正把这种车推向市场呢？除非有新的保险计划能保这种情况，而且法律允许这种保险计划。大半个世纪前阿西莫夫以前写的关于机器人三定律的小说，许多都是这方面的思想实验。
有许多人会说，要防止人工智能作恶，最重要的是要有个开关，能随时把它关掉。这是个很好的想法，然而实践起来不见得容易。首先是，你怎么知道这个开关真地能关掉人工智能呢？论文里头讨论了一些“可验证性”方面的问题，也就包括怎么能验证，这开关一切换，人工智能就彻底关掉了。现代软硬件系统过于复杂，很多东西验证起来不是那么容易。就算你把机器的电源线拔了，要是系统里头有备用电源，人工智能也还能接着跑一会。与此相关的一点是，很多人都知道，你的手机就算关机了，也可能还有一些部件其实开着，可能会被某些外来信号激活，运行一些东西。这么复杂的电路，这么复杂的软件，你怎么知道这种事情不会发生呢？
我的一个想法是，应该从最底层起的软硬件设计都开源，用可验证的方法设计装配这些东西，确保任何人随时都能查看源码和数据，验证没有恶意或无意中的漏洞。而且应该让所有软硬件厂商结成联盟，把那些开关、或类似机器人三定律的制约（如果能表达成系统能理解的东西的话），写成所有厂商都应遵循的协议。所有厂商只用这些开源部件，只支持这些开源部件，而且不能让任何一个厂商的势力大到纵跨系统多个层次的地步，这样大家互相制约，谁也没能力独力开发出秘密的能完整运行的人工智能来违反这些协议，也许能最大限度推迟那一天的到来。
当然，人和机器并不必然成为敌人。机器智能也许可以超越人类，人类意识也许可以上载到机器中，也许人类文明进化的下一阶段就是人机结合，这都没啥大不了的。怎样定义人，也许本来就是只在一定历史阶段才有意义的问题，嘿嘿。
How Elon Musk and Y Combinator Plan to Stop Computers From Taking Over
They’re funding a new organization, OpenAI, to pursue the most advanced forms of artificial intelligence — and give the results to the public
As if the field of AI wasn’t competitive enough — with giants like Google, Apple, Facebook, Microsoft and even car companies like Toyota scrambling to hire researchers — there’s now a new entry, with a twist. It’s a non-profit venture called OpenAI, announced today, that vows to make its results public and its patents royalty-free, all to ensure that the scary prospect of computers surpassing human intelligence may not be the dystopia that some people fear. Funding comes from a group of tech luminaries including Elon Musk, Reid Hoffman, Peter Thiel, Jessica Livingston and Amazon Web Services. They have collectively pledged more than a billion dollars to be paid over a long time period. The co-chairs are Musk and Sam Altman, the CEO of Y Combinator, whose research group is also a funder. (As is Altman himself.)
Musk, a well-known critic of AI, isn’t a surprise. But Y Combinator? Yep. That’s the tech accelerator that started 10 years ago as a summer project that funded six startup companies by paying founders “ramen wages” and giving them gourmet advice so they could quickly ramp up their businesses. Since then, YC has helped launch almost 1,000 companies, including Dropbox, Airbnb, and Stripe, and has recently started a research division. For the past two years, it’s been led by Altman, whose company Loopt was in the initial class of 2005, and sold in 2012 for $43.4 million. Though YC and Altman are funders, and Altman is co-chair, OpenAI is a separate, independent venture.
Essentially, OpenAI is a research lab meant to counteract large corporations who may gain too much power by owning super-intelligence systems devoted to profits, as well as governments which may use AI to gain power and even oppress their citizenry. It may sound quixotic, but the team has already scored some marquee hires, including former Stripe CTO Greg Brockman (who will be OpenAI’s CTO) and world-class researcher Ilya Sutskever, who was formerly at Google and was one of the famed group of young scientists studying under neural net pioneer Geoff Hinton in Toronto. He’ll be OpenAI’s research director. The rest of the lineup includes top young talent whose resumes include major academic groups, Facebook AI and DeepMind, the AI company Google snapped up in 2014. There is also a stellar board of advisors including Alan Kay, a pioneering computer scientist.
OpenAI’s leaders spoke to me about the project and its aspirations. The interviews were conducted in two parts, first with Altman and then another session with Altman, Musk, and Brockman. I combined the interviews and edited for space and clarity.
How did this come about?
Sam Altman: We launched YC Research about a month and a half ago, but I had been thinking about AI for a long time and so had Elon. If you think about the things that are most important to the future of the world, I think good AI is probably one of the highest things on that list. So we are creating OpenAI. The organization is trying to develop a human positive AI. And because it’s a non-profit, it will be freely owned by the world.
Elon Musk: As you know, I’ve had some concerns about AI for some time. And I’ve had many conversations with Sam and with Reid [Hoffman], Peter Thiel, and others. And we were just thinking, “Is there some way to insure, or increase, the probability that AI would develop in a beneficial way?” And as a result of a number of conversations, we came to the conclusion that having a 501c3, a non-profit, with no obligation to maximize profitability, would probably be a good thing to do. And also we’re going to be very focused on safety.
And then philosophically there’s an important element here: we want AI to be widespread. There’s two schools of thought — do you want many AIs, or a small number of AIs? We think probably many is good. And to the degree that you can tie it to an extension of individual human will, that is also good.
Human will?
Musk: As in an AI extension of yourself, such that each person is essentially symbiotic with AI as opposed to the AI being a large central intelligence that’s kind of an other. If you think about how you use, say, applications on the internet, you’ve got your email and you’ve got the social media and with apps on your phone — they effectively make you superhuman and you don’t think of them as being other, you think of them as being an extension of yourself. So to the degree that we can guide AI in that direction, we want to do that. And we’ve found a number of like-minded engineers and researchers in the AI field who feel similarly.
Altman: We think the best way AI can develop is if it’s about individual empowerment and making humans better, and made freely available to everyone, not a single entity that is a million times more powerful than any human. Because we are not a for-profit company, like a Google, we can focus not on trying to enrich our shareholders, but what we believe is the actual best thing for the future of humanity.
Doesn’t Google share its developments with the public, like it just did with machine learning?
Altman: They certainly do share a lot of their research. As time rolls on and we get closer to something that surpasses human intelligence, there is some question how much Google will share.
Couldn’t your stuff in OpenAI surpass human intelligence?
Altman: I expect that it will, but it will just be open source and useable by everyone instead of useable by, say, just Google. Anything the group develops will be available to everyone. If you take it and repurpose it you don’t have to share that. But any of the work that we do will be available to everyone.
If I’m Dr. Evil and I use it, won’t you be empowering me?
Musk: I think that’s an excellent question and it’s something that we debated quite a bit.
Altman: There are a few different thoughts about this. Just like humans protect against Dr. Evil by the fact that most humans are good, and the collective force of humanity can contain the bad elements, we think its far more likely that many, many AIs, will work to stop the occasional bad actors than the idea that there is a single AI a billion times more powerful than anything else. If that one thing goes off the rails or if Dr. Evil gets that one thing and there is nothing to counteract it, then we’re really in a bad place.
Will you have oversight over what comes out of OpenAI?
Altman: We do want to build out an oversight for it over time. It’ll start just with Elon and me. We’re still a long, long way from actually developing real AI. But I think we’ll have plenty of time to build out an oversight function.
Musk: I do intend to spend time with the team, basically spending an afternoon in the office every week or two just getting updates, providing any feedback that I have and just getting a much deeper understanding of where things are in AI and whether we are close to something dangerous or not. I’m going to be super conscious personally of safety. This is something that I am quite concerned about. And if we do see something that we think is potentially a safety risk, we will want to make that public.
What’s an example of bad AI?
Altman: Well, there’s all the science fiction stuff, which I think is years off, like The Terminator or something like that. I’m not worried about that any time in the short term. One thing that I do think is going to be a challenge — although not what I consider bad AI — is just the massive automation and job elimination that’s going to happen. Another example of bad AI that people talk about are AI-like programs that hack into computers that are far better than any human. That’s already happening today.
Are you starting with a system that’s built already?
Altman: No. This is going to start like any research lab and it’s going to look like a research lab for a long time. No one knows how to build this yet. We have eight researchers starting on day one and a few more will be joining over the next few months. For now they are going to use the YC office space and as they grow they’ll move out on their own. They will be playing with ideas and writing software to see if they can advance the current state of the art of AI.
Will outsiders contribute?
Altman: Absolutely. One of the advantages of doing this as a totally open program is that the labs can collaborate with anyone because they can share information freely. It’s very hard to go collaborate with employees at Google because they have a bunch of confidentiality provisions.
Sam, since OpenAI will initially be in the YC office, will your startups have access to the OpenAI work? [UPDATE: Altman now tells me the office will be based in San Francisco.]
Altman: If OpenAI develops really great technology and anyone can use it for free, that will benefit any technology company. But no more so than that. However, we are going to ask YC companies to make whatever data they are comfortable making available to OpenAI. And Elon is also going to figure out what data Tesla and Space X can share.
What would be an example of the kind of data that might be shared?
Altman: So many things. All of the Reddit data would be a very useful training set, for example. You can imagine all of the Tesla self-driving car video information being very valuable. Huge volumes of data are really important. If you think about how humans get smarter, you read a book, you get smarter, I read a book, I get smarter. But we don’t both get smarter from the book the other person read. But, using Teslas as an example, if one single Tesla learned something about a new condition every Tesla instantly gets the benefit of that intelligence.
Musk: In general we don’t have a ton of specific plans because this is really just the incipient stage of the company; it’s kind of the embryonic stage. But certainly Tesla will have an enormous amount of data, of real world data, because of the millions of miles accumulated per day from our fleet of vehicles. Probably Tesla will have more real world data than any other company in the world.
Tesla Motors CEO and Product Architect Elon Musk, Y Combinator President Sam Altman onstage during ‘the Vanity Fair New Establishment Summit n San Francisco, California. (Photo by Mike Windle/Getty Images for Vanity Fair)
AI needs a lot of computation. What will be your infrastructure?
Altman: We are partnering with Amazon Web Services. They are donating a huge amount of infrastructure to the effort.
And there is a billion dollars committed to this?
Musk: I think it’s fair to say that the commitment actually is some number in excess of a billion. We don’t want to give an exact breakdown but there are significant contributions from all the people mentioned in the blog piece.
Over what period of time?
Altman: However long it takes to build. We’ll be as frugal as we can but this is probably a multi-decade project that requires a lot of people and a lot of hardware.
And you don’t have to make money?
Musk: Correct. This is not a for-profit investment. It is possible that it could generate revenue in the future in the same way that the Stanford Research Institute is a 501c3 that generates revenue. So there could be revenue in the future, but there wouldn’t be profits. There wouldn’t be profits that would just enrich shareholders, there wouldn’t be a share price or anything. We think that’s probably good.
Elon, you earlier invested in the AI company DeepMind, for what seems to me to be the same reasons — to make sure AI has oversight. Then Google bought the company. Is this a second try at that?
Musk: I should say that I’m not really an investor in any normal sense of the word. I don’t seek to make investments for financial return. I put money into the companies that I help create and I might invest to help a friend, or because there’s some cause that I believe in or something I’m concerned about. I am really not diversified beyond my own company in any material sense of the word. But yeah, my sort of “investment,” in quotes, for DeepMind was just to get a better understanding of AI and to keep an eye on it, if you will.
You will be competing for the best scientists now who might go to Deep Mind or Facebook or Microsoft?
Altman: Our recruiting is going pretty well so far. One thing that really appeals to researchers is freedom and openness and the ability to share what they’re working on, which at any of the industrial labs you don’t have to the same degree. We were able to attract such a high-quality initial team that other people now want to join just to work with that team. And then finally I think our mission and our vision and our structure really appeals to people.
How many researchers will you eventually hire? Hundreds?
Altman: Maybe
I want to return to the idea that by sharing AI, we might not suffer the worst of its negative consequences. Isn’t there a risk that by making it more available, you’ll be increasing the potential dangers?.
Altman: I wish I could count the hours that I have spent with Elon debating this topic and with others as well and I am still not a hundred percent certain. You can never be a hundred percent certain, right? But play out the different scenarios. Security through secrecy on technology has just not worked very often. If only one person gets to have it, how do you decide if that should be Google or the U.S. government or the Chinese government or ISIS or who? There are lots of bad humans in the world and yet humanity has continued to thrive. However, what would happen if one of those humans were a billion times more powerful than another human?
Musk: I think the best defense against the misuse of AI is to empower as many people as possible to have AI. If everyone has AI powers, then there’s not any one person or a small set of individuals who can have AI superpower.
Elon, you are the CEO of two companies and chair of a third. One wouldn’t think you have a lot of spare time to devote to a new project.
Musk: Yeah, that’s true. But AI safety has been preying on my mind for quite some time, so I think I’ll take the trade-off in peace of mind.
评论 (0)