Actually, the more important reply is this: Regardless of whether you believe that Eliezer has both good intentions and a good plan - I consider the first more probable than the second - the problem is so complex that Eliezer can't solve it on his own. To believe that a group of two, or ten, people on their own, crippled by groupthink, will solve this problem in isolation and get it right the first time, is not just hubris, it's full-blown megalomania.
Re: obviousness of possible failure modes
Date: 2010-05-18 04:09 am (UTC)