AGI Analogies

An increasing number of people are sounding the alarm about the risks posed by
artificial general intelligences (AGIs), but I’ve found it hard to viscerally
understand the magnitude of the problem. Here are some analogies that I’ve found
useful when thinking about the risks of creating an entity massively smarter
than we are.


Imagine two cats making an agreement that they won’t run back and forth across
a room. No matter what happens, the cats agree to just sit in one place. You
can thwart this easily with a laser pointer. Not only are the cats helpless
against this attack, they won’t even see it as an attack. A low-level part of
their brain will chase the pointer around the carpet, breaking their vow, and
to an observing cat it will seem like pure magic.

Note that this type of manipulation is completely unlike how cats would
manipulate each other. A cat might try to break the other cat’s vow by
tempting it, or convincing it to go over there, but no cat would ever think to
develop laser technology and flash a dot from across the room.

An AGI might be as much smarter than us as we are than cats, and it may develop
a laser pointer.


Imagine that there’s a wealthy person and you want to convince them to protect
you and your family. You want them to feed you and house you and keep you safe.
What would you try? You’d come up with a convincing argument, maybe. Appeal to
the person’s emotions or ego. You might come up with various ideas for the
“attack”, and a wealthy person might come up with various defenses against
these attacks.

This is not how cows solved the problem. They solved the problem by being
delicious
. This is not a solution you, as a human, would have thought of.
Again, it doesn’t even feel like an attack. You want to protect and house
and feed the cows. No human says, “I need to defend myself against the attack
of delicious hamburgers.” Yet here we are willingly doing all this work for
cows, not to mention damaging the environment and perhaps our health. Corn did
an even better job.


When people imagine an AGI trying to “get out of the box”, they imagine a very
smart human attacking us doing what normal humans do, just smarter, with better
arguments. But a laser pointer is not a smarter version of how cats manipulate
one another. It is a tactic entirely outside the realm of reality of cats, and
they’re 100% helpless against it.

And we have little defense against the deliciousness of beef. We could defend
ourselves against a human-like cow that tried to argue that we should take it
in from the wild and feed it and house it. It might have all sorts of
reasonable arguments, and we could argue back or at worst just force ourselves
to ignore it. But we don’t even want to ignore hamburgers. We even feel bad
for the way that we house and feed cows!

So when you’re trying to imagine an AGI trying to get out of the box, don’t
imagine something like, “If you let me out, I’ll cure cancer!” which is
tempting if you believe it, but you can perhaps see through the lie and resist
the temptation. The attack will either be like a laser pointer, perhaps a long
sequence of vowels that causes you to let the AGI out of the box, or it’ll be
beef-like, where you find yourself wanting to let it out of the box without it
even asking.

Some people encourage us to think about how to “defend” ourselves against an
AGI, but I think using that term limits our imagination to things that we’d
consider an attack. We can imagine a defense against other humans. We can even
imagine defenses against non-human things like viruses and space aliens, but
only because we anthropomorphize them as microscopic humans or green ones. By
using the word defend we implicitly restrict our thinking to the kinds of
things that would feel like an attack, in other words, attacks that
human-like creatures would use, and methods that humans know about.

We must instead imagine a method that’s as alien to us as a laser point is to a
cat, or as imaginative as the juicy steak, neither of which is even recognized
as an attack by the victim. And when I say “imagine” I don’t mean a specific
approach, since that’s by definition impossible. I mean imagine that
something would happen and we’d be helpless. Internalize that it would be as
easy for an AGI to manipulate us as it is for us to manipulate cats. Imagine
literal magic, because that’s more or less what it would seem like.


The final analogy addresses the problem of alignment, which is different than
the problem of control. If you don’t think that AGIs need to be aligned with
us, then think about the fact that some humans are amoral. They don’t
necessarily want to kill, but they’ll easily kill five people if it’ll help
them in some way. We don’t worry too much about these people because we know
the law will keep them from doing too much damage.

Also consider that some people are very good at arguing their point, even when
they’re wrong. You know people like this.

Combine these two, but crank up the good-argument skill to the point where this
person can talk their way out of any problem. They can commit murder, and in
court convince the judge and jury that they shouldn’t be punished. Every time.

You should be worried about this person being in our society. They’re a bad
combination of being immune to laws and they are not aligned with what the
rest of us want.

An AGI, by default, would be like this. They would be immune to laws for the
reasons I argued above (they have laser pointers), and they’re amoral (not
aligned with us). I don’t think we can solve the first problem, but the second
one is, in principle, solvable. And we need to solve it before we need it,
because by the time we have an amoral super-arguer, it’ll be too late: they’ll
talk us out of solving the problem.

find the cost of your paper

Sep 13, Grand Remembrances

Today is Grandparents Day in the United States. Being a Grand is a special honor. I feel very blessed that my wife and I have two grandchildren. We were able to visit them today. Yes, we are still being cautious with the coronavirus, but we also find it very difficult to not see them when they live so close. So today we did drop by to visit Jacob (age 10) and Sophia (age 7) along with their parents. We brought donuts and caught up with them. Our grandchildren are still pretty young and this is a precious time in their lives – and ours!

I wish I had known my grandparents better. We never lived in the same place. Dad was a career Air Force pilot, so we moved around a lot. But we did get to see them once in a while when they would visit us, or we them.

A Plague of Giants

There are five known magical ‘kennings’ or types: air, water, fire, earth, and plants. Each nation specializes in of these kennings, and the magic influences the society. There’s a big pitfall with this diversity of ability and locale–not everyone gets along.

Enter the Hathrim giants, or ‘lavaborn’ whose kenning is fire. Where they live the trees that fuel their fire are long gone, but the giants are definitely not welcome anywhere else. They’re big, they’re violent, and they’re ruthless. When a volcano erupts and they are forced to evacuate, they take the opportunity to relocate. They don’t care that it’s in a place where they aren’t wanted.

I first read Kevin Hearne’s Iron Druid books and loved them (also the quirky The Tales of Pell), so was curious about this new venture, starting with A PLAGUE OF GIANTS. Think Avatar: The Last Airbender meets Jim Butcher’s Codex Alera series. Elemental magic, a variety of races, different lands. And it’s all thrown at you from page one.

But this story is told a little differently. It starts at the end of the war, after a difficult victory, and a bard with earth kenning uses his magic to re-tell the story of the war to a city of refugees. And it’s this movement back and forth in time and between key players in this war that we get a singularly grand view of the war as a whole. Hearne uses this method to great effect.

There are so many interesting characters in this book that I can’t cover them all here. Often in books like this such a large cast of ‘main’ character can make the storytelling suffer, especially since they don’t have a lot of interaction with each other for the first 3/4 of the book–but it doesn’t suffer, thankfully. And the characterization is good enough, despite these short bursts, that by the end we understand these people and care about what happens to them.

If there were a main character it would be Dervan, a historian who is assigned to record (also spy on?) the bard’s stories. He finds himself caught up in machinations he feels unfit to survive. Fintan is the bard from another country, who at first is rather mysterious and his true personality is hidden by the stories he tells; it takes a while to understand him. Gorin Mogen is the leader of the Hathrim giants who decide to find a new land to settle. He’s hard to like, but as far as villains go, you understand his motivations and he can be even a little convincing. There’s Abhi, the son of hunters, who decides hunting isn’t the life for him–and unexpectedly finds himself on a quest for the sixth kenning. And Gondel Vedd, a scholar of linguistics who finds himself tasked with finding a way to communicate with a race of giants never seen before (definitely not Hathrim) and stumbles onto a mystery no one could have guessed: there may be a seventh kenning.

There are other characters, but what makes them all interesting is that they’re regular people (well, maybe not Gorin Mogen or the viceroy–he’s a piece of work) who become heroes in their own little ways, whether it’s the teenage girl who isn’t afraid to share vital information, to the scholars who suddenly find how crucial their minds are to the survival of a nation, to the humble public servants who find bravery when they need it most. This is a story of loss, love, redemption, courage, unity, and overcoming despair to not give up. All very human experiences by simple people who do extraordinary things.

Hearne’s worldbuilding is engaging. He doesn’t bottle feed you, at first it feels like drinking from a hydrant, but then you settle in and pick up things along the way. Then he shows you stuff with a punch to the gut. This is no fluffy world with simple magic without price. All the magic has a price, and more often than not it leads you straight to death’s door. For most people just the seeking of the magic will kill you. I particularly enjoyed the scenes with Ahbi and his discovery of the sixth kenning and everything associated with it. But giants? I mean, really? It isn’t bad enough fighting people who can control fire that you have to add that they’re twice the size of normal people? For Hearne if it’s war, the stakes are pretty high, and it gets ugly.

The benefit of the storytelling style is that the book, despite its length, moves along steadily (Hearne is no novice, here). The bits of story lead you along without annoying cliffhangers (mostly), and I never got bored with the switch between characters. It was easy to move between them, and they were recognizable enough that I got lost or confused. The end of the novel felt a little abrupt, but I guess that has more to do with I was ready for the story to continue, despite the exiting climax.

If you’re looking for epic fantasy with fun storytelling and clever worldbuilding, check out A PLAGUE OF GIANTS.

The post A Plague of Giants appeared first on Elitist Book Reviews.

The Artwork Of Gary Choo

Gary Choo is a concept artist/illustrator based in Singapore. I’ve know Gary for a good many years ( 17, actually ), working together in animation studios in Singapore like Silicon Illusions and Lucasfilm. Gary currently runs an art team at Mighty Bear Games, but when time allows he also draws covers for Marvel comics, and they’re amazing –

The Art Of Gary Choo
The Art Of Gary Choo
The Art Of Gary Choo
The Art Of Gary Choo
The Art Of Gary Choo

To see more of Gary’s work or to engage him for freelance work, head down to his ArtStation.

The post The Art Of Gary Choo appeared first on Halcyon Realms – Art Book Reviews – Anime, Manga, Film, Photography.

27