My path to AGI

History / Edit / PDF / EPUB / BIB /
Created: October 22, 2015 / Updated: November 2, 2024 / Status: in progress / 18 min read (~3490 words)
Artificial General Intelligence

If you are interested in a more digestible path through the same content I've experienced, I suggest you look up my suggested path to AGI article. It basically contains most of the stuff I've gone through myself, but it is organized in a constructive manner to a new comer (at least, in my opinion).

If you have comments and suggestions, feel free to let me know through the comments!

Note: The dates in parenthesis indicate when I started and finished reading a given book/paper, not their publication date.

Stuart Russell, Peter Norvig

Only reading the content without doing any exercises is about 60-65h of reading.

Douglas Engelbart

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller

YKY

Ray Kurzweil

Andrew Ng

Alan M. Turing, B. Jack Copeland

  • On Computable Numbers, with an Application to the Entscheidungsproblem (2015-06 - ?)
  • Intelligent Machinery (2015-10)
  • Computing Machinery and Intelligence (2015-10)
  • Can Digital Computer Think? (2015-10)
  • Solvable and Unsolvable Problems (2015-10)
  • Systems of Logic Based on Ordinals (2016-01 - ?)
  • Can Automatic Calculating Machines Be Said to Think?
  • Lecture on the Automated Computing Engine
  • The Chemical Basis of Morphogenesis
  • Chess

Tom Murphy VII

Paul R. Halmos

Claude E. Shannon

Matt Mahoney

Marcus Hutter

Shane Legg

Douglas R. Hofstadter

Marvin Minsky

John von Neumann

  • Tensorflow 2016-01

See PHP-Brain

For about two weeks, I spent my time trying to find ways to do efficient string matching and document learning. The reason I wanted to do this was to attempt to learn language through examples. Provided with enough (hopefully valid) documents, it would be possible to assess if a given sentence is valid by matching it with previously seen sentences. It would also allow me to feed it a bunch of my chatlogs so that it may learn my vocabulary behavior and potentially predict what I would type next (basically autocompletion).

With a program that could do not much more than receive texts and match them against previously seen documents, I wanted to figure out a way to match generic data instead of words/sentences.

However, it wasn't only about matching generic data. It was also about being able to match data of different types with one another. In other words, if an image contained a sentence within one of its region, I'd like to be able to reuse that region to store the sentence as well as this region of the image. Thus, using only one run of the data I have two different informations: a sentence and a part of an image.

Furthermore, the idea is also to be able to relate concepts of varying degree of abstraction with one another.

In other to get an idea of how this could be accomplished, I started to reflect on the architecture of the human body and how the brain processes signal/information.

I began by designing a model-based brain, in other words, something that would look like a brain but from the inside would be something different. The main idea was to have a loop that would run forever, which would select tasks to execute. Input from the environment as well as actions to undertake (output) were considered to be tasks. The brain would have to process an input task, which would consist of a stream of data to be parsed and looked for patterns. An output task would be the emission of a stream of data (a stored pattern we expect to produce the desired result) to a target actuator. In other words, the brain would have to learn how to use its outputs through pattern emission, like it would try to learn to decipher the input streams it would receive.

After playing with the idea for a while, I came across an issue: what should I do when a task may take several seconds, even maybe minutes or hours? It doesn't make sense to stop processing everything else until said task is complete. That is not a real time approach. Thus, it meant we'd have to implement some sort of system to do task preemption. PHP being a single threaded language, this would prove to be more complicated than I wanted to deal with. Thus, I decided to move to a language that would support multithreading, C#.

See Sharp-Brain

The first thing I did when moving to C# was to build a complete body in an object oriented fashion. Body, head, brain, eyes, ears, nerves and so on. The head had eyes, which were attached to their own optic nerve, which would then connect to the brain. The brain was connected to a set of nerves, such as the optic nerve, which could be used either to receive or transmit data/signal.

Since I was now using a multithreaded language, I made each input a thread of its own. Thus, there would be 2 threads for the eyes, one per eye, 2 threads for the ears, one per ear, a thread for the brain, and so on. This would be the model of a human.

However, it was possible to generalize the idea to a robot. If we replaced the eyes with a single webcam, then the architecture would be similar in the sense that the eye, now a webcam, would take a frame and transmit it over the optic nerve to the brain. The brain would receive this array of bytes and process it (whatever that would mean, at this point no processing was done within the brain other than receive the data).

The problem at this point though was that the brain would now build its own network of understanding/patterns, which would make it more opaque and difficult to understand compared to a system of tasks.

Ben Goertzel, Cassio Pennachin

Pei Wang

Ben Goertzel

Peter Voss

Hugo de Garis

Patrick Hammer, Tony Lofthouse, Pei Wang

Jürgen Schmidhuber

Jürgen Schmidhuber

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller

David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, Demis Hassabis

Łukasz Kaiser

Vladimir G. Red'ko

Alex Graves, Greg Wayne, Ivo Danihelka

Keith A. Hoyes

José Luis Bermúdez

Breden Lake

Douglas Lenat

Douglas Lenat

Douglas Lenat

Ken Haase

G.D. Ritchie, F.K. Hanna

Allen Newell, John Clifford Shaw, Herbert A. Simon

Allen Newell, Herbert A. Simon

Douglas Hofstadter, Melanie Mitchell

J.R. Lucas

Began research related to automated programming. The main goal is to determine what is doable at the moment (based on current knowledge of the field).

Began research related to automated refactoring. The main goal is to determine a list of refactoring tools that can be developed to simplify the job of refactoring of programmers.

Alan Biermann

Francesco Zanoni

George Pólya

Allen Newell

Mark F. Bear, Barry W. Connors, Michael A. Paradiso

Patrick J. Hurley

Cordell Green, David Barstow

Peter Flach

Jeff Hawkins

John McCarthy

Tom Schaul, Dan Horgan, Karol Gregor, David Silver

Charles Rich, Richard C. Waters

Carl E. Hewitt, Brian Smith

David Marr

Began research related to automated research. The main goal is to determine a basic research flow and automate most of the process (researching papers, establishing important papers, determine a reading order, create a wordcloud of the most important terms, define which papers cite which papers, etc.).

John E. Hopcroft, Rajeev Motwani, Jeffrey D. Ullman

John von Neumann

Roman V. Yampolskiy

John Storrs Hall

John Storrs Hall

Richard Yonck

Masafumi Oizumi, Larissa Albantakis, Giulio Tononi

Tai-Hung Chen, Chun-Han Tseng, Chia-Ping Chen

Sam Abrahams, Danijar Hafner, Erik Erwitt, Ariel Scarpinelli

E. Mark Gold

J. Roland Olsson

Marcus Hutter

Marcus Hutter

Began research related to automated test generator. The main goal of this project is to use the knowledge I've acquired thinking about automated programming and refactoring and attempt to apply it to test generation. One of the goals is to develop a tool which will automate most of the process of testing PHP OOP code, specifically MVC-oriented websites using frameworks such as Laravel or Symfony.

My research into static analysis is related to my research on automated programming, automated refactoring and automated test generator. The main goal is to produce a piece of software that will be able to reason as a programmer would reason about a program in his head.

Patrice Godefroid, Peli de Halleux, Aditya Nori, Sriram Rajamani, Wolfram Schulte, Nikolai Tillmann, Michael Y. Levin

Paul Jorgensen

Thomas Fahringer, Bernhard Scholz

Flemming Nielson, Hanne R. Nielson, Chris Hankin

Toby Segaran, Colin Evans, Jamie Taylor

Ian Goodfellow, Yoshua Bengio, Aaron Courville

David Silver

Marek Rosa, Jan Feyereisl, The GoodAI Collective

I wrote a polymer/javascript-based application which dealt with triples in order to manage knowledge. Everything is considered to be a node and nodes can be associated to one another through anonymous.

See Knowledge base.

Yoshua Bengio, Jérôme Louradour, Ronan Collobert, Jason Weston

Joe Z. Tsien

Kun Xie, Grace E. Fox, Jun Liu, Cheng Lyu, Jason C. Lee, Hui Kuang, Stephanie Jacobs, Meng Li, Tianming Liu, Sen Song, Joe Z. Tsien

Lion Kimbro

J.R. Quinlan

Began experimenting with the problem of handwriting recognition. The main goal is to be able to feed a page through a camera/webcam/scanner and have its text content extracted.

Hugo Larochelle

Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, Pushmeet Kohli

Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio

Alex Graves, Jürgen Schmidhuber

Diederik P. Kingma, Max Welling

Yuxuan Wang, RJ Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J. Weiss, Navdeep Jaitly, Zongheng Yang, Ying Xiao, Zhifeng Chen, Samy Bengio, Quoc Le, Yannis Agiomyrgiannakis, Rob Clark, Rif A. Saurous

Kedar Tatwawadi

Théodore Bluche, Jérôme Louradour, Ronaldo Messina

Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio

Matej Balog, Alexander L. Gaunt, Marc Brockschmidt, Sebastian Nowozin, Daniel Tarlow

Barret Zoph, Quoc V. Le

Miltiadis Allamanis, Marc Brockschmidt

Bradly C. Stadie, Pieter Abbeel, Ilya Sutskever

Thomas M. Cover, Joy A. Thomas

Sercan O. Arik, Mike Chrzanowski, Adam Coates, Gregory Diamos, Andrew Gibiansky, Yongguo Kang, Xian Li, John Miller, Andrew Ng, Jonathan Raiman, Shubho Sengupta, Mohammad Shoeybi

Sercan Arik, Gregory Diamos, Andrew Gibiansky, John Miller, Kainan Peng, Wei Ping, Jonathan Raiman, Yanqi Zhou

Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R. Hruschka, Tom M. Mitchell

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin A. Riedmiller, Andreas Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, Demis Hassabis

Vicent Wiegel

Alex Graves, Santiago Fernandez, Juergen Schmidhuber

Nal Kalchbrenner, Ivo Danihelka, Alex Graves

Lukasz Kaiser, Aidan N. Gomez, Noam Shazeer, Ashish Vaswani, Niki Parmar, Llion Jones, Jakob Uszkoreit

Ken Kansky, Tom Silver, David A. Mély, Mohamed Eldawy, Miguel Lázaro-Gredilla, Xinghua Lou, Nimrod Dorfman, Szymon Sidor, Scott Phoenix, Dileep George

David Silver, Hado van Hasselt, Matteo Hessel, Tom Schaul, Arthur Guez, Tim Harley, Gabriel Dulac-Arnold, David Reichert, Neil Rabinowitz, André Barreto, Thomas Degris

Alex Graves, Marc G. Bellemare, Jacob Menick, Rémi Munos, Koray Kavukcuoglu

Manuel Lopes, Pierre-Yves Oudeyer

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia

Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron C. Courville, Ruslan Salakhutdinov, Richard S. Zemel, Yoshua Bengio

Paul F. Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, Dario Amodei

Pedro Domingos

Brian Christian, Tom Griffiths

Misha Denil, Sergio Gomez Colmenarejo, Serkan Cabi, David Saxton, Nando de Freitas

David Sterratt, Bruce Graham, Andrew Gillies, David Willshaw

Chris Eliasmith

Scott Aaronson

Nick Bostrom

David Eagleman

Yoshua Bengio

Jakob N. Foerster, Richard Y. Chen, Maruan Al-Shedivat, Shimon Whiteson, Pieter Abbeel, Igor Mordatch

Oriol Vinyals, Timo Ewalds, Sergey Bartunov, Petko Georgiev, Alexander Sasha Vezhnevets, Michelle Yeo, Alireza Makhzani, Heinrich Küttler, John Agapiou, Julian Schrittwieser, John Quan, Stephen Gaffney, Stig Petersen, Karen Simonyan, Tom Schaul, Hado van Hasselt, David Silver, Timothy P. Lillicrap, Kevin Calderone, Paul Keet, Anthony Brunasso, David Lawrence, Anders Ekermo, Jacob Repp, Rodney Tsing

Max Tegmark

John Johnston

Scott Gilbert

David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas R Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy P. Lillicrap, Fan Xin Hui, Laurent Sifre, George van den Driessche, Thore Graepel, Demis Hassabis

Thomas Anthony, Zheng Tian, David Barber

Matteo Hessel, Joseph Modayil, Hado van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Daniel Horgan, Bilal Piot, Mohammad Gheshlaghi Azar, David Silver

Carlos Florensa, David Held, Markus Wulfmeier, Pieter Abbeel

Yoshua Bengio

Wei Ping, Kainan Peng, Andrew Gibiansky, Sercan Ömer Arik, Ajay Kannan, Sharan Narang, Jonathan Raiman, John Miller

David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, Demis Hassabis

Levente Kocsis, Csaba Szepesvári

Ronen I. Brafman, Moshe Tennenholtz

Max Tegmark

Irving John Good

Yoav Freund, Robert E. Schapire

I have recorded a ton of audio of myself while I use my computer at home. I wanted to be able to find the slices of time where I speak and say something intelligible in order to extract those regions and then potentially send them to a service such as Google Voice to have it converted into text.

Burr Settles, Brendan Meeder

Hans Moravec

Shane Legg

Karl Friston

Yingfei Xiong, Jie Wang, Runfa Yan, Jiachen Zhang, Shi Han, Gang Huang, Lu Zhang

Satinder P. Singh, Andrew G. Barto, Nuttapong Chentanez

Xuan-Bach D. Le, David Lo, Claire Le Goues

Neil C. Rabinowitz, Frank Perbet, H. Francis Song, Chiyuan Zhang, S. M. Ali Eslami, Matthew M Botvinick

Jurgen Schmidhuber

Jurgen Schmidhuber