GPT-J is the largest model the EleutherAI has released to date. A 6-billion parameters language model is comparable in performance to the GPT-3 version of similar size — 6.7 billion parameters (but GPT-3 is over 175 billion parameters now).
This model was trained upon the Pile, which is an 825 GiB diverse, open-source language modeling data set that consists of 22 smaller, high-quality datasets combined together. For more information, please read this paper.
I admit that the story below came nicely out of the model, but I needed to guide it a few times, re-starting and re-seeding the conversation (to keep it meaningful). Otherwise, I simply followed the rules described in the prior post.
Now, will an all-knowing artificial intelligence govern us in the future?
A few years ago, the word “singularity” entered the popular lexicon. In this new era, which futurist Ray Kurzweil predicts will occur in the early 21st century, humanity will merge with technology to form a “transcendent” intelligence. This machine consciousness will surpass us in every way, and humanity will be a distant memory.
We are not there yet, but the concept of a singularity is beginning to make some waves in science and engineering. “The Singularity” is a term used by futurists to describe the future state of our technology and society in which the pace of scientific and technological development will lead to exponential growth in computational power, artificial intelligence, and longevity. (AK: actually, the concept was introduced in 1993 here, if you want to read more.)
Why all this fuss? Because the concepts of intelligence and consciousness have always been hard to define. While some of the current AI research is more like the U.S. space program of the 1950s, seeking a grand unified theory of intelligence, other AI researchers are more like the International Space Station program, focused on getting practical results as soon as possible. But there’s a chance that we’re on the verge of the technological Singularity, a time of significant change.
What is the Singularity? In the early days of AI, researchers like John McCarthy at Dartmouth College and Marvin Minsky at MIT thought AI would require a human-like level of intelligence. While this is still a possibility, it’s not the only one. The world of AI is so varied, with so many different ways of representing intelligence and consciousness, that it’s hard to say where exactly we are on the path to the Singularity. The goal of superintelligence is still largely a dream, but a few researchers have made significant progress towards the theoretical Singularity. (AK: this reference to superintelligence reminded me of my story The Interview from the I, AI collection of short stories.)
The Singularity as a theory
The concept of the Singularity as a future state of humanity is not a new idea. In the 1970s, Vernor Vinge first proposed it, arguing that the development of AI would lead to a new state of being that transcends human beings. The idea is still alive today, but it’s become increasingly focused on the concept of an all-knowing AI.