University of Wisconsin-Madison, UC Berkeley, and Google Brain Introduce Nystromformer: A Nystrom-based Algorithm for Approximating Self-Attention

  • by

Early days of research in Natural language processing established long-term dependencies. It also brought the vanishing gradient problem in front of us because nascent models were handling the input sequences one by one, without parallelization. Recently, revolutionary transformer-based architectures and the self-attention mechanisms have enabled token pairs’ interactions across complete sequences resulting in modeling arbitrary dependencies in a constant number of layers. The above helped achieve state-of-the-art performance across many… Read More »University of Wisconsin-Madison, UC Berkeley, and Google Brain Introduce Nystromformer: A Nystrom-based Algorithm for Approximating Self-Attention

AI Researchers Create Video Game Playing Model That Can Remember Past Events

  • by

A team of researchers at Uber’s AI lab have recently developed a system of AI algorithms that outperformed both human players and other AI systems at classic Atari video games. The AI system developed by the researchers is capable of remembering previously successful strategies, creating new strategies based on what worked in the past. The study’s research team believes that the algorithms they developed have potential applications in other technical… Read More »AI Researchers Create Video Game Playing Model That Can Remember Past Events

My vision of the post singularity world

  • by

I think China will create an Homo Deus that will decide that organic lifeforms are too inefficient and decide to not produce rice, starving all homo sapiens that depend on China’s infrastructure. In the West, I think a coalition of Google and the US Government will create the Homo Deus that will protect the West from attacks from the East. Elon Musk will create a libertarian colony on Mars. submitted… Read More »My vision of the post singularity world